2024-04-17T05:39:57.694 INFO:root:teuthology version: 0.0.1.dev267+g6c63784 2024-04-17T05:39:57.694 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_rishabh@teuthology --name rishabh-2024-04-17_04:58:44-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi --verbose --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-workunit/fs/test_o_trunc}} --archive /home/teuthworker/archive/rishabh-2024-04-17_04:58:44-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7659857 -- /home/teuthworker/archive/rishabh-2024-04-17_04:58:44-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7659857/orig.config.yaml 2024-04-17T05:39:57.752 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-17T05:39:57.838 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2024-04-17_04:58:44-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7659857 branch: wip-rishabh-testing-20240416.193735 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-workunit/fs/test_o_trunc}} email: null first_in_suite: false job_id: '7659857' kernel: client: sha1: distro flavor: default kdb: true sha1: 3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true name: rishabh-2024-04-17_04:58:44-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-rishabh-testing-20240416.193735 ceph: cephfs: max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability sha1: a8c84f5997950fd7d008204839253291d52e6eda subvols: create: 2 subvol_options: --namespace-isolated ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount dry_run: true install: ceph: flavor: default sha1: a8c84f5997950fd7d008204839253291d52e6eda kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=crc - wsync syntax: v2 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-testing-20240416.193735 sha1: a8c84f5997950fd7d008204839253291d52e6eda owner: scheduled_rishabh@teuthology priority: 50 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 3521 sha1: a8c84f5997950fd7d008204839253291d52e6eda sleep_before_teardown: 0 subset: 410/512 suite: fs suite_branch: wip-rishabh-testing-20240416.193735 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: a8c84f5997950fd7d008204839253291d52e6eda syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi047.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAyH9yLAOLCPCYe4zUsNI5uUo3/ta4J3zO6kUkeDDp3R2Qtlq6FJ6lrUxUd/0waxq3ltYQfg3a338iSr3i/mHJI= smithi098.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBvkz9GNp+nmgIMc4TsrGcI2waA7uwk7Htp3UJzERtGcqMNo+582u2UwYhAOJuEUHnonN6+vTJoaMMOTREdIANc= smithi103.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBP/aJTTQebpmIo7EIVE10vSvjKUddAPHud2MF8XGw9AaLoJSd3P004sMYTRQRgoCwk5tqUtNwIiRUi9zjbyGBeA= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - exec: mon.a: - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - fs/test_o_trunc.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 6c637841c215537a4502385240412f1966e0faab timestamp: 2024-04-17_04:58:44 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-17T05:39:57.839 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa; will attempt to use it 2024-04-17T05:39:57.839 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks 2024-04-17T05:39:57.839 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-17T05:39:57.841 INFO:teuthology.task.internal:Checking packages... 2024-04-17T05:39:57.864 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'a8c84f5997950fd7d008204839253291d52e6eda' 2024-04-17T05:39:57.864 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-17T05:39:57.864 INFO:teuthology.packaging:ref: None 2024-04-17T05:39:57.864 INFO:teuthology.packaging:tag: None 2024-04-17T05:39:57.864 INFO:teuthology.packaging:branch: wip-rishabh-testing-20240416.193735 2024-04-17T05:39:57.864 INFO:teuthology.packaging:sha1: a8c84f5997950fd7d008204839253291d52e6eda 2024-04-17T05:39:57.864 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-rishabh-testing-20240416.193735 2024-04-17T05:39:58.079 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3133.ga8c84f59 2024-04-17T05:39:58.080 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-17T05:39:58.087 INFO:teuthology.task.internal:no buildpackages task found 2024-04-17T05:39:58.087 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-17T05:39:58.093 INFO:teuthology.task.internal:Saving configuration 2024-04-17T05:39:58.113 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-17T05:39:58.119 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-17T05:39:58.148 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi047.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-17_04:58:44-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7659857', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-17 05:34:22.101894', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAyH9yLAOLCPCYe4zUsNI5uUo3/ta4J3zO6kUkeDDp3R2Qtlq6FJ6lrUxUd/0waxq3ltYQfg3a338iSr3i/mHJI='} 2024-04-17T05:39:58.173 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi098.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-17_04:58:44-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7659857', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-17 05:34:22.098825', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBvkz9GNp+nmgIMc4TsrGcI2waA7uwk7Htp3UJzERtGcqMNo+582u2UwYhAOJuEUHnonN6+vTJoaMMOTREdIANc='} 2024-04-17T05:39:58.193 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi103.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-17_04:58:44-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7659857', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-17 05:34:22.100492', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBP/aJTTQebpmIo7EIVE10vSvjKUddAPHud2MF8XGw9AaLoJSd3P004sMYTRQRgoCwk5tqUtNwIiRUi9zjbyGBeA='} 2024-04-17T05:39:58.193 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-17T05:39:58.199 INFO:teuthology.task.internal:roles: ubuntu@smithi047.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-04-17T05:39:58.199 INFO:teuthology.task.internal:roles: ubuntu@smithi098.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-04-17T05:39:58.199 INFO:teuthology.task.internal:roles: ubuntu@smithi103.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-04-17T05:39:58.199 INFO:teuthology.run_tasks:Running task console_log... 2024-04-17T05:39:58.301 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f0eee258160>, signals=[15]) 2024-04-17T05:39:58.302 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-17T05:39:58.308 INFO:teuthology.task.internal:Opening connections... 2024-04-17T05:39:58.308 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi047.front.sepia.ceph.com 2024-04-17T05:39:58.310 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi047.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T05:39:58.424 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi098.front.sepia.ceph.com 2024-04-17T05:39:58.425 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T05:39:58.510 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi103.front.sepia.ceph.com 2024-04-17T05:39:58.510 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi103.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T05:39:58.601 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-17T05:39:58.608 DEBUG:teuthology.orchestra.run.smithi047:> uname -m 2024-04-17T05:39:58.626 INFO:teuthology.orchestra.run.smithi047.stdout:x86_64 2024-04-17T05:39:58.627 DEBUG:teuthology.orchestra.run.smithi047:> cat /etc/os-release 2024-04-17T05:39:58.683 INFO:teuthology.orchestra.run.smithi047.stdout:NAME="CentOS Stream" 2024-04-17T05:39:58.684 INFO:teuthology.orchestra.run.smithi047.stdout:VERSION="9" 2024-04-17T05:39:58.684 INFO:teuthology.orchestra.run.smithi047.stdout:ID="centos" 2024-04-17T05:39:58.684 INFO:teuthology.orchestra.run.smithi047.stdout:ID_LIKE="rhel fedora" 2024-04-17T05:39:58.684 INFO:teuthology.orchestra.run.smithi047.stdout:VERSION_ID="9" 2024-04-17T05:39:58.684 INFO:teuthology.orchestra.run.smithi047.stdout:PLATFORM_ID="platform:el9" 2024-04-17T05:39:58.684 INFO:teuthology.orchestra.run.smithi047.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-17T05:39:58.684 INFO:teuthology.orchestra.run.smithi047.stdout:ANSI_COLOR="0;31" 2024-04-17T05:39:58.684 INFO:teuthology.orchestra.run.smithi047.stdout:LOGO="fedora-logo-icon" 2024-04-17T05:39:58.684 INFO:teuthology.orchestra.run.smithi047.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-17T05:39:58.684 INFO:teuthology.orchestra.run.smithi047.stdout:HOME_URL="https://centos.org/" 2024-04-17T05:39:58.684 INFO:teuthology.orchestra.run.smithi047.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-17T05:39:58.684 INFO:teuthology.orchestra.run.smithi047.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-17T05:39:58.684 INFO:teuthology.orchestra.run.smithi047.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-17T05:39:58.685 INFO:teuthology.lock.ops:Updating smithi047.front.sepia.ceph.com on lock server 2024-04-17T05:39:58.707 DEBUG:teuthology.orchestra.run.smithi098:> uname -m 2024-04-17T05:39:58.724 INFO:teuthology.orchestra.run.smithi098.stdout:x86_64 2024-04-17T05:39:58.725 DEBUG:teuthology.orchestra.run.smithi098:> cat /etc/os-release 2024-04-17T05:39:58.781 INFO:teuthology.orchestra.run.smithi098.stdout:NAME="CentOS Stream" 2024-04-17T05:39:58.781 INFO:teuthology.orchestra.run.smithi098.stdout:VERSION="9" 2024-04-17T05:39:58.782 INFO:teuthology.orchestra.run.smithi098.stdout:ID="centos" 2024-04-17T05:39:58.782 INFO:teuthology.orchestra.run.smithi098.stdout:ID_LIKE="rhel fedora" 2024-04-17T05:39:58.782 INFO:teuthology.orchestra.run.smithi098.stdout:VERSION_ID="9" 2024-04-17T05:39:58.782 INFO:teuthology.orchestra.run.smithi098.stdout:PLATFORM_ID="platform:el9" 2024-04-17T05:39:58.782 INFO:teuthology.orchestra.run.smithi098.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-17T05:39:58.782 INFO:teuthology.orchestra.run.smithi098.stdout:ANSI_COLOR="0;31" 2024-04-17T05:39:58.782 INFO:teuthology.orchestra.run.smithi098.stdout:LOGO="fedora-logo-icon" 2024-04-17T05:39:58.782 INFO:teuthology.orchestra.run.smithi098.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-17T05:39:58.782 INFO:teuthology.orchestra.run.smithi098.stdout:HOME_URL="https://centos.org/" 2024-04-17T05:39:58.782 INFO:teuthology.orchestra.run.smithi098.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-17T05:39:58.782 INFO:teuthology.orchestra.run.smithi098.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-17T05:39:58.782 INFO:teuthology.orchestra.run.smithi098.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-17T05:39:58.783 INFO:teuthology.lock.ops:Updating smithi098.front.sepia.ceph.com on lock server 2024-04-17T05:39:58.798 DEBUG:teuthology.orchestra.run.smithi103:> uname -m 2024-04-17T05:39:58.815 INFO:teuthology.orchestra.run.smithi103.stdout:x86_64 2024-04-17T05:39:58.816 DEBUG:teuthology.orchestra.run.smithi103:> cat /etc/os-release 2024-04-17T05:39:58.874 INFO:teuthology.orchestra.run.smithi103.stdout:NAME="CentOS Stream" 2024-04-17T05:39:58.874 INFO:teuthology.orchestra.run.smithi103.stdout:VERSION="9" 2024-04-17T05:39:58.874 INFO:teuthology.orchestra.run.smithi103.stdout:ID="centos" 2024-04-17T05:39:58.874 INFO:teuthology.orchestra.run.smithi103.stdout:ID_LIKE="rhel fedora" 2024-04-17T05:39:58.875 INFO:teuthology.orchestra.run.smithi103.stdout:VERSION_ID="9" 2024-04-17T05:39:58.875 INFO:teuthology.orchestra.run.smithi103.stdout:PLATFORM_ID="platform:el9" 2024-04-17T05:39:58.875 INFO:teuthology.orchestra.run.smithi103.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-17T05:39:58.875 INFO:teuthology.orchestra.run.smithi103.stdout:ANSI_COLOR="0;31" 2024-04-17T05:39:58.875 INFO:teuthology.orchestra.run.smithi103.stdout:LOGO="fedora-logo-icon" 2024-04-17T05:39:58.875 INFO:teuthology.orchestra.run.smithi103.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-17T05:39:58.875 INFO:teuthology.orchestra.run.smithi103.stdout:HOME_URL="https://centos.org/" 2024-04-17T05:39:58.875 INFO:teuthology.orchestra.run.smithi103.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-17T05:39:58.875 INFO:teuthology.orchestra.run.smithi103.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-17T05:39:58.875 INFO:teuthology.orchestra.run.smithi103.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-17T05:39:58.875 INFO:teuthology.lock.ops:Updating smithi103.front.sepia.ceph.com on lock server 2024-04-17T05:39:58.898 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-17T05:39:58.909 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-17T05:39:58.933 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-17T05:39:58.933 DEBUG:teuthology.orchestra.run.smithi047:> test '!' -e /home/ubuntu/cephtest 2024-04-17T05:39:58.935 DEBUG:teuthology.orchestra.run.smithi098:> test '!' -e /home/ubuntu/cephtest 2024-04-17T05:39:58.938 DEBUG:teuthology.orchestra.run.smithi103:> test '!' -e /home/ubuntu/cephtest 2024-04-17T05:39:58.952 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-17T05:39:58.958 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-17T05:39:58.958 DEBUG:teuthology.orchestra.run.smithi047:> test -z $(ls -A /var/lib/ceph) 2024-04-17T05:39:58.993 DEBUG:teuthology.orchestra.run.smithi098:> test -z $(ls -A /var/lib/ceph) 2024-04-17T05:39:58.995 DEBUG:teuthology.orchestra.run.smithi103:> test -z $(ls -A /var/lib/ceph) 2024-04-17T05:39:59.093 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-17T05:39:59.174 INFO:teuthology.run_tasks:Running task kernel... 2024-04-17T05:39:59.188 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': '3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8'} 2024-04-17T05:39:59.188 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-04-17T05:39:59.188 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-04-17T05:39:59.188 DEBUG:teuthology.orchestra.run.smithi047:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-17T05:39:59.205 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T05:39:59.205 DEBUG:teuthology.orchestra.run.smithi047:> uname -r 2024-04-17T05:39:59.263 INFO:teuthology.orchestra.run.smithi047.stdout:5.14.0-361.el9.x86_64 2024-04-17T05:39:59.263 INFO:teuthology.task.kernel:Running kernel on smithi047: 5.14.0-361.el9.x86_64 2024-04-17T05:39:59.263 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum install -y kernel 2024-04-17T05:40:01.366 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS-9 - AppStream 15 MB/s | 6.7 MB 00:00 2024-04-17T05:40:02.784 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS-9 - BaseOS 7.4 MB/s | 2.0 MB 00:00 2024-04-17T05:40:04.141 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS Stream 9 - BaseOS 8.0 MB/s | 8.0 MB 00:00 2024-04-17T05:40:06.175 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS Stream 9 - AppStream 22 MB/s | 19 MB 00:00 2024-04-17T05:40:11.201 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS Stream 9 - CRB 4.7 MB/s | 6.2 MB 00:01 2024-04-17T05:40:12.643 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS Stream 9 - Extras packages 66 kB/s | 16 kB 00:00 2024-04-17T05:40:14.161 INFO:teuthology.orchestra.run.smithi047.stdout:Extra Packages for Enterprise Linux 15 MB/s | 21 MB 00:01 2024-04-17T05:40:19.353 INFO:teuthology.orchestra.run.smithi047.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-04-17T05:40:21.818 INFO:teuthology.orchestra.run.smithi047.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-17T05:40:21.818 INFO:teuthology.orchestra.run.smithi047.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-04-17T05:40:21.918 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-04-17T05:40:21.934 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-04-17T05:40:21.935 INFO:teuthology.orchestra.run.smithi047.stdout: Package Arch Version Repository Size 2024-04-17T05:40:21.935 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-04-17T05:40:21.935 INFO:teuthology.orchestra.run.smithi047.stdout:Installing: 2024-04-17T05:40:21.935 INFO:teuthology.orchestra.run.smithi047.stdout: kernel x86_64 5.14.0-438.el9 CentOS-BaseOS 277 k 2024-04-17T05:40:21.935 INFO:teuthology.orchestra.run.smithi047.stdout:Installing dependencies: 2024-04-17T05:40:21.935 INFO:teuthology.orchestra.run.smithi047.stdout: kernel-core x86_64 5.14.0-438.el9 CentOS-BaseOS 15 M 2024-04-17T05:40:21.935 INFO:teuthology.orchestra.run.smithi047.stdout: kernel-modules x86_64 5.14.0-438.el9 CentOS-BaseOS 34 M 2024-04-17T05:40:21.935 INFO:teuthology.orchestra.run.smithi047.stdout: kernel-modules-core x86_64 5.14.0-438.el9 CentOS-BaseOS 28 M 2024-04-17T05:40:21.935 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-17T05:40:21.935 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction Summary 2024-04-17T05:40:21.935 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-04-17T05:40:21.935 INFO:teuthology.orchestra.run.smithi047.stdout:Install 4 Packages 2024-04-17T05:40:21.935 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-17T05:40:21.936 INFO:teuthology.orchestra.run.smithi047.stdout:Total download size: 77 M 2024-04-17T05:40:21.936 INFO:teuthology.orchestra.run.smithi047.stdout:Installed size: 123 M 2024-04-17T05:40:21.936 INFO:teuthology.orchestra.run.smithi047.stdout:Downloading Packages: 2024-04-17T05:40:22.296 INFO:teuthology.orchestra.run.smithi047.stdout:(1/4): kernel-5.14.0-438.el9.x86_64.rpm 771 kB/s | 277 kB 00:00 2024-04-17T05:40:22.446 INFO:teuthology.orchestra.run.smithi047.stdout:(2/4): kernel-core-5.14.0-438.el9.x86_64.rpm 30 MB/s | 15 MB 00:00 2024-04-17T05:40:23.022 INFO:teuthology.orchestra.run.smithi047.stdout:(3/4): kernel-modules-5.14.0-438.el9.x86_64.rpm 31 MB/s | 34 MB 00:01 2024-04-17T05:40:23.505 INFO:teuthology.orchestra.run.smithi047.stdout:(4/4): kernel-modules-core-5.14.0-438.el9.x86_6 23 MB/s | 28 MB 00:01 2024-04-17T05:40:23.507 INFO:teuthology.orchestra.run.smithi047.stdout:-------------------------------------------------------------------------------- 2024-04-17T05:40:23.508 INFO:teuthology.orchestra.run.smithi047.stdout:Total 49 MB/s | 77 MB 00:01 2024-04-17T05:40:23.508 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction check 2024-04-17T05:40:23.944 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction check succeeded. 2024-04-17T05:40:23.944 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction test 2024-04-17T05:40:24.796 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction test succeeded. 2024-04-17T05:40:24.797 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction 2024-04-17T05:40:27.228 INFO:teuthology.orchestra.run.smithi047.stdout: Preparing : 1/1 2024-04-17T05:40:27.868 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : kernel-modules-core-5.14.0-438.el9.x86_64 1/4 2024-04-17T05:40:27.965 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : kernel-core-5.14.0-438.el9.x86_64 2/4 2024-04-17T05:40:30.825 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: kernel-core-5.14.0-438.el9.x86_64 2/4 2024-04-17T05:40:30.958 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : kernel-modules-5.14.0-438.el9.x86_64 3/4 2024-04-17T05:40:36.023 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: kernel-modules-5.14.0-438.el9.x86_64 3/4 2024-04-17T05:40:36.066 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : kernel-5.14.0-438.el9.x86_64 4/4 2024-04-17T05:40:41.160 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: kernel-modules-core-5.14.0-438.el9.x86_64 4/4 2024-04-17T05:41:11.828 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: kernel-core-5.14.0-438.el9.x86_64 4/4 2024-04-17T05:41:11.828 INFO:teuthology.orchestra.run.smithi047.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-04-17T05:41:11.828 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-17T05:41:11.873 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: kernel-modules-5.14.0-438.el9.x86_64 4/4 2024-04-17T05:41:12.245 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: kernel-5.14.0-438.el9.x86_64 4/4 2024-04-17T05:41:12.245 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : kernel-5.14.0-438.el9.x86_64 1/4 2024-04-17T05:41:12.246 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : kernel-core-5.14.0-438.el9.x86_64 2/4 2024-04-17T05:41:12.246 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : kernel-modules-5.14.0-438.el9.x86_64 3/4 2024-04-17T05:41:12.636 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : kernel-modules-core-5.14.0-438.el9.x86_64 4/4 2024-04-17T05:41:12.636 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-17T05:41:12.636 INFO:teuthology.orchestra.run.smithi047.stdout:Installed: 2024-04-17T05:41:12.636 INFO:teuthology.orchestra.run.smithi047.stdout: kernel-5.14.0-438.el9.x86_64 2024-04-17T05:41:12.636 INFO:teuthology.orchestra.run.smithi047.stdout: kernel-core-5.14.0-438.el9.x86_64 2024-04-17T05:41:12.636 INFO:teuthology.orchestra.run.smithi047.stdout: kernel-modules-5.14.0-438.el9.x86_64 2024-04-17T05:41:12.636 INFO:teuthology.orchestra.run.smithi047.stdout: kernel-modules-core-5.14.0-438.el9.x86_64 2024-04-17T05:41:12.637 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-17T05:41:12.637 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-04-17T05:41:12.947 DEBUG:teuthology.orchestra.run.smithi047:> rpm -q kernel | sort -rV | head -n 1 2024-04-17T05:41:13.056 INFO:teuthology.orchestra.run.smithi047.stdout:kernel-5.14.0-438.el9.x86_64 2024-04-17T05:41:13.057 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-438.el9.x86_64 2024-04-17T05:41:13.057 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-438.el9.x86_64 2024-04-17T05:41:13.057 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-04-17T05:41:13.057 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-04-17T05:41:13.057 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-04-17T05:41:13.057 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi047.front.sepia.ceph.com, path=None, version=distro) 2024-04-17T05:41:13.057 DEBUG:teuthology.orchestra.run.smithi047:> rpm -q kernel | sort -rV | head -n 1 2024-04-17T05:41:13.140 INFO:teuthology.orchestra.run.smithi047.stdout:kernel-5.14.0-438.el9.x86_64 2024-04-17T05:41:13.141 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-438.el9.x86_64 2024-04-17T05:41:13.141 DEBUG:teuthology.orchestra.run.smithi047:> sudo rpm -qi grub2-tools 2024-04-17T05:41:13.179 INFO:teuthology.orchestra.run.smithi047.stdout:Name : grub2-tools 2024-04-17T05:41:13.179 INFO:teuthology.orchestra.run.smithi047.stdout:Epoch : 1 2024-04-17T05:41:13.179 INFO:teuthology.orchestra.run.smithi047.stdout:Version : 2.06 2024-04-17T05:41:13.179 INFO:teuthology.orchestra.run.smithi047.stdout:Release : 68.el9 2024-04-17T05:41:13.179 INFO:teuthology.orchestra.run.smithi047.stdout:Architecture: x86_64 2024-04-17T05:41:13.179 INFO:teuthology.orchestra.run.smithi047.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-04-17T05:41:13.179 INFO:teuthology.orchestra.run.smithi047.stdout:Group : Unspecified 2024-04-17T05:41:13.179 INFO:teuthology.orchestra.run.smithi047.stdout:Size : 8264254 2024-04-17T05:41:13.179 INFO:teuthology.orchestra.run.smithi047.stdout:License : GPLv3+ 2024-04-17T05:41:13.179 INFO:teuthology.orchestra.run.smithi047.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-17T05:41:13.179 INFO:teuthology.orchestra.run.smithi047.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-17T05:41:13.180 INFO:teuthology.orchestra.run.smithi047.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-17T05:41:13.180 INFO:teuthology.orchestra.run.smithi047.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-17T05:41:13.180 INFO:teuthology.orchestra.run.smithi047.stdout:Packager : builder@centos.org 2024-04-17T05:41:13.180 INFO:teuthology.orchestra.run.smithi047.stdout:Vendor : CentOS 2024-04-17T05:41:13.180 INFO:teuthology.orchestra.run.smithi047.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-17T05:41:13.180 INFO:teuthology.orchestra.run.smithi047.stdout:Summary : Support tools for GRUB. 2024-04-17T05:41:13.180 INFO:teuthology.orchestra.run.smithi047.stdout:Description : 2024-04-17T05:41:13.180 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-17T05:41:13.180 INFO:teuthology.orchestra.run.smithi047.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-17T05:41:13.180 INFO:teuthology.orchestra.run.smithi047.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-17T05:41:13.180 INFO:teuthology.orchestra.run.smithi047.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-17T05:41:13.180 INFO:teuthology.orchestra.run.smithi047.stdout:hardware devices. 2024-04-17T05:41:13.180 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-17T05:41:13.180 INFO:teuthology.orchestra.run.smithi047.stdout:This subpackage provides tools for support of all platforms. 2024-04-17T05:41:13.181 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-17T05:41:13.182 INFO:teuthology.task.kernel:Updating grub on smithi047 to boot 5.14.0-438.el9.x86_64 2024-04-17T05:41:13.182 DEBUG:teuthology.orchestra.run.smithi047:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-17T05:41:13.798 INFO:teuthology.orchestra.run.smithi047.stderr:Generating grub configuration file ... 2024-04-17T05:41:15.275 INFO:teuthology.orchestra.run.smithi047.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-17T05:41:15.295 INFO:teuthology.orchestra.run.smithi047.stderr:done 2024-04-17T05:41:15.297 DEBUG:teuthology.orchestra.run.smithi047:> mktemp 2024-04-17T05:41:15.313 INFO:teuthology.orchestra.run.smithi047.stdout:/tmp/tmp.ODz9c0Hi4m 2024-04-17T05:41:15.313 DEBUG:teuthology.orchestra.run.smithi047:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.ODz9c0Hi4m 2024-04-17T05:41:15.385 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 0666 /tmp/tmp.ODz9c0Hi4m 2024-04-17T05:41:15.529 DEBUG:teuthology.orchestra.remote:smithi047:/tmp/tmp.ODz9c0Hi4m is 6KB 2024-04-17T05:41:15.577 DEBUG:teuthology.orchestra.run.smithi047:> rm -fr /tmp/tmp.ODz9c0Hi4m 2024-04-17T05:41:15.591 DEBUG:teuthology.orchestra.run.smithi047:> sudo /bin/ls /boot/loader/entries || true 2024-04-17T05:41:15.659 INFO:teuthology.orchestra.run.smithi047.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-04-17T05:41:15.659 INFO:teuthology.orchestra.run.smithi047.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-04-17T05:41:15.659 INFO:teuthology.orchestra.run.smithi047.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-04-17T05:41:15.659 INFO:teuthology.orchestra.run.smithi047.stdout:15949e3c88704300882482eddfd7baae-5.14.0-438.el9.x86_64.conf 2024-04-17T05:41:15.661 DEBUG:teuthology.orchestra.run.smithi047:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-438.el9.x86_64 2024-04-17T05:41:15.823 DEBUG:teuthology.orchestra.run.smithi047:> sudo shutdown -r now 2024-04-17T05:41:45.827 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-04-17T05:41:45.827 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi047.front.sepia.ceph.com' 2024-04-17T05:41:45.828 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi047.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T05:42:04.318 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.47 2024-04-17T05:42:13.319 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi047.front.sepia.ceph.com' 2024-04-17T05:42:13.320 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi047.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T05:43:13.323 DEBUG:teuthology.orchestra.remote:timed out 2024-04-17T05:43:25.324 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi047.front.sepia.ceph.com' 2024-04-17T05:43:25.325 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi047.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T05:43:25.660 DEBUG:teuthology.orchestra.run.smithi047:> true 2024-04-17T05:43:26.048 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi047.front.sepia.ceph.com' 2024-04-17T05:43:26.048 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-438.el9.x86_64"... 2024-04-17T05:43:26.048 DEBUG:teuthology.orchestra.run.smithi047:> uname -r 2024-04-17T05:43:26.065 INFO:teuthology.orchestra.run.smithi047.stdout:5.14.0-438.el9.x86_64 2024-04-17T05:43:26.066 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-438.el9.x86_64 vs 5.14.0-438.el9.x86_64 2024-04-17T05:43:26.066 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-17T05:43:26.066 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-17T05:43:27.066 DEBUG:teuthology.parallel:result is None 2024-04-17T05:43:27.067 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-17T05:43:27.075 INFO:teuthology.task.internal:Creating test directory... 2024-04-17T05:43:27.075 DEBUG:teuthology.orchestra.run.smithi047:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-17T05:43:27.078 DEBUG:teuthology.orchestra.run.smithi098:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-17T05:43:27.080 DEBUG:teuthology.orchestra.run.smithi103:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-17T05:43:27.099 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-17T05:43:27.105 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-17T05:43:27.114 INFO:teuthology.task.internal:Creating archive directory... 2024-04-17T05:43:27.114 DEBUG:teuthology.orchestra.run.smithi047:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-17T05:43:27.136 DEBUG:teuthology.orchestra.run.smithi098:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-17T05:43:27.139 DEBUG:teuthology.orchestra.run.smithi103:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-17T05:43:27.277 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-17T05:43:27.284 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-17T05:43:27.284 DEBUG:teuthology.orchestra.run.smithi098:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-17T05:43:27.300 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T05:43:27.300 DEBUG:teuthology.orchestra.run.smithi103:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-17T05:43:27.317 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T05:43:27.317 DEBUG:teuthology.orchestra.run.smithi047:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-17T05:43:27.320 DEBUG:teuthology.orchestra.run.smithi098:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-17T05:43:27.343 DEBUG:teuthology.orchestra.run.smithi103:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-17T05:43:27.394 INFO:teuthology.orchestra.run.smithi098.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-17T05:43:27.412 INFO:teuthology.orchestra.run.smithi047.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-17T05:43:27.412 INFO:teuthology.orchestra.run.smithi103.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-17T05:43:27.418 INFO:teuthology.orchestra.run.smithi098.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-17T05:43:27.435 INFO:teuthology.orchestra.run.smithi103.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-17T05:43:27.438 INFO:teuthology.orchestra.run.smithi047.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-17T05:43:27.440 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-17T05:43:27.447 INFO:teuthology.task.internal:Configuring sudo... 2024-04-17T05:43:27.447 DEBUG:teuthology.orchestra.run.smithi047:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-17T05:43:27.450 DEBUG:teuthology.orchestra.run.smithi098:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-17T05:43:27.452 DEBUG:teuthology.orchestra.run.smithi103:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-17T05:43:27.481 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-17T05:43:27.490 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-17T05:43:27.491 DEBUG:teuthology.orchestra.run.smithi047:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-17T05:43:27.523 DEBUG:teuthology.orchestra.run.smithi098:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-17T05:43:27.525 DEBUG:teuthology.orchestra.run.smithi103:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-17T05:43:27.541 DEBUG:teuthology.orchestra.run.smithi047:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-17T05:43:27.623 DEBUG:teuthology.orchestra.run.smithi047:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-17T05:43:27.675 DEBUG:teuthology.orchestra.run.smithi047:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-17T05:43:27.759 DEBUG:teuthology.orchestra.run.smithi047:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-17T05:43:27.804 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-17T05:43:27.804 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-17T05:43:27.874 DEBUG:teuthology.orchestra.run.smithi098:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-17T05:43:27.920 DEBUG:teuthology.orchestra.run.smithi098:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-17T05:43:27.975 DEBUG:teuthology.orchestra.run.smithi098:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-17T05:43:28.060 DEBUG:teuthology.orchestra.run.smithi098:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-17T05:43:28.105 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-04-17T05:43:28.105 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-17T05:43:28.174 DEBUG:teuthology.orchestra.run.smithi103:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-17T05:43:28.218 DEBUG:teuthology.orchestra.run.smithi103:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-17T05:43:28.273 DEBUG:teuthology.orchestra.run.smithi103:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-17T05:43:28.358 DEBUG:teuthology.orchestra.run.smithi103:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-17T05:43:28.404 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-04-17T05:43:28.404 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-17T05:43:28.473 DEBUG:teuthology.orchestra.run.smithi047:> sudo service rsyslog restart 2024-04-17T05:43:28.476 DEBUG:teuthology.orchestra.run.smithi098:> sudo service rsyslog restart 2024-04-17T05:43:28.478 DEBUG:teuthology.orchestra.run.smithi103:> sudo service rsyslog restart 2024-04-17T05:43:28.512 INFO:teuthology.orchestra.run.smithi098.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-17T05:43:28.522 INFO:teuthology.orchestra.run.smithi047.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-17T05:43:28.551 INFO:teuthology.orchestra.run.smithi103.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-17T05:43:29.060 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-17T05:43:29.069 INFO:teuthology.task.internal:Starting timer... 2024-04-17T05:43:29.069 INFO:teuthology.run_tasks:Running task pcp... 2024-04-17T05:43:29.079 INFO:teuthology.run_tasks:Running task selinux... 2024-04-17T05:43:29.088 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-04-17T05:43:29.088 DEBUG:teuthology.orchestra.run.smithi047:> sudo service auditd rotate 2024-04-17T05:43:29.162 INFO:teuthology.orchestra.run.smithi047.stdout:Rotating logs: 2024-04-17T05:43:29.164 DEBUG:teuthology.orchestra.run.smithi098:> sudo service auditd rotate 2024-04-17T05:43:29.213 INFO:teuthology.orchestra.run.smithi098.stdout:Rotating logs: 2024-04-17T05:43:29.214 DEBUG:teuthology.orchestra.run.smithi103:> sudo service auditd rotate 2024-04-17T05:43:29.265 INFO:teuthology.orchestra.run.smithi103.stdout:Rotating logs: 2024-04-17T05:43:29.267 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-17T05:43:29.267 DEBUG:teuthology.orchestra.run.smithi047:> /usr/sbin/getenforce 2024-04-17T05:43:29.293 INFO:teuthology.orchestra.run.smithi047.stdout:Permissive 2024-04-17T05:43:29.293 DEBUG:teuthology.orchestra.run.smithi098:> /usr/sbin/getenforce 2024-04-17T05:43:29.326 INFO:teuthology.orchestra.run.smithi098.stdout:Permissive 2024-04-17T05:43:29.326 DEBUG:teuthology.orchestra.run.smithi103:> /usr/sbin/getenforce 2024-04-17T05:43:29.346 INFO:teuthology.orchestra.run.smithi103.stdout:Permissive 2024-04-17T05:43:29.346 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi047.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi098.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi103.front.sepia.ceph.com': 'permissive'} 2024-04-17T05:43:29.347 DEBUG:teuthology.orchestra.run.smithi047:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-17T05:43:29.377 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T05:43:29.377 DEBUG:teuthology.orchestra.run.smithi098:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-17T05:43:29.405 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T05:43:29.406 DEBUG:teuthology.orchestra.run.smithi103:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-17T05:43:29.434 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T05:43:29.435 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-17T05:43:29.435 DEBUG:teuthology.orchestra.run.smithi047:> sudo /usr/sbin/setenforce permissive 2024-04-17T05:43:29.470 DEBUG:teuthology.orchestra.run.smithi098:> sudo /usr/sbin/setenforce permissive 2024-04-17T05:43:29.511 DEBUG:teuthology.orchestra.run.smithi103:> sudo /usr/sbin/setenforce permissive 2024-04-17T05:43:29.547 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-17T05:43:29.557 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-17T05:43:29.572 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-17T05:43:29.573 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi047.front.sepia.ceph.com,smithi098.front.sepia.ceph.com,smithi103.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-17T05:48:07.185 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi047.front.sepia.ceph.com'), Remote(name='ubuntu@smithi098.front.sepia.ceph.com'), Remote(name='ubuntu@smithi103.front.sepia.ceph.com')] 2024-04-17T05:48:07.186 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi047.front.sepia.ceph.com' 2024-04-17T05:48:07.187 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi047.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T05:48:07.277 DEBUG:teuthology.orchestra.run.smithi047:> true 2024-04-17T05:48:07.354 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi047.front.sepia.ceph.com' 2024-04-17T05:48:07.355 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi098.front.sepia.ceph.com' 2024-04-17T05:48:07.355 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T05:48:07.440 DEBUG:teuthology.orchestra.run.smithi098:> true 2024-04-17T05:48:07.508 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi098.front.sepia.ceph.com' 2024-04-17T05:48:07.509 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi103.front.sepia.ceph.com' 2024-04-17T05:48:07.509 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi103.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T05:48:07.600 DEBUG:teuthology.orchestra.run.smithi103:> true 2024-04-17T05:48:07.671 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi103.front.sepia.ceph.com' 2024-04-17T05:48:07.672 INFO:teuthology.run_tasks:Running task clock... 2024-04-17T05:48:07.682 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-17T05:48:07.682 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-17T05:48:07.683 DEBUG:teuthology.orchestra.run.smithi047:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-17T05:48:07.686 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-17T05:48:07.686 DEBUG:teuthology.orchestra.run.smithi098:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-17T05:48:07.688 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-17T05:48:07.688 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-17T05:48:07.716 INFO:teuthology.orchestra.run.smithi098.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-17T05:48:07.718 INFO:teuthology.orchestra.run.smithi047.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-17T05:48:07.732 INFO:teuthology.orchestra.run.smithi098.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-17T05:48:07.736 INFO:teuthology.orchestra.run.smithi047.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-17T05:48:07.744 INFO:teuthology.orchestra.run.smithi103.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-17T05:48:07.761 INFO:teuthology.orchestra.run.smithi103.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-17T05:48:07.762 INFO:teuthology.orchestra.run.smithi098.stderr:sudo: ntpd: command not found 2024-04-17T05:48:07.768 INFO:teuthology.orchestra.run.smithi047.stderr:sudo: ntpd: command not found 2024-04-17T05:48:07.775 INFO:teuthology.orchestra.run.smithi098.stdout:506 Cannot talk to daemon 2024-04-17T05:48:07.783 INFO:teuthology.orchestra.run.smithi047.stdout:506 Cannot talk to daemon 2024-04-17T05:48:07.791 INFO:teuthology.orchestra.run.smithi098.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-17T05:48:07.791 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: ntpd: command not found 2024-04-17T05:48:07.801 INFO:teuthology.orchestra.run.smithi047.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-17T05:48:07.805 INFO:teuthology.orchestra.run.smithi103.stdout:506 Cannot talk to daemon 2024-04-17T05:48:07.807 INFO:teuthology.orchestra.run.smithi098.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-17T05:48:07.818 INFO:teuthology.orchestra.run.smithi047.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-17T05:48:07.821 INFO:teuthology.orchestra.run.smithi103.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-17T05:48:07.838 INFO:teuthology.orchestra.run.smithi103.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-17T05:48:07.851 INFO:teuthology.orchestra.run.smithi098.stderr:bash: line 1: ntpq: command not found 2024-04-17T05:48:07.854 INFO:teuthology.orchestra.run.smithi098.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-17T05:48:07.854 INFO:teuthology.orchestra.run.smithi098.stdout:=============================================================================== 2024-04-17T05:48:07.855 INFO:teuthology.orchestra.run.smithi098.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-17T05:48:07.855 INFO:teuthology.orchestra.run.smithi098.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-17T05:48:07.855 INFO:teuthology.orchestra.run.smithi098.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-17T05:48:07.855 INFO:teuthology.orchestra.run.smithi098.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-17T05:48:07.873 INFO:teuthology.orchestra.run.smithi047.stderr:bash: line 1: ntpq: command not found 2024-04-17T05:48:07.876 INFO:teuthology.orchestra.run.smithi047.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-17T05:48:07.876 INFO:teuthology.orchestra.run.smithi047.stdout:=============================================================================== 2024-04-17T05:48:07.877 INFO:teuthology.orchestra.run.smithi047.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-17T05:48:07.877 INFO:teuthology.orchestra.run.smithi047.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-17T05:48:07.877 INFO:teuthology.orchestra.run.smithi047.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-17T05:48:07.877 INFO:teuthology.orchestra.run.smithi047.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-17T05:48:07.881 INFO:teuthology.orchestra.run.smithi103.stderr:bash: line 1: ntpq: command not found 2024-04-17T05:48:07.885 INFO:teuthology.orchestra.run.smithi103.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-17T05:48:07.885 INFO:teuthology.orchestra.run.smithi103.stdout:=============================================================================== 2024-04-17T05:48:07.885 INFO:teuthology.orchestra.run.smithi103.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-17T05:48:07.885 INFO:teuthology.orchestra.run.smithi103.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-17T05:48:07.885 INFO:teuthology.orchestra.run.smithi103.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-17T05:48:07.885 INFO:teuthology.orchestra.run.smithi103.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-17T05:48:07.886 INFO:teuthology.run_tasks:Running task install... 2024-04-17T05:48:07.895 DEBUG:teuthology.task.install:project ceph 2024-04-17T05:48:07.895 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda'}} 2024-04-17T05:48:07.895 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda'} 2024-04-17T05:48:07.895 INFO:teuthology.task.install:Using flavor: default 2024-04-17T05:48:07.903 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-17T05:48:07.903 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-17T05:48:07.904 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda', 'tag': None, 'wait_for_package': False} 2024-04-17T05:48:07.904 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a8c84f5997950fd7d008204839253291d52e6eda 2024-04-17T05:48:07.906 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda', 'tag': None, 'wait_for_package': False} 2024-04-17T05:48:07.907 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a8c84f5997950fd7d008204839253291d52e6eda 2024-04-17T05:48:07.908 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda', 'tag': None, 'wait_for_package': False} 2024-04-17T05:48:07.908 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a8c84f5997950fd7d008204839253291d52e6eda 2024-04-17T05:48:08.050 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/ 2024-04-17T05:48:08.050 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3133.ga8c84f59 2024-04-17T05:48:08.061 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/ 2024-04-17T05:48:08.061 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3133.ga8c84f59 2024-04-17T05:48:08.204 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/ 2024-04-17T05:48:08.205 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3133.ga8c84f59 2024-04-17T05:48:08.255 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-17T05:48:08.255 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-17T05:48:08.255 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-17T05:48:08.256 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-17T05:48:08.256 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-04-17T05:48:08.256 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-17T05:48:08.284 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-17T05:48:08.284 DEBUG:teuthology.orchestra.run.smithi098:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a8c84f5997950fd7d008204839253291d52e6eda/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-17T05:48:08.285 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-17T05:48:08.286 DEBUG:teuthology.orchestra.run.smithi047:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a8c84f5997950fd7d008204839253291d52e6eda/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-17T05:48:08.352 DEBUG:teuthology.orchestra.run.smithi098:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-17T05:48:08.354 DEBUG:teuthology.orchestra.run.smithi047:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-17T05:48:08.379 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-17T05:48:08.379 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-04-17T05:48:08.379 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-17T05:48:08.407 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-17T05:48:08.407 DEBUG:teuthology.orchestra.run.smithi103:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a8c84f5997950fd7d008204839253291d52e6eda/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-17T05:48:08.431 DEBUG:teuthology.orchestra.run.smithi098:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-17T05:48:08.436 DEBUG:teuthology.orchestra.run.smithi047:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-17T05:48:08.476 DEBUG:teuthology.orchestra.run.smithi103:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-17T05:48:08.498 INFO:teuthology.orchestra.run.smithi098.stdout:check_obsoletes = 1 2024-04-17T05:48:08.499 DEBUG:teuthology.orchestra.run.smithi098:> sudo yum clean all 2024-04-17T05:48:08.503 INFO:teuthology.orchestra.run.smithi047.stdout:check_obsoletes = 1 2024-04-17T05:48:08.505 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum clean all 2024-04-17T05:48:08.556 DEBUG:teuthology.orchestra.run.smithi103:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-17T05:48:08.622 INFO:teuthology.orchestra.run.smithi103.stdout:check_obsoletes = 1 2024-04-17T05:48:08.624 DEBUG:teuthology.orchestra.run.smithi103:> sudo yum clean all 2024-04-17T05:48:08.735 INFO:teuthology.orchestra.run.smithi098.stdout:74 files removed 2024-04-17T05:48:08.762 DEBUG:teuthology.orchestra.run.smithi098:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-17T05:48:08.763 INFO:teuthology.orchestra.run.smithi047.stdout:74 files removed 2024-04-17T05:48:08.793 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-17T05:48:08.860 INFO:teuthology.orchestra.run.smithi103.stdout:74 files removed 2024-04-17T05:48:08.888 DEBUG:teuthology.orchestra.run.smithi103:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-17T05:48:09.478 INFO:teuthology.orchestra.run.smithi098.stdout:ceph packages for x86_64 198 kB/s | 84 kB 00:00 2024-04-17T05:48:09.515 INFO:teuthology.orchestra.run.smithi047.stdout:ceph packages for x86_64 203 kB/s | 84 kB 00:00 2024-04-17T05:48:09.563 INFO:teuthology.orchestra.run.smithi103.stdout:ceph packages for x86_64 217 kB/s | 84 kB 00:00 2024-04-17T05:48:09.794 INFO:teuthology.orchestra.run.smithi098.stdout:ceph noarch packages 42 kB/s | 12 kB 00:00 2024-04-17T05:48:09.874 INFO:teuthology.orchestra.run.smithi047.stdout:ceph noarch packages 38 kB/s | 12 kB 00:00 2024-04-17T05:48:09.880 INFO:teuthology.orchestra.run.smithi103.stdout:ceph noarch packages 42 kB/s | 12 kB 00:00 2024-04-17T05:48:10.145 INFO:teuthology.orchestra.run.smithi098.stdout:ceph source packages 5.9 kB/s | 1.9 kB 00:00 2024-04-17T05:48:10.225 INFO:teuthology.orchestra.run.smithi047.stdout:ceph source packages 6.1 kB/s | 1.9 kB 00:00 2024-04-17T05:48:10.239 INFO:teuthology.orchestra.run.smithi103.stdout:ceph source packages 5.8 kB/s | 1.9 kB 00:00 2024-04-17T05:48:10.547 INFO:teuthology.orchestra.run.smithi098.stdout:CentOS-9 - AppStream 18 MB/s | 6.7 MB 00:00 2024-04-17T05:48:10.625 INFO:teuthology.orchestra.run.smithi103.stdout:CentOS-9 - AppStream 19 MB/s | 6.7 MB 00:00 2024-04-17T05:48:10.885 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS-9 - AppStream 11 MB/s | 6.7 MB 00:00 2024-04-17T05:48:11.990 INFO:teuthology.orchestra.run.smithi098.stdout:CentOS-9 - BaseOS 5.8 MB/s | 2.0 MB 00:00 2024-04-17T05:48:11.991 INFO:teuthology.orchestra.run.smithi103.stdout:CentOS-9 - BaseOS 7.8 MB/s | 2.0 MB 00:00 2024-04-17T05:48:12.311 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS-9 - BaseOS 7.4 MB/s | 2.0 MB 00:00 2024-04-17T05:48:12.822 INFO:teuthology.orchestra.run.smithi098.stdout:CentOS Stream 9 - BaseOS 16 MB/s | 8.0 MB 00:00 2024-04-17T05:48:13.749 INFO:teuthology.orchestra.run.smithi103.stdout:CentOS Stream 9 - BaseOS 5.6 MB/s | 8.0 MB 00:01 2024-04-17T05:48:16.443 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS Stream 9 - BaseOS 2.1 MB/s | 8.0 MB 00:03 2024-04-17T05:48:17.300 INFO:teuthology.orchestra.run.smithi103.stdout:CentOS Stream 9 - AppStream 7.8 MB/s | 19 MB 00:02 2024-04-17T05:48:17.440 INFO:teuthology.orchestra.run.smithi098.stdout:CentOS Stream 9 - AppStream 5.4 MB/s | 19 MB 00:03 2024-04-17T05:48:18.227 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS Stream 9 - AppStream 30 MB/s | 19 MB 00:00 2024-04-17T05:48:21.349 INFO:teuthology.orchestra.run.smithi098.stdout:CentOS Stream 9 - CRB 14 MB/s | 6.2 MB 00:00 2024-04-17T05:48:21.559 INFO:teuthology.orchestra.run.smithi103.stdout:CentOS Stream 9 - CRB 8.4 MB/s | 6.2 MB 00:00 2024-04-17T05:48:22.842 INFO:teuthology.orchestra.run.smithi098.stdout:CentOS Stream 9 - Extras packages 50 kB/s | 16 kB 00:00 2024-04-17T05:48:22.920 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS Stream 9 - CRB 6.1 MB/s | 6.2 MB 00:01 2024-04-17T05:48:23.019 INFO:teuthology.orchestra.run.smithi103.stdout:CentOS Stream 9 - Extras packages 53 kB/s | 16 kB 00:00 2024-04-17T05:48:23.943 INFO:teuthology.orchestra.run.smithi098.stdout:Extra Packages for Enterprise Linux 22 MB/s | 21 MB 00:00 2024-04-17T05:48:24.547 INFO:teuthology.orchestra.run.smithi103.stdout:Extra Packages for Enterprise Linux 15 MB/s | 21 MB 00:01 2024-04-17T05:48:24.595 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS Stream 9 - Extras packages 38 kB/s | 16 kB 00:00 2024-04-17T05:48:25.630 INFO:teuthology.orchestra.run.smithi047.stdout:Extra Packages for Enterprise Linux 24 MB/s | 21 MB 00:00 2024-04-17T05:48:29.085 INFO:teuthology.orchestra.run.smithi098.stdout:lab-extras 15 kB/s | 1.7 kB 00:00 2024-04-17T05:48:29.587 INFO:teuthology.orchestra.run.smithi103.stdout:lab-extras 11 kB/s | 1.7 kB 00:00 2024-04-17T05:48:30.772 INFO:teuthology.orchestra.run.smithi047.stdout:lab-extras 39 kB/s | 1.7 kB 00:00 2024-04-17T05:48:31.724 INFO:teuthology.orchestra.run.smithi098.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-17T05:48:31.725 INFO:teuthology.orchestra.run.smithi098.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-17T05:48:31.739 INFO:teuthology.orchestra.run.smithi098.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-04-17T05:48:31.740 INFO:teuthology.orchestra.run.smithi098.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-17T05:48:31.741 INFO:teuthology.orchestra.run.smithi098.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-04-17T05:48:31.742 INFO:teuthology.orchestra.run.smithi098.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-04-17T05:48:31.744 INFO:teuthology.orchestra.run.smithi098.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-17T05:48:31.747 INFO:teuthology.orchestra.run.smithi098.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-04-17T05:48:31.748 INFO:teuthology.orchestra.run.smithi098.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-04-17T05:48:31.828 INFO:teuthology.orchestra.run.smithi098.stdout:Dependencies resolved. 2024-04-17T05:48:31.841 INFO:teuthology.orchestra.run.smithi098.stdout:============================================================================================ 2024-04-17T05:48:31.841 INFO:teuthology.orchestra.run.smithi098.stdout: Package Arch Version Repository Size 2024-04-17T05:48:31.841 INFO:teuthology.orchestra.run.smithi098.stdout:============================================================================================ 2024-04-17T05:48:31.841 INFO:teuthology.orchestra.run.smithi098.stdout:Installing: 2024-04-17T05:48:31.841 INFO:teuthology.orchestra.run.smithi098.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-04-17T05:48:31.841 INFO:teuthology.orchestra.run.smithi098.stdout: ceph x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 6.6 k 2024-04-17T05:48:31.841 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-base x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 5.4 M 2024-04-17T05:48:31.841 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-fuse x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 893 k 2024-04-17T05:48:31.841 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 145 k 2024-04-17T05:48:31.841 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 1.5 M 2024-04-17T05:48:31.842 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 142 k 2024-04-17T05:48:31.842 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 3.5 M 2024-04-17T05:48:31.842 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 7.4 M 2024-04-17T05:48:31.842 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-rook noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 49 k 2024-04-17T05:48:31.842 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-radosgw x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 11 M 2024-04-17T05:48:31.842 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-test x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 49 M 2024-04-17T05:48:31.842 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-volume noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 273 k 2024-04-17T05:48:31.842 INFO:teuthology.orchestra.run.smithi098.stdout: cephadm noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 763 k 2024-04-17T05:48:31.842 INFO:teuthology.orchestra.run.smithi098.stdout: cephfs-mirror x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 225 k 2024-04-17T05:48:31.842 INFO:teuthology.orchestra.run.smithi098.stdout: cephfs-shell x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 27 k 2024-04-17T05:48:31.842 INFO:teuthology.orchestra.run.smithi098.stdout: cephfs-top noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 18 k 2024-04-17T05:48:31.842 INFO:teuthology.orchestra.run.smithi098.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-17T05:48:31.842 INFO:teuthology.orchestra.run.smithi098.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-04-17T05:48:31.843 INFO:teuthology.orchestra.run.smithi098.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-04-17T05:48:31.843 INFO:teuthology.orchestra.run.smithi098.stdout: libcephfs-devel x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 32 k 2024-04-17T05:48:31.843 INFO:teuthology.orchestra.run.smithi098.stdout: libcephfs2 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 748 k 2024-04-17T05:48:31.843 INFO:teuthology.orchestra.run.smithi098.stdout: librados-devel x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 126 k 2024-04-17T05:48:31.843 INFO:teuthology.orchestra.run.smithi098.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-04-17T05:48:31.843 INFO:teuthology.orchestra.run.smithi098.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-17T05:48:31.843 INFO:teuthology.orchestra.run.smithi098.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-17T05:48:31.843 INFO:teuthology.orchestra.run.smithi098.stdout: postgresql x86_64 13.14-1.el9 CentOS-AppStream 1.5 M 2024-04-17T05:48:31.843 INFO:teuthology.orchestra.run.smithi098.stdout: postgresql-contrib x86_64 13.14-1.el9 CentOS-AppStream 817 k 2024-04-17T05:48:31.843 INFO:teuthology.orchestra.run.smithi098.stdout: postgresql-server x86_64 13.14-1.el9 CentOS-AppStream 5.7 M 2024-04-17T05:48:31.843 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cephfs x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 162 k 2024-04-17T05:48:31.843 INFO:teuthology.orchestra.run.smithi098.stdout: python3-rados x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 321 k 2024-04-17T05:48:31.844 INFO:teuthology.orchestra.run.smithi098.stdout: python3-rbd x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 299 k 2024-04-17T05:48:31.844 INFO:teuthology.orchestra.run.smithi098.stdout: python3-rgw x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 99 k 2024-04-17T05:48:31.844 INFO:teuthology.orchestra.run.smithi098.stdout: rbd-fuse x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 85 k 2024-04-17T05:48:31.844 INFO:teuthology.orchestra.run.smithi098.stdout: rbd-mirror x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 3.1 M 2024-04-17T05:48:31.844 INFO:teuthology.orchestra.run.smithi098.stdout: rbd-nbd x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 170 k 2024-04-17T05:48:31.844 INFO:teuthology.orchestra.run.smithi098.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-17T05:48:31.844 INFO:teuthology.orchestra.run.smithi098.stdout:Upgrading: 2024-04-17T05:48:31.844 INFO:teuthology.orchestra.run.smithi098.stdout: NetworkManager x86_64 1:1.47.4-1.el9 CentOS-BaseOS 2.3 M 2024-04-17T05:48:31.844 INFO:teuthology.orchestra.run.smithi098.stdout: NetworkManager-libnm x86_64 1:1.47.4-1.el9 CentOS-BaseOS 1.8 M 2024-04-17T05:48:31.844 INFO:teuthology.orchestra.run.smithi098.stdout: NetworkManager-team x86_64 1:1.47.4-1.el9 CentOS-BaseOS 41 k 2024-04-17T05:48:31.844 INFO:teuthology.orchestra.run.smithi098.stdout: NetworkManager-tui x86_64 1:1.47.4-1.el9 CentOS-BaseOS 248 k 2024-04-17T05:48:31.844 INFO:teuthology.orchestra.run.smithi098.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-04-17T05:48:31.844 INFO:teuthology.orchestra.run.smithi098.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-04-17T05:48:31.845 INFO:teuthology.orchestra.run.smithi098.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-04-17T05:48:31.845 INFO:teuthology.orchestra.run.smithi098.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-04-17T05:48:31.845 INFO:teuthology.orchestra.run.smithi098.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-04-17T05:48:31.845 INFO:teuthology.orchestra.run.smithi098.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-04-17T05:48:31.845 INFO:teuthology.orchestra.run.smithi098.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-04-17T05:48:31.845 INFO:teuthology.orchestra.run.smithi098.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-04-17T05:48:31.845 INFO:teuthology.orchestra.run.smithi098.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-04-17T05:48:31.845 INFO:teuthology.orchestra.run.smithi098.stdout: librados2 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 3.4 M 2024-04-17T05:48:31.845 INFO:teuthology.orchestra.run.smithi098.stdout: librbd1 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 3.1 M 2024-04-17T05:48:31.845 INFO:teuthology.orchestra.run.smithi098.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-04-17T05:48:31.845 INFO:teuthology.orchestra.run.smithi098.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-04-17T05:48:31.845 INFO:teuthology.orchestra.run.smithi098.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-04-17T05:48:31.845 INFO:teuthology.orchestra.run.smithi098.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-04-17T05:48:31.846 INFO:teuthology.orchestra.run.smithi098.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-04-17T05:48:31.846 INFO:teuthology.orchestra.run.smithi098.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-04-17T05:48:31.846 INFO:teuthology.orchestra.run.smithi098.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-04-17T05:48:31.846 INFO:teuthology.orchestra.run.smithi098.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-04-17T05:48:31.846 INFO:teuthology.orchestra.run.smithi098.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-04-17T05:48:31.846 INFO:teuthology.orchestra.run.smithi098.stdout:Installing dependencies: 2024-04-17T05:48:31.846 INFO:teuthology.orchestra.run.smithi098.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-17T05:48:31.846 INFO:teuthology.orchestra.run.smithi098.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-04-17T05:48:31.846 INFO:teuthology.orchestra.run.smithi098.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-04-17T05:48:31.846 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-common x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 21 M 2024-04-17T05:48:31.846 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 34 k 2024-04-17T05:48:31.846 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mds x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 2.4 M 2024-04-17T05:48:31.846 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 249 k 2024-04-17T05:48:31.846 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mon x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 4.7 M 2024-04-17T05:48:31.847 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-osd x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 17 M 2024-04-17T05:48:31.847 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 16 k 2024-04-17T05:48:31.847 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-selinux x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 25 k 2024-04-17T05:48:31.847 INFO:teuthology.orchestra.run.smithi098.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-04-17T05:48:31.847 INFO:teuthology.orchestra.run.smithi098.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-04-17T05:48:31.847 INFO:teuthology.orchestra.run.smithi098.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-04-17T05:48:31.847 INFO:teuthology.orchestra.run.smithi098.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-04-17T05:48:31.847 INFO:teuthology.orchestra.run.smithi098.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-17T05:48:31.847 INFO:teuthology.orchestra.run.smithi098.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-17T05:48:31.847 INFO:teuthology.orchestra.run.smithi098.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-04-17T05:48:31.847 INFO:teuthology.orchestra.run.smithi098.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-17T05:48:31.847 INFO:teuthology.orchestra.run.smithi098.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-17T05:48:31.847 INFO:teuthology.orchestra.run.smithi098.stdout: libcephsqlite x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 163 k 2024-04-17T05:48:31.848 INFO:teuthology.orchestra.run.smithi098.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-17T05:48:31.848 INFO:teuthology.orchestra.run.smithi098.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-17T05:48:31.848 INFO:teuthology.orchestra.run.smithi098.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-04-17T05:48:31.848 INFO:teuthology.orchestra.run.smithi098.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-17T05:48:31.848 INFO:teuthology.orchestra.run.smithi098.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-04-17T05:48:31.848 INFO:teuthology.orchestra.run.smithi098.stdout: libradosstriper1 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 501 k 2024-04-17T05:48:31.848 INFO:teuthology.orchestra.run.smithi098.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-04-17T05:48:31.848 INFO:teuthology.orchestra.run.smithi098.stdout: librgw2 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 5.4 M 2024-04-17T05:48:31.848 INFO:teuthology.orchestra.run.smithi098.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-17T05:48:31.848 INFO:teuthology.orchestra.run.smithi098.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-04-17T05:48:31.848 INFO:teuthology.orchestra.run.smithi098.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-04-17T05:48:31.848 INFO:teuthology.orchestra.run.smithi098.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-04-17T05:48:31.848 INFO:teuthology.orchestra.run.smithi098.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-17T05:48:31.848 INFO:teuthology.orchestra.run.smithi098.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-17T05:48:31.849 INFO:teuthology.orchestra.run.smithi098.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-04-17T05:48:31.849 INFO:teuthology.orchestra.run.smithi098.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-04-17T05:48:31.849 INFO:teuthology.orchestra.run.smithi098.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-04-17T05:48:31.849 INFO:teuthology.orchestra.run.smithi098.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-17T05:48:31.849 INFO:teuthology.orchestra.run.smithi098.stdout: postgresql-private-libs x86_64 13.14-1.el9 CentOS-AppStream 132 k 2024-04-17T05:48:31.849 INFO:teuthology.orchestra.run.smithi098.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-04-17T05:48:31.849 INFO:teuthology.orchestra.run.smithi098.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-17T05:48:31.849 INFO:teuthology.orchestra.run.smithi098.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-17T05:48:31.849 INFO:teuthology.orchestra.run.smithi098.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-04-17T05:48:31.849 INFO:teuthology.orchestra.run.smithi098.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-04-17T05:48:31.849 INFO:teuthology.orchestra.run.smithi098.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-17T05:48:31.849 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-17T05:48:31.849 INFO:teuthology.orchestra.run.smithi098.stdout: python3-ceph-argparse x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 45 k 2024-04-17T05:48:31.850 INFO:teuthology.orchestra.run.smithi098.stdout: python3-ceph-common x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 135 k 2024-04-17T05:48:31.850 INFO:teuthology.orchestra.run.smithi098.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-17T05:48:31.850 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-04-17T05:48:31.850 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-17T05:48:31.850 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-17T05:48:31.850 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-17T05:48:31.850 INFO:teuthology.orchestra.run.smithi098.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-17T05:48:31.850 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-04-17T05:48:31.850 INFO:teuthology.orchestra.run.smithi098.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-04-17T05:48:31.850 INFO:teuthology.orchestra.run.smithi098.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-17T05:48:31.850 INFO:teuthology.orchestra.run.smithi098.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-17T05:48:31.850 INFO:teuthology.orchestra.run.smithi098.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-17T05:48:31.851 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-17T05:48:31.851 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-17T05:48:31.851 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-17T05:48:31.851 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-17T05:48:31.851 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-17T05:48:31.851 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-17T05:48:31.851 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-04-17T05:48:31.851 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-04-17T05:48:31.851 INFO:teuthology.orchestra.run.smithi098.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-17T05:48:31.851 INFO:teuthology.orchestra.run.smithi098.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-17T05:48:31.851 INFO:teuthology.orchestra.run.smithi098.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-04-17T05:48:31.851 INFO:teuthology.orchestra.run.smithi098.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-04-17T05:48:31.851 INFO:teuthology.orchestra.run.smithi098.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-17T05:48:31.851 INFO:teuthology.orchestra.run.smithi098.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-17T05:48:31.852 INFO:teuthology.orchestra.run.smithi098.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-04-17T05:48:31.852 INFO:teuthology.orchestra.run.smithi098.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-04-17T05:48:31.852 INFO:teuthology.orchestra.run.smithi098.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-04-17T05:48:31.852 INFO:teuthology.orchestra.run.smithi098.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-04-17T05:48:31.852 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-17T05:48:31.852 INFO:teuthology.orchestra.run.smithi098.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-04-17T05:48:31.852 INFO:teuthology.orchestra.run.smithi098.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-17T05:48:31.852 INFO:teuthology.orchestra.run.smithi098.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-04-17T05:48:31.852 INFO:teuthology.orchestra.run.smithi098.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-04-17T05:48:31.852 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-17T05:48:31.852 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-04-17T05:48:31.852 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-04-17T05:48:31.852 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-04-17T05:48:31.853 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-17T05:48:31.853 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-04-17T05:48:31.853 INFO:teuthology.orchestra.run.smithi098.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-17T05:48:31.853 INFO:teuthology.orchestra.run.smithi098.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-17T05:48:31.853 INFO:teuthology.orchestra.run.smithi098.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-04-17T05:48:31.853 INFO:teuthology.orchestra.run.smithi098.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-17T05:48:31.853 INFO:teuthology.orchestra.run.smithi098.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-17T05:48:31.853 INFO:teuthology.orchestra.run.smithi098.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-04-17T05:48:31.853 INFO:teuthology.orchestra.run.smithi098.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-17T05:48:31.853 INFO:teuthology.orchestra.run.smithi098.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-04-17T05:48:31.853 INFO:teuthology.orchestra.run.smithi098.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-17T05:48:31.853 INFO:teuthology.orchestra.run.smithi098.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-04-17T05:48:31.853 INFO:teuthology.orchestra.run.smithi098.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-04-17T05:48:31.853 INFO:teuthology.orchestra.run.smithi098.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-17T05:48:31.853 INFO:teuthology.orchestra.run.smithi098.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-17T05:48:31.854 INFO:teuthology.orchestra.run.smithi098.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-17T05:48:31.854 INFO:teuthology.orchestra.run.smithi098.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-17T05:48:31.854 INFO:teuthology.orchestra.run.smithi098.stdout: qatlib x86_64 24.02.0-1.el9 CentOS-AppStream 220 k 2024-04-17T05:48:31.854 INFO:teuthology.orchestra.run.smithi098.stdout: qatzip-libs x86_64 1.2.0-1.el9 CentOS-AppStream 46 k 2024-04-17T05:48:31.854 INFO:teuthology.orchestra.run.smithi098.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-17T05:48:31.854 INFO:teuthology.orchestra.run.smithi098.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-04-17T05:48:31.854 INFO:teuthology.orchestra.run.smithi098.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-17T05:48:31.854 INFO:teuthology.orchestra.run.smithi098.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-04-17T05:48:31.854 INFO:teuthology.orchestra.run.smithi098.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-04-17T05:48:31.854 INFO:teuthology.orchestra.run.smithi098.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-04-17T05:48:31.854 INFO:teuthology.orchestra.run.smithi098.stdout:Installing weak dependencies: 2024-04-17T05:48:31.854 INFO:teuthology.orchestra.run.smithi098.stdout: qatlib-service x86_64 24.02.0-1.el9 CentOS-AppStream 35 k 2024-04-17T05:48:31.854 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-04-17T05:48:31.855 INFO:teuthology.orchestra.run.smithi098.stdout:Transaction Summary 2024-04-17T05:48:31.855 INFO:teuthology.orchestra.run.smithi098.stdout:============================================================================================ 2024-04-17T05:48:31.855 INFO:teuthology.orchestra.run.smithi098.stdout:Install 150 Packages 2024-04-17T05:48:31.855 INFO:teuthology.orchestra.run.smithi098.stdout:Upgrade 24 Packages 2024-04-17T05:48:31.855 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-04-17T05:48:31.855 INFO:teuthology.orchestra.run.smithi098.stdout:Total download size: 229 M 2024-04-17T05:48:31.855 INFO:teuthology.orchestra.run.smithi098.stdout:Downloading Packages: 2024-04-17T05:48:32.227 INFO:teuthology.orchestra.run.smithi103.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-17T05:48:32.228 INFO:teuthology.orchestra.run.smithi103.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-17T05:48:32.243 INFO:teuthology.orchestra.run.smithi103.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-04-17T05:48:32.244 INFO:teuthology.orchestra.run.smithi103.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-17T05:48:32.245 INFO:teuthology.orchestra.run.smithi103.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-04-17T05:48:32.246 INFO:teuthology.orchestra.run.smithi103.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-04-17T05:48:32.248 INFO:teuthology.orchestra.run.smithi103.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-17T05:48:32.251 INFO:teuthology.orchestra.run.smithi103.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-04-17T05:48:32.252 INFO:teuthology.orchestra.run.smithi103.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-04-17T05:48:32.333 INFO:teuthology.orchestra.run.smithi103.stdout:Dependencies resolved. 2024-04-17T05:48:32.354 INFO:teuthology.orchestra.run.smithi103.stdout:============================================================================================ 2024-04-17T05:48:32.354 INFO:teuthology.orchestra.run.smithi103.stdout: Package Arch Version Repository Size 2024-04-17T05:48:32.354 INFO:teuthology.orchestra.run.smithi103.stdout:============================================================================================ 2024-04-17T05:48:32.354 INFO:teuthology.orchestra.run.smithi103.stdout:Installing: 2024-04-17T05:48:32.355 INFO:teuthology.orchestra.run.smithi103.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-04-17T05:48:32.355 INFO:teuthology.orchestra.run.smithi103.stdout: ceph x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 6.6 k 2024-04-17T05:48:32.355 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-base x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 5.4 M 2024-04-17T05:48:32.355 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-fuse x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 893 k 2024-04-17T05:48:32.355 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 145 k 2024-04-17T05:48:32.355 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 1.5 M 2024-04-17T05:48:32.355 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 142 k 2024-04-17T05:48:32.355 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 3.5 M 2024-04-17T05:48:32.355 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 7.4 M 2024-04-17T05:48:32.355 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-rook noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 49 k 2024-04-17T05:48:32.355 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-radosgw x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 11 M 2024-04-17T05:48:32.355 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-test x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 49 M 2024-04-17T05:48:32.355 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-volume noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 273 k 2024-04-17T05:48:32.355 INFO:teuthology.orchestra.run.smithi103.stdout: cephadm noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 763 k 2024-04-17T05:48:32.356 INFO:teuthology.orchestra.run.smithi103.stdout: cephfs-mirror x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 225 k 2024-04-17T05:48:32.356 INFO:teuthology.orchestra.run.smithi103.stdout: cephfs-shell x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 27 k 2024-04-17T05:48:32.356 INFO:teuthology.orchestra.run.smithi103.stdout: cephfs-top noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 18 k 2024-04-17T05:48:32.356 INFO:teuthology.orchestra.run.smithi103.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-17T05:48:32.356 INFO:teuthology.orchestra.run.smithi103.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-04-17T05:48:32.356 INFO:teuthology.orchestra.run.smithi103.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-04-17T05:48:32.356 INFO:teuthology.orchestra.run.smithi103.stdout: libcephfs-devel x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 32 k 2024-04-17T05:48:32.356 INFO:teuthology.orchestra.run.smithi103.stdout: libcephfs2 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 748 k 2024-04-17T05:48:32.356 INFO:teuthology.orchestra.run.smithi103.stdout: librados-devel x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 126 k 2024-04-17T05:48:32.356 INFO:teuthology.orchestra.run.smithi103.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-04-17T05:48:32.356 INFO:teuthology.orchestra.run.smithi103.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-17T05:48:32.356 INFO:teuthology.orchestra.run.smithi103.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-17T05:48:32.356 INFO:teuthology.orchestra.run.smithi103.stdout: postgresql x86_64 13.14-1.el9 CentOS-AppStream 1.5 M 2024-04-17T05:48:32.356 INFO:teuthology.orchestra.run.smithi103.stdout: postgresql-contrib x86_64 13.14-1.el9 CentOS-AppStream 817 k 2024-04-17T05:48:32.357 INFO:teuthology.orchestra.run.smithi103.stdout: postgresql-server x86_64 13.14-1.el9 CentOS-AppStream 5.7 M 2024-04-17T05:48:32.357 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cephfs x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 162 k 2024-04-17T05:48:32.357 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rados x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 321 k 2024-04-17T05:48:32.357 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rbd x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 299 k 2024-04-17T05:48:32.357 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rgw x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 99 k 2024-04-17T05:48:32.357 INFO:teuthology.orchestra.run.smithi103.stdout: rbd-fuse x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 85 k 2024-04-17T05:48:32.357 INFO:teuthology.orchestra.run.smithi103.stdout: rbd-mirror x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 3.1 M 2024-04-17T05:48:32.357 INFO:teuthology.orchestra.run.smithi103.stdout: rbd-nbd x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 170 k 2024-04-17T05:48:32.357 INFO:teuthology.orchestra.run.smithi103.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-17T05:48:32.357 INFO:teuthology.orchestra.run.smithi103.stdout:Upgrading: 2024-04-17T05:48:32.357 INFO:teuthology.orchestra.run.smithi103.stdout: NetworkManager x86_64 1:1.47.4-1.el9 CentOS-BaseOS 2.3 M 2024-04-17T05:48:32.357 INFO:teuthology.orchestra.run.smithi103.stdout: NetworkManager-libnm x86_64 1:1.47.4-1.el9 CentOS-BaseOS 1.8 M 2024-04-17T05:48:32.357 INFO:teuthology.orchestra.run.smithi103.stdout: NetworkManager-team x86_64 1:1.47.4-1.el9 CentOS-BaseOS 41 k 2024-04-17T05:48:32.357 INFO:teuthology.orchestra.run.smithi103.stdout: NetworkManager-tui x86_64 1:1.47.4-1.el9 CentOS-BaseOS 248 k 2024-04-17T05:48:32.358 INFO:teuthology.orchestra.run.smithi103.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-04-17T05:48:32.358 INFO:teuthology.orchestra.run.smithi103.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-04-17T05:48:32.358 INFO:teuthology.orchestra.run.smithi103.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-04-17T05:48:32.358 INFO:teuthology.orchestra.run.smithi103.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-04-17T05:48:32.358 INFO:teuthology.orchestra.run.smithi103.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-04-17T05:48:32.358 INFO:teuthology.orchestra.run.smithi103.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-04-17T05:48:32.358 INFO:teuthology.orchestra.run.smithi103.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-04-17T05:48:32.358 INFO:teuthology.orchestra.run.smithi103.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-04-17T05:48:32.358 INFO:teuthology.orchestra.run.smithi103.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-04-17T05:48:32.358 INFO:teuthology.orchestra.run.smithi103.stdout: librados2 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 3.4 M 2024-04-17T05:48:32.358 INFO:teuthology.orchestra.run.smithi103.stdout: librbd1 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 3.1 M 2024-04-17T05:48:32.358 INFO:teuthology.orchestra.run.smithi103.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-04-17T05:48:32.358 INFO:teuthology.orchestra.run.smithi103.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-04-17T05:48:32.358 INFO:teuthology.orchestra.run.smithi103.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-04-17T05:48:32.359 INFO:teuthology.orchestra.run.smithi103.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-04-17T05:48:32.359 INFO:teuthology.orchestra.run.smithi103.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-04-17T05:48:32.359 INFO:teuthology.orchestra.run.smithi103.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-04-17T05:48:32.359 INFO:teuthology.orchestra.run.smithi103.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-04-17T05:48:32.359 INFO:teuthology.orchestra.run.smithi103.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-04-17T05:48:32.359 INFO:teuthology.orchestra.run.smithi103.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-04-17T05:48:32.359 INFO:teuthology.orchestra.run.smithi103.stdout:Installing dependencies: 2024-04-17T05:48:32.359 INFO:teuthology.orchestra.run.smithi103.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-17T05:48:32.359 INFO:teuthology.orchestra.run.smithi103.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-04-17T05:48:32.359 INFO:teuthology.orchestra.run.smithi103.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-04-17T05:48:32.359 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-common x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 21 M 2024-04-17T05:48:32.359 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 34 k 2024-04-17T05:48:32.359 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mds x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 2.4 M 2024-04-17T05:48:32.359 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 249 k 2024-04-17T05:48:32.359 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mon x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 4.7 M 2024-04-17T05:48:32.360 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-osd x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 17 M 2024-04-17T05:48:32.360 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 16 k 2024-04-17T05:48:32.360 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-selinux x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 25 k 2024-04-17T05:48:32.360 INFO:teuthology.orchestra.run.smithi103.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-04-17T05:48:32.360 INFO:teuthology.orchestra.run.smithi103.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-04-17T05:48:32.360 INFO:teuthology.orchestra.run.smithi103.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-04-17T05:48:32.360 INFO:teuthology.orchestra.run.smithi103.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-04-17T05:48:32.360 INFO:teuthology.orchestra.run.smithi103.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-17T05:48:32.360 INFO:teuthology.orchestra.run.smithi103.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-17T05:48:32.360 INFO:teuthology.orchestra.run.smithi103.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-04-17T05:48:32.360 INFO:teuthology.orchestra.run.smithi103.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-17T05:48:32.360 INFO:teuthology.orchestra.run.smithi103.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-17T05:48:32.360 INFO:teuthology.orchestra.run.smithi103.stdout: libcephsqlite x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 163 k 2024-04-17T05:48:32.360 INFO:teuthology.orchestra.run.smithi103.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-17T05:48:32.360 INFO:teuthology.orchestra.run.smithi103.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-17T05:48:32.361 INFO:teuthology.orchestra.run.smithi103.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-04-17T05:48:32.361 INFO:teuthology.orchestra.run.smithi103.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-17T05:48:32.361 INFO:teuthology.orchestra.run.smithi103.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-04-17T05:48:32.361 INFO:teuthology.orchestra.run.smithi103.stdout: libradosstriper1 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 501 k 2024-04-17T05:48:32.361 INFO:teuthology.orchestra.run.smithi103.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-04-17T05:48:32.361 INFO:teuthology.orchestra.run.smithi103.stdout: librgw2 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 5.4 M 2024-04-17T05:48:32.361 INFO:teuthology.orchestra.run.smithi103.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-17T05:48:32.361 INFO:teuthology.orchestra.run.smithi103.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-04-17T05:48:32.361 INFO:teuthology.orchestra.run.smithi103.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-04-17T05:48:32.361 INFO:teuthology.orchestra.run.smithi103.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-04-17T05:48:32.361 INFO:teuthology.orchestra.run.smithi103.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-17T05:48:32.361 INFO:teuthology.orchestra.run.smithi103.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-17T05:48:32.361 INFO:teuthology.orchestra.run.smithi103.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-04-17T05:48:32.362 INFO:teuthology.orchestra.run.smithi103.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-04-17T05:48:32.362 INFO:teuthology.orchestra.run.smithi103.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-04-17T05:48:32.362 INFO:teuthology.orchestra.run.smithi103.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-17T05:48:32.362 INFO:teuthology.orchestra.run.smithi103.stdout: postgresql-private-libs x86_64 13.14-1.el9 CentOS-AppStream 132 k 2024-04-17T05:48:32.362 INFO:teuthology.orchestra.run.smithi103.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-04-17T05:48:32.362 INFO:teuthology.orchestra.run.smithi103.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-17T05:48:32.362 INFO:teuthology.orchestra.run.smithi103.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-17T05:48:32.362 INFO:teuthology.orchestra.run.smithi103.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-04-17T05:48:32.362 INFO:teuthology.orchestra.run.smithi103.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-04-17T05:48:32.362 INFO:teuthology.orchestra.run.smithi103.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-17T05:48:32.362 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-17T05:48:32.362 INFO:teuthology.orchestra.run.smithi103.stdout: python3-ceph-argparse x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 45 k 2024-04-17T05:48:32.362 INFO:teuthology.orchestra.run.smithi103.stdout: python3-ceph-common x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 135 k 2024-04-17T05:48:32.363 INFO:teuthology.orchestra.run.smithi103.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-17T05:48:32.363 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-04-17T05:48:32.363 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-17T05:48:32.363 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-17T05:48:32.363 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-17T05:48:32.363 INFO:teuthology.orchestra.run.smithi103.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-17T05:48:32.363 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-04-17T05:48:32.363 INFO:teuthology.orchestra.run.smithi103.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-04-17T05:48:32.363 INFO:teuthology.orchestra.run.smithi103.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-17T05:48:32.363 INFO:teuthology.orchestra.run.smithi103.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-17T05:48:32.363 INFO:teuthology.orchestra.run.smithi103.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-17T05:48:32.363 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-17T05:48:32.363 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-17T05:48:32.364 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-17T05:48:32.364 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-17T05:48:32.364 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-17T05:48:32.364 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-17T05:48:32.364 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-04-17T05:48:32.364 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-04-17T05:48:32.364 INFO:teuthology.orchestra.run.smithi103.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-17T05:48:32.364 INFO:teuthology.orchestra.run.smithi103.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-17T05:48:32.364 INFO:teuthology.orchestra.run.smithi103.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-04-17T05:48:32.364 INFO:teuthology.orchestra.run.smithi103.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-04-17T05:48:32.364 INFO:teuthology.orchestra.run.smithi103.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-17T05:48:32.364 INFO:teuthology.orchestra.run.smithi103.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-17T05:48:32.364 INFO:teuthology.orchestra.run.smithi103.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-04-17T05:48:32.364 INFO:teuthology.orchestra.run.smithi103.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-04-17T05:48:32.364 INFO:teuthology.orchestra.run.smithi103.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-04-17T05:48:32.365 INFO:teuthology.orchestra.run.smithi103.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-04-17T05:48:32.365 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-17T05:48:32.365 INFO:teuthology.orchestra.run.smithi103.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-04-17T05:48:32.365 INFO:teuthology.orchestra.run.smithi103.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-17T05:48:32.365 INFO:teuthology.orchestra.run.smithi103.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-04-17T05:48:32.365 INFO:teuthology.orchestra.run.smithi103.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-04-17T05:48:32.365 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-17T05:48:32.365 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-04-17T05:48:32.365 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-04-17T05:48:32.365 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-04-17T05:48:32.365 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-17T05:48:32.365 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-04-17T05:48:32.365 INFO:teuthology.orchestra.run.smithi103.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-17T05:48:32.366 INFO:teuthology.orchestra.run.smithi103.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-17T05:48:32.366 INFO:teuthology.orchestra.run.smithi103.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-04-17T05:48:32.366 INFO:teuthology.orchestra.run.smithi103.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-17T05:48:32.366 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-17T05:48:32.366 INFO:teuthology.orchestra.run.smithi103.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-04-17T05:48:32.366 INFO:teuthology.orchestra.run.smithi103.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-17T05:48:32.366 INFO:teuthology.orchestra.run.smithi103.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-04-17T05:48:32.366 INFO:teuthology.orchestra.run.smithi103.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-17T05:48:32.366 INFO:teuthology.orchestra.run.smithi103.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-04-17T05:48:32.366 INFO:teuthology.orchestra.run.smithi103.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-04-17T05:48:32.366 INFO:teuthology.orchestra.run.smithi103.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-17T05:48:32.366 INFO:teuthology.orchestra.run.smithi103.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-17T05:48:32.366 INFO:teuthology.orchestra.run.smithi103.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-17T05:48:32.366 INFO:teuthology.orchestra.run.smithi103.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-17T05:48:32.367 INFO:teuthology.orchestra.run.smithi103.stdout: qatlib x86_64 24.02.0-1.el9 CentOS-AppStream 220 k 2024-04-17T05:48:32.367 INFO:teuthology.orchestra.run.smithi103.stdout: qatzip-libs x86_64 1.2.0-1.el9 CentOS-AppStream 46 k 2024-04-17T05:48:32.367 INFO:teuthology.orchestra.run.smithi103.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-17T05:48:32.367 INFO:teuthology.orchestra.run.smithi103.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-04-17T05:48:32.367 INFO:teuthology.orchestra.run.smithi103.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-17T05:48:32.367 INFO:teuthology.orchestra.run.smithi103.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-04-17T05:48:32.367 INFO:teuthology.orchestra.run.smithi103.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-04-17T05:48:32.367 INFO:teuthology.orchestra.run.smithi103.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-04-17T05:48:32.367 INFO:teuthology.orchestra.run.smithi103.stdout:Installing weak dependencies: 2024-04-17T05:48:32.367 INFO:teuthology.orchestra.run.smithi103.stdout: qatlib-service x86_64 24.02.0-1.el9 CentOS-AppStream 35 k 2024-04-17T05:48:32.367 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-04-17T05:48:32.367 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction Summary 2024-04-17T05:48:32.367 INFO:teuthology.orchestra.run.smithi103.stdout:============================================================================================ 2024-04-17T05:48:32.367 INFO:teuthology.orchestra.run.smithi103.stdout:Install 150 Packages 2024-04-17T05:48:32.368 INFO:teuthology.orchestra.run.smithi103.stdout:Upgrade 24 Packages 2024-04-17T05:48:32.368 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-04-17T05:48:32.368 INFO:teuthology.orchestra.run.smithi103.stdout:Total download size: 229 M 2024-04-17T05:48:32.368 INFO:teuthology.orchestra.run.smithi103.stdout:Downloading Packages: 2024-04-17T05:48:32.778 INFO:teuthology.orchestra.run.smithi098.stdout:(1/174): ceph-19.0.0-3133.ga8c84f59.el9.x86_64. 21 kB/s | 6.6 kB 00:00 2024-04-17T05:48:33.355 INFO:teuthology.orchestra.run.smithi103.stdout:(1/174): ceph-19.0.0-3133.ga8c84f59.el9.x86_64. 25 kB/s | 6.6 kB 00:00 2024-04-17T05:48:33.602 INFO:teuthology.orchestra.run.smithi047.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-17T05:48:33.603 INFO:teuthology.orchestra.run.smithi047.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-17T05:48:33.616 INFO:teuthology.orchestra.run.smithi047.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-04-17T05:48:33.617 INFO:teuthology.orchestra.run.smithi047.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-17T05:48:33.617 INFO:teuthology.orchestra.run.smithi047.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-04-17T05:48:33.618 INFO:teuthology.orchestra.run.smithi047.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-04-17T05:48:33.620 INFO:teuthology.orchestra.run.smithi047.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-17T05:48:33.623 INFO:teuthology.orchestra.run.smithi047.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-04-17T05:48:33.624 INFO:teuthology.orchestra.run.smithi047.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-04-17T05:48:33.711 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-04-17T05:48:33.733 INFO:teuthology.orchestra.run.smithi047.stdout:============================================================================================ 2024-04-17T05:48:33.733 INFO:teuthology.orchestra.run.smithi047.stdout: Package Arch Version Repository Size 2024-04-17T05:48:33.734 INFO:teuthology.orchestra.run.smithi047.stdout:============================================================================================ 2024-04-17T05:48:33.734 INFO:teuthology.orchestra.run.smithi047.stdout:Installing: 2024-04-17T05:48:33.734 INFO:teuthology.orchestra.run.smithi047.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-04-17T05:48:33.734 INFO:teuthology.orchestra.run.smithi047.stdout: ceph x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 6.6 k 2024-04-17T05:48:33.734 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-base x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 5.4 M 2024-04-17T05:48:33.734 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-fuse x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 893 k 2024-04-17T05:48:33.734 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 145 k 2024-04-17T05:48:33.734 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 1.5 M 2024-04-17T05:48:33.734 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 142 k 2024-04-17T05:48:33.734 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 3.5 M 2024-04-17T05:48:33.734 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 7.4 M 2024-04-17T05:48:33.734 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-rook noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 49 k 2024-04-17T05:48:33.735 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-radosgw x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 11 M 2024-04-17T05:48:33.735 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-test x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 49 M 2024-04-17T05:48:33.735 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-volume noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 273 k 2024-04-17T05:48:33.735 INFO:teuthology.orchestra.run.smithi047.stdout: cephadm noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 763 k 2024-04-17T05:48:33.735 INFO:teuthology.orchestra.run.smithi047.stdout: cephfs-mirror x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 225 k 2024-04-17T05:48:33.735 INFO:teuthology.orchestra.run.smithi047.stdout: cephfs-shell x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 27 k 2024-04-17T05:48:33.735 INFO:teuthology.orchestra.run.smithi047.stdout: cephfs-top noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 18 k 2024-04-17T05:48:33.735 INFO:teuthology.orchestra.run.smithi047.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-17T05:48:33.735 INFO:teuthology.orchestra.run.smithi047.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-04-17T05:48:33.735 INFO:teuthology.orchestra.run.smithi047.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-04-17T05:48:33.735 INFO:teuthology.orchestra.run.smithi047.stdout: libcephfs-devel x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 32 k 2024-04-17T05:48:33.735 INFO:teuthology.orchestra.run.smithi047.stdout: libcephfs2 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 748 k 2024-04-17T05:48:33.735 INFO:teuthology.orchestra.run.smithi047.stdout: librados-devel x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 126 k 2024-04-17T05:48:33.735 INFO:teuthology.orchestra.run.smithi047.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-04-17T05:48:33.735 INFO:teuthology.orchestra.run.smithi047.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-17T05:48:33.736 INFO:teuthology.orchestra.run.smithi047.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-17T05:48:33.736 INFO:teuthology.orchestra.run.smithi047.stdout: postgresql x86_64 13.14-1.el9 CentOS-AppStream 1.5 M 2024-04-17T05:48:33.736 INFO:teuthology.orchestra.run.smithi047.stdout: postgresql-contrib x86_64 13.14-1.el9 CentOS-AppStream 817 k 2024-04-17T05:48:33.736 INFO:teuthology.orchestra.run.smithi047.stdout: postgresql-server x86_64 13.14-1.el9 CentOS-AppStream 5.7 M 2024-04-17T05:48:33.736 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cephfs x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 162 k 2024-04-17T05:48:33.736 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rados x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 321 k 2024-04-17T05:48:33.736 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rbd x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 299 k 2024-04-17T05:48:33.736 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rgw x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 99 k 2024-04-17T05:48:33.736 INFO:teuthology.orchestra.run.smithi047.stdout: rbd-fuse x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 85 k 2024-04-17T05:48:33.736 INFO:teuthology.orchestra.run.smithi047.stdout: rbd-mirror x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 3.1 M 2024-04-17T05:48:33.736 INFO:teuthology.orchestra.run.smithi047.stdout: rbd-nbd x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 170 k 2024-04-17T05:48:33.736 INFO:teuthology.orchestra.run.smithi047.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-17T05:48:33.737 INFO:teuthology.orchestra.run.smithi047.stdout:Upgrading: 2024-04-17T05:48:33.737 INFO:teuthology.orchestra.run.smithi047.stdout: NetworkManager x86_64 1:1.47.4-1.el9 CentOS-BaseOS 2.3 M 2024-04-17T05:48:33.737 INFO:teuthology.orchestra.run.smithi047.stdout: NetworkManager-libnm x86_64 1:1.47.4-1.el9 CentOS-BaseOS 1.8 M 2024-04-17T05:48:33.737 INFO:teuthology.orchestra.run.smithi047.stdout: NetworkManager-team x86_64 1:1.47.4-1.el9 CentOS-BaseOS 41 k 2024-04-17T05:48:33.737 INFO:teuthology.orchestra.run.smithi047.stdout: NetworkManager-tui x86_64 1:1.47.4-1.el9 CentOS-BaseOS 248 k 2024-04-17T05:48:33.737 INFO:teuthology.orchestra.run.smithi047.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-04-17T05:48:33.737 INFO:teuthology.orchestra.run.smithi047.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-04-17T05:48:33.737 INFO:teuthology.orchestra.run.smithi047.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-04-17T05:48:33.737 INFO:teuthology.orchestra.run.smithi047.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-04-17T05:48:33.737 INFO:teuthology.orchestra.run.smithi047.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-04-17T05:48:33.737 INFO:teuthology.orchestra.run.smithi047.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-04-17T05:48:33.737 INFO:teuthology.orchestra.run.smithi047.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-04-17T05:48:33.737 INFO:teuthology.orchestra.run.smithi047.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-04-17T05:48:33.737 INFO:teuthology.orchestra.run.smithi047.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-04-17T05:48:33.738 INFO:teuthology.orchestra.run.smithi047.stdout: librados2 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 3.4 M 2024-04-17T05:48:33.738 INFO:teuthology.orchestra.run.smithi047.stdout: librbd1 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 3.1 M 2024-04-17T05:48:33.738 INFO:teuthology.orchestra.run.smithi047.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-04-17T05:48:33.738 INFO:teuthology.orchestra.run.smithi047.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-04-17T05:48:33.738 INFO:teuthology.orchestra.run.smithi047.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-04-17T05:48:33.738 INFO:teuthology.orchestra.run.smithi047.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-04-17T05:48:33.738 INFO:teuthology.orchestra.run.smithi047.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-04-17T05:48:33.738 INFO:teuthology.orchestra.run.smithi047.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-04-17T05:48:33.738 INFO:teuthology.orchestra.run.smithi047.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-04-17T05:48:33.738 INFO:teuthology.orchestra.run.smithi047.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-04-17T05:48:33.738 INFO:teuthology.orchestra.run.smithi047.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-04-17T05:48:33.738 INFO:teuthology.orchestra.run.smithi047.stdout:Installing dependencies: 2024-04-17T05:48:33.738 INFO:teuthology.orchestra.run.smithi047.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-17T05:48:33.738 INFO:teuthology.orchestra.run.smithi047.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-04-17T05:48:33.739 INFO:teuthology.orchestra.run.smithi047.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-04-17T05:48:33.739 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-common x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 21 M 2024-04-17T05:48:33.739 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 34 k 2024-04-17T05:48:33.739 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mds x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 2.4 M 2024-04-17T05:48:33.739 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 249 k 2024-04-17T05:48:33.739 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mon x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 4.7 M 2024-04-17T05:48:33.739 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-osd x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 17 M 2024-04-17T05:48:33.739 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 16 k 2024-04-17T05:48:33.739 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-selinux x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 25 k 2024-04-17T05:48:33.739 INFO:teuthology.orchestra.run.smithi047.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-04-17T05:48:33.739 INFO:teuthology.orchestra.run.smithi047.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-04-17T05:48:33.739 INFO:teuthology.orchestra.run.smithi047.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-04-17T05:48:33.739 INFO:teuthology.orchestra.run.smithi047.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-04-17T05:48:33.739 INFO:teuthology.orchestra.run.smithi047.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-17T05:48:33.740 INFO:teuthology.orchestra.run.smithi047.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-17T05:48:33.740 INFO:teuthology.orchestra.run.smithi047.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-04-17T05:48:33.740 INFO:teuthology.orchestra.run.smithi047.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-17T05:48:33.740 INFO:teuthology.orchestra.run.smithi047.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-17T05:48:33.740 INFO:teuthology.orchestra.run.smithi047.stdout: libcephsqlite x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 163 k 2024-04-17T05:48:33.740 INFO:teuthology.orchestra.run.smithi047.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-17T05:48:33.740 INFO:teuthology.orchestra.run.smithi047.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-17T05:48:33.740 INFO:teuthology.orchestra.run.smithi047.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-04-17T05:48:33.740 INFO:teuthology.orchestra.run.smithi047.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-17T05:48:33.740 INFO:teuthology.orchestra.run.smithi047.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-04-17T05:48:33.740 INFO:teuthology.orchestra.run.smithi047.stdout: libradosstriper1 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 501 k 2024-04-17T05:48:33.740 INFO:teuthology.orchestra.run.smithi047.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-04-17T05:48:33.740 INFO:teuthology.orchestra.run.smithi047.stdout: librgw2 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 5.4 M 2024-04-17T05:48:33.740 INFO:teuthology.orchestra.run.smithi047.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-17T05:48:33.741 INFO:teuthology.orchestra.run.smithi047.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-04-17T05:48:33.741 INFO:teuthology.orchestra.run.smithi047.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-04-17T05:48:33.741 INFO:teuthology.orchestra.run.smithi047.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-04-17T05:48:33.741 INFO:teuthology.orchestra.run.smithi047.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-17T05:48:33.741 INFO:teuthology.orchestra.run.smithi047.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-17T05:48:33.741 INFO:teuthology.orchestra.run.smithi047.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-04-17T05:48:33.741 INFO:teuthology.orchestra.run.smithi047.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-04-17T05:48:33.741 INFO:teuthology.orchestra.run.smithi047.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-04-17T05:48:33.741 INFO:teuthology.orchestra.run.smithi047.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-17T05:48:33.741 INFO:teuthology.orchestra.run.smithi047.stdout: postgresql-private-libs x86_64 13.14-1.el9 CentOS-AppStream 132 k 2024-04-17T05:48:33.741 INFO:teuthology.orchestra.run.smithi047.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-04-17T05:48:33.741 INFO:teuthology.orchestra.run.smithi047.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-17T05:48:33.741 INFO:teuthology.orchestra.run.smithi047.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-17T05:48:33.741 INFO:teuthology.orchestra.run.smithi047.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-04-17T05:48:33.741 INFO:teuthology.orchestra.run.smithi047.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-04-17T05:48:33.742 INFO:teuthology.orchestra.run.smithi047.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-17T05:48:33.742 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-17T05:48:33.742 INFO:teuthology.orchestra.run.smithi047.stdout: python3-ceph-argparse x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 45 k 2024-04-17T05:48:33.742 INFO:teuthology.orchestra.run.smithi047.stdout: python3-ceph-common x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 135 k 2024-04-17T05:48:33.742 INFO:teuthology.orchestra.run.smithi047.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-17T05:48:33.742 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-04-17T05:48:33.742 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-17T05:48:33.742 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-17T05:48:33.742 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-17T05:48:33.743 INFO:teuthology.orchestra.run.smithi047.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-17T05:48:33.743 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-04-17T05:48:33.743 INFO:teuthology.orchestra.run.smithi047.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-04-17T05:48:33.743 INFO:teuthology.orchestra.run.smithi047.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-17T05:48:33.743 INFO:teuthology.orchestra.run.smithi047.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-17T05:48:33.743 INFO:teuthology.orchestra.run.smithi047.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-17T05:48:33.743 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-17T05:48:33.743 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-17T05:48:33.743 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-17T05:48:33.743 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-17T05:48:33.743 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-17T05:48:33.743 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-17T05:48:33.743 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-04-17T05:48:33.743 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-04-17T05:48:33.744 INFO:teuthology.orchestra.run.smithi047.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-17T05:48:33.744 INFO:teuthology.orchestra.run.smithi047.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-17T05:48:33.744 INFO:teuthology.orchestra.run.smithi047.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-04-17T05:48:33.744 INFO:teuthology.orchestra.run.smithi047.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-04-17T05:48:33.744 INFO:teuthology.orchestra.run.smithi047.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-17T05:48:33.744 INFO:teuthology.orchestra.run.smithi047.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-17T05:48:33.744 INFO:teuthology.orchestra.run.smithi047.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-04-17T05:48:33.744 INFO:teuthology.orchestra.run.smithi047.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-04-17T05:48:33.744 INFO:teuthology.orchestra.run.smithi047.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-04-17T05:48:33.744 INFO:teuthology.orchestra.run.smithi047.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-04-17T05:48:33.744 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-17T05:48:33.744 INFO:teuthology.orchestra.run.smithi047.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-04-17T05:48:33.744 INFO:teuthology.orchestra.run.smithi047.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-17T05:48:33.744 INFO:teuthology.orchestra.run.smithi047.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-04-17T05:48:33.745 INFO:teuthology.orchestra.run.smithi047.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-04-17T05:48:33.745 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-17T05:48:33.745 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-04-17T05:48:33.745 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-04-17T05:48:33.745 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-04-17T05:48:33.745 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-17T05:48:33.745 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-04-17T05:48:33.745 INFO:teuthology.orchestra.run.smithi047.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-17T05:48:33.745 INFO:teuthology.orchestra.run.smithi047.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-17T05:48:33.745 INFO:teuthology.orchestra.run.smithi047.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-04-17T05:48:33.745 INFO:teuthology.orchestra.run.smithi047.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-17T05:48:33.745 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-17T05:48:33.745 INFO:teuthology.orchestra.run.smithi047.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-04-17T05:48:33.745 INFO:teuthology.orchestra.run.smithi047.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-17T05:48:33.745 INFO:teuthology.orchestra.run.smithi047.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-04-17T05:48:33.746 INFO:teuthology.orchestra.run.smithi047.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-17T05:48:33.746 INFO:teuthology.orchestra.run.smithi047.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-04-17T05:48:33.746 INFO:teuthology.orchestra.run.smithi047.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-04-17T05:48:33.746 INFO:teuthology.orchestra.run.smithi047.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-17T05:48:33.746 INFO:teuthology.orchestra.run.smithi047.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-17T05:48:33.746 INFO:teuthology.orchestra.run.smithi047.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-17T05:48:33.746 INFO:teuthology.orchestra.run.smithi047.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-17T05:48:33.746 INFO:teuthology.orchestra.run.smithi047.stdout: qatlib x86_64 24.02.0-1.el9 CentOS-AppStream 220 k 2024-04-17T05:48:33.746 INFO:teuthology.orchestra.run.smithi047.stdout: qatzip-libs x86_64 1.2.0-1.el9 CentOS-AppStream 46 k 2024-04-17T05:48:33.746 INFO:teuthology.orchestra.run.smithi047.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-17T05:48:33.746 INFO:teuthology.orchestra.run.smithi047.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-04-17T05:48:33.746 INFO:teuthology.orchestra.run.smithi047.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-17T05:48:33.746 INFO:teuthology.orchestra.run.smithi047.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-04-17T05:48:33.746 INFO:teuthology.orchestra.run.smithi047.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-04-17T05:48:33.747 INFO:teuthology.orchestra.run.smithi047.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-04-17T05:48:33.747 INFO:teuthology.orchestra.run.smithi047.stdout:Installing weak dependencies: 2024-04-17T05:48:33.747 INFO:teuthology.orchestra.run.smithi047.stdout: qatlib-service x86_64 24.02.0-1.el9 CentOS-AppStream 35 k 2024-04-17T05:48:33.747 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-17T05:48:33.747 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction Summary 2024-04-17T05:48:33.747 INFO:teuthology.orchestra.run.smithi047.stdout:============================================================================================ 2024-04-17T05:48:33.747 INFO:teuthology.orchestra.run.smithi047.stdout:Install 150 Packages 2024-04-17T05:48:33.747 INFO:teuthology.orchestra.run.smithi047.stdout:Upgrade 24 Packages 2024-04-17T05:48:33.747 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-17T05:48:33.748 INFO:teuthology.orchestra.run.smithi047.stdout:Total download size: 229 M 2024-04-17T05:48:33.748 INFO:teuthology.orchestra.run.smithi047.stdout:Downloading Packages: 2024-04-17T05:48:33.795 INFO:teuthology.orchestra.run.smithi098.stdout:(2/174): ceph-fuse-19.0.0-3133.ga8c84f59.el9.x8 878 kB/s | 893 kB 00:01 2024-04-17T05:48:33.895 INFO:teuthology.orchestra.run.smithi098.stdout:(3/174): ceph-immutable-object-cache-19.0.0-313 1.4 MB/s | 145 kB 00:00 2024-04-17T05:48:34.037 INFO:teuthology.orchestra.run.smithi098.stdout:(4/174): ceph-base-19.0.0-3133.ga8c84f59.el9.x8 3.4 MB/s | 5.4 MB 00:01 2024-04-17T05:48:34.138 INFO:teuthology.orchestra.run.smithi103.stdout:(2/174): ceph-fuse-19.0.0-3133.ga8c84f59.el9.x8 1.1 MB/s | 893 kB 00:00 2024-04-17T05:48:34.221 INFO:teuthology.orchestra.run.smithi098.stdout:(5/174): ceph-mgr-19.0.0-3133.ga8c84f59.el9.x86 8.2 MB/s | 1.5 MB 00:00 2024-04-17T05:48:34.263 INFO:teuthology.orchestra.run.smithi103.stdout:(3/174): ceph-immutable-object-cache-19.0.0-313 1.1 MB/s | 145 kB 00:00 2024-04-17T05:48:34.490 INFO:teuthology.orchestra.run.smithi047.stdout:(1/174): ceph-19.0.0-3133.ga8c84f59.el9.x86_64. 20 kB/s | 6.6 kB 00:00 2024-04-17T05:48:34.580 INFO:teuthology.orchestra.run.smithi103.stdout:(4/174): ceph-base-19.0.0-3133.ga8c84f59.el9.x8 3.6 MB/s | 5.4 MB 00:01 2024-04-17T05:48:34.721 INFO:teuthology.orchestra.run.smithi098.stdout:(6/174): ceph-mon-19.0.0-3133.ga8c84f59.el9.x86 9.4 MB/s | 4.7 MB 00:00 2024-04-17T05:48:34.843 INFO:teuthology.orchestra.run.smithi103.stdout:(5/174): ceph-mgr-19.0.0-3133.ga8c84f59.el9.x86 5.8 MB/s | 1.5 MB 00:00 2024-04-17T05:48:34.955 INFO:teuthology.orchestra.run.smithi098.stdout:(7/174): ceph-mds-19.0.0-3133.ga8c84f59.el9.x86 2.2 MB/s | 2.4 MB 00:01 2024-04-17T05:48:35.432 INFO:teuthology.orchestra.run.smithi047.stdout:(2/174): ceph-fuse-19.0.0-3133.ga8c84f59.el9.x8 948 kB/s | 893 kB 00:00 2024-04-17T05:48:35.632 INFO:teuthology.orchestra.run.smithi047.stdout:(3/174): ceph-immutable-object-cache-19.0.0-313 725 kB/s | 145 kB 00:00 2024-04-17T05:48:35.673 INFO:teuthology.orchestra.run.smithi103.stdout:(6/174): ceph-mds-19.0.0-3133.ga8c84f59.el9.x86 1.7 MB/s | 2.4 MB 00:01 2024-04-17T05:48:35.981 INFO:teuthology.orchestra.run.smithi103.stdout:(7/174): ceph-mon-19.0.0-3133.ga8c84f59.el9.x86 4.1 MB/s | 4.7 MB 00:01 2024-04-17T05:48:36.874 INFO:teuthology.orchestra.run.smithi047.stdout:(4/174): ceph-mds-19.0.0-3133.ga8c84f59.el9.x86 1.9 MB/s | 2.4 MB 00:01 2024-04-17T05:48:37.408 INFO:teuthology.orchestra.run.smithi047.stdout:(5/174): ceph-mgr-19.0.0-3133.ga8c84f59.el9.x86 2.8 MB/s | 1.5 MB 00:00 2024-04-17T05:48:37.488 INFO:teuthology.orchestra.run.smithi098.stdout:(8/174): ceph-osd-19.0.0-3133.ga8c84f59.el9.x86 6.1 MB/s | 17 MB 00:02 2024-04-17T05:48:37.516 INFO:teuthology.orchestra.run.smithi047.stdout:(6/174): ceph-base-19.0.0-3133.ga8c84f59.el9.x8 1.6 MB/s | 5.4 MB 00:03 2024-04-17T05:48:37.555 INFO:teuthology.orchestra.run.smithi098.stdout:(9/174): ceph-selinux-19.0.0-3133.ga8c84f59.el9 381 kB/s | 25 kB 00:00 2024-04-17T05:48:38.500 INFO:teuthology.orchestra.run.smithi047.stdout:(7/174): ceph-mon-19.0.0-3133.ga8c84f59.el9.x86 4.3 MB/s | 4.7 MB 00:01 2024-04-17T05:48:38.573 INFO:teuthology.orchestra.run.smithi103.stdout:(8/174): ceph-common-19.0.0-3133.ga8c84f59.el9. 3.9 MB/s | 21 MB 00:05 2024-04-17T05:48:38.630 INFO:teuthology.orchestra.run.smithi098.stdout:(10/174): ceph-radosgw-19.0.0-3133.ga8c84f59.el 2.9 MB/s | 11 MB 00:03 2024-04-17T05:48:38.681 INFO:teuthology.orchestra.run.smithi103.stdout:(9/174): ceph-selinux-19.0.0-3133.ga8c84f59.el9 232 kB/s | 25 kB 00:00 2024-04-17T05:48:38.739 INFO:teuthology.orchestra.run.smithi098.stdout:(11/174): cephfs-mirror-19.0.0-3133.ga8c84f59.e 2.0 MB/s | 225 kB 00:00 2024-04-17T05:48:38.848 INFO:teuthology.orchestra.run.smithi098.stdout:(12/174): cephfs-shell-19.0.0-3133.ga8c84f59.el 250 kB/s | 27 kB 00:00 2024-04-17T05:48:38.956 INFO:teuthology.orchestra.run.smithi098.stdout:(13/174): libcephfs-devel-19.0.0-3133.ga8c84f59 292 kB/s | 32 kB 00:00 2024-04-17T05:48:39.090 INFO:teuthology.orchestra.run.smithi103.stdout:(10/174): ceph-osd-19.0.0-3133.ga8c84f59.el9.x8 4.9 MB/s | 17 MB 00:03 2024-04-17T05:48:39.207 INFO:teuthology.orchestra.run.smithi103.stdout:(11/174): cephfs-mirror-19.0.0-3133.ga8c84f59.e 1.9 MB/s | 225 kB 00:00 2024-04-17T05:48:39.307 INFO:teuthology.orchestra.run.smithi098.stdout:(14/174): libcephfs2-19.0.0-3133.ga8c84f59.el9. 2.1 MB/s | 748 kB 00:00 2024-04-17T05:48:39.315 INFO:teuthology.orchestra.run.smithi103.stdout:(12/174): cephfs-shell-19.0.0-3133.ga8c84f59.el 249 kB/s | 27 kB 00:00 2024-04-17T05:48:39.424 INFO:teuthology.orchestra.run.smithi098.stdout:(15/174): libcephsqlite-19.0.0-3133.ga8c84f59.e 1.4 MB/s | 163 kB 00:00 2024-04-17T05:48:39.424 INFO:teuthology.orchestra.run.smithi103.stdout:(13/174): libcephfs-devel-19.0.0-3133.ga8c84f59 293 kB/s | 32 kB 00:00 2024-04-17T05:48:39.524 INFO:teuthology.orchestra.run.smithi098.stdout:(16/174): librados-devel-19.0.0-3133.ga8c84f59. 1.2 MB/s | 126 kB 00:00 2024-04-17T05:48:39.633 INFO:teuthology.orchestra.run.smithi103.stdout:(14/174): ceph-radosgw-19.0.0-3133.ga8c84f59.el 2.9 MB/s | 11 MB 00:03 2024-04-17T05:48:39.683 INFO:teuthology.orchestra.run.smithi103.stdout:(15/174): libcephfs2-19.0.0-3133.ga8c84f59.el9. 2.8 MB/s | 748 kB 00:00 2024-04-17T05:48:39.716 INFO:teuthology.orchestra.run.smithi103.stdout:(16/174): libcephsqlite-19.0.0-3133.ga8c84f59.e 1.9 MB/s | 163 kB 00:00 2024-04-17T05:48:39.758 INFO:teuthology.orchestra.run.smithi098.stdout:(17/174): libradosstriper1-19.0.0-3133.ga8c84f5 2.1 MB/s | 501 kB 00:00 2024-04-17T05:48:39.775 INFO:teuthology.orchestra.run.smithi103.stdout:(17/174): librados-devel-19.0.0-3133.ga8c84f59. 1.3 MB/s | 126 kB 00:00 2024-04-17T05:48:39.850 INFO:teuthology.orchestra.run.smithi047.stdout:(8/174): ceph-radosgw-19.0.0-3133.ga8c84f59.el9 7.8 MB/s | 11 MB 00:01 2024-04-17T05:48:39.884 INFO:teuthology.orchestra.run.smithi103.stdout:(18/174): libradosstriper1-19.0.0-3133.ga8c84f5 2.9 MB/s | 501 kB 00:00 2024-04-17T05:48:39.933 INFO:teuthology.orchestra.run.smithi047.stdout:(9/174): ceph-selinux-19.0.0-3133.ga8c84f59.el9 303 kB/s | 25 kB 00:00 2024-04-17T05:48:39.942 INFO:teuthology.orchestra.run.smithi103.stdout:(19/174): python3-ceph-argparse-19.0.0-3133.ga8 780 kB/s | 45 kB 00:00 2024-04-17T05:48:40.042 INFO:teuthology.orchestra.run.smithi103.stdout:(20/174): python3-ceph-common-19.0.0-3133.ga8c8 1.3 MB/s | 135 kB 00:00 2024-04-17T05:48:40.143 INFO:teuthology.orchestra.run.smithi103.stdout:(21/174): python3-cephfs-19.0.0-3133.ga8c84f59. 1.6 MB/s | 162 kB 00:00 2024-04-17T05:48:40.318 INFO:teuthology.orchestra.run.smithi103.stdout:(22/174): python3-rados-19.0.0-3133.ga8c84f59.e 1.8 MB/s | 321 kB 00:00 2024-04-17T05:48:40.502 INFO:teuthology.orchestra.run.smithi103.stdout:(23/174): python3-rbd-19.0.0-3133.ga8c84f59.el9 1.6 MB/s | 299 kB 00:00 2024-04-17T05:48:40.602 INFO:teuthology.orchestra.run.smithi103.stdout:(24/174): python3-rgw-19.0.0-3133.ga8c84f59.el9 994 kB/s | 99 kB 00:00 2024-04-17T05:48:40.702 INFO:teuthology.orchestra.run.smithi103.stdout:(25/174): rbd-fuse-19.0.0-3133.ga8c84f59.el9.x8 852 kB/s | 85 kB 00:00 2024-04-17T05:48:41.109 INFO:teuthology.orchestra.run.smithi047.stdout:(10/174): ceph-common-19.0.0-3133.ga8c84f59.el9 3.0 MB/s | 21 MB 00:06 2024-04-17T05:48:41.292 INFO:teuthology.orchestra.run.smithi047.stdout:(11/174): cephfs-mirror-19.0.0-3133.ga8c84f59.e 1.2 MB/s | 225 kB 00:00 2024-04-17T05:48:41.400 INFO:teuthology.orchestra.run.smithi047.stdout:(12/174): cephfs-shell-19.0.0-3133.ga8c84f59.el 250 kB/s | 27 kB 00:00 2024-04-17T05:48:41.402 INFO:teuthology.orchestra.run.smithi103.stdout:(26/174): librgw2-19.0.0-3133.ga8c84f59.el9.x86 3.3 MB/s | 5.4 MB 00:01 2024-04-17T05:48:41.501 INFO:teuthology.orchestra.run.smithi047.stdout:(13/174): libcephfs-devel-19.0.0-3133.ga8c84f59 317 kB/s | 32 kB 00:00 2024-04-17T05:48:41.511 INFO:teuthology.orchestra.run.smithi103.stdout:(27/174): rbd-nbd-19.0.0-3133.ga8c84f59.el9.x86 1.5 MB/s | 170 kB 00:00 2024-04-17T05:48:41.608 INFO:teuthology.orchestra.run.smithi098.stdout:(18/174): librgw2-19.0.0-3133.ga8c84f59.el9.x86 2.9 MB/s | 5.4 MB 00:01 2024-04-17T05:48:41.611 INFO:teuthology.orchestra.run.smithi103.stdout:(28/174): ceph-grafana-dashboards-19.0.0-3133.g 342 kB/s | 34 kB 00:00 2024-04-17T05:48:41.699 INFO:teuthology.orchestra.run.smithi098.stdout:(19/174): python3-ceph-argparse-19.0.0-3133.ga8 494 kB/s | 45 kB 00:00 2024-04-17T05:48:41.703 INFO:teuthology.orchestra.run.smithi103.stdout:(29/174): ceph-mgr-cephadm-19.0.0-3133.ga8c84f5 1.5 MB/s | 142 kB 00:00 2024-04-17T05:48:41.792 INFO:teuthology.orchestra.run.smithi098.stdout:(20/174): python3-ceph-common-19.0.0-3133.ga8c8 1.4 MB/s | 135 kB 00:00 2024-04-17T05:48:41.818 INFO:teuthology.orchestra.run.smithi047.stdout:(14/174): libcephfs2-19.0.0-3133.ga8c84f59.el9. 2.3 MB/s | 748 kB 00:00 2024-04-17T05:48:41.878 INFO:teuthology.orchestra.run.smithi103.stdout:(30/174): rbd-mirror-19.0.0-3133.ga8c84f59.el9. 2.6 MB/s | 3.1 MB 00:01 2024-04-17T05:48:41.884 INFO:teuthology.orchestra.run.smithi098.stdout:(21/174): python3-cephfs-19.0.0-3133.ga8c84f59. 1.7 MB/s | 162 kB 00:00 2024-04-17T05:48:41.926 INFO:teuthology.orchestra.run.smithi047.stdout:(15/174): libcephsqlite-19.0.0-3133.ga8c84f59.e 1.5 MB/s | 163 kB 00:00 2024-04-17T05:48:42.026 INFO:teuthology.orchestra.run.smithi047.stdout:(16/174): librados-devel-19.0.0-3133.ga8c84f59. 1.2 MB/s | 126 kB 00:00 2024-04-17T05:48:42.067 INFO:teuthology.orchestra.run.smithi098.stdout:(22/174): python3-rados-19.0.0-3133.ga8c84f59.e 1.7 MB/s | 321 kB 00:00 2024-04-17T05:48:42.251 INFO:teuthology.orchestra.run.smithi098.stdout:(23/174): python3-rbd-19.0.0-3133.ga8c84f59.el9 1.6 MB/s | 299 kB 00:00 2024-04-17T05:48:42.293 INFO:teuthology.orchestra.run.smithi047.stdout:(17/174): libradosstriper1-19.0.0-3133.ga8c84f5 1.8 MB/s | 501 kB 00:00 2024-04-17T05:48:42.351 INFO:teuthology.orchestra.run.smithi098.stdout:(24/174): python3-rgw-19.0.0-3133.ga8c84f59.el9 994 kB/s | 99 kB 00:00 2024-04-17T05:48:42.451 INFO:teuthology.orchestra.run.smithi098.stdout:(25/174): rbd-fuse-19.0.0-3133.ga8c84f59.el9.x8 851 kB/s | 85 kB 00:00 2024-04-17T05:48:42.671 INFO:teuthology.orchestra.run.smithi103.stdout:(31/174): ceph-mgr-dashboard-19.0.0-3133.ga8c84 3.7 MB/s | 3.5 MB 00:00 2024-04-17T05:48:42.779 INFO:teuthology.orchestra.run.smithi103.stdout:(32/174): ceph-mgr-modules-core-19.0.0-3133.ga8 2.2 MB/s | 249 kB 00:00 2024-04-17T05:48:42.879 INFO:teuthology.orchestra.run.smithi103.stdout:(33/174): ceph-mgr-rook-19.0.0-3133.ga8c84f59.e 494 kB/s | 49 kB 00:00 2024-04-17T05:48:42.980 INFO:teuthology.orchestra.run.smithi103.stdout:(34/174): ceph-prometheus-alerts-19.0.0-3133.ga 164 kB/s | 16 kB 00:00 2024-04-17T05:48:43.088 INFO:teuthology.orchestra.run.smithi103.stdout:(35/174): ceph-volume-19.0.0-3133.ga8c84f59.el9 2.5 MB/s | 273 kB 00:00 2024-04-17T05:48:43.363 INFO:teuthology.orchestra.run.smithi103.stdout:(36/174): cephadm-19.0.0-3133.ga8c84f59.el9.noa 2.7 MB/s | 763 kB 00:00 2024-04-17T05:48:43.464 INFO:teuthology.orchestra.run.smithi103.stdout:(37/174): cephfs-top-19.0.0-3133.ga8c84f59.el9. 182 kB/s | 18 kB 00:00 2024-04-17T05:48:43.502 INFO:teuthology.orchestra.run.smithi098.stdout:(26/174): rbd-mirror-19.0.0-3133.ga8c84f59.el9. 2.9 MB/s | 3.1 MB 00:01 2024-04-17T05:48:43.594 INFO:teuthology.orchestra.run.smithi103.stdout:(38/174): automake-1.16.2-8.el9.noarch.rpm 5.0 MB/s | 662 kB 00:00 2024-04-17T05:48:43.610 INFO:teuthology.orchestra.run.smithi098.stdout:(27/174): rbd-nbd-19.0.0-3133.ga8c84f59.el9.x86 1.5 MB/s | 170 kB 00:00 2024-04-17T05:48:43.652 INFO:teuthology.orchestra.run.smithi103.stdout:(39/174): bison-3.7.4-5.el9.x86_64.rpm 15 MB/s | 920 kB 00:00 2024-04-17T05:48:43.686 INFO:teuthology.orchestra.run.smithi103.stdout:(40/174): boost-program-options-1.75.0-8.el9.x8 3.1 MB/s | 106 kB 00:00 2024-04-17T05:48:43.702 INFO:teuthology.orchestra.run.smithi098.stdout:(28/174): ceph-grafana-dashboards-19.0.0-3133.g 372 kB/s | 34 kB 00:00 2024-04-17T05:48:43.728 INFO:teuthology.orchestra.run.smithi103.stdout:(41/174): flex-2.6.4-9.el9.x86_64.rpm 7.2 MB/s | 307 kB 00:00 2024-04-17T05:48:43.762 INFO:teuthology.orchestra.run.smithi103.stdout:(42/174): flexiblas-3.0.4-8.el9.x86_64.rpm 947 kB/s | 32 kB 00:00 2024-04-17T05:48:43.803 INFO:teuthology.orchestra.run.smithi098.stdout:(29/174): ceph-mgr-cephadm-19.0.0-3133.ga8c84f5 1.4 MB/s | 142 kB 00:00 2024-04-17T05:48:43.854 INFO:teuthology.orchestra.run.smithi103.stdout:(43/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 33 MB/s | 3.0 MB 00:00 2024-04-17T05:48:43.887 INFO:teuthology.orchestra.run.smithi103.stdout:(44/174): flexiblas-openblas-openmp-3.0.4-8.el9 517 kB/s | 17 kB 00:00 2024-04-17T05:48:43.921 INFO:teuthology.orchestra.run.smithi103.stdout:(45/174): libaio-devel-0.3.111-13.el9.x86_64.rp 302 kB/s | 10 kB 00:00 2024-04-17T05:48:43.946 INFO:teuthology.orchestra.run.smithi103.stdout:(46/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 6.2 MB/s | 159 kB 00:00 2024-04-17T05:48:43.980 INFO:teuthology.orchestra.run.smithi103.stdout:(47/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.3 MB/s | 44 kB 00:00 2024-04-17T05:48:43.985 INFO:teuthology.orchestra.run.smithi047.stdout:(18/174): librgw2-19.0.0-3133.ga8c84f59.el9.x86 3.2 MB/s | 5.4 MB 00:01 2024-04-17T05:48:44.030 INFO:teuthology.orchestra.run.smithi103.stdout:(48/174): librdkafka-1.6.1-102.el9.x86_64.rpm 13 MB/s | 661 kB 00:00 2024-04-17T05:48:44.081 INFO:teuthology.orchestra.run.smithi103.stdout:(49/174): libtool-2.4.6-45.el9.x86_64.rpm 11 MB/s | 578 kB 00:00 2024-04-17T05:48:44.114 INFO:teuthology.orchestra.run.smithi103.stdout:(50/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.4 MB/s | 46 kB 00:00 2024-04-17T05:48:44.127 INFO:teuthology.orchestra.run.smithi047.stdout:(19/174): python3-ceph-argparse-19.0.0-3133.ga8 318 kB/s | 45 kB 00:00 2024-04-17T05:48:44.156 INFO:teuthology.orchestra.run.smithi103.stdout:(51/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 6.6 MB/s | 282 kB 00:00 2024-04-17T05:48:44.190 INFO:teuthology.orchestra.run.smithi103.stdout:(52/174): lua-5.4.4-4.el9.x86_64.rpm 5.5 MB/s | 187 kB 00:00 2024-04-17T05:48:44.219 INFO:teuthology.orchestra.run.smithi047.stdout:(20/174): python3-ceph-common-19.0.0-3133.ga8c8 1.4 MB/s | 135 kB 00:00 2024-04-17T05:48:44.223 INFO:teuthology.orchestra.run.smithi103.stdout:(53/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 6.3 MB/s | 217 kB 00:00 2024-04-17T05:48:44.257 INFO:teuthology.orchestra.run.smithi103.stdout:(54/174): openblas-0.3.21-2.el9.x86_64.rpm 1.0 MB/s | 34 kB 00:00 2024-04-17T05:48:44.328 INFO:teuthology.orchestra.run.smithi047.stdout:(21/174): python3-cephfs-19.0.0-3133.ga8c84f59. 1.5 MB/s | 162 kB 00:00 2024-04-17T05:48:44.383 INFO:teuthology.orchestra.run.smithi103.stdout:(55/174): openblas-openmp-0.3.21-2.el9.x86_64.r 38 MB/s | 4.7 MB 00:00 2024-04-17T05:48:44.449 INFO:teuthology.orchestra.run.smithi103.stdout:(56/174): postgresql-13.14-1.el9.x86_64.rpm 23 MB/s | 1.5 MB 00:00 2024-04-17T05:48:44.495 INFO:teuthology.orchestra.run.smithi047.stdout:(22/174): python3-rados-19.0.0-3133.ga8c84f59.e 1.9 MB/s | 321 kB 00:00 2024-04-17T05:48:44.669 INFO:teuthology.orchestra.run.smithi103.stdout:(57/174): ceph-mgr-diskprediction-local-19.0.0- 2.7 MB/s | 7.4 MB 00:02 2024-04-17T05:48:44.687 INFO:teuthology.orchestra.run.smithi047.stdout:(23/174): ceph-osd-19.0.0-3133.ga8c84f59.el9.x8 2.3 MB/s | 17 MB 00:07 2024-04-17T05:48:44.700 INFO:teuthology.orchestra.run.smithi103.stdout:(58/174): postgresql-contrib-13.14-1.el9.x86_64 3.2 MB/s | 817 kB 00:00 2024-04-17T05:48:44.720 INFO:teuthology.orchestra.run.smithi047.stdout:(24/174): python3-rbd-19.0.0-3133.ga8c84f59.el9 1.3 MB/s | 299 kB 00:00 2024-04-17T05:48:44.725 INFO:teuthology.orchestra.run.smithi103.stdout:(59/174): postgresql-private-libs-13.14-1.el9.x 2.4 MB/s | 132 kB 00:00 2024-04-17T05:48:44.746 INFO:teuthology.orchestra.run.smithi047.stdout:(25/174): python3-rgw-19.0.0-3133.ga8c84f59.el9 1.7 MB/s | 99 kB 00:00 2024-04-17T05:48:44.753 INFO:teuthology.orchestra.run.smithi098.stdout:(30/174): ceph-mgr-dashboard-19.0.0-3133.ga8c84 3.7 MB/s | 3.5 MB 00:00 2024-04-17T05:48:44.804 INFO:teuthology.orchestra.run.smithi047.stdout:(26/174): rbd-fuse-19.0.0-3133.ga8c84f59.el9.x8 1.0 MB/s | 85 kB 00:00 2024-04-17T05:48:44.843 INFO:teuthology.orchestra.run.smithi103.stdout:(60/174): postgresql-server-13.14-1.el9.x86_64. 40 MB/s | 5.7 MB 00:00 2024-04-17T05:48:44.876 INFO:teuthology.orchestra.run.smithi103.stdout:(61/174): protobuf-3.14.0-13.el9.x86_64.rpm 6.8 MB/s | 1.0 MB 00:00 2024-04-17T05:48:44.888 INFO:teuthology.orchestra.run.smithi047.stdout:(27/174): rbd-nbd-19.0.0-3133.ga8c84f59.el9.x86 2.0 MB/s | 170 kB 00:00 2024-04-17T05:48:44.901 INFO:teuthology.orchestra.run.smithi103.stdout:(62/174): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 78 kB 00:00 2024-04-17T05:48:44.980 INFO:teuthology.orchestra.run.smithi047.stdout:(28/174): ceph-grafana-dashboards-19.0.0-3133.g 373 kB/s | 34 kB 00:00 2024-04-17T05:48:45.035 INFO:teuthology.orchestra.run.smithi103.stdout:(63/174): python3-babel-2.9.1-2.el9.noarch.rpm 36 MB/s | 5.8 MB 00:00 2024-04-17T05:48:45.052 INFO:teuthology.orchestra.run.smithi103.stdout:(64/174): python3-jsonpatch-1.21-16.el9.noarch. 165 kB/s | 24 kB 00:00 2024-04-17T05:48:45.069 INFO:teuthology.orchestra.run.smithi103.stdout:(65/174): python3-jsonpointer-2.0-4.el9.noarch. 540 kB/s | 18 kB 00:00 2024-04-17T05:48:45.072 INFO:teuthology.orchestra.run.smithi047.stdout:(29/174): ceph-mgr-cephadm-19.0.0-3133.ga8c84f5 1.5 MB/s | 142 kB 00:00 2024-04-17T05:48:45.094 INFO:teuthology.orchestra.run.smithi103.stdout:(66/174): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 144 kB 00:00 2024-04-17T05:48:45.111 INFO:teuthology.orchestra.run.smithi103.stdout:(67/174): python3-markupsafe-1.1.1-12.el9.x86_6 759 kB/s | 32 kB 00:00 2024-04-17T05:48:45.220 INFO:teuthology.orchestra.run.smithi103.stdout:(68/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 38 MB/s | 4.8 MB 00:00 2024-04-17T05:48:45.245 INFO:teuthology.orchestra.run.smithi103.stdout:(69/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.6 MB/s | 217 kB 00:00 2024-04-17T05:48:45.270 INFO:teuthology.orchestra.run.smithi103.stdout:(70/174): python3-oauthlib-3.1.1-5.el9.noarch.r 3.4 MB/s | 174 kB 00:00 2024-04-17T05:48:45.296 INFO:teuthology.orchestra.run.smithi103.stdout:(71/174): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 69 kB 00:00 2024-04-17T05:48:45.321 INFO:teuthology.orchestra.run.smithi103.stdout:(72/174): python3-prettytable-0.7.2-27.el9.noar 823 kB/s | 41 kB 00:00 2024-04-17T05:48:45.346 INFO:teuthology.orchestra.run.smithi103.stdout:(73/174): python3-protobuf-3.14.0-13.el9.noarch 4.6 MB/s | 238 kB 00:00 2024-04-17T05:48:45.380 INFO:teuthology.orchestra.run.smithi103.stdout:(74/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.2 MB/s | 132 kB 00:00 2024-04-17T05:48:45.414 INFO:teuthology.orchestra.run.smithi103.stdout:(75/174): python3-pyasn1-modules-0.4.8-6.el9.no 3.1 MB/s | 211 kB 00:00 2024-04-17T05:48:45.439 INFO:teuthology.orchestra.run.smithi103.stdout:(76/174): python3-pytz-2021.1-5.el9.noarch.rpm 811 kB/s | 47 kB 00:00 2024-04-17T05:48:45.462 INFO:teuthology.orchestra.run.smithi098.stdout:(31/174): ceph-common-19.0.0-3133.ga8c84f59.el9 1.6 MB/s | 21 MB 00:13 2024-04-17T05:48:45.464 INFO:teuthology.orchestra.run.smithi103.stdout:(77/174): python3-requests-oauthlib-1.3.0-12.el 852 kB/s | 43 kB 00:00 2024-04-17T05:48:45.631 INFO:teuthology.orchestra.run.smithi103.stdout:(78/174): python3-toml-0.10.2-6.el9.noarch.rpm 235 kB/s | 39 kB 00:00 2024-04-17T05:48:45.637 INFO:teuthology.orchestra.run.smithi098.stdout:(32/174): ceph-mgr-modules-core-19.0.0-3133.ga8 1.4 MB/s | 249 kB 00:00 2024-04-17T05:48:45.664 INFO:teuthology.orchestra.run.smithi047.stdout:(30/174): rbd-mirror-19.0.0-3133.ga8c84f59.el9. 3.4 MB/s | 3.1 MB 00:00 2024-04-17T05:48:45.687 INFO:teuthology.orchestra.run.smithi098.stdout:(33/174): ceph-mgr-rook-19.0.0-3133.ga8c84f59.e 987 kB/s | 49 kB 00:00 2024-04-17T05:48:45.771 INFO:teuthology.orchestra.run.smithi098.stdout:(34/174): ceph-prometheus-alerts-19.0.0-3133.ga 198 kB/s | 16 kB 00:00 2024-04-17T05:48:45.815 INFO:teuthology.orchestra.run.smithi103.stdout:(79/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 42 MB/s | 16 MB 00:00 2024-04-17T05:48:45.840 INFO:teuthology.orchestra.run.smithi103.stdout:(80/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 199 kB/s | 41 kB 00:00 2024-04-17T05:48:45.866 INFO:teuthology.orchestra.run.smithi103.stdout:(81/174): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 220 kB 00:00 2024-04-17T05:48:45.924 INFO:teuthology.orchestra.run.smithi103.stdout:(82/174): qatlib-service-24.02.0-1.el9.x86_64.r 417 kB/s | 35 kB 00:00 2024-04-17T05:48:45.929 INFO:teuthology.orchestra.run.smithi098.stdout:(35/174): ceph-volume-19.0.0-3133.ga8c84f59.el9 1.7 MB/s | 273 kB 00:00 2024-04-17T05:48:45.949 INFO:teuthology.orchestra.run.smithi103.stdout:(83/174): qatzip-libs-1.2.0-1.el9.x86_64.rpm 556 kB/s | 46 kB 00:00 2024-04-17T05:48:45.974 INFO:teuthology.orchestra.run.smithi103.stdout:(84/174): socat-1.7.4.1-5.el9.x86_64.rpm 5.8 MB/s | 300 kB 00:00 2024-04-17T05:48:46.000 INFO:teuthology.orchestra.run.smithi103.stdout:(85/174): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 56 kB 00:00 2024-04-17T05:48:46.025 INFO:teuthology.orchestra.run.smithi103.stdout:(86/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 63 kB 00:00 2024-04-17T05:48:46.050 INFO:teuthology.orchestra.run.smithi103.stdout:(87/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 877 kB/s | 44 kB 00:00 2024-04-17T05:48:46.072 INFO:teuthology.orchestra.run.smithi103.stdout:(88/174): fuse-2.9.9-15.el9.x86_64.rpm 1.7 MB/s | 79 kB 00:00 2024-04-17T05:48:46.105 INFO:teuthology.orchestra.run.smithi103.stdout:(89/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.3 MB/s | 241 kB 00:00 2024-04-17T05:48:46.113 INFO:teuthology.orchestra.run.smithi098.stdout:(36/174): cephadm-19.0.0-3133.ga8c84f59.el9.noa 4.1 MB/s | 763 kB 00:00 2024-04-17T05:48:46.139 INFO:teuthology.orchestra.run.smithi103.stdout:(90/174): python3-cryptography-36.0.1-4.el9.x86 17 MB/s | 1.1 MB 00:00 2024-04-17T05:48:46.156 INFO:teuthology.orchestra.run.smithi103.stdout:(91/174): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 103 kB 00:00 2024-04-17T05:48:46.172 INFO:teuthology.orchestra.run.smithi098.stdout:(37/174): cephfs-top-19.0.0-3133.ga8c84f59.el9. 312 kB/s | 18 kB 00:00 2024-04-17T05:48:46.181 INFO:teuthology.orchestra.run.smithi103.stdout:(92/174): python3-pycparser-2.20-6.el9.noarch.r 2.9 MB/s | 124 kB 00:00 2024-04-17T05:48:46.207 INFO:teuthology.orchestra.run.smithi103.stdout:(93/174): python3-urllib3-1.26.5-5.el9.noarch.r 3.7 MB/s | 187 kB 00:00 2024-04-17T05:48:46.298 INFO:teuthology.orchestra.run.smithi047.stdout:(31/174): ceph-test-19.0.0-3133.ga8c84f59.el9.x 7.7 MB/s | 49 MB 00:06 2024-04-17T05:48:46.314 INFO:teuthology.orchestra.run.smithi098.stdout:(38/174): automake-1.16.2-8.el9.noarch.rpm 4.6 MB/s | 662 kB 00:00 2024-04-17T05:48:46.349 INFO:teuthology.orchestra.run.smithi103.stdout:(94/174): jq-1.6-16.el9.x86_64.rpm 1.1 MB/s | 187 kB 00:00 2024-04-17T05:48:46.356 INFO:teuthology.orchestra.run.smithi047.stdout:(32/174): ceph-mgr-dashboard-19.0.0-3133.ga8c84 2.8 MB/s | 3.5 MB 00:01 2024-04-17T05:48:46.364 INFO:teuthology.orchestra.run.smithi098.stdout:(39/174): bison-3.7.4-5.el9.x86_64.rpm 18 MB/s | 920 kB 00:00 2024-04-17T05:48:46.381 INFO:teuthology.orchestra.run.smithi047.stdout:(33/174): ceph-mgr-modules-core-19.0.0-3133.ga8 2.9 MB/s | 249 kB 00:00 2024-04-17T05:48:46.398 INFO:teuthology.orchestra.run.smithi098.stdout:(40/174): boost-program-options-1.75.0-8.el9.x8 3.1 MB/s | 106 kB 00:00 2024-04-17T05:48:46.399 INFO:teuthology.orchestra.run.smithi103.stdout:(95/174): libgfortran-11.4.1-3.el9.x86_64.rpm 4.1 MB/s | 803 kB 00:00 2024-04-17T05:48:46.415 INFO:teuthology.orchestra.run.smithi047.stdout:(34/174): ceph-mgr-rook-19.0.0-3133.ga8c84f59.e 842 kB/s | 49 kB 00:00 2024-04-17T05:48:46.424 INFO:teuthology.orchestra.run.smithi103.stdout:(96/174): libquadmath-11.4.1-3.el9.x86_64.rpm 2.5 MB/s | 195 kB 00:00 2024-04-17T05:48:46.444 INFO:teuthology.orchestra.run.smithi098.stdout:(41/174): flex-2.6.4-9.el9.x86_64.rpm 6.5 MB/s | 307 kB 00:00 2024-04-17T05:48:46.448 INFO:teuthology.orchestra.run.smithi047.stdout:(35/174): ceph-prometheus-alerts-19.0.0-3133.ga 246 kB/s | 16 kB 00:00 2024-04-17T05:48:46.449 INFO:teuthology.orchestra.run.smithi103.stdout:(97/174): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-04-17T05:48:46.478 INFO:teuthology.orchestra.run.smithi098.stdout:(42/174): flexiblas-3.0.4-8.el9.x86_64.rpm 946 kB/s | 32 kB 00:00 2024-04-17T05:48:46.540 INFO:teuthology.orchestra.run.smithi047.stdout:(36/174): ceph-volume-19.0.0-3133.ga8c84f59.el9 2.1 MB/s | 273 kB 00:00 2024-04-17T05:48:46.570 INFO:teuthology.orchestra.run.smithi098.stdout:(43/174): ceph-mgr-diskprediction-local-19.0.0- 4.1 MB/s | 7.4 MB 00:01 2024-04-17T05:48:46.573 INFO:teuthology.orchestra.run.smithi047.stdout:(37/174): cephadm-19.0.0-3133.ga8c84f59.el9.noa 6.0 MB/s | 763 kB 00:00 2024-04-17T05:48:46.616 INFO:teuthology.orchestra.run.smithi047.stdout:(38/174): cephfs-top-19.0.0-3133.ga8c84f59.el9. 243 kB/s | 18 kB 00:00 2024-04-17T05:48:46.645 INFO:teuthology.orchestra.run.smithi098.stdout:(44/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 18 MB/s | 3.0 MB 00:00 2024-04-17T05:48:46.671 INFO:teuthology.orchestra.run.smithi098.stdout:(45/174): flexiblas-openblas-openmp-3.0.4-8.el9 174 kB/s | 17 kB 00:00 2024-04-17T05:48:46.696 INFO:teuthology.orchestra.run.smithi098.stdout:(46/174): libaio-devel-0.3.111-13.el9.x86_64.rp 199 kB/s | 10 kB 00:00 2024-04-17T05:48:46.722 INFO:teuthology.orchestra.run.smithi098.stdout:(47/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 159 kB 00:00 2024-04-17T05:48:46.747 INFO:teuthology.orchestra.run.smithi098.stdout:(48/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 885 kB/s | 44 kB 00:00 2024-04-17T05:48:46.772 INFO:teuthology.orchestra.run.smithi098.stdout:(49/174): librdkafka-1.6.1-102.el9.x86_64.rpm 13 MB/s | 661 kB 00:00 2024-04-17T05:48:46.774 INFO:teuthology.orchestra.run.smithi047.stdout:(39/174): automake-1.16.2-8.el9.noarch.rpm 3.2 MB/s | 662 kB 00:00 2024-04-17T05:48:46.798 INFO:teuthology.orchestra.run.smithi098.stdout:(50/174): libtool-2.4.6-45.el9.x86_64.rpm 11 MB/s | 578 kB 00:00 2024-04-17T05:48:46.808 INFO:teuthology.orchestra.run.smithi047.stdout:(40/174): bison-3.7.4-5.el9.x86_64.rpm 5.5 MB/s | 920 kB 00:00 2024-04-17T05:48:46.814 INFO:teuthology.orchestra.run.smithi098.stdout:(51/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.1 MB/s | 46 kB 00:00 2024-04-17T05:48:46.833 INFO:teuthology.orchestra.run.smithi047.stdout:(41/174): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 106 kB 00:00 2024-04-17T05:48:46.840 INFO:teuthology.orchestra.run.smithi098.stdout:(52/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 6.6 MB/s | 282 kB 00:00 2024-04-17T05:48:46.858 INFO:teuthology.orchestra.run.smithi047.stdout:(42/174): flex-2.6.4-9.el9.x86_64.rpm 6.0 MB/s | 307 kB 00:00 2024-04-17T05:48:46.865 INFO:teuthology.orchestra.run.smithi098.stdout:(53/174): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2024-04-17T05:48:46.883 INFO:teuthology.orchestra.run.smithi047.stdout:(43/174): flexiblas-3.0.4-8.el9.x86_64.rpm 638 kB/s | 32 kB 00:00 2024-04-17T05:48:46.890 INFO:teuthology.orchestra.run.smithi098.stdout:(54/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 4.2 MB/s | 217 kB 00:00 2024-04-17T05:48:46.907 INFO:teuthology.orchestra.run.smithi098.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 822 kB/s | 34 kB 00:00 2024-04-17T05:48:46.984 INFO:teuthology.orchestra.run.smithi047.stdout:(44/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 24 MB/s | 3.0 MB 00:00 2024-04-17T05:48:47.000 INFO:teuthology.orchestra.run.smithi103.stdout:(98/174): elfutils-libelf-devel-0.190-2.el9.x86 37 kB/s | 21 kB 00:00 2024-04-17T05:48:47.009 INFO:teuthology.orchestra.run.smithi047.stdout:(45/174): flexiblas-openblas-openmp-3.0.4-8.el9 138 kB/s | 17 kB 00:00 2024-04-17T05:48:47.034 INFO:teuthology.orchestra.run.smithi047.stdout:(46/174): libaio-devel-0.3.111-13.el9.x86_64.rp 200 kB/s | 10 kB 00:00 2024-04-17T05:48:47.041 INFO:teuthology.orchestra.run.smithi098.stdout:(56/174): openblas-openmp-0.3.21-2.el9.x86_64.r 32 MB/s | 4.7 MB 00:00 2024-04-17T05:48:47.058 INFO:teuthology.orchestra.run.smithi103.stdout:(99/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 42 kB/s | 25 kB 00:00 2024-04-17T05:48:47.059 INFO:teuthology.orchestra.run.smithi047.stdout:(47/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 159 kB 00:00 2024-04-17T05:48:47.085 INFO:teuthology.orchestra.run.smithi047.stdout:(48/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 888 kB/s | 44 kB 00:00 2024-04-17T05:48:47.091 INFO:teuthology.orchestra.run.smithi098.stdout:(57/174): postgresql-13.14-1.el9.x86_64.rpm 8.3 MB/s | 1.5 MB 00:00 2024-04-17T05:48:47.118 INFO:teuthology.orchestra.run.smithi047.stdout:(49/174): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 661 kB 00:00 2024-04-17T05:48:47.133 INFO:teuthology.orchestra.run.smithi098.stdout:(58/174): postgresql-contrib-13.14-1.el9.x86_64 8.9 MB/s | 817 kB 00:00 2024-04-17T05:48:47.144 INFO:teuthology.orchestra.run.smithi047.stdout:(50/174): libtool-2.4.6-45.el9.x86_64.rpm 9.9 MB/s | 578 kB 00:00 2024-04-17T05:48:47.159 INFO:teuthology.orchestra.run.smithi098.stdout:(59/174): postgresql-private-libs-13.14-1.el9.x 2.0 MB/s | 132 kB 00:00 2024-04-17T05:48:47.169 INFO:teuthology.orchestra.run.smithi047.stdout:(51/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 937 kB/s | 46 kB 00:00 2024-04-17T05:48:47.270 INFO:teuthology.orchestra.run.smithi047.stdout:(52/174): ceph-mgr-diskprediction-local-19.0.0- 4.6 MB/s | 7.4 MB 00:01 2024-04-17T05:48:47.276 INFO:teuthology.orchestra.run.smithi098.stdout:(60/174): postgresql-server-13.14-1.el9.x86_64. 40 MB/s | 5.7 MB 00:00 2024-04-17T05:48:47.295 INFO:teuthology.orchestra.run.smithi047.stdout:(53/174): lua-5.4.4-4.el9.x86_64.rpm 1.5 MB/s | 187 kB 00:00 2024-04-17T05:48:47.310 INFO:teuthology.orchestra.run.smithi098.stdout:(61/174): protobuf-3.14.0-13.el9.x86_64.rpm 6.8 MB/s | 1.0 MB 00:00 2024-04-17T05:48:47.320 INFO:teuthology.orchestra.run.smithi047.stdout:(54/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.6 MB/s | 282 kB 00:00 2024-04-17T05:48:47.326 INFO:teuthology.orchestra.run.smithi098.stdout:(62/174): python3-attrs-20.3.0-7.el9.noarch.rpm 1.6 MB/s | 78 kB 00:00 2024-04-17T05:48:47.345 INFO:teuthology.orchestra.run.smithi047.stdout:(55/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.8 MB/s | 217 kB 00:00 2024-04-17T05:48:47.370 INFO:teuthology.orchestra.run.smithi047.stdout:(56/174): openblas-0.3.21-2.el9.x86_64.rpm 459 kB/s | 34 kB 00:00 2024-04-17T05:48:47.437 INFO:teuthology.orchestra.run.smithi047.stdout:(57/174): postgresql-13.14-1.el9.x86_64.rpm 17 MB/s | 1.5 MB 00:00 2024-04-17T05:48:47.452 INFO:teuthology.orchestra.run.smithi098.stdout:(63/174): python3-babel-2.9.1-2.el9.noarch.rpm 41 MB/s | 5.8 MB 00:00 2024-04-17T05:48:47.469 INFO:teuthology.orchestra.run.smithi098.stdout:(64/174): python3-jsonpatch-1.21-16.el9.noarch. 173 kB/s | 24 kB 00:00 2024-04-17T05:48:47.486 INFO:teuthology.orchestra.run.smithi098.stdout:(65/174): python3-jsonpointer-2.0-4.el9.noarch. 539 kB/s | 18 kB 00:00 2024-04-17T05:48:47.488 INFO:teuthology.orchestra.run.smithi047.stdout:(58/174): postgresql-contrib-13.14-1.el9.x86_64 6.8 MB/s | 817 kB 00:00 2024-04-17T05:48:47.519 INFO:teuthology.orchestra.run.smithi098.stdout:(66/174): python3-mako-1.1.4-6.el9.noarch.rpm 2.8 MB/s | 144 kB 00:00 2024-04-17T05:48:47.544 INFO:teuthology.orchestra.run.smithi098.stdout:(67/174): python3-markupsafe-1.1.1-12.el9.x86_6 541 kB/s | 32 kB 00:00 2024-04-17T05:48:47.554 INFO:teuthology.orchestra.run.smithi047.stdout:(59/174): openblas-openmp-0.3.21-2.el9.x86_64.r 23 MB/s | 4.7 MB 00:00 2024-04-17T05:48:47.580 INFO:teuthology.orchestra.run.smithi047.stdout:(60/174): postgresql-private-libs-13.14-1.el9.x 952 kB/s | 132 kB 00:00 2024-04-17T05:48:47.662 INFO:teuthology.orchestra.run.smithi098.stdout:(68/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 34 MB/s | 4.8 MB 00:00 2024-04-17T05:48:47.680 INFO:teuthology.orchestra.run.smithi047.stdout:(61/174): postgresql-server-13.14-1.el9.x86_64. 30 MB/s | 5.7 MB 00:00 2024-04-17T05:48:47.687 INFO:teuthology.orchestra.run.smithi098.stdout:(69/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.5 MB/s | 217 kB 00:00 2024-04-17T05:48:47.705 INFO:teuthology.orchestra.run.smithi047.stdout:(62/174): python3-attrs-20.3.0-7.el9.noarch.rpm 636 kB/s | 78 kB 00:00 2024-04-17T05:48:47.720 INFO:teuthology.orchestra.run.smithi098.stdout:(70/174): python3-packaging-20.9-5.el9.noarch.r 2.0 MB/s | 69 kB 00:00 2024-04-17T05:48:47.737 INFO:teuthology.orchestra.run.smithi098.stdout:(71/174): python3-oauthlib-3.1.1-5.el9.noarch.r 3.4 MB/s | 174 kB 00:00 2024-04-17T05:48:47.739 INFO:teuthology.orchestra.run.smithi047.stdout:(63/174): protobuf-3.14.0-13.el9.x86_64.rpm 6.5 MB/s | 1.0 MB 00:00 2024-04-17T05:48:47.763 INFO:teuthology.orchestra.run.smithi098.stdout:(72/174): python3-prettytable-0.7.2-27.el9.noar 986 kB/s | 41 kB 00:00 2024-04-17T05:48:47.788 INFO:teuthology.orchestra.run.smithi098.stdout:(73/174): python3-protobuf-3.14.0-13.el9.noarch 4.7 MB/s | 238 kB 00:00 2024-04-17T05:48:47.813 INFO:teuthology.orchestra.run.smithi098.stdout:(74/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 132 kB 00:00 2024-04-17T05:48:47.831 INFO:teuthology.orchestra.run.smithi047.stdout:(64/174): python3-babel-2.9.1-2.el9.noarch.rpm 38 MB/s | 5.8 MB 00:00 2024-04-17T05:48:47.838 INFO:teuthology.orchestra.run.smithi098.stdout:(75/174): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 211 kB 00:00 2024-04-17T05:48:47.848 INFO:teuthology.orchestra.run.smithi047.stdout:(65/174): python3-jsonpatch-1.21-16.el9.noarch. 178 kB/s | 24 kB 00:00 2024-04-17T05:48:47.864 INFO:teuthology.orchestra.run.smithi098.stdout:(76/174): python3-pytz-2021.1-5.el9.noarch.rpm 950 kB/s | 47 kB 00:00 2024-04-17T05:48:47.864 INFO:teuthology.orchestra.run.smithi047.stdout:(66/174): python3-jsonpointer-2.0-4.el9.noarch. 148 kB/s | 18 kB 00:00 2024-04-17T05:48:47.889 INFO:teuthology.orchestra.run.smithi098.stdout:(77/174): python3-requests-oauthlib-1.3.0-12.el 850 kB/s | 43 kB 00:00 2024-04-17T05:48:47.890 INFO:teuthology.orchestra.run.smithi047.stdout:(67/174): python3-mako-1.1.4-6.el9.noarch.rpm 2.4 MB/s | 144 kB 00:00 2024-04-17T05:48:47.906 INFO:teuthology.orchestra.run.smithi047.stdout:(68/174): python3-markupsafe-1.1.1-12.el9.x86_6 762 kB/s | 32 kB 00:00 2024-04-17T05:48:48.017 INFO:teuthology.orchestra.run.smithi103.stdout:(100/174): python3-devel-3.9.17-2.el9.x86_64.rp 256 kB/s | 245 kB 00:00 2024-04-17T05:48:48.032 INFO:teuthology.orchestra.run.smithi047.stdout:(69/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 29 MB/s | 4.8 MB 00:00 2024-04-17T05:48:48.057 INFO:teuthology.orchestra.run.smithi047.stdout:(70/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.3 MB/s | 217 kB 00:00 2024-04-17T05:48:48.082 INFO:teuthology.orchestra.run.smithi047.stdout:(71/174): python3-oauthlib-3.1.1-5.el9.noarch.r 1.0 MB/s | 174 kB 00:00 2024-04-17T05:48:48.108 INFO:teuthology.orchestra.run.smithi047.stdout:(72/174): python3-packaging-20.9-5.el9.noarch.r 922 kB/s | 69 kB 00:00 2024-04-17T05:48:48.133 INFO:teuthology.orchestra.run.smithi047.stdout:(73/174): python3-prettytable-0.7.2-27.el9.noar 826 kB/s | 41 kB 00:00 2024-04-17T05:48:48.158 INFO:teuthology.orchestra.run.smithi047.stdout:(74/174): python3-protobuf-3.14.0-13.el9.noarch 3.1 MB/s | 238 kB 00:00 2024-04-17T05:48:48.184 INFO:teuthology.orchestra.run.smithi047.stdout:(75/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 132 kB 00:00 2024-04-17T05:48:48.209 INFO:teuthology.orchestra.run.smithi047.stdout:(76/174): python3-pytz-2021.1-5.el9.noarch.rpm 951 kB/s | 47 kB 00:00 2024-04-17T05:48:48.234 INFO:teuthology.orchestra.run.smithi047.stdout:(77/174): python3-pyasn1-modules-0.4.8-6.el9.no 2.7 MB/s | 211 kB 00:00 2024-04-17T05:48:48.259 INFO:teuthology.orchestra.run.smithi047.stdout:(78/174): python3-requests-oauthlib-1.3.0-12.el 571 kB/s | 43 kB 00:00 2024-04-17T05:48:48.348 INFO:teuthology.orchestra.run.smithi098.stdout:(78/174): ceph-test-19.0.0-3133.ga8c84f59.el9.x 4.6 MB/s | 49 MB 00:10 2024-04-17T05:48:48.465 INFO:teuthology.orchestra.run.smithi098.stdout:(79/174): python3-toml-0.10.2-6.el9.noarch.rpm 67 kB/s | 39 kB 00:00 2024-04-17T05:48:48.468 INFO:teuthology.orchestra.run.smithi047.stdout:(79/174): python3-toml-0.10.2-6.el9.noarch.rpm 188 kB/s | 39 kB 00:00 2024-04-17T05:48:48.493 INFO:teuthology.orchestra.run.smithi047.stdout:(80/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 178 kB/s | 41 kB 00:00 2024-04-17T05:48:48.635 INFO:teuthology.orchestra.run.smithi047.stdout:(81/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 37 MB/s | 16 MB 00:00 2024-04-17T05:48:48.660 INFO:teuthology.orchestra.run.smithi047.stdout:(82/174): qatlib-24.02.0-1.el9.x86_64.rpm 1.1 MB/s | 220 kB 00:00 2024-04-17T05:48:48.685 INFO:teuthology.orchestra.run.smithi047.stdout:(83/174): qatlib-service-24.02.0-1.el9.x86_64.r 182 kB/s | 35 kB 00:00 2024-04-17T05:48:48.710 INFO:teuthology.orchestra.run.smithi047.stdout:(84/174): qatzip-libs-1.2.0-1.el9.x86_64.rpm 621 kB/s | 46 kB 00:00 2024-04-17T05:48:48.736 INFO:teuthology.orchestra.run.smithi047.stdout:(85/174): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 56 kB 00:00 2024-04-17T05:48:48.761 INFO:teuthology.orchestra.run.smithi047.stdout:(86/174): socat-1.7.4.1-5.el9.x86_64.rpm 3.9 MB/s | 300 kB 00:00 2024-04-17T05:48:48.782 INFO:teuthology.orchestra.run.smithi098.stdout:(80/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 17 MB/s | 16 MB 00:00 2024-04-17T05:48:48.786 INFO:teuthology.orchestra.run.smithi047.stdout:(87/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 829 kB/s | 63 kB 00:00 2024-04-17T05:48:48.807 INFO:teuthology.orchestra.run.smithi098.stdout:(81/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 90 kB/s | 41 kB 00:00 2024-04-17T05:48:48.811 INFO:teuthology.orchestra.run.smithi047.stdout:(88/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 590 kB/s | 44 kB 00:00 2024-04-17T05:48:48.832 INFO:teuthology.orchestra.run.smithi098.stdout:(82/174): qatlib-24.02.0-1.el9.x86_64.rpm 601 kB/s | 220 kB 00:00 2024-04-17T05:48:48.837 INFO:teuthology.orchestra.run.smithi047.stdout:(89/174): fuse-2.9.9-15.el9.x86_64.rpm 1.5 MB/s | 79 kB 00:00 2024-04-17T05:48:48.853 INFO:teuthology.orchestra.run.smithi047.stdout:(90/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.5 MB/s | 241 kB 00:00 2024-04-17T05:48:48.858 INFO:teuthology.orchestra.run.smithi098.stdout:(83/174): qatlib-service-24.02.0-1.el9.x86_64.r 466 kB/s | 35 kB 00:00 2024-04-17T05:48:48.883 INFO:teuthology.orchestra.run.smithi098.stdout:(84/174): qatzip-libs-1.2.0-1.el9.x86_64.rpm 929 kB/s | 46 kB 00:00 2024-04-17T05:48:48.887 INFO:teuthology.orchestra.run.smithi047.stdout:(91/174): python3-cryptography-36.0.1-4.el9.x86 15 MB/s | 1.1 MB 00:00 2024-04-17T05:48:48.908 INFO:teuthology.orchestra.run.smithi098.stdout:(85/174): socat-1.7.4.1-5.el9.x86_64.rpm 3.9 MB/s | 300 kB 00:00 2024-04-17T05:48:48.912 INFO:teuthology.orchestra.run.smithi047.stdout:(92/174): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 103 kB 00:00 2024-04-17T05:48:48.933 INFO:teuthology.orchestra.run.smithi098.stdout:(86/174): uuid-1.6.2-55.el9.x86_64.rpm 748 kB/s | 56 kB 00:00 2024-04-17T05:48:48.938 INFO:teuthology.orchestra.run.smithi047.stdout:(93/174): python3-pycparser-2.20-6.el9.noarch.r 1.5 MB/s | 124 kB 00:00 2024-04-17T05:48:48.959 INFO:teuthology.orchestra.run.smithi098.stdout:(87/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 881 kB/s | 44 kB 00:00 2024-04-17T05:48:48.963 INFO:teuthology.orchestra.run.smithi047.stdout:(94/174): python3-urllib3-1.26.5-5.el9.noarch.r 2.4 MB/s | 187 kB 00:00 2024-04-17T05:48:48.984 INFO:teuthology.orchestra.run.smithi103.stdout:(101/174): python3-jinja2-2.11.3-5.el9.noarch.r 257 kB/s | 248 kB 00:00 2024-04-17T05:48:48.985 INFO:teuthology.orchestra.run.smithi098.stdout:(88/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 829 kB/s | 63 kB 00:00 2024-04-17T05:48:49.009 INFO:teuthology.orchestra.run.smithi098.stdout:(89/174): fuse-2.9.9-15.el9.x86_64.rpm 1.0 MB/s | 79 kB 00:00 2024-04-17T05:48:49.035 INFO:teuthology.orchestra.run.smithi098.stdout:(90/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.1 MB/s | 241 kB 00:00 2024-04-17T05:48:49.060 INFO:teuthology.orchestra.run.smithi098.stdout:(91/174): python3-ply-3.11-14.el9.noarch.rpm 2.0 MB/s | 103 kB 00:00 2024-04-17T05:48:49.102 INFO:teuthology.orchestra.run.smithi098.stdout:(92/174): python3-cryptography-36.0.1-4.el9.x86 13 MB/s | 1.1 MB 00:00 2024-04-17T05:48:49.127 INFO:teuthology.orchestra.run.smithi098.stdout:(93/174): python3-pycparser-2.20-6.el9.noarch.r 1.3 MB/s | 124 kB 00:00 2024-04-17T05:48:49.152 INFO:teuthology.orchestra.run.smithi098.stdout:(94/174): python3-urllib3-1.26.5-5.el9.noarch.r 2.0 MB/s | 187 kB 00:00 2024-04-17T05:48:49.291 INFO:teuthology.orchestra.run.smithi098.stdout:(95/174): jq-1.6-16.el9.x86_64.rpm 1.1 MB/s | 187 kB 00:00 2024-04-17T05:48:49.316 INFO:teuthology.orchestra.run.smithi098.stdout:(96/174): libquadmath-11.4.1-3.el9.x86_64.rpm 1.2 MB/s | 195 kB 00:00 2024-04-17T05:48:49.342 INFO:teuthology.orchestra.run.smithi098.stdout:(97/174): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-04-17T05:48:49.392 INFO:teuthology.orchestra.run.smithi098.stdout:(98/174): libgfortran-11.4.1-3.el9.x86_64.rpm 3.0 MB/s | 803 kB 00:00 2024-04-17T05:48:49.442 INFO:teuthology.orchestra.run.smithi098.stdout:(99/174): elfutils-libelf-devel-0.190-2.el9.x86 171 kB/s | 21 kB 00:00 2024-04-17T05:48:49.468 INFO:teuthology.orchestra.run.smithi098.stdout:(100/174): libuuid-devel-2.37.4-18.el9.x86_64.r 202 kB/s | 25 kB 00:00 2024-04-17T05:48:49.471 INFO:teuthology.orchestra.run.smithi047.stdout:(95/174): jq-1.6-16.el9.x86_64.rpm 351 kB/s | 187 kB 00:00 2024-04-17T05:48:49.497 INFO:teuthology.orchestra.run.smithi047.stdout:(96/174): libquadmath-11.4.1-3.el9.x86_64.rpm 366 kB/s | 195 kB 00:00 2024-04-17T05:48:49.568 INFO:teuthology.orchestra.run.smithi098.stdout:(101/174): python3-devel-3.9.17-2.el9.x86_64.rp 1.9 MB/s | 245 kB 00:00 2024-04-17T05:48:49.589 INFO:teuthology.orchestra.run.smithi047.stdout:(97/174): python3-requests-2.25.1-8.el9.noarch. 1.0 MB/s | 125 kB 00:00 2024-04-17T05:48:49.593 INFO:teuthology.orchestra.run.smithi098.stdout:(102/174): python3-jinja2-2.11.3-5.el9.noarch.r 1.9 MB/s | 248 kB 00:00 2024-04-17T05:48:49.618 INFO:teuthology.orchestra.run.smithi098.stdout:(103/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.8 MB/s | 94 kB 00:00 2024-04-17T05:48:49.644 INFO:teuthology.orchestra.run.smithi098.stdout:(104/174): lua-devel-5.4.4-4.el9.x86_64.rpm 445 kB/s | 22 kB 00:00 2024-04-17T05:48:49.701 INFO:teuthology.orchestra.run.smithi103.stdout:(102/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 131 kB/s | 94 kB 00:00 2024-04-17T05:48:49.714 INFO:teuthology.orchestra.run.smithi047.stdout:(98/174): libgfortran-11.4.1-3.el9.x86_64.rpm 1.0 MB/s | 803 kB 00:00 2024-04-17T05:48:49.727 INFO:teuthology.orchestra.run.smithi098.stdout:(105/174): protobuf-compiler-3.14.0-13.el9.x86_ 7.8 MB/s | 863 kB 00:00 2024-04-17T05:48:49.785 INFO:teuthology.orchestra.run.smithi103.stdout:(103/174): lua-devel-5.4.4-4.el9.x86_64.rpm 269 kB/s | 22 kB 00:00 2024-04-17T05:48:49.786 INFO:teuthology.orchestra.run.smithi098.stdout:(106/174): openssl-devel-3.0.7-27.el9.x86_64.rp 10 MB/s | 4.1 MB 00:00 2024-04-17T05:48:49.910 INFO:teuthology.orchestra.run.smithi103.stdout:(104/174): protobuf-compiler-3.14.0-13.el9.x86_ 6.7 MB/s | 863 kB 00:00 2024-04-17T05:48:49.961 INFO:teuthology.orchestra.run.smithi098.stdout:(107/174): grpc-data-1.46.7-10.el9.noarch.rpm 131 kB/s | 19 kB 00:00 2024-04-17T05:48:50.028 INFO:teuthology.orchestra.run.smithi098.stdout:(108/174): abseil-cpp-20211102.0-3.el9.x86_64.r 1.4 MB/s | 548 kB 00:00 2024-04-17T05:48:50.053 INFO:teuthology.orchestra.run.smithi098.stdout:(109/174): gperftools-libs-2.9.1-2.el9.x86_64.r 951 kB/s | 309 kB 00:00 2024-04-17T05:48:50.071 INFO:teuthology.orchestra.run.smithi098.stdout:(110/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 618 kB/s | 26 kB 00:00 2024-04-17T05:48:50.096 INFO:teuthology.orchestra.run.smithi098.stdout:(111/174): liboath-2.6.7-2.el9.x86_64.rpm 1.1 MB/s | 49 kB 00:00 2024-04-17T05:48:50.114 INFO:teuthology.orchestra.run.smithi047.stdout:(99/174): elfutils-libelf-devel-0.190-2.el9.x86 35 kB/s | 21 kB 00:00 2024-04-17T05:48:50.144 INFO:teuthology.orchestra.run.smithi103.stdout:(105/174): abseil-cpp-20211102.0-3.el9.x86_64.r 2.3 MB/s | 548 kB 00:00 2024-04-17T05:48:50.146 INFO:teuthology.orchestra.run.smithi098.stdout:(112/174): libunwind-1.6.2-1.el9.x86_64.rpm 899 kB/s | 67 kB 00:00 2024-04-17T05:48:50.172 INFO:teuthology.orchestra.run.smithi098.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 2.0 MB/s | 151 kB 00:00 2024-04-17T05:48:50.193 INFO:teuthology.orchestra.run.smithi103.stdout:(106/174): gperftools-libs-2.9.1-2.el9.x86_64.r 6.1 MB/s | 309 kB 00:00 2024-04-17T05:48:50.223 INFO:teuthology.orchestra.run.smithi047.stdout:(100/174): libuuid-devel-2.37.4-18.el9.x86_64.r 40 kB/s | 25 kB 00:00 2024-04-17T05:48:50.244 INFO:teuthology.orchestra.run.smithi103.stdout:(107/174): grpc-data-1.46.7-10.el9.noarch.rpm 388 kB/s | 19 kB 00:00 2024-04-17T05:48:50.388 INFO:teuthology.orchestra.run.smithi098.stdout:(114/174): python3-asyncssh-2.13.2-5.el9.noarch 2.5 MB/s | 548 kB 00:00 2024-04-17T05:48:50.411 INFO:teuthology.orchestra.run.smithi103.stdout:(108/174): libarrow-9.0.0-10.el9.x86_64.rpm 26 MB/s | 4.4 MB 00:00 2024-04-17T05:48:50.422 INFO:teuthology.orchestra.run.smithi098.stdout:(115/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.3 MB/s | 43 kB 00:00 2024-04-17T05:48:50.461 INFO:teuthology.orchestra.run.smithi103.stdout:(109/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 521 kB/s | 26 kB 00:00 2024-04-17T05:48:50.464 INFO:teuthology.orchestra.run.smithi098.stdout:(116/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 2.6 MB/s | 838 kB 00:00 2024-04-17T05:48:50.489 INFO:teuthology.orchestra.run.smithi098.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 479 kB/s | 32 kB 00:00 2024-04-17T05:48:50.511 INFO:teuthology.orchestra.run.smithi103.stdout:(110/174): liboath-2.6.7-2.el9.x86_64.rpm 970 kB/s | 49 kB 00:00 2024-04-17T05:48:50.514 INFO:teuthology.orchestra.run.smithi098.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 298 kB/s | 15 kB 00:00 2024-04-17T05:48:50.562 INFO:teuthology.orchestra.run.smithi103.stdout:(111/174): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-04-17T05:48:50.573 INFO:teuthology.orchestra.run.smithi098.stdout:(119/174): python3-cheroot-8.6.0-4.el9.noarch.r 2.0 MB/s | 172 kB 00:00 2024-04-17T05:48:50.612 INFO:teuthology.orchestra.run.smithi103.stdout:(112/174): luarocks-3.9.2-1.el9.noarch.rpm 2.9 MB/s | 151 kB 00:00 2024-04-17T05:48:50.679 INFO:teuthology.orchestra.run.smithi103.stdout:(113/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 12 MB/s | 838 kB 00:00 2024-04-17T05:48:50.690 INFO:teuthology.orchestra.run.smithi098.stdout:(120/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.5 MB/s | 303 kB 00:00 2024-04-17T05:48:50.732 INFO:teuthology.orchestra.run.smithi098.stdout:(121/174): python3-colorama-0.4.4-12.el9.noarch 848 kB/s | 35 kB 00:00 2024-04-17T05:48:50.738 INFO:teuthology.orchestra.run.smithi103.stdout:(114/174): python3-asyncssh-2.13.2-5.el9.noarch 9.2 MB/s | 548 kB 00:00 2024-04-17T05:48:50.788 INFO:teuthology.orchestra.run.smithi103.stdout:(115/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 867 kB/s | 43 kB 00:00 2024-04-17T05:48:50.824 INFO:teuthology.orchestra.run.smithi098.stdout:(122/174): python3-google-auth-2.28.2-1.el9.noa 2.2 MB/s | 203 kB 00:00 2024-04-17T05:48:50.838 INFO:teuthology.orchestra.run.smithi103.stdout:(116/174): python3-cachetools-4.2.4-1.el9.noarc 646 kB/s | 32 kB 00:00 2024-04-17T05:48:50.889 INFO:teuthology.orchestra.run.smithi103.stdout:(117/174): python3-certifi-2021.10.8-2.el9.noar 300 kB/s | 15 kB 00:00 2024-04-17T05:48:50.939 INFO:teuthology.orchestra.run.smithi103.stdout:(118/174): python3-cheroot-8.6.0-4.el9.noarch.r 3.4 MB/s | 172 kB 00:00 2024-04-17T05:48:50.989 INFO:teuthology.orchestra.run.smithi103.stdout:(119/174): python3-cherrypy-18.6.1-2.el9.noarch 7.0 MB/s | 358 kB 00:00 2024-04-17T05:48:50.997 INFO:teuthology.orchestra.run.smithi098.stdout:(123/174): python3-cherrypy-18.6.1-2.el9.noarch 743 kB/s | 358 kB 00:00 2024-04-17T05:48:51.015 INFO:teuthology.orchestra.run.smithi047.stdout:(101/174): python3-devel-3.9.17-2.el9.x86_64.rp 273 kB/s | 245 kB 00:00 2024-04-17T05:48:51.048 INFO:teuthology.orchestra.run.smithi103.stdout:(120/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.1 MB/s | 303 kB 00:00 2024-04-17T05:48:51.098 INFO:teuthology.orchestra.run.smithi103.stdout:(121/174): python3-colorama-0.4.4-12.el9.noarch 705 kB/s | 35 kB 00:00 2024-04-17T05:48:51.141 INFO:teuthology.orchestra.run.smithi098.stdout:(124/174): python3-grpcio-tools-1.46.7-10.el9.x 1.0 MB/s | 144 kB 00:00 2024-04-17T05:48:51.149 INFO:teuthology.orchestra.run.smithi103.stdout:(122/174): python3-google-auth-2.28.2-1.el9.noa 4.0 MB/s | 203 kB 00:00 2024-04-17T05:48:51.165 INFO:teuthology.orchestra.run.smithi047.stdout:(102/174): python3-jinja2-2.11.3-5.el9.noarch.r 263 kB/s | 248 kB 00:00 2024-04-17T05:48:51.183 INFO:teuthology.orchestra.run.smithi098.stdout:(125/174): python3-jaraco-8.2.1-3.el9.noarch.rp 258 kB/s | 11 kB 00:00 2024-04-17T05:48:51.225 INFO:teuthology.orchestra.run.smithi098.stdout:(126/174): python3-jaraco-classes-3.2.1-5.el9.n 424 kB/s | 18 kB 00:00 2024-04-17T05:48:51.232 INFO:teuthology.orchestra.run.smithi103.stdout:(123/174): python3-grpcio-1.46.7-10.el9.x86_64. 24 MB/s | 2.0 MB 00:00 2024-04-17T05:48:51.267 INFO:teuthology.orchestra.run.smithi098.stdout:(127/174): python3-jaraco-collections-3.0.0-8.e 556 kB/s | 23 kB 00:00 2024-04-17T05:48:51.283 INFO:teuthology.orchestra.run.smithi103.stdout:(124/174): python3-grpcio-tools-1.46.7-10.el9.x 2.8 MB/s | 144 kB 00:00 2024-04-17T05:48:51.298 INFO:teuthology.orchestra.run.smithi047.stdout:(103/174): lua-devel-5.4.4-4.el9.x86_64.rpm 168 kB/s | 22 kB 00:00 2024-04-17T05:48:51.309 INFO:teuthology.orchestra.run.smithi098.stdout:(128/174): python3-jaraco-functools-3.5.0-2.el9 468 kB/s | 19 kB 00:00 2024-04-17T05:48:51.325 INFO:teuthology.orchestra.run.smithi103.stdout:(125/174): python3-jaraco-8.2.1-3.el9.noarch.rp 257 kB/s | 11 kB 00:00 2024-04-17T05:48:51.351 INFO:teuthology.orchestra.run.smithi098.stdout:(129/174): python3-jaraco-text-3.2.0-6.el9.noar 473 kB/s | 20 kB 00:00 2024-04-17T05:48:51.375 INFO:teuthology.orchestra.run.smithi103.stdout:(126/174): python3-jaraco-classes-3.2.1-5.el9.n 352 kB/s | 18 kB 00:00 2024-04-17T05:48:51.426 INFO:teuthology.orchestra.run.smithi103.stdout:(127/174): python3-jaraco-collections-3.0.0-8.e 465 kB/s | 23 kB 00:00 2024-04-17T05:48:51.449 INFO:teuthology.orchestra.run.smithi047.stdout:(104/174): protobuf-compiler-3.14.0-13.el9.x86_ 5.6 MB/s | 863 kB 00:00 2024-04-17T05:48:51.476 INFO:teuthology.orchestra.run.smithi103.stdout:(128/174): python3-jaraco-functools-3.5.0-2.el9 389 kB/s | 19 kB 00:00 2024-04-17T05:48:51.502 INFO:teuthology.orchestra.run.smithi098.stdout:(130/174): libarrow-9.0.0-10.el9.x86_64.rpm 2.9 MB/s | 4.4 MB 00:01 2024-04-17T05:48:51.526 INFO:teuthology.orchestra.run.smithi103.stdout:(129/174): python3-jaraco-text-3.2.0-6.el9.noar 394 kB/s | 20 kB 00:00 2024-04-17T05:48:51.568 INFO:teuthology.orchestra.run.smithi098.stdout:(131/174): python3-grpcio-1.46.7-10.el9.x86_64. 2.7 MB/s | 2.0 MB 00:00 2024-04-17T05:48:51.593 INFO:teuthology.orchestra.run.smithi103.stdout:(130/174): python3-kubernetes-26.1.0-2.el9.noar 15 MB/s | 1.0 MB 00:00 2024-04-17T05:48:51.594 INFO:teuthology.orchestra.run.smithi098.stdout:(132/174): python3-logutils-0.3.5-21.el9.noarch 505 kB/s | 46 kB 00:00 2024-04-17T05:48:51.636 INFO:teuthology.orchestra.run.smithi098.stdout:(133/174): python3-more-itertools-8.12.0-2.el9. 1.2 MB/s | 79 kB 00:00 2024-04-17T05:48:51.643 INFO:teuthology.orchestra.run.smithi103.stdout:(131/174): python3-logutils-0.3.5-21.el9.noarch 926 kB/s | 46 kB 00:00 2024-04-17T05:48:51.661 INFO:teuthology.orchestra.run.smithi098.stdout:(134/174): python3-natsort-7.1.1-5.el9.noarch.r 862 kB/s | 58 kB 00:00 2024-04-17T05:48:51.694 INFO:teuthology.orchestra.run.smithi103.stdout:(132/174): python3-more-itertools-8.12.0-2.el9. 1.5 MB/s | 79 kB 00:00 2024-04-17T05:48:51.703 INFO:teuthology.orchestra.run.smithi098.stdout:(135/174): python3-portend-3.1.0-2.el9.noarch.r 396 kB/s | 16 kB 00:00 2024-04-17T05:48:51.744 INFO:teuthology.orchestra.run.smithi103.stdout:(133/174): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-04-17T05:48:51.745 INFO:teuthology.orchestra.run.smithi098.stdout:(136/174): python3-pecan-1.4.2-3.el9.noarch.rpm 2.4 MB/s | 272 kB 00:00 2024-04-17T05:48:51.770 INFO:teuthology.orchestra.run.smithi098.stdout:(137/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.3 MB/s | 90 kB 00:00 2024-04-17T05:48:51.791 INFO:teuthology.orchestra.run.smithi047.stdout:(105/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 121 kB/s | 94 kB 00:00 2024-04-17T05:48:51.794 INFO:teuthology.orchestra.run.smithi103.stdout:(134/174): python3-pecan-1.4.2-3.el9.noarch.rpm 5.3 MB/s | 272 kB 00:00 2024-04-17T05:48:51.795 INFO:teuthology.orchestra.run.smithi098.stdout:(138/174): python3-pyperclip-1.8.2-2.el9.noarch 557 kB/s | 28 kB 00:00 2024-04-17T05:48:51.821 INFO:teuthology.orchestra.run.smithi098.stdout:(139/174): python3-repoze-lru-0.7-16.el9.noarch 610 kB/s | 31 kB 00:00 2024-04-17T05:48:51.845 INFO:teuthology.orchestra.run.smithi103.stdout:(135/174): python3-portend-3.1.0-2.el9.noarch.r 329 kB/s | 16 kB 00:00 2024-04-17T05:48:51.863 INFO:teuthology.orchestra.run.smithi098.stdout:(140/174): python3-rsa-4.9-2.el9.noarch.rpm 1.4 MB/s | 59 kB 00:00 2024-04-17T05:48:51.888 INFO:teuthology.orchestra.run.smithi098.stdout:(141/174): python3-routes-2.5.1-5.el9.noarch.rp 2.0 MB/s | 188 kB 00:00 2024-04-17T05:48:51.895 INFO:teuthology.orchestra.run.smithi103.stdout:(136/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.8 MB/s | 90 kB 00:00 2024-04-17T05:48:51.913 INFO:teuthology.orchestra.run.smithi098.stdout:(142/174): python3-tempora-5.0.0-2.el9.noarch.r 711 kB/s | 36 kB 00:00 2024-04-17T05:48:51.939 INFO:teuthology.orchestra.run.smithi098.stdout:(143/174): python3-typing-extensions-4.4.0-2.el 1.0 MB/s | 51 kB 00:00 2024-04-17T05:48:51.945 INFO:teuthology.orchestra.run.smithi103.stdout:(137/174): python3-pyperclip-1.8.2-2.el9.noarch 564 kB/s | 28 kB 00:00 2024-04-17T05:48:51.996 INFO:teuthology.orchestra.run.smithi103.stdout:(138/174): python3-repoze-lru-0.7-16.el9.noarch 614 kB/s | 31 kB 00:00 2024-04-17T05:48:52.006 INFO:teuthology.orchestra.run.smithi098.stdout:(144/174): python3-websocket-client-1.2.3-2.el9 1.3 MB/s | 90 kB 00:00 2024-04-17T05:48:52.031 INFO:teuthology.orchestra.run.smithi098.stdout:(145/174): python3-webob-1.8.7-6.el9.noarch.rpm 1.9 MB/s | 230 kB 00:00 2024-04-17T05:48:52.046 INFO:teuthology.orchestra.run.smithi103.stdout:(139/174): python3-routes-2.5.1-5.el9.noarch.rp 3.7 MB/s | 188 kB 00:00 2024-04-17T05:48:52.049 INFO:teuthology.orchestra.run.smithi047.stdout:(106/174): abseil-cpp-20211102.0-3.el9.x86_64.r 913 kB/s | 548 kB 00:00 2024-04-17T05:48:52.073 INFO:teuthology.orchestra.run.smithi098.stdout:(146/174): python3-zc-lockfile-2.0-10.el9.noarc 479 kB/s | 20 kB 00:00 2024-04-17T05:48:52.096 INFO:teuthology.orchestra.run.smithi103.stdout:(140/174): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-04-17T05:48:52.141 INFO:teuthology.orchestra.run.smithi047.stdout:(107/174): grpc-data-1.46.7-10.el9.noarch.rpm 212 kB/s | 19 kB 00:00 2024-04-17T05:48:52.146 INFO:teuthology.orchestra.run.smithi103.stdout:(141/174): python3-tempora-5.0.0-2.el9.noarch.r 717 kB/s | 36 kB 00:00 2024-04-17T05:48:52.148 INFO:teuthology.orchestra.run.smithi098.stdout:(147/174): python3-werkzeug-2.0.3-3.el9.1.noarc 2.9 MB/s | 427 kB 00:00 2024-04-17T05:48:52.173 INFO:teuthology.orchestra.run.smithi098.stdout:(148/174): re2-20211101-3.el9.x86_64.rpm 1.9 MB/s | 192 kB 00:00 2024-04-17T05:48:52.205 INFO:teuthology.orchestra.run.smithi103.stdout:(142/174): python3-typing-extensions-4.4.0-2.el 875 kB/s | 51 kB 00:00 2024-04-17T05:48:52.250 INFO:teuthology.orchestra.run.smithi047.stdout:(108/174): gperftools-libs-2.9.1-2.el9.x86_64.r 673 kB/s | 309 kB 00:00 2024-04-17T05:48:52.255 INFO:teuthology.orchestra.run.smithi103.stdout:(143/174): python3-webob-1.8.7-6.el9.noarch.rpm 4.5 MB/s | 230 kB 00:00 2024-04-17T05:48:52.265 INFO:teuthology.orchestra.run.smithi098.stdout:(149/174): python3-kubernetes-26.1.0-2.el9.noar 1.1 MB/s | 1.0 MB 00:00 2024-04-17T05:48:52.306 INFO:teuthology.orchestra.run.smithi103.stdout:(144/174): python3-websocket-client-1.2.3-2.el9 1.8 MB/s | 90 kB 00:00 2024-04-17T05:48:52.356 INFO:teuthology.orchestra.run.smithi103.stdout:(145/174): python3-werkzeug-2.0.3-3.el9.1.noarc 8.3 MB/s | 427 kB 00:00 2024-04-17T05:48:52.407 INFO:teuthology.orchestra.run.smithi103.stdout:(146/174): python3-zc-lockfile-2.0-10.el9.noarc 400 kB/s | 20 kB 00:00 2024-04-17T05:48:52.425 INFO:teuthology.orchestra.run.smithi047.stdout:(109/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 148 kB/s | 26 kB 00:00 2024-04-17T05:48:52.457 INFO:teuthology.orchestra.run.smithi103.stdout:(147/174): re2-20211101-3.el9.x86_64.rpm 3.8 MB/s | 192 kB 00:00 2024-04-17T05:48:52.533 INFO:teuthology.orchestra.run.smithi047.stdout:(110/174): liboath-2.6.7-2.el9.x86_64.rpm 449 kB/s | 49 kB 00:00 2024-04-17T05:48:52.540 INFO:teuthology.orchestra.run.smithi103.stdout:(148/174): thrift-0.15.0-2.el9.x86_64.rpm 19 MB/s | 1.6 MB 00:00 2024-04-17T05:48:52.616 INFO:teuthology.orchestra.run.smithi098.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 3.4 MB/s | 1.6 MB 00:00 2024-04-17T05:48:52.634 INFO:teuthology.orchestra.run.smithi047.stdout:(111/174): libunwind-1.6.2-1.el9.x86_64.rpm 674 kB/s | 67 kB 00:00 2024-04-17T05:48:52.708 INFO:teuthology.orchestra.run.smithi098.stdout:(151/174): NetworkManager-1.47.4-1.el9.x86_64.r 25 MB/s | 2.3 MB 00:00 2024-04-17T05:48:52.734 INFO:teuthology.orchestra.run.smithi047.stdout:(112/174): luarocks-3.9.2-1.el9.noarch.rpm 1.5 MB/s | 151 kB 00:00 2024-04-17T05:48:52.791 INFO:teuthology.orchestra.run.smithi098.stdout:(152/174): NetworkManager-libnm-1.47.4-1.el9.x8 22 MB/s | 1.8 MB 00:00 2024-04-17T05:48:52.825 INFO:teuthology.orchestra.run.smithi098.stdout:(153/174): NetworkManager-team-1.47.4-1.el9.x86 1.2 MB/s | 41 kB 00:00 2024-04-17T05:48:52.867 INFO:teuthology.orchestra.run.smithi098.stdout:(154/174): NetworkManager-tui-1.47.4-1.el9.x86_ 5.8 MB/s | 248 kB 00:00 2024-04-17T05:48:52.909 INFO:teuthology.orchestra.run.smithi098.stdout:(155/174): xfsdump-3.1.12-4.el9.x86_64.rpm 7.6 MB/s | 321 kB 00:00 2024-04-17T05:48:52.976 INFO:teuthology.orchestra.run.smithi047.stdout:(113/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.4 MB/s | 838 kB 00:00 2024-04-17T05:48:52.984 INFO:teuthology.orchestra.run.smithi098.stdout:(156/174): librbd1-19.0.0-3133.ga8c84f59.el9.x8 4.4 MB/s | 3.1 MB 00:00 2024-04-17T05:48:53.034 INFO:teuthology.orchestra.run.smithi098.stdout:(157/174): acl-2.3.1-4.el9.x86_64.rpm 569 kB/s | 71 kB 00:00 2024-04-17T05:48:53.060 INFO:teuthology.orchestra.run.smithi098.stdout:(158/174): elfutils-debuginfod-client-0.190-2.e 485 kB/s | 36 kB 00:00 2024-04-17T05:48:53.110 INFO:teuthology.orchestra.run.smithi098.stdout:(159/174): elfutils-libelf-0.190-2.el9.x86_64.r 2.5 MB/s | 192 kB 00:00 2024-04-17T05:48:53.126 INFO:teuthology.orchestra.run.smithi047.stdout:(114/174): libarrow-9.0.0-10.el9.x86_64.rpm 4.5 MB/s | 4.4 MB 00:00 2024-04-17T05:48:53.151 INFO:teuthology.orchestra.run.smithi047.stdout:(115/174): python3-asyncssh-2.13.2-5.el9.noarch 3.1 MB/s | 548 kB 00:00 2024-04-17T05:48:53.152 INFO:teuthology.orchestra.run.smithi098.stdout:(160/174): elfutils-libs-0.190-2.el9.x86_64.rpm 2.7 MB/s | 256 kB 00:00 2024-04-17T05:48:53.177 INFO:teuthology.orchestra.run.smithi098.stdout:(161/174): libacl-2.3.1-4.el9.x86_64.rpm 344 kB/s | 23 kB 00:00 2024-04-17T05:48:53.202 INFO:teuthology.orchestra.run.smithi098.stdout:(162/174): libblkid-2.37.4-18.el9.x86_64.rpm 2.1 MB/s | 108 kB 00:00 2024-04-17T05:48:53.218 INFO:teuthology.orchestra.run.smithi047.stdout:(116/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 473 kB/s | 43 kB 00:00 2024-04-17T05:48:53.228 INFO:teuthology.orchestra.run.smithi098.stdout:(163/174): libfdisk-2.37.4-18.el9.x86_64.rpm 3.0 MB/s | 155 kB 00:00 2024-04-17T05:48:53.244 INFO:teuthology.orchestra.run.smithi047.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 350 kB/s | 32 kB 00:00 2024-04-17T05:48:53.253 INFO:teuthology.orchestra.run.smithi098.stdout:(164/174): libmount-2.37.4-18.el9.x86_64.rpm 2.6 MB/s | 136 kB 00:00 2024-04-17T05:48:53.311 INFO:teuthology.orchestra.run.smithi047.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 163 kB/s | 15 kB 00:00 2024-04-17T05:48:53.328 INFO:teuthology.orchestra.run.smithi098.stdout:(165/174): librados2-19.0.0-3133.ga8c84f59.el9. 2.9 MB/s | 3.4 MB 00:01 2024-04-17T05:48:53.344 INFO:teuthology.orchestra.run.smithi047.stdout:(119/174): python3-cheroot-8.6.0-4.el9.noarch.r 1.7 MB/s | 172 kB 00:00 2024-04-17T05:48:53.353 INFO:teuthology.orchestra.run.smithi098.stdout:(166/174): libsmartcols-2.37.4-18.el9.x86_64.rp 509 kB/s | 64 kB 00:00 2024-04-17T05:48:53.375 INFO:teuthology.orchestra.run.smithi103.stdout:(149/174): ceph-test-19.0.0-3133.ga8c84f59.el9. 3.3 MB/s | 49 MB 00:14 2024-04-17T05:48:53.387 INFO:teuthology.orchestra.run.smithi098.stdout:(167/174): libuuid-2.37.4-18.el9.x86_64.rpm 214 kB/s | 29 kB 00:00 2024-04-17T05:48:53.411 INFO:teuthology.orchestra.run.smithi047.stdout:(120/174): python3-cherrypy-18.6.1-2.el9.noarch 3.5 MB/s | 358 kB 00:00 2024-04-17T05:48:53.445 INFO:teuthology.orchestra.run.smithi047.stdout:(121/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.0 MB/s | 303 kB 00:00 2024-04-17T05:48:53.467 INFO:teuthology.orchestra.run.smithi103.stdout:(150/174): openssl-devel-3.0.7-27.el9.x86_64.rp 654 kB/s | 4.1 MB 00:06 2024-04-17T05:48:53.487 INFO:teuthology.orchestra.run.smithi098.stdout:(168/174): openssl-3.0.7-27.el9.x86_64.rpm 7.6 MB/s | 1.2 MB 00:00 2024-04-17T05:48:53.503 INFO:teuthology.orchestra.run.smithi047.stdout:(122/174): python3-colorama-0.4.4-12.el9.noarch 384 kB/s | 35 kB 00:00 2024-04-17T05:48:53.530 INFO:teuthology.orchestra.run.smithi098.stdout:(169/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 12 MB/s | 2.2 MB 00:00 2024-04-17T05:48:53.554 INFO:teuthology.orchestra.run.smithi047.stdout:(123/174): python3-google-auth-2.28.2-1.el9.noa 1.8 MB/s | 203 kB 00:00 2024-04-17T05:48:53.555 INFO:teuthology.orchestra.run.smithi098.stdout:(170/174): util-linux-core-2.37.4-18.el9.x86_64 6.8 MB/s | 465 kB 00:00 2024-04-17T05:48:53.567 INFO:teuthology.orchestra.run.smithi103.stdout:(151/174): NetworkManager-1.47.4-1.el9.x86_64.r 23 MB/s | 2.3 MB 00:00 2024-04-17T05:48:53.572 INFO:teuthology.orchestra.run.smithi098.stdout:(171/174): util-linux-user-2.37.4-18.el9.x86_64 743 kB/s | 31 kB 00:00 2024-04-17T05:48:53.622 INFO:teuthology.orchestra.run.smithi098.stdout:(172/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 16 MB/s | 1.1 MB 00:00 2024-04-17T05:48:53.654 INFO:teuthology.orchestra.run.smithi047.stdout:(124/174): python3-grpcio-tools-1.46.7-10.el9.x 1.4 MB/s | 144 kB 00:00 2024-04-17T05:48:53.659 INFO:teuthology.orchestra.run.smithi103.stdout:(152/174): NetworkManager-libnm-1.47.4-1.el9.x8 20 MB/s | 1.8 MB 00:00 2024-04-17T05:48:53.672 INFO:teuthology.orchestra.run.smithi098.stdout:(173/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 742 kB/s | 74 kB 00:00 2024-04-17T05:48:53.763 INFO:teuthology.orchestra.run.smithi047.stdout:(125/174): python3-jaraco-8.2.1-3.el9.noarch.rp 99 kB/s | 11 kB 00:00 2024-04-17T05:48:53.834 INFO:teuthology.orchestra.run.smithi103.stdout:(153/174): librados2-19.0.0-3133.ga8c84f59.el9. 2.6 MB/s | 3.4 MB 00:01 2024-04-17T05:48:53.860 INFO:teuthology.orchestra.run.smithi103.stdout:(154/174): NetworkManager-team-1.47.4-1.el9.x86 203 kB/s | 41 kB 00:00 2024-04-17T05:48:53.896 INFO:teuthology.orchestra.run.smithi047.stdout:(126/174): python3-grpcio-1.46.7-10.el9.x86_64. 5.2 MB/s | 2.0 MB 00:00 2024-04-17T05:48:53.901 INFO:teuthology.orchestra.run.smithi103.stdout:(155/174): NetworkManager-tui-1.47.4-1.el9.x86_ 3.6 MB/s | 248 kB 00:00 2024-04-17T05:48:53.921 INFO:teuthology.orchestra.run.smithi047.stdout:(127/174): python3-jaraco-classes-3.2.1-5.el9.n 111 kB/s | 18 kB 00:00 2024-04-17T05:48:53.935 INFO:teuthology.orchestra.run.smithi103.stdout:(156/174): xfsdump-3.1.12-4.el9.x86_64.rpm 4.2 MB/s | 321 kB 00:00 2024-04-17T05:48:53.989 INFO:teuthology.orchestra.run.smithi047.stdout:(128/174): python3-jaraco-collections-3.0.0-8.e 252 kB/s | 23 kB 00:00 2024-04-17T05:48:54.014 INFO:teuthology.orchestra.run.smithi047.stdout:(129/174): python3-jaraco-functools-3.5.0-2.el9 211 kB/s | 19 kB 00:00 2024-04-17T05:48:54.018 INFO:teuthology.orchestra.run.smithi103.stdout:(157/174): acl-2.3.1-4.el9.x86_64.rpm 610 kB/s | 71 kB 00:00 2024-04-17T05:48:54.035 INFO:teuthology.orchestra.run.smithi103.stdout:(158/174): elfutils-debuginfod-client-0.190-2.e 364 kB/s | 36 kB 00:00 2024-04-17T05:48:54.080 INFO:teuthology.orchestra.run.smithi047.stdout:(130/174): python3-jaraco-text-3.2.0-6.el9.noar 214 kB/s | 20 kB 00:00 2024-04-17T05:48:54.102 INFO:teuthology.orchestra.run.smithi103.stdout:(159/174): elfutils-libelf-0.190-2.el9.x86_64.r 2.8 MB/s | 192 kB 00:00 2024-04-17T05:48:54.127 INFO:teuthology.orchestra.run.smithi103.stdout:(160/174): elfutils-libs-0.190-2.el9.x86_64.rpm 2.7 MB/s | 256 kB 00:00 2024-04-17T05:48:54.153 INFO:teuthology.orchestra.run.smithi103.stdout:(161/174): libacl-2.3.1-4.el9.x86_64.rpm 458 kB/s | 23 kB 00:00 2024-04-17T05:48:54.172 INFO:teuthology.orchestra.run.smithi047.stdout:(131/174): python3-logutils-0.3.5-21.el9.noarch 504 kB/s | 46 kB 00:00 2024-04-17T05:48:54.178 INFO:teuthology.orchestra.run.smithi103.stdout:(162/174): libblkid-2.37.4-18.el9.x86_64.rpm 2.1 MB/s | 108 kB 00:00 2024-04-17T05:48:54.203 INFO:teuthology.orchestra.run.smithi103.stdout:(163/174): libfdisk-2.37.4-18.el9.x86_64.rpm 3.0 MB/s | 155 kB 00:00 2024-04-17T05:48:54.214 INFO:teuthology.orchestra.run.smithi098.stdout:(174/174): util-linux-2.37.4-18.el9.x86_64.rpm 2.8 MB/s | 2.3 MB 00:00 2024-04-17T05:48:54.221 INFO:teuthology.orchestra.run.smithi098.stdout:-------------------------------------------------------------------------------- 2024-04-17T05:48:54.221 INFO:teuthology.orchestra.run.smithi098.stdout:Total 10 MB/s | 229 MB 00:22 2024-04-17T05:48:54.229 INFO:teuthology.orchestra.run.smithi103.stdout:(164/174): libmount-2.37.4-18.el9.x86_64.rpm 2.6 MB/s | 136 kB 00:00 2024-04-17T05:48:54.254 INFO:teuthology.orchestra.run.smithi103.stdout:(165/174): libsmartcols-2.37.4-18.el9.x86_64.rp 1.2 MB/s | 64 kB 00:00 2024-04-17T05:48:54.279 INFO:teuthology.orchestra.run.smithi103.stdout:(166/174): libuuid-2.37.4-18.el9.x86_64.rpm 573 kB/s | 29 kB 00:00 2024-04-17T05:48:54.346 INFO:teuthology.orchestra.run.smithi047.stdout:(132/174): python3-more-itertools-8.12.0-2.el9. 454 kB/s | 79 kB 00:00 2024-04-17T05:48:54.355 INFO:teuthology.orchestra.run.smithi103.stdout:(167/174): openssl-3.0.7-27.el9.x86_64.rpm 12 MB/s | 1.2 MB 00:00 2024-04-17T05:48:54.430 INFO:teuthology.orchestra.run.smithi103.stdout:(168/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 14 MB/s | 2.2 MB 00:00 2024-04-17T05:48:54.456 INFO:teuthology.orchestra.run.smithi047.stdout:(133/174): python3-kubernetes-26.1.0-2.el9.noar 2.3 MB/s | 1.0 MB 00:00 2024-04-17T05:48:54.473 INFO:teuthology.orchestra.run.smithi047.stdout:(134/174): python3-natsort-7.1.1-5.el9.noarch.r 455 kB/s | 58 kB 00:00 2024-04-17T05:48:54.489 INFO:teuthology.orchestra.run.smithi103.stdout:(169/174): util-linux-2.37.4-18.el9.x86_64.rpm 17 MB/s | 2.3 MB 00:00 2024-04-17T05:48:54.514 INFO:teuthology.orchestra.run.smithi103.stdout:(170/174): util-linux-core-2.37.4-18.el9.x86_64 5.5 MB/s | 465 kB 00:00 2024-04-17T05:48:54.539 INFO:teuthology.orchestra.run.smithi103.stdout:(171/174): util-linux-user-2.37.4-18.el9.x86_64 623 kB/s | 31 kB 00:00 2024-04-17T05:48:54.573 INFO:teuthology.orchestra.run.smithi103.stdout:(172/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 18 MB/s | 1.1 MB 00:00 2024-04-17T05:48:54.573 INFO:teuthology.orchestra.run.smithi047.stdout:(135/174): python3-portend-3.1.0-2.el9.noarch.r 164 kB/s | 16 kB 00:00 2024-04-17T05:48:54.602 INFO:teuthology.orchestra.run.smithi098.stdout:Running transaction check 2024-04-17T05:48:54.657 INFO:teuthology.orchestra.run.smithi047.stdout:(136/174): python3-pecan-1.4.2-3.el9.noarch.rpm 1.3 MB/s | 272 kB 00:00 2024-04-17T05:48:54.682 INFO:teuthology.orchestra.run.smithi047.stdout:(137/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 828 kB/s | 90 kB 00:00 2024-04-17T05:48:54.750 INFO:teuthology.orchestra.run.smithi047.stdout:(138/174): python3-pyperclip-1.8.2-2.el9.noarch 307 kB/s | 28 kB 00:00 2024-04-17T05:48:54.800 INFO:teuthology.orchestra.run.smithi047.stdout:(139/174): python3-repoze-lru-0.7-16.el9.noarch 263 kB/s | 31 kB 00:00 2024-04-17T05:48:54.850 INFO:teuthology.orchestra.run.smithi047.stdout:(140/174): python3-routes-2.5.1-5.el9.noarch.rp 1.8 MB/s | 188 kB 00:00 2024-04-17T05:48:54.871 INFO:teuthology.orchestra.run.smithi098.stdout:Transaction check succeeded. 2024-04-17T05:48:54.871 INFO:teuthology.orchestra.run.smithi098.stdout:Running transaction test 2024-04-17T05:48:54.892 INFO:teuthology.orchestra.run.smithi047.stdout:(141/174): python3-rsa-4.9-2.el9.noarch.rpm 643 kB/s | 59 kB 00:00 2024-04-17T05:48:54.942 INFO:teuthology.orchestra.run.smithi047.stdout:(142/174): python3-tempora-5.0.0-2.el9.noarch.r 389 kB/s | 36 kB 00:00 2024-04-17T05:48:54.984 INFO:teuthology.orchestra.run.smithi047.stdout:(143/174): python3-typing-extensions-4.4.0-2.el 554 kB/s | 51 kB 00:00 2024-04-17T05:48:55.043 INFO:teuthology.orchestra.run.smithi047.stdout:(144/174): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-04-17T05:48:55.076 INFO:teuthology.orchestra.run.smithi047.stdout:(145/174): python3-websocket-client-1.2.3-2.el9 977 kB/s | 90 kB 00:00 2024-04-17T05:48:55.107 INFO:teuthology.orchestra.run.smithi103.stdout:(173/174): librbd1-19.0.0-3133.ga8c84f59.el9.x8 1.8 MB/s | 3.1 MB 00:01 2024-04-17T05:48:55.168 INFO:teuthology.orchestra.run.smithi047.stdout:(146/174): python3-zc-lockfile-2.0-10.el9.noarc 217 kB/s | 20 kB 00:00 2024-04-17T05:48:55.218 INFO:teuthology.orchestra.run.smithi047.stdout:(147/174): python3-werkzeug-2.0.3-3.el9.1.noarc 2.4 MB/s | 427 kB 00:00 2024-04-17T05:48:55.248 INFO:teuthology.orchestra.run.smithi103.stdout:(174/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 105 kB/s | 74 kB 00:00 2024-04-17T05:48:55.255 INFO:teuthology.orchestra.run.smithi103.stdout:-------------------------------------------------------------------------------- 2024-04-17T05:48:55.255 INFO:teuthology.orchestra.run.smithi103.stdout:Total 10 MB/s | 229 MB 00:22 2024-04-17T05:48:55.269 INFO:teuthology.orchestra.run.smithi047.stdout:(148/174): re2-20211101-3.el9.x86_64.rpm 1.9 MB/s | 192 kB 00:00 2024-04-17T05:48:55.618 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction check 2024-04-17T05:48:55.636 INFO:teuthology.orchestra.run.smithi047.stdout:(149/174): thrift-0.15.0-2.el9.x86_64.rpm 3.8 MB/s | 1.6 MB 00:00 2024-04-17T05:48:55.795 INFO:teuthology.orchestra.run.smithi047.stdout:(150/174): librados2-19.0.0-3133.ga8c84f59.el9. 6.5 MB/s | 3.4 MB 00:00 2024-04-17T05:48:55.870 INFO:teuthology.orchestra.run.smithi047.stdout:(151/174): openssl-devel-3.0.7-27.el9.x86_64.rp 687 kB/s | 4.1 MB 00:06 2024-04-17T05:48:55.890 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction check succeeded. 2024-04-17T05:48:55.890 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction test 2024-04-17T05:48:55.930 INFO:teuthology.orchestra.run.smithi047.stdout:(152/174): NetworkManager-1.47.4-1.el9.x86_64.r 17 MB/s | 2.3 MB 00:00 2024-04-17T05:48:55.970 INFO:teuthology.orchestra.run.smithi047.stdout:(153/174): NetworkManager-libnm-1.47.4-1.el9.x8 19 MB/s | 1.8 MB 00:00 2024-04-17T05:48:55.996 INFO:teuthology.orchestra.run.smithi047.stdout:(154/174): NetworkManager-team-1.47.4-1.el9.x86 636 kB/s | 41 kB 00:00 2024-04-17T05:48:56.021 INFO:teuthology.orchestra.run.smithi047.stdout:(155/174): NetworkManager-tui-1.47.4-1.el9.x86_ 4.9 MB/s | 248 kB 00:00 2024-04-17T05:48:56.046 INFO:teuthology.orchestra.run.smithi047.stdout:(156/174): xfsdump-3.1.12-4.el9.x86_64.rpm 6.3 MB/s | 321 kB 00:00 2024-04-17T05:48:56.246 INFO:teuthology.orchestra.run.smithi047.stdout:(157/174): librbd1-19.0.0-3133.ga8c84f59.el9.x8 5.2 MB/s | 3.1 MB 00:00 2024-04-17T05:48:56.322 INFO:teuthology.orchestra.run.smithi047.stdout:(158/174): elfutils-debuginfod-client-0.190-2.e 132 kB/s | 36 kB 00:00 2024-04-17T05:48:56.372 INFO:teuthology.orchestra.run.smithi047.stdout:(159/174): acl-2.3.1-4.el9.x86_64.rpm 204 kB/s | 71 kB 00:00 2024-04-17T05:48:56.489 INFO:teuthology.orchestra.run.smithi047.stdout:(160/174): libacl-2.3.1-4.el9.x86_64.rpm 198 kB/s | 23 kB 00:00 2024-04-17T05:48:56.587 INFO:teuthology.orchestra.run.smithi098.stdout:Transaction test succeeded. 2024-04-17T05:48:56.587 INFO:teuthology.orchestra.run.smithi098.stdout:Running transaction 2024-04-17T05:48:56.697 INFO:teuthology.orchestra.run.smithi047.stdout:(161/174): libblkid-2.37.4-18.el9.x86_64.rpm 520 kB/s | 108 kB 00:00 2024-04-17T05:48:56.797 INFO:teuthology.orchestra.run.smithi047.stdout:(162/174): elfutils-libs-0.190-2.el9.x86_64.rpm 537 kB/s | 256 kB 00:00 2024-04-17T05:48:56.823 INFO:teuthology.orchestra.run.smithi047.stdout:(163/174): libfdisk-2.37.4-18.el9.x86_64.rpm 1.2 MB/s | 155 kB 00:00 2024-04-17T05:48:56.856 INFO:teuthology.orchestra.run.smithi047.stdout:(164/174): elfutils-libelf-0.190-2.el9.x86_64.r 315 kB/s | 192 kB 00:00 2024-04-17T05:48:56.907 INFO:teuthology.orchestra.run.smithi047.stdout:(165/174): libmount-2.37.4-18.el9.x86_64.rpm 1.2 MB/s | 136 kB 00:00 2024-04-17T05:48:56.932 INFO:teuthology.orchestra.run.smithi047.stdout:(166/174): libsmartcols-2.37.4-18.el9.x86_64.rp 587 kB/s | 64 kB 00:00 2024-04-17T05:48:56.965 INFO:teuthology.orchestra.run.smithi047.stdout:(167/174): libuuid-2.37.4-18.el9.x86_64.rpm 264 kB/s | 29 kB 00:00 2024-04-17T05:48:57.274 INFO:teuthology.orchestra.run.smithi047.stdout:(168/174): openssl-3.0.7-27.el9.x86_64.rpm 3.3 MB/s | 1.2 MB 00:00 2024-04-17T05:48:57.391 INFO:teuthology.orchestra.run.smithi047.stdout:(169/174): util-linux-core-2.37.4-18.el9.x86_64 3.9 MB/s | 465 kB 00:00 2024-04-17T05:48:57.483 INFO:teuthology.orchestra.run.smithi047.stdout:(170/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 3.9 MB/s | 2.2 MB 00:00 2024-04-17T05:48:57.508 INFO:teuthology.orchestra.run.smithi047.stdout:(171/174): util-linux-user-2.37.4-18.el9.x86_64 266 kB/s | 31 kB 00:00 2024-04-17T05:48:57.559 INFO:teuthology.orchestra.run.smithi047.stdout:(172/174): util-linux-2.37.4-18.el9.x86_64.rpm 3.8 MB/s | 2.3 MB 00:00 2024-04-17T05:48:57.610 INFO:teuthology.orchestra.run.smithi047.stdout:(173/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 8.5 MB/s | 1.1 MB 00:00 2024-04-17T05:48:57.628 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction test succeeded. 2024-04-17T05:48:57.628 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction 2024-04-17T05:48:58.244 INFO:teuthology.orchestra.run.smithi047.stdout:(174/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 101 kB/s | 74 kB 00:00 2024-04-17T05:48:58.248 INFO:teuthology.orchestra.run.smithi047.stdout:-------------------------------------------------------------------------------- 2024-04-17T05:48:58.249 INFO:teuthology.orchestra.run.smithi047.stdout:Total 9.4 MB/s | 229 MB 00:24 2024-04-17T05:48:58.644 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction check 2024-04-17T05:48:58.943 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction check succeeded. 2024-04-17T05:48:58.943 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction test 2024-04-17T05:49:01.083 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction test succeeded. 2024-04-17T05:49:01.083 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction 2024-04-17T05:49:03.454 INFO:teuthology.orchestra.run.smithi098.stdout: Preparing : 1/1 2024-04-17T05:49:03.576 INFO:teuthology.orchestra.run.smithi098.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-04-17T05:49:03.602 INFO:teuthology.orchestra.run.smithi098.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-04-17T05:49:03.625 INFO:teuthology.orchestra.run.smithi098.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-17T05:49:03.634 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-17T05:49:03.674 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-04-17T05:49:03.727 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-04-17T05:49:03.761 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-04-17T05:49:03.787 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-04-17T05:49:03.922 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-04-17T05:49:04.341 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-04-17T05:49:04.405 INFO:teuthology.orchestra.run.smithi098.stdout: Upgrading : librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 10/198 2024-04-17T05:49:04.529 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 10/198 2024-04-17T05:49:04.565 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 11/198 2024-04-17T05:49:04.647 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 11/198 2024-04-17T05:49:04.685 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-rados-2:19.0.0-3133.ga8c84f59.el9.x86_64 12/198 2024-04-17T05:49:04.727 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 13/198 2024-04-17T05:49:04.747 INFO:teuthology.orchestra.run.smithi103.stdout: Preparing : 1/1 2024-04-17T05:49:04.862 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-04-17T05:49:04.889 INFO:teuthology.orchestra.run.smithi098.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-04-17T05:49:04.895 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-04-17T05:49:04.933 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : postgresql-13.14-1.el9.x86_64 15/198 2024-04-17T05:49:04.934 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-17T05:49:04.943 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-17T05:49:04.953 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 16/198 2024-04-17T05:49:04.993 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-04-17T05:49:05.040 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 16/198 2024-04-17T05:49:05.046 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-04-17T05:49:05.062 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86 17/198 2024-04-17T05:49:05.079 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-04-17T05:49:05.118 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86 17/198 2024-04-17T05:49:05.131 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-04-17T05:49:05.157 INFO:teuthology.orchestra.run.smithi098.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-04-17T05:49:05.164 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-04-17T05:49:05.181 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-17T05:49:05.196 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-17T05:49:05.389 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-17T05:49:05.420 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-17T05:49:05.452 INFO:teuthology.orchestra.run.smithi098.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-04-17T05:49:05.483 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-04-17T05:49:05.540 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 10/198 2024-04-17T05:49:05.559 INFO:teuthology.orchestra.run.smithi098.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-04-17T05:49:05.644 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 10/198 2024-04-17T05:49:05.654 INFO:teuthology.orchestra.run.smithi098.stdout: Upgrading : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 23/198 2024-04-17T05:49:05.674 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 11/198 2024-04-17T05:49:05.715 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 23/198 2024-04-17T05:49:05.749 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 11/198 2024-04-17T05:49:05.778 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-rados-2:19.0.0-3133.ga8c84f59.el9.x86_64 12/198 2024-04-17T05:49:05.800 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-17T05:49:05.819 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 13/198 2024-04-17T05:49:05.901 INFO:teuthology.orchestra.run.smithi098.stdout: Upgrading : NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-17T05:49:05.993 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-04-17T05:49:06.043 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : postgresql-13.14-1.el9.x86_64 15/198 2024-04-17T05:49:06.071 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 16/198 2024-04-17T05:49:06.093 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-17T05:49:06.158 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 16/198 2024-04-17T05:49:06.181 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86 17/198 2024-04-17T05:49:06.206 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-04-17T05:49:06.228 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86 17/198 2024-04-17T05:49:06.276 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-04-17T05:49:06.299 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-17T05:49:06.300 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-04-17T05:49:06.314 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-17T05:49:06.329 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-04-17T05:49:06.356 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-04-17T05:49:06.392 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-04-17T05:49:06.466 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-04-17T05:49:06.507 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-17T05:49:06.520 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-04-17T05:49:06.539 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-17T05:49:06.558 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-04-17T05:49:06.579 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-04-17T05:49:06.631 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-04-17T05:49:06.665 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-04-17T05:49:06.674 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-04-17T05:49:06.717 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-04-17T05:49:06.750 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-04-17T05:49:06.777 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-04-17T05:49:06.790 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 23/198 2024-04-17T05:49:06.849 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-04-17T05:49:06.850 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 23/198 2024-04-17T05:49:06.907 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-ceph-common-2:19.0.0-3133.ga8c84f59.el9. 39/198 2024-04-17T05:49:06.914 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-17T05:49:06.951 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-ceph-argparse-2:19.0.0-3133.ga8c84f59.el 40/198 2024-04-17T05:49:06.977 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-17T05:49:06.982 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-cephfs-2:19.0.0-3133.ga8c84f59.el9.x86_6 41/198 2024-04-17T05:49:07.048 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-04-17T05:49:07.094 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-04-17T05:49:07.167 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-04-17T05:49:07.214 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-17T05:49:07.308 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-04-17T05:49:07.319 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-04-17T05:49:07.363 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-04-17T05:49:07.412 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-04-17T05:49:07.457 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-04-17T05:49:07.484 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-04-17T05:49:07.519 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-04-17T05:49:07.593 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-04-17T05:49:07.647 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-04-17T05:49:07.686 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-04-17T05:49:07.750 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-04-17T05:49:07.785 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-04-17T05:49:07.829 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-04-17T05:49:07.861 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-04-17T05:49:07.886 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-04-17T05:49:07.889 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-04-17T05:49:07.960 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-04-17T05:49:07.967 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-04-17T05:49:07.994 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-ceph-common-2:19.0.0-3133.ga8c84f59.el9. 39/198 2024-04-17T05:49:08.038 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-ceph-argparse-2:19.0.0-3133.ga8c84f59.el 40/198 2024-04-17T05:49:08.059 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-04-17T05:49:08.066 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-cephfs-2:19.0.0-3133.ga8c84f59.el9.x86_6 41/198 2024-04-17T05:49:08.104 INFO:teuthology.orchestra.run.smithi098.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-17T05:49:08.113 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-17T05:49:08.134 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-04-17T05:49:08.144 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 51/198 2024-04-17T05:49:08.184 INFO:teuthology.orchestra.run.smithi098.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-04-17T05:49:08.189 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-04-17T05:49:08.222 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : librados-devel-2:19.0.0-3133.ga8c84f59.el9.x86_6 53/198 2024-04-17T05:49:08.252 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-04-17T05:49:08.262 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-04-17T05:49:08.276 INFO:teuthology.orchestra.run.smithi047.stdout: Preparing : 1/1 2024-04-17T05:49:08.365 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-04-17T05:49:08.392 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-04-17T05:49:08.395 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-04-17T05:49:08.430 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-17T05:49:08.439 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-17T05:49:08.450 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-04-17T05:49:08.487 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-04-17T05:49:08.498 INFO:teuthology.orchestra.run.smithi098.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-04-17T05:49:08.543 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-04-17T05:49:08.583 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-04-17T05:49:08.618 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-04-17T05:49:08.691 INFO:teuthology.orchestra.run.smithi098.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-04-17T05:49:08.763 INFO:teuthology.orchestra.run.smithi098.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-04-17T05:49:08.763 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-04-17T05:49:08.807 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-04-17T05:49:08.939 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-04-17T05:49:08.966 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-04-17T05:49:08.985 INFO:teuthology.orchestra.run.smithi098.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-04-17T05:49:09.040 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-04-17T05:49:09.045 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-04-17T05:49:09.117 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-04-17T05:49:09.128 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-04-17T05:49:09.145 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-04-17T05:49:09.156 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-04-17T05:49:09.174 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-17T05:49:09.176 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 10/198 2024-04-17T05:49:09.184 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-17T05:49:09.193 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-04-17T05:49:09.214 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 51/198 2024-04-17T05:49:09.239 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-04-17T05:49:09.255 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-04-17T05:49:09.276 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-04-17T05:49:09.283 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 10/198 2024-04-17T05:49:09.302 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : librados-devel-2:19.0.0-3133.ga8c84f59.el9.x86_6 53/198 2024-04-17T05:49:09.310 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 11/198 2024-04-17T05:49:09.318 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-04-17T05:49:09.331 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-04-17T05:49:09.361 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-04-17T05:49:09.403 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 11/198 2024-04-17T05:49:09.431 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-rados-2:19.0.0-3133.ga8c84f59.el9.x86_64 12/198 2024-04-17T05:49:09.472 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 13/198 2024-04-17T05:49:09.493 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-04-17T05:49:09.542 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-04-17T05:49:09.557 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-04-17T05:49:09.571 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-04-17T05:49:09.613 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-04-17T05:49:09.657 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-04-17T05:49:09.706 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-04-17T05:49:09.707 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : postgresql-13.14-1.el9.x86_64 15/198 2024-04-17T05:49:09.734 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 16/198 2024-04-17T05:49:09.738 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-04-17T05:49:09.747 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-04-17T05:49:09.787 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-04-17T05:49:09.793 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-04-17T05:49:09.813 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-04-17T05:49:09.830 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 16/198 2024-04-17T05:49:09.837 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-04-17T05:49:09.843 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-04-17T05:49:09.860 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86 17/198 2024-04-17T05:49:09.908 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86 17/198 2024-04-17T05:49:09.909 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-04-17T05:49:09.947 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-04-17T05:49:09.969 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-17T05:49:09.979 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-04-17T05:49:09.986 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-17T05:49:10.030 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-04-17T05:49:10.038 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-04-17T05:49:10.085 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-04-17T05:49:10.138 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-04-17T05:49:10.157 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-04-17T05:49:10.179 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-17T05:49:10.187 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-04-17T05:49:10.197 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-04-17T05:49:10.209 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-17T05:49:10.234 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-04-17T05:49:10.236 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-04-17T05:49:10.240 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-04-17T05:49:10.280 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-04-17T05:49:10.291 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-04-17T05:49:10.317 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-04-17T05:49:10.360 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-04-17T05:49:10.403 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-04-17T05:49:10.429 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-04-17T05:49:10.484 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-04-17T05:49:10.533 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-04-17T05:49:10.535 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-04-17T05:49:10.560 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 23/198 2024-04-17T05:49:10.574 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-04-17T05:49:10.605 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-04-17T05:49:10.619 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 23/198 2024-04-17T05:49:10.654 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-04-17T05:49:10.663 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-04-17T05:49:10.698 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-04-17T05:49:10.730 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-04-17T05:49:10.748 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-04-17T05:49:10.761 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-04-17T05:49:10.789 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-04-17T05:49:10.789 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-04-17T05:49:10.820 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-04-17T05:49:10.823 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-04-17T05:49:10.845 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-17T05:49:10.846 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-04-17T05:49:10.863 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-04-17T05:49:10.871 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-04-17T05:49:10.894 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-04-17T05:49:10.924 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-04-17T05:49:10.942 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-04-17T05:49:10.999 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-04-17T05:49:11.029 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-17T05:49:11.034 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-04-17T05:49:11.062 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-04-17T05:49:11.067 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-04-17T05:49:11.109 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-04-17T05:49:11.144 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-04-17T05:49:11.161 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-04-17T05:49:11.205 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-04-17T05:49:11.233 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-04-17T05:49:11.244 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-04-17T05:49:11.249 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-17T05:49:11.300 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-04-17T05:49:11.323 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-04-17T05:49:11.375 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-04-17T05:49:11.380 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-04-17T05:49:11.406 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-04-17T05:49:11.453 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-04-17T05:49:11.475 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-04-17T05:49:11.481 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-04-17T05:49:11.509 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-04-17T05:49:11.526 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-04-17T05:49:11.546 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-04-17T05:49:11.552 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-04-17T05:49:11.589 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-04-17T05:49:11.605 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-04-17T05:49:11.654 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-04-17T05:49:11.665 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-04-17T05:49:11.699 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-04-17T05:49:11.731 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-04-17T05:49:11.734 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-04-17T05:49:11.762 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-04-17T05:49:11.771 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-04-17T05:49:11.790 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-04-17T05:49:11.824 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-04-17T05:49:11.839 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-04-17T05:49:11.897 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-04-17T05:49:11.897 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-04-17T05:49:11.897 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-04-17T05:49:11.923 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-04-17T05:49:11.925 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-04-17T05:49:11.956 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-04-17T05:49:11.983 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-04-17T05:49:12.001 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-04-17T05:49:12.044 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-04-17T05:49:12.054 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-04-17T05:49:12.078 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-04-17T05:49:12.087 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-ceph-common-2:19.0.0-3133.ga8c84f59.el9. 39/198 2024-04-17T05:49:12.119 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-04-17T05:49:12.132 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-ceph-argparse-2:19.0.0-3133.ga8c84f59.el 40/198 2024-04-17T05:49:12.159 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-cephfs-2:19.0.0-3133.ga8c84f59.el9.x86_6 41/198 2024-04-17T05:49:12.169 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-04-17T05:49:12.241 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-04-17T05:49:12.259 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-04-17T05:49:12.283 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-04-17T05:49:12.324 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-04-17T05:49:12.341 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-04-17T05:49:12.359 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-04-17T05:49:12.363 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-04-17T05:49:12.385 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-04-17T05:49:12.397 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-04-17T05:49:12.425 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-04-17T05:49:12.445 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-04-17T05:49:12.472 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-04-17T05:49:12.497 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-04-17T05:49:12.524 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-04-17T05:49:12.553 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-04-17T05:49:12.556 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-04-17T05:49:12.568 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-04-17T05:49:12.618 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-04-17T05:49:12.724 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-04-17T05:49:12.763 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-04-17T05:49:12.839 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-04-17T05:49:13.105 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-04-17T05:49:13.191 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-04-17T05:49:13.283 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-04-17T05:49:13.340 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-04-17T05:49:13.341 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-17T05:49:13.350 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-17T05:49:13.382 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 51/198 2024-04-17T05:49:13.390 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-04-17T05:49:13.424 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-04-17T05:49:13.431 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-04-17T05:49:13.469 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : librados-devel-2:19.0.0-3133.ga8c84f59.el9.x86_6 53/198 2024-04-17T05:49:13.472 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-04-17T05:49:13.472 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-04-17T05:49:13.499 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-04-17T05:49:13.552 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-04-17T05:49:13.585 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-04-17T05:49:13.592 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-04-17T05:49:13.769 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-04-17T05:49:13.798 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-04-17T05:49:13.799 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-04-17T05:49:13.876 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-04-17T05:49:13.980 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-04-17T05:49:14.068 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-04-17T05:49:14.112 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-04-17T05:49:14.251 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-04-17T05:49:14.298 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-04-17T05:49:14.354 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-04-17T05:49:14.426 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-04-17T05:49:14.456 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-04-17T05:49:14.493 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-04-17T05:49:14.510 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-04-17T05:49:14.541 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-04-17T05:49:14.576 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-04-17T05:49:14.619 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-04-17T05:49:14.629 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-04-17T05:49:14.662 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-04-17T05:49:14.799 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-04-17T05:49:14.800 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-04-17T05:49:14.841 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-04-17T05:49:14.872 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-04-17T05:49:14.880 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-04-17T05:49:14.911 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-04-17T05:49:14.913 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-04-17T05:49:14.942 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-04-17T05:49:14.979 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-04-17T05:49:15.011 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-04-17T05:49:15.054 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-04-17T05:49:15.095 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-04-17T05:49:15.120 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-04-17T05:49:15.145 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-04-17T05:49:15.233 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-04-17T05:49:15.311 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-04-17T05:49:15.342 INFO:teuthology.orchestra.run.smithi098.stdout: Upgrading : librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 124/198 2024-04-17T05:49:15.383 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-04-17T05:49:15.433 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 124/198 2024-04-17T05:49:15.463 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-rbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 125/198 2024-04-17T05:49:15.489 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-04-17T05:49:15.500 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-04-17T05:49:15.537 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-04-17T05:49:15.595 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-04-17T05:49:15.650 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-04-17T05:49:15.832 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-04-17T05:49:15.841 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-04-17T05:49:15.891 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-04-17T05:49:15.909 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-04-17T05:49:15.940 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-04-17T05:49:15.971 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-04-17T05:49:15.980 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-04-17T05:49:15.999 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-04-17T05:49:16.012 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 128/198 2024-04-17T05:49:16.034 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-04-17T05:49:16.066 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 128/198 2024-04-17T05:49:16.086 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-04-17T05:49:16.088 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-rgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 129/198 2024-04-17T05:49:16.121 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-04-17T05:49:16.151 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-04-17T05:49:16.179 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-04-17T05:49:16.214 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-04-17T05:49:16.253 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-04-17T05:49:16.284 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-04-17T05:49:16.308 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-04-17T05:49:16.314 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-04-17T05:49:16.337 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 124/198 2024-04-17T05:49:16.393 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-04-17T05:49:16.427 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 124/198 2024-04-17T05:49:16.433 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-04-17T05:49:16.459 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-rbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 125/198 2024-04-17T05:49:16.466 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-04-17T05:49:16.495 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-04-17T05:49:16.508 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-04-17T05:49:16.542 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-04-17T05:49:16.635 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-04-17T05:49:16.723 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-04-17T05:49:16.765 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-04-17T05:49:16.805 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-04-17T05:49:16.852 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-04-17T05:49:16.952 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-04-17T05:49:16.972 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-04-17T05:49:16.999 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 128/198 2024-04-17T05:49:17.007 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-04-17T05:49:17.053 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 128/198 2024-04-17T05:49:17.084 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-rgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 129/198 2024-04-17T05:49:17.766 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-04-17T05:49:17.821 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-04-17T05:49:17.863 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-04-17T05:49:17.912 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-04-17T05:49:18.001 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-04-17T05:49:18.038 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-04-17T05:49:18.223 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-04-17T05:49:18.253 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-04-17T05:49:18.333 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-04-17T05:49:18.343 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 130/198 2024-04-17T05:49:18.385 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 130/198 2024-04-17T05:49:18.922 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 130/198 2024-04-17T05:49:18.972 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 131/198 2024-04-17T05:49:19.002 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-04-17T05:49:19.054 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 131/198 2024-04-17T05:49:19.054 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-17T05:49:19.054 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-17T05:49:19.054 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-04-17T05:49:19.081 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 132/198 2024-04-17T05:49:19.121 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-04-17T05:49:19.385 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 130/198 2024-04-17T05:49:19.431 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 130/198 2024-04-17T05:49:19.967 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 130/198 2024-04-17T05:49:20.001 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 131/198 2024-04-17T05:49:20.082 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 131/198 2024-04-17T05:49:20.082 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-17T05:49:20.082 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-17T05:49:20.082 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-04-17T05:49:20.102 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 132/198 2024-04-17T05:49:20.413 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-04-17T05:49:20.504 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-04-17T05:49:20.534 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-04-17T05:49:20.565 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-04-17T05:49:20.593 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-04-17T05:49:20.923 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-04-17T05:49:20.948 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 124/198 2024-04-17T05:49:21.022 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 124/198 2024-04-17T05:49:21.052 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-rbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 125/198 2024-04-17T05:49:21.097 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-04-17T05:49:21.602 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-04-17T05:49:21.642 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 128/198 2024-04-17T05:49:21.714 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 128/198 2024-04-17T05:49:21.744 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-rgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 129/198 2024-04-17T05:49:24.137 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 130/198 2024-04-17T05:49:24.166 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 130/198 2024-04-17T05:49:24.731 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 130/198 2024-04-17T05:49:24.769 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 131/198 2024-04-17T05:49:24.859 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 131/198 2024-04-17T05:49:24.860 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-17T05:49:24.860 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-17T05:49:24.860 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-17T05:49:24.903 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 132/198 2024-04-17T05:49:30.702 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 132/198 2024-04-17T05:49:30.702 INFO:teuthology.orchestra.run.smithi098.stdout:skipping the directory /sys 2024-04-17T05:49:30.702 INFO:teuthology.orchestra.run.smithi098.stdout:skipping the directory /proc 2024-04-17T05:49:30.702 INFO:teuthology.orchestra.run.smithi098.stdout:skipping the directory /mnt 2024-04-17T05:49:30.703 INFO:teuthology.orchestra.run.smithi098.stdout:skipping the directory /var/tmp 2024-04-17T05:49:30.703 INFO:teuthology.orchestra.run.smithi098.stdout:skipping the directory /home 2024-04-17T05:49:30.703 INFO:teuthology.orchestra.run.smithi098.stdout:skipping the directory /root 2024-04-17T05:49:30.703 INFO:teuthology.orchestra.run.smithi098.stdout:skipping the directory /tmp 2024-04-17T05:49:30.703 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-04-17T05:49:30.866 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 133/198 2024-04-17T05:49:30.882 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 133/198 2024-04-17T05:49:30.882 INFO:teuthology.orchestra.run.smithi098.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-17T05:49:30.882 INFO:teuthology.orchestra.run.smithi098.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-17T05:49:30.882 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-04-17T05:49:32.148 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 132/198 2024-04-17T05:49:32.148 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /sys 2024-04-17T05:49:32.148 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /proc 2024-04-17T05:49:32.148 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /mnt 2024-04-17T05:49:32.149 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /var/tmp 2024-04-17T05:49:32.149 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /home 2024-04-17T05:49:32.149 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /root 2024-04-17T05:49:32.149 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /tmp 2024-04-17T05:49:32.149 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-04-17T05:49:32.314 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 133/198 2024-04-17T05:49:32.328 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 133/198 2024-04-17T05:49:32.329 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-17T05:49:32.329 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-17T05:49:32.329 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-04-17T05:49:32.345 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 134/198 2024-04-17T05:49:32.370 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 134/198 2024-04-17T05:49:32.370 INFO:teuthology.orchestra.run.smithi098.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-17T05:49:32.370 INFO:teuthology.orchestra.run.smithi098.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-17T05:49:32.370 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-17T05:49:32.371 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-17T05:49:32.371 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-04-17T05:49:32.595 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 135/198 2024-04-17T05:49:32.617 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 135/198 2024-04-17T05:49:32.617 INFO:teuthology.orchestra.run.smithi098.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-17T05:49:32.617 INFO:teuthology.orchestra.run.smithi098.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-17T05:49:32.617 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-17T05:49:32.617 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-17T05:49:32.617 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-04-17T05:49:33.062 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 136/198 2024-04-17T05:49:33.085 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 136/198 2024-04-17T05:49:33.085 INFO:teuthology.orchestra.run.smithi098.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-17T05:49:33.085 INFO:teuthology.orchestra.run.smithi098.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-17T05:49:33.085 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-17T05:49:33.086 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-17T05:49:33.086 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-04-17T05:49:33.173 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-04-17T05:49:33.658 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 138/198 2024-04-17T05:49:33.688 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 138/198 2024-04-17T05:49:33.720 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3133.ga8c84f59.e 139/198 2024-04-17T05:49:33.781 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3133.ga8c84f59. 140/198 2024-04-17T05:49:33.792 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 134/198 2024-04-17T05:49:33.811 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noa 141/198 2024-04-17T05:49:33.817 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 134/198 2024-04-17T05:49:33.817 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-17T05:49:33.817 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-17T05:49:33.818 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-17T05:49:33.818 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-17T05:49:33.818 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-04-17T05:49:34.043 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 135/198 2024-04-17T05:49:34.064 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 135/198 2024-04-17T05:49:34.065 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-17T05:49:34.065 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-17T05:49:34.065 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-17T05:49:34.065 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-17T05:49:34.065 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-04-17T05:49:34.494 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 136/198 2024-04-17T05:49:34.515 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 136/198 2024-04-17T05:49:34.516 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-17T05:49:34.516 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-17T05:49:34.516 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-17T05:49:34.516 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-17T05:49:34.516 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-04-17T05:49:34.604 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-04-17T05:49:34.836 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noa 141/198 2024-04-17T05:49:34.871 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.n 142/198 2024-04-17T05:49:34.989 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 138/198 2024-04-17T05:49:35.019 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 138/198 2024-04-17T05:49:35.043 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3133.ga8c84f59.e 139/198 2024-04-17T05:49:35.095 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3133.ga8c84f59. 140/198 2024-04-17T05:49:35.117 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noa 141/198 2024-04-17T05:49:35.825 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.n 142/198 2024-04-17T05:49:35.846 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c 143/198 2024-04-17T05:49:35.944 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c 143/198 2024-04-17T05:49:36.142 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3133.ga8c84f59.el 144/198 2024-04-17T05:49:36.143 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noa 141/198 2024-04-17T05:49:36.174 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 145/198 2024-04-17T05:49:36.178 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.n 142/198 2024-04-17T05:49:36.197 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 145/198 2024-04-17T05:49:36.197 INFO:teuthology.orchestra.run.smithi098.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-17T05:49:36.197 INFO:teuthology.orchestra.run.smithi098.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-17T05:49:36.197 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-17T05:49:36.197 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-17T05:49:36.197 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-04-17T05:49:36.241 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 146/198 2024-04-17T05:49:36.252 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 146/198 2024-04-17T05:49:36.297 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-2:19.0.0-3133.ga8c84f59.el9.x86_64 147/198 2024-04-17T05:49:37.132 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.n 142/198 2024-04-17T05:49:37.148 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 132/198 2024-04-17T05:49:37.148 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /sys 2024-04-17T05:49:37.148 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /proc 2024-04-17T05:49:37.148 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /mnt 2024-04-17T05:49:37.148 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /var/tmp 2024-04-17T05:49:37.148 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /home 2024-04-17T05:49:37.148 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /root 2024-04-17T05:49:37.148 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /tmp 2024-04-17T05:49:37.149 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-17T05:49:37.161 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c 143/198 2024-04-17T05:49:37.224 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-04-17T05:49:37.260 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 149/198 2024-04-17T05:49:37.261 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c 143/198 2024-04-17T05:49:37.283 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 149/198 2024-04-17T05:49:37.283 INFO:teuthology.orchestra.run.smithi098.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-17T05:49:37.283 INFO:teuthology.orchestra.run.smithi098.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-17T05:49:37.283 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-17T05:49:37.284 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-17T05:49:37.284 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-04-17T05:49:37.325 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 133/198 2024-04-17T05:49:37.328 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3133.ga8c84 150/198 2024-04-17T05:49:37.340 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 133/198 2024-04-17T05:49:37.340 INFO:teuthology.orchestra.run.smithi047.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-17T05:49:37.340 INFO:teuthology.orchestra.run.smithi047.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-17T05:49:37.340 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-17T05:49:37.350 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3133.ga8c84 150/198 2024-04-17T05:49:37.350 INFO:teuthology.orchestra.run.smithi098.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-17T05:49:37.350 INFO:teuthology.orchestra.run.smithi098.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-17T05:49:37.350 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-04-17T05:49:37.395 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 151/198 2024-04-17T05:49:37.415 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3133.ga8c84f59.el 144/198 2024-04-17T05:49:37.417 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 151/198 2024-04-17T05:49:37.418 INFO:teuthology.orchestra.run.smithi098.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-17T05:49:37.418 INFO:teuthology.orchestra.run.smithi098.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-17T05:49:37.418 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-17T05:49:37.418 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-17T05:49:37.418 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-04-17T05:49:37.439 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 145/198 2024-04-17T05:49:37.462 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 145/198 2024-04-17T05:49:37.462 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-17T05:49:37.462 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-17T05:49:37.462 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-17T05:49:37.462 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-17T05:49:37.462 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-04-17T05:49:37.507 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 146/198 2024-04-17T05:49:37.519 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 146/198 2024-04-17T05:49:37.554 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-2:19.0.0-3133.ga8c84f59.el9.x86_64 147/198 2024-04-17T05:49:37.721 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 152/198 2024-04-17T05:49:37.743 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 152/198 2024-04-17T05:49:37.743 INFO:teuthology.orchestra.run.smithi098.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-17T05:49:37.743 INFO:teuthology.orchestra.run.smithi098.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-17T05:49:37.743 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-17T05:49:37.743 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-17T05:49:37.744 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-04-17T05:49:38.464 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-04-17T05:49:38.483 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 149/198 2024-04-17T05:49:38.506 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 149/198 2024-04-17T05:49:38.506 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-17T05:49:38.506 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-17T05:49:38.506 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-17T05:49:38.506 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-17T05:49:38.506 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-04-17T05:49:38.551 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3133.ga8c84 150/198 2024-04-17T05:49:38.572 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3133.ga8c84 150/198 2024-04-17T05:49:38.573 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-17T05:49:38.573 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-17T05:49:38.573 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-04-17T05:49:38.618 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 151/198 2024-04-17T05:49:38.640 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 151/198 2024-04-17T05:49:38.640 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-17T05:49:38.640 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-17T05:49:38.640 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-17T05:49:38.640 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-17T05:49:38.640 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-04-17T05:49:38.861 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 134/198 2024-04-17T05:49:38.888 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 134/198 2024-04-17T05:49:38.888 INFO:teuthology.orchestra.run.smithi047.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-17T05:49:38.888 INFO:teuthology.orchestra.run.smithi047.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-17T05:49:38.888 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-17T05:49:38.888 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-17T05:49:38.888 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-17T05:49:38.952 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 152/198 2024-04-17T05:49:38.974 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 152/198 2024-04-17T05:49:38.974 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-17T05:49:38.975 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-17T05:49:38.975 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-17T05:49:38.975 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-17T05:49:38.975 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-04-17T05:49:39.121 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 135/198 2024-04-17T05:49:39.144 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 135/198 2024-04-17T05:49:39.144 INFO:teuthology.orchestra.run.smithi047.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-17T05:49:39.144 INFO:teuthology.orchestra.run.smithi047.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-17T05:49:39.144 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-17T05:49:39.144 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-17T05:49:39.144 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-17T05:49:39.605 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 136/198 2024-04-17T05:49:39.628 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 136/198 2024-04-17T05:49:39.628 INFO:teuthology.orchestra.run.smithi047.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-17T05:49:39.628 INFO:teuthology.orchestra.run.smithi047.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-17T05:49:39.628 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-17T05:49:39.628 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-17T05:49:39.628 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-17T05:49:39.723 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-04-17T05:49:40.152 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 138/198 2024-04-17T05:49:40.180 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 138/198 2024-04-17T05:49:40.213 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3133.ga8c84f59.e 139/198 2024-04-17T05:49:40.275 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3133.ga8c84f59. 140/198 2024-04-17T05:49:40.303 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noa 141/198 2024-04-17T05:49:41.398 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noa 141/198 2024-04-17T05:49:41.430 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.n 142/198 2024-04-17T05:49:42.326 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-test-2:19.0.0-3133.ga8c84f59.el9.x86_64 153/198 2024-04-17T05:49:42.367 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : rbd-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 154/198 2024-04-17T05:49:42.397 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : rbd-nbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 155/198 2024-04-17T05:49:42.425 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-04-17T05:49:42.443 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.n 142/198 2024-04-17T05:49:42.463 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c 143/198 2024-04-17T05:49:42.569 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c 143/198 2024-04-17T05:49:42.601 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : cephfs-shell-2:19.0.0-3133.ga8c84f59.el9.x86_64 157/198 2024-04-17T05:49:42.707 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 158/198 2024-04-17T05:49:42.747 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 159/198 2024-04-17T05:49:42.748 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3133.ga8c84f59.el 144/198 2024-04-17T05:49:42.774 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 145/198 2024-04-17T05:49:42.786 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-04-17T05:49:42.797 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 145/198 2024-04-17T05:49:42.798 INFO:teuthology.orchestra.run.smithi047.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-17T05:49:42.798 INFO:teuthology.orchestra.run.smithi047.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-17T05:49:42.798 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-17T05:49:42.798 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-17T05:49:42.798 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-17T05:49:42.839 INFO:teuthology.orchestra.run.smithi098.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-04-17T05:49:42.841 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 146/198 2024-04-17T05:49:42.853 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 146/198 2024-04-17T05:49:42.882 INFO:teuthology.orchestra.run.smithi098.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-04-17T05:49:42.901 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-2:19.0.0-3133.ga8c84f59.el9.x86_64 147/198 2024-04-17T05:49:42.907 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libcephfs-devel-2:19.0.0-3133.ga8c84f59.el9.x86_ 163/198 2024-04-17T05:49:42.947 INFO:teuthology.orchestra.run.smithi098.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-04-17T05:49:42.977 INFO:teuthology.orchestra.run.smithi098.stdout: Upgrading : NetworkManager-team-1:1.47.4-1.el9.x86_64 165/198 2024-04-17T05:49:43.031 INFO:teuthology.orchestra.run.smithi098.stdout: Upgrading : NetworkManager-tui-1:1.47.4-1.el9.x86_64 166/198 2024-04-17T05:49:43.100 INFO:teuthology.orchestra.run.smithi098.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-04-17T05:49:43.134 INFO:teuthology.orchestra.run.smithi098.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-04-17T05:49:43.532 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-test-2:19.0.0-3133.ga8c84f59.el9.x86_64 153/198 2024-04-17T05:49:43.574 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : rbd-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 154/198 2024-04-17T05:49:43.612 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : rbd-nbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 155/198 2024-04-17T05:49:43.640 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-04-17T05:49:43.751 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-17T05:49:43.791 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-17T05:49:43.810 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-17T05:49:43.818 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : cephfs-shell-2:19.0.0-3133.ga8c84f59.el9.x86_64 157/198 2024-04-17T05:49:43.897 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-04-17T05:49:43.931 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 158/198 2024-04-17T05:49:43.946 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 149/198 2024-04-17T05:49:43.969 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 149/198 2024-04-17T05:49:43.970 INFO:teuthology.orchestra.run.smithi047.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-17T05:49:43.970 INFO:teuthology.orchestra.run.smithi047.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-17T05:49:43.970 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-17T05:49:43.970 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-17T05:49:43.970 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-17T05:49:43.971 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 159/198 2024-04-17T05:49:44.002 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-04-17T05:49:44.013 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3133.ga8c84 150/198 2024-04-17T05:49:44.035 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3133.ga8c84 150/198 2024-04-17T05:49:44.035 INFO:teuthology.orchestra.run.smithi047.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-17T05:49:44.036 INFO:teuthology.orchestra.run.smithi047.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-17T05:49:44.036 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-17T05:49:44.072 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-04-17T05:49:44.080 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 151/198 2024-04-17T05:49:44.103 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 151/198 2024-04-17T05:49:44.103 INFO:teuthology.orchestra.run.smithi047.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-17T05:49:44.103 INFO:teuthology.orchestra.run.smithi047.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-17T05:49:44.103 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-17T05:49:44.103 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-17T05:49:44.103 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-17T05:49:44.107 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-04-17T05:49:44.140 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libcephfs-devel-2:19.0.0-3133.ga8c84f59.el9.x86_ 163/198 2024-04-17T05:49:44.172 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-04-17T05:49:44.201 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : NetworkManager-team-1:1.47.4-1.el9.x86_64 165/198 2024-04-17T05:49:44.256 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : NetworkManager-tui-1:1.47.4-1.el9.x86_64 166/198 2024-04-17T05:49:44.318 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-04-17T05:49:44.350 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-04-17T05:49:44.423 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 152/198 2024-04-17T05:49:44.446 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 152/198 2024-04-17T05:49:44.446 INFO:teuthology.orchestra.run.smithi047.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-17T05:49:44.446 INFO:teuthology.orchestra.run.smithi047.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-17T05:49:44.446 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-17T05:49:44.446 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-17T05:49:44.446 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-17T05:49:44.628 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : cephfs-top-2:19.0.0-3133.ga8c84f59.el9.noarch 170/198 2024-04-17T05:49:44.711 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-04-17T05:49:44.752 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-04-17T05:49:44.833 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-04-17T05:49:44.865 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-04-17T05:49:44.890 INFO:teuthology.orchestra.run.smithi098.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-04-17T05:49:44.916 INFO:teuthology.orchestra.run.smithi098.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-04-17T05:49:44.939 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-17T05:49:44.942 INFO:teuthology.orchestra.run.smithi098.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-04-17T05:49:44.942 INFO:teuthology.orchestra.run.smithi098.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-17T05:49:44.974 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-17T05:49:44.993 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-17T05:49:45.030 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-17T05:49:45.059 INFO:teuthology.orchestra.run.smithi098.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-04-17T05:49:45.059 INFO:teuthology.orchestra.run.smithi098.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-17T05:49:45.136 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-17T05:49:45.162 INFO:teuthology.orchestra.run.smithi098.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-04-17T05:49:45.196 INFO:teuthology.orchestra.run.smithi098.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-04-17T05:49:45.221 INFO:teuthology.orchestra.run.smithi098.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-04-17T05:49:45.246 INFO:teuthology.orchestra.run.smithi098.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-04-17T05:49:45.278 INFO:teuthology.orchestra.run.smithi098.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-04-17T05:49:45.327 INFO:teuthology.orchestra.run.smithi098.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-04-17T05:49:45.359 INFO:teuthology.orchestra.run.smithi098.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-04-17T05:49:45.384 INFO:teuthology.orchestra.run.smithi098.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-04-17T05:49:45.410 INFO:teuthology.orchestra.run.smithi098.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-04-17T05:49:45.435 INFO:teuthology.orchestra.run.smithi098.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-04-17T05:49:45.452 INFO:teuthology.orchestra.run.smithi098.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-04-17T05:49:45.459 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-17T05:49:45.459 INFO:teuthology.orchestra.run.smithi098.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-17T05:49:45.630 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-17T05:49:45.630 INFO:teuthology.orchestra.run.smithi098.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-17T05:49:45.697 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-17T05:49:45.728 INFO:teuthology.orchestra.run.smithi098.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-04-17T05:49:45.755 INFO:teuthology.orchestra.run.smithi098.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-04-17T05:49:45.781 INFO:teuthology.orchestra.run.smithi098.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-04-17T05:49:45.806 INFO:teuthology.orchestra.run.smithi098.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-04-17T05:49:45.825 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : cephfs-top-2:19.0.0-3133.ga8c84f59.el9.noarch 170/198 2024-04-17T05:49:45.870 INFO:teuthology.orchestra.run.smithi098.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-17T05:49:45.904 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-04-17T05:49:45.935 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-04-17T05:49:46.025 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-04-17T05:49:46.066 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-04-17T05:49:46.091 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-04-17T05:49:46.117 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-04-17T05:49:46.143 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-04-17T05:49:46.143 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-17T05:49:46.231 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-17T05:49:46.261 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-04-17T05:49:46.261 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-17T05:49:46.346 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-17T05:49:46.372 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-04-17T05:49:46.405 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-04-17T05:49:46.431 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-04-17T05:49:46.455 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-04-17T05:49:46.489 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-04-17T05:49:46.537 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-04-17T05:49:46.577 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-04-17T05:49:46.603 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-04-17T05:49:46.628 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-04-17T05:49:46.653 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-04-17T05:49:46.679 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-04-17T05:49:46.685 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-17T05:49:46.685 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-17T05:49:46.873 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-17T05:49:46.874 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-17T05:49:46.948 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-17T05:49:46.988 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-04-17T05:49:47.006 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-04-17T05:49:47.033 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-04-17T05:49:47.057 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-04-17T05:49:47.121 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-17T05:49:49.311 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-test-2:19.0.0-3133.ga8c84f59.el9.x86_64 153/198 2024-04-17T05:49:49.352 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : rbd-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 154/198 2024-04-17T05:49:49.382 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : rbd-nbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 155/198 2024-04-17T05:49:49.410 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-04-17T05:49:49.609 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : cephfs-shell-2:19.0.0-3133.ga8c84f59.el9.x86_64 157/198 2024-04-17T05:49:49.730 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 158/198 2024-04-17T05:49:49.775 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 159/198 2024-04-17T05:49:49.824 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-04-17T05:49:49.891 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-04-17T05:49:49.934 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-04-17T05:49:49.958 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libcephfs-devel-2:19.0.0-3133.ga8c84f59.el9.x86_ 163/198 2024-04-17T05:49:49.990 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-04-17T05:49:50.020 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : NetworkManager-team-1:1.47.4-1.el9.x86_64 165/198 2024-04-17T05:49:50.085 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : NetworkManager-tui-1:1.47.4-1.el9.x86_64 166/198 2024-04-17T05:49:50.152 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-04-17T05:49:50.193 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-04-17T05:49:50.851 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-17T05:49:50.892 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-17T05:49:50.912 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-17T05:49:51.367 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-17T05:49:51.368 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-2:19.0.0-3133.ga8c84f59.el9.x86_64 1/198 2024-04-17T05:49:51.368 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 2/198 2024-04-17T05:49:51.368 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 3/198 2024-04-17T05:49:51.368 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 4/198 2024-04-17T05:49:51.368 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3133.ga8c84 5/198 2024-04-17T05:49:51.368 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 6/198 2024-04-17T05:49:51.368 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 7/198 2024-04-17T05:49:51.368 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 8/198 2024-04-17T05:49:51.368 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 9/198 2024-04-17T05:49:51.368 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 10/198 2024-04-17T05:49:51.368 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 11/198 2024-04-17T05:49:51.369 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-test-2:19.0.0-3133.ga8c84f59.el9.x86_64 12/198 2024-04-17T05:49:51.369 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 13/198 2024-04-17T05:49:51.369 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : cephfs-shell-2:19.0.0-3133.ga8c84f59.el9.x86_64 14/198 2024-04-17T05:49:51.369 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libcephfs-devel-2:19.0.0-3133.ga8c84f59.el9.x86_ 15/198 2024-04-17T05:49:51.369 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 16/198 2024-04-17T05:49:51.369 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 17/198 2024-04-17T05:49:51.369 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : librados-devel-2:19.0.0-3133.ga8c84f59.el9.x86_6 18/198 2024-04-17T05:49:51.369 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86 19/198 2024-04-17T05:49:51.369 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 20/198 2024-04-17T05:49:51.369 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3133.ga8c84f59.el 21/198 2024-04-17T05:49:51.369 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-ceph-common-2:19.0.0-3133.ga8c84f59.el9. 22/198 2024-04-17T05:49:51.369 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-cephfs-2:19.0.0-3133.ga8c84f59.el9.x86_6 23/198 2024-04-17T05:49:51.369 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-rados-2:19.0.0-3133.ga8c84f59.el9.x86_64 24/198 2024-04-17T05:49:51.369 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-rbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 25/198 2024-04-17T05:49:51.370 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-rgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 26/198 2024-04-17T05:49:51.370 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : rbd-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 27/198 2024-04-17T05:49:51.370 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 28/198 2024-04-17T05:49:51.370 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : rbd-nbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 29/198 2024-04-17T05:49:51.389 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3133.ga8c84f59. 30/198 2024-04-17T05:49:51.389 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noa 31/198 2024-04-17T05:49:51.390 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.n 32/198 2024-04-17T05:49:51.390 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c 33/198 2024-04-17T05:49:51.390 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3133.ga8c84f59.el 34/198 2024-04-17T05:49:51.390 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 35/198 2024-04-17T05:49:51.390 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3133.ga8c84f59.e 36/198 2024-04-17T05:49:51.390 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 37/198 2024-04-17T05:49:51.390 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 38/198 2024-04-17T05:49:51.390 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : cephfs-top-2:19.0.0-3133.ga8c84f59.el9.noarch 39/198 2024-04-17T05:49:51.390 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-04-17T05:49:51.390 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-04-17T05:49:51.391 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-04-17T05:49:51.391 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-04-17T05:49:51.391 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-04-17T05:49:51.391 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-04-17T05:49:51.391 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-04-17T05:49:51.391 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-04-17T05:49:51.391 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-04-17T05:49:51.391 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-04-17T05:49:51.391 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-04-17T05:49:51.391 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-04-17T05:49:51.392 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-04-17T05:49:51.392 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-04-17T05:49:51.392 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-04-17T05:49:51.392 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-04-17T05:49:51.392 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-04-17T05:49:51.392 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-04-17T05:49:51.392 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : postgresql-13.14-1.el9.x86_64 58/198 2024-04-17T05:49:51.392 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 59/198 2024-04-17T05:49:51.392 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 60/198 2024-04-17T05:49:51.392 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 61/198 2024-04-17T05:49:51.392 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-04-17T05:49:51.393 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-04-17T05:49:51.393 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-04-17T05:49:51.393 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-04-17T05:49:51.393 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-04-17T05:49:51.393 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-04-17T05:49:51.393 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-04-17T05:49:51.393 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-04-17T05:49:51.393 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-04-17T05:49:51.393 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-04-17T05:49:51.393 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-04-17T05:49:51.393 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-04-17T05:49:51.394 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-04-17T05:49:51.394 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-04-17T05:49:51.394 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-04-17T05:49:51.394 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-04-17T05:49:51.394 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-04-17T05:49:51.394 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-04-17T05:49:51.394 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-04-17T05:49:51.394 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-04-17T05:49:51.394 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 82/198 2024-04-17T05:49:51.394 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 83/198 2024-04-17T05:49:51.395 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 84/198 2024-04-17T05:49:51.395 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 85/198 2024-04-17T05:49:51.395 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 86/198 2024-04-17T05:49:51.395 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 87/198 2024-04-17T05:49:51.395 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 88/198 2024-04-17T05:49:51.395 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 89/198 2024-04-17T05:49:51.395 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 90/198 2024-04-17T05:49:51.395 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 91/198 2024-04-17T05:49:51.395 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-ply-3.11-14.el9.noarch 92/198 2024-04-17T05:49:51.395 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 93/198 2024-04-17T05:49:51.395 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 94/198 2024-04-17T05:49:51.395 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : jq-1.6-16.el9.x86_64 95/198 2024-04-17T05:49:51.396 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 96/198 2024-04-17T05:49:51.396 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 97/198 2024-04-17T05:49:51.396 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 98/198 2024-04-17T05:49:51.396 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 99/198 2024-04-17T05:49:51.396 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 100/198 2024-04-17T05:49:51.396 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 101/198 2024-04-17T05:49:51.396 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 102/198 2024-04-17T05:49:51.396 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 103/198 2024-04-17T05:49:51.396 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-04-17T05:49:51.396 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-04-17T05:49:51.396 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-04-17T05:49:51.396 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-04-17T05:49:51.397 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-04-17T05:49:51.397 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-04-17T05:49:51.397 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-04-17T05:49:51.397 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-04-17T05:49:51.397 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-04-17T05:49:51.397 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-04-17T05:49:51.397 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-04-17T05:49:51.397 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-04-17T05:49:51.397 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-04-17T05:49:51.397 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-04-17T05:49:51.397 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-04-17T05:49:51.398 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-04-17T05:49:51.398 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-04-17T05:49:51.398 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-04-17T05:49:51.398 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-04-17T05:49:51.398 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-04-17T05:49:51.398 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-04-17T05:49:51.398 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-04-17T05:49:51.398 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-04-17T05:49:51.398 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-04-17T05:49:51.398 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-04-17T05:49:51.398 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-04-17T05:49:51.398 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-04-17T05:49:51.399 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-04-17T05:49:51.399 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-04-17T05:49:51.399 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-04-17T05:49:51.399 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-04-17T05:49:51.399 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-04-17T05:49:51.399 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-04-17T05:49:51.399 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-04-17T05:49:51.399 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-04-17T05:49:51.399 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-04-17T05:49:51.399 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-04-17T05:49:51.399 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-04-17T05:49:51.400 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-04-17T05:49:51.400 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-04-17T05:49:51.400 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-04-17T05:49:51.400 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-04-17T05:49:51.400 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-04-17T05:49:51.400 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-04-17T05:49:51.400 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-04-17T05:49:51.400 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-04-17T05:49:51.400 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-04-17T05:49:51.400 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 151/198 2024-04-17T05:49:51.400 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-04-17T05:49:51.400 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 153/198 2024-04-17T05:49:51.401 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-04-17T05:49:51.401 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 155/198 2024-04-17T05:49:51.401 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 156/198 2024-04-17T05:49:51.401 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 157/198 2024-04-17T05:49:51.401 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/198 2024-04-17T05:49:51.401 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 159/198 2024-04-17T05:49:51.401 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 160/198 2024-04-17T05:49:51.401 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 161/198 2024-04-17T05:49:51.401 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 162/198 2024-04-17T05:49:51.401 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 163/198 2024-04-17T05:49:51.401 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 164/198 2024-04-17T05:49:51.402 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-04-17T05:49:51.402 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-04-17T05:49:51.402 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-04-17T05:49:51.402 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-04-17T05:49:51.402 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-04-17T05:49:51.402 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-04-17T05:49:51.402 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-04-17T05:49:51.402 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-04-17T05:49:51.402 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-04-17T05:49:51.402 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-04-17T05:49:51.402 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-04-17T05:49:51.403 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-04-17T05:49:51.403 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-04-17T05:49:51.403 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-04-17T05:49:51.403 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-04-17T05:49:51.403 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-04-17T05:49:51.403 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-04-17T05:49:51.403 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-04-17T05:49:51.403 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-04-17T05:49:51.403 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-04-17T05:49:51.403 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-04-17T05:49:51.403 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-04-17T05:49:51.403 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-04-17T05:49:51.404 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-04-17T05:49:51.404 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-04-17T05:49:51.404 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-04-17T05:49:51.404 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-04-17T05:49:51.404 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-04-17T05:49:51.404 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-04-17T05:49:51.404 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-04-17T05:49:51.404 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-04-17T05:49:51.404 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-04-17T05:49:51.404 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-04-17T05:49:51.784 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : cephfs-top-2:19.0.0-3133.ga8c84f59.el9.noarch 170/198 2024-04-17T05:49:51.853 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-04-17T05:49:51.895 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-04-17T05:49:51.985 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-04-17T05:49:52.025 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-04-17T05:49:52.049 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-04-17T05:49:52.091 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-04-17T05:49:52.117 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-04-17T05:49:52.117 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-17T05:49:52.222 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-17T05:49:52.251 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-04-17T05:49:52.252 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-17T05:49:52.336 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-17T05:49:52.362 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-04-17T05:49:52.404 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-04-17T05:49:52.429 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-04-17T05:49:52.454 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-04-17T05:49:52.489 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-04-17T05:49:52.527 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-04-17T05:49:52.559 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-04-17T05:49:52.584 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-04-17T05:49:52.609 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-04-17T05:49:52.643 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-04-17T05:49:52.668 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-04-17T05:49:52.675 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-17T05:49:52.675 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-17T05:49:52.855 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-17T05:49:52.855 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-17T05:49:52.896 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-17T05:49:52.896 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-2:19.0.0-3133.ga8c84f59.el9.x86_64 1/198 2024-04-17T05:49:52.896 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 2/198 2024-04-17T05:49:52.896 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 3/198 2024-04-17T05:49:52.896 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 4/198 2024-04-17T05:49:52.896 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3133.ga8c84 5/198 2024-04-17T05:49:52.897 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 6/198 2024-04-17T05:49:52.897 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 7/198 2024-04-17T05:49:52.897 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 8/198 2024-04-17T05:49:52.897 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 9/198 2024-04-17T05:49:52.897 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 10/198 2024-04-17T05:49:52.897 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 11/198 2024-04-17T05:49:52.897 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-test-2:19.0.0-3133.ga8c84f59.el9.x86_64 12/198 2024-04-17T05:49:52.897 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 13/198 2024-04-17T05:49:52.897 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : cephfs-shell-2:19.0.0-3133.ga8c84f59.el9.x86_64 14/198 2024-04-17T05:49:52.897 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libcephfs-devel-2:19.0.0-3133.ga8c84f59.el9.x86_ 15/198 2024-04-17T05:49:52.897 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 16/198 2024-04-17T05:49:52.897 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 17/198 2024-04-17T05:49:52.897 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librados-devel-2:19.0.0-3133.ga8c84f59.el9.x86_6 18/198 2024-04-17T05:49:52.897 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86 19/198 2024-04-17T05:49:52.897 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 20/198 2024-04-17T05:49:52.898 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3133.ga8c84f59.el 21/198 2024-04-17T05:49:52.898 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-ceph-common-2:19.0.0-3133.ga8c84f59.el9. 22/198 2024-04-17T05:49:52.898 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-cephfs-2:19.0.0-3133.ga8c84f59.el9.x86_6 23/198 2024-04-17T05:49:52.898 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-rados-2:19.0.0-3133.ga8c84f59.el9.x86_64 24/198 2024-04-17T05:49:52.898 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-rbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 25/198 2024-04-17T05:49:52.898 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-rgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 26/198 2024-04-17T05:49:52.913 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : rbd-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 27/198 2024-04-17T05:49:52.914 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 28/198 2024-04-17T05:49:52.914 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : rbd-nbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 29/198 2024-04-17T05:49:52.914 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3133.ga8c84f59. 30/198 2024-04-17T05:49:52.914 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noa 31/198 2024-04-17T05:49:52.914 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.n 32/198 2024-04-17T05:49:52.914 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c 33/198 2024-04-17T05:49:52.914 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3133.ga8c84f59.el 34/198 2024-04-17T05:49:52.914 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 35/198 2024-04-17T05:49:52.914 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3133.ga8c84f59.e 36/198 2024-04-17T05:49:52.914 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 37/198 2024-04-17T05:49:52.914 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 38/198 2024-04-17T05:49:52.914 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : cephfs-top-2:19.0.0-3133.ga8c84f59.el9.noarch 39/198 2024-04-17T05:49:52.914 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-04-17T05:49:52.915 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-04-17T05:49:52.915 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-04-17T05:49:52.915 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-04-17T05:49:52.915 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-04-17T05:49:52.915 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-04-17T05:49:52.915 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-04-17T05:49:52.915 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-04-17T05:49:52.915 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-04-17T05:49:52.915 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-04-17T05:49:52.915 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-04-17T05:49:52.915 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-04-17T05:49:52.915 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-04-17T05:49:52.915 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-04-17T05:49:52.915 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-04-17T05:49:52.915 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-04-17T05:49:52.916 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-04-17T05:49:52.916 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-04-17T05:49:52.916 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : postgresql-13.14-1.el9.x86_64 58/198 2024-04-17T05:49:52.916 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 59/198 2024-04-17T05:49:52.916 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 60/198 2024-04-17T05:49:52.916 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 61/198 2024-04-17T05:49:52.916 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-04-17T05:49:52.916 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-04-17T05:49:52.916 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-04-17T05:49:52.916 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-04-17T05:49:52.916 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-04-17T05:49:52.916 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-04-17T05:49:52.916 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-04-17T05:49:52.916 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-04-17T05:49:52.917 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-04-17T05:49:52.917 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-04-17T05:49:52.917 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-04-17T05:49:52.917 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-04-17T05:49:52.917 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-04-17T05:49:52.917 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-04-17T05:49:52.917 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-04-17T05:49:52.917 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-04-17T05:49:52.917 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-04-17T05:49:52.917 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-04-17T05:49:52.917 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-04-17T05:49:52.917 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-04-17T05:49:52.917 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 82/198 2024-04-17T05:49:52.917 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 83/198 2024-04-17T05:49:52.918 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 84/198 2024-04-17T05:49:52.918 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 85/198 2024-04-17T05:49:52.918 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 86/198 2024-04-17T05:49:52.918 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 87/198 2024-04-17T05:49:52.918 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 88/198 2024-04-17T05:49:52.918 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 89/198 2024-04-17T05:49:52.918 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 90/198 2024-04-17T05:49:52.918 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 91/198 2024-04-17T05:49:52.918 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-ply-3.11-14.el9.noarch 92/198 2024-04-17T05:49:52.918 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 93/198 2024-04-17T05:49:52.918 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 94/198 2024-04-17T05:49:52.918 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : jq-1.6-16.el9.x86_64 95/198 2024-04-17T05:49:52.918 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 96/198 2024-04-17T05:49:52.918 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 97/198 2024-04-17T05:49:52.919 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 98/198 2024-04-17T05:49:52.919 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 99/198 2024-04-17T05:49:52.919 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 100/198 2024-04-17T05:49:52.919 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 101/198 2024-04-17T05:49:52.919 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 102/198 2024-04-17T05:49:52.919 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 103/198 2024-04-17T05:49:52.919 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-04-17T05:49:52.919 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-04-17T05:49:52.919 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-04-17T05:49:52.919 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-04-17T05:49:52.919 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-04-17T05:49:52.919 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-04-17T05:49:52.919 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-04-17T05:49:52.919 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-04-17T05:49:52.919 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-04-17T05:49:52.920 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-04-17T05:49:52.920 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-04-17T05:49:52.920 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-04-17T05:49:52.920 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-04-17T05:49:52.920 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-04-17T05:49:52.920 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-04-17T05:49:52.920 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-04-17T05:49:52.920 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-04-17T05:49:52.920 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-04-17T05:49:52.920 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-04-17T05:49:52.920 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-04-17T05:49:52.920 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-04-17T05:49:52.920 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-04-17T05:49:52.920 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-04-17T05:49:52.920 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-04-17T05:49:52.921 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-04-17T05:49:52.921 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-04-17T05:49:52.921 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-04-17T05:49:52.921 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-04-17T05:49:52.921 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-04-17T05:49:52.921 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-04-17T05:49:52.921 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-04-17T05:49:52.921 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-04-17T05:49:52.921 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-04-17T05:49:52.921 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-04-17T05:49:52.921 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-04-17T05:49:52.921 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-04-17T05:49:52.921 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-04-17T05:49:52.921 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-04-17T05:49:52.921 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-04-17T05:49:52.922 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-04-17T05:49:52.922 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-04-17T05:49:52.922 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-04-17T05:49:52.922 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-04-17T05:49:52.922 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-04-17T05:49:52.922 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-04-17T05:49:52.922 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-04-17T05:49:52.922 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-04-17T05:49:52.922 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 151/198 2024-04-17T05:49:52.922 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-04-17T05:49:52.922 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 153/198 2024-04-17T05:49:52.922 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-04-17T05:49:52.922 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 155/198 2024-04-17T05:49:52.923 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 156/198 2024-04-17T05:49:52.923 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 157/198 2024-04-17T05:49:52.923 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/198 2024-04-17T05:49:52.923 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 159/198 2024-04-17T05:49:52.923 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 160/198 2024-04-17T05:49:52.923 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 161/198 2024-04-17T05:49:52.923 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 162/198 2024-04-17T05:49:52.923 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 163/198 2024-04-17T05:49:52.923 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 164/198 2024-04-17T05:49:52.923 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-04-17T05:49:52.923 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-04-17T05:49:52.923 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-04-17T05:49:52.923 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-04-17T05:49:52.923 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-04-17T05:49:52.924 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-04-17T05:49:52.924 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-04-17T05:49:52.924 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-04-17T05:49:52.924 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-04-17T05:49:52.924 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-04-17T05:49:52.924 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-04-17T05:49:52.924 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-04-17T05:49:52.924 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-04-17T05:49:52.924 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-04-17T05:49:52.924 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-04-17T05:49:52.924 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-04-17T05:49:52.924 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-04-17T05:49:52.924 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-04-17T05:49:52.924 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-04-17T05:49:52.924 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-04-17T05:49:52.925 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-04-17T05:49:52.925 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-04-17T05:49:52.925 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-04-17T05:49:52.925 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-04-17T05:49:52.925 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-04-17T05:49:52.925 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-04-17T05:49:52.925 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-04-17T05:49:52.925 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-04-17T05:49:52.925 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-04-17T05:49:52.925 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-04-17T05:49:52.925 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-04-17T05:49:52.925 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-04-17T05:49:52.925 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-04-17T05:49:52.937 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-17T05:49:52.969 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-04-17T05:49:52.996 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-04-17T05:49:53.022 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-04-17T05:49:53.047 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-04-17T05:49:53.115 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-17T05:49:54.512 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-04-17T05:49:54.513 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-04-17T05:49:54.513 INFO:teuthology.orchestra.run.smithi098.stdout:Upgraded: 2024-04-17T05:49:54.513 INFO:teuthology.orchestra.run.smithi098.stdout: NetworkManager-1:1.47.4-1.el9.x86_64 2024-04-17T05:49:54.513 INFO:teuthology.orchestra.run.smithi098.stdout: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 2024-04-17T05:49:54.513 INFO:teuthology.orchestra.run.smithi098.stdout: NetworkManager-team-1:1.47.4-1.el9.x86_64 2024-04-17T05:49:54.513 INFO:teuthology.orchestra.run.smithi098.stdout: NetworkManager-tui-1:1.47.4-1.el9.x86_64 2024-04-17T05:49:54.513 INFO:teuthology.orchestra.run.smithi098.stdout: acl-2.3.1-4.el9.x86_64 2024-04-17T05:49:54.513 INFO:teuthology.orchestra.run.smithi098.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-04-17T05:49:54.513 INFO:teuthology.orchestra.run.smithi098.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-04-17T05:49:54.513 INFO:teuthology.orchestra.run.smithi098.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-04-17T05:49:54.513 INFO:teuthology.orchestra.run.smithi098.stdout: libacl-2.3.1-4.el9.x86_64 2024-04-17T05:49:54.514 INFO:teuthology.orchestra.run.smithi098.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-04-17T05:49:54.514 INFO:teuthology.orchestra.run.smithi098.stdout: libblkid-2.37.4-18.el9.x86_64 2024-04-17T05:49:54.514 INFO:teuthology.orchestra.run.smithi098.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-04-17T05:49:54.514 INFO:teuthology.orchestra.run.smithi098.stdout: libmount-2.37.4-18.el9.x86_64 2024-04-17T05:49:54.514 INFO:teuthology.orchestra.run.smithi098.stdout: librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T05:49:54.514 INFO:teuthology.orchestra.run.smithi098.stdout: librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T05:49:54.514 INFO:teuthology.orchestra.run.smithi098.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-04-17T05:49:54.514 INFO:teuthology.orchestra.run.smithi098.stdout: libuuid-2.37.4-18.el9.x86_64 2024-04-17T05:49:54.514 INFO:teuthology.orchestra.run.smithi098.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-04-17T05:49:54.514 INFO:teuthology.orchestra.run.smithi098.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-04-17T05:49:54.514 INFO:teuthology.orchestra.run.smithi098.stdout: util-linux-2.37.4-18.el9.x86_64 2024-04-17T05:49:54.514 INFO:teuthology.orchestra.run.smithi098.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-04-17T05:49:54.514 INFO:teuthology.orchestra.run.smithi098.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-04-17T05:49:54.515 INFO:teuthology.orchestra.run.smithi098.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-04-17T05:49:54.515 INFO:teuthology.orchestra.run.smithi098.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-04-17T05:49:54.515 INFO:teuthology.orchestra.run.smithi098.stdout:Installed: 2024-04-17T05:49:54.515 INFO:teuthology.orchestra.run.smithi098.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-17T05:49:54.515 INFO:teuthology.orchestra.run.smithi098.stdout: automake-1.16.2-8.el9.noarch 2024-04-17T05:49:54.515 INFO:teuthology.orchestra.run.smithi098.stdout: bison-3.7.4-5.el9.x86_64 2024-04-17T05:49:54.515 INFO:teuthology.orchestra.run.smithi098.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-17T05:49:54.515 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T05:49:54.515 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T05:49:54.515 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T05:49:54.515 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T05:49:54.516 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-grafana-dashboards-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-17T05:49:54.516 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-immutable-object-cache-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T05:49:54.516 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T05:49:54.516 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T05:49:54.516 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-17T05:49:54.516 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-17T05:49:54.516 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-17T05:49:54.516 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-modules-core-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-17T05:49:54.516 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-17T05:49:54.516 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T05:49:54.516 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T05:49:54.516 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-prometheus-alerts-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-17T05:49:54.516 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T05:49:54.517 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T05:49:54.517 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-test-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T05:49:54.517 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-17T05:49:54.517 INFO:teuthology.orchestra.run.smithi098.stdout: cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-17T05:49:54.517 INFO:teuthology.orchestra.run.smithi098.stdout: cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T05:49:54.517 INFO:teuthology.orchestra.run.smithi098.stdout: cephfs-shell-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T05:49:54.517 INFO:teuthology.orchestra.run.smithi098.stdout: cephfs-top-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-17T05:49:54.517 INFO:teuthology.orchestra.run.smithi098.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-17T05:49:54.517 INFO:teuthology.orchestra.run.smithi098.stdout: flex-2.6.4-9.el9.x86_64 2024-04-17T05:49:54.517 INFO:teuthology.orchestra.run.smithi098.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-17T05:49:54.517 INFO:teuthology.orchestra.run.smithi098.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-17T05:49:54.517 INFO:teuthology.orchestra.run.smithi098.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-17T05:49:54.517 INFO:teuthology.orchestra.run.smithi098.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-17T05:49:54.518 INFO:teuthology.orchestra.run.smithi098.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-17T05:49:54.518 INFO:teuthology.orchestra.run.smithi098.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-17T05:49:54.518 INFO:teuthology.orchestra.run.smithi098.stdout: jq-1.6-16.el9.x86_64 2024-04-17T05:49:54.518 INFO:teuthology.orchestra.run.smithi098.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-17T05:49:54.518 INFO:teuthology.orchestra.run.smithi098.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-17T05:49:54.518 INFO:teuthology.orchestra.run.smithi098.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-17T05:49:54.518 INFO:teuthology.orchestra.run.smithi098.stdout: libcephfs-devel-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T05:49:54.518 INFO:teuthology.orchestra.run.smithi098.stdout: libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T05:49:54.518 INFO:teuthology.orchestra.run.smithi098.stdout: libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T05:49:54.518 INFO:teuthology.orchestra.run.smithi098.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-17T05:49:54.518 INFO:teuthology.orchestra.run.smithi098.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-17T05:49:54.518 INFO:teuthology.orchestra.run.smithi098.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-17T05:49:54.518 INFO:teuthology.orchestra.run.smithi098.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-17T05:49:54.518 INFO:teuthology.orchestra.run.smithi098.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-17T05:49:54.519 INFO:teuthology.orchestra.run.smithi098.stdout: librados-devel-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T05:49:54.519 INFO:teuthology.orchestra.run.smithi098.stdout: libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T05:49:54.519 INFO:teuthology.orchestra.run.smithi098.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-17T05:49:54.519 INFO:teuthology.orchestra.run.smithi098.stdout: librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T05:49:54.519 INFO:teuthology.orchestra.run.smithi098.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-17T05:49:54.519 INFO:teuthology.orchestra.run.smithi098.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-17T05:49:54.519 INFO:teuthology.orchestra.run.smithi098.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-17T05:49:54.519 INFO:teuthology.orchestra.run.smithi098.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-17T05:49:54.519 INFO:teuthology.orchestra.run.smithi098.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-17T05:49:54.519 INFO:teuthology.orchestra.run.smithi098.stdout: lua-5.4.4-4.el9.x86_64 2024-04-17T05:49:54.519 INFO:teuthology.orchestra.run.smithi098.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-17T05:49:54.519 INFO:teuthology.orchestra.run.smithi098.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-17T05:49:54.519 INFO:teuthology.orchestra.run.smithi098.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-04-17T05:49:54.520 INFO:teuthology.orchestra.run.smithi098.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-17T05:49:54.520 INFO:teuthology.orchestra.run.smithi098.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-17T05:49:54.520 INFO:teuthology.orchestra.run.smithi098.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-17T05:49:54.520 INFO:teuthology.orchestra.run.smithi098.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-17T05:49:54.520 INFO:teuthology.orchestra.run.smithi098.stdout: postgresql-13.14-1.el9.x86_64 2024-04-17T05:49:54.520 INFO:teuthology.orchestra.run.smithi098.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-17T05:49:54.520 INFO:teuthology.orchestra.run.smithi098.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-17T05:49:54.520 INFO:teuthology.orchestra.run.smithi098.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-17T05:49:54.520 INFO:teuthology.orchestra.run.smithi098.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-17T05:49:54.520 INFO:teuthology.orchestra.run.smithi098.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-17T05:49:54.520 INFO:teuthology.orchestra.run.smithi098.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-17T05:49:54.520 INFO:teuthology.orchestra.run.smithi098.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-17T05:49:54.520 INFO:teuthology.orchestra.run.smithi098.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-17T05:49:54.520 INFO:teuthology.orchestra.run.smithi098.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-17T05:49:54.521 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-17T05:49:54.521 INFO:teuthology.orchestra.run.smithi098.stdout: python3-ceph-argparse-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T05:49:54.521 INFO:teuthology.orchestra.run.smithi098.stdout: python3-ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T05:49:54.521 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cephfs-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T05:49:54.521 INFO:teuthology.orchestra.run.smithi098.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-17T05:49:54.521 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-17T05:49:54.521 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-17T05:49:54.521 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-17T05:49:54.521 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-17T05:49:54.521 INFO:teuthology.orchestra.run.smithi098.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-17T05:49:54.521 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-17T05:49:54.521 INFO:teuthology.orchestra.run.smithi098.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-04-17T05:49:54.521 INFO:teuthology.orchestra.run.smithi098.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-17T05:49:54.522 INFO:teuthology.orchestra.run.smithi098.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-17T05:49:54.522 INFO:teuthology.orchestra.run.smithi098.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-17T05:49:54.522 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-17T05:49:54.522 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-17T05:49:54.522 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-17T05:49:54.522 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-17T05:49:54.522 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-17T05:49:54.522 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-17T05:49:54.522 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-17T05:49:54.522 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-17T05:49:54.522 INFO:teuthology.orchestra.run.smithi098.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-17T05:49:54.522 INFO:teuthology.orchestra.run.smithi098.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-17T05:49:54.523 INFO:teuthology.orchestra.run.smithi098.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-17T05:49:54.523 INFO:teuthology.orchestra.run.smithi098.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-17T05:49:54.523 INFO:teuthology.orchestra.run.smithi098.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-17T05:49:54.523 INFO:teuthology.orchestra.run.smithi098.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-17T05:49:54.523 INFO:teuthology.orchestra.run.smithi098.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-17T05:49:54.523 INFO:teuthology.orchestra.run.smithi098.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-17T05:49:54.523 INFO:teuthology.orchestra.run.smithi098.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-17T05:49:54.523 INFO:teuthology.orchestra.run.smithi098.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-17T05:49:54.523 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-17T05:49:54.523 INFO:teuthology.orchestra.run.smithi098.stdout: python3-ply-3.11-14.el9.noarch 2024-04-17T05:49:54.523 INFO:teuthology.orchestra.run.smithi098.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-17T05:49:54.523 INFO:teuthology.orchestra.run.smithi098.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-17T05:49:54.523 INFO:teuthology.orchestra.run.smithi098.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-17T05:49:54.523 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-17T05:49:54.523 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-17T05:49:54.524 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-17T05:49:54.524 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-17T05:49:54.524 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-17T05:49:54.524 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-17T05:49:54.524 INFO:teuthology.orchestra.run.smithi098.stdout: python3-rados-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T05:49:54.524 INFO:teuthology.orchestra.run.smithi098.stdout: python3-rbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T05:49:54.524 INFO:teuthology.orchestra.run.smithi098.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-17T05:49:54.524 INFO:teuthology.orchestra.run.smithi098.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-17T05:49:54.524 INFO:teuthology.orchestra.run.smithi098.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-17T05:49:54.524 INFO:teuthology.orchestra.run.smithi098.stdout: python3-rgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T05:49:54.524 INFO:teuthology.orchestra.run.smithi098.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-17T05:49:54.524 INFO:teuthology.orchestra.run.smithi098.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-17T05:49:54.524 INFO:teuthology.orchestra.run.smithi098.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-17T05:49:54.525 INFO:teuthology.orchestra.run.smithi098.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-17T05:49:54.525 INFO:teuthology.orchestra.run.smithi098.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-17T05:49:54.525 INFO:teuthology.orchestra.run.smithi098.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-17T05:49:54.525 INFO:teuthology.orchestra.run.smithi098.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-17T05:49:54.525 INFO:teuthology.orchestra.run.smithi098.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-17T05:49:54.525 INFO:teuthology.orchestra.run.smithi098.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-17T05:49:54.525 INFO:teuthology.orchestra.run.smithi098.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-17T05:49:54.525 INFO:teuthology.orchestra.run.smithi098.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-17T05:49:54.525 INFO:teuthology.orchestra.run.smithi098.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-17T05:49:54.525 INFO:teuthology.orchestra.run.smithi098.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-17T05:49:54.525 INFO:teuthology.orchestra.run.smithi098.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-17T05:49:54.525 INFO:teuthology.orchestra.run.smithi098.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-17T05:49:54.525 INFO:teuthology.orchestra.run.smithi098.stdout: rbd-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T05:49:54.525 INFO:teuthology.orchestra.run.smithi098.stdout: rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T05:49:54.526 INFO:teuthology.orchestra.run.smithi098.stdout: rbd-nbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T05:49:54.526 INFO:teuthology.orchestra.run.smithi098.stdout: re2-1:20211101-3.el9.x86_64 2024-04-17T05:49:54.526 INFO:teuthology.orchestra.run.smithi098.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-17T05:49:54.526 INFO:teuthology.orchestra.run.smithi098.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-17T05:49:54.526 INFO:teuthology.orchestra.run.smithi098.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-17T05:49:54.526 INFO:teuthology.orchestra.run.smithi098.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-17T05:49:54.526 INFO:teuthology.orchestra.run.smithi098.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-17T05:49:54.527 INFO:teuthology.orchestra.run.smithi098.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-04-17T05:49:54.527 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-04-17T05:49:54.527 INFO:teuthology.orchestra.run.smithi098.stdout:Complete! 2024-04-17T05:49:54.971 DEBUG:teuthology.parallel:result is None 2024-04-17T05:49:56.083 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-04-17T05:49:56.083 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-04-17T05:49:56.083 INFO:teuthology.orchestra.run.smithi103.stdout:Upgraded: 2024-04-17T05:49:56.083 INFO:teuthology.orchestra.run.smithi103.stdout: NetworkManager-1:1.47.4-1.el9.x86_64 2024-04-17T05:49:56.083 INFO:teuthology.orchestra.run.smithi103.stdout: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 2024-04-17T05:49:56.083 INFO:teuthology.orchestra.run.smithi103.stdout: NetworkManager-team-1:1.47.4-1.el9.x86_64 2024-04-17T05:49:56.084 INFO:teuthology.orchestra.run.smithi103.stdout: NetworkManager-tui-1:1.47.4-1.el9.x86_64 2024-04-17T05:49:56.084 INFO:teuthology.orchestra.run.smithi103.stdout: acl-2.3.1-4.el9.x86_64 2024-04-17T05:49:56.084 INFO:teuthology.orchestra.run.smithi103.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-04-17T05:49:56.084 INFO:teuthology.orchestra.run.smithi103.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-04-17T05:49:56.084 INFO:teuthology.orchestra.run.smithi103.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-04-17T05:49:56.084 INFO:teuthology.orchestra.run.smithi103.stdout: libacl-2.3.1-4.el9.x86_64 2024-04-17T05:49:56.084 INFO:teuthology.orchestra.run.smithi103.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-04-17T05:49:56.084 INFO:teuthology.orchestra.run.smithi103.stdout: libblkid-2.37.4-18.el9.x86_64 2024-04-17T05:49:56.084 INFO:teuthology.orchestra.run.smithi103.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-04-17T05:49:56.084 INFO:teuthology.orchestra.run.smithi103.stdout: libmount-2.37.4-18.el9.x86_64 2024-04-17T05:49:56.085 INFO:teuthology.orchestra.run.smithi103.stdout: librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T05:49:56.085 INFO:teuthology.orchestra.run.smithi103.stdout: librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T05:49:56.085 INFO:teuthology.orchestra.run.smithi103.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-04-17T05:49:56.085 INFO:teuthology.orchestra.run.smithi103.stdout: libuuid-2.37.4-18.el9.x86_64 2024-04-17T05:49:56.085 INFO:teuthology.orchestra.run.smithi103.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-04-17T05:49:56.085 INFO:teuthology.orchestra.run.smithi103.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-04-17T05:49:56.085 INFO:teuthology.orchestra.run.smithi103.stdout: util-linux-2.37.4-18.el9.x86_64 2024-04-17T05:49:56.085 INFO:teuthology.orchestra.run.smithi103.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-04-17T05:49:56.085 INFO:teuthology.orchestra.run.smithi103.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-04-17T05:49:56.085 INFO:teuthology.orchestra.run.smithi103.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-04-17T05:49:56.085 INFO:teuthology.orchestra.run.smithi103.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-04-17T05:49:56.085 INFO:teuthology.orchestra.run.smithi103.stdout:Installed: 2024-04-17T05:49:56.085 INFO:teuthology.orchestra.run.smithi103.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-17T05:49:56.085 INFO:teuthology.orchestra.run.smithi103.stdout: automake-1.16.2-8.el9.noarch 2024-04-17T05:49:56.085 INFO:teuthology.orchestra.run.smithi103.stdout: bison-3.7.4-5.el9.x86_64 2024-04-17T05:49:56.086 INFO:teuthology.orchestra.run.smithi103.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-17T05:49:56.086 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T05:49:56.086 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T05:49:56.086 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T05:49:56.086 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T05:49:56.086 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-grafana-dashboards-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-17T05:49:56.086 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-immutable-object-cache-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T05:49:56.086 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T05:49:56.086 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T05:49:56.086 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-17T05:49:56.086 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-17T05:49:56.086 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-17T05:49:56.087 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-modules-core-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-17T05:49:56.087 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-17T05:49:56.087 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T05:49:56.087 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T05:49:56.087 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-prometheus-alerts-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-17T05:49:56.087 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T05:49:56.087 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T05:49:56.087 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-test-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T05:49:56.087 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-17T05:49:56.087 INFO:teuthology.orchestra.run.smithi103.stdout: cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-17T05:49:56.087 INFO:teuthology.orchestra.run.smithi103.stdout: cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T05:49:56.087 INFO:teuthology.orchestra.run.smithi103.stdout: cephfs-shell-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T05:49:56.087 INFO:teuthology.orchestra.run.smithi103.stdout: cephfs-top-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-17T05:49:56.088 INFO:teuthology.orchestra.run.smithi103.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-17T05:49:56.088 INFO:teuthology.orchestra.run.smithi103.stdout: flex-2.6.4-9.el9.x86_64 2024-04-17T05:49:56.088 INFO:teuthology.orchestra.run.smithi103.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-17T05:49:56.088 INFO:teuthology.orchestra.run.smithi103.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-17T05:49:56.088 INFO:teuthology.orchestra.run.smithi103.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-17T05:49:56.088 INFO:teuthology.orchestra.run.smithi103.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-17T05:49:56.088 INFO:teuthology.orchestra.run.smithi103.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-17T05:49:56.088 INFO:teuthology.orchestra.run.smithi103.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-17T05:49:56.088 INFO:teuthology.orchestra.run.smithi103.stdout: jq-1.6-16.el9.x86_64 2024-04-17T05:49:56.088 INFO:teuthology.orchestra.run.smithi103.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-17T05:49:56.088 INFO:teuthology.orchestra.run.smithi103.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-17T05:49:56.088 INFO:teuthology.orchestra.run.smithi103.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-17T05:49:56.088 INFO:teuthology.orchestra.run.smithi103.stdout: libcephfs-devel-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T05:49:56.088 INFO:teuthology.orchestra.run.smithi103.stdout: libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T05:49:56.089 INFO:teuthology.orchestra.run.smithi103.stdout: libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T05:49:56.089 INFO:teuthology.orchestra.run.smithi103.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-17T05:49:56.089 INFO:teuthology.orchestra.run.smithi103.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-17T05:49:56.089 INFO:teuthology.orchestra.run.smithi103.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-17T05:49:56.089 INFO:teuthology.orchestra.run.smithi103.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-17T05:49:56.089 INFO:teuthology.orchestra.run.smithi103.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-17T05:49:56.089 INFO:teuthology.orchestra.run.smithi103.stdout: librados-devel-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T05:49:56.089 INFO:teuthology.orchestra.run.smithi103.stdout: libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T05:49:56.089 INFO:teuthology.orchestra.run.smithi103.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-17T05:49:56.089 INFO:teuthology.orchestra.run.smithi103.stdout: librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T05:49:56.089 INFO:teuthology.orchestra.run.smithi103.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-17T05:49:56.089 INFO:teuthology.orchestra.run.smithi103.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-17T05:49:56.089 INFO:teuthology.orchestra.run.smithi103.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-17T05:49:56.089 INFO:teuthology.orchestra.run.smithi103.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-17T05:49:56.090 INFO:teuthology.orchestra.run.smithi103.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-17T05:49:56.090 INFO:teuthology.orchestra.run.smithi103.stdout: lua-5.4.4-4.el9.x86_64 2024-04-17T05:49:56.090 INFO:teuthology.orchestra.run.smithi103.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-17T05:49:56.090 INFO:teuthology.orchestra.run.smithi103.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-17T05:49:56.090 INFO:teuthology.orchestra.run.smithi103.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-04-17T05:49:56.090 INFO:teuthology.orchestra.run.smithi103.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-17T05:49:56.090 INFO:teuthology.orchestra.run.smithi103.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-17T05:49:56.090 INFO:teuthology.orchestra.run.smithi103.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-17T05:49:56.090 INFO:teuthology.orchestra.run.smithi103.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-17T05:49:56.090 INFO:teuthology.orchestra.run.smithi103.stdout: postgresql-13.14-1.el9.x86_64 2024-04-17T05:49:56.090 INFO:teuthology.orchestra.run.smithi103.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-17T05:49:56.090 INFO:teuthology.orchestra.run.smithi103.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-17T05:49:56.090 INFO:teuthology.orchestra.run.smithi103.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-17T05:49:56.090 INFO:teuthology.orchestra.run.smithi103.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-17T05:49:56.091 INFO:teuthology.orchestra.run.smithi103.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-17T05:49:56.091 INFO:teuthology.orchestra.run.smithi103.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-17T05:49:56.091 INFO:teuthology.orchestra.run.smithi103.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-17T05:49:56.091 INFO:teuthology.orchestra.run.smithi103.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-17T05:49:56.091 INFO:teuthology.orchestra.run.smithi103.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-17T05:49:56.091 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-17T05:49:56.091 INFO:teuthology.orchestra.run.smithi103.stdout: python3-ceph-argparse-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T05:49:56.091 INFO:teuthology.orchestra.run.smithi103.stdout: python3-ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T05:49:56.091 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cephfs-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T05:49:56.091 INFO:teuthology.orchestra.run.smithi103.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-17T05:49:56.091 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-17T05:49:56.091 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-17T05:49:56.091 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-17T05:49:56.091 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-17T05:49:56.092 INFO:teuthology.orchestra.run.smithi103.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-17T05:49:56.092 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-17T05:49:56.092 INFO:teuthology.orchestra.run.smithi103.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-04-17T05:49:56.092 INFO:teuthology.orchestra.run.smithi103.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-17T05:49:56.092 INFO:teuthology.orchestra.run.smithi103.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-17T05:49:56.092 INFO:teuthology.orchestra.run.smithi103.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-17T05:49:56.092 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-17T05:49:56.092 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-17T05:49:56.092 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-17T05:49:56.092 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-17T05:49:56.092 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-17T05:49:56.092 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-17T05:49:56.092 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-17T05:49:56.093 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-17T05:49:56.093 INFO:teuthology.orchestra.run.smithi103.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-17T05:49:56.093 INFO:teuthology.orchestra.run.smithi103.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-17T05:49:56.093 INFO:teuthology.orchestra.run.smithi103.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-17T05:49:56.093 INFO:teuthology.orchestra.run.smithi103.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-17T05:49:56.093 INFO:teuthology.orchestra.run.smithi103.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-17T05:49:56.093 INFO:teuthology.orchestra.run.smithi103.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-17T05:49:56.093 INFO:teuthology.orchestra.run.smithi103.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-17T05:49:56.093 INFO:teuthology.orchestra.run.smithi103.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-17T05:49:56.093 INFO:teuthology.orchestra.run.smithi103.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-17T05:49:56.093 INFO:teuthology.orchestra.run.smithi103.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-17T05:49:56.093 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-17T05:49:56.093 INFO:teuthology.orchestra.run.smithi103.stdout: python3-ply-3.11-14.el9.noarch 2024-04-17T05:49:56.093 INFO:teuthology.orchestra.run.smithi103.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-17T05:49:56.094 INFO:teuthology.orchestra.run.smithi103.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-17T05:49:56.094 INFO:teuthology.orchestra.run.smithi103.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-17T05:49:56.094 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-17T05:49:56.094 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-17T05:49:56.094 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-17T05:49:56.094 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-17T05:49:56.094 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-17T05:49:56.094 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-17T05:49:56.094 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rados-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T05:49:56.094 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T05:49:56.094 INFO:teuthology.orchestra.run.smithi103.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-17T05:49:56.094 INFO:teuthology.orchestra.run.smithi103.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-17T05:49:56.094 INFO:teuthology.orchestra.run.smithi103.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-17T05:49:56.095 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T05:49:56.095 INFO:teuthology.orchestra.run.smithi103.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-17T05:49:56.095 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-17T05:49:56.095 INFO:teuthology.orchestra.run.smithi103.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-17T05:49:56.095 INFO:teuthology.orchestra.run.smithi103.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-17T05:49:56.095 INFO:teuthology.orchestra.run.smithi103.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-17T05:49:56.095 INFO:teuthology.orchestra.run.smithi103.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-17T05:49:56.095 INFO:teuthology.orchestra.run.smithi103.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-17T05:49:56.095 INFO:teuthology.orchestra.run.smithi103.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-17T05:49:56.095 INFO:teuthology.orchestra.run.smithi103.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-17T05:49:56.095 INFO:teuthology.orchestra.run.smithi103.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-17T05:49:56.095 INFO:teuthology.orchestra.run.smithi103.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-17T05:49:56.095 INFO:teuthology.orchestra.run.smithi103.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-17T05:49:56.095 INFO:teuthology.orchestra.run.smithi103.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-17T05:49:56.096 INFO:teuthology.orchestra.run.smithi103.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-17T05:49:56.096 INFO:teuthology.orchestra.run.smithi103.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-17T05:49:56.096 INFO:teuthology.orchestra.run.smithi103.stdout: rbd-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T05:49:56.096 INFO:teuthology.orchestra.run.smithi103.stdout: rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T05:49:56.096 INFO:teuthology.orchestra.run.smithi103.stdout: rbd-nbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T05:49:56.096 INFO:teuthology.orchestra.run.smithi103.stdout: re2-1:20211101-3.el9.x86_64 2024-04-17T05:49:56.096 INFO:teuthology.orchestra.run.smithi103.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-17T05:49:56.096 INFO:teuthology.orchestra.run.smithi103.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-17T05:49:56.096 INFO:teuthology.orchestra.run.smithi103.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-17T05:49:56.096 INFO:teuthology.orchestra.run.smithi103.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-17T05:49:56.096 INFO:teuthology.orchestra.run.smithi103.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-17T05:49:56.097 INFO:teuthology.orchestra.run.smithi103.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-04-17T05:49:56.097 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-04-17T05:49:56.097 INFO:teuthology.orchestra.run.smithi103.stdout:Complete! 2024-04-17T05:49:56.536 DEBUG:teuthology.parallel:result is None 2024-04-17T05:49:58.574 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-17T05:49:58.575 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-2:19.0.0-3133.ga8c84f59.el9.x86_64 1/198 2024-04-17T05:49:58.575 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 2/198 2024-04-17T05:49:58.575 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 3/198 2024-04-17T05:49:58.575 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 4/198 2024-04-17T05:49:58.575 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3133.ga8c84 5/198 2024-04-17T05:49:58.575 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 6/198 2024-04-17T05:49:58.575 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 7/198 2024-04-17T05:49:58.575 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 8/198 2024-04-17T05:49:58.575 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 9/198 2024-04-17T05:49:58.575 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 10/198 2024-04-17T05:49:58.575 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 11/198 2024-04-17T05:49:58.575 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-test-2:19.0.0-3133.ga8c84f59.el9.x86_64 12/198 2024-04-17T05:49:58.576 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 13/198 2024-04-17T05:49:58.576 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : cephfs-shell-2:19.0.0-3133.ga8c84f59.el9.x86_64 14/198 2024-04-17T05:49:58.576 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libcephfs-devel-2:19.0.0-3133.ga8c84f59.el9.x86_ 15/198 2024-04-17T05:49:58.576 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 16/198 2024-04-17T05:49:58.576 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 17/198 2024-04-17T05:49:58.576 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librados-devel-2:19.0.0-3133.ga8c84f59.el9.x86_6 18/198 2024-04-17T05:49:58.576 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86 19/198 2024-04-17T05:49:58.576 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 20/198 2024-04-17T05:49:58.576 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3133.ga8c84f59.el 21/198 2024-04-17T05:49:58.576 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-ceph-common-2:19.0.0-3133.ga8c84f59.el9. 22/198 2024-04-17T05:49:58.576 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-cephfs-2:19.0.0-3133.ga8c84f59.el9.x86_6 23/198 2024-04-17T05:49:58.576 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-rados-2:19.0.0-3133.ga8c84f59.el9.x86_64 24/198 2024-04-17T05:49:58.576 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-rbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 25/198 2024-04-17T05:49:58.576 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-rgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 26/198 2024-04-17T05:49:58.576 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : rbd-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 27/198 2024-04-17T05:49:58.577 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 28/198 2024-04-17T05:49:58.577 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : rbd-nbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 29/198 2024-04-17T05:49:58.577 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3133.ga8c84f59. 30/198 2024-04-17T05:49:58.577 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noa 31/198 2024-04-17T05:49:58.577 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.n 32/198 2024-04-17T05:49:58.577 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c 33/198 2024-04-17T05:49:58.577 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3133.ga8c84f59.el 34/198 2024-04-17T05:49:58.577 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 35/198 2024-04-17T05:49:58.577 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3133.ga8c84f59.e 36/198 2024-04-17T05:49:58.577 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 37/198 2024-04-17T05:49:58.577 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 38/198 2024-04-17T05:49:58.577 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : cephfs-top-2:19.0.0-3133.ga8c84f59.el9.noarch 39/198 2024-04-17T05:49:58.577 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-04-17T05:49:58.577 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-04-17T05:49:58.578 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-04-17T05:49:58.578 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-04-17T05:49:58.578 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-04-17T05:49:58.578 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-04-17T05:49:58.578 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-04-17T05:49:58.578 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-04-17T05:49:58.578 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-04-17T05:49:58.578 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-04-17T05:49:58.578 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-04-17T05:49:58.578 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-04-17T05:49:58.578 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-04-17T05:49:58.578 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-04-17T05:49:58.578 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-04-17T05:49:58.579 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-04-17T05:49:58.579 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-04-17T05:49:58.579 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-04-17T05:49:58.579 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : postgresql-13.14-1.el9.x86_64 58/198 2024-04-17T05:49:58.579 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 59/198 2024-04-17T05:49:58.579 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 60/198 2024-04-17T05:49:58.579 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 61/198 2024-04-17T05:49:58.579 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-04-17T05:49:58.579 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-04-17T05:49:58.579 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-04-17T05:49:58.579 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-04-17T05:49:58.579 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-04-17T05:49:58.580 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-04-17T05:49:58.580 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-04-17T05:49:58.580 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-04-17T05:49:58.580 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-04-17T05:49:58.580 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-04-17T05:49:58.580 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-04-17T05:49:58.580 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-04-17T05:49:58.580 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-04-17T05:49:58.580 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-04-17T05:49:58.580 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-04-17T05:49:58.580 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-04-17T05:49:58.580 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-04-17T05:49:58.580 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-04-17T05:49:58.580 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-04-17T05:49:58.581 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-04-17T05:49:58.581 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 82/198 2024-04-17T05:49:58.581 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 83/198 2024-04-17T05:49:58.581 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 84/198 2024-04-17T05:49:58.581 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 85/198 2024-04-17T05:49:58.581 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 86/198 2024-04-17T05:49:58.581 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 87/198 2024-04-17T05:49:58.581 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 88/198 2024-04-17T05:49:58.581 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 89/198 2024-04-17T05:49:58.581 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 90/198 2024-04-17T05:49:58.581 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 91/198 2024-04-17T05:49:58.581 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-ply-3.11-14.el9.noarch 92/198 2024-04-17T05:49:58.581 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 93/198 2024-04-17T05:49:58.581 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 94/198 2024-04-17T05:49:58.582 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : jq-1.6-16.el9.x86_64 95/198 2024-04-17T05:49:58.582 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 96/198 2024-04-17T05:49:58.582 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 97/198 2024-04-17T05:49:58.582 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 98/198 2024-04-17T05:49:58.582 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 99/198 2024-04-17T05:49:58.582 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 100/198 2024-04-17T05:49:58.582 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 101/198 2024-04-17T05:49:58.582 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 102/198 2024-04-17T05:49:58.582 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 103/198 2024-04-17T05:49:58.582 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-04-17T05:49:58.582 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-04-17T05:49:58.582 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-04-17T05:49:58.582 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-04-17T05:49:58.582 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-04-17T05:49:58.583 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-04-17T05:49:58.583 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-04-17T05:49:58.583 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-04-17T05:49:58.583 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-04-17T05:49:58.583 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-04-17T05:49:58.583 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-04-17T05:49:58.583 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-04-17T05:49:58.583 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-04-17T05:49:58.583 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-04-17T05:49:58.583 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-04-17T05:49:58.583 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-04-17T05:49:58.583 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-04-17T05:49:58.583 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-04-17T05:49:58.583 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-04-17T05:49:58.583 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-04-17T05:49:58.584 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-04-17T05:49:58.584 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-04-17T05:49:58.584 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-04-17T05:49:58.584 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-04-17T05:49:58.584 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-04-17T05:49:58.584 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-04-17T05:49:58.584 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-04-17T05:49:58.584 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-04-17T05:49:58.584 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-04-17T05:49:58.584 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-04-17T05:49:58.584 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-04-17T05:49:58.584 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-04-17T05:49:58.584 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-04-17T05:49:58.584 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-04-17T05:49:58.585 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-04-17T05:49:58.585 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-04-17T05:49:58.585 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-04-17T05:49:58.585 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-04-17T05:49:58.585 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-04-17T05:49:58.585 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-04-17T05:49:58.585 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-04-17T05:49:58.585 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-04-17T05:49:58.585 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-04-17T05:49:58.585 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-04-17T05:49:58.585 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-04-17T05:49:58.585 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-04-17T05:49:58.585 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-04-17T05:49:58.585 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 151/198 2024-04-17T05:49:58.586 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-04-17T05:49:58.586 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 153/198 2024-04-17T05:49:58.586 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-04-17T05:49:58.586 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 155/198 2024-04-17T05:49:58.586 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 156/198 2024-04-17T05:49:58.586 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 157/198 2024-04-17T05:49:58.586 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/198 2024-04-17T05:49:58.586 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 159/198 2024-04-17T05:49:58.586 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 160/198 2024-04-17T05:49:58.586 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 161/198 2024-04-17T05:49:58.586 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 162/198 2024-04-17T05:49:58.586 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 163/198 2024-04-17T05:49:58.586 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 164/198 2024-04-17T05:49:58.586 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-04-17T05:49:58.587 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-04-17T05:49:58.587 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-04-17T05:49:58.587 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-04-17T05:49:58.587 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-04-17T05:49:58.587 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-04-17T05:49:58.587 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-04-17T05:49:58.587 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-04-17T05:49:58.587 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-04-17T05:49:58.587 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-04-17T05:49:58.587 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-04-17T05:49:58.587 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-04-17T05:49:58.587 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-04-17T05:49:58.587 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-04-17T05:49:58.587 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-04-17T05:49:58.587 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-04-17T05:49:58.587 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-04-17T05:49:58.588 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-04-17T05:49:58.588 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-04-17T05:49:58.588 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-04-17T05:49:58.588 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-04-17T05:49:58.588 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-04-17T05:49:58.588 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-04-17T05:49:58.588 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-04-17T05:49:58.588 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-04-17T05:49:58.588 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-04-17T05:49:58.588 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-04-17T05:49:58.588 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-04-17T05:49:58.588 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-04-17T05:49:58.588 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-04-17T05:49:58.588 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-04-17T05:49:58.589 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-04-17T05:49:58.589 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-04-17T05:50:01.794 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-04-17T05:50:01.794 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-17T05:50:01.794 INFO:teuthology.orchestra.run.smithi047.stdout:Upgraded: 2024-04-17T05:50:01.794 INFO:teuthology.orchestra.run.smithi047.stdout: NetworkManager-1:1.47.4-1.el9.x86_64 2024-04-17T05:50:01.794 INFO:teuthology.orchestra.run.smithi047.stdout: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 2024-04-17T05:50:01.794 INFO:teuthology.orchestra.run.smithi047.stdout: NetworkManager-team-1:1.47.4-1.el9.x86_64 2024-04-17T05:50:01.794 INFO:teuthology.orchestra.run.smithi047.stdout: NetworkManager-tui-1:1.47.4-1.el9.x86_64 2024-04-17T05:50:01.795 INFO:teuthology.orchestra.run.smithi047.stdout: acl-2.3.1-4.el9.x86_64 2024-04-17T05:50:01.795 INFO:teuthology.orchestra.run.smithi047.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-04-17T05:50:01.795 INFO:teuthology.orchestra.run.smithi047.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-04-17T05:50:01.795 INFO:teuthology.orchestra.run.smithi047.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-04-17T05:50:01.795 INFO:teuthology.orchestra.run.smithi047.stdout: libacl-2.3.1-4.el9.x86_64 2024-04-17T05:50:01.795 INFO:teuthology.orchestra.run.smithi047.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-04-17T05:50:01.795 INFO:teuthology.orchestra.run.smithi047.stdout: libblkid-2.37.4-18.el9.x86_64 2024-04-17T05:50:01.795 INFO:teuthology.orchestra.run.smithi047.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-04-17T05:50:01.795 INFO:teuthology.orchestra.run.smithi047.stdout: libmount-2.37.4-18.el9.x86_64 2024-04-17T05:50:01.795 INFO:teuthology.orchestra.run.smithi047.stdout: librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T05:50:01.795 INFO:teuthology.orchestra.run.smithi047.stdout: librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T05:50:01.795 INFO:teuthology.orchestra.run.smithi047.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-04-17T05:50:01.795 INFO:teuthology.orchestra.run.smithi047.stdout: libuuid-2.37.4-18.el9.x86_64 2024-04-17T05:50:01.796 INFO:teuthology.orchestra.run.smithi047.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-04-17T05:50:01.796 INFO:teuthology.orchestra.run.smithi047.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-04-17T05:50:01.796 INFO:teuthology.orchestra.run.smithi047.stdout: util-linux-2.37.4-18.el9.x86_64 2024-04-17T05:50:01.796 INFO:teuthology.orchestra.run.smithi047.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-04-17T05:50:01.796 INFO:teuthology.orchestra.run.smithi047.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-04-17T05:50:01.796 INFO:teuthology.orchestra.run.smithi047.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-04-17T05:50:01.796 INFO:teuthology.orchestra.run.smithi047.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-04-17T05:50:01.796 INFO:teuthology.orchestra.run.smithi047.stdout:Installed: 2024-04-17T05:50:01.796 INFO:teuthology.orchestra.run.smithi047.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-17T05:50:01.796 INFO:teuthology.orchestra.run.smithi047.stdout: automake-1.16.2-8.el9.noarch 2024-04-17T05:50:01.796 INFO:teuthology.orchestra.run.smithi047.stdout: bison-3.7.4-5.el9.x86_64 2024-04-17T05:50:01.796 INFO:teuthology.orchestra.run.smithi047.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-17T05:50:01.796 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T05:50:01.796 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T05:50:01.797 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T05:50:01.797 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T05:50:01.797 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-grafana-dashboards-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-17T05:50:01.797 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-immutable-object-cache-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T05:50:01.797 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T05:50:01.797 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T05:50:01.797 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-17T05:50:01.797 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-17T05:50:01.797 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-17T05:50:01.797 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-modules-core-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-17T05:50:01.797 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-17T05:50:01.797 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T05:50:01.797 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T05:50:01.798 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-prometheus-alerts-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-17T05:50:01.798 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T05:50:01.798 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T05:50:01.798 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-test-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T05:50:01.798 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-17T05:50:01.798 INFO:teuthology.orchestra.run.smithi047.stdout: cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-17T05:50:01.798 INFO:teuthology.orchestra.run.smithi047.stdout: cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T05:50:01.798 INFO:teuthology.orchestra.run.smithi047.stdout: cephfs-shell-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T05:50:01.798 INFO:teuthology.orchestra.run.smithi047.stdout: cephfs-top-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-17T05:50:01.798 INFO:teuthology.orchestra.run.smithi047.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-17T05:50:01.798 INFO:teuthology.orchestra.run.smithi047.stdout: flex-2.6.4-9.el9.x86_64 2024-04-17T05:50:01.798 INFO:teuthology.orchestra.run.smithi047.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-17T05:50:01.798 INFO:teuthology.orchestra.run.smithi047.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-17T05:50:01.799 INFO:teuthology.orchestra.run.smithi047.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-17T05:50:01.799 INFO:teuthology.orchestra.run.smithi047.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-17T05:50:01.799 INFO:teuthology.orchestra.run.smithi047.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-17T05:50:01.799 INFO:teuthology.orchestra.run.smithi047.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-17T05:50:01.799 INFO:teuthology.orchestra.run.smithi047.stdout: jq-1.6-16.el9.x86_64 2024-04-17T05:50:01.799 INFO:teuthology.orchestra.run.smithi047.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-17T05:50:01.799 INFO:teuthology.orchestra.run.smithi047.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-17T05:50:01.799 INFO:teuthology.orchestra.run.smithi047.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-17T05:50:01.799 INFO:teuthology.orchestra.run.smithi047.stdout: libcephfs-devel-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T05:50:01.799 INFO:teuthology.orchestra.run.smithi047.stdout: libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T05:50:01.799 INFO:teuthology.orchestra.run.smithi047.stdout: libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T05:50:01.799 INFO:teuthology.orchestra.run.smithi047.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-17T05:50:01.799 INFO:teuthology.orchestra.run.smithi047.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-17T05:50:01.800 INFO:teuthology.orchestra.run.smithi047.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-17T05:50:01.800 INFO:teuthology.orchestra.run.smithi047.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-17T05:50:01.800 INFO:teuthology.orchestra.run.smithi047.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-17T05:50:01.800 INFO:teuthology.orchestra.run.smithi047.stdout: librados-devel-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T05:50:01.800 INFO:teuthology.orchestra.run.smithi047.stdout: libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T05:50:01.800 INFO:teuthology.orchestra.run.smithi047.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-17T05:50:01.800 INFO:teuthology.orchestra.run.smithi047.stdout: librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T05:50:01.800 INFO:teuthology.orchestra.run.smithi047.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-17T05:50:01.800 INFO:teuthology.orchestra.run.smithi047.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-17T05:50:01.800 INFO:teuthology.orchestra.run.smithi047.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-17T05:50:01.800 INFO:teuthology.orchestra.run.smithi047.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-17T05:50:01.800 INFO:teuthology.orchestra.run.smithi047.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-17T05:50:01.800 INFO:teuthology.orchestra.run.smithi047.stdout: lua-5.4.4-4.el9.x86_64 2024-04-17T05:50:01.800 INFO:teuthology.orchestra.run.smithi047.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-17T05:50:01.800 INFO:teuthology.orchestra.run.smithi047.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-17T05:50:01.801 INFO:teuthology.orchestra.run.smithi047.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-04-17T05:50:01.801 INFO:teuthology.orchestra.run.smithi047.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-17T05:50:01.801 INFO:teuthology.orchestra.run.smithi047.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-17T05:50:01.801 INFO:teuthology.orchestra.run.smithi047.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-17T05:50:01.801 INFO:teuthology.orchestra.run.smithi047.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-17T05:50:01.801 INFO:teuthology.orchestra.run.smithi047.stdout: postgresql-13.14-1.el9.x86_64 2024-04-17T05:50:01.801 INFO:teuthology.orchestra.run.smithi047.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-17T05:50:01.801 INFO:teuthology.orchestra.run.smithi047.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-17T05:50:01.801 INFO:teuthology.orchestra.run.smithi047.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-17T05:50:01.801 INFO:teuthology.orchestra.run.smithi047.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-17T05:50:01.801 INFO:teuthology.orchestra.run.smithi047.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-17T05:50:01.801 INFO:teuthology.orchestra.run.smithi047.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-17T05:50:01.801 INFO:teuthology.orchestra.run.smithi047.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-17T05:50:01.802 INFO:teuthology.orchestra.run.smithi047.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-17T05:50:01.802 INFO:teuthology.orchestra.run.smithi047.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-17T05:50:01.802 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-17T05:50:01.802 INFO:teuthology.orchestra.run.smithi047.stdout: python3-ceph-argparse-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T05:50:01.802 INFO:teuthology.orchestra.run.smithi047.stdout: python3-ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T05:50:01.802 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cephfs-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T05:50:01.802 INFO:teuthology.orchestra.run.smithi047.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-17T05:50:01.802 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-17T05:50:01.802 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-17T05:50:01.802 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-17T05:50:01.802 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-17T05:50:01.802 INFO:teuthology.orchestra.run.smithi047.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-17T05:50:01.802 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-17T05:50:01.803 INFO:teuthology.orchestra.run.smithi047.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-04-17T05:50:01.803 INFO:teuthology.orchestra.run.smithi047.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-17T05:50:01.803 INFO:teuthology.orchestra.run.smithi047.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-17T05:50:01.803 INFO:teuthology.orchestra.run.smithi047.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-17T05:50:01.803 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-17T05:50:01.803 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-17T05:50:01.803 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-17T05:50:01.803 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-17T05:50:01.803 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-17T05:50:01.803 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-17T05:50:01.803 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-17T05:50:01.803 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-17T05:50:01.803 INFO:teuthology.orchestra.run.smithi047.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-17T05:50:01.804 INFO:teuthology.orchestra.run.smithi047.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-17T05:50:01.804 INFO:teuthology.orchestra.run.smithi047.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-17T05:50:01.804 INFO:teuthology.orchestra.run.smithi047.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-17T05:50:01.804 INFO:teuthology.orchestra.run.smithi047.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-17T05:50:01.804 INFO:teuthology.orchestra.run.smithi047.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-17T05:50:01.804 INFO:teuthology.orchestra.run.smithi047.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-17T05:50:01.804 INFO:teuthology.orchestra.run.smithi047.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-17T05:50:01.804 INFO:teuthology.orchestra.run.smithi047.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-17T05:50:01.804 INFO:teuthology.orchestra.run.smithi047.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-17T05:50:01.804 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-17T05:50:01.804 INFO:teuthology.orchestra.run.smithi047.stdout: python3-ply-3.11-14.el9.noarch 2024-04-17T05:50:01.804 INFO:teuthology.orchestra.run.smithi047.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-17T05:50:01.804 INFO:teuthology.orchestra.run.smithi047.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-17T05:50:01.804 INFO:teuthology.orchestra.run.smithi047.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-17T05:50:01.805 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-17T05:50:01.805 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-17T05:50:01.805 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-17T05:50:01.805 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-17T05:50:01.805 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-17T05:50:01.805 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-17T05:50:01.805 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rados-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T05:50:01.805 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T05:50:01.805 INFO:teuthology.orchestra.run.smithi047.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-17T05:50:01.805 INFO:teuthology.orchestra.run.smithi047.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-17T05:50:01.805 INFO:teuthology.orchestra.run.smithi047.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-17T05:50:01.805 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T05:50:01.805 INFO:teuthology.orchestra.run.smithi047.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-17T05:50:01.805 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-17T05:50:01.806 INFO:teuthology.orchestra.run.smithi047.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-17T05:50:01.806 INFO:teuthology.orchestra.run.smithi047.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-17T05:50:01.806 INFO:teuthology.orchestra.run.smithi047.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-17T05:50:01.806 INFO:teuthology.orchestra.run.smithi047.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-17T05:50:01.806 INFO:teuthology.orchestra.run.smithi047.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-17T05:50:01.806 INFO:teuthology.orchestra.run.smithi047.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-17T05:50:01.806 INFO:teuthology.orchestra.run.smithi047.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-17T05:50:01.806 INFO:teuthology.orchestra.run.smithi047.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-17T05:50:01.806 INFO:teuthology.orchestra.run.smithi047.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-17T05:50:01.806 INFO:teuthology.orchestra.run.smithi047.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-17T05:50:01.806 INFO:teuthology.orchestra.run.smithi047.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-17T05:50:01.806 INFO:teuthology.orchestra.run.smithi047.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-17T05:50:01.806 INFO:teuthology.orchestra.run.smithi047.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-17T05:50:01.806 INFO:teuthology.orchestra.run.smithi047.stdout: rbd-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T05:50:01.807 INFO:teuthology.orchestra.run.smithi047.stdout: rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T05:50:01.807 INFO:teuthology.orchestra.run.smithi047.stdout: rbd-nbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T05:50:01.807 INFO:teuthology.orchestra.run.smithi047.stdout: re2-1:20211101-3.el9.x86_64 2024-04-17T05:50:01.807 INFO:teuthology.orchestra.run.smithi047.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-17T05:50:01.807 INFO:teuthology.orchestra.run.smithi047.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-17T05:50:01.807 INFO:teuthology.orchestra.run.smithi047.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-17T05:50:01.807 INFO:teuthology.orchestra.run.smithi047.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-17T05:50:01.807 INFO:teuthology.orchestra.run.smithi047.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-17T05:50:01.807 INFO:teuthology.orchestra.run.smithi047.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-04-17T05:50:01.807 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-17T05:50:01.808 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-04-17T05:50:02.232 DEBUG:teuthology.parallel:result is None 2024-04-17T05:50:02.233 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a8c84f5997950fd7d008204839253291d52e6eda 2024-04-17T05:50:02.463 DEBUG:teuthology.orchestra.run.smithi047:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-17T05:50:02.486 INFO:teuthology.orchestra.run.smithi047.stdout:19.0.0-3133.ga8c84f59.el9 2024-04-17T05:50:02.487 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3133.ga8c84f59.el9 2024-04-17T05:50:02.487 INFO:teuthology.task.install:The correct ceph version 19.0.0-3133.ga8c84f59 is installed. 2024-04-17T05:50:02.489 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a8c84f5997950fd7d008204839253291d52e6eda 2024-04-17T05:50:02.654 DEBUG:teuthology.orchestra.run.smithi098:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-17T05:50:02.680 INFO:teuthology.orchestra.run.smithi098.stdout:19.0.0-3133.ga8c84f59.el9 2024-04-17T05:50:02.680 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3133.ga8c84f59.el9 2024-04-17T05:50:02.681 INFO:teuthology.task.install:The correct ceph version 19.0.0-3133.ga8c84f59 is installed. 2024-04-17T05:50:02.682 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a8c84f5997950fd7d008204839253291d52e6eda 2024-04-17T05:50:02.846 DEBUG:teuthology.orchestra.run.smithi103:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-17T05:50:02.872 INFO:teuthology.orchestra.run.smithi103.stdout:19.0.0-3133.ga8c84f59.el9 2024-04-17T05:50:02.872 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3133.ga8c84f59.el9 2024-04-17T05:50:02.872 INFO:teuthology.task.install:The correct ceph version 19.0.0-3133.ga8c84f59 is installed. 2024-04-17T05:50:02.874 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-17T05:50:02.874 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-17T05:50:02.874 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-17T05:50:02.911 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-04-17T05:50:02.911 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-17T05:50:02.946 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-04-17T05:50:02.946 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-17T05:50:02.982 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-17T05:50:02.982 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-17T05:50:02.982 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/usr/bin/daemon-helper 2024-04-17T05:50:03.014 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-17T05:50:03.086 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-04-17T05:50:03.086 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/usr/bin/daemon-helper 2024-04-17T05:50:03.118 DEBUG:teuthology.orchestra.run.smithi098:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-17T05:50:03.188 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-04-17T05:50:03.188 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/usr/bin/daemon-helper 2024-04-17T05:50:03.220 DEBUG:teuthology.orchestra.run.smithi103:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-17T05:50:03.290 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-17T05:50:03.291 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-17T05:50:03.291 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-17T05:50:03.323 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-17T05:50:03.397 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-04-17T05:50:03.397 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-17T05:50:03.428 DEBUG:teuthology.orchestra.run.smithi098:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-17T05:50:03.498 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-04-17T05:50:03.499 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-17T05:50:03.530 DEBUG:teuthology.orchestra.run.smithi103:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-17T05:50:03.601 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-17T05:50:03.601 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-17T05:50:03.601 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/usr/bin/stdin-killer 2024-04-17T05:50:03.635 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-17T05:50:03.706 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-04-17T05:50:03.707 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/usr/bin/stdin-killer 2024-04-17T05:50:03.737 DEBUG:teuthology.orchestra.run.smithi098:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-17T05:50:03.807 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-04-17T05:50:03.807 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/usr/bin/stdin-killer 2024-04-17T05:50:03.838 DEBUG:teuthology.orchestra.run.smithi103:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-17T05:50:03.910 INFO:teuthology.run_tasks:Running task cephadm... 2024-04-17T05:50:04.011 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-04-17T05:50:04.011 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda 2024-04-17T05:50:04.012 INFO:tasks.cephadm:Cluster fsid is 565db2d8-fc7e-11ee-bc91-c7b262605968 2024-04-17T05:50:04.012 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-04-17T05:50:04.012 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.47', 'mon.b': '172.21.15.98', 'mon.c': '172.21.15.103'} 2024-04-17T05:50:04.012 INFO:tasks.cephadm:First mon is mon.a on smithi047 2024-04-17T05:50:04.013 INFO:tasks.cephadm:First mgr is x 2024-04-17T05:50:04.013 INFO:tasks.cephadm:Normalizing hostnames... 2024-04-17T05:50:04.013 DEBUG:teuthology.orchestra.run.smithi047:> sudo hostname $(hostname -s) 2024-04-17T05:50:04.046 DEBUG:teuthology.orchestra.run.smithi098:> sudo hostname $(hostname -s) 2024-04-17T05:50:04.077 DEBUG:teuthology.orchestra.run.smithi103:> sudo hostname $(hostname -s) 2024-04-17T05:50:04.109 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-04-17T05:50:04.109 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a8c84f5997950fd7d008204839253291d52e6eda 2024-04-17T05:50:04.340 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/78637/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3133-ga8c84f59', 'node_name': '172.21.5.35+adami05', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3133.ga8c84f59'}, 'url': 'https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-04-16 21:41:56.138585', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-rishabh-testing-20240416.193735', 'chacra_url': 'https://2.chacra.ceph.com/repos/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-04-17T05:50:04.484 INFO:tasks.util.chacra:got chacra host 2.chacra.ceph.com, ref wip-rishabh-testing-20240416.193735, sha1 a8c84f5997950fd7d008204839253291d52e6eda from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=a8c84f5997950fd7d008204839253291d52e6eda 2024-04-17T05:50:04.485 INFO:tasks.cephadm:Discovered cachra url: https://2.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/x86_64/flavors/default/cephadm 2024-04-17T05:50:04.486 DEBUG:teuthology.orchestra.run.smithi047:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-17T05:50:05.104 INFO:teuthology.orchestra.run.smithi047.stdout:-rw-r--r--. 1 ubuntu ubuntu 781646 Apr 17 05:50 /home/ubuntu/cephtest/cephadm 2024-04-17T05:50:05.104 DEBUG:teuthology.orchestra.run.smithi098:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-17T05:50:05.681 INFO:teuthology.orchestra.run.smithi098.stdout:-rw-r--r--. 1 ubuntu ubuntu 781646 Apr 17 05:50 /home/ubuntu/cephtest/cephadm 2024-04-17T05:50:05.681 DEBUG:teuthology.orchestra.run.smithi103:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-17T05:50:06.263 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-r--r--. 1 ubuntu ubuntu 781646 Apr 17 05:50 /home/ubuntu/cephtest/cephadm 2024-04-17T05:50:06.263 DEBUG:teuthology.orchestra.run.smithi047:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-17T05:50:06.283 DEBUG:teuthology.orchestra.run.smithi098:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-17T05:50:06.302 DEBUG:teuthology.orchestra.run.smithi103:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-17T05:50:06.338 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda on all hosts... 2024-04-17T05:50:06.338 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda pull 2024-04-17T05:50:06.341 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda pull 2024-04-17T05:50:06.345 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda pull 2024-04-17T05:50:07.261 INFO:teuthology.orchestra.run.smithi047.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda... 2024-04-17T05:50:07.270 INFO:teuthology.orchestra.run.smithi103.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda... 2024-04-17T05:50:07.289 INFO:teuthology.orchestra.run.smithi098.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda... 2024-04-17T05:50:45.732 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-17T05:50:45.732 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", 2024-04-17T05:50:45.732 INFO:teuthology.orchestra.run.smithi047.stdout: "image_id": "76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d", 2024-04-17T05:50:45.733 INFO:teuthology.orchestra.run.smithi047.stdout: "repo_digests": [ 2024-04-17T05:50:45.733 INFO:teuthology.orchestra.run.smithi047.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1" 2024-04-17T05:50:45.733 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-17T05:50:45.733 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-17T05:50:46.608 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-04-17T05:50:46.608 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", 2024-04-17T05:50:46.608 INFO:teuthology.orchestra.run.smithi098.stdout: "image_id": "76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d", 2024-04-17T05:50:46.608 INFO:teuthology.orchestra.run.smithi098.stdout: "repo_digests": [ 2024-04-17T05:50:46.608 INFO:teuthology.orchestra.run.smithi098.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1" 2024-04-17T05:50:46.608 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-04-17T05:50:46.608 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-04-17T05:50:57.571 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-04-17T05:50:57.572 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", 2024-04-17T05:50:57.572 INFO:teuthology.orchestra.run.smithi103.stdout: "image_id": "76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d", 2024-04-17T05:50:57.572 INFO:teuthology.orchestra.run.smithi103.stdout: "repo_digests": [ 2024-04-17T05:50:57.572 INFO:teuthology.orchestra.run.smithi103.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1" 2024-04-17T05:50:57.572 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-04-17T05:50:57.572 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-04-17T05:50:57.595 DEBUG:teuthology.orchestra.run.smithi047:> sudo mkdir -p /etc/ceph 2024-04-17T05:50:57.633 DEBUG:teuthology.orchestra.run.smithi098:> sudo mkdir -p /etc/ceph 2024-04-17T05:50:57.668 DEBUG:teuthology.orchestra.run.smithi103:> sudo mkdir -p /etc/ceph 2024-04-17T05:50:57.698 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 777 /etc/ceph 2024-04-17T05:50:57.730 DEBUG:teuthology.orchestra.run.smithi098:> sudo chmod 777 /etc/ceph 2024-04-17T05:50:57.760 DEBUG:teuthology.orchestra.run.smithi103:> sudo chmod 777 /etc/ceph 2024-04-17T05:50:57.786 INFO:tasks.cephadm:Writing seed config... 2024-04-17T05:50:57.788 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-04-17T05:50:57.788 INFO:tasks.cephadm: override: [client] debug client = 20 2024-04-17T05:50:57.788 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-04-17T05:50:57.788 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-04-17T05:50:57.788 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-04-17T05:50:57.789 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-04-17T05:50:57.789 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-04-17T05:50:57.789 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-04-17T05:50:57.789 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-04-17T05:50:57.789 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-04-17T05:50:57.789 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-04-17T05:50:57.789 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-04-17T05:50:57.789 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-04-17T05:50:57.789 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-04-17T05:50:57.789 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-04-17T05:50:57.790 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-04-17T05:50:57.790 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-04-17T05:50:57.790 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-04-17T05:50:57.790 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-04-17T05:50:57.790 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-04-17T05:50:57.790 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-04-17T05:50:57.790 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-04-17T05:50:57.790 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-04-17T05:50:57.790 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-04-17T05:50:57.790 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-04-17T05:50:57.790 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-04-17T05:50:57.790 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-04-17T05:50:57.790 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-04-17T05:50:57.791 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-04-17T05:50:57.791 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-04-17T05:50:57.791 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-04-17T05:50:57.791 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-04-17T05:50:57.791 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-04-17T05:50:57.791 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-04-17T05:50:57.791 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-04-17T05:50:57.791 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-04-17T05:50:57.791 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-04-17T05:50:57.791 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-04-17T05:50:57.791 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-04-17T05:50:57.791 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-04-17T05:50:57.791 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-04-17T05:50:57.791 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-04-17T05:50:57.791 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-04-17T05:50:57.792 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-04-17T05:50:57.792 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-04-17T05:50:57.792 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-04-17T05:50:57.792 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-04-17T05:50:57.793 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-17T05:50:57.793 DEBUG:teuthology.orchestra.run.smithi047:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-04-17T05:50:57.811 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 565db2d8-fc7e-11ee-bc91-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-04-17T05:50:57.811 DEBUG:teuthology.orchestra.run.smithi047:mon.a> sudo journalctl -f -n 0 -u ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@mon.a.service 2024-04-17T05:50:57.854 DEBUG:teuthology.orchestra.run.smithi047:mgr.x> sudo journalctl -f -n 0 -u ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@mgr.x.service 2024-04-17T05:50:57.897 INFO:tasks.cephadm:Bootstrapping... 2024-04-17T05:50:57.898 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda -v bootstrap --fsid 565db2d8-fc7e-11ee-bc91-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.47 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-04-17T05:50:58.101 INFO:teuthology.orchestra.run.smithi047.stdout:-------------------------------------------------------------------------------- 2024-04-17T05:50:58.101 INFO:teuthology.orchestra.run.smithi047.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda', '-v', 'bootstrap', '--fsid', '565db2d8-fc7e-11ee-bc91-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.47', '--skip-admin-label'] 2024-04-17T05:50:58.101 INFO:teuthology.orchestra.run.smithi047.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-04-17T05:50:58.102 INFO:teuthology.orchestra.run.smithi047.stdout:Verifying podman|docker is present... 2024-04-17T05:50:58.127 INFO:teuthology.orchestra.run.smithi047.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-17T05:50:58.127 INFO:teuthology.orchestra.run.smithi047.stdout:Verifying lvm2 is present... 2024-04-17T05:50:58.127 INFO:teuthology.orchestra.run.smithi047.stdout:Verifying time synchronization is in place... 2024-04-17T05:50:58.135 INFO:teuthology.orchestra.run.smithi047.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-17T05:50:58.135 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-17T05:50:58.141 INFO:teuthology.orchestra.run.smithi047.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-17T05:50:58.141 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stdout inactive 2024-04-17T05:50:58.148 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stdout enabled 2024-04-17T05:50:58.154 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stdout active 2024-04-17T05:50:58.154 INFO:teuthology.orchestra.run.smithi047.stdout:Unit chronyd.service is enabled and running 2024-04-17T05:50:58.154 INFO:teuthology.orchestra.run.smithi047.stdout:Repeating the final host check... 2024-04-17T05:50:58.180 INFO:teuthology.orchestra.run.smithi047.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-17T05:50:58.180 INFO:teuthology.orchestra.run.smithi047.stdout:podman (/bin/podman) version 4.9.4 is present 2024-04-17T05:50:58.180 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl is present 2024-04-17T05:50:58.180 INFO:teuthology.orchestra.run.smithi047.stdout:lvcreate is present 2024-04-17T05:50:58.186 INFO:teuthology.orchestra.run.smithi047.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-17T05:50:58.186 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-17T05:50:58.193 INFO:teuthology.orchestra.run.smithi047.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-17T05:50:58.193 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stdout inactive 2024-04-17T05:50:58.199 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stdout enabled 2024-04-17T05:50:58.206 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stdout active 2024-04-17T05:50:58.206 INFO:teuthology.orchestra.run.smithi047.stdout:Unit chronyd.service is enabled and running 2024-04-17T05:50:58.206 INFO:teuthology.orchestra.run.smithi047.stdout:Host looks OK 2024-04-17T05:50:58.206 INFO:teuthology.orchestra.run.smithi047.stdout:Cluster fsid: 565db2d8-fc7e-11ee-bc91-c7b262605968 2024-04-17T05:50:58.206 INFO:teuthology.orchestra.run.smithi047.stdout:Acquiring lock 140290005210016 on /run/cephadm/565db2d8-fc7e-11ee-bc91-c7b262605968.lock 2024-04-17T05:50:58.206 INFO:teuthology.orchestra.run.smithi047.stdout:Lock 140290005210016 acquired on /run/cephadm/565db2d8-fc7e-11ee-bc91-c7b262605968.lock 2024-04-17T05:50:58.207 INFO:teuthology.orchestra.run.smithi047.stdout:Verifying IP 172.21.15.47 port 3300 ... 2024-04-17T05:50:58.207 INFO:teuthology.orchestra.run.smithi047.stdout:Verifying IP 172.21.15.47 port 6789 ... 2024-04-17T05:50:58.207 INFO:teuthology.orchestra.run.smithi047.stdout:Base mon IP(s) is [172.21.15.47:3300, 172.21.15.47:6789], mon addrv is [v2:172.21.15.47:3300,v1:172.21.15.47:6789] 2024-04-17T05:50:58.210 INFO:teuthology.orchestra.run.smithi047.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.47 metric 100 2024-04-17T05:50:58.210 INFO:teuthology.orchestra.run.smithi047.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.47 metric 100 2024-04-17T05:50:58.213 INFO:teuthology.orchestra.run.smithi047.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-04-17T05:50:58.213 INFO:teuthology.orchestra.run.smithi047.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-04-17T05:50:58.213 INFO:teuthology.orchestra.run.smithi047.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-04-17T05:50:58.215 INFO:teuthology.orchestra.run.smithi047.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-04-17T05:50:58.215 INFO:teuthology.orchestra.run.smithi047.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-04-17T05:50:58.215 INFO:teuthology.orchestra.run.smithi047.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-17T05:50:58.215 INFO:teuthology.orchestra.run.smithi047.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-04-17T05:50:58.215 INFO:teuthology.orchestra.run.smithi047.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:148e/64 scope link noprefixroute 2024-04-17T05:50:58.216 INFO:teuthology.orchestra.run.smithi047.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-17T05:50:58.216 INFO:teuthology.orchestra.run.smithi047.stdout:Mon IP `172.21.15.47` is in CIDR network `172.21.0.0/20` 2024-04-17T05:50:58.216 INFO:teuthology.orchestra.run.smithi047.stdout:Mon IP `172.21.15.47` is in CIDR network `172.21.0.0/20` 2024-04-17T05:50:58.216 INFO:teuthology.orchestra.run.smithi047.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-04-17T05:50:58.217 INFO:teuthology.orchestra.run.smithi047.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-04-17T05:50:58.217 INFO:teuthology.orchestra.run.smithi047.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda... 2024-04-17T05:50:58.643 INFO:teuthology.orchestra.run.smithi047.stdout:/bin/podman: stdout 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d 2024-04-17T05:50:58.643 INFO:teuthology.orchestra.run.smithi047.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda... 2024-04-17T05:50:58.643 INFO:teuthology.orchestra.run.smithi047.stdout:/bin/podman: stderr Getting image source signatures 2024-04-17T05:50:58.643 INFO:teuthology.orchestra.run.smithi047.stdout:/bin/podman: stderr Copying blob sha256:9473d3fd79377e8e2904ab0f292b04da81cb2f2bfccde15ec927b481a725f6f4 2024-04-17T05:50:58.643 INFO:teuthology.orchestra.run.smithi047.stdout:/bin/podman: stderr Copying blob sha256:426fede9fffa2c7dfe40a4f9322a34794b72405170134d54d2c30d9cc6c6a5be 2024-04-17T05:50:58.643 INFO:teuthology.orchestra.run.smithi047.stdout:/bin/podman: stderr Copying config sha256:76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d 2024-04-17T05:50:58.643 INFO:teuthology.orchestra.run.smithi047.stdout:/bin/podman: stderr Writing manifest to image destination 2024-04-17T05:50:59.572 INFO:teuthology.orchestra.run.smithi047.stdout:ceph: stdout ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev) 2024-04-17T05:50:59.573 INFO:teuthology.orchestra.run.smithi047.stdout:Ceph version: ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev) 2024-04-17T05:50:59.573 INFO:teuthology.orchestra.run.smithi047.stdout:Extracting ceph user uid/gid from container image... 2024-04-17T05:51:07.684 INFO:teuthology.orchestra.run.smithi047.stdout:stat: stdout 167 167 2024-04-17T05:51:07.684 INFO:teuthology.orchestra.run.smithi047.stdout:Creating initial keys... 2024-04-17T05:51:13.816 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph-authtool: stdout AQBOYx9mKcUAExAAlt9p1+6uZdY05Dlp7/hBGg== 2024-04-17T05:51:15.297 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph-authtool: stdout AQBSYx9mTU41BxAAlv9O74mPDnv56lsVpM3rbw== 2024-04-17T05:51:16.144 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph-authtool: stdout AQBTYx9mZrbVJRAA9DsUNLhd+FxbN0dm/lH53A== 2024-04-17T05:51:16.145 INFO:teuthology.orchestra.run.smithi047.stdout:Creating initial monmap... 2024-04-17T05:51:17.041 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-17T05:51:17.041 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-04-17T05:51:17.042 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 565db2d8-fc7e-11ee-bc91-c7b262605968 2024-04-17T05:51:17.042 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-17T05:51:17.042 INFO:teuthology.orchestra.run.smithi047.stdout:monmaptool for a [v2:172.21.15.47:3300,v1:172.21.15.47:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-17T05:51:17.042 INFO:teuthology.orchestra.run.smithi047.stdout:setting min_mon_release = quincy 2024-04-17T05:51:17.042 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/monmaptool: set fsid to 565db2d8-fc7e-11ee-bc91-c7b262605968 2024-04-17T05:51:17.042 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-17T05:51:17.042 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-17T05:51:17.042 INFO:teuthology.orchestra.run.smithi047.stdout:Creating mon... 2024-04-17T05:51:17.939 INFO:teuthology.orchestra.run.smithi047.stdout:create mon.a on 2024-04-17T05:51:18.144 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-04-17T05:51:18.325 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-04-17T05:51:18.501 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-565db2d8-fc7e-11ee-bc91-c7b262605968.target → /etc/systemd/system/ceph-565db2d8-fc7e-11ee-bc91-c7b262605968.target. 2024-04-17T05:51:18.501 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-565db2d8-fc7e-11ee-bc91-c7b262605968.target → /etc/systemd/system/ceph-565db2d8-fc7e-11ee-bc91-c7b262605968.target. 2024-04-17T05:51:18.739 INFO:teuthology.orchestra.run.smithi047.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@mon.a 2024-04-17T05:51:18.739 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stderr Failed to reset failed state of unit ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@mon.a.service: Unit ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@mon.a.service not loaded. 2024-04-17T05:51:18.971 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-565db2d8-fc7e-11ee-bc91-c7b262605968.target.wants/ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@mon.a.service → /etc/systemd/system/ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@.service. 2024-04-17T05:51:19.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:19 smithi047 podman[31834]: 2024-04-17 05:51:19.156292523 +0000 UTC m=+0.017379425 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda 2024-04-17T05:51:19.913 INFO:teuthology.orchestra.run.smithi047.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-17T05:51:19.913 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stdout disabled 2024-04-17T05:51:19.920 INFO:teuthology.orchestra.run.smithi047.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-17T05:51:19.920 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stdout inactive 2024-04-17T05:51:19.920 INFO:teuthology.orchestra.run.smithi047.stdout:firewalld.service is not enabled 2024-04-17T05:51:19.920 INFO:teuthology.orchestra.run.smithi047.stdout:Not possible to enable service . firewalld.service is not available 2024-04-17T05:51:19.920 INFO:teuthology.orchestra.run.smithi047.stdout:Waiting for mon to start... 2024-04-17T05:51:19.920 INFO:teuthology.orchestra.run.smithi047.stdout:Waiting for mon... 2024-04-17T05:51:20.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:19 smithi047 podman[31834]: 2024-04-17 05:51:19.673400782 +0000 UTC m=+0.534487663 container create 9ad99d0cb556c680c9130a79ee04de960c94e8eb7d26665809d6633a23c4c62c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mon-a, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-04-17T05:51:20.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:19 smithi047 podman[31834]: 2024-04-17 05:51:19.842574919 +0000 UTC m=+0.703661797 container init 9ad99d0cb556c680c9130a79ee04de960c94e8eb7d26665809d6633a23c4c62c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mon-a, ceph=True, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-04-17T05:51:20.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:19 smithi047 podman[31834]: 2024-04-17 05:51:19.846066921 +0000 UTC m=+0.707153803 container start 9ad99d0cb556c680c9130a79ee04de960c94e8eb7d26665809d6633a23c4c62c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mon-a, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-04-17T05:51:20.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:19 smithi047 ceph-mon[31852]: mkfs 565db2d8-fc7e-11ee-bc91-c7b262605968 2024-04-17T05:51:20.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:19 smithi047 ceph-mon[31852]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-17T05:51:20.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:19 smithi047 bash[31834]: 9ad99d0cb556c680c9130a79ee04de960c94e8eb7d26665809d6633a23c4c62c 2024-04-17T05:51:20.026 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:19 smithi047 systemd[1]: Started Ceph mon.a for 565db2d8-fc7e-11ee-bc91-c7b262605968. 2024-04-17T05:51:20.928 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout cluster: 2024-04-17T05:51:20.928 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout id: 565db2d8-fc7e-11ee-bc91-c7b262605968 2024-04-17T05:51:20.928 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-04-17T05:51:20.928 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 2024-04-17T05:51:20.928 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout services: 2024-04-17T05:51:20.929 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.497477s) 2024-04-17T05:51:20.929 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-04-17T05:51:20.929 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-04-17T05:51:20.929 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 2024-04-17T05:51:20.929 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout data: 2024-04-17T05:51:20.929 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-04-17T05:51:20.929 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-04-17T05:51:20.929 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-04-17T05:51:20.929 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout pgs: 2024-04-17T05:51:20.929 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 2024-04-17T05:51:20.929 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:20.303+0000 7f8467e2a700 1 Processor -- start 2024-04-17T05:51:20.929 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:20.304+0000 7f8467e2a700 1 -- start start 2024-04-17T05:51:20.930 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:20.304+0000 7f8467e2a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8460109020 0x7f846010b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:51:20.930 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:20.304+0000 7f8467e2a700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f846007be50 con 0x7f8460109020 2024-04-17T05:51:20.930 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:20.304+0000 7f8465bc6700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8460109020 0x7f846010b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:51:20.930 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:20.304+0000 7f8465bc6700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8460109020 0x7f846010b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57678/0 (socket says 172.21.15.47:57678) 2024-04-17T05:51:20.930 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:20.304+0000 7f8465bc6700 1 -- 172.21.15.47:0/3230560798 learned_addr learned my addr 172.21.15.47:0/3230560798 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:51:20.930 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:20.305+0000 7f8465bc6700 1 -- 172.21.15.47:0/3230560798 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f846010b950 con 0x7f8460109020 2024-04-17T05:51:20.930 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:20.305+0000 7f8465bc6700 1 --2- 172.21.15.47:0/3230560798 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8460109020 0x7f846010b410 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f845c009e80 tx=0x7f845c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6037444d17eed032 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:51:20.930 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:20.305+0000 7f8464bc4700 1 -- 172.21.15.47:0/3230560798 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f845c017070 con 0x7f8460109020 2024-04-17T05:51:20.930 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:20.305+0000 7f8464bc4700 1 -- 172.21.15.47:0/3230560798 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f845c003a20 con 0x7f8460109020 2024-04-17T05:51:20.930 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:20.305+0000 7f8464bc4700 1 -- 172.21.15.47:0/3230560798 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f845c003d20 con 0x7f8460109020 2024-04-17T05:51:20.930 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:20.306+0000 7f8467e2a700 1 -- 172.21.15.47:0/3230560798 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8460109020 msgr2=0x7f846010b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:51:20.930 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:20.306+0000 7f8467e2a700 1 --2- 172.21.15.47:0/3230560798 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8460109020 0x7f846010b410 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f845c009e80 tx=0x7f845c00b330 comp rx=0 tx=0).stop 2024-04-17T05:51:20.931 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:20.306+0000 7f8467e2a700 1 -- 172.21.15.47:0/3230560798 shutdown_connections 2024-04-17T05:51:20.931 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:20.306+0000 7f8467e2a700 1 --2- 172.21.15.47:0/3230560798 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8460109020 0x7f846010b410 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:51:20.931 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:20.306+0000 7f8467e2a700 1 -- 172.21.15.47:0/3230560798 >> 172.21.15.47:0/3230560798 conn(0x7f8460102a10 msgr2=0x7f8460104e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:51:20.931 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:20.306+0000 7f8467e2a700 1 -- 172.21.15.47:0/3230560798 shutdown_connections 2024-04-17T05:51:20.931 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:20.306+0000 7f8467e2a700 1 -- 172.21.15.47:0/3230560798 wait complete. 2024-04-17T05:51:20.931 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:20.306+0000 7f8467e2a700 1 Processor -- start 2024-04-17T05:51:20.931 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:20.306+0000 7f8467e2a700 1 -- start start 2024-04-17T05:51:20.932 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:20.306+0000 7f8467e2a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8460109020 0x7f84601088a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:51:20.932 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:20.307+0000 7f8467e2a700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f846010bdc0 con 0x7f8460109020 2024-04-17T05:51:20.932 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:20.307+0000 7f8465bc6700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8460109020 0x7f84601088a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:51:20.932 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:20.307+0000 7f8465bc6700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8460109020 0x7f84601088a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57688/0 (socket says 172.21.15.47:57688) 2024-04-17T05:51:20.932 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:20.307+0000 7f8465bc6700 1 -- 172.21.15.47:0/760630984 learned_addr learned my addr 172.21.15.47:0/760630984 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:51:20.932 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:20.307+0000 7f8465bc6700 1 -- 172.21.15.47:0/760630984 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8460108de0 con 0x7f8460109020 2024-04-17T05:51:20.932 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:20.307+0000 7f8465bc6700 1 --2- 172.21.15.47:0/760630984 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8460109020 0x7f84601088a0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f846007b290 tx=0x7f845c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:51:20.932 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:20.307+0000 7f8456ffd700 1 -- 172.21.15.47:0/760630984 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f845c022020 con 0x7f8460109020 2024-04-17T05:51:20.932 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:20.307+0000 7f8456ffd700 1 -- 172.21.15.47:0/760630984 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f845c018040 con 0x7f8460109020 2024-04-17T05:51:20.932 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:20.307+0000 7f8456ffd700 1 -- 172.21.15.47:0/760630984 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f845c017070 con 0x7f8460109020 2024-04-17T05:51:20.932 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:20.307+0000 7f8467e2a700 1 -- 172.21.15.47:0/760630984 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8460106fd0 con 0x7f8460109020 2024-04-17T05:51:20.932 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:20.307+0000 7f8467e2a700 1 -- 172.21.15.47:0/760630984 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8460107510 con 0x7f8460109020 2024-04-17T05:51:20.933 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:20.308+0000 7f8456ffd700 1 -- 172.21.15.47:0/760630984 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f845c025020 con 0x7f8460109020 2024-04-17T05:51:20.933 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:20.308+0000 7f8456ffd700 1 -- 172.21.15.47:0/760630984 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f845c00f050 con 0x7f8460109020 2024-04-17T05:51:20.933 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:20.308+0000 7f8467e2a700 1 -- 172.21.15.47:0/760630984 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84600640b0 con 0x7f8460109020 2024-04-17T05:51:20.933 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:20.311+0000 7f8456ffd700 1 -- 172.21.15.47:0/760630984 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7f845c02c030 con 0x7f8460109020 2024-04-17T05:51:20.933 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:20.383+0000 7f8467e2a700 1 -- 172.21.15.47:0/760630984 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f84601955f0 con 0x7f8460109020 2024-04-17T05:51:20.933 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:20.383+0000 7f8456ffd700 1 -- 172.21.15.47:0/760630984 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f845c01e480 con 0x7f8460109020 2024-04-17T05:51:20.933 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:20.384+0000 7f8467e2a700 1 -- 172.21.15.47:0/760630984 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8460109020 msgr2=0x7f84601088a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:51:20.933 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:20.384+0000 7f8467e2a700 1 --2- 172.21.15.47:0/760630984 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8460109020 0x7f84601088a0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f846007b290 tx=0x7f845c004ad0 comp rx=0 tx=0).stop 2024-04-17T05:51:20.933 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:20.384+0000 7f8467e2a700 1 -- 172.21.15.47:0/760630984 shutdown_connections 2024-04-17T05:51:20.933 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:20.384+0000 7f8467e2a700 1 --2- 172.21.15.47:0/760630984 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8460109020 0x7f84601088a0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:51:20.933 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:20.384+0000 7f8467e2a700 1 -- 172.21.15.47:0/760630984 >> 172.21.15.47:0/760630984 conn(0x7f8460102a10 msgr2=0x7f8460104d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:51:20.934 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:20.384+0000 7f8467e2a700 1 -- 172.21.15.47:0/760630984 shutdown_connections 2024-04-17T05:51:20.934 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:20.384+0000 7f8467e2a700 1 -- 172.21.15.47:0/760630984 wait complete. 2024-04-17T05:51:20.934 INFO:teuthology.orchestra.run.smithi047.stdout:mon is available 2024-04-17T05:51:20.934 INFO:teuthology.orchestra.run.smithi047.stdout:Assimilating anything we can from ceph.conf... 2024-04-17T05:51:21.216 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:20 smithi047 ceph-mon[31852]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-17T05:51:21.216 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:20 smithi047 ceph-mon[31852]: monmap epoch 1 2024-04-17T05:51:21.216 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:20 smithi047 ceph-mon[31852]: fsid 565db2d8-fc7e-11ee-bc91-c7b262605968 2024-04-17T05:51:21.216 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:20 smithi047 ceph-mon[31852]: last_changed 2024-04-17T05:51:16.462437+0000 2024-04-17T05:51:21.216 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:20 smithi047 ceph-mon[31852]: created 2024-04-17T05:51:16.462437+0000 2024-04-17T05:51:21.216 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:20 smithi047 ceph-mon[31852]: min_mon_release 19 (squid) 2024-04-17T05:51:21.216 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:20 smithi047 ceph-mon[31852]: election_strategy: 1 2024-04-17T05:51:21.216 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:20 smithi047 ceph-mon[31852]: 0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-17T05:51:21.216 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:20 smithi047 ceph-mon[31852]: fsmap 2024-04-17T05:51:21.216 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:20 smithi047 ceph-mon[31852]: osdmap e1: 0 total, 0 up, 0 in 2024-04-17T05:51:21.217 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:20 smithi047 ceph-mon[31852]: mgrmap e1: no daemons active 2024-04-17T05:51:21.217 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:20 smithi047 ceph-mon[31852]: from='client.? 172.21.15.47:0/760630984' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-04-17T05:51:21.944 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 2024-04-17T05:51:21.945 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout [global] 2024-04-17T05:51:21.945 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout fsid = 565db2d8-fc7e-11ee-bc91-c7b262605968 2024-04-17T05:51:21.945 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.47:3300,v1:172.21.15.47:6789] 2024-04-17T05:51:21.945 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-17T05:51:21.945 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-17T05:51:21.945 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-17T05:51:21.945 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-17T05:51:21.945 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 2024-04-17T05:51:21.945 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-17T05:51:21.945 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-17T05:51:21.945 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 2024-04-17T05:51:21.946 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout [osd] 2024-04-17T05:51:21.946 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-17T05:51:21.946 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-17T05:51:21.946 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-17T05:51:21.946 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-17T05:51:21.946 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-17T05:51:21.946 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:21.318+0000 7f12172c7700 1 Processor -- start 2024-04-17T05:51:21.946 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:21.318+0000 7f12172c7700 1 -- start start 2024-04-17T05:51:21.946 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:21.318+0000 7f12172c7700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1210108540 0x7f1210108960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:51:21.946 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:21.318+0000 7f12172c7700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f1210108f30 con 0x7f1210108540 2024-04-17T05:51:21.946 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:21.318+0000 7f1215063700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1210108540 0x7f1210108960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:51:21.947 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:21.318+0000 7f1215063700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1210108540 0x7f1210108960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57700/0 (socket says 172.21.15.47:57700) 2024-04-17T05:51:21.947 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:21.318+0000 7f1215063700 1 -- 172.21.15.47:0/729177414 learned_addr learned my addr 172.21.15.47:0/729177414 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:51:21.947 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:21.319+0000 7f1215063700 1 -- 172.21.15.47:0/729177414 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12101096f0 con 0x7f1210108540 2024-04-17T05:51:21.947 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:21.319+0000 7f1215063700 1 --2- 172.21.15.47:0/729177414 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1210108540 0x7f1210108960 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f120c0098a0 tx=0x7f120c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfbdfcc04544f6ba server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:51:21.947 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:21.319+0000 7f1207fff700 1 -- 172.21.15.47:0/729177414 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f120c004160 con 0x7f1210108540 2024-04-17T05:51:21.948 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:21.319+0000 7f1207fff700 1 -- 172.21.15.47:0/729177414 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f120c004300 con 0x7f1210108540 2024-04-17T05:51:21.948 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:21.319+0000 7f1207fff700 1 -- 172.21.15.47:0/729177414 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f120c0036a0 con 0x7f1210108540 2024-04-17T05:51:21.948 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:21.319+0000 7f12172c7700 1 -- 172.21.15.47:0/729177414 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1210108540 msgr2=0x7f1210108960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:51:21.948 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:21.319+0000 7f12172c7700 1 --2- 172.21.15.47:0/729177414 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1210108540 0x7f1210108960 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f120c0098a0 tx=0x7f120c009bb0 comp rx=0 tx=0).stop 2024-04-17T05:51:21.948 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:21.320+0000 7f12172c7700 1 -- 172.21.15.47:0/729177414 shutdown_connections 2024-04-17T05:51:21.948 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:21.320+0000 7f12172c7700 1 --2- 172.21.15.47:0/729177414 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1210108540 0x7f1210108960 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:51:21.948 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:21.320+0000 7f12172c7700 1 -- 172.21.15.47:0/729177414 >> 172.21.15.47:0/729177414 conn(0x7f1210103b80 msgr2=0x7f1210105fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:51:21.948 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:21.320+0000 7f12172c7700 1 -- 172.21.15.47:0/729177414 shutdown_connections 2024-04-17T05:51:21.948 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:21.320+0000 7f12172c7700 1 -- 172.21.15.47:0/729177414 wait complete. 2024-04-17T05:51:21.948 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:21.320+0000 7f12172c7700 1 Processor -- start 2024-04-17T05:51:21.948 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:21.320+0000 7f12172c7700 1 -- start start 2024-04-17T05:51:21.948 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:21.320+0000 7f12172c7700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1210108540 0x7f121019bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:51:21.948 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:21.320+0000 7f12172c7700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f1210109c20 con 0x7f1210108540 2024-04-17T05:51:21.949 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:21.321+0000 7f1215063700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1210108540 0x7f121019bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:51:21.949 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:21.321+0000 7f1215063700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1210108540 0x7f121019bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57702/0 (socket says 172.21.15.47:57702) 2024-04-17T05:51:21.949 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:21.321+0000 7f1215063700 1 -- 172.21.15.47:0/139123705 learned_addr learned my addr 172.21.15.47:0/139123705 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:51:21.949 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:21.321+0000 7f1215063700 1 -- 172.21.15.47:0/139123705 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f121019c170 con 0x7f1210108540 2024-04-17T05:51:21.949 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:21.321+0000 7f1215063700 1 --2- 172.21.15.47:0/139123705 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1210108540 0x7f121019bc30 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f120c000c00 tx=0x7f120c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:51:21.949 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:21.321+0000 7f12067fc700 1 -- 172.21.15.47:0/139123705 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f120c003db0 con 0x7f1210108540 2024-04-17T05:51:21.949 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:21.321+0000 7f12067fc700 1 -- 172.21.15.47:0/139123705 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f120c011630 con 0x7f1210108540 2024-04-17T05:51:21.949 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:21.321+0000 7f12172c7700 1 -- 172.21.15.47:0/139123705 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f121019c400 con 0x7f1210108540 2024-04-17T05:51:21.949 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:21.321+0000 7f12067fc700 1 -- 172.21.15.47:0/139123705 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f120c011950 con 0x7f1210108540 2024-04-17T05:51:21.949 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:21.321+0000 7f12172c7700 1 -- 172.21.15.47:0/139123705 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f121019f0f0 con 0x7f1210108540 2024-04-17T05:51:21.949 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:21.321+0000 7f12067fc700 1 -- 172.21.15.47:0/139123705 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f120c011af0 con 0x7f1210108540 2024-04-17T05:51:21.949 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:21.321+0000 7f12067fc700 1 -- 172.21.15.47:0/139123705 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f120c01c450 con 0x7f1210108540 2024-04-17T05:51:21.949 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:21.322+0000 7f12172c7700 1 -- 172.21.15.47:0/139123705 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12100640b0 con 0x7f1210108540 2024-04-17T05:51:21.950 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:21.324+0000 7f12067fc700 1 -- 172.21.15.47:0/139123705 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7f120c032020 con 0x7f1210108540 2024-04-17T05:51:21.950 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:21.396+0000 7f12172c7700 1 -- 172.21.15.47:0/139123705 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f121002e1d0 con 0x7f1210108540 2024-04-17T05:51:21.950 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:21.399+0000 7f12067fc700 1 -- 172.21.15.47:0/139123705 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+429 (secure 0 0 0) 0x7f120c004260 con 0x7f1210108540 2024-04-17T05:51:21.950 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:21.400+0000 7f12067fc700 1 -- 172.21.15.47:0/139123705 <== mon.0 v2:172.21.15.47:3300/0 8 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f120c01cb60 con 0x7f1210108540 2024-04-17T05:51:21.950 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:21.400+0000 7f12172c7700 1 -- 172.21.15.47:0/139123705 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1210108540 msgr2=0x7f121019bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:51:21.950 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:21.400+0000 7f12172c7700 1 --2- 172.21.15.47:0/139123705 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1210108540 0x7f121019bc30 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f120c000c00 tx=0x7f120c004750 comp rx=0 tx=0).stop 2024-04-17T05:51:21.950 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:21.400+0000 7f12172c7700 1 -- 172.21.15.47:0/139123705 shutdown_connections 2024-04-17T05:51:21.950 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:21.400+0000 7f12172c7700 1 --2- 172.21.15.47:0/139123705 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1210108540 0x7f121019bc30 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:51:21.950 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:21.400+0000 7f12172c7700 1 -- 172.21.15.47:0/139123705 >> 172.21.15.47:0/139123705 conn(0x7f1210103b80 msgr2=0x7f1210192840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:51:21.950 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:21.400+0000 7f12172c7700 1 -- 172.21.15.47:0/139123705 shutdown_connections 2024-04-17T05:51:21.950 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:21.400+0000 7f12172c7700 1 -- 172.21.15.47:0/139123705 wait complete. 2024-04-17T05:51:21.951 INFO:teuthology.orchestra.run.smithi047.stdout:Generating new minimal ceph.conf... 2024-04-17T05:51:22.168 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:21 smithi047 ceph-mon[31852]: from='client.? 172.21.15.47:0/139123705' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-17T05:51:22.168 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:21 smithi047 ceph-mon[31852]: from='client.? 172.21.15.47:0/139123705' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-04-17T05:51:22.983 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:22.377+0000 7f29eb543700 1 Processor -- start 2024-04-17T05:51:22.983 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:22.377+0000 7f29eb543700 1 -- start start 2024-04-17T05:51:22.983 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:22.377+0000 7f29eb543700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f29e4108fb0 0x7f29e410b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:51:22.984 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:22.377+0000 7f29eb543700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f29e407be90 con 0x7f29e4108fb0 2024-04-17T05:51:22.984 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:22.378+0000 7f29e92df700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f29e4108fb0 0x7f29e410b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:51:22.984 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:22.378+0000 7f29e92df700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f29e4108fb0 0x7f29e410b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57712/0 (socket says 172.21.15.47:57712) 2024-04-17T05:51:22.984 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:22.378+0000 7f29e92df700 1 -- 172.21.15.47:0/488386742 learned_addr learned my addr 172.21.15.47:0/488386742 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:51:22.984 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:22.378+0000 7f29e92df700 1 -- 172.21.15.47:0/488386742 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29e410b8e0 con 0x7f29e4108fb0 2024-04-17T05:51:22.984 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:22.378+0000 7f29e92df700 1 --2- 172.21.15.47:0/488386742 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f29e4108fb0 0x7f29e410b3a0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f29d80098a0 tx=0x7f29d8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d22a5dc66e4893f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:51:22.984 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:22.378+0000 7f29d7fff700 1 -- 172.21.15.47:0/488386742 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f29d8004160 con 0x7f29e4108fb0 2024-04-17T05:51:22.984 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:22.378+0000 7f29d7fff700 1 -- 172.21.15.47:0/488386742 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f29d8004300 con 0x7f29e4108fb0 2024-04-17T05:51:22.984 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:22.378+0000 7f29d7fff700 1 -- 172.21.15.47:0/488386742 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f29d8011b50 con 0x7f29e4108fb0 2024-04-17T05:51:22.984 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:22.379+0000 7f29eb543700 1 -- 172.21.15.47:0/488386742 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f29e4108fb0 msgr2=0x7f29e410b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:51:22.984 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:22.379+0000 7f29eb543700 1 --2- 172.21.15.47:0/488386742 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f29e4108fb0 0x7f29e410b3a0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f29d80098a0 tx=0x7f29d8009bb0 comp rx=0 tx=0).stop 2024-04-17T05:51:22.984 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:22.379+0000 7f29eb543700 1 -- 172.21.15.47:0/488386742 shutdown_connections 2024-04-17T05:51:22.984 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:22.379+0000 7f29eb543700 1 --2- 172.21.15.47:0/488386742 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f29e4108fb0 0x7f29e410b3a0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:51:22.985 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:22.379+0000 7f29eb543700 1 -- 172.21.15.47:0/488386742 >> 172.21.15.47:0/488386742 conn(0x7f29e4102a00 msgr2=0x7f29e4104e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:51:22.985 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:22.379+0000 7f29eb543700 1 -- 172.21.15.47:0/488386742 shutdown_connections 2024-04-17T05:51:22.985 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:22.379+0000 7f29eb543700 1 -- 172.21.15.47:0/488386742 wait complete. 2024-04-17T05:51:22.985 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:22.380+0000 7f29eb543700 1 Processor -- start 2024-04-17T05:51:22.985 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:22.380+0000 7f29eb543700 1 -- start start 2024-04-17T05:51:22.985 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:22.380+0000 7f29eb543700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f29e4108fb0 0x7f29e419c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:51:22.985 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:22.380+0000 7f29eb543700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f29e410c900 con 0x7f29e4108fb0 2024-04-17T05:51:22.985 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:22.380+0000 7f29e92df700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f29e4108fb0 0x7f29e419c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:51:22.985 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:22.380+0000 7f29e92df700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f29e4108fb0 0x7f29e419c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57728/0 (socket says 172.21.15.47:57728) 2024-04-17T05:51:22.985 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:22.380+0000 7f29e92df700 1 -- 172.21.15.47:0/2219385533 learned_addr learned my addr 172.21.15.47:0/2219385533 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:51:22.985 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:22.380+0000 7f29e92df700 1 -- 172.21.15.47:0/2219385533 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29e419ca10 con 0x7f29e4108fb0 2024-04-17T05:51:22.986 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:22.380+0000 7f29e92df700 1 --2- 172.21.15.47:0/2219385533 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f29e4108fb0 0x7f29e419c4d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f29d8011670 tx=0x7f29d8011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:51:22.986 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:22.380+0000 7f29d67fc700 1 -- 172.21.15.47:0/2219385533 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f29d80039c0 con 0x7f29e4108fb0 2024-04-17T05:51:22.986 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:22.380+0000 7f29d67fc700 1 -- 172.21.15.47:0/2219385533 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f29d8011e50 con 0x7f29e4108fb0 2024-04-17T05:51:22.986 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:22.380+0000 7f29eb543700 1 -- 172.21.15.47:0/2219385533 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29e419cca0 con 0x7f29e4108fb0 2024-04-17T05:51:22.987 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:22.380+0000 7f29d67fc700 1 -- 172.21.15.47:0/2219385533 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f29d801a5c0 con 0x7f29e4108fb0 2024-04-17T05:51:22.987 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:22.381+0000 7f29eb543700 1 -- 172.21.15.47:0/2219385533 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29e419f990 con 0x7f29e4108fb0 2024-04-17T05:51:22.987 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:22.381+0000 7f29d67fc700 1 -- 172.21.15.47:0/2219385533 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f29d801ac50 con 0x7f29e4108fb0 2024-04-17T05:51:22.987 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:22.381+0000 7f29d67fc700 1 -- 172.21.15.47:0/2219385533 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f29d8019700 con 0x7f29e4108fb0 2024-04-17T05:51:22.987 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:22.381+0000 7f29eb543700 1 -- 172.21.15.47:0/2219385533 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29e40640b0 con 0x7f29e4108fb0 2024-04-17T05:51:22.987 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:22.384+0000 7f29d67fc700 1 -- 172.21.15.47:0/2219385533 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7f29d8003b60 con 0x7f29e4108fb0 2024-04-17T05:51:22.987 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:22.454+0000 7f29eb543700 1 -- 172.21.15.47:0/2219385533 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f29e419fb60 con 0x7f29e4108fb0 2024-04-17T05:51:22.988 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:22.454+0000 7f29d67fc700 1 -- 172.21.15.47:0/2219385533 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7f29d8019950 con 0x7f29e4108fb0 2024-04-17T05:51:22.988 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:22.455+0000 7f29eb543700 1 -- 172.21.15.47:0/2219385533 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f29e4108fb0 msgr2=0x7f29e419c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:51:22.988 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:22.455+0000 7f29eb543700 1 --2- 172.21.15.47:0/2219385533 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f29e4108fb0 0x7f29e419c4d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f29d8011670 tx=0x7f29d8011b60 comp rx=0 tx=0).stop 2024-04-17T05:51:22.988 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:22.455+0000 7f29eb543700 1 -- 172.21.15.47:0/2219385533 shutdown_connections 2024-04-17T05:51:22.988 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:22.455+0000 7f29eb543700 1 --2- 172.21.15.47:0/2219385533 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f29e4108fb0 0x7f29e419c4d0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:51:22.988 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:22.455+0000 7f29eb543700 1 -- 172.21.15.47:0/2219385533 >> 172.21.15.47:0/2219385533 conn(0x7f29e4102a00 msgr2=0x7f29e41035b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:51:22.988 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:22.455+0000 7f29eb543700 1 -- 172.21.15.47:0/2219385533 shutdown_connections 2024-04-17T05:51:22.988 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:22.455+0000 7f29eb543700 1 -- 172.21.15.47:0/2219385533 wait complete. 2024-04-17T05:51:22.988 INFO:teuthology.orchestra.run.smithi047.stdout:Restarting the monitor... 2024-04-17T05:51:23.246 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:22 smithi047 ceph-mon[31852]: from='client.? 172.21.15.47:0/2219385533' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:51:23.246 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:23 smithi047 systemd[1]: Stopping Ceph mon.a for 565db2d8-fc7e-11ee-bc91-c7b262605968... 2024-04-17T05:51:23.596 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:23 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mon-a[31848]: 2024-04-17T05:51:23.246+0000 7f0b903ae700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-17T05:51:23.596 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:23 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mon-a[31848]: 2024-04-17T05:51:23.246+0000 7f0b903ae700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-04-17T05:51:23.596 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:23 smithi047 podman[32097]: 2024-04-17 05:51:23.330042497 +0000 UTC m=+0.180567362 container died 9ad99d0cb556c680c9130a79ee04de960c94e8eb7d26665809d6633a23c4c62c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mon-a, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-17T05:51:23.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:23 smithi047 podman[32097]: 2024-04-17 05:51:23.697446219 +0000 UTC m=+0.547971072 container cleanup 9ad99d0cb556c680c9130a79ee04de960c94e8eb7d26665809d6633a23c4c62c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mon-a, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-04-17T05:51:23.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:23 smithi047 bash[32097]: ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mon-a 2024-04-17T05:51:24.271 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:23 smithi047 podman[32111]: 2024-04-17 05:51:23.931952591 +0000 UTC m=+0.604211667 container remove 9ad99d0cb556c680c9130a79ee04de960c94e8eb7d26665809d6633a23c4c62c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mon-a, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2) 2024-04-17T05:51:24.272 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 systemd[1]: ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@mon.a.service: Deactivated successfully. 2024-04-17T05:51:24.272 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 systemd[1]: Stopped Ceph mon.a for 565db2d8-fc7e-11ee-bc91-c7b262605968. 2024-04-17T05:51:24.272 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 systemd[1]: Starting Ceph mon.a for 565db2d8-fc7e-11ee-bc91-c7b262605968... 2024-04-17T05:51:24.524 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 podman[32204]: 2024-04-17 05:51:24.274226854 +0000 UTC m=+0.108883560 container create d01e5baa55cce7efcd28623ab8975076add23c7e219e8db2d41dc3fb0ba7723d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mon-a, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, ceph=True) 2024-04-17T05:51:24.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 podman[32204]: 2024-04-17 05:51:24.182962584 +0000 UTC m=+0.017619300 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda 2024-04-17T05:51:24.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 podman[32204]: 2024-04-17 05:51:24.468328785 +0000 UTC m=+0.302985492 container init d01e5baa55cce7efcd28623ab8975076add23c7e219e8db2d41dc3fb0ba7723d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mon-a, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-04-17T05:51:24.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 podman[32204]: 2024-04-17 05:51:24.471594071 +0000 UTC m=+0.306250778 container start d01e5baa55cce7efcd28623ab8975076add23c7e219e8db2d41dc3fb0ba7723d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mon-a, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-17T05:51:24.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: set uid:gid to 167:167 (ceph:ceph) 2024-04-17T05:51:24.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev), process ceph-mon, pid 2 2024-04-17T05:51:24.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: pidfile_write: ignore empty --pid-file 2024-04-17T05:51:24.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: load: jerasure load: lrc 2024-04-17T05:51:24.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: RocksDB version: 7.9.2 2024-04-17T05:51:24.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Git sha 0 2024-04-17T05:51:24.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Compile date 2024-04-17 00:07:32 2024-04-17T05:51:24.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: DB SUMMARY 2024-04-17T05:51:24.526 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: DB Session ID: 9GCT4XYE35W1TBQ67D1S 2024-04-17T05:51:24.526 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: CURRENT file: CURRENT 2024-04-17T05:51:24.526 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: IDENTITY file: IDENTITY 2024-04-17T05:51:24.526 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-04-17T05:51:24.526 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-04-17T05:51:24.526 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 105640 ; 2024-04-17T05:51:24.526 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.error_if_exists: 0 2024-04-17T05:51:24.526 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.create_if_missing: 0 2024-04-17T05:51:24.526 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.paranoid_checks: 1 2024-04-17T05:51:24.526 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-17T05:51:24.526 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-17T05:51:24.526 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-17T05:51:24.526 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.env: 0x55dbb45b4cc0 2024-04-17T05:51:24.527 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.fs: PosixFileSystem 2024-04-17T05:51:24.527 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.info_log: 0x55dbb619bc20 2024-04-17T05:51:24.527 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.max_file_opening_threads: 16 2024-04-17T05:51:24.527 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.statistics: (nil) 2024-04-17T05:51:24.527 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.use_fsync: 0 2024-04-17T05:51:24.527 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.max_log_file_size: 0 2024-04-17T05:51:24.527 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-17T05:51:24.527 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-17T05:51:24.527 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.keep_log_file_num: 1000 2024-04-17T05:51:24.527 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.recycle_log_file_num: 0 2024-04-17T05:51:24.527 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.allow_fallocate: 1 2024-04-17T05:51:24.527 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.allow_mmap_reads: 0 2024-04-17T05:51:24.527 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.allow_mmap_writes: 0 2024-04-17T05:51:24.527 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.use_direct_reads: 0 2024-04-17T05:51:24.527 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-17T05:51:24.528 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.create_missing_column_families: 0 2024-04-17T05:51:24.528 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.db_log_dir: 2024-04-17T05:51:24.528 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.wal_dir: 2024-04-17T05:51:24.528 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-17T05:51:24.528 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-17T05:51:24.528 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-17T05:51:24.528 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-17T05:51:24.528 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-17T05:51:24.528 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-17T05:51:24.528 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.advise_random_on_open: 1 2024-04-17T05:51:24.528 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.db_write_buffer_size: 0 2024-04-17T05:51:24.528 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.write_buffer_manager: 0x55dbb616a1e0 2024-04-17T05:51:24.528 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-17T05:51:24.528 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-17T05:51:24.528 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-17T05:51:24.529 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.rate_limiter: (nil) 2024-04-17T05:51:24.529 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-17T05:51:24.530 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.wal_recovery_mode: 2 2024-04-17T05:51:24.530 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.enable_thread_tracking: 0 2024-04-17T05:51:24.530 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.enable_pipelined_write: 0 2024-04-17T05:51:24.530 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.unordered_write: 0 2024-04-17T05:51:24.530 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-17T05:51:24.530 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-17T05:51:24.530 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-17T05:51:24.530 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-17T05:51:24.531 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.row_cache: None 2024-04-17T05:51:24.531 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.wal_filter: None 2024-04-17T05:51:24.531 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-17T05:51:24.531 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.allow_ingest_behind: 0 2024-04-17T05:51:24.531 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.two_write_queues: 0 2024-04-17T05:51:24.531 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.manual_wal_flush: 0 2024-04-17T05:51:24.531 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.wal_compression: 0 2024-04-17T05:51:24.531 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.atomic_flush: 0 2024-04-17T05:51:24.531 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-17T05:51:24.531 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-17T05:51:24.531 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-17T05:51:24.531 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.log_readahead_size: 0 2024-04-17T05:51:24.531 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-17T05:51:24.531 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.best_efforts_recovery: 0 2024-04-17T05:51:24.532 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-17T05:51:24.532 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-17T05:51:24.532 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.allow_data_in_errors: 0 2024-04-17T05:51:24.532 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.db_host_id: __hostname__ 2024-04-17T05:51:24.532 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-17T05:51:24.532 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.max_background_jobs: 2 2024-04-17T05:51:24.532 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.max_background_compactions: -1 2024-04-17T05:51:24.532 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.max_subcompactions: 1 2024-04-17T05:51:24.532 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-17T05:51:24.532 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-17T05:51:24.532 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-17T05:51:24.532 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.max_total_wal_size: 0 2024-04-17T05:51:24.532 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-17T05:51:24.532 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-17T05:51:24.532 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-17T05:51:24.533 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-17T05:51:24.533 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.max_open_files: -1 2024-04-17T05:51:24.533 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.bytes_per_sync: 0 2024-04-17T05:51:24.533 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-17T05:51:24.533 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-17T05:51:24.533 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.compaction_readahead_size: 0 2024-04-17T05:51:24.533 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.max_background_flushes: -1 2024-04-17T05:51:24.533 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Compression algorithms supported: 2024-04-17T05:51:24.533 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-17T05:51:24.533 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: kZSTD supported: 0 2024-04-17T05:51:24.533 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: kXpressCompression supported: 0 2024-04-17T05:51:24.533 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: kLZ4HCCompression supported: 1 2024-04-17T05:51:24.533 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: kZlibCompression supported: 1 2024-04-17T05:51:24.533 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: kSnappyCompression supported: 1 2024-04-17T05:51:24.534 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: kLZ4Compression supported: 1 2024-04-17T05:51:24.534 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: kBZip2Compression supported: 0 2024-04-17T05:51:24.534 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-17T05:51:24.534 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-17T05:51:24.534 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-04-17T05:51:24.534 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-17T05:51:24.534 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-17T05:51:24.534 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.merge_operator: 2024-04-17T05:51:24.534 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.compaction_filter: None 2024-04-17T05:51:24.534 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.compaction_filter_factory: None 2024-04-17T05:51:24.534 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.sst_partitioner_factory: None 2024-04-17T05:51:24.534 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-17T05:51:24.534 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-17T05:51:24.534 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55dbb619bde0) 2024-04-17T05:51:24.535 INFO:journalctl@ceph.mon.a.smithi047.stdout: cache_index_and_filter_blocks: 1 2024-04-17T05:51:24.535 INFO:journalctl@ceph.mon.a.smithi047.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-17T05:51:24.535 INFO:journalctl@ceph.mon.a.smithi047.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-17T05:51:24.535 INFO:journalctl@ceph.mon.a.smithi047.stdout: pin_top_level_index_and_filter: 1 2024-04-17T05:51:24.535 INFO:journalctl@ceph.mon.a.smithi047.stdout: index_type: 0 2024-04-17T05:51:24.535 INFO:journalctl@ceph.mon.a.smithi047.stdout: data_block_index_type: 0 2024-04-17T05:51:24.535 INFO:journalctl@ceph.mon.a.smithi047.stdout: index_shortening: 1 2024-04-17T05:51:24.535 INFO:journalctl@ceph.mon.a.smithi047.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-17T05:51:24.535 INFO:journalctl@ceph.mon.a.smithi047.stdout: checksum: 4 2024-04-17T05:51:24.535 INFO:journalctl@ceph.mon.a.smithi047.stdout: no_block_cache: 0 2024-04-17T05:51:24.535 INFO:journalctl@ceph.mon.a.smithi047.stdout: block_cache: 0x55dbb619d090 2024-04-17T05:51:24.535 INFO:journalctl@ceph.mon.a.smithi047.stdout: block_cache_name: BinnedLRUCache 2024-04-17T05:51:24.535 INFO:journalctl@ceph.mon.a.smithi047.stdout: block_cache_options: 2024-04-17T05:51:24.535 INFO:journalctl@ceph.mon.a.smithi047.stdout: capacity : 536870912 2024-04-17T05:51:24.536 INFO:journalctl@ceph.mon.a.smithi047.stdout: num_shard_bits : 4 2024-04-17T05:51:24.536 INFO:journalctl@ceph.mon.a.smithi047.stdout: strict_capacity_limit : 0 2024-04-17T05:51:24.536 INFO:journalctl@ceph.mon.a.smithi047.stdout: high_pri_pool_ratio: 0.000 2024-04-17T05:51:24.536 INFO:journalctl@ceph.mon.a.smithi047.stdout: block_cache_compressed: (nil) 2024-04-17T05:51:24.536 INFO:journalctl@ceph.mon.a.smithi047.stdout: persistent_cache: (nil) 2024-04-17T05:51:24.536 INFO:journalctl@ceph.mon.a.smithi047.stdout: block_size: 4096 2024-04-17T05:51:24.536 INFO:journalctl@ceph.mon.a.smithi047.stdout: block_size_deviation: 10 2024-04-17T05:51:24.536 INFO:journalctl@ceph.mon.a.smithi047.stdout: block_restart_interval: 16 2024-04-17T05:51:24.536 INFO:journalctl@ceph.mon.a.smithi047.stdout: index_block_restart_interval: 1 2024-04-17T05:51:24.536 INFO:journalctl@ceph.mon.a.smithi047.stdout: metadata_block_size: 4096 2024-04-17T05:51:24.536 INFO:journalctl@ceph.mon.a.smithi047.stdout: partition_filters: 0 2024-04-17T05:51:24.536 INFO:journalctl@ceph.mon.a.smithi047.stdout: use_delta_encoding: 1 2024-04-17T05:51:24.536 INFO:journalctl@ceph.mon.a.smithi047.stdout: filter_policy: bloomfilter 2024-04-17T05:51:24.536 INFO:journalctl@ceph.mon.a.smithi047.stdout: whole_key_filtering: 1 2024-04-17T05:51:24.536 INFO:journalctl@ceph.mon.a.smithi047.stdout: verify_compression: 0 2024-04-17T05:51:24.537 INFO:journalctl@ceph.mon.a.smithi047.stdout: read_amp_bytes_per_bit: 0 2024-04-17T05:51:24.537 INFO:journalctl@ceph.mon.a.smithi047.stdout: format_version: 5 2024-04-17T05:51:24.537 INFO:journalctl@ceph.mon.a.smithi047.stdout: enable_index_compression: 1 2024-04-17T05:51:24.537 INFO:journalctl@ceph.mon.a.smithi047.stdout: block_align: 0 2024-04-17T05:51:24.537 INFO:journalctl@ceph.mon.a.smithi047.stdout: max_auto_readahead_size: 262144 2024-04-17T05:51:24.537 INFO:journalctl@ceph.mon.a.smithi047.stdout: prepopulate_block_cache: 0 2024-04-17T05:51:24.537 INFO:journalctl@ceph.mon.a.smithi047.stdout: initial_auto_readahead_size: 8192 2024-04-17T05:51:24.537 INFO:journalctl@ceph.mon.a.smithi047.stdout: num_file_reads_for_auto_readahead: 2 2024-04-17T05:51:24.537 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.write_buffer_size: 33554432 2024-04-17T05:51:24.537 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.max_write_buffer_number: 2 2024-04-17T05:51:24.537 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.compression: NoCompression 2024-04-17T05:51:24.537 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.bottommost_compression: Disabled 2024-04-17T05:51:24.537 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.prefix_extractor: nullptr 2024-04-17T05:51:24.537 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-17T05:51:24.538 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.num_levels: 7 2024-04-17T05:51:24.538 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-17T05:51:24.538 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-17T05:51:24.538 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-17T05:51:24.538 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-17T05:51:24.538 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-17T05:51:24.538 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-17T05:51:24.538 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-17T05:51:24.538 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-17T05:51:24.538 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-17T05:51:24.538 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-17T05:51:24.538 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-17T05:51:24.538 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-17T05:51:24.538 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-17T05:51:24.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.compression_opts.level: 32767 2024-04-17T05:51:24.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.compression_opts.strategy: 0 2024-04-17T05:51:24.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-17T05:51:24.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-17T05:51:24.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-17T05:51:24.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-17T05:51:24.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.compression_opts.enabled: false 2024-04-17T05:51:24.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-17T05:51:24.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-17T05:51:24.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-17T05:51:24.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-17T05:51:24.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.target_file_size_base: 67108864 2024-04-17T05:51:24.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-17T05:51:24.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-17T05:51:24.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-17T05:51:24.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-17T05:51:24.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-17T05:51:24.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-17T05:51:24.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-17T05:51:24.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-17T05:51:24.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-17T05:51:24.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-17T05:51:24.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-17T05:51:24.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-17T05:51:24.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-17T05:51:24.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-17T05:51:24.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.arena_block_size: 1048576 2024-04-17T05:51:24.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-17T05:51:24.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-17T05:51:24.541 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.disable_auto_compactions: 0 2024-04-17T05:51:24.541 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-17T05:51:24.541 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-17T05:51:24.541 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-17T05:51:24.541 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-17T05:51:24.541 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-17T05:51:24.541 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-17T05:51:24.541 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-17T05:51:24.541 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-17T05:51:24.541 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-17T05:51:24.541 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-17T05:51:24.541 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-17T05:51:24.541 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.inplace_update_support: 0 2024-04-17T05:51:24.541 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-17T05:51:24.541 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-17T05:51:24.542 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-17T05:51:24.542 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-17T05:51:24.542 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.bloom_locality: 0 2024-04-17T05:51:24.542 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.max_successive_merges: 0 2024-04-17T05:51:24.542 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-17T05:51:24.542 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.paranoid_file_checks: 0 2024-04-17T05:51:24.542 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.force_consistency_checks: 1 2024-04-17T05:51:24.542 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.report_bg_io_stats: 0 2024-04-17T05:51:24.542 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.ttl: 2592000 2024-04-17T05:51:24.542 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-17T05:51:24.542 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-17T05:51:24.542 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-17T05:51:24.545 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.enable_blob_files: false 2024-04-17T05:51:24.546 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.min_blob_size: 0 2024-04-17T05:51:24.546 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.blob_file_size: 268435456 2024-04-17T05:51:24.547 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-17T05:51:24.547 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-17T05:51:24.547 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-17T05:51:24.547 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-17T05:51:24.547 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-17T05:51:24.547 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.blob_file_starting_level: 0 2024-04-17T05:51:24.547 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-17T05:51:24.547 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-04-17T05:51:24.548 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-04-17T05:51:24.548 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 056794c3-dd9e-4d23-9c77-c466576c1698 2024-04-17T05:51:24.548 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713333084500825, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-04-17T05:51:24.548 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-04-17T05:51:24.548 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713333084502373, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 100709, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 328, "table_properties": {"data_size": 98603, "index_size": 298, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14306, "raw_average_key_size": 49, "raw_value_size": 90849, "raw_average_value_size": 316, "num_data_blocks": 12, "num_entries": 287, "num_filter_entries": 287, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1713333084, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "056794c3-dd9e-4d23-9c77-c466576c1698", "db_session_id": "9GCT4XYE35W1TBQ67D1S", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-04-17T05:51:24.548 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713333084502482, "job": 1, "event": "recovery_finished"} 2024-04-17T05:51:24.548 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-04-17T05:51:24.548 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-17T05:51:24.548 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55dbb628a000 2024-04-17T05:51:24.549 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: DB pointer 0x55dbb6274000 2024-04-17T05:51:24.549 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-17T05:51:24.549 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-17T05:51:24.549 INFO:journalctl@ceph.mon.a.smithi047.stdout: ** DB Stats ** 2024-04-17T05:51:24.549 INFO:journalctl@ceph.mon.a.smithi047.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-17T05:51:24.549 INFO:journalctl@ceph.mon.a.smithi047.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-17T05:51:24.549 INFO:journalctl@ceph.mon.a.smithi047.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-17T05:51:24.549 INFO:journalctl@ceph.mon.a.smithi047.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-17T05:51:24.549 INFO:journalctl@ceph.mon.a.smithi047.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-17T05:51:24.549 INFO:journalctl@ceph.mon.a.smithi047.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-17T05:51:24.549 INFO:journalctl@ceph.mon.a.smithi047.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-17T05:51:24.549 INFO:journalctl@ceph.mon.a.smithi047.stdout: 2024-04-17T05:51:24.550 INFO:journalctl@ceph.mon.a.smithi047.stdout: ** Compaction Stats [default] ** 2024-04-17T05:51:24.550 INFO:journalctl@ceph.mon.a.smithi047.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-17T05:51:24.550 INFO:journalctl@ceph.mon.a.smithi047.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-17T05:51:24.550 INFO:journalctl@ceph.mon.a.smithi047.stdout: L0 2/0 100.21 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 99.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-17T05:51:24.550 INFO:journalctl@ceph.mon.a.smithi047.stdout: Sum 2/0 100.21 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 99.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-17T05:51:24.550 INFO:journalctl@ceph.mon.a.smithi047.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 99.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-17T05:51:24.550 INFO:journalctl@ceph.mon.a.smithi047.stdout: 2024-04-17T05:51:24.550 INFO:journalctl@ceph.mon.a.smithi047.stdout: ** Compaction Stats [default] ** 2024-04-17T05:51:24.551 INFO:journalctl@ceph.mon.a.smithi047.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-17T05:51:24.551 INFO:journalctl@ceph.mon.a.smithi047.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-17T05:51:24.551 INFO:journalctl@ceph.mon.a.smithi047.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 99.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-17T05:51:24.551 INFO:journalctl@ceph.mon.a.smithi047.stdout: 2024-04-17T05:51:24.551 INFO:journalctl@ceph.mon.a.smithi047.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-17T05:51:24.551 INFO:journalctl@ceph.mon.a.smithi047.stdout: 2024-04-17T05:51:24.551 INFO:journalctl@ceph.mon.a.smithi047.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-17T05:51:24.551 INFO:journalctl@ceph.mon.a.smithi047.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-17T05:51:24.551 INFO:journalctl@ceph.mon.a.smithi047.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-17T05:51:24.551 INFO:journalctl@ceph.mon.a.smithi047.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-17T05:51:24.551 INFO:journalctl@ceph.mon.a.smithi047.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-17T05:51:24.551 INFO:journalctl@ceph.mon.a.smithi047.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-17T05:51:24.551 INFO:journalctl@ceph.mon.a.smithi047.stdout: Cumulative compaction: 0.00 GB write, 20.78 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-17T05:51:24.552 INFO:journalctl@ceph.mon.a.smithi047.stdout: Interval compaction: 0.00 GB write, 20.78 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-17T05:51:24.552 INFO:journalctl@ceph.mon.a.smithi047.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-17T05:51:24.552 INFO:journalctl@ceph.mon.a.smithi047.stdout: Block cache BinnedLRUCache@0x55dbb619d090#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-04-17T05:51:24.552 INFO:journalctl@ceph.mon.a.smithi047.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-04-17T05:51:24.552 INFO:journalctl@ceph.mon.a.smithi047.stdout: 2024-04-17T05:51:24.552 INFO:journalctl@ceph.mon.a.smithi047.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-17T05:51:24.552 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: starting mon.a rank 0 at public addrs [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] at bind addrs [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 565db2d8-fc7e-11ee-bc91-c7b262605968 2024-04-17T05:51:24.552 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: mon.a@-1(???) e1 preinit fsid 565db2d8-fc7e-11ee-bc91-c7b262605968 2024-04-17T05:51:24.552 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: mon.a@-1(???).mds e1 new map 2024-04-17T05:51:24.552 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: mon.a@-1(???).mds e1 print_map 2024-04-17T05:51:24.552 INFO:journalctl@ceph.mon.a.smithi047.stdout: e1 2024-04-17T05:51:24.552 INFO:journalctl@ceph.mon.a.smithi047.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-17T05:51:24.553 INFO:journalctl@ceph.mon.a.smithi047.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-17T05:51:24.553 INFO:journalctl@ceph.mon.a.smithi047.stdout: legacy client fscid: -1 2024-04-17T05:51:24.553 INFO:journalctl@ceph.mon.a.smithi047.stdout: 2024-04-17T05:51:24.553 INFO:journalctl@ceph.mon.a.smithi047.stdout: No filesystems configured 2024-04-17T05:51:24.553 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-17T05:51:24.553 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-17T05:51:24.553 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-17T05:51:24.553 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-17T05:51:24.553 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-04-17T05:51:24.553 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 bash[32204]: d01e5baa55cce7efcd28623ab8975076add23c7e219e8db2d41dc3fb0ba7723d 2024-04-17T05:51:24.553 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 systemd[1]: Started Ceph mon.a for 565db2d8-fc7e-11ee-bc91-c7b262605968. 2024-04-17T05:51:24.554 INFO:teuthology.orchestra.run.smithi047.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-04-17T05:51:24.835 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-17T05:51:24.835 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: monmap epoch 1 2024-04-17T05:51:24.835 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: fsid 565db2d8-fc7e-11ee-bc91-c7b262605968 2024-04-17T05:51:24.835 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: last_changed 2024-04-17T05:51:16.462437+0000 2024-04-17T05:51:24.835 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: created 2024-04-17T05:51:16.462437+0000 2024-04-17T05:51:24.835 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: min_mon_release 19 (squid) 2024-04-17T05:51:24.836 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: election_strategy: 1 2024-04-17T05:51:24.836 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: 0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-17T05:51:24.836 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: fsmap 2024-04-17T05:51:24.836 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: osdmap e1: 0 total, 0 up, 0 in 2024-04-17T05:51:24.836 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:24 smithi047 ceph-mon[32223]: mgrmap e1: no daemons active 2024-04-17T05:51:25.566 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:24.941+0000 7f703d1f6700 1 Processor -- start 2024-04-17T05:51:25.566 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:24.941+0000 7f703d1f6700 1 -- start start 2024-04-17T05:51:25.566 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:24.941+0000 7f703d1f6700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7038107400 0x7f7038107820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:51:25.566 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:24.941+0000 7f703d1f6700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f7038107df0 con 0x7f7038107400 2024-04-17T05:51:25.566 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:24.941+0000 7f7036d9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7038107400 0x7f7038107820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:51:25.566 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:24.941+0000 7f7036d9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7038107400 0x7f7038107820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48622/0 (socket says 172.21.15.47:48622) 2024-04-17T05:51:25.567 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:24.941+0000 7f7036d9d700 1 -- 172.21.15.47:0/2616881042 learned_addr learned my addr 172.21.15.47:0/2616881042 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:51:25.567 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:24.942+0000 7f7036d9d700 1 -- 172.21.15.47:0/2616881042 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7038108600 con 0x7f7038107400 2024-04-17T05:51:25.567 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:24.942+0000 7f7036d9d700 1 --2- 172.21.15.47:0/2616881042 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7038107400 0x7f7038107820 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f70280098a0 tx=0x7f7028009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c02815ea0ed0693c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:51:25.567 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:24.942+0000 7f7035d9b700 1 -- 172.21.15.47:0/2616881042 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7028004160 con 0x7f7038107400 2024-04-17T05:51:25.567 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:24.942+0000 7f7035d9b700 1 -- 172.21.15.47:0/2616881042 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7028004300 con 0x7f7038107400 2024-04-17T05:51:25.567 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:24.942+0000 7f7035d9b700 1 -- 172.21.15.47:0/2616881042 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7028011b50 con 0x7f7038107400 2024-04-17T05:51:25.567 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:24.942+0000 7f703d1f6700 1 -- 172.21.15.47:0/2616881042 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7038107400 msgr2=0x7f7038107820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:51:25.567 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:24.942+0000 7f703d1f6700 1 --2- 172.21.15.47:0/2616881042 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7038107400 0x7f7038107820 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f70280098a0 tx=0x7f7028009bb0 comp rx=0 tx=0).stop 2024-04-17T05:51:25.567 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:24.943+0000 7f703d1f6700 1 -- 172.21.15.47:0/2616881042 shutdown_connections 2024-04-17T05:51:25.567 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:24.943+0000 7f703d1f6700 1 --2- 172.21.15.47:0/2616881042 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7038107400 0x7f7038107820 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:51:25.567 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:24.943+0000 7f703d1f6700 1 -- 172.21.15.47:0/2616881042 >> 172.21.15.47:0/2616881042 conn(0x7f7038102a00 msgr2=0x7f7038104e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:51:25.567 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:24.943+0000 7f703d1f6700 1 -- 172.21.15.47:0/2616881042 shutdown_connections 2024-04-17T05:51:25.568 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:24.943+0000 7f703d1f6700 1 -- 172.21.15.47:0/2616881042 wait complete. 2024-04-17T05:51:25.568 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:24.943+0000 7f703d1f6700 1 Processor -- start 2024-04-17T05:51:25.568 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:24.944+0000 7f703d1f6700 1 -- start start 2024-04-17T05:51:25.568 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:24.944+0000 7f703d1f6700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7038107400 0x7f70381925b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:51:25.568 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:24.944+0000 7f703d1f6700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f703810b9b0 con 0x7f7038107400 2024-04-17T05:51:25.568 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:24.944+0000 7f7036d9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7038107400 0x7f70381925b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:51:25.568 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:24.944+0000 7f7036d9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7038107400 0x7f70381925b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48624/0 (socket says 172.21.15.47:48624) 2024-04-17T05:51:25.568 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:24.944+0000 7f7036d9d700 1 -- 172.21.15.47:0/3430061871 learned_addr learned my addr 172.21.15.47:0/3430061871 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:51:25.568 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:24.944+0000 7f7036d9d700 1 -- 172.21.15.47:0/3430061871 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7038192af0 con 0x7f7038107400 2024-04-17T05:51:25.568 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:24.944+0000 7f7036d9d700 1 --2- 172.21.15.47:0/3430061871 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7038107400 0x7f70381925b0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7028011670 tx=0x7f7028011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:51:25.568 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:24.944+0000 7f701ffff700 1 -- 172.21.15.47:0/3430061871 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f70280039c0 con 0x7f7038107400 2024-04-17T05:51:25.568 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:24.945+0000 7f701ffff700 1 -- 172.21.15.47:0/3430061871 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7028011e50 con 0x7f7038107400 2024-04-17T05:51:25.568 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:24.945+0000 7f703d1f6700 1 -- 172.21.15.47:0/3430061871 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7038190ce0 con 0x7f7038107400 2024-04-17T05:51:25.568 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:24.945+0000 7f701ffff700 1 -- 172.21.15.47:0/3430061871 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f702801a5c0 con 0x7f7038107400 2024-04-17T05:51:25.569 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:24.945+0000 7f703d1f6700 1 -- 172.21.15.47:0/3430061871 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7038191220 con 0x7f7038107400 2024-04-17T05:51:25.569 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:24.945+0000 7f701ffff700 1 -- 172.21.15.47:0/3430061871 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f702801ac50 con 0x7f7038107400 2024-04-17T05:51:25.569 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:24.945+0000 7f701ffff700 1 -- 172.21.15.47:0/3430061871 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7028019700 con 0x7f7038107400 2024-04-17T05:51:25.569 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:24.946+0000 7f703d1f6700 1 -- 172.21.15.47:0/3430061871 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70380640b0 con 0x7f7038107400 2024-04-17T05:51:25.569 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:24.948+0000 7f701ffff700 1 -- 172.21.15.47:0/3430061871 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7f7028003b60 con 0x7f7038107400 2024-04-17T05:51:25.569 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:25.018+0000 7f703d1f6700 1 -- 172.21.15.47:0/3430061871 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f70381962b0 con 0x7f7038107400 2024-04-17T05:51:25.569 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:25.020+0000 7f701ffff700 1 -- 172.21.15.47:0/3430061871 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f7028019900 con 0x7f7038107400 2024-04-17T05:51:25.569 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:25.020+0000 7f703d1f6700 1 -- 172.21.15.47:0/3430061871 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7038107400 msgr2=0x7f70381925b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:51:25.569 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:25.020+0000 7f703d1f6700 1 --2- 172.21.15.47:0/3430061871 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7038107400 0x7f70381925b0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7028011670 tx=0x7f7028011b60 comp rx=0 tx=0).stop 2024-04-17T05:51:25.569 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:25.021+0000 7f703d1f6700 1 -- 172.21.15.47:0/3430061871 shutdown_connections 2024-04-17T05:51:25.569 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:25.021+0000 7f703d1f6700 1 --2- 172.21.15.47:0/3430061871 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7038107400 0x7f70381925b0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:51:25.569 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:25.021+0000 7f703d1f6700 1 -- 172.21.15.47:0/3430061871 >> 172.21.15.47:0/3430061871 conn(0x7f7038102a00 msgr2=0x7f7038103940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:51:25.570 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:25.021+0000 7f703d1f6700 1 -- 172.21.15.47:0/3430061871 shutdown_connections 2024-04-17T05:51:25.570 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:25.021+0000 7f703d1f6700 1 -- 172.21.15.47:0/3430061871 wait complete. 2024-04-17T05:51:25.570 INFO:teuthology.orchestra.run.smithi047.stdout:Wrote config to /etc/ceph/ceph.conf 2024-04-17T05:51:25.597 INFO:teuthology.orchestra.run.smithi047.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-04-17T05:51:25.597 INFO:teuthology.orchestra.run.smithi047.stdout:Creating mgr... 2024-04-17T05:51:25.597 INFO:teuthology.orchestra.run.smithi047.stdout:Verifying port 0.0.0.0:9283 ... 2024-04-17T05:51:25.598 INFO:teuthology.orchestra.run.smithi047.stdout:Verifying port 0.0.0.0:8765 ... 2024-04-17T05:51:25.851 INFO:teuthology.orchestra.run.smithi047.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@mgr.x 2024-04-17T05:51:25.851 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stderr Failed to reset failed state of unit ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@mgr.x.service: Unit ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@mgr.x.service not loaded. 2024-04-17T05:51:26.049 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-565db2d8-fc7e-11ee-bc91-c7b262605968.target.wants/ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@.service. 2024-04-17T05:51:26.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:26 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/3430061871' entity='client.admin' 2024-04-17T05:51:26.139 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 05:51:26 smithi047 systemd[1]: Starting Ceph mgr.x for 565db2d8-fc7e-11ee-bc91-c7b262605968... 2024-04-17T05:51:26.543 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 05:51:26 smithi047 podman[32438]: 2024-04-17 05:51:26.185683597 +0000 UTC m=+0.017549845 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda 2024-04-17T05:51:26.543 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 05:51:26 smithi047 podman[32438]: 2024-04-17 05:51:26.302024862 +0000 UTC m=+0.133891096 container create b8f280f6ff25fce571f13037d77ede78db5b2c8ce28f0135be7d9abc3d5c33ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-17T05:51:26.543 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 05:51:26 smithi047 podman[32438]: 2024-04-17 05:51:26.46235689 +0000 UTC m=+0.294223124 container init b8f280f6ff25fce571f13037d77ede78db5b2c8ce28f0135be7d9abc3d5c33ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_BRANCH=HEAD) 2024-04-17T05:51:26.543 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 05:51:26 smithi047 podman[32438]: 2024-04-17 05:51:26.465099641 +0000 UTC m=+0.296965883 container start b8f280f6ff25fce571f13037d77ede78db5b2c8ce28f0135be7d9abc3d5c33ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_BRANCH=HEAD) 2024-04-17T05:51:26.556 INFO:teuthology.orchestra.run.smithi047.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-17T05:51:26.556 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stdout disabled 2024-04-17T05:51:26.564 INFO:teuthology.orchestra.run.smithi047.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-17T05:51:26.564 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stdout inactive 2024-04-17T05:51:26.564 INFO:teuthology.orchestra.run.smithi047.stdout:firewalld.service is not enabled 2024-04-17T05:51:26.564 INFO:teuthology.orchestra.run.smithi047.stdout:Not possible to enable service . firewalld.service is not available 2024-04-17T05:51:26.572 INFO:teuthology.orchestra.run.smithi047.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-17T05:51:26.572 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stdout disabled 2024-04-17T05:51:26.580 INFO:teuthology.orchestra.run.smithi047.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-17T05:51:26.580 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stdout inactive 2024-04-17T05:51:26.580 INFO:teuthology.orchestra.run.smithi047.stdout:firewalld.service is not enabled 2024-04-17T05:51:26.580 INFO:teuthology.orchestra.run.smithi047.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-04-17T05:51:26.581 INFO:teuthology.orchestra.run.smithi047.stdout:Waiting for mgr to start... 2024-04-17T05:51:26.581 INFO:teuthology.orchestra.run.smithi047.stdout:Waiting for mgr... 2024-04-17T05:51:26.833 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 05:51:26 smithi047 bash[32438]: b8f280f6ff25fce571f13037d77ede78db5b2c8ce28f0135be7d9abc3d5c33ff 2024-04-17T05:51:26.833 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 05:51:26 smithi047 systemd[1]: Started Ceph mgr.x for 565db2d8-fc7e-11ee-bc91-c7b262605968. 2024-04-17T05:51:27.105 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 05:51:27 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T05:51:27.033+0000 7f59b9068200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-17T05:51:27.388 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:27 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/3422926166' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-17T05:51:27.389 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 05:51:27 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T05:51:27.137+0000 7f59b9068200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-17T05:51:27.629 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 2024-04-17T05:51:27.629 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout { 2024-04-17T05:51:27.629 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "fsid": "565db2d8-fc7e-11ee-bc91-c7b262605968", 2024-04-17T05:51:27.629 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "health": { 2024-04-17T05:51:27.629 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-17T05:51:27.629 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-17T05:51:27.629 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-17T05:51:27.630 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-04-17T05:51:27.630 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-17T05:51:27.630 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-17T05:51:27.630 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 0 2024-04-17T05:51:27.630 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout ], 2024-04-17T05:51:27.630 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-17T05:51:27.630 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "a" 2024-04-17T05:51:27.630 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout ], 2024-04-17T05:51:27.630 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-04-17T05:51:27.630 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-17T05:51:27.630 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-17T05:51:27.630 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-17T05:51:27.630 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-17T05:51:27.630 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-04-17T05:51:27.631 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-17T05:51:27.631 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-17T05:51:27.631 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-17T05:51:27.631 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-17T05:51:27.631 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-17T05:51:27.631 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-17T05:51:27.638 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-17T05:51:27.639 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-17T05:51:27.639 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-04-17T05:51:27.639 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-17T05:51:27.639 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-17T05:51:27.639 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-17T05:51:27.639 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-17T05:51:27.639 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-17T05:51:27.639 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-17T05:51:27.639 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-17T05:51:27.639 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-17T05:51:27.639 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-17T05:51:27.639 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-04-17T05:51:27.639 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-17T05:51:27.639 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-17T05:51:27.640 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-17T05:51:27.640 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-17T05:51:27.640 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-04-17T05:51:27.640 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-17T05:51:27.640 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-17T05:51:27.640 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-17T05:51:27.640 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-17T05:51:27.640 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-17T05:51:27.640 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-17T05:51:27.640 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "restful" 2024-04-17T05:51:27.640 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout ], 2024-04-17T05:51:27.640 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-17T05:51:27.640 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-04-17T05:51:27.640 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-17T05:51:27.641 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-17T05:51:27.641 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "modified": "2024-04-17T05:51:19.889468+0000", 2024-04-17T05:51:27.641 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-17T05:51:27.641 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-04-17T05:51:27.641 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-17T05:51:27.641 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout } 2024-04-17T05:51:27.641 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:27.024+0000 7f1f1880e700 1 Processor -- start 2024-04-17T05:51:27.641 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:27.024+0000 7f1f1880e700 1 -- start start 2024-04-17T05:51:27.641 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:27.025+0000 7f1f1880e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f14108580 0x7f1f141089a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:51:27.641 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:27.025+0000 7f1f1880e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f1f14108f70 con 0x7f1f14108580 2024-04-17T05:51:27.641 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:27.025+0000 7f1f1259c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f14108580 0x7f1f141089a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:51:27.641 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:27.025+0000 7f1f1259c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f14108580 0x7f1f141089a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48644/0 (socket says 172.21.15.47:48644) 2024-04-17T05:51:27.641 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:27.025+0000 7f1f1259c700 1 -- 172.21.15.47:0/1935442789 learned_addr learned my addr 172.21.15.47:0/1935442789 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:51:27.641 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:27.025+0000 7f1f1259c700 1 -- 172.21.15.47:0/1935442789 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f141096e0 con 0x7f1f14108580 2024-04-17T05:51:27.641 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:27.026+0000 7f1f1259c700 1 --2- 172.21.15.47:0/1935442789 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f14108580 0x7f1f141089a0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f1f040098a0 tx=0x7f1f04009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=687004f902ee8e59 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:51:27.642 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:27.026+0000 7f1f1159a700 1 -- 172.21.15.47:0/1935442789 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1f04004160 con 0x7f1f14108580 2024-04-17T05:51:27.642 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:27.026+0000 7f1f1159a700 1 -- 172.21.15.47:0/1935442789 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1f04004300 con 0x7f1f14108580 2024-04-17T05:51:27.642 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:27.026+0000 7f1f1159a700 1 -- 172.21.15.47:0/1935442789 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1f04011b50 con 0x7f1f14108580 2024-04-17T05:51:27.642 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:27.026+0000 7f1f1880e700 1 -- 172.21.15.47:0/1935442789 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f14108580 msgr2=0x7f1f141089a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:51:27.642 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:27.026+0000 7f1f1880e700 1 --2- 172.21.15.47:0/1935442789 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f14108580 0x7f1f141089a0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f1f040098a0 tx=0x7f1f04009bb0 comp rx=0 tx=0).stop 2024-04-17T05:51:27.642 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:27.027+0000 7f1f1880e700 1 -- 172.21.15.47:0/1935442789 shutdown_connections 2024-04-17T05:51:27.642 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:27.027+0000 7f1f1880e700 1 --2- 172.21.15.47:0/1935442789 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f14108580 0x7f1f141089a0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:51:27.642 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:27.027+0000 7f1f1880e700 1 -- 172.21.15.47:0/1935442789 >> 172.21.15.47:0/1935442789 conn(0x7f1f14103ba0 msgr2=0x7f1f14105fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:51:27.642 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:27.027+0000 7f1f1880e700 1 -- 172.21.15.47:0/1935442789 shutdown_connections 2024-04-17T05:51:27.642 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:27.027+0000 7f1f1880e700 1 -- 172.21.15.47:0/1935442789 wait complete. 2024-04-17T05:51:27.642 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:27.027+0000 7f1f1880e700 1 Processor -- start 2024-04-17T05:51:27.642 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:27.027+0000 7f1f1880e700 1 -- start start 2024-04-17T05:51:27.642 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:27.028+0000 7f1f1880e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f14108580 0x7f1f1407db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:51:27.642 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:27.028+0000 7f1f1880e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f1f1410ca90 con 0x7f1f14108580 2024-04-17T05:51:27.643 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:27.028+0000 7f1f1259c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f14108580 0x7f1f1407db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:51:27.643 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:27.028+0000 7f1f1259c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f14108580 0x7f1f1407db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48650/0 (socket says 172.21.15.47:48650) 2024-04-17T05:51:27.643 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:27.028+0000 7f1f1259c700 1 -- 172.21.15.47:0/3422926166 learned_addr learned my addr 172.21.15.47:0/3422926166 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:51:27.643 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:27.028+0000 7f1f1259c700 1 -- 172.21.15.47:0/3422926166 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f1407e0d0 con 0x7f1f14108580 2024-04-17T05:51:27.643 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:27.028+0000 7f1f1259c700 1 --2- 172.21.15.47:0/3422926166 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f14108580 0x7f1f1407db90 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f1f04011670 tx=0x7f1f04011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:51:27.643 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:27.028+0000 7f1f0b7fe700 1 -- 172.21.15.47:0/3422926166 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1f040039c0 con 0x7f1f14108580 2024-04-17T05:51:27.643 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:27.028+0000 7f1f0b7fe700 1 -- 172.21.15.47:0/3422926166 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1f04011e50 con 0x7f1f14108580 2024-04-17T05:51:27.643 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:27.028+0000 7f1f1880e700 1 -- 172.21.15.47:0/3422926166 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f1407c2c0 con 0x7f1f14108580 2024-04-17T05:51:27.643 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:27.028+0000 7f1f0b7fe700 1 -- 172.21.15.47:0/3422926166 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1f0401a5c0 con 0x7f1f14108580 2024-04-17T05:51:27.643 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:27.028+0000 7f1f1880e700 1 -- 172.21.15.47:0/3422926166 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f1407c800 con 0x7f1f14108580 2024-04-17T05:51:27.643 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:27.029+0000 7f1f0b7fe700 1 -- 172.21.15.47:0/3422926166 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1f0401ac50 con 0x7f1f14108580 2024-04-17T05:51:27.643 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:27.029+0000 7f1f0b7fe700 1 -- 172.21.15.47:0/3422926166 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f1f04019700 con 0x7f1f14108580 2024-04-17T05:51:27.643 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:27.029+0000 7f1f1880e700 1 -- 172.21.15.47:0/3422926166 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f140640b0 con 0x7f1f14108580 2024-04-17T05:51:27.644 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:27.032+0000 7f1f0b7fe700 1 -- 172.21.15.47:0/3422926166 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7f1f04003b60 con 0x7f1f14108580 2024-04-17T05:51:27.644 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:27.105+0000 7f1f1880e700 1 -- 172.21.15.47:0/3422926166 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f1f1407cd30 con 0x7f1f14108580 2024-04-17T05:51:27.644 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:27.106+0000 7f1f0b7fe700 1 -- 172.21.15.47:0/3422926166 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f1f04019900 con 0x7f1f14108580 2024-04-17T05:51:27.644 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:27.107+0000 7f1f1880e700 1 -- 172.21.15.47:0/3422926166 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f14108580 msgr2=0x7f1f1407db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:51:27.644 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:27.107+0000 7f1f1880e700 1 --2- 172.21.15.47:0/3422926166 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f14108580 0x7f1f1407db90 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f1f04011670 tx=0x7f1f04011b60 comp rx=0 tx=0).stop 2024-04-17T05:51:27.644 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:27.107+0000 7f1f1880e700 1 -- 172.21.15.47:0/3422926166 shutdown_connections 2024-04-17T05:51:27.644 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:27.107+0000 7f1f1880e700 1 --2- 172.21.15.47:0/3422926166 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f14108580 0x7f1f1407db90 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:51:27.644 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:27.107+0000 7f1f1880e700 1 -- 172.21.15.47:0/3422926166 >> 172.21.15.47:0/3422926166 conn(0x7f1f14103ba0 msgr2=0x7f1f14104720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:51:27.644 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:27.107+0000 7f1f1880e700 1 -- 172.21.15.47:0/3422926166 shutdown_connections 2024-04-17T05:51:27.644 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:27.107+0000 7f1f1880e700 1 -- 172.21.15.47:0/3422926166 wait complete. 2024-04-17T05:51:27.644 INFO:teuthology.orchestra.run.smithi047.stdout:mgr not available, waiting (1/15)... 2024-04-17T05:51:28.138 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 05:51:27 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T05:51:27.814+0000 7f59b9068200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-17T05:51:28.139 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 05:51:27 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T05:51:27.908+0000 7f59b9068200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-17T05:51:28.139 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 05:51:28 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T05:51:28.051+0000 7f59b9068200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-17T05:51:28.518 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 05:51:28 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T05:51:28.152+0000 7f59b9068200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-17T05:51:28.518 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 05:51:28 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T05:51:28.249+0000 7f59b9068200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-17T05:51:28.518 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 05:51:28 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T05:51:28.519+0000 7f59b9068200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-17T05:51:28.816 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 05:51:28 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T05:51:28.632+0000 7f59b9068200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-17T05:51:28.816 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 05:51:28 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T05:51:28.720+0000 7f59b9068200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-17T05:51:28.816 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 05:51:28 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T05:51:28.817+0000 7f59b9068200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-17T05:51:29.138 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 05:51:28 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T05:51:28.995+0000 7f59b9068200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-17T05:51:30.027 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 05:51:30 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T05:51:30.005+0000 7f59b9068200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-17T05:51:30.380 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:30 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/2443738544' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-17T05:51:30.380 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 05:51:30 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T05:51:30.154+0000 7f59b9068200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-17T05:51:30.638 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 05:51:30 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T05:51:30.508+0000 7f59b9068200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-17T05:51:30.639 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 05:51:30 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T05:51:30.600+0000 7f59b9068200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-17T05:51:30.710 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 2024-04-17T05:51:30.711 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout { 2024-04-17T05:51:30.711 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "fsid": "565db2d8-fc7e-11ee-bc91-c7b262605968", 2024-04-17T05:51:30.711 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "health": { 2024-04-17T05:51:30.711 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-17T05:51:30.711 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-17T05:51:30.711 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-17T05:51:30.711 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-04-17T05:51:30.711 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-17T05:51:30.711 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-17T05:51:30.711 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 0 2024-04-17T05:51:30.711 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout ], 2024-04-17T05:51:30.711 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-17T05:51:30.712 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "a" 2024-04-17T05:51:30.712 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout ], 2024-04-17T05:51:30.712 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-04-17T05:51:30.712 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-17T05:51:30.712 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-17T05:51:30.712 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-17T05:51:30.712 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-17T05:51:30.712 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-04-17T05:51:30.712 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-17T05:51:30.712 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-17T05:51:30.712 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-17T05:51:30.712 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-17T05:51:30.721 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-17T05:51:30.721 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-17T05:51:30.721 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-17T05:51:30.721 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-17T05:51:30.721 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-04-17T05:51:30.721 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-17T05:51:30.721 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-17T05:51:30.721 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-17T05:51:30.721 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-17T05:51:30.721 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-17T05:51:30.721 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-17T05:51:30.721 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-17T05:51:30.722 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-17T05:51:30.722 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-17T05:51:30.722 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-04-17T05:51:30.722 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-17T05:51:30.722 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-17T05:51:30.722 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-17T05:51:30.722 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-17T05:51:30.722 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-04-17T05:51:30.722 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-17T05:51:30.722 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-17T05:51:30.722 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-17T05:51:30.722 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-17T05:51:30.722 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-17T05:51:30.722 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-17T05:51:30.722 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "restful" 2024-04-17T05:51:30.723 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout ], 2024-04-17T05:51:30.723 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-17T05:51:30.723 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-04-17T05:51:30.723 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-17T05:51:30.723 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-17T05:51:30.723 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "modified": "2024-04-17T05:51:19.889468+0000", 2024-04-17T05:51:30.723 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-17T05:51:30.723 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-04-17T05:51:30.723 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-17T05:51:30.723 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout } 2024-04-17T05:51:30.723 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:30.053+0000 7f65f9e4a700 1 Processor -- start 2024-04-17T05:51:30.723 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:30.053+0000 7f65f9e4a700 1 -- start start 2024-04-17T05:51:30.724 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:30.053+0000 7f65f9e4a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65f4108ff0 0x7f65f410b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:51:30.724 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:30.053+0000 7f65f9e4a700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f65f407be50 con 0x7f65f4108ff0 2024-04-17T05:51:30.724 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:30.053+0000 7f65f37fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65f4108ff0 0x7f65f410b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:51:30.724 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:30.053+0000 7f65f37fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65f4108ff0 0x7f65f410b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48664/0 (socket says 172.21.15.47:48664) 2024-04-17T05:51:30.724 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:30.054+0000 7f65f37fe700 1 -- 172.21.15.47:0/1500340289 learned_addr learned my addr 172.21.15.47:0/1500340289 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:51:30.724 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:30.054+0000 7f65f37fe700 1 -- 172.21.15.47:0/1500340289 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65f410b920 con 0x7f65f4108ff0 2024-04-17T05:51:30.724 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:30.054+0000 7f65f37fe700 1 --2- 172.21.15.47:0/1500340289 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65f4108ff0 0x7f65f410b3e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f65e40098a0 tx=0x7f65e4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a90698813c18366 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:51:30.724 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:30.054+0000 7f65f2ffd700 1 -- 172.21.15.47:0/1500340289 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f65e4004160 con 0x7f65f4108ff0 2024-04-17T05:51:30.724 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:30.054+0000 7f65f2ffd700 1 -- 172.21.15.47:0/1500340289 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f65e4004300 con 0x7f65f4108ff0 2024-04-17T05:51:30.724 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:30.054+0000 7f65f2ffd700 1 -- 172.21.15.47:0/1500340289 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f65e4011b50 con 0x7f65f4108ff0 2024-04-17T05:51:30.724 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:30.055+0000 7f65f9e4a700 1 -- 172.21.15.47:0/1500340289 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65f4108ff0 msgr2=0x7f65f410b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:51:30.725 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:30.055+0000 7f65f9e4a700 1 --2- 172.21.15.47:0/1500340289 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65f4108ff0 0x7f65f410b3e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f65e40098a0 tx=0x7f65e4009bb0 comp rx=0 tx=0).stop 2024-04-17T05:51:30.725 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:30.055+0000 7f65f9e4a700 1 -- 172.21.15.47:0/1500340289 shutdown_connections 2024-04-17T05:51:30.725 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:30.055+0000 7f65f9e4a700 1 --2- 172.21.15.47:0/1500340289 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65f4108ff0 0x7f65f410b3e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:51:30.725 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:30.055+0000 7f65f9e4a700 1 -- 172.21.15.47:0/1500340289 >> 172.21.15.47:0/1500340289 conn(0x7f65f41029c0 msgr2=0x7f65f4104de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:51:30.725 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:30.055+0000 7f65f9e4a700 1 -- 172.21.15.47:0/1500340289 shutdown_connections 2024-04-17T05:51:30.725 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:30.055+0000 7f65f9e4a700 1 -- 172.21.15.47:0/1500340289 wait complete. 2024-04-17T05:51:30.725 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:30.056+0000 7f65f9e4a700 1 Processor -- start 2024-04-17T05:51:30.725 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:30.056+0000 7f65f9e4a700 1 -- start start 2024-04-17T05:51:30.725 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:30.056+0000 7f65f9e4a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65f4108ff0 0x7f65f4108870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:51:30.725 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:30.056+0000 7f65f9e4a700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f65f410c8e0 con 0x7f65f4108ff0 2024-04-17T05:51:30.725 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:30.056+0000 7f65f37fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65f4108ff0 0x7f65f4108870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:51:30.725 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:30.056+0000 7f65f37fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65f4108ff0 0x7f65f4108870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48674/0 (socket says 172.21.15.47:48674) 2024-04-17T05:51:30.725 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:30.056+0000 7f65f37fe700 1 -- 172.21.15.47:0/2443738544 learned_addr learned my addr 172.21.15.47:0/2443738544 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:51:30.725 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:30.056+0000 7f65f37fe700 1 -- 172.21.15.47:0/2443738544 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65f4108db0 con 0x7f65f4108ff0 2024-04-17T05:51:30.726 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:30.056+0000 7f65f37fe700 1 --2- 172.21.15.47:0/2443738544 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65f4108ff0 0x7f65f4108870 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f65e4011670 tx=0x7f65e4011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:51:30.726 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:30.057+0000 7f65f17fa700 1 -- 172.21.15.47:0/2443738544 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f65e40039c0 con 0x7f65f4108ff0 2024-04-17T05:51:30.726 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:30.057+0000 7f65f9e4a700 1 -- 172.21.15.47:0/2443738544 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65f4106fa0 con 0x7f65f4108ff0 2024-04-17T05:51:30.726 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:30.057+0000 7f65f17fa700 1 -- 172.21.15.47:0/2443738544 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f65e4011e50 con 0x7f65f4108ff0 2024-04-17T05:51:30.726 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:30.057+0000 7f65f17fa700 1 -- 172.21.15.47:0/2443738544 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f65e401a5c0 con 0x7f65f4108ff0 2024-04-17T05:51:30.726 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:30.057+0000 7f65f9e4a700 1 -- 172.21.15.47:0/2443738544 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65f41074e0 con 0x7f65f4108ff0 2024-04-17T05:51:30.726 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:30.057+0000 7f65f17fa700 1 -- 172.21.15.47:0/2443738544 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f65e401ac50 con 0x7f65f4108ff0 2024-04-17T05:51:30.726 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:30.057+0000 7f65f17fa700 1 -- 172.21.15.47:0/2443738544 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f65e4019700 con 0x7f65f4108ff0 2024-04-17T05:51:30.726 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:30.058+0000 7f65f9e4a700 1 -- 172.21.15.47:0/2443738544 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65f40640b0 con 0x7f65f4108ff0 2024-04-17T05:51:30.726 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:30.060+0000 7f65f17fa700 1 -- 172.21.15.47:0/2443738544 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7f65e4003b60 con 0x7f65f4108ff0 2024-04-17T05:51:30.726 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:30.146+0000 7f65f9e4a700 1 -- 172.21.15.47:0/2443738544 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f65f4107820 con 0x7f65f4108ff0 2024-04-17T05:51:30.726 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:30.146+0000 7f65f17fa700 1 -- 172.21.15.47:0/2443738544 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f65e40199a0 con 0x7f65f4108ff0 2024-04-17T05:51:30.726 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:30.147+0000 7f65f9e4a700 1 -- 172.21.15.47:0/2443738544 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65f4108ff0 msgr2=0x7f65f4108870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:51:30.727 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:30.147+0000 7f65f9e4a700 1 --2- 172.21.15.47:0/2443738544 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65f4108ff0 0x7f65f4108870 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f65e4011670 tx=0x7f65e4011b60 comp rx=0 tx=0).stop 2024-04-17T05:51:30.727 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:30.147+0000 7f65f9e4a700 1 -- 172.21.15.47:0/2443738544 shutdown_connections 2024-04-17T05:51:30.727 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:30.147+0000 7f65f9e4a700 1 --2- 172.21.15.47:0/2443738544 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65f4108ff0 0x7f65f4108870 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:51:30.727 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:30.147+0000 7f65f9e4a700 1 -- 172.21.15.47:0/2443738544 >> 172.21.15.47:0/2443738544 conn(0x7f65f41029c0 msgr2=0x7f65f41034b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:51:30.727 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:30.147+0000 7f65f9e4a700 1 -- 172.21.15.47:0/2443738544 shutdown_connections 2024-04-17T05:51:30.727 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:30.147+0000 7f65f9e4a700 1 -- 172.21.15.47:0/2443738544 wait complete. 2024-04-17T05:51:30.727 INFO:teuthology.orchestra.run.smithi047.stdout:mgr not available, waiting (2/15)... 2024-04-17T05:51:30.992 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 05:51:30 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T05:51:30.803+0000 7f59b9068200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-17T05:51:31.548 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 05:51:30 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T05:51:30.993+0000 7f59b9068200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-17T05:51:31.548 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 05:51:31 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T05:51:31.286+0000 7f59b9068200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-17T05:51:31.548 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 05:51:31 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T05:51:31.549+0000 7f59b9068200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-17T05:51:31.888 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 05:51:31 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T05:51:31.792+0000 7f59b9068200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-17T05:51:32.889 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 05:51:32 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T05:51:32.623+0000 7f59b9068200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-17T05:51:33.550 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:33 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/2901123967' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-17T05:51:33.550 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 05:51:33 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T05:51:33.223+0000 7f59b9068200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-17T05:51:33.550 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 05:51:33 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T05:51:33.324+0000 7f59b9068200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-17T05:51:33.550 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 05:51:33 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T05:51:33.550+0000 7f59b9068200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-17T05:51:33.725 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 2024-04-17T05:51:33.725 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout { 2024-04-17T05:51:33.725 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "fsid": "565db2d8-fc7e-11ee-bc91-c7b262605968", 2024-04-17T05:51:33.725 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "health": { 2024-04-17T05:51:33.725 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-17T05:51:33.725 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-17T05:51:33.726 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-17T05:51:33.726 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-04-17T05:51:33.726 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-17T05:51:33.726 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-17T05:51:33.726 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 0 2024-04-17T05:51:33.726 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout ], 2024-04-17T05:51:33.726 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-17T05:51:33.726 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "a" 2024-04-17T05:51:33.726 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout ], 2024-04-17T05:51:33.726 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-04-17T05:51:33.726 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-17T05:51:33.726 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-17T05:51:33.726 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-17T05:51:33.726 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-17T05:51:33.727 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-04-17T05:51:33.727 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-17T05:51:33.727 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-17T05:51:33.727 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-17T05:51:33.737 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-17T05:51:33.737 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-17T05:51:33.737 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-17T05:51:33.737 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-17T05:51:33.737 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-17T05:51:33.737 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-04-17T05:51:33.737 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-17T05:51:33.737 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-17T05:51:33.738 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-17T05:51:33.738 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-17T05:51:33.738 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-17T05:51:33.738 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-17T05:51:33.738 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-17T05:51:33.738 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-17T05:51:33.738 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-17T05:51:33.738 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-04-17T05:51:33.738 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-17T05:51:33.738 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-17T05:51:33.738 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-17T05:51:33.738 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-17T05:51:33.738 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-04-17T05:51:33.739 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-17T05:51:33.739 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-17T05:51:33.739 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-17T05:51:33.739 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-17T05:51:33.739 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-17T05:51:33.739 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-17T05:51:33.739 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "restful" 2024-04-17T05:51:33.739 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout ], 2024-04-17T05:51:33.739 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-17T05:51:33.739 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-04-17T05:51:33.739 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-17T05:51:33.739 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-17T05:51:33.739 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "modified": "2024-04-17T05:51:19.889468+0000", 2024-04-17T05:51:33.739 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-17T05:51:33.739 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-04-17T05:51:33.740 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-17T05:51:33.740 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout } 2024-04-17T05:51:33.740 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:33.101+0000 7fd98e9b3700 1 Processor -- start 2024-04-17T05:51:33.740 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:33.101+0000 7fd98e9b3700 1 -- start start 2024-04-17T05:51:33.740 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:33.101+0000 7fd98e9b3700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd988108fb0 0x7fd98810b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:51:33.740 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:33.101+0000 7fd98e9b3700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fd98807bed0 con 0x7fd988108fb0 2024-04-17T05:51:33.740 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:33.102+0000 7fd987fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd988108fb0 0x7fd98810b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:51:33.740 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:33.102+0000 7fd987fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd988108fb0 0x7fd98810b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48678/0 (socket says 172.21.15.47:48678) 2024-04-17T05:51:33.740 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:33.102+0000 7fd987fff700 1 -- 172.21.15.47:0/570984816 learned_addr learned my addr 172.21.15.47:0/570984816 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:51:33.740 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:33.102+0000 7fd987fff700 1 -- 172.21.15.47:0/570984816 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd98810b8e0 con 0x7fd988108fb0 2024-04-17T05:51:33.740 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:33.102+0000 7fd987fff700 1 --2- 172.21.15.47:0/570984816 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd988108fb0 0x7fd98810b3a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd9780098a0 tx=0x7fd978009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6142d5b050ac80e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:51:33.740 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:33.102+0000 7fd986ffd700 1 -- 172.21.15.47:0/570984816 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd978004160 con 0x7fd988108fb0 2024-04-17T05:51:33.740 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:33.102+0000 7fd986ffd700 1 -- 172.21.15.47:0/570984816 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd978004300 con 0x7fd988108fb0 2024-04-17T05:51:33.740 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:33.102+0000 7fd986ffd700 1 -- 172.21.15.47:0/570984816 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd978011b50 con 0x7fd988108fb0 2024-04-17T05:51:33.741 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:33.103+0000 7fd98e9b3700 1 -- 172.21.15.47:0/570984816 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd988108fb0 msgr2=0x7fd98810b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:51:33.741 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:33.103+0000 7fd98e9b3700 1 --2- 172.21.15.47:0/570984816 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd988108fb0 0x7fd98810b3a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd9780098a0 tx=0x7fd978009bb0 comp rx=0 tx=0).stop 2024-04-17T05:51:33.741 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:33.103+0000 7fd98e9b3700 1 -- 172.21.15.47:0/570984816 shutdown_connections 2024-04-17T05:51:33.741 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:33.103+0000 7fd98e9b3700 1 --2- 172.21.15.47:0/570984816 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd988108fb0 0x7fd98810b3a0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:51:33.741 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:33.103+0000 7fd98e9b3700 1 -- 172.21.15.47:0/570984816 >> 172.21.15.47:0/570984816 conn(0x7fd988102a20 msgr2=0x7fd988104e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:51:33.741 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:33.103+0000 7fd98e9b3700 1 -- 172.21.15.47:0/570984816 shutdown_connections 2024-04-17T05:51:33.741 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:33.103+0000 7fd98e9b3700 1 -- 172.21.15.47:0/570984816 wait complete. 2024-04-17T05:51:33.741 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:33.103+0000 7fd98e9b3700 1 Processor -- start 2024-04-17T05:51:33.741 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:33.104+0000 7fd98e9b3700 1 -- start start 2024-04-17T05:51:33.741 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:33.104+0000 7fd98e9b3700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd988108fb0 0x7fd98819c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:51:33.741 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:33.104+0000 7fd98e9b3700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fd98810c910 con 0x7fd988108fb0 2024-04-17T05:51:33.741 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:33.104+0000 7fd987fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd988108fb0 0x7fd98819c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:51:33.741 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:33.104+0000 7fd987fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd988108fb0 0x7fd98819c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48680/0 (socket says 172.21.15.47:48680) 2024-04-17T05:51:33.741 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:33.104+0000 7fd987fff700 1 -- 172.21.15.47:0/2901123967 learned_addr learned my addr 172.21.15.47:0/2901123967 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:51:33.741 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:33.104+0000 7fd987fff700 1 -- 172.21.15.47:0/2901123967 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd98819caa0 con 0x7fd988108fb0 2024-04-17T05:51:33.742 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:33.104+0000 7fd987fff700 1 --2- 172.21.15.47:0/2901123967 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd988108fb0 0x7fd98819c560 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd978009fb0 tx=0x7fd978011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:51:33.742 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:33.105+0000 7fd9857fa700 1 -- 172.21.15.47:0/2901123967 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd9780038b0 con 0x7fd988108fb0 2024-04-17T05:51:33.742 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:33.105+0000 7fd9857fa700 1 -- 172.21.15.47:0/2901123967 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd978011e50 con 0x7fd988108fb0 2024-04-17T05:51:33.742 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:33.105+0000 7fd98e9b3700 1 -- 172.21.15.47:0/2901123967 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd98819cd30 con 0x7fd988108fb0 2024-04-17T05:51:33.742 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:33.105+0000 7fd9857fa700 1 -- 172.21.15.47:0/2901123967 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd978019420 con 0x7fd988108fb0 2024-04-17T05:51:33.742 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:33.105+0000 7fd98e9b3700 1 -- 172.21.15.47:0/2901123967 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd98819fa20 con 0x7fd988108fb0 2024-04-17T05:51:33.742 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:33.105+0000 7fd9857fa700 1 -- 172.21.15.47:0/2901123967 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd978019ad0 con 0x7fd988108fb0 2024-04-17T05:51:33.742 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:33.105+0000 7fd9857fa700 1 -- 172.21.15.47:0/2901123967 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd978024980 con 0x7fd988108fb0 2024-04-17T05:51:33.742 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:33.106+0000 7fd98e9b3700 1 -- 172.21.15.47:0/2901123967 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9880640b0 con 0x7fd988108fb0 2024-04-17T05:51:33.742 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:33.108+0000 7fd9857fa700 1 -- 172.21.15.47:0/2901123967 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7fd978043e50 con 0x7fd988108fb0 2024-04-17T05:51:33.742 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:33.181+0000 7fd98e9b3700 1 -- 172.21.15.47:0/2901123967 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fd98819fbf0 con 0x7fd988108fb0 2024-04-17T05:51:33.742 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:33.182+0000 7fd9857fa700 1 -- 172.21.15.47:0/2901123967 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fd978032020 con 0x7fd988108fb0 2024-04-17T05:51:33.742 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:33.183+0000 7fd98e9b3700 1 -- 172.21.15.47:0/2901123967 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd988108fb0 msgr2=0x7fd98819c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:51:33.743 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:33.183+0000 7fd98e9b3700 1 --2- 172.21.15.47:0/2901123967 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd988108fb0 0x7fd98819c560 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd978009fb0 tx=0x7fd978011bf0 comp rx=0 tx=0).stop 2024-04-17T05:51:33.743 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:33.183+0000 7fd98e9b3700 1 -- 172.21.15.47:0/2901123967 shutdown_connections 2024-04-17T05:51:33.743 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:33.183+0000 7fd98e9b3700 1 --2- 172.21.15.47:0/2901123967 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd988108fb0 0x7fd98819c560 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:51:33.743 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:33.183+0000 7fd98e9b3700 1 -- 172.21.15.47:0/2901123967 >> 172.21.15.47:0/2901123967 conn(0x7fd988102a20 msgr2=0x7fd9881035d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:51:33.743 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:33.183+0000 7fd98e9b3700 1 -- 172.21.15.47:0/2901123967 shutdown_connections 2024-04-17T05:51:33.743 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:33.183+0000 7fd98e9b3700 1 -- 172.21.15.47:0/2901123967 wait complete. 2024-04-17T05:51:33.743 INFO:teuthology.orchestra.run.smithi047.stdout:mgr not available, waiting (3/15)... 2024-04-17T05:51:34.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:34 smithi047 ceph-mon[32223]: Activating manager daemon x 2024-04-17T05:51:34.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:34 smithi047 ceph-mon[32223]: mgrmap e2: x(active, starting, since 0.00305803s) 2024-04-17T05:51:34.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:34 smithi047 ceph-mon[32223]: from='mgr.14100 172.21.15.47:0/3455075800' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-17T05:51:34.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:34 smithi047 ceph-mon[32223]: from='mgr.14100 172.21.15.47:0/3455075800' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-17T05:51:34.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:34 smithi047 ceph-mon[32223]: from='mgr.14100 172.21.15.47:0/3455075800' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-17T05:51:34.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:34 smithi047 ceph-mon[32223]: from='mgr.14100 172.21.15.47:0/3455075800' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-17T05:51:34.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:34 smithi047 ceph-mon[32223]: from='mgr.14100 172.21.15.47:0/3455075800' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-17T05:51:34.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:34 smithi047 ceph-mon[32223]: Manager daemon x is now available 2024-04-17T05:51:34.640 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:34 smithi047 ceph-mon[32223]: from='mgr.14100 172.21.15.47:0/3455075800' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-17T05:51:34.640 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:34 smithi047 ceph-mon[32223]: from='mgr.14100 172.21.15.47:0/3455075800' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-17T05:51:34.640 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:34 smithi047 ceph-mon[32223]: from='mgr.14100 172.21.15.47:0/3455075800' entity='mgr.x' 2024-04-17T05:51:34.640 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:34 smithi047 ceph-mon[32223]: from='mgr.14100 172.21.15.47:0/3455075800' entity='mgr.x' 2024-04-17T05:51:34.640 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:34 smithi047 ceph-mon[32223]: from='mgr.14100 172.21.15.47:0/3455075800' entity='mgr.x' 2024-04-17T05:51:35.837 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:35 smithi047 ceph-mon[32223]: mgrmap e3: x(active, since 1.0071s) 2024-04-17T05:51:36.703 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:36 smithi047 ceph-mon[32223]: mgrmap e4: x(active, since 2s) 2024-04-17T05:51:36.703 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:36 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/1172132760' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-17T05:51:36.999 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 2024-04-17T05:51:36.999 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout { 2024-04-17T05:51:36.999 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "fsid": "565db2d8-fc7e-11ee-bc91-c7b262605968", 2024-04-17T05:51:36.999 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "health": { 2024-04-17T05:51:36.999 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-17T05:51:36.999 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-17T05:51:37.000 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-17T05:51:37.000 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-04-17T05:51:37.000 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-17T05:51:37.000 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-17T05:51:37.000 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 0 2024-04-17T05:51:37.000 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout ], 2024-04-17T05:51:37.000 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-17T05:51:37.000 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "a" 2024-04-17T05:51:37.000 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout ], 2024-04-17T05:51:37.000 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "quorum_age": 11, 2024-04-17T05:51:37.000 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-17T05:51:37.000 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-17T05:51:37.000 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-17T05:51:37.000 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-17T05:51:37.001 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-04-17T05:51:37.001 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-17T05:51:37.001 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-17T05:51:37.001 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-17T05:51:37.001 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-17T05:51:37.001 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-17T05:51:37.001 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-17T05:51:37.001 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-17T05:51:37.001 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-17T05:51:37.001 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-04-17T05:51:37.001 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-17T05:51:37.001 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-17T05:51:37.001 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-17T05:51:37.001 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-17T05:51:37.002 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-17T05:51:37.002 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-17T05:51:37.002 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-17T05:51:37.002 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-17T05:51:37.002 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-17T05:51:37.002 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-04-17T05:51:37.002 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-17T05:51:37.002 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-17T05:51:37.002 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-17T05:51:37.002 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-17T05:51:37.002 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-04-17T05:51:37.002 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-17T05:51:37.002 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-17T05:51:37.002 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-17T05:51:37.003 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-17T05:51:37.003 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-17T05:51:37.003 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-17T05:51:37.003 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "restful" 2024-04-17T05:51:37.003 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout ], 2024-04-17T05:51:37.003 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-17T05:51:37.003 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-04-17T05:51:37.003 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-17T05:51:37.003 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-17T05:51:37.003 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "modified": "2024-04-17T05:51:19.889468+0000", 2024-04-17T05:51:37.003 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-17T05:51:37.003 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-04-17T05:51:37.003 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-17T05:51:37.003 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout } 2024-04-17T05:51:37.004 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:36.115+0000 7f261c27c700 1 Processor -- start 2024-04-17T05:51:37.004 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:36.115+0000 7f261c27c700 1 -- start start 2024-04-17T05:51:37.004 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:36.115+0000 7f261c27c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2614107400 0x7f2614107820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:51:37.004 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:36.115+0000 7f261c27c700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f2614107df0 con 0x7f2614107400 2024-04-17T05:51:37.004 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:36.115+0000 7f261a018700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2614107400 0x7f2614107820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:51:37.004 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:36.115+0000 7f261a018700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2614107400 0x7f2614107820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48786/0 (socket says 172.21.15.47:48786) 2024-04-17T05:51:37.004 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:36.115+0000 7f261a018700 1 -- 172.21.15.47:0/3529847241 learned_addr learned my addr 172.21.15.47:0/3529847241 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:51:37.004 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:36.115+0000 7f261a018700 1 -- 172.21.15.47:0/3529847241 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2614108600 con 0x7f2614107400 2024-04-17T05:51:37.004 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:36.115+0000 7f261a018700 1 --2- 172.21.15.47:0/3529847241 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2614107400 0x7f2614107820 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f26080098a0 tx=0x7f2608009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0ab2d6d29bdb182 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:51:37.004 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:36.115+0000 7f2619016700 1 -- 172.21.15.47:0/3529847241 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2608004160 con 0x7f2614107400 2024-04-17T05:51:37.004 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:36.116+0000 7f2619016700 1 -- 172.21.15.47:0/3529847241 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2608004300 con 0x7f2614107400 2024-04-17T05:51:37.004 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:36.116+0000 7f2619016700 1 -- 172.21.15.47:0/3529847241 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2608011b50 con 0x7f2614107400 2024-04-17T05:51:37.004 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:36.116+0000 7f261c27c700 1 -- 172.21.15.47:0/3529847241 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2614107400 msgr2=0x7f2614107820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:51:37.004 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:36.116+0000 7f261c27c700 1 --2- 172.21.15.47:0/3529847241 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2614107400 0x7f2614107820 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f26080098a0 tx=0x7f2608009bb0 comp rx=0 tx=0).stop 2024-04-17T05:51:37.005 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:36.116+0000 7f261c27c700 1 -- 172.21.15.47:0/3529847241 shutdown_connections 2024-04-17T05:51:37.005 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:36.116+0000 7f261c27c700 1 --2- 172.21.15.47:0/3529847241 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2614107400 0x7f2614107820 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:51:37.005 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:36.116+0000 7f261c27c700 1 -- 172.21.15.47:0/3529847241 >> 172.21.15.47:0/3529847241 conn(0x7f26141029c0 msgr2=0x7f2614104de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:51:37.005 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:36.116+0000 7f261c27c700 1 -- 172.21.15.47:0/3529847241 shutdown_connections 2024-04-17T05:51:37.005 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:36.116+0000 7f261c27c700 1 -- 172.21.15.47:0/3529847241 wait complete. 2024-04-17T05:51:37.005 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:36.117+0000 7f261c27c700 1 Processor -- start 2024-04-17T05:51:37.005 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:36.117+0000 7f261c27c700 1 -- start start 2024-04-17T05:51:37.005 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:36.117+0000 7f261c27c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2614107400 0x7f261419c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:51:37.005 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:36.117+0000 7f261c27c700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f261410b9b0 con 0x7f2614107400 2024-04-17T05:51:37.005 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:36.117+0000 7f261a018700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2614107400 0x7f261419c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:51:37.005 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:36.117+0000 7f261a018700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2614107400 0x7f261419c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48800/0 (socket says 172.21.15.47:48800) 2024-04-17T05:51:37.005 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:36.117+0000 7f261a018700 1 -- 172.21.15.47:0/1172132760 learned_addr learned my addr 172.21.15.47:0/1172132760 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:51:37.005 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:36.117+0000 7f261a018700 1 -- 172.21.15.47:0/1172132760 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f261419ca50 con 0x7f2614107400 2024-04-17T05:51:37.006 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:36.118+0000 7f261a018700 1 --2- 172.21.15.47:0/1172132760 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2614107400 0x7f261419c510 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f2608011670 tx=0x7f2608011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:51:37.006 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:36.118+0000 7f26077fe700 1 -- 172.21.15.47:0/1172132760 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f26080039c0 con 0x7f2614107400 2024-04-17T05:51:37.006 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:36.118+0000 7f26077fe700 1 -- 172.21.15.47:0/1172132760 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2608011e50 con 0x7f2614107400 2024-04-17T05:51:37.006 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:36.118+0000 7f261c27c700 1 -- 172.21.15.47:0/1172132760 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f261419cce0 con 0x7f2614107400 2024-04-17T05:51:37.006 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:36.118+0000 7f26077fe700 1 -- 172.21.15.47:0/1172132760 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f260801a5c0 con 0x7f2614107400 2024-04-17T05:51:37.006 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:36.118+0000 7f261c27c700 1 -- 172.21.15.47:0/1172132760 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f261419f9d0 con 0x7f2614107400 2024-04-17T05:51:37.006 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:36.119+0000 7f26077fe700 1 -- 172.21.15.47:0/1172132760 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f260801a870 con 0x7f2614107400 2024-04-17T05:51:37.006 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:36.119+0000 7f261c27c700 1 -- 172.21.15.47:0/1172132760 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26140640b0 con 0x7f2614107400 2024-04-17T05:51:37.006 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:36.119+0000 7f26077fe700 1 --2- 172.21.15.47:0/1172132760 >> [v2:172.21.15.47:6800/953064786,v1:172.21.15.47:6801/953064786] conn(0x7f2600039560 0x7f260003ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:51:37.006 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:36.119+0000 7f26077fe700 1 -- 172.21.15.47:0/1172132760 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f260804db90 con 0x7f2614107400 2024-04-17T05:51:37.006 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:36.119+0000 7f2619817700 1 --2- 172.21.15.47:0/1172132760 >> [v2:172.21.15.47:6800/953064786,v1:172.21.15.47:6801/953064786] conn(0x7f2600039560 0x7f260003ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:51:37.006 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:36.120+0000 7f2619817700 1 --2- 172.21.15.47:0/1172132760 >> [v2:172.21.15.47:6800/953064786,v1:172.21.15.47:6801/953064786] conn(0x7f2600039560 0x7f260003ba20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f2610006fd0 tx=0x7f2610006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:51:37.006 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:36.125+0000 7f26077fe700 1 -- 172.21.15.47:0/1172132760 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f2608019c70 con 0x7f2614107400 2024-04-17T05:51:37.006 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:36.417+0000 7f261c27c700 1 -- 172.21.15.47:0/1172132760 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f26140008d0 con 0x7f2614107400 2024-04-17T05:51:37.007 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:36.418+0000 7f26077fe700 1 -- 172.21.15.47:0/1172132760 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f260802c9c0 con 0x7f2614107400 2024-04-17T05:51:37.007 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:36.420+0000 7f261c27c700 1 -- 172.21.15.47:0/1172132760 >> [v2:172.21.15.47:6800/953064786,v1:172.21.15.47:6801/953064786] conn(0x7f2600039560 msgr2=0x7f260003ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:51:37.007 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:36.420+0000 7f261c27c700 1 --2- 172.21.15.47:0/1172132760 >> [v2:172.21.15.47:6800/953064786,v1:172.21.15.47:6801/953064786] conn(0x7f2600039560 0x7f260003ba20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f2610006fd0 tx=0x7f2610006e40 comp rx=0 tx=0).stop 2024-04-17T05:51:37.007 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:36.420+0000 7f261c27c700 1 -- 172.21.15.47:0/1172132760 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2614107400 msgr2=0x7f261419c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:51:37.007 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:36.420+0000 7f261c27c700 1 --2- 172.21.15.47:0/1172132760 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2614107400 0x7f261419c510 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f2608011670 tx=0x7f2608011b60 comp rx=0 tx=0).stop 2024-04-17T05:51:37.007 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:36.420+0000 7f261c27c700 1 -- 172.21.15.47:0/1172132760 shutdown_connections 2024-04-17T05:51:37.007 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:36.420+0000 7f261c27c700 1 --2- 172.21.15.47:0/1172132760 >> [v2:172.21.15.47:6800/953064786,v1:172.21.15.47:6801/953064786] conn(0x7f2600039560 0x7f260003ba20 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:51:37.007 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:36.420+0000 7f261c27c700 1 --2- 172.21.15.47:0/1172132760 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2614107400 0x7f261419c510 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:51:37.007 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:36.420+0000 7f261c27c700 1 -- 172.21.15.47:0/1172132760 >> 172.21.15.47:0/1172132760 conn(0x7f26141029c0 msgr2=0x7f2614103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:51:37.007 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:36.420+0000 7f261c27c700 1 -- 172.21.15.47:0/1172132760 shutdown_connections 2024-04-17T05:51:37.007 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:36.420+0000 7f261c27c700 1 -- 172.21.15.47:0/1172132760 wait complete. 2024-04-17T05:51:37.007 INFO:teuthology.orchestra.run.smithi047.stdout:mgr is available 2024-04-17T05:51:37.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:37 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/2356832090' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-17T05:51:38.153 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 2024-04-17T05:51:38.153 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout [global] 2024-04-17T05:51:38.153 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout fsid = 565db2d8-fc7e-11ee-bc91-c7b262605968 2024-04-17T05:51:38.153 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-17T05:51:38.154 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-17T05:51:38.154 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-17T05:51:38.154 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-17T05:51:38.154 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 2024-04-17T05:51:38.154 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-17T05:51:38.154 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-17T05:51:38.154 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 2024-04-17T05:51:38.154 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout [osd] 2024-04-17T05:51:38.154 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-17T05:51:38.154 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-17T05:51:38.154 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-17T05:51:38.155 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-17T05:51:38.155 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-17T05:51:38.155 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:37.376+0000 7f3c26897700 1 Processor -- start 2024-04-17T05:51:38.155 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:37.376+0000 7f3c26897700 1 -- start start 2024-04-17T05:51:38.155 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:37.376+0000 7f3c26897700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3c20109050 0x7f3c2010b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:51:38.155 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:37.376+0000 7f3c26897700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f3c2007be30 con 0x7f3c20109050 2024-04-17T05:51:38.155 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:37.377+0000 7f3c1ffff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3c20109050 0x7f3c2010b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:51:38.155 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:37.377+0000 7f3c1ffff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3c20109050 0x7f3c2010b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48808/0 (socket says 172.21.15.47:48808) 2024-04-17T05:51:38.155 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:37.377+0000 7f3c1ffff700 1 -- 172.21.15.47:0/2019447595 learned_addr learned my addr 172.21.15.47:0/2019447595 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:51:38.155 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:37.377+0000 7f3c1ffff700 1 -- 172.21.15.47:0/2019447595 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c2010b980 con 0x7f3c20109050 2024-04-17T05:51:38.155 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:37.377+0000 7f3c1ffff700 1 --2- 172.21.15.47:0/2019447595 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3c20109050 0x7f3c2010b440 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f3c100098a0 tx=0x7f3c10009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba4217bc29431e3c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:51:38.156 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:37.377+0000 7f3c1effd700 1 -- 172.21.15.47:0/2019447595 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3c10004160 con 0x7f3c20109050 2024-04-17T05:51:38.156 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:37.377+0000 7f3c1effd700 1 -- 172.21.15.47:0/2019447595 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3c10004300 con 0x7f3c20109050 2024-04-17T05:51:38.156 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:37.377+0000 7f3c1effd700 1 -- 172.21.15.47:0/2019447595 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3c10011b50 con 0x7f3c20109050 2024-04-17T05:51:38.156 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:37.378+0000 7f3c26897700 1 -- 172.21.15.47:0/2019447595 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3c20109050 msgr2=0x7f3c2010b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:51:38.156 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:37.378+0000 7f3c26897700 1 --2- 172.21.15.47:0/2019447595 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3c20109050 0x7f3c2010b440 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f3c100098a0 tx=0x7f3c10009bb0 comp rx=0 tx=0).stop 2024-04-17T05:51:38.156 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:37.378+0000 7f3c26897700 1 -- 172.21.15.47:0/2019447595 shutdown_connections 2024-04-17T05:51:38.156 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:37.378+0000 7f3c26897700 1 --2- 172.21.15.47:0/2019447595 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3c20109050 0x7f3c2010b440 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:51:38.156 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:37.378+0000 7f3c26897700 1 -- 172.21.15.47:0/2019447595 >> 172.21.15.47:0/2019447595 conn(0x7f3c20102a20 msgr2=0x7f3c20104e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:51:38.156 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:37.378+0000 7f3c26897700 1 -- 172.21.15.47:0/2019447595 shutdown_connections 2024-04-17T05:51:38.156 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:37.378+0000 7f3c26897700 1 -- 172.21.15.47:0/2019447595 wait complete. 2024-04-17T05:51:38.156 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:37.378+0000 7f3c26897700 1 Processor -- start 2024-04-17T05:51:38.157 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:37.379+0000 7f3c26897700 1 -- start start 2024-04-17T05:51:38.157 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:37.379+0000 7f3c26897700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3c20109050 0x7f3c201088d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:51:38.157 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:37.379+0000 7f3c26897700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f3c2010c940 con 0x7f3c20109050 2024-04-17T05:51:38.157 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:37.379+0000 7f3c1ffff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3c20109050 0x7f3c201088d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:51:38.157 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:37.379+0000 7f3c1ffff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3c20109050 0x7f3c201088d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48814/0 (socket says 172.21.15.47:48814) 2024-04-17T05:51:38.157 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:37.379+0000 7f3c1ffff700 1 -- 172.21.15.47:0/2356832090 learned_addr learned my addr 172.21.15.47:0/2356832090 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:51:38.157 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:37.379+0000 7f3c1ffff700 1 -- 172.21.15.47:0/2356832090 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c20108e10 con 0x7f3c20109050 2024-04-17T05:51:38.157 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:37.379+0000 7f3c1ffff700 1 --2- 172.21.15.47:0/2356832090 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3c20109050 0x7f3c201088d0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f3c10009fb0 tx=0x7f3c10011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:51:38.157 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:37.380+0000 7f3c1d7fa700 1 -- 172.21.15.47:0/2356832090 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3c100038b0 con 0x7f3c20109050 2024-04-17T05:51:38.157 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:37.380+0000 7f3c1d7fa700 1 -- 172.21.15.47:0/2356832090 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3c10011e50 con 0x7f3c20109050 2024-04-17T05:51:38.157 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:37.380+0000 7f3c26897700 1 -- 172.21.15.47:0/2356832090 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c20107000 con 0x7f3c20109050 2024-04-17T05:51:38.157 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:37.380+0000 7f3c1d7fa700 1 -- 172.21.15.47:0/2356832090 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3c10019420 con 0x7f3c20109050 2024-04-17T05:51:38.158 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:37.380+0000 7f3c26897700 1 -- 172.21.15.47:0/2356832090 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c20107540 con 0x7f3c20109050 2024-04-17T05:51:38.158 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:37.380+0000 7f3c1d7fa700 1 -- 172.21.15.47:0/2356832090 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f3c1001a430 con 0x7f3c20109050 2024-04-17T05:51:38.158 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:37.381+0000 7f3c1d7fa700 1 --2- 172.21.15.47:0/2356832090 >> [v2:172.21.15.47:6800/953064786,v1:172.21.15.47:6801/953064786] conn(0x7f3c08039510 0x7f3c0803b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:51:38.158 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:37.381+0000 7f3c26897700 1 -- 172.21.15.47:0/2356832090 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c200640b0 con 0x7f3c20109050 2024-04-17T05:51:38.158 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:37.381+0000 7f3c1d7fa700 1 -- 172.21.15.47:0/2356832090 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f3c1004d8c0 con 0x7f3c20109050 2024-04-17T05:51:38.158 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:37.381+0000 7f3c1f7fe700 1 --2- 172.21.15.47:0/2356832090 >> [v2:172.21.15.47:6800/953064786,v1:172.21.15.47:6801/953064786] conn(0x7f3c08039510 0x7f3c0803b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:51:38.158 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:37.381+0000 7f3c1f7fe700 1 --2- 172.21.15.47:0/2356832090 >> [v2:172.21.15.47:6800/953064786,v1:172.21.15.47:6801/953064786] conn(0x7f3c08039510 0x7f3c0803b9d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3c14006fd0 tx=0x7f3c14006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:51:38.158 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:37.386+0000 7f3c1d7fa700 1 -- 172.21.15.47:0/2356832090 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f3c1004d2e0 con 0x7f3c20109050 2024-04-17T05:51:38.158 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:37.592+0000 7f3c26897700 1 -- 172.21.15.47:0/2356832090 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f3c20107e50 con 0x7f3c20109050 2024-04-17T05:51:38.158 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:37.593+0000 7f3c1d7fa700 1 -- 172.21.15.47:0/2356832090 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+373 (secure 0 0 0) 0x7f3c10003a50 con 0x7f3c20109050 2024-04-17T05:51:38.158 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:37.595+0000 7f3c26897700 1 -- 172.21.15.47:0/2356832090 >> [v2:172.21.15.47:6800/953064786,v1:172.21.15.47:6801/953064786] conn(0x7f3c08039510 msgr2=0x7f3c0803b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:51:38.159 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:37.595+0000 7f3c26897700 1 --2- 172.21.15.47:0/2356832090 >> [v2:172.21.15.47:6800/953064786,v1:172.21.15.47:6801/953064786] conn(0x7f3c08039510 0x7f3c0803b9d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3c14006fd0 tx=0x7f3c14006e40 comp rx=0 tx=0).stop 2024-04-17T05:51:38.159 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:37.595+0000 7f3c26897700 1 -- 172.21.15.47:0/2356832090 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3c20109050 msgr2=0x7f3c201088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:51:38.159 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:37.595+0000 7f3c26897700 1 --2- 172.21.15.47:0/2356832090 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3c20109050 0x7f3c201088d0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f3c10009fb0 tx=0x7f3c10011bf0 comp rx=0 tx=0).stop 2024-04-17T05:51:38.159 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:37.595+0000 7f3c26897700 1 -- 172.21.15.47:0/2356832090 shutdown_connections 2024-04-17T05:51:38.159 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:37.595+0000 7f3c26897700 1 --2- 172.21.15.47:0/2356832090 >> [v2:172.21.15.47:6800/953064786,v1:172.21.15.47:6801/953064786] conn(0x7f3c08039510 0x7f3c0803b9d0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:51:38.159 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:37.595+0000 7f3c26897700 1 --2- 172.21.15.47:0/2356832090 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3c20109050 0x7f3c201088d0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:51:38.159 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:37.595+0000 7f3c26897700 1 -- 172.21.15.47:0/2356832090 >> 172.21.15.47:0/2356832090 conn(0x7f3c20102a20 msgr2=0x7f3c201035a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:51:38.159 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:37.595+0000 7f3c26897700 1 -- 172.21.15.47:0/2356832090 shutdown_connections 2024-04-17T05:51:38.159 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:37.596+0000 7f3c26897700 1 -- 172.21.15.47:0/2356832090 wait complete. 2024-04-17T05:51:38.159 INFO:teuthology.orchestra.run.smithi047.stdout:Enabling cephadm module... 2024-04-17T05:51:39.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:38 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/3710451117' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-04-17T05:51:39.139 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 05:51:38 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: ignoring --setuser ceph since I am not root 2024-04-17T05:51:39.139 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 05:51:38 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: ignoring --setgroup ceph since I am not root 2024-04-17T05:51:39.471 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:38.569+0000 7f53e09fe700 1 Processor -- start 2024-04-17T05:51:39.471 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:38.570+0000 7f53e09fe700 1 -- start start 2024-04-17T05:51:39.471 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:38.570+0000 7f53e09fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53dc07de20 0x7f53dc07c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:51:39.472 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:38.570+0000 7f53e09fe700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f53dc07c860 con 0x7f53dc07de20 2024-04-17T05:51:39.472 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:38.570+0000 7f53dad9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53dc07de20 0x7f53dc07c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:51:39.472 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:38.570+0000 7f53dad9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53dc07de20 0x7f53dc07c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48830/0 (socket says 172.21.15.47:48830) 2024-04-17T05:51:39.472 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:38.570+0000 7f53dad9d700 1 -- 172.21.15.47:0/3338327239 learned_addr learned my addr 172.21.15.47:0/3338327239 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:51:39.472 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:38.570+0000 7f53dad9d700 1 -- 172.21.15.47:0/3338327239 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53dc07c9e0 con 0x7f53dc07de20 2024-04-17T05:51:39.472 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:38.570+0000 7f53dad9d700 1 --2- 172.21.15.47:0/3338327239 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53dc07de20 0x7f53dc07c320 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f53c40098a0 tx=0x7f53c4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c14ee36d3f31db48 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:51:39.472 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:38.571+0000 7f53d9d9b700 1 -- 172.21.15.47:0/3338327239 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f53c4004160 con 0x7f53dc07de20 2024-04-17T05:51:39.472 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:38.571+0000 7f53d9d9b700 1 -- 172.21.15.47:0/3338327239 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f53c4004300 con 0x7f53dc07de20 2024-04-17T05:51:39.472 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:38.571+0000 7f53d9d9b700 1 -- 172.21.15.47:0/3338327239 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f53c4011b50 con 0x7f53dc07de20 2024-04-17T05:51:39.472 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:38.571+0000 7f53e09fe700 1 -- 172.21.15.47:0/3338327239 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53dc07de20 msgr2=0x7f53dc07c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:51:39.472 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:38.571+0000 7f53e09fe700 1 --2- 172.21.15.47:0/3338327239 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53dc07de20 0x7f53dc07c320 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f53c40098a0 tx=0x7f53c4009bb0 comp rx=0 tx=0).stop 2024-04-17T05:51:39.472 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:38.571+0000 7f53e09fe700 1 -- 172.21.15.47:0/3338327239 shutdown_connections 2024-04-17T05:51:39.472 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:38.571+0000 7f53e09fe700 1 --2- 172.21.15.47:0/3338327239 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53dc07de20 0x7f53dc07c320 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:51:39.472 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:38.571+0000 7f53e09fe700 1 -- 172.21.15.47:0/3338327239 >> 172.21.15.47:0/3338327239 conn(0x7f53dc103a30 msgr2=0x7f53dc105e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:51:39.473 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:38.571+0000 7f53e09fe700 1 -- 172.21.15.47:0/3338327239 shutdown_connections 2024-04-17T05:51:39.473 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:38.571+0000 7f53e09fe700 1 -- 172.21.15.47:0/3338327239 wait complete. 2024-04-17T05:51:39.473 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:38.572+0000 7f53e09fe700 1 Processor -- start 2024-04-17T05:51:39.473 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:38.572+0000 7f53e09fe700 1 -- start start 2024-04-17T05:51:39.473 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:38.572+0000 7f53e09fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53dc07de20 0x7f53dc19e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:51:39.473 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:38.572+0000 7f53e09fe700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f53dc10e9a0 con 0x7f53dc07de20 2024-04-17T05:51:39.473 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:38.573+0000 7f53dad9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53dc07de20 0x7f53dc19e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:51:39.473 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:38.573+0000 7f53dad9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53dc07de20 0x7f53dc19e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48836/0 (socket says 172.21.15.47:48836) 2024-04-17T05:51:39.473 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:38.573+0000 7f53dad9d700 1 -- 172.21.15.47:0/3710451117 learned_addr learned my addr 172.21.15.47:0/3710451117 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:51:39.473 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:38.573+0000 7f53dad9d700 1 -- 172.21.15.47:0/3710451117 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53dc19ea80 con 0x7f53dc07de20 2024-04-17T05:51:39.473 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:38.573+0000 7f53dad9d700 1 --2- 172.21.15.47:0/3710451117 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53dc07de20 0x7f53dc19e540 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f53c4009fb0 tx=0x7f53c4011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:51:39.473 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:38.573+0000 7f53d3fff700 1 -- 172.21.15.47:0/3710451117 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f53c40038b0 con 0x7f53dc07de20 2024-04-17T05:51:39.473 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:38.573+0000 7f53d3fff700 1 -- 172.21.15.47:0/3710451117 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f53c4011e50 con 0x7f53dc07de20 2024-04-17T05:51:39.473 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:38.573+0000 7f53d3fff700 1 -- 172.21.15.47:0/3710451117 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f53c4019420 con 0x7f53dc07de20 2024-04-17T05:51:39.474 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:38.573+0000 7f53e09fe700 1 -- 172.21.15.47:0/3710451117 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53dc19ed10 con 0x7f53dc07de20 2024-04-17T05:51:39.474 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:38.573+0000 7f53e09fe700 1 -- 172.21.15.47:0/3710451117 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53dc1a1a00 con 0x7f53dc07de20 2024-04-17T05:51:39.474 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:38.574+0000 7f53d3fff700 1 -- 172.21.15.47:0/3710451117 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f53c40196d0 con 0x7f53dc07de20 2024-04-17T05:51:39.474 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:38.574+0000 7f53e09fe700 1 -- 172.21.15.47:0/3710451117 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53dc069b80 con 0x7f53dc07de20 2024-04-17T05:51:39.474 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:38.574+0000 7f53d3fff700 1 --2- 172.21.15.47:0/3710451117 >> [v2:172.21.15.47:6800/953064786,v1:172.21.15.47:6801/953064786] conn(0x7f53c8039560 0x7f53c803ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:51:39.474 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:38.574+0000 7f53d3fff700 1 -- 172.21.15.47:0/3710451117 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f53c404d920 con 0x7f53dc07de20 2024-04-17T05:51:39.474 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:38.574+0000 7f53da59c700 1 --2- 172.21.15.47:0/3710451117 >> [v2:172.21.15.47:6800/953064786,v1:172.21.15.47:6801/953064786] conn(0x7f53c8039560 0x7f53c803ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:51:39.474 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:38.575+0000 7f53da59c700 1 --2- 172.21.15.47:0/3710451117 >> [v2:172.21.15.47:6800/953064786,v1:172.21.15.47:6801/953064786] conn(0x7f53c8039560 0x7f53c803ba20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f53cc006fd0 tx=0x7f53cc006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:51:39.474 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:38.580+0000 7f53d3fff700 1 -- 172.21.15.47:0/3710451117 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f53c400ba30 con 0x7f53dc07de20 2024-04-17T05:51:39.474 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:38.849+0000 7f53e09fe700 1 -- 172.21.15.47:0/3710451117 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f53dc1a1bd0 con 0x7f53dc07de20 2024-04-17T05:51:39.474 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:38.900+0000 7f53d3fff700 1 -- 172.21.15.47:0/3710451117 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f53dc1a1bd0 con 0x7f53dc07de20 2024-04-17T05:51:39.474 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:38.902+0000 7f53d3fff700 1 -- 172.21.15.47:0/3710451117 <== mon.0 v2:172.21.15.47:3300/0 8 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f53c4003a50 con 0x7f53dc07de20 2024-04-17T05:51:39.474 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:38.903+0000 7f53e09fe700 1 -- 172.21.15.47:0/3710451117 >> [v2:172.21.15.47:6800/953064786,v1:172.21.15.47:6801/953064786] conn(0x7f53c8039560 msgr2=0x7f53c803ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:51:39.477 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:38.903+0000 7f53e09fe700 1 --2- 172.21.15.47:0/3710451117 >> [v2:172.21.15.47:6800/953064786,v1:172.21.15.47:6801/953064786] conn(0x7f53c8039560 0x7f53c803ba20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f53cc006fd0 tx=0x7f53cc006e40 comp rx=0 tx=0).stop 2024-04-17T05:51:39.477 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:38.903+0000 7f53e09fe700 1 -- 172.21.15.47:0/3710451117 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53dc07de20 msgr2=0x7f53dc19e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:51:39.477 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:38.903+0000 7f53e09fe700 1 --2- 172.21.15.47:0/3710451117 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53dc07de20 0x7f53dc19e540 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f53c4009fb0 tx=0x7f53c4011bf0 comp rx=0 tx=0).stop 2024-04-17T05:51:39.478 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:38.903+0000 7f53e09fe700 1 -- 172.21.15.47:0/3710451117 shutdown_connections 2024-04-17T05:51:39.478 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:38.903+0000 7f53e09fe700 1 --2- 172.21.15.47:0/3710451117 >> [v2:172.21.15.47:6800/953064786,v1:172.21.15.47:6801/953064786] conn(0x7f53c8039560 0x7f53c803ba20 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:51:39.478 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:38.903+0000 7f53e09fe700 1 --2- 172.21.15.47:0/3710451117 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53dc07de20 0x7f53dc19e540 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:51:39.478 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:38.903+0000 7f53e09fe700 1 -- 172.21.15.47:0/3710451117 >> 172.21.15.47:0/3710451117 conn(0x7f53dc103a30 msgr2=0x7f53dc1045e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:51:39.478 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:38.904+0000 7f53e09fe700 1 -- 172.21.15.47:0/3710451117 shutdown_connections 2024-04-17T05:51:39.478 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:38.904+0000 7f53e09fe700 1 -- 172.21.15.47:0/3710451117 wait complete. 2024-04-17T05:51:39.800 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 05:51:39 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T05:51:39.474+0000 7f9591735200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-17T05:51:39.800 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 05:51:39 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T05:51:39.573+0000 7f9591735200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-17T05:51:40.109 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:39 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/3710451117' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-04-17T05:51:40.109 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:39 smithi047 ceph-mon[32223]: mgrmap e5: x(active, since 5s) 2024-04-17T05:51:40.389 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 05:51:40 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T05:51:40.247+0000 7f9591735200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-17T05:51:40.389 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 05:51:40 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T05:51:40.342+0000 7f9591735200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-17T05:51:40.770 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 05:51:40 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T05:51:40.487+0000 7f9591735200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-17T05:51:40.770 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 05:51:40 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T05:51:40.586+0000 7f9591735200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-17T05:51:40.770 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 05:51:40 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T05:51:40.671+0000 7f9591735200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-17T05:51:40.779 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout { 2024-04-17T05:51:40.779 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-04-17T05:51:40.780 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-17T05:51:40.780 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-17T05:51:40.780 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-17T05:51:40.780 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout } 2024-04-17T05:51:40.780 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:39.903+0000 7fb119c4f700 1 Processor -- start 2024-04-17T05:51:40.780 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:39.903+0000 7fb119c4f700 1 -- start start 2024-04-17T05:51:40.780 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:39.903+0000 7fb119c4f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb114107400 0x7fb114107820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:51:40.780 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:39.903+0000 7fb119c4f700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb114107df0 con 0x7fb114107400 2024-04-17T05:51:40.780 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:39.904+0000 7fb1137fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb114107400 0x7fb114107820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:51:40.780 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:39.904+0000 7fb1137fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb114107400 0x7fb114107820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48866/0 (socket says 172.21.15.47:48866) 2024-04-17T05:51:40.780 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:39.904+0000 7fb1137fe700 1 -- 172.21.15.47:0/66661825 learned_addr learned my addr 172.21.15.47:0/66661825 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:51:40.780 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:39.904+0000 7fb1137fe700 1 -- 172.21.15.47:0/66661825 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb114108600 con 0x7fb114107400 2024-04-17T05:51:40.780 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:39.904+0000 7fb1137fe700 1 --2- 172.21.15.47:0/66661825 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb114107400 0x7fb114107820 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc0098a0 tx=0x7fb0fc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3176c5dc9395bd46 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:51:40.780 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:39.906+0000 7fb1127fc700 1 -- 172.21.15.47:0/66661825 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb0fc004160 con 0x7fb114107400 2024-04-17T05:51:40.781 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:39.906+0000 7fb1127fc700 1 -- 172.21.15.47:0/66661825 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb0fc004300 con 0x7fb114107400 2024-04-17T05:51:40.781 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:39.906+0000 7fb1127fc700 1 -- 172.21.15.47:0/66661825 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb0fc011b50 con 0x7fb114107400 2024-04-17T05:51:40.781 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:39.907+0000 7fb119c4f700 1 -- 172.21.15.47:0/66661825 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb114107400 msgr2=0x7fb114107820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:51:40.781 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:39.907+0000 7fb119c4f700 1 --2- 172.21.15.47:0/66661825 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb114107400 0x7fb114107820 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc0098a0 tx=0x7fb0fc009bb0 comp rx=0 tx=0).stop 2024-04-17T05:51:40.781 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:39.907+0000 7fb119c4f700 1 -- 172.21.15.47:0/66661825 shutdown_connections 2024-04-17T05:51:40.781 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:39.907+0000 7fb119c4f700 1 --2- 172.21.15.47:0/66661825 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb114107400 0x7fb114107820 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:51:40.781 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:39.907+0000 7fb119c4f700 1 -- 172.21.15.47:0/66661825 >> 172.21.15.47:0/66661825 conn(0x7fb1141029c0 msgr2=0x7fb114104de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:51:40.781 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:39.907+0000 7fb119c4f700 1 -- 172.21.15.47:0/66661825 shutdown_connections 2024-04-17T05:51:40.781 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:39.907+0000 7fb119c4f700 1 -- 172.21.15.47:0/66661825 wait complete. 2024-04-17T05:51:40.781 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:39.907+0000 7fb119c4f700 1 Processor -- start 2024-04-17T05:51:40.781 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:39.908+0000 7fb119c4f700 1 -- start start 2024-04-17T05:51:40.781 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:39.908+0000 7fb119c4f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb114107400 0x7fb1141925b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:51:40.781 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:39.908+0000 7fb119c4f700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb11410b9b0 con 0x7fb114107400 2024-04-17T05:51:40.781 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:39.908+0000 7fb1137fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb114107400 0x7fb1141925b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:51:40.782 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:39.908+0000 7fb1137fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb114107400 0x7fb1141925b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48880/0 (socket says 172.21.15.47:48880) 2024-04-17T05:51:40.782 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:39.908+0000 7fb1137fe700 1 -- 172.21.15.47:0/3497394 learned_addr learned my addr 172.21.15.47:0/3497394 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:51:40.782 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:39.908+0000 7fb1137fe700 1 -- 172.21.15.47:0/3497394 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb114192af0 con 0x7fb114107400 2024-04-17T05:51:40.782 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:39.908+0000 7fb1137fe700 1 --2- 172.21.15.47:0/3497394 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb114107400 0x7fb1141925b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc000c00 tx=0x7fb0fc011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:51:40.782 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:39.909+0000 7fb110ff9700 1 -- 172.21.15.47:0/3497394 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb0fc003b20 con 0x7fb114107400 2024-04-17T05:51:40.782 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:39.909+0000 7fb110ff9700 1 -- 172.21.15.47:0/3497394 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb0fc003cc0 con 0x7fb114107400 2024-04-17T05:51:40.782 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:39.909+0000 7fb119c4f700 1 -- 172.21.15.47:0/3497394 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb114190ce0 con 0x7fb114107400 2024-04-17T05:51:40.782 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:39.909+0000 7fb110ff9700 1 -- 172.21.15.47:0/3497394 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb0fc024910 con 0x7fb114107400 2024-04-17T05:51:40.782 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:39.909+0000 7fb119c4f700 1 -- 172.21.15.47:0/3497394 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb114191220 con 0x7fb114107400 2024-04-17T05:51:40.782 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:39.910+0000 7fb110ff9700 1 -- 172.21.15.47:0/3497394 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7fb0fc022020 con 0x7fb114107400 2024-04-17T05:51:40.782 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:39.910+0000 7fb119c4f700 1 -- 172.21.15.47:0/3497394 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1140640b0 con 0x7fb114107400 2024-04-17T05:51:40.782 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:39.910+0000 7fb110ff9700 1 --2- 172.21.15.47:0/3497394 >> [v2:172.21.15.47:6800/953064786,v1:172.21.15.47:6801/953064786] conn(0x7fb100039600 0x7fb10003bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:51:40.782 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:39.910+0000 7fb110ff9700 1 -- 172.21.15.47:0/3497394 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb0fc011e00 con 0x7fb114107400 2024-04-17T05:51:40.782 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:39.910+0000 7fb112ffd700 1 -- 172.21.15.47:0/3497394 >> [v2:172.21.15.47:6800/953064786,v1:172.21.15.47:6801/953064786] conn(0x7fb100039600 msgr2=0x7fb10003bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/953064786 2024-04-17T05:51:40.783 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:39.910+0000 7fb112ffd700 1 --2- 172.21.15.47:0/3497394 >> [v2:172.21.15.47:6800/953064786,v1:172.21.15.47:6801/953064786] conn(0x7fb100039600 0x7fb10003bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-17T05:51:40.783 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:39.915+0000 7fb110ff9700 1 -- 172.21.15.47:0/3497394 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fb0fc02cdf0 con 0x7fb114107400 2024-04-17T05:51:40.783 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:40.111+0000 7fb112ffd700 1 -- 172.21.15.47:0/3497394 >> [v2:172.21.15.47:6800/953064786,v1:172.21.15.47:6801/953064786] conn(0x7fb100039600 msgr2=0x7fb10003bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/953064786 2024-04-17T05:51:40.783 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:40.111+0000 7fb112ffd700 1 --2- 172.21.15.47:0/3497394 >> [v2:172.21.15.47:6800/953064786,v1:172.21.15.47:6801/953064786] conn(0x7fb100039600 0x7fb10003bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-17T05:51:40.783 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:40.188+0000 7fb119c4f700 1 -- 172.21.15.47:0/3497394 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fb114196330 con 0x7fb114107400 2024-04-17T05:51:40.783 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:40.188+0000 7fb110ff9700 1 -- 172.21.15.47:0/3497394 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7fb0fc00b5a0 con 0x7fb114107400 2024-04-17T05:51:40.783 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:40.191+0000 7fb119c4f700 1 -- 172.21.15.47:0/3497394 >> [v2:172.21.15.47:6800/953064786,v1:172.21.15.47:6801/953064786] conn(0x7fb100039600 msgr2=0x7fb10003bac0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-17T05:51:40.783 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:40.191+0000 7fb119c4f700 1 --2- 172.21.15.47:0/3497394 >> [v2:172.21.15.47:6800/953064786,v1:172.21.15.47:6801/953064786] conn(0x7fb100039600 0x7fb10003bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:51:40.783 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:40.191+0000 7fb119c4f700 1 -- 172.21.15.47:0/3497394 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb114107400 msgr2=0x7fb1141925b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:51:40.783 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:40.191+0000 7fb119c4f700 1 --2- 172.21.15.47:0/3497394 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb114107400 0x7fb1141925b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc000c00 tx=0x7fb0fc011c10 comp rx=0 tx=0).stop 2024-04-17T05:51:40.783 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:40.191+0000 7fb119c4f700 1 -- 172.21.15.47:0/3497394 shutdown_connections 2024-04-17T05:51:40.783 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:40.191+0000 7fb119c4f700 1 --2- 172.21.15.47:0/3497394 >> [v2:172.21.15.47:6800/953064786,v1:172.21.15.47:6801/953064786] conn(0x7fb100039600 0x7fb10003bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:51:40.783 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:40.191+0000 7fb119c4f700 1 --2- 172.21.15.47:0/3497394 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb114107400 0x7fb1141925b0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:51:40.783 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:40.191+0000 7fb119c4f700 1 -- 172.21.15.47:0/3497394 >> 172.21.15.47:0/3497394 conn(0x7fb1141029c0 msgr2=0x7fb114103880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:51:40.784 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:40.191+0000 7fb119c4f700 1 -- 172.21.15.47:0/3497394 shutdown_connections 2024-04-17T05:51:40.784 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:40.191+0000 7fb119c4f700 1 -- 172.21.15.47:0/3497394 wait complete. 2024-04-17T05:51:40.784 INFO:teuthology.orchestra.run.smithi047.stdout:Waiting for the mgr to restart... 2024-04-17T05:51:40.784 INFO:teuthology.orchestra.run.smithi047.stdout:Waiting for mgr epoch 5... 2024-04-17T05:51:41.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:40 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/3497394' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-17T05:51:41.066 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 05:51:40 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T05:51:40.954+0000 7f9591735200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-17T05:51:41.066 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 05:51:41 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T05:51:41.067+0000 7f9591735200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-17T05:51:41.389 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 05:51:41 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T05:51:41.158+0000 7f9591735200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-17T05:51:41.389 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 05:51:41 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T05:51:41.258+0000 7f9591735200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-17T05:51:41.808 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 05:51:41 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T05:51:41.437+0000 7f9591735200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-17T05:51:42.889 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 05:51:42 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T05:51:42.428+0000 7f9591735200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-17T05:51:42.889 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 05:51:42 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T05:51:42.562+0000 7f9591735200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-17T05:51:43.195 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 05:51:42 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T05:51:42.903+0000 7f9591735200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-17T05:51:43.196 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 05:51:42 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T05:51:42.995+0000 7f9591735200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-17T05:51:43.196 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 05:51:43 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T05:51:43.197+0000 7f9591735200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-17T05:51:43.638 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 05:51:43 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T05:51:43.388+0000 7f9591735200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-17T05:51:43.943 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 05:51:43 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T05:51:43.682+0000 7f9591735200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-17T05:51:43.943 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 05:51:43 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T05:51:43.945+0000 7f9591735200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-17T05:51:44.210 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 05:51:44 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T05:51:44.189+0000 7f9591735200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-17T05:51:45.389 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 05:51:45 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T05:51:45.045+0000 7f9591735200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-17T05:51:45.751 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 05:51:45 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T05:51:45.652+0000 7f9591735200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-17T05:51:45.751 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 05:51:45 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T05:51:45.753+0000 7f9591735200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-17T05:51:46.028 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 05:51:45 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T05:51:45.977+0000 7f9591735200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-17T05:51:46.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:46 smithi047 ceph-mon[32223]: Active manager daemon x restarted 2024-04-17T05:51:46.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:46 smithi047 ceph-mon[32223]: Activating manager daemon x 2024-04-17T05:51:46.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:46 smithi047 ceph-mon[32223]: osdmap e2: 0 total, 0 up, 0 in 2024-04-17T05:51:46.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:46 smithi047 ceph-mon[32223]: mgrmap e6: x(active, starting, since 0.00323286s) 2024-04-17T05:51:46.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:46 smithi047 ceph-mon[32223]: from='mgr.14120 172.21.15.47:0/2362399306' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-17T05:51:46.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:46 smithi047 ceph-mon[32223]: from='mgr.14120 172.21.15.47:0/2362399306' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-17T05:51:46.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:46 smithi047 ceph-mon[32223]: from='mgr.14120 172.21.15.47:0/2362399306' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-17T05:51:46.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:46 smithi047 ceph-mon[32223]: from='mgr.14120 172.21.15.47:0/2362399306' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-17T05:51:46.390 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:46 smithi047 ceph-mon[32223]: from='mgr.14120 172.21.15.47:0/2362399306' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-17T05:51:46.390 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:46 smithi047 ceph-mon[32223]: Manager daemon x is now available 2024-04-17T05:51:46.390 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:46 smithi047 ceph-mon[32223]: from='mgr.14120 172.21.15.47:0/2362399306' entity='mgr.x' 2024-04-17T05:51:46.390 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:46 smithi047 ceph-mon[32223]: from='mgr.14120 172.21.15.47:0/2362399306' entity='mgr.x' 2024-04-17T05:51:46.390 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:46 smithi047 ceph-mon[32223]: from='mgr.14120 172.21.15.47:0/2362399306' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T05:51:46.390 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:46 smithi047 ceph-mon[32223]: from='mgr.14120 172.21.15.47:0/2362399306' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T05:51:46.390 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:46 smithi047 ceph-mon[32223]: from='mgr.14120 172.21.15.47:0/2362399306' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-17T05:51:47.310 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:47 smithi047 ceph-mon[32223]: Found migration_current of "None". Setting to last migration. 2024-04-17T05:51:47.310 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:47 smithi047 ceph-mon[32223]: from='mgr.14120 172.21.15.47:0/2362399306' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-17T05:51:47.311 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:47 smithi047 ceph-mon[32223]: mgrmap e7: x(active, since 1.00524s) 2024-04-17T05:51:47.529 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout { 2024-04-17T05:51:47.529 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-04-17T05:51:47.529 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-17T05:51:47.529 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout } 2024-04-17T05:51:47.529 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:41.203+0000 7f9767896700 1 Processor -- start 2024-04-17T05:51:47.529 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:41.203+0000 7f9767896700 1 -- start start 2024-04-17T05:51:47.529 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:41.203+0000 7f9767896700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9760108530 0x7f9760108950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:51:47.529 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:41.203+0000 7f9767896700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f9760108f20 con 0x7f9760108530 2024-04-17T05:51:47.529 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:41.204+0000 7f9765632700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9760108530 0x7f9760108950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:51:47.529 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:41.204+0000 7f9765632700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9760108530 0x7f9760108950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48888/0 (socket says 172.21.15.47:48888) 2024-04-17T05:51:47.530 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:41.204+0000 7f9765632700 1 -- 172.21.15.47:0/2779365691 learned_addr learned my addr 172.21.15.47:0/2779365691 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:51:47.530 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:41.204+0000 7f9765632700 1 -- 172.21.15.47:0/2779365691 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97601096e0 con 0x7f9760108530 2024-04-17T05:51:47.530 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:41.204+0000 7f9765632700 1 --2- 172.21.15.47:0/2779365691 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9760108530 0x7f9760108950 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f975c0098a0 tx=0x7f975c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b04291250386227 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:51:47.530 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:41.204+0000 7f9753fff700 1 -- 172.21.15.47:0/2779365691 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f975c004160 con 0x7f9760108530 2024-04-17T05:51:47.530 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:41.204+0000 7f9753fff700 1 -- 172.21.15.47:0/2779365691 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f975c004300 con 0x7f9760108530 2024-04-17T05:51:47.530 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:41.204+0000 7f9753fff700 1 -- 172.21.15.47:0/2779365691 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f975c011b50 con 0x7f9760108530 2024-04-17T05:51:47.530 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:41.205+0000 7f9767896700 1 -- 172.21.15.47:0/2779365691 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9760108530 msgr2=0x7f9760108950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:51:47.530 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:41.205+0000 7f9767896700 1 --2- 172.21.15.47:0/2779365691 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9760108530 0x7f9760108950 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f975c0098a0 tx=0x7f975c009bb0 comp rx=0 tx=0).stop 2024-04-17T05:51:47.530 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:41.205+0000 7f9767896700 1 -- 172.21.15.47:0/2779365691 shutdown_connections 2024-04-17T05:51:47.530 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:41.205+0000 7f9767896700 1 --2- 172.21.15.47:0/2779365691 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9760108530 0x7f9760108950 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:51:47.530 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:41.205+0000 7f9767896700 1 -- 172.21.15.47:0/2779365691 >> 172.21.15.47:0/2779365691 conn(0x7f9760103af0 msgr2=0x7f9760105f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:51:47.530 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:41.205+0000 7f9767896700 1 -- 172.21.15.47:0/2779365691 shutdown_connections 2024-04-17T05:51:47.530 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:41.205+0000 7f9767896700 1 -- 172.21.15.47:0/2779365691 wait complete. 2024-04-17T05:51:47.531 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:41.206+0000 7f9767896700 1 Processor -- start 2024-04-17T05:51:47.531 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:41.206+0000 7f9767896700 1 -- start start 2024-04-17T05:51:47.531 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:41.206+0000 7f9767896700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9760108530 0x7f976007db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:51:47.531 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:41.206+0000 7f9767896700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f976010ca90 con 0x7f9760108530 2024-04-17T05:51:47.531 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:41.206+0000 7f9765632700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9760108530 0x7f976007db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:51:47.531 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:41.206+0000 7f9765632700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9760108530 0x7f976007db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48904/0 (socket says 172.21.15.47:48904) 2024-04-17T05:51:47.531 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:41.206+0000 7f9765632700 1 -- 172.21.15.47:0/456224205 learned_addr learned my addr 172.21.15.47:0/456224205 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:51:47.531 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:41.206+0000 7f9765632700 1 -- 172.21.15.47:0/456224205 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f976007e0d0 con 0x7f9760108530 2024-04-17T05:51:47.531 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:41.206+0000 7f9765632700 1 --2- 172.21.15.47:0/456224205 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9760108530 0x7f976007db90 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f975c011850 tx=0x7f975c011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:51:47.531 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:41.206+0000 7f97527fc700 1 -- 172.21.15.47:0/456224205 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f975c003af0 con 0x7f9760108530 2024-04-17T05:51:47.531 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:41.207+0000 7f97527fc700 1 -- 172.21.15.47:0/456224205 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f975c020070 con 0x7f9760108530 2024-04-17T05:51:47.531 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:41.207+0000 7f97527fc700 1 -- 172.21.15.47:0/456224205 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f975c0195a0 con 0x7f9760108530 2024-04-17T05:51:47.531 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:41.207+0000 7f9767896700 1 -- 172.21.15.47:0/456224205 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f976007c2c0 con 0x7f9760108530 2024-04-17T05:51:47.532 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:41.207+0000 7f9767896700 1 -- 172.21.15.47:0/456224205 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f976007c800 con 0x7f9760108530 2024-04-17T05:51:47.532 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:41.207+0000 7f97527fc700 1 -- 172.21.15.47:0/456224205 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f975c0198a0 con 0x7f9760108530 2024-04-17T05:51:47.532 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:41.208+0000 7f97527fc700 1 --2- 172.21.15.47:0/456224205 >> [v2:172.21.15.47:6800/953064786,v1:172.21.15.47:6801/953064786] conn(0x7f974c0395b0 0x7f974c03ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:51:47.532 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:41.208+0000 7f9767896700 1 -- 172.21.15.47:0/456224205 --> [v2:172.21.15.47:6800/953064786,v1:172.21.15.47:6801/953064786] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f9760069b80 con 0x7f974c0395b0 2024-04-17T05:51:47.532 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:41.208+0000 7f97527fc700 1 -- 172.21.15.47:0/456224205 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f975c011e00 con 0x7f9760108530 2024-04-17T05:51:47.532 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:41.208+0000 7f9764e31700 1 -- 172.21.15.47:0/456224205 >> [v2:172.21.15.47:6800/953064786,v1:172.21.15.47:6801/953064786] conn(0x7f974c0395b0 msgr2=0x7f974c03ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/953064786 2024-04-17T05:51:47.532 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:41.208+0000 7f9764e31700 1 --2- 172.21.15.47:0/456224205 >> [v2:172.21.15.47:6800/953064786,v1:172.21.15.47:6801/953064786] conn(0x7f974c0395b0 0x7f974c03ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-17T05:51:47.532 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:41.409+0000 7f9764e31700 1 -- 172.21.15.47:0/456224205 >> [v2:172.21.15.47:6800/953064786,v1:172.21.15.47:6801/953064786] conn(0x7f974c0395b0 msgr2=0x7f974c03ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/953064786 2024-04-17T05:51:47.532 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:41.409+0000 7f9764e31700 1 --2- 172.21.15.47:0/456224205 >> [v2:172.21.15.47:6800/953064786,v1:172.21.15.47:6801/953064786] conn(0x7f974c0395b0 0x7f974c03ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-17T05:51:47.532 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:41.810+0000 7f9764e31700 1 -- 172.21.15.47:0/456224205 >> [v2:172.21.15.47:6800/953064786,v1:172.21.15.47:6801/953064786] conn(0x7f974c0395b0 msgr2=0x7f974c03ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/953064786 2024-04-17T05:51:47.532 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:41.810+0000 7f9764e31700 1 --2- 172.21.15.47:0/456224205 >> [v2:172.21.15.47:6800/953064786,v1:172.21.15.47:6801/953064786] conn(0x7f974c0395b0 0x7f974c03ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-17T05:51:47.532 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:42.611+0000 7f9764e31700 1 -- 172.21.15.47:0/456224205 >> [v2:172.21.15.47:6800/953064786,v1:172.21.15.47:6801/953064786] conn(0x7f974c0395b0 msgr2=0x7f974c03ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/953064786 2024-04-17T05:51:47.532 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:42.611+0000 7f9764e31700 1 --2- 172.21.15.47:0/456224205 >> [v2:172.21.15.47:6800/953064786,v1:172.21.15.47:6801/953064786] conn(0x7f974c0395b0 0x7f974c03ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-17T05:51:47.532 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:44.212+0000 7f9764e31700 1 -- 172.21.15.47:0/456224205 >> [v2:172.21.15.47:6800/953064786,v1:172.21.15.47:6801/953064786] conn(0x7f974c0395b0 msgr2=0x7f974c03ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/953064786 2024-04-17T05:51:47.533 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:44.212+0000 7f9764e31700 1 --2- 172.21.15.47:0/456224205 >> [v2:172.21.15.47:6800/953064786,v1:172.21.15.47:6801/953064786] conn(0x7f974c0395b0 0x7f974c03ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-17T05:51:47.533 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:45.980+0000 7f97527fc700 1 -- 172.21.15.47:0/456224205 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mgrmap(e 6) ==== 46325+0+0 (secure 0 0 0) 0x7f975c01a430 con 0x7f9760108530 2024-04-17T05:51:47.533 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:45.980+0000 7f97527fc700 1 -- 172.21.15.47:0/456224205 >> [v2:172.21.15.47:6800/953064786,v1:172.21.15.47:6801/953064786] conn(0x7f974c0395b0 msgr2=0x7f974c03ba70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-17T05:51:47.533 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:45.980+0000 7f97527fc700 1 --2- 172.21.15.47:0/456224205 >> [v2:172.21.15.47:6800/953064786,v1:172.21.15.47:6801/953064786] conn(0x7f974c0395b0 0x7f974c03ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:51:47.533 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:46.981+0000 7f97527fc700 1 -- 172.21.15.47:0/456224205 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mgrmap(e 7) ==== 46452+0+0 (secure 0 0 0) 0x7f975c00f2e0 con 0x7f9760108530 2024-04-17T05:51:47.533 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:46.982+0000 7f97527fc700 1 --2- 172.21.15.47:0/456224205 >> [v2:172.21.15.47:6800/3632157175,v1:172.21.15.47:6801/3632157175] conn(0x7f974c03ccb0 0x7f974c03f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:51:47.533 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:46.982+0000 7f97527fc700 1 -- 172.21.15.47:0/456224205 --> [v2:172.21.15.47:6800/3632157175,v1:172.21.15.47:6801/3632157175] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f975c024a80 con 0x7f974c03ccb0 2024-04-17T05:51:47.533 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:46.982+0000 7f9764e31700 1 --2- 172.21.15.47:0/456224205 >> [v2:172.21.15.47:6800/3632157175,v1:172.21.15.47:6801/3632157175] conn(0x7f974c03ccb0 0x7f974c03f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:51:47.533 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:46.983+0000 7f9764e31700 1 --2- 172.21.15.47:0/456224205 >> [v2:172.21.15.47:6800/3632157175,v1:172.21.15.47:6801/3632157175] conn(0x7f974c03ccb0 0x7f974c03f0a0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f9754003a10 tx=0x7f9754009290 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:51:47.533 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:46.983+0000 7f97527fc700 1 -- 172.21.15.47:0/456224205 <== mgr.14120 v2:172.21.15.47:6800/3632157175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f975c024a80 con 0x7f974c03ccb0 2024-04-17T05:51:47.533 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:46.990+0000 7f9767896700 1 -- 172.21.15.47:0/456224205 --> [v2:172.21.15.47:6800/3632157175,v1:172.21.15.47:6801/3632157175] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f9760104d90 con 0x7f974c03ccb0 2024-04-17T05:51:47.533 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:46.990+0000 7f97527fc700 1 -- 172.21.15.47:0/456224205 <== mgr.14120 v2:172.21.15.47:6800/3632157175 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f9760104d90 con 0x7f974c03ccb0 2024-04-17T05:51:47.533 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:46.990+0000 7f9767896700 1 -- 172.21.15.47:0/456224205 >> [v2:172.21.15.47:6800/3632157175,v1:172.21.15.47:6801/3632157175] conn(0x7f974c03ccb0 msgr2=0x7f974c03f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:51:47.534 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:46.990+0000 7f9767896700 1 --2- 172.21.15.47:0/456224205 >> [v2:172.21.15.47:6800/3632157175,v1:172.21.15.47:6801/3632157175] conn(0x7f974c03ccb0 0x7f974c03f0a0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f9754003a10 tx=0x7f9754009290 comp rx=0 tx=0).stop 2024-04-17T05:51:47.534 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:46.991+0000 7f9767896700 1 -- 172.21.15.47:0/456224205 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9760108530 msgr2=0x7f976007db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:51:47.534 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:46.991+0000 7f9767896700 1 --2- 172.21.15.47:0/456224205 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9760108530 0x7f976007db90 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f975c011850 tx=0x7f975c011c10 comp rx=0 tx=0).stop 2024-04-17T05:51:47.534 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:46.991+0000 7f9767896700 1 -- 172.21.15.47:0/456224205 shutdown_connections 2024-04-17T05:51:47.534 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:46.991+0000 7f9767896700 1 --2- 172.21.15.47:0/456224205 >> [v2:172.21.15.47:6800/3632157175,v1:172.21.15.47:6801/3632157175] conn(0x7f974c03ccb0 0x7f974c03f0a0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:51:47.534 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:46.991+0000 7f9767896700 1 --2- 172.21.15.47:0/456224205 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9760108530 0x7f976007db90 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:51:47.534 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:46.991+0000 7f9767896700 1 --2- 172.21.15.47:0/456224205 >> [v2:172.21.15.47:6800/953064786,v1:172.21.15.47:6801/953064786] conn(0x7f974c0395b0 0x7f974c03ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:51:47.534 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:46.991+0000 7f9767896700 1 -- 172.21.15.47:0/456224205 >> 172.21.15.47:0/456224205 conn(0x7f9760103af0 msgr2=0x7f9760104680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:51:47.534 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:46.991+0000 7f9767896700 1 -- 172.21.15.47:0/456224205 shutdown_connections 2024-04-17T05:51:47.534 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:46.991+0000 7f9767896700 1 -- 172.21.15.47:0/456224205 wait complete. 2024-04-17T05:51:47.534 INFO:teuthology.orchestra.run.smithi047.stdout:mgr epoch 5 is available 2024-04-17T05:51:47.534 INFO:teuthology.orchestra.run.smithi047.stdout:Setting orchestrator backend to cephadm... 2024-04-17T05:51:48.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:48 smithi047 ceph-mon[32223]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-17T05:51:48.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:48 smithi047 ceph-mon[32223]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-17T05:51:48.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:48 smithi047 ceph-mon[32223]: from='mgr.14120 172.21.15.47:0/2362399306' entity='mgr.x' 2024-04-17T05:51:48.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:48 smithi047 ceph-mon[32223]: from='mgr.14120 172.21.15.47:0/2362399306' entity='mgr.x' 2024-04-17T05:51:48.766 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:47.927+0000 7f3e79f69700 1 Processor -- start 2024-04-17T05:51:48.766 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:47.927+0000 7f3e79f69700 1 -- start start 2024-04-17T05:51:48.766 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:47.927+0000 7f3e79f69700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3e74109030 0x7f3e7410b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:51:48.766 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:47.927+0000 7f3e79f69700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f3e7407be50 con 0x7f3e74109030 2024-04-17T05:51:48.766 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:47.927+0000 7f3e737fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3e74109030 0x7f3e7410b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:51:48.767 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:47.927+0000 7f3e737fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3e74109030 0x7f3e7410b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57802/0 (socket says 172.21.15.47:57802) 2024-04-17T05:51:48.767 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:47.927+0000 7f3e737fe700 1 -- 172.21.15.47:0/2934290648 learned_addr learned my addr 172.21.15.47:0/2934290648 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:51:48.767 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:47.928+0000 7f3e737fe700 1 -- 172.21.15.47:0/2934290648 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e7410b960 con 0x7f3e74109030 2024-04-17T05:51:48.767 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:47.928+0000 7f3e737fe700 1 --2- 172.21.15.47:0/2934290648 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3e74109030 0x7f3e7410b420 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3e640098a0 tx=0x7f3e64009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5649be6eadc53832 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:51:48.767 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:47.928+0000 7f3e727fc700 1 -- 172.21.15.47:0/2934290648 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3e64004160 con 0x7f3e74109030 2024-04-17T05:51:48.767 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:47.928+0000 7f3e727fc700 1 -- 172.21.15.47:0/2934290648 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3e64004300 con 0x7f3e74109030 2024-04-17T05:51:48.767 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:47.928+0000 7f3e727fc700 1 -- 172.21.15.47:0/2934290648 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3e64011b50 con 0x7f3e74109030 2024-04-17T05:51:48.767 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:47.928+0000 7f3e79f69700 1 -- 172.21.15.47:0/2934290648 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3e74109030 msgr2=0x7f3e7410b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:51:48.767 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:47.928+0000 7f3e79f69700 1 --2- 172.21.15.47:0/2934290648 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3e74109030 0x7f3e7410b420 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3e640098a0 tx=0x7f3e64009bb0 comp rx=0 tx=0).stop 2024-04-17T05:51:48.767 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:47.928+0000 7f3e79f69700 1 -- 172.21.15.47:0/2934290648 shutdown_connections 2024-04-17T05:51:48.768 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:47.928+0000 7f3e79f69700 1 --2- 172.21.15.47:0/2934290648 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3e74109030 0x7f3e7410b420 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:51:48.768 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:47.929+0000 7f3e79f69700 1 -- 172.21.15.47:0/2934290648 >> 172.21.15.47:0/2934290648 conn(0x7f3e74102a20 msgr2=0x7f3e74104e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:51:48.768 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:47.929+0000 7f3e79f69700 1 -- 172.21.15.47:0/2934290648 shutdown_connections 2024-04-17T05:51:48.768 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:47.929+0000 7f3e79f69700 1 -- 172.21.15.47:0/2934290648 wait complete. 2024-04-17T05:51:48.768 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:47.929+0000 7f3e79f69700 1 Processor -- start 2024-04-17T05:51:48.768 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:47.929+0000 7f3e79f69700 1 -- start start 2024-04-17T05:51:48.768 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:47.930+0000 7f3e79f69700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3e74109030 0x7f3e7419c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:51:48.768 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:47.930+0000 7f3e79f69700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f3e7410c920 con 0x7f3e74109030 2024-04-17T05:51:48.768 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:47.930+0000 7f3e737fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3e74109030 0x7f3e7419c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:51:48.768 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:47.930+0000 7f3e737fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3e74109030 0x7f3e7419c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57808/0 (socket says 172.21.15.47:57808) 2024-04-17T05:51:48.768 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:47.930+0000 7f3e737fe700 1 -- 172.21.15.47:0/2066482502 learned_addr learned my addr 172.21.15.47:0/2066482502 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:51:48.768 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:47.930+0000 7f3e737fe700 1 -- 172.21.15.47:0/2066482502 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e7419cab0 con 0x7f3e74109030 2024-04-17T05:51:48.768 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:47.930+0000 7f3e737fe700 1 --2- 172.21.15.47:0/2066482502 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3e74109030 0x7f3e7419c570 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3e64009fb0 tx=0x7f3e64011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:51:48.769 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:47.930+0000 7f3e70ff9700 1 -- 172.21.15.47:0/2066482502 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3e640038b0 con 0x7f3e74109030 2024-04-17T05:51:48.769 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:47.930+0000 7f3e79f69700 1 -- 172.21.15.47:0/2066482502 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e7419cd40 con 0x7f3e74109030 2024-04-17T05:51:48.769 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:47.930+0000 7f3e79f69700 1 -- 172.21.15.47:0/2066482502 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e7419fa30 con 0x7f3e74109030 2024-04-17T05:51:48.769 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:47.930+0000 7f3e70ff9700 1 -- 172.21.15.47:0/2066482502 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3e64011e50 con 0x7f3e74109030 2024-04-17T05:51:48.769 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:47.930+0000 7f3e70ff9700 1 -- 172.21.15.47:0/2066482502 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3e64019420 con 0x7f3e74109030 2024-04-17T05:51:48.769 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:47.931+0000 7f3e70ff9700 1 -- 172.21.15.47:0/2066482502 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 7) ==== 46452+0+0 (secure 0 0 0) 0x7f3e64019750 con 0x7f3e74109030 2024-04-17T05:51:48.769 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:47.932+0000 7f3e79f69700 1 -- 172.21.15.47:0/2066482502 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e740640b0 con 0x7f3e74109030 2024-04-17T05:51:48.769 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:47.932+0000 7f3e70ff9700 1 --2- 172.21.15.47:0/2066482502 >> [v2:172.21.15.47:6800/3632157175,v1:172.21.15.47:6801/3632157175] conn(0x7f3e5c0394a0 0x7f3e5c03b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:51:48.769 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:47.932+0000 7f3e70ff9700 1 -- 172.21.15.47:0/2066482502 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f3e6404d820 con 0x7f3e74109030 2024-04-17T05:51:48.769 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:47.932+0000 7f3e72ffd700 1 --2- 172.21.15.47:0/2066482502 >> [v2:172.21.15.47:6800/3632157175,v1:172.21.15.47:6801/3632157175] conn(0x7f3e5c0394a0 0x7f3e5c03b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:51:48.769 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:47.932+0000 7f3e72ffd700 1 --2- 172.21.15.47:0/2066482502 >> [v2:172.21.15.47:6800/3632157175,v1:172.21.15.47:6801/3632157175] conn(0x7f3e5c0394a0 0x7f3e5c03b960 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3e68006fd0 tx=0x7f3e68006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:51:48.769 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:47.941+0000 7f3e70ff9700 1 -- 172.21.15.47:0/2066482502 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f3e64003a50 con 0x7f3e74109030 2024-04-17T05:51:48.769 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:48.141+0000 7f3e70ff9700 1 -- 172.21.15.47:0/2066482502 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f3e6402d770 con 0x7f3e74109030 2024-04-17T05:51:48.769 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:48.169+0000 7f3e79f69700 1 -- 172.21.15.47:0/2066482502 --> [v2:172.21.15.47:6800/3632157175,v1:172.21.15.47:6801/3632157175] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f3e74103d30 con 0x7f3e5c0394a0 2024-04-17T05:51:48.770 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:48.174+0000 7f3e70ff9700 1 -- 172.21.15.47:0/2066482502 <== mgr.14120 v2:172.21.15.47:6800/3632157175 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3e74103d30 con 0x7f3e5c0394a0 2024-04-17T05:51:48.770 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:48.177+0000 7f3e79f69700 1 -- 172.21.15.47:0/2066482502 >> [v2:172.21.15.47:6800/3632157175,v1:172.21.15.47:6801/3632157175] conn(0x7f3e5c0394a0 msgr2=0x7f3e5c03b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:51:48.770 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:48.177+0000 7f3e79f69700 1 --2- 172.21.15.47:0/2066482502 >> [v2:172.21.15.47:6800/3632157175,v1:172.21.15.47:6801/3632157175] conn(0x7f3e5c0394a0 0x7f3e5c03b960 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3e68006fd0 tx=0x7f3e68006e40 comp rx=0 tx=0).stop 2024-04-17T05:51:48.770 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:48.177+0000 7f3e79f69700 1 -- 172.21.15.47:0/2066482502 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3e74109030 msgr2=0x7f3e7419c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:51:48.770 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:48.177+0000 7f3e79f69700 1 --2- 172.21.15.47:0/2066482502 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3e74109030 0x7f3e7419c570 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3e64009fb0 tx=0x7f3e64011bf0 comp rx=0 tx=0).stop 2024-04-17T05:51:48.770 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:48.177+0000 7f3e79f69700 1 -- 172.21.15.47:0/2066482502 shutdown_connections 2024-04-17T05:51:48.770 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:48.177+0000 7f3e79f69700 1 --2- 172.21.15.47:0/2066482502 >> [v2:172.21.15.47:6800/3632157175,v1:172.21.15.47:6801/3632157175] conn(0x7f3e5c0394a0 0x7f3e5c03b960 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:51:48.770 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:48.177+0000 7f3e79f69700 1 --2- 172.21.15.47:0/2066482502 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3e74109030 0x7f3e7419c570 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:51:48.770 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:48.177+0000 7f3e79f69700 1 -- 172.21.15.47:0/2066482502 >> 172.21.15.47:0/2066482502 conn(0x7f3e74102a20 msgr2=0x7f3e74103620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:51:48.770 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:48.177+0000 7f3e79f69700 1 -- 172.21.15.47:0/2066482502 shutdown_connections 2024-04-17T05:51:48.770 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:48.177+0000 7f3e79f69700 1 -- 172.21.15.47:0/2066482502 wait complete. 2024-04-17T05:51:49.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:49 smithi047 ceph-mon[32223]: mgrmap e8: x(active, since 2s) 2024-04-17T05:51:49.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:49 smithi047 ceph-mon[32223]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T05:51:49.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:49 smithi047 ceph-mon[32223]: from='mgr.14120 172.21.15.47:0/2362399306' entity='mgr.x' 2024-04-17T05:51:49.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:49 smithi047 ceph-mon[32223]: from='mgr.14120 172.21.15.47:0/2362399306' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T05:51:49.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:49 smithi047 ceph-mon[32223]: [17/Apr/2024:05:51:48] ENGINE Bus STARTING 2024-04-17T05:51:49.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:49 smithi047 ceph-mon[32223]: [17/Apr/2024:05:51:48] ENGINE Serving on http://172.21.15.47:8765 2024-04-17T05:51:49.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:49 smithi047 ceph-mon[32223]: from='mgr.14120 172.21.15.47:0/2362399306' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T05:51:50.023 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout value unchanged 2024-04-17T05:51:50.023 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:49.240+0000 7f45c7a97700 1 Processor -- start 2024-04-17T05:51:50.023 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:49.240+0000 7f45c7a97700 1 -- start start 2024-04-17T05:51:50.023 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:49.241+0000 7f45c7a97700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45c0108580 0x7f45c01089a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:51:50.023 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:49.241+0000 7f45c7a97700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f45c0108f70 con 0x7f45c0108580 2024-04-17T05:51:50.023 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:49.241+0000 7f45c5833700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45c0108580 0x7f45c01089a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:51:50.023 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:49.241+0000 7f45c5833700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45c0108580 0x7f45c01089a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57814/0 (socket says 172.21.15.47:57814) 2024-04-17T05:51:50.023 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:49.241+0000 7f45c5833700 1 -- 172.21.15.47:0/3623566053 learned_addr learned my addr 172.21.15.47:0/3623566053 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:51:50.023 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:49.241+0000 7f45c5833700 1 -- 172.21.15.47:0/3623566053 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45c0109730 con 0x7f45c0108580 2024-04-17T05:51:50.023 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:49.241+0000 7f45c5833700 1 --2- 172.21.15.47:0/3623566053 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45c0108580 0x7f45c01089a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f45bc0098a0 tx=0x7f45bc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebf7826bdd4cbb66 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:51:50.023 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:49.241+0000 7f45c4831700 1 -- 172.21.15.47:0/3623566053 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f45bc004160 con 0x7f45c0108580 2024-04-17T05:51:50.023 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:49.242+0000 7f45c4831700 1 -- 172.21.15.47:0/3623566053 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f45bc004300 con 0x7f45c0108580 2024-04-17T05:51:50.024 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:49.242+0000 7f45c4831700 1 -- 172.21.15.47:0/3623566053 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f45bc011b50 con 0x7f45c0108580 2024-04-17T05:51:50.024 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:49.242+0000 7f45c7a97700 1 -- 172.21.15.47:0/3623566053 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45c0108580 msgr2=0x7f45c01089a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:51:50.024 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:49.242+0000 7f45c7a97700 1 --2- 172.21.15.47:0/3623566053 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45c0108580 0x7f45c01089a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f45bc0098a0 tx=0x7f45bc009bb0 comp rx=0 tx=0).stop 2024-04-17T05:51:50.024 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:49.242+0000 7f45c7a97700 1 -- 172.21.15.47:0/3623566053 shutdown_connections 2024-04-17T05:51:50.024 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:49.242+0000 7f45c7a97700 1 --2- 172.21.15.47:0/3623566053 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45c0108580 0x7f45c01089a0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:51:50.024 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:49.242+0000 7f45c7a97700 1 -- 172.21.15.47:0/3623566053 >> 172.21.15.47:0/3623566053 conn(0x7f45c0103be0 msgr2=0x7f45c0106000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:51:50.024 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:49.242+0000 7f45c7a97700 1 -- 172.21.15.47:0/3623566053 shutdown_connections 2024-04-17T05:51:50.024 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:49.242+0000 7f45c7a97700 1 -- 172.21.15.47:0/3623566053 wait complete. 2024-04-17T05:51:50.024 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:49.243+0000 7f45c7a97700 1 Processor -- start 2024-04-17T05:51:50.024 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:49.243+0000 7f45c7a97700 1 -- start start 2024-04-17T05:51:50.024 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:49.243+0000 7f45c7a97700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45c0108580 0x7f45c019c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:51:50.024 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:49.243+0000 7f45c7a97700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f45c010cae0 con 0x7f45c0108580 2024-04-17T05:51:50.024 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:49.243+0000 7f45c5833700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45c0108580 0x7f45c019c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:51:50.024 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:49.243+0000 7f45c5833700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45c0108580 0x7f45c019c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57826/0 (socket says 172.21.15.47:57826) 2024-04-17T05:51:50.024 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:49.243+0000 7f45c5833700 1 -- 172.21.15.47:0/1514478229 learned_addr learned my addr 172.21.15.47:0/1514478229 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:51:50.025 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:49.243+0000 7f45c5833700 1 -- 172.21.15.47:0/1514478229 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45c019ca90 con 0x7f45c0108580 2024-04-17T05:51:50.025 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:49.243+0000 7f45c5833700 1 --2- 172.21.15.47:0/1514478229 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45c0108580 0x7f45c019c550 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f45bc009fb0 tx=0x7f45bc011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:51:50.025 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:49.244+0000 7f45b2ffd700 1 -- 172.21.15.47:0/1514478229 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f45bc0038b0 con 0x7f45c0108580 2024-04-17T05:51:50.025 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:49.244+0000 7f45b2ffd700 1 -- 172.21.15.47:0/1514478229 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f45bc011e50 con 0x7f45c0108580 2024-04-17T05:51:50.025 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:49.244+0000 7f45b2ffd700 1 -- 172.21.15.47:0/1514478229 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f45bc019550 con 0x7f45c0108580 2024-04-17T05:51:50.025 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:49.244+0000 7f45c7a97700 1 -- 172.21.15.47:0/1514478229 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45c019cd20 con 0x7f45c0108580 2024-04-17T05:51:50.025 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:49.244+0000 7f45c7a97700 1 -- 172.21.15.47:0/1514478229 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45c019fa10 con 0x7f45c0108580 2024-04-17T05:51:50.025 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:49.245+0000 7f45b2ffd700 1 -- 172.21.15.47:0/1514478229 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f45bc01a430 con 0x7f45c0108580 2024-04-17T05:51:50.025 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:49.245+0000 7f45b2ffd700 1 --2- 172.21.15.47:0/1514478229 >> [v2:172.21.15.47:6800/3632157175,v1:172.21.15.47:6801/3632157175] conn(0x7f45ac0395b0 0x7f45ac03ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:51:50.025 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:49.245+0000 7f45b2ffd700 1 -- 172.21.15.47:0/1514478229 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f45bc04d910 con 0x7f45c0108580 2024-04-17T05:51:50.025 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:49.245+0000 7f45c7a97700 1 -- 172.21.15.47:0/1514478229 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45c0069b80 con 0x7f45c0108580 2024-04-17T05:51:50.025 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:49.245+0000 7f45c5032700 1 --2- 172.21.15.47:0/1514478229 >> [v2:172.21.15.47:6800/3632157175,v1:172.21.15.47:6801/3632157175] conn(0x7f45ac0395b0 0x7f45ac03ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:51:50.025 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:49.246+0000 7f45c5032700 1 --2- 172.21.15.47:0/1514478229 >> [v2:172.21.15.47:6800/3632157175,v1:172.21.15.47:6801/3632157175] conn(0x7f45ac0395b0 0x7f45ac03ba70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f45b4006fd0 tx=0x7f45b4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:51:50.025 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:49.250+0000 7f45b2ffd700 1 -- 172.21.15.47:0/1514478229 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f45bc020240 con 0x7f45c0108580 2024-04-17T05:51:50.026 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:49.462+0000 7f45c7a97700 1 -- 172.21.15.47:0/1514478229 --> [v2:172.21.15.47:6800/3632157175,v1:172.21.15.47:6801/3632157175] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f45c0104e70 con 0x7f45ac0395b0 2024-04-17T05:51:50.026 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:49.463+0000 7f45b2ffd700 1 -- 172.21.15.47:0/1514478229 <== mgr.14120 v2:172.21.15.47:6800/3632157175 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f45c0104e70 con 0x7f45ac0395b0 2024-04-17T05:51:50.026 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:49.465+0000 7f45c7a97700 1 -- 172.21.15.47:0/1514478229 >> [v2:172.21.15.47:6800/3632157175,v1:172.21.15.47:6801/3632157175] conn(0x7f45ac0395b0 msgr2=0x7f45ac03ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:51:50.026 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:49.465+0000 7f45c7a97700 1 --2- 172.21.15.47:0/1514478229 >> [v2:172.21.15.47:6800/3632157175,v1:172.21.15.47:6801/3632157175] conn(0x7f45ac0395b0 0x7f45ac03ba70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f45b4006fd0 tx=0x7f45b4006e40 comp rx=0 tx=0).stop 2024-04-17T05:51:50.026 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:49.465+0000 7f45c7a97700 1 -- 172.21.15.47:0/1514478229 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45c0108580 msgr2=0x7f45c019c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:51:50.026 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:49.465+0000 7f45c7a97700 1 --2- 172.21.15.47:0/1514478229 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45c0108580 0x7f45c019c550 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f45bc009fb0 tx=0x7f45bc011bf0 comp rx=0 tx=0).stop 2024-04-17T05:51:50.026 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:49.465+0000 7f45c7a97700 1 -- 172.21.15.47:0/1514478229 shutdown_connections 2024-04-17T05:51:50.026 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:49.465+0000 7f45c7a97700 1 --2- 172.21.15.47:0/1514478229 >> [v2:172.21.15.47:6800/3632157175,v1:172.21.15.47:6801/3632157175] conn(0x7f45ac0395b0 0x7f45ac03ba70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:51:50.026 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:49.465+0000 7f45c7a97700 1 --2- 172.21.15.47:0/1514478229 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45c0108580 0x7f45c019c550 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:51:50.026 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:49.465+0000 7f45c7a97700 1 -- 172.21.15.47:0/1514478229 >> 172.21.15.47:0/1514478229 conn(0x7f45c0103be0 msgr2=0x7f45c0104760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:51:50.026 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:49.465+0000 7f45c7a97700 1 -- 172.21.15.47:0/1514478229 shutdown_connections 2024-04-17T05:51:50.026 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:49.465+0000 7f45c7a97700 1 -- 172.21.15.47:0/1514478229 wait complete. 2024-04-17T05:51:50.026 INFO:teuthology.orchestra.run.smithi047.stdout:Generating ssh key... 2024-04-17T05:51:50.388 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:50 smithi047 ceph-mon[32223]: [17/Apr/2024:05:51:49] ENGINE Serving on https://172.21.15.47:7150 2024-04-17T05:51:50.388 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:50 smithi047 ceph-mon[32223]: [17/Apr/2024:05:51:49] ENGINE Bus STARTED 2024-04-17T05:51:50.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:50 smithi047 ceph-mon[32223]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T05:51:51.345 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:51 smithi047 ceph-mon[32223]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T05:51:51.345 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:51 smithi047 ceph-mon[32223]: Generating ssh key... 2024-04-17T05:51:51.345 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:51 smithi047 ceph-mon[32223]: from='mgr.14120 172.21.15.47:0/2362399306' entity='mgr.x' 2024-04-17T05:51:51.346 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:51 smithi047 ceph-mon[32223]: from='mgr.14120 172.21.15.47:0/2362399306' entity='mgr.x' 2024-04-17T05:51:51.346 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 05:51:51 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: Generating public/private rsa key pair. 2024-04-17T05:51:51.346 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 05:51:51 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: Your identification has been saved in /tmp/tmpsptyihyq/key. 2024-04-17T05:51:51.346 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 05:51:51 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: Your public key has been saved in /tmp/tmpsptyihyq/key.pub. 2024-04-17T05:51:51.346 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 05:51:51 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: The key fingerprint is: 2024-04-17T05:51:51.346 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 05:51:51 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: SHA256:j5HWRSfSuwzyT/Zsio3rwV3VO7Bvh/TspHrNEyf8H+c ceph-565db2d8-fc7e-11ee-bc91-c7b262605968 2024-04-17T05:51:51.346 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 05:51:51 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: The key's randomart image is: 2024-04-17T05:51:51.346 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 05:51:51 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: +---[RSA 3072]----+ 2024-04-17T05:51:51.346 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 05:51:51 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: | ..o . | 2024-04-17T05:51:51.346 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 05:51:51 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: | o.o .| 2024-04-17T05:51:51.346 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 05:51:51 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: | .o o| 2024-04-17T05:51:51.346 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 05:51:51 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: | .o... o..| 2024-04-17T05:51:51.347 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 05:51:51 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: | So.o +oo | 2024-04-17T05:51:51.347 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 05:51:51 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: | . =..=o+=o| 2024-04-17T05:51:51.347 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 05:51:51 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: | . ++.o.OO| 2024-04-17T05:51:51.347 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 05:51:51 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: | =. **B| 2024-04-17T05:51:51.347 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 05:51:51 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: | .=.+=. E| 2024-04-17T05:51:51.347 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 05:51:51 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: +----[SHA256]-----+ 2024-04-17T05:51:51.630 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:50.461+0000 7ff26c47c700 1 Processor -- start 2024-04-17T05:51:51.631 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:50.461+0000 7ff26c47c700 1 -- start start 2024-04-17T05:51:51.631 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:50.461+0000 7ff26c47c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff264108540 0x7ff264108960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:51:51.631 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:50.461+0000 7ff26c47c700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff264108f30 con 0x7ff264108540 2024-04-17T05:51:51.631 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:50.461+0000 7ff26a218700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff264108540 0x7ff264108960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:51:51.631 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:50.461+0000 7ff26a218700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff264108540 0x7ff264108960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57832/0 (socket says 172.21.15.47:57832) 2024-04-17T05:51:51.631 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:50.461+0000 7ff26a218700 1 -- 172.21.15.47:0/3386656288 learned_addr learned my addr 172.21.15.47:0/3386656288 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:51:51.631 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:50.462+0000 7ff26a218700 1 -- 172.21.15.47:0/3386656288 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2641096f0 con 0x7ff264108540 2024-04-17T05:51:51.631 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:50.462+0000 7ff26a218700 1 --2- 172.21.15.47:0/3386656288 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff264108540 0x7ff264108960 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff2600098a0 tx=0x7ff260009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d96366b7b44f48aa server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:51:51.631 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:50.462+0000 7ff269216700 1 -- 172.21.15.47:0/3386656288 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff260004160 con 0x7ff264108540 2024-04-17T05:51:51.631 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:50.462+0000 7ff269216700 1 -- 172.21.15.47:0/3386656288 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff260004300 con 0x7ff264108540 2024-04-17T05:51:51.631 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:50.462+0000 7ff269216700 1 -- 172.21.15.47:0/3386656288 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff260011b50 con 0x7ff264108540 2024-04-17T05:51:51.632 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:50.463+0000 7ff26c47c700 1 -- 172.21.15.47:0/3386656288 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff264108540 msgr2=0x7ff264108960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:51:51.632 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:50.463+0000 7ff26c47c700 1 --2- 172.21.15.47:0/3386656288 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff264108540 0x7ff264108960 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff2600098a0 tx=0x7ff260009bb0 comp rx=0 tx=0).stop 2024-04-17T05:51:51.632 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:50.463+0000 7ff26c47c700 1 -- 172.21.15.47:0/3386656288 shutdown_connections 2024-04-17T05:51:51.632 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:50.463+0000 7ff26c47c700 1 --2- 172.21.15.47:0/3386656288 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff264108540 0x7ff264108960 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:51:51.632 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:50.463+0000 7ff26c47c700 1 -- 172.21.15.47:0/3386656288 >> 172.21.15.47:0/3386656288 conn(0x7ff264103b80 msgr2=0x7ff264105fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:51:51.632 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:50.463+0000 7ff26c47c700 1 -- 172.21.15.47:0/3386656288 shutdown_connections 2024-04-17T05:51:51.632 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:50.463+0000 7ff26c47c700 1 -- 172.21.15.47:0/3386656288 wait complete. 2024-04-17T05:51:51.632 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:50.463+0000 7ff26c47c700 1 Processor -- start 2024-04-17T05:51:51.632 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:50.463+0000 7ff26c47c700 1 -- start start 2024-04-17T05:51:51.632 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:50.464+0000 7ff26c47c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff264108540 0x7ff26419c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:51:51.632 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:50.464+0000 7ff26c47c700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff26410caa0 con 0x7ff264108540 2024-04-17T05:51:51.632 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:50.464+0000 7ff26a218700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff264108540 0x7ff26419c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:51:51.632 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:50.464+0000 7ff26a218700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff264108540 0x7ff26419c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57836/0 (socket says 172.21.15.47:57836) 2024-04-17T05:51:51.633 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:50.464+0000 7ff26a218700 1 -- 172.21.15.47:0/695774846 learned_addr learned my addr 172.21.15.47:0/695774846 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:51:51.633 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:50.464+0000 7ff26a218700 1 -- 172.21.15.47:0/695774846 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff26419ca20 con 0x7ff264108540 2024-04-17T05:51:51.633 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:50.464+0000 7ff26a218700 1 --2- 172.21.15.47:0/695774846 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff264108540 0x7ff26419c4e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff260011670 tx=0x7ff260011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:51:51.633 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:50.464+0000 7ff25b7fe700 1 -- 172.21.15.47:0/695774846 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff2600039c0 con 0x7ff264108540 2024-04-17T05:51:51.633 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:50.464+0000 7ff25b7fe700 1 -- 172.21.15.47:0/695774846 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff260011e50 con 0x7ff264108540 2024-04-17T05:51:51.633 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:50.464+0000 7ff26c47c700 1 -- 172.21.15.47:0/695774846 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff26419ccb0 con 0x7ff264108540 2024-04-17T05:51:51.633 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:50.464+0000 7ff25b7fe700 1 -- 172.21.15.47:0/695774846 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff26001a5c0 con 0x7ff264108540 2024-04-17T05:51:51.633 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:50.464+0000 7ff26c47c700 1 -- 172.21.15.47:0/695774846 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff26419f9a0 con 0x7ff264108540 2024-04-17T05:51:51.633 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:50.465+0000 7ff25b7fe700 1 -- 172.21.15.47:0/695774846 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7ff26001a870 con 0x7ff264108540 2024-04-17T05:51:51.633 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:50.465+0000 7ff26c47c700 1 -- 172.21.15.47:0/695774846 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2640640b0 con 0x7ff264108540 2024-04-17T05:51:51.633 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:50.466+0000 7ff25b7fe700 1 --2- 172.21.15.47:0/695774846 >> [v2:172.21.15.47:6800/3632157175,v1:172.21.15.47:6801/3632157175] conn(0x7ff2500395b0 0x7ff25003ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:51:51.633 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:50.466+0000 7ff25b7fe700 1 -- 172.21.15.47:0/695774846 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ff26004da80 con 0x7ff264108540 2024-04-17T05:51:51.634 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:50.466+0000 7ff269a17700 1 --2- 172.21.15.47:0/695774846 >> [v2:172.21.15.47:6800/3632157175,v1:172.21.15.47:6801/3632157175] conn(0x7ff2500395b0 0x7ff25003ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:51:51.634 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:50.466+0000 7ff269a17700 1 --2- 172.21.15.47:0/695774846 >> [v2:172.21.15.47:6800/3632157175,v1:172.21.15.47:6801/3632157175] conn(0x7ff2500395b0 0x7ff25003ba70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff254006fd0 tx=0x7ff254006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:51:51.634 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:50.471+0000 7ff25b7fe700 1 -- 172.21.15.47:0/695774846 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7ff26002d3f0 con 0x7ff264108540 2024-04-17T05:51:51.634 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:50.678+0000 7ff26c47c700 1 -- 172.21.15.47:0/695774846 --> [v2:172.21.15.47:6800/3632157175,v1:172.21.15.47:6801/3632157175] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7ff264104dd0 con 0x7ff2500395b0 2024-04-17T05:51:51.634 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:51.054+0000 7ff25b7fe700 1 -- 172.21.15.47:0/695774846 <== mgr.14120 v2:172.21.15.47:6800/3632157175 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff264104dd0 con 0x7ff2500395b0 2024-04-17T05:51:51.634 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:51.057+0000 7ff26c47c700 1 -- 172.21.15.47:0/695774846 >> [v2:172.21.15.47:6800/3632157175,v1:172.21.15.47:6801/3632157175] conn(0x7ff2500395b0 msgr2=0x7ff25003ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:51:51.634 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:51.057+0000 7ff26c47c700 1 --2- 172.21.15.47:0/695774846 >> [v2:172.21.15.47:6800/3632157175,v1:172.21.15.47:6801/3632157175] conn(0x7ff2500395b0 0x7ff25003ba70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff254006fd0 tx=0x7ff254006e40 comp rx=0 tx=0).stop 2024-04-17T05:51:51.634 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:51.057+0000 7ff26c47c700 1 -- 172.21.15.47:0/695774846 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff264108540 msgr2=0x7ff26419c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:51:51.634 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:51.057+0000 7ff26c47c700 1 --2- 172.21.15.47:0/695774846 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff264108540 0x7ff26419c4e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff260011670 tx=0x7ff260011b60 comp rx=0 tx=0).stop 2024-04-17T05:51:51.634 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:51.057+0000 7ff26c47c700 1 -- 172.21.15.47:0/695774846 shutdown_connections 2024-04-17T05:51:51.634 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:51.057+0000 7ff26c47c700 1 --2- 172.21.15.47:0/695774846 >> [v2:172.21.15.47:6800/3632157175,v1:172.21.15.47:6801/3632157175] conn(0x7ff2500395b0 0x7ff25003ba70 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:51:51.634 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:51.057+0000 7ff26c47c700 1 --2- 172.21.15.47:0/695774846 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff264108540 0x7ff26419c4e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:51:51.634 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:51.057+0000 7ff26c47c700 1 -- 172.21.15.47:0/695774846 >> 172.21.15.47:0/695774846 conn(0x7ff264103b80 msgr2=0x7ff2641046c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:51:51.635 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:51.057+0000 7ff26c47c700 1 -- 172.21.15.47:0/695774846 shutdown_connections 2024-04-17T05:51:51.635 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:51.057+0000 7ff26c47c700 1 -- 172.21.15.47:0/695774846 wait complete. 2024-04-17T05:51:52.918 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDXDAtDHw0jii0KMV41FXBKNtkjOglzKcQhGLS0Kp5FMXfVIyR8qMDYWBlKAMPr+hpfasq1z8x1/zxJx9tq605jMqHIWcNqTuJFwZt6kJHJNW4tawoeHKyXr3QSYQwH7zcfX498XufYKhrMRt/vmQLLPvx1vcZ8QtKPaexF+/mVQRL/cvYqwwNmZgSpkGhwzG3gkNZML27yuh2bsGhu3VtY3NkzuonSxWXZMDeAGZVqE6cW0x/Mg7apTEPgWTgN09ng3I+bDj0+eN6jZwxNoh7qqLaVtcbmvPrNph4vgBGVzYGJaRsIThmjiN2E2jBv2IO9ku5ng/Hg88b2xDdB3F3VSKk53P/gwoDmqf+6Ah2Zflqeeebj63c4nFjRMXfi/kHRof2lr+OofeFYOyqFfXUKWJytd7CZztVhhMm0c1yVMhCM7Stvk7cfodJgIEC2RWP0+R9KgKqjI095xsmZS3ARk4RunLbf1D7vo1pFQjiIUkzKaNtsYYOFv0UO5gTjoLk= ceph-565db2d8-fc7e-11ee-bc91-c7b262605968 2024-04-17T05:51:52.919 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:52.097+0000 7f3e33ded700 1 Processor -- start 2024-04-17T05:51:52.919 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:52.097+0000 7f3e33ded700 1 -- start start 2024-04-17T05:51:52.919 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:52.097+0000 7f3e33ded700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3e2c106c20 0x7f3e2c107040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:51:52.919 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:52.097+0000 7f3e33ded700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f3e2c107610 con 0x7f3e2c106c20 2024-04-17T05:51:52.919 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:52.097+0000 7f3e31b89700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3e2c106c20 0x7f3e2c107040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:51:52.919 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:52.097+0000 7f3e31b89700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3e2c106c20 0x7f3e2c107040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57852/0 (socket says 172.21.15.47:57852) 2024-04-17T05:51:52.919 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:52.097+0000 7f3e31b89700 1 -- 172.21.15.47:0/2725318290 learned_addr learned my addr 172.21.15.47:0/2725318290 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:51:52.919 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:52.098+0000 7f3e31b89700 1 -- 172.21.15.47:0/2725318290 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e2c107e20 con 0x7f3e2c106c20 2024-04-17T05:51:52.919 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:52.098+0000 7f3e31b89700 1 --2- 172.21.15.47:0/2725318290 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3e2c106c20 0x7f3e2c107040 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3e2800bb40 tx=0x7f3e2800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=702333026d87d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:51:52.919 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:52.098+0000 7f3e30b87700 1 -- 172.21.15.47:0/2725318290 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3e2800dad0 con 0x7f3e2c106c20 2024-04-17T05:51:52.919 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:52.098+0000 7f3e30b87700 1 -- 172.21.15.47:0/2725318290 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3e280044f0 con 0x7f3e2c106c20 2024-04-17T05:51:52.919 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:52.098+0000 7f3e30b87700 1 -- 172.21.15.47:0/2725318290 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3e28017410 con 0x7f3e2c106c20 2024-04-17T05:51:52.919 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:52.099+0000 7f3e33ded700 1 -- 172.21.15.47:0/2725318290 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3e2c106c20 msgr2=0x7f3e2c107040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:51:52.920 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:52.099+0000 7f3e33ded700 1 --2- 172.21.15.47:0/2725318290 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3e2c106c20 0x7f3e2c107040 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3e2800bb40 tx=0x7f3e2800be50 comp rx=0 tx=0).stop 2024-04-17T05:51:52.920 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:52.099+0000 7f3e33ded700 1 -- 172.21.15.47:0/2725318290 shutdown_connections 2024-04-17T05:51:52.920 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:52.099+0000 7f3e33ded700 1 --2- 172.21.15.47:0/2725318290 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3e2c106c20 0x7f3e2c107040 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:51:52.920 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:52.099+0000 7f3e33ded700 1 -- 172.21.15.47:0/2725318290 >> 172.21.15.47:0/2725318290 conn(0x7f3e2c102240 msgr2=0x7f3e2c104660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:51:52.920 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:52.099+0000 7f3e33ded700 1 -- 172.21.15.47:0/2725318290 shutdown_connections 2024-04-17T05:51:52.920 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:52.099+0000 7f3e33ded700 1 -- 172.21.15.47:0/2725318290 wait complete. 2024-04-17T05:51:52.920 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:52.100+0000 7f3e33ded700 1 Processor -- start 2024-04-17T05:51:52.920 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:52.100+0000 7f3e33ded700 1 -- start start 2024-04-17T05:51:52.920 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:52.100+0000 7f3e33ded700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3e2c106c20 0x7f3e2c19e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:51:52.920 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:52.100+0000 7f3e33ded700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f3e2c108540 con 0x7f3e2c106c20 2024-04-17T05:51:52.920 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:52.100+0000 7f3e31b89700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3e2c106c20 0x7f3e2c19e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:51:52.920 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:52.100+0000 7f3e31b89700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3e2c106c20 0x7f3e2c19e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57868/0 (socket says 172.21.15.47:57868) 2024-04-17T05:51:52.920 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:52.100+0000 7f3e31b89700 1 -- 172.21.15.47:0/45081040 learned_addr learned my addr 172.21.15.47:0/45081040 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:51:52.920 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:52.100+0000 7f3e31b89700 1 -- 172.21.15.47:0/45081040 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e2c19ec40 con 0x7f3e2c106c20 2024-04-17T05:51:52.921 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:52.100+0000 7f3e31b89700 1 --2- 172.21.15.47:0/45081040 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3e2c106c20 0x7f3e2c19e700 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3e2800bb10 tx=0x7f3e280047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:51:52.921 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:52.100+0000 7f3e22ffd700 1 -- 172.21.15.47:0/45081040 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3e28004110 con 0x7f3e2c106c20 2024-04-17T05:51:52.921 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:52.101+0000 7f3e22ffd700 1 -- 172.21.15.47:0/45081040 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3e28024070 con 0x7f3e2c106c20 2024-04-17T05:51:52.921 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:52.101+0000 7f3e33ded700 1 -- 172.21.15.47:0/45081040 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e2c19eed0 con 0x7f3e2c106c20 2024-04-17T05:51:52.921 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:52.101+0000 7f3e22ffd700 1 -- 172.21.15.47:0/45081040 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3e2801fa40 con 0x7f3e2c106c20 2024-04-17T05:51:52.921 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:52.101+0000 7f3e33ded700 1 -- 172.21.15.47:0/45081040 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e2c1a1bc0 con 0x7f3e2c106c20 2024-04-17T05:51:52.921 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:52.101+0000 7f3e22ffd700 1 -- 172.21.15.47:0/45081040 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f3e280177b0 con 0x7f3e2c106c20 2024-04-17T05:51:52.921 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:52.102+0000 7f3e33ded700 1 -- 172.21.15.47:0/45081040 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e2c0640b0 con 0x7f3e2c106c20 2024-04-17T05:51:52.921 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:52.102+0000 7f3e22ffd700 1 --2- 172.21.15.47:0/45081040 >> [v2:172.21.15.47:6800/3632157175,v1:172.21.15.47:6801/3632157175] conn(0x7f3e180395b0 0x7f3e1803ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:51:52.921 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:52.102+0000 7f3e22ffd700 1 -- 172.21.15.47:0/45081040 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f3e2804dab0 con 0x7f3e2c106c20 2024-04-17T05:51:52.921 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:52.102+0000 7f3e31388700 1 --2- 172.21.15.47:0/45081040 >> [v2:172.21.15.47:6800/3632157175,v1:172.21.15.47:6801/3632157175] conn(0x7f3e180395b0 0x7f3e1803ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:51:52.921 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:52.102+0000 7f3e31388700 1 --2- 172.21.15.47:0/45081040 >> [v2:172.21.15.47:6800/3632157175,v1:172.21.15.47:6801/3632157175] conn(0x7f3e180395b0 0x7f3e1803ba70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3e1c006fd0 tx=0x7f3e1c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:51:52.921 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:52.107+0000 7f3e22ffd700 1 -- 172.21.15.47:0/45081040 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f3e28012070 con 0x7f3e2c106c20 2024-04-17T05:51:52.921 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:52.315+0000 7f3e33ded700 1 -- 172.21.15.47:0/45081040 --> [v2:172.21.15.47:6800/3632157175,v1:172.21.15.47:6801/3632157175] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f3e2c0008d0 con 0x7f3e180395b0 2024-04-17T05:51:52.922 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:52.316+0000 7f3e22ffd700 1 -- 172.21.15.47:0/45081040 <== mgr.14120 v2:172.21.15.47:6800/3632157175 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f3e2c0008d0 con 0x7f3e180395b0 2024-04-17T05:51:52.922 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:52.318+0000 7f3e33ded700 1 -- 172.21.15.47:0/45081040 >> [v2:172.21.15.47:6800/3632157175,v1:172.21.15.47:6801/3632157175] conn(0x7f3e180395b0 msgr2=0x7f3e1803ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:51:52.922 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:52.318+0000 7f3e33ded700 1 --2- 172.21.15.47:0/45081040 >> [v2:172.21.15.47:6800/3632157175,v1:172.21.15.47:6801/3632157175] conn(0x7f3e180395b0 0x7f3e1803ba70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3e1c006fd0 tx=0x7f3e1c006e40 comp rx=0 tx=0).stop 2024-04-17T05:51:52.922 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:52.318+0000 7f3e33ded700 1 -- 172.21.15.47:0/45081040 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3e2c106c20 msgr2=0x7f3e2c19e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:51:52.922 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:52.318+0000 7f3e33ded700 1 --2- 172.21.15.47:0/45081040 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3e2c106c20 0x7f3e2c19e700 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3e2800bb10 tx=0x7f3e280047c0 comp rx=0 tx=0).stop 2024-04-17T05:51:52.922 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:52.318+0000 7f3e33ded700 1 -- 172.21.15.47:0/45081040 shutdown_connections 2024-04-17T05:51:52.922 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:52.318+0000 7f3e33ded700 1 --2- 172.21.15.47:0/45081040 >> [v2:172.21.15.47:6800/3632157175,v1:172.21.15.47:6801/3632157175] conn(0x7f3e180395b0 0x7f3e1803ba70 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:51:52.922 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:52.318+0000 7f3e33ded700 1 --2- 172.21.15.47:0/45081040 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3e2c106c20 0x7f3e2c19e700 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:51:52.922 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:52.318+0000 7f3e33ded700 1 -- 172.21.15.47:0/45081040 >> 172.21.15.47:0/45081040 conn(0x7f3e2c102240 msgr2=0x7f3e2c190ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:51:52.922 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:52.318+0000 7f3e33ded700 1 -- 172.21.15.47:0/45081040 shutdown_connections 2024-04-17T05:51:52.922 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:52.318+0000 7f3e33ded700 1 -- 172.21.15.47:0/45081040 wait complete. 2024-04-17T05:51:52.922 INFO:teuthology.orchestra.run.smithi047.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-04-17T05:51:52.922 INFO:teuthology.orchestra.run.smithi047.stdout:Adding key to root@localhost authorized_keys... 2024-04-17T05:51:52.922 INFO:teuthology.orchestra.run.smithi047.stdout:Adding host smithi047... 2024-04-17T05:51:53.280 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:53 smithi047 ceph-mon[32223]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T05:51:54.154 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:54 smithi047 ceph-mon[32223]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi047", "addr": "172.21.15.47", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T05:51:55.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:55 smithi047 ceph-mon[32223]: Deploying cephadm binary to smithi047 2024-04-17T05:51:58.592 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout Added host 'smithi047' with addr '172.21.15.47' 2024-04-17T05:51:58.593 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:53.332+0000 7f0ce10b2700 1 Processor -- start 2024-04-17T05:51:58.593 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:53.332+0000 7f0ce10b2700 1 -- start start 2024-04-17T05:51:58.593 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:53.332+0000 7f0ce10b2700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0cdc108fd0 0x7f0cdc10b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:51:58.593 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:53.332+0000 7f0ce10b2700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f0cdc07be70 con 0x7f0cdc108fd0 2024-04-17T05:51:58.593 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:53.332+0000 7f0cdad9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0cdc108fd0 0x7f0cdc10b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:51:58.593 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:53.332+0000 7f0cdad9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0cdc108fd0 0x7f0cdc10b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57878/0 (socket says 172.21.15.47:57878) 2024-04-17T05:51:58.593 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:53.332+0000 7f0cdad9d700 1 -- 172.21.15.47:0/1922504849 learned_addr learned my addr 172.21.15.47:0/1922504849 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:51:58.593 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:53.333+0000 7f0cdad9d700 1 -- 172.21.15.47:0/1922504849 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cdc10b900 con 0x7f0cdc108fd0 2024-04-17T05:51:58.593 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:53.333+0000 7f0cdad9d700 1 --2- 172.21.15.47:0/1922504849 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0cdc108fd0 0x7f0cdc10b3c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc009e80 tx=0x7f0ccc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e27357207afcda7 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:51:58.593 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:53.333+0000 7f0cd9d9b700 1 -- 172.21.15.47:0/1922504849 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0ccc017070 con 0x7f0cdc108fd0 2024-04-17T05:51:58.593 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:53.333+0000 7f0cd9d9b700 1 -- 172.21.15.47:0/1922504849 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0ccc00b8c0 con 0x7f0cdc108fd0 2024-04-17T05:51:58.593 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:53.333+0000 7f0cd9d9b700 1 -- 172.21.15.47:0/1922504849 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0ccc0045c0 con 0x7f0cdc108fd0 2024-04-17T05:51:58.594 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:53.333+0000 7f0ce10b2700 1 -- 172.21.15.47:0/1922504849 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0cdc108fd0 msgr2=0x7f0cdc10b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:51:58.594 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:53.333+0000 7f0ce10b2700 1 --2- 172.21.15.47:0/1922504849 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0cdc108fd0 0x7f0cdc10b3c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc009e80 tx=0x7f0ccc00b330 comp rx=0 tx=0).stop 2024-04-17T05:51:58.594 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:53.333+0000 7f0ce10b2700 1 -- 172.21.15.47:0/1922504849 shutdown_connections 2024-04-17T05:51:58.594 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:53.333+0000 7f0ce10b2700 1 --2- 172.21.15.47:0/1922504849 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0cdc108fd0 0x7f0cdc10b3c0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:51:58.594 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:53.333+0000 7f0ce10b2700 1 -- 172.21.15.47:0/1922504849 >> 172.21.15.47:0/1922504849 conn(0x7f0cdc102a00 msgr2=0x7f0cdc104e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:51:58.594 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:53.334+0000 7f0ce10b2700 1 -- 172.21.15.47:0/1922504849 shutdown_connections 2024-04-17T05:51:58.594 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:53.334+0000 7f0ce10b2700 1 -- 172.21.15.47:0/1922504849 wait complete. 2024-04-17T05:51:58.594 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:53.334+0000 7f0ce10b2700 1 Processor -- start 2024-04-17T05:51:58.594 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:53.334+0000 7f0ce10b2700 1 -- start start 2024-04-17T05:51:58.594 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:53.335+0000 7f0ce10b2700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0cdc108fd0 0x7f0cdc19c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:51:58.594 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:53.335+0000 7f0ce10b2700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f0cdc10c8d0 con 0x7f0cdc108fd0 2024-04-17T05:51:58.594 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:53.335+0000 7f0cdad9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0cdc108fd0 0x7f0cdc19c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:51:58.595 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:53.335+0000 7f0cdad9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0cdc108fd0 0x7f0cdc19c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57890/0 (socket says 172.21.15.47:57890) 2024-04-17T05:51:58.595 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:53.335+0000 7f0cdad9d700 1 -- 172.21.15.47:0/3303903947 learned_addr learned my addr 172.21.15.47:0/3303903947 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:51:58.595 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:53.335+0000 7f0cdad9d700 1 -- 172.21.15.47:0/3303903947 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cdc19c9e0 con 0x7f0cdc108fd0 2024-04-17T05:51:58.595 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:53.335+0000 7f0cdad9d700 1 --2- 172.21.15.47:0/3303903947 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0cdc108fd0 0x7f0cdc19c4a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc0072a0 tx=0x7f0ccc00bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:51:58.595 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:53.335+0000 7f0cc3fff700 1 -- 172.21.15.47:0/3303903947 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0ccc020020 con 0x7f0cdc108fd0 2024-04-17T05:51:58.595 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:53.335+0000 7f0ce10b2700 1 -- 172.21.15.47:0/3303903947 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cdc19cc70 con 0x7f0cdc108fd0 2024-04-17T05:51:58.595 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:53.335+0000 7f0cc3fff700 1 -- 172.21.15.47:0/3303903947 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0ccc003b50 con 0x7f0cdc108fd0 2024-04-17T05:51:58.595 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:53.335+0000 7f0cc3fff700 1 -- 172.21.15.47:0/3303903947 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0ccc017040 con 0x7f0cdc108fd0 2024-04-17T05:51:58.595 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:53.335+0000 7f0ce10b2700 1 -- 172.21.15.47:0/3303903947 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cdc19f960 con 0x7f0cdc108fd0 2024-04-17T05:51:58.595 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:53.336+0000 7f0cc3fff700 1 -- 172.21.15.47:0/3303903947 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f0ccc023020 con 0x7f0cdc108fd0 2024-04-17T05:51:58.595 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:53.336+0000 7f0cc3fff700 1 --2- 172.21.15.47:0/3303903947 >> [v2:172.21.15.47:6800/3632157175,v1:172.21.15.47:6801/3632157175] conn(0x7f0cc4039560 0x7f0cc403ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:51:58.595 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:53.336+0000 7f0cc3fff700 1 -- 172.21.15.47:0/3303903947 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f0ccc00f050 con 0x7f0cdc108fd0 2024-04-17T05:51:58.595 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:53.337+0000 7f0cda59c700 1 --2- 172.21.15.47:0/3303903947 >> [v2:172.21.15.47:6800/3632157175,v1:172.21.15.47:6801/3632157175] conn(0x7f0cc4039560 0x7f0cc403ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:51:58.595 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:53.337+0000 7f0ce10b2700 1 -- 172.21.15.47:0/3303903947 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cdc0640b0 con 0x7f0cdc108fd0 2024-04-17T05:51:58.596 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:53.337+0000 7f0cda59c700 1 --2- 172.21.15.47:0/3303903947 >> [v2:172.21.15.47:6800/3632157175,v1:172.21.15.47:6801/3632157175] conn(0x7f0cc4039560 0x7f0cc403ba20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0cd0006fd0 tx=0x7f0cd0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:51:58.596 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:53.342+0000 7f0cc3fff700 1 -- 172.21.15.47:0/3303903947 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f0ccc02a020 con 0x7f0cdc108fd0 2024-04-17T05:51:58.596 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:53.553+0000 7f0ce10b2700 1 -- 172.21.15.47:0/3303903947 --> [v2:172.21.15.47:6800/3632157175,v1:172.21.15.47:6801/3632157175] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi047", "addr": "172.21.15.47", "target": ["mon-mgr", ""]}) -- 0x7f0cdc103cc0 con 0x7f0cc4039560 2024-04-17T05:51:58.596 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:58.025+0000 7f0cc3fff700 1 -- 172.21.15.47:0/3303903947 <== mgr.14120 v2:172.21.15.47:6800/3632157175 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f0cdc103cc0 con 0x7f0cc4039560 2024-04-17T05:51:58.596 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:58.028+0000 7f0ce10b2700 1 -- 172.21.15.47:0/3303903947 >> [v2:172.21.15.47:6800/3632157175,v1:172.21.15.47:6801/3632157175] conn(0x7f0cc4039560 msgr2=0x7f0cc403ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:51:58.596 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:58.028+0000 7f0ce10b2700 1 --2- 172.21.15.47:0/3303903947 >> [v2:172.21.15.47:6800/3632157175,v1:172.21.15.47:6801/3632157175] conn(0x7f0cc4039560 0x7f0cc403ba20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0cd0006fd0 tx=0x7f0cd0006e40 comp rx=0 tx=0).stop 2024-04-17T05:51:58.596 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:58.028+0000 7f0ce10b2700 1 -- 172.21.15.47:0/3303903947 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0cdc108fd0 msgr2=0x7f0cdc19c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:51:58.596 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:58.028+0000 7f0ce10b2700 1 --2- 172.21.15.47:0/3303903947 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0cdc108fd0 0x7f0cdc19c4a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc0072a0 tx=0x7f0ccc00bd50 comp rx=0 tx=0).stop 2024-04-17T05:51:58.596 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:58.028+0000 7f0ce10b2700 1 -- 172.21.15.47:0/3303903947 shutdown_connections 2024-04-17T05:51:58.596 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:58.028+0000 7f0ce10b2700 1 --2- 172.21.15.47:0/3303903947 >> [v2:172.21.15.47:6800/3632157175,v1:172.21.15.47:6801/3632157175] conn(0x7f0cc4039560 0x7f0cc403ba20 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:51:58.596 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:58.028+0000 7f0ce10b2700 1 --2- 172.21.15.47:0/3303903947 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0cdc108fd0 0x7f0cdc19c4a0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:51:58.596 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:58.028+0000 7f0ce10b2700 1 -- 172.21.15.47:0/3303903947 >> 172.21.15.47:0/3303903947 conn(0x7f0cdc102a00 msgr2=0x7f0cdc1035b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:51:58.596 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:58.028+0000 7f0ce10b2700 1 -- 172.21.15.47:0/3303903947 shutdown_connections 2024-04-17T05:51:58.596 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:58.028+0000 7f0ce10b2700 1 -- 172.21.15.47:0/3303903947 wait complete. 2024-04-17T05:51:58.597 INFO:teuthology.orchestra.run.smithi047.stdout:Deploying unmanaged mon service... 2024-04-17T05:51:59.216 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:59 smithi047 ceph-mon[32223]: from='mgr.14120 172.21.15.47:0/2362399306' entity='mgr.x' 2024-04-17T05:51:59.217 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:59 smithi047 ceph-mon[32223]: Added host smithi047 2024-04-17T05:51:59.217 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:51:59 smithi047 ceph-mon[32223]: from='mgr.14120 172.21.15.47:0/2362399306' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T05:52:00.583 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:00 smithi047 ceph-mon[32223]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T05:52:00.583 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:00 smithi047 ceph-mon[32223]: Saving service mon spec with placement count:5 2024-04-17T05:52:00.583 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:00 smithi047 ceph-mon[32223]: from='mgr.14120 172.21.15.47:0/2362399306' entity='mgr.x' 2024-04-17T05:52:00.851 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-04-17T05:52:00.851 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:59.055+0000 7f2feaf04700 1 Processor -- start 2024-04-17T05:52:00.851 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:59.055+0000 7f2feaf04700 1 -- start start 2024-04-17T05:52:00.851 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:59.055+0000 7f2feaf04700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2fe407de50 0x7f2fe407c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:52:00.851 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:59.055+0000 7f2feaf04700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f2fe407c840 con 0x7f2fe407de50 2024-04-17T05:52:00.851 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:59.055+0000 7f2fe8ca0700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2fe407de50 0x7f2fe407c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:52:00.851 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:59.055+0000 7f2fe8ca0700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2fe407de50 0x7f2fe407c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43582/0 (socket says 172.21.15.47:43582) 2024-04-17T05:52:00.851 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:59.055+0000 7f2fe8ca0700 1 -- 172.21.15.47:0/598756491 learned_addr learned my addr 172.21.15.47:0/598756491 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:52:00.851 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:59.056+0000 7f2fe8ca0700 1 -- 172.21.15.47:0/598756491 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fe407c9c0 con 0x7f2fe407de50 2024-04-17T05:52:00.851 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:59.056+0000 7f2fe8ca0700 1 --2- 172.21.15.47:0/598756491 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2fe407de50 0x7f2fe407c300 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2fd80098a0 tx=0x7f2fd8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65988eda6f34cb1e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:52:00.851 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:59.056+0000 7f2fe37fe700 1 -- 172.21.15.47:0/598756491 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2fd8004160 con 0x7f2fe407de50 2024-04-17T05:52:00.851 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:59.056+0000 7f2fe37fe700 1 -- 172.21.15.47:0/598756491 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2fd8004300 con 0x7f2fe407de50 2024-04-17T05:52:00.852 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:59.056+0000 7f2fe37fe700 1 -- 172.21.15.47:0/598756491 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2fd8011b50 con 0x7f2fe407de50 2024-04-17T05:52:00.852 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:59.056+0000 7f2feaf04700 1 -- 172.21.15.47:0/598756491 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2fe407de50 msgr2=0x7f2fe407c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:52:00.852 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:59.056+0000 7f2feaf04700 1 --2- 172.21.15.47:0/598756491 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2fe407de50 0x7f2fe407c300 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2fd80098a0 tx=0x7f2fd8009bb0 comp rx=0 tx=0).stop 2024-04-17T05:52:00.852 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:59.056+0000 7f2feaf04700 1 -- 172.21.15.47:0/598756491 shutdown_connections 2024-04-17T05:52:00.852 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:59.056+0000 7f2feaf04700 1 --2- 172.21.15.47:0/598756491 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2fe407de50 0x7f2fe407c300 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:52:00.852 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:59.056+0000 7f2feaf04700 1 -- 172.21.15.47:0/598756491 >> 172.21.15.47:0/598756491 conn(0x7f2fe4103b80 msgr2=0x7f2fe4105fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:52:00.852 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:59.056+0000 7f2feaf04700 1 -- 172.21.15.47:0/598756491 shutdown_connections 2024-04-17T05:52:00.852 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:59.057+0000 7f2feaf04700 1 -- 172.21.15.47:0/598756491 wait complete. 2024-04-17T05:52:00.852 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:59.057+0000 7f2feaf04700 1 Processor -- start 2024-04-17T05:52:00.852 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:59.057+0000 7f2feaf04700 1 -- start start 2024-04-17T05:52:00.852 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:59.057+0000 7f2feaf04700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2fe407de50 0x7f2fe410b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:52:00.852 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:59.057+0000 7f2feaf04700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f2fe410ead0 con 0x7f2fe407de50 2024-04-17T05:52:00.852 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:59.057+0000 7f2fe8ca0700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2fe407de50 0x7f2fe410b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:52:00.852 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:59.057+0000 7f2fe8ca0700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2fe407de50 0x7f2fe410b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43596/0 (socket says 172.21.15.47:43596) 2024-04-17T05:52:00.852 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:59.057+0000 7f2fe8ca0700 1 -- 172.21.15.47:0/3838008264 learned_addr learned my addr 172.21.15.47:0/3838008264 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:52:00.853 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:59.058+0000 7f2fe8ca0700 1 -- 172.21.15.47:0/3838008264 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fe410bcf0 con 0x7f2fe407de50 2024-04-17T05:52:00.853 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:59.058+0000 7f2fe8ca0700 1 --2- 172.21.15.47:0/3838008264 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2fe407de50 0x7f2fe410b7b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2fd8011670 tx=0x7f2fd8011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:52:00.853 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:59.058+0000 7f2fe1ffb700 1 -- 172.21.15.47:0/3838008264 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2fd80039c0 con 0x7f2fe407de50 2024-04-17T05:52:00.853 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:59.058+0000 7f2fe1ffb700 1 -- 172.21.15.47:0/3838008264 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2fd8011e50 con 0x7f2fe407de50 2024-04-17T05:52:00.853 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:59.058+0000 7f2fe1ffb700 1 -- 172.21.15.47:0/3838008264 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2fd801a5c0 con 0x7f2fe407de50 2024-04-17T05:52:00.853 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:59.058+0000 7f2feaf04700 1 -- 172.21.15.47:0/3838008264 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fe410bf80 con 0x7f2fe407de50 2024-04-17T05:52:00.853 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:59.058+0000 7f2feaf04700 1 -- 172.21.15.47:0/3838008264 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fe4108540 con 0x7f2fe407de50 2024-04-17T05:52:00.853 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:59.059+0000 7f2fe1ffb700 1 -- 172.21.15.47:0/3838008264 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f2fd8024430 con 0x7f2fe407de50 2024-04-17T05:52:00.853 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:59.059+0000 7f2feaf04700 1 -- 172.21.15.47:0/3838008264 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fe410c150 con 0x7f2fe407de50 2024-04-17T05:52:00.853 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:59.059+0000 7f2fe1ffb700 1 --2- 172.21.15.47:0/3838008264 >> [v2:172.21.15.47:6800/3632157175,v1:172.21.15.47:6801/3632157175] conn(0x7f2fd40395b0 0x7f2fd403ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:52:00.853 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:59.059+0000 7f2fe1ffb700 1 -- 172.21.15.47:0/3838008264 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f2fd804db80 con 0x7f2fe407de50 2024-04-17T05:52:00.853 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:59.059+0000 7f2fe3fff700 1 --2- 172.21.15.47:0/3838008264 >> [v2:172.21.15.47:6800/3632157175,v1:172.21.15.47:6801/3632157175] conn(0x7f2fd40395b0 0x7f2fd403ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:52:00.853 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:59.060+0000 7f2fe3fff700 1 --2- 172.21.15.47:0/3838008264 >> [v2:172.21.15.47:6800/3632157175,v1:172.21.15.47:6801/3632157175] conn(0x7f2fd40395b0 0x7f2fd403ba70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f2fd0006fd0 tx=0x7f2fd0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:52:00.853 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:59.065+0000 7f2fe1ffb700 1 -- 172.21.15.47:0/3838008264 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f2fe410c150 con 0x7f2fe407de50 2024-04-17T05:52:00.854 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:59.292+0000 7f2feaf04700 1 -- 172.21.15.47:0/3838008264 --> [v2:172.21.15.47:6800/3632157175,v1:172.21.15.47:6801/3632157175] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f2fe4108ce0 con 0x7f2fd40395b0 2024-04-17T05:52:00.854 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:59.297+0000 7f2fe1ffb700 1 -- 172.21.15.47:0/3838008264 <== mgr.14120 v2:172.21.15.47:6800/3632157175 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f2fe4108ce0 con 0x7f2fd40395b0 2024-04-17T05:52:00.854 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:59.299+0000 7f2feaf04700 1 -- 172.21.15.47:0/3838008264 >> [v2:172.21.15.47:6800/3632157175,v1:172.21.15.47:6801/3632157175] conn(0x7f2fd40395b0 msgr2=0x7f2fd403ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:52:00.854 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:59.299+0000 7f2feaf04700 1 --2- 172.21.15.47:0/3838008264 >> [v2:172.21.15.47:6800/3632157175,v1:172.21.15.47:6801/3632157175] conn(0x7f2fd40395b0 0x7f2fd403ba70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f2fd0006fd0 tx=0x7f2fd0006e40 comp rx=0 tx=0).stop 2024-04-17T05:52:00.854 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:59.299+0000 7f2feaf04700 1 -- 172.21.15.47:0/3838008264 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2fe407de50 msgr2=0x7f2fe410b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:52:00.854 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:59.299+0000 7f2feaf04700 1 --2- 172.21.15.47:0/3838008264 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2fe407de50 0x7f2fe410b7b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2fd8011670 tx=0x7f2fd8011b60 comp rx=0 tx=0).stop 2024-04-17T05:52:00.854 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:59.299+0000 7f2feaf04700 1 -- 172.21.15.47:0/3838008264 shutdown_connections 2024-04-17T05:52:00.854 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:59.299+0000 7f2feaf04700 1 --2- 172.21.15.47:0/3838008264 >> [v2:172.21.15.47:6800/3632157175,v1:172.21.15.47:6801/3632157175] conn(0x7f2fd40395b0 0x7f2fd403ba70 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:52:00.854 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:59.299+0000 7f2feaf04700 1 --2- 172.21.15.47:0/3838008264 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2fe407de50 0x7f2fe410b7b0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:52:00.854 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:59.299+0000 7f2feaf04700 1 -- 172.21.15.47:0/3838008264 >> 172.21.15.47:0/3838008264 conn(0x7f2fe4103b80 msgr2=0x7f2fe4104730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:52:00.854 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:59.299+0000 7f2feaf04700 1 -- 172.21.15.47:0/3838008264 shutdown_connections 2024-04-17T05:52:00.854 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:51:59.299+0000 7f2feaf04700 1 -- 172.21.15.47:0/3838008264 wait complete. 2024-04-17T05:52:00.855 INFO:teuthology.orchestra.run.smithi047.stdout:Deploying unmanaged mgr service... 2024-04-17T05:52:01.840 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:01 smithi047 ceph-mon[32223]: from='mgr.14120 172.21.15.47:0/2362399306' entity='mgr.x' 2024-04-17T05:52:01.840 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:01 smithi047 ceph-mon[32223]: from='mgr.14120 172.21.15.47:0/2362399306' entity='mgr.x' 2024-04-17T05:52:02.148 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-04-17T05:52:02.148 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:01.294+0000 7f199ae36700 1 Processor -- start 2024-04-17T05:52:02.148 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:01.294+0000 7f199ae36700 1 -- start start 2024-04-17T05:52:02.148 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:01.294+0000 7f199ae36700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1994108f70 0x7f199410b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:52:02.148 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:01.294+0000 7f199ae36700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f199407bed0 con 0x7f1994108f70 2024-04-17T05:52:02.148 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:01.295+0000 7f1998bd2700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1994108f70 0x7f199410b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:52:02.148 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:01.295+0000 7f1998bd2700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1994108f70 0x7f199410b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43598/0 (socket says 172.21.15.47:43598) 2024-04-17T05:52:02.148 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:01.295+0000 7f1998bd2700 1 -- 172.21.15.47:0/3749444677 learned_addr learned my addr 172.21.15.47:0/3749444677 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:52:02.148 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:01.295+0000 7f1998bd2700 1 -- 172.21.15.47:0/3749444677 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f199410b8a0 con 0x7f1994108f70 2024-04-17T05:52:02.149 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:01.295+0000 7f1998bd2700 1 --2- 172.21.15.47:0/3749444677 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1994108f70 0x7f199410b360 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f19880098a0 tx=0x7f1988009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25743757cbea6f82 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:52:02.149 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:01.295+0000 7f19937fe700 1 -- 172.21.15.47:0/3749444677 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1988004160 con 0x7f1994108f70 2024-04-17T05:52:02.149 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:01.295+0000 7f19937fe700 1 -- 172.21.15.47:0/3749444677 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1988004300 con 0x7f1994108f70 2024-04-17T05:52:02.149 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:01.295+0000 7f19937fe700 1 -- 172.21.15.47:0/3749444677 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1988011b20 con 0x7f1994108f70 2024-04-17T05:52:02.149 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:01.296+0000 7f199ae36700 1 -- 172.21.15.47:0/3749444677 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1994108f70 msgr2=0x7f199410b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:52:02.149 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:01.296+0000 7f199ae36700 1 --2- 172.21.15.47:0/3749444677 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1994108f70 0x7f199410b360 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f19880098a0 tx=0x7f1988009bb0 comp rx=0 tx=0).stop 2024-04-17T05:52:02.149 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:01.296+0000 7f199ae36700 1 -- 172.21.15.47:0/3749444677 shutdown_connections 2024-04-17T05:52:02.149 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:01.296+0000 7f199ae36700 1 --2- 172.21.15.47:0/3749444677 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1994108f70 0x7f199410b360 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:52:02.149 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:01.296+0000 7f199ae36700 1 -- 172.21.15.47:0/3749444677 >> 172.21.15.47:0/3749444677 conn(0x7f1994102a00 msgr2=0x7f1994104e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:52:02.149 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:01.296+0000 7f199ae36700 1 -- 172.21.15.47:0/3749444677 shutdown_connections 2024-04-17T05:52:02.149 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:01.296+0000 7f199ae36700 1 -- 172.21.15.47:0/3749444677 wait complete. 2024-04-17T05:52:02.149 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:01.297+0000 7f199ae36700 1 Processor -- start 2024-04-17T05:52:02.150 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:01.297+0000 7f199ae36700 1 -- start start 2024-04-17T05:52:02.150 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:01.297+0000 7f199ae36700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1994108f70 0x7f199419e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:52:02.150 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:01.297+0000 7f199ae36700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f199410c910 con 0x7f1994108f70 2024-04-17T05:52:02.150 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:01.297+0000 7f1998bd2700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1994108f70 0x7f199419e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:52:02.150 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:01.297+0000 7f1998bd2700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1994108f70 0x7f199419e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43612/0 (socket says 172.21.15.47:43612) 2024-04-17T05:52:02.150 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:01.297+0000 7f1998bd2700 1 -- 172.21.15.47:0/2517268981 learned_addr learned my addr 172.21.15.47:0/2517268981 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:52:02.150 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:01.297+0000 7f1998bd2700 1 -- 172.21.15.47:0/2517268981 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f199419ebd0 con 0x7f1994108f70 2024-04-17T05:52:02.150 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:01.297+0000 7f1998bd2700 1 --2- 172.21.15.47:0/2517268981 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1994108f70 0x7f199419e690 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f19880117e0 tx=0x7f1988011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:52:02.150 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:01.298+0000 7f1991ffb700 1 -- 172.21.15.47:0/2517268981 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1988003bf0 con 0x7f1994108f70 2024-04-17T05:52:02.150 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:01.298+0000 7f1991ffb700 1 -- 172.21.15.47:0/2517268981 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f198801a910 con 0x7f1994108f70 2024-04-17T05:52:02.150 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:01.298+0000 7f199ae36700 1 -- 172.21.15.47:0/2517268981 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f199419ee60 con 0x7f1994108f70 2024-04-17T05:52:02.150 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:01.298+0000 7f1991ffb700 1 -- 172.21.15.47:0/2517268981 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1988024970 con 0x7f1994108f70 2024-04-17T05:52:02.150 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:01.298+0000 7f199ae36700 1 -- 172.21.15.47:0/2517268981 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f199419f2c0 con 0x7f1994108f70 2024-04-17T05:52:02.150 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:01.299+0000 7f1991ffb700 1 -- 172.21.15.47:0/2517268981 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f1988022020 con 0x7f1994108f70 2024-04-17T05:52:02.151 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:01.299+0000 7f199ae36700 1 -- 172.21.15.47:0/2517268981 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19940640b0 con 0x7f1994108f70 2024-04-17T05:52:02.151 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:01.299+0000 7f1991ffb700 1 --2- 172.21.15.47:0/2517268981 >> [v2:172.21.15.47:6800/3632157175,v1:172.21.15.47:6801/3632157175] conn(0x7f19840395b0 0x7f198403ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:52:02.151 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:01.299+0000 7f1991ffb700 1 -- 172.21.15.47:0/2517268981 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f198804d9d0 con 0x7f1994108f70 2024-04-17T05:52:02.151 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:01.299+0000 7f1993fff700 1 --2- 172.21.15.47:0/2517268981 >> [v2:172.21.15.47:6800/3632157175,v1:172.21.15.47:6801/3632157175] conn(0x7f19840395b0 0x7f198403ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:52:02.151 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:01.300+0000 7f1993fff700 1 --2- 172.21.15.47:0/2517268981 >> [v2:172.21.15.47:6800/3632157175,v1:172.21.15.47:6801/3632157175] conn(0x7f19840395b0 0x7f198403ba70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f1980006fd0 tx=0x7f1980006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:52:02.151 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:01.304+0000 7f1991ffb700 1 -- 172.21.15.47:0/2517268981 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f1988011e20 con 0x7f1994108f70 2024-04-17T05:52:02.151 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:01.530+0000 7f199ae36700 1 -- 172.21.15.47:0/2517268981 --> [v2:172.21.15.47:6800/3632157175,v1:172.21.15.47:6801/3632157175] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f1994103b50 con 0x7f19840395b0 2024-04-17T05:52:02.151 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:01.536+0000 7f1991ffb700 1 -- 172.21.15.47:0/2517268981 <== mgr.14120 v2:172.21.15.47:6800/3632157175 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f1994103b50 con 0x7f19840395b0 2024-04-17T05:52:02.151 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:01.538+0000 7f199ae36700 1 -- 172.21.15.47:0/2517268981 >> [v2:172.21.15.47:6800/3632157175,v1:172.21.15.47:6801/3632157175] conn(0x7f19840395b0 msgr2=0x7f198403ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:52:02.151 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:01.538+0000 7f199ae36700 1 --2- 172.21.15.47:0/2517268981 >> [v2:172.21.15.47:6800/3632157175,v1:172.21.15.47:6801/3632157175] conn(0x7f19840395b0 0x7f198403ba70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f1980006fd0 tx=0x7f1980006e40 comp rx=0 tx=0).stop 2024-04-17T05:52:02.151 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:01.538+0000 7f199ae36700 1 -- 172.21.15.47:0/2517268981 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1994108f70 msgr2=0x7f199419e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:52:02.151 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:01.538+0000 7f199ae36700 1 --2- 172.21.15.47:0/2517268981 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1994108f70 0x7f199419e690 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f19880117e0 tx=0x7f1988011c10 comp rx=0 tx=0).stop 2024-04-17T05:52:02.152 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:01.538+0000 7f199ae36700 1 -- 172.21.15.47:0/2517268981 shutdown_connections 2024-04-17T05:52:02.152 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:01.538+0000 7f199ae36700 1 --2- 172.21.15.47:0/2517268981 >> [v2:172.21.15.47:6800/3632157175,v1:172.21.15.47:6801/3632157175] conn(0x7f19840395b0 0x7f198403ba70 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:52:02.152 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:01.538+0000 7f199ae36700 1 --2- 172.21.15.47:0/2517268981 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1994108f70 0x7f199419e690 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:52:02.152 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:01.538+0000 7f199ae36700 1 -- 172.21.15.47:0/2517268981 >> 172.21.15.47:0/2517268981 conn(0x7f1994102a00 msgr2=0x7f1994103440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:52:02.152 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:01.538+0000 7f199ae36700 1 -- 172.21.15.47:0/2517268981 shutdown_connections 2024-04-17T05:52:02.152 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:01.538+0000 7f199ae36700 1 -- 172.21.15.47:0/2517268981 wait complete. 2024-04-17T05:52:02.784 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:02 smithi047 ceph-mon[32223]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T05:52:02.785 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:02 smithi047 ceph-mon[32223]: Saving service mgr spec with placement count:2 2024-04-17T05:52:02.785 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:02 smithi047 ceph-mon[32223]: from='mgr.14120 172.21.15.47:0/2362399306' entity='mgr.x' 2024-04-17T05:52:03.909 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:02.551+0000 7f4ffc225700 1 Processor -- start 2024-04-17T05:52:03.909 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:02.552+0000 7f4ffc225700 1 -- start start 2024-04-17T05:52:03.909 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:02.552+0000 7f4ffc225700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ff4108ff0 0x7f4ff410b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:52:03.909 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:02.552+0000 7f4ffc225700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f4ff407be90 con 0x7f4ff4108ff0 2024-04-17T05:52:03.909 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:02.552+0000 7f4ff9fc1700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ff4108ff0 0x7f4ff410b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:52:03.910 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:02.552+0000 7f4ff9fc1700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ff4108ff0 0x7f4ff410b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43622/0 (socket says 172.21.15.47:43622) 2024-04-17T05:52:03.910 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:02.552+0000 7f4ff9fc1700 1 -- 172.21.15.47:0/1622479835 learned_addr learned my addr 172.21.15.47:0/1622479835 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:52:03.910 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:02.552+0000 7f4ff9fc1700 1 -- 172.21.15.47:0/1622479835 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ff410b920 con 0x7f4ff4108ff0 2024-04-17T05:52:03.910 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:02.553+0000 7f4ff9fc1700 1 --2- 172.21.15.47:0/1622479835 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ff4108ff0 0x7f4ff410b3e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4ff00098a0 tx=0x7f4ff0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c608e9ac4e29eb57 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:52:03.910 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:02.553+0000 7f4ff8fbf700 1 -- 172.21.15.47:0/1622479835 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4ff0004160 con 0x7f4ff4108ff0 2024-04-17T05:52:03.910 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:02.553+0000 7f4ff8fbf700 1 -- 172.21.15.47:0/1622479835 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ff0004300 con 0x7f4ff4108ff0 2024-04-17T05:52:03.910 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:02.553+0000 7f4ff8fbf700 1 -- 172.21.15.47:0/1622479835 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4ff0011b20 con 0x7f4ff4108ff0 2024-04-17T05:52:03.910 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:02.553+0000 7f4ffc225700 1 -- 172.21.15.47:0/1622479835 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ff4108ff0 msgr2=0x7f4ff410b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:52:03.910 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:02.553+0000 7f4ffc225700 1 --2- 172.21.15.47:0/1622479835 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ff4108ff0 0x7f4ff410b3e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4ff00098a0 tx=0x7f4ff0009bb0 comp rx=0 tx=0).stop 2024-04-17T05:52:03.910 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:02.553+0000 7f4ffc225700 1 -- 172.21.15.47:0/1622479835 shutdown_connections 2024-04-17T05:52:03.910 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:02.553+0000 7f4ffc225700 1 --2- 172.21.15.47:0/1622479835 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ff4108ff0 0x7f4ff410b3e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:52:03.910 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:02.553+0000 7f4ffc225700 1 -- 172.21.15.47:0/1622479835 >> 172.21.15.47:0/1622479835 conn(0x7f4ff4102a20 msgr2=0x7f4ff4104e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:52:03.911 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:02.554+0000 7f4ffc225700 1 -- 172.21.15.47:0/1622479835 shutdown_connections 2024-04-17T05:52:03.911 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:02.554+0000 7f4ffc225700 1 -- 172.21.15.47:0/1622479835 wait complete. 2024-04-17T05:52:03.911 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:02.554+0000 7f4ffc225700 1 Processor -- start 2024-04-17T05:52:03.911 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:02.554+0000 7f4ffc225700 1 -- start start 2024-04-17T05:52:03.911 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:02.554+0000 7f4ffc225700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ff4108ff0 0x7f4ff419e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:52:03.911 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:02.554+0000 7f4ffc225700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f4ff410c980 con 0x7f4ff4108ff0 2024-04-17T05:52:03.911 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:02.554+0000 7f4ff9fc1700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ff4108ff0 0x7f4ff419e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:52:03.911 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:02.554+0000 7f4ff9fc1700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ff4108ff0 0x7f4ff419e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43638/0 (socket says 172.21.15.47:43638) 2024-04-17T05:52:03.911 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:02.554+0000 7f4ff9fc1700 1 -- 172.21.15.47:0/1936267546 learned_addr learned my addr 172.21.15.47:0/1936267546 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:52:03.911 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:02.555+0000 7f4ff9fc1700 1 -- 172.21.15.47:0/1936267546 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ff419ec40 con 0x7f4ff4108ff0 2024-04-17T05:52:03.911 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:02.555+0000 7f4ff9fc1700 1 --2- 172.21.15.47:0/1936267546 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ff4108ff0 0x7f4ff419e700 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4ff00116a0 tx=0x7f4ff0011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:52:03.911 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:02.555+0000 7f4fe6ffd700 1 -- 172.21.15.47:0/1936267546 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4ff0003a40 con 0x7f4ff4108ff0 2024-04-17T05:52:03.911 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:02.555+0000 7f4fe6ffd700 1 -- 172.21.15.47:0/1936267546 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ff0003be0 con 0x7f4ff4108ff0 2024-04-17T05:52:03.912 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:02.555+0000 7f4ffc225700 1 -- 172.21.15.47:0/1936267546 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ff419eed0 con 0x7f4ff4108ff0 2024-04-17T05:52:03.912 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:02.555+0000 7f4fe6ffd700 1 -- 172.21.15.47:0/1936267546 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4ff001a750 con 0x7f4ff4108ff0 2024-04-17T05:52:03.912 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:02.555+0000 7f4ffc225700 1 -- 172.21.15.47:0/1936267546 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ff419f330 con 0x7f4ff4108ff0 2024-04-17T05:52:03.912 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:02.556+0000 7f4fe6ffd700 1 -- 172.21.15.47:0/1936267546 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f4ff001aa00 con 0x7f4ff4108ff0 2024-04-17T05:52:03.912 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:02.556+0000 7f4ffc225700 1 -- 172.21.15.47:0/1936267546 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ff40640b0 con 0x7f4ff4108ff0 2024-04-17T05:52:03.912 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:02.556+0000 7f4fe6ffd700 1 --2- 172.21.15.47:0/1936267546 >> [v2:172.21.15.47:6800/3632157175,v1:172.21.15.47:6801/3632157175] conn(0x7f4fe00395b0 0x7f4fe003ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:52:03.912 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:02.556+0000 7f4fe6ffd700 1 -- 172.21.15.47:0/1936267546 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f4ff004dd90 con 0x7f4ff4108ff0 2024-04-17T05:52:03.912 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:02.556+0000 7f4ff97c0700 1 --2- 172.21.15.47:0/1936267546 >> [v2:172.21.15.47:6800/3632157175,v1:172.21.15.47:6801/3632157175] conn(0x7f4fe00395b0 0x7f4fe003ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:52:03.912 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:02.557+0000 7f4ff97c0700 1 --2- 172.21.15.47:0/1936267546 >> [v2:172.21.15.47:6800/3632157175,v1:172.21.15.47:6801/3632157175] conn(0x7f4fe00395b0 0x7f4fe003ba70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4fe8006fd0 tx=0x7f4fe8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:52:03.912 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:02.561+0000 7f4fe6ffd700 1 -- 172.21.15.47:0/1936267546 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f4ff0011e20 con 0x7f4ff4108ff0 2024-04-17T05:52:03.912 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:02.786+0000 7f4ffc225700 1 -- 172.21.15.47:0/1936267546 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f4ff41a1bc0 con 0x7f4ff4108ff0 2024-04-17T05:52:03.912 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:02.789+0000 7f4fe6ffd700 1 -- 172.21.15.47:0/1936267546 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f4ff001ada0 con 0x7f4ff4108ff0 2024-04-17T05:52:03.913 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:02.791+0000 7f4ffc225700 1 -- 172.21.15.47:0/1936267546 >> [v2:172.21.15.47:6800/3632157175,v1:172.21.15.47:6801/3632157175] conn(0x7f4fe00395b0 msgr2=0x7f4fe003ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:52:03.913 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:02.791+0000 7f4ffc225700 1 --2- 172.21.15.47:0/1936267546 >> [v2:172.21.15.47:6800/3632157175,v1:172.21.15.47:6801/3632157175] conn(0x7f4fe00395b0 0x7f4fe003ba70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4fe8006fd0 tx=0x7f4fe8006e40 comp rx=0 tx=0).stop 2024-04-17T05:52:03.913 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:02.791+0000 7f4ffc225700 1 -- 172.21.15.47:0/1936267546 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ff4108ff0 msgr2=0x7f4ff419e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:52:03.913 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:02.791+0000 7f4ffc225700 1 --2- 172.21.15.47:0/1936267546 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ff4108ff0 0x7f4ff419e700 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4ff00116a0 tx=0x7f4ff0011ba0 comp rx=0 tx=0).stop 2024-04-17T05:52:03.913 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:02.791+0000 7f4ffc225700 1 -- 172.21.15.47:0/1936267546 shutdown_connections 2024-04-17T05:52:03.913 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:02.791+0000 7f4ffc225700 1 --2- 172.21.15.47:0/1936267546 >> [v2:172.21.15.47:6800/3632157175,v1:172.21.15.47:6801/3632157175] conn(0x7f4fe00395b0 0x7f4fe003ba70 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:52:03.913 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:02.791+0000 7f4ffc225700 1 --2- 172.21.15.47:0/1936267546 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ff4108ff0 0x7f4ff419e700 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:52:03.913 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:02.791+0000 7f4ffc225700 1 -- 172.21.15.47:0/1936267546 >> 172.21.15.47:0/1936267546 conn(0x7f4ff4102a20 msgr2=0x7f4ff4103470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:52:03.913 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:02.791+0000 7f4ffc225700 1 -- 172.21.15.47:0/1936267546 shutdown_connections 2024-04-17T05:52:03.913 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:02.791+0000 7f4ffc225700 1 -- 172.21.15.47:0/1936267546 wait complete. 2024-04-17T05:52:04.018 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:03 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/1936267546' entity='client.admin' 2024-04-17T05:52:04.019 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:03 smithi047 ceph-mon[32223]: from='mgr.14120 172.21.15.47:0/2362399306' entity='mgr.x' 2024-04-17T05:52:05.199 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:04.351+0000 7facfe78f700 1 Processor -- start 2024-04-17T05:52:05.199 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:04.351+0000 7facfe78f700 1 -- start start 2024-04-17T05:52:05.200 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:04.351+0000 7facfe78f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7facf8107400 0x7facf8107820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:52:05.200 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:04.351+0000 7facfe78f700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7facf8107df0 con 0x7facf8107400 2024-04-17T05:52:05.200 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:04.352+0000 7facf7fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7facf8107400 0x7facf8107820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:52:05.200 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:04.352+0000 7facf7fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7facf8107400 0x7facf8107820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43640/0 (socket says 172.21.15.47:43640) 2024-04-17T05:52:05.200 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:04.352+0000 7facf7fff700 1 -- 172.21.15.47:0/3497133291 learned_addr learned my addr 172.21.15.47:0/3497133291 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:52:05.200 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:04.352+0000 7facf7fff700 1 -- 172.21.15.47:0/3497133291 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facf8108600 con 0x7facf8107400 2024-04-17T05:52:05.200 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:04.352+0000 7facf7fff700 1 --2- 172.21.15.47:0/3497133291 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7facf8107400 0x7facf8107820 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7face80098a0 tx=0x7face8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b59c31786c79bdcf server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:52:05.200 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:04.352+0000 7facf6ffd700 1 -- 172.21.15.47:0/3497133291 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7face8004160 con 0x7facf8107400 2024-04-17T05:52:05.200 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:04.353+0000 7facf6ffd700 1 -- 172.21.15.47:0/3497133291 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7face8004300 con 0x7facf8107400 2024-04-17T05:52:05.200 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:04.353+0000 7facf6ffd700 1 -- 172.21.15.47:0/3497133291 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7face8011b20 con 0x7facf8107400 2024-04-17T05:52:05.200 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:04.353+0000 7facfe78f700 1 -- 172.21.15.47:0/3497133291 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7facf8107400 msgr2=0x7facf8107820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:52:05.200 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:04.353+0000 7facfe78f700 1 --2- 172.21.15.47:0/3497133291 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7facf8107400 0x7facf8107820 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7face80098a0 tx=0x7face8009bb0 comp rx=0 tx=0).stop 2024-04-17T05:52:05.200 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:04.353+0000 7facfe78f700 1 -- 172.21.15.47:0/3497133291 shutdown_connections 2024-04-17T05:52:05.200 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:04.353+0000 7facfe78f700 1 --2- 172.21.15.47:0/3497133291 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7facf8107400 0x7facf8107820 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:52:05.201 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:04.353+0000 7facfe78f700 1 -- 172.21.15.47:0/3497133291 >> 172.21.15.47:0/3497133291 conn(0x7facf81029c0 msgr2=0x7facf8104de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:52:05.201 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:04.353+0000 7facfe78f700 1 -- 172.21.15.47:0/3497133291 shutdown_connections 2024-04-17T05:52:05.201 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:04.353+0000 7facfe78f700 1 -- 172.21.15.47:0/3497133291 wait complete. 2024-04-17T05:52:05.201 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:04.354+0000 7facfe78f700 1 Processor -- start 2024-04-17T05:52:05.201 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:04.354+0000 7facfe78f700 1 -- start start 2024-04-17T05:52:05.201 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:04.354+0000 7facfe78f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7facf8107400 0x7facf819a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:52:05.201 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:04.354+0000 7facfe78f700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7facf807aff0 con 0x7facf8107400 2024-04-17T05:52:05.201 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:04.355+0000 7facf7fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7facf8107400 0x7facf819a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:52:05.201 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:04.355+0000 7facf7fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7facf8107400 0x7facf819a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43648/0 (socket says 172.21.15.47:43648) 2024-04-17T05:52:05.201 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:04.355+0000 7facf7fff700 1 -- 172.21.15.47:0/1275221436 learned_addr learned my addr 172.21.15.47:0/1275221436 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:52:05.201 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:04.355+0000 7facf7fff700 1 -- 172.21.15.47:0/1275221436 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facf819a780 con 0x7facf8107400 2024-04-17T05:52:05.201 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:04.355+0000 7facf7fff700 1 --2- 172.21.15.47:0/1275221436 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7facf8107400 0x7facf819a240 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7face80117e0 tx=0x7face8011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:52:05.201 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:04.355+0000 7facf57fa700 1 -- 172.21.15.47:0/1275221436 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7face8003bf0 con 0x7facf8107400 2024-04-17T05:52:05.201 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:04.355+0000 7facf57fa700 1 -- 172.21.15.47:0/1275221436 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7face801a910 con 0x7facf8107400 2024-04-17T05:52:05.202 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:04.355+0000 7facfe78f700 1 -- 172.21.15.47:0/1275221436 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facf819aa10 con 0x7facf8107400 2024-04-17T05:52:05.202 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:04.355+0000 7facf57fa700 1 -- 172.21.15.47:0/1275221436 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7face8024970 con 0x7facf8107400 2024-04-17T05:52:05.202 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:04.355+0000 7facfe78f700 1 -- 172.21.15.47:0/1275221436 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facf819ae70 con 0x7facf8107400 2024-04-17T05:52:05.202 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:04.356+0000 7facf57fa700 1 -- 172.21.15.47:0/1275221436 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7face8022020 con 0x7facf8107400 2024-04-17T05:52:05.202 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:04.356+0000 7facf57fa700 1 --2- 172.21.15.47:0/1275221436 >> [v2:172.21.15.47:6800/3632157175,v1:172.21.15.47:6801/3632157175] conn(0x7face003b810 0x7face003dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:52:05.202 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:04.356+0000 7facf57fa700 1 -- 172.21.15.47:0/1275221436 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7face804ea50 con 0x7facf8107400 2024-04-17T05:52:05.202 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:04.357+0000 7facf77fe700 1 --2- 172.21.15.47:0/1275221436 >> [v2:172.21.15.47:6800/3632157175,v1:172.21.15.47:6801/3632157175] conn(0x7face003b810 0x7face003dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:52:05.202 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:04.357+0000 7facfe78f700 1 -- 172.21.15.47:0/1275221436 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facf80640b0 con 0x7facf8107400 2024-04-17T05:52:05.202 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:04.357+0000 7facf77fe700 1 --2- 172.21.15.47:0/1275221436 >> [v2:172.21.15.47:6800/3632157175,v1:172.21.15.47:6801/3632157175] conn(0x7face003b810 0x7face003dcd0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7facec006fd0 tx=0x7facec006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:52:05.202 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:04.363+0000 7facf57fa700 1 -- 172.21.15.47:0/1275221436 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7face8020c10 con 0x7facf8107400 2024-04-17T05:52:05.202 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:04.574+0000 7facfe78f700 1 -- 172.21.15.47:0/1275221436 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7facf81937a0 con 0x7facf8107400 2024-04-17T05:52:05.202 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:04.577+0000 7facf57fa700 1 -- 172.21.15.47:0/1275221436 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7facf81937a0 con 0x7facf8107400 2024-04-17T05:52:05.203 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:04.579+0000 7facfe78f700 1 -- 172.21.15.47:0/1275221436 >> [v2:172.21.15.47:6800/3632157175,v1:172.21.15.47:6801/3632157175] conn(0x7face003b810 msgr2=0x7face003dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:52:05.203 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:04.579+0000 7facfe78f700 1 --2- 172.21.15.47:0/1275221436 >> [v2:172.21.15.47:6800/3632157175,v1:172.21.15.47:6801/3632157175] conn(0x7face003b810 0x7face003dcd0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7facec006fd0 tx=0x7facec006e40 comp rx=0 tx=0).stop 2024-04-17T05:52:05.203 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:04.579+0000 7facfe78f700 1 -- 172.21.15.47:0/1275221436 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7facf8107400 msgr2=0x7facf819a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:52:05.203 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:04.579+0000 7facfe78f700 1 --2- 172.21.15.47:0/1275221436 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7facf8107400 0x7facf819a240 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7face80117e0 tx=0x7face8011ba0 comp rx=0 tx=0).stop 2024-04-17T05:52:05.203 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:04.579+0000 7facfe78f700 1 -- 172.21.15.47:0/1275221436 shutdown_connections 2024-04-17T05:52:05.203 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:04.579+0000 7facfe78f700 1 --2- 172.21.15.47:0/1275221436 >> [v2:172.21.15.47:6800/3632157175,v1:172.21.15.47:6801/3632157175] conn(0x7face003b810 0x7face003dcd0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:52:05.203 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:04.579+0000 7facfe78f700 1 --2- 172.21.15.47:0/1275221436 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7facf8107400 0x7facf819a240 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:52:05.203 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:04.579+0000 7facfe78f700 1 -- 172.21.15.47:0/1275221436 >> 172.21.15.47:0/1275221436 conn(0x7facf81029c0 msgr2=0x7facf8190880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:52:05.203 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:04.580+0000 7facfe78f700 1 -- 172.21.15.47:0/1275221436 shutdown_connections 2024-04-17T05:52:05.203 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:04.580+0000 7facfe78f700 1 -- 172.21.15.47:0/1275221436 wait complete. 2024-04-17T05:52:05.203 INFO:teuthology.orchestra.run.smithi047.stdout:Enabling the dashboard module... 2024-04-17T05:52:05.795 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:05 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/1275221436' entity='client.admin' 2024-04-17T05:52:05.795 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:05 smithi047 ceph-mon[32223]: from='mgr.14120 172.21.15.47:0/2362399306' entity='mgr.x' 2024-04-17T05:52:06.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:06 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/221764256' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-04-17T05:52:06.889 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 05:52:06 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: ignoring --setuser ceph since I am not root 2024-04-17T05:52:06.889 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 05:52:06 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: ignoring --setgroup ceph since I am not root 2024-04-17T05:52:07.195 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:05.667+0000 7f7892484700 1 Processor -- start 2024-04-17T05:52:07.196 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:05.667+0000 7f7892484700 1 -- start start 2024-04-17T05:52:07.196 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:05.667+0000 7f7892484700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f788c108500 0x7f788c108920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:52:07.196 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:05.667+0000 7f7892484700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f788c108ef0 con 0x7f788c108500 2024-04-17T05:52:07.196 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:05.667+0000 7f788bfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f788c108500 0x7f788c108920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:52:07.196 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:05.667+0000 7f788bfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f788c108500 0x7f788c108920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:42146/0 (socket says 172.21.15.47:42146) 2024-04-17T05:52:07.196 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:05.667+0000 7f788bfff700 1 -- 172.21.15.47:0/3629100055 learned_addr learned my addr 172.21.15.47:0/3629100055 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:52:07.196 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:05.668+0000 7f788bfff700 1 -- 172.21.15.47:0/3629100055 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f788c1096b0 con 0x7f788c108500 2024-04-17T05:52:07.196 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:05.668+0000 7f788bfff700 1 --2- 172.21.15.47:0/3629100055 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f788c108500 0x7f788c108920 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f787c0098a0 tx=0x7f787c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aafbf6dd5c2b711b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:52:07.196 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:05.668+0000 7f788affd700 1 -- 172.21.15.47:0/3629100055 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f787c004160 con 0x7f788c108500 2024-04-17T05:52:07.196 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:05.668+0000 7f788affd700 1 -- 172.21.15.47:0/3629100055 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f787c004300 con 0x7f788c108500 2024-04-17T05:52:07.196 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:05.668+0000 7f788affd700 1 -- 172.21.15.47:0/3629100055 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f787c011b20 con 0x7f788c108500 2024-04-17T05:52:07.196 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:05.669+0000 7f7892484700 1 -- 172.21.15.47:0/3629100055 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f788c108500 msgr2=0x7f788c108920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:52:07.196 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:05.669+0000 7f7892484700 1 --2- 172.21.15.47:0/3629100055 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f788c108500 0x7f788c108920 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f787c0098a0 tx=0x7f787c009bb0 comp rx=0 tx=0).stop 2024-04-17T05:52:07.196 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:05.669+0000 7f7892484700 1 -- 172.21.15.47:0/3629100055 shutdown_connections 2024-04-17T05:52:07.197 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:05.669+0000 7f7892484700 1 --2- 172.21.15.47:0/3629100055 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f788c108500 0x7f788c108920 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:52:07.197 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:05.669+0000 7f7892484700 1 -- 172.21.15.47:0/3629100055 >> 172.21.15.47:0/3629100055 conn(0x7f788c103b20 msgr2=0x7f788c105f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:52:07.197 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:05.669+0000 7f7892484700 1 -- 172.21.15.47:0/3629100055 shutdown_connections 2024-04-17T05:52:07.197 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:05.669+0000 7f7892484700 1 -- 172.21.15.47:0/3629100055 wait complete. 2024-04-17T05:52:07.197 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:05.669+0000 7f7892484700 1 Processor -- start 2024-04-17T05:52:07.197 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:05.669+0000 7f7892484700 1 -- start start 2024-04-17T05:52:07.197 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:05.670+0000 7f7892484700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f788c108500 0x7f788c19c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:52:07.197 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:05.670+0000 7f7892484700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f788c10ca60 con 0x7f788c108500 2024-04-17T05:52:07.197 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:05.670+0000 7f788bfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f788c108500 0x7f788c19c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:52:07.197 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:05.670+0000 7f788bfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f788c108500 0x7f788c19c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:42154/0 (socket says 172.21.15.47:42154) 2024-04-17T05:52:07.197 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:05.670+0000 7f788bfff700 1 -- 172.21.15.47:0/221764256 learned_addr learned my addr 172.21.15.47:0/221764256 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:52:07.197 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:05.670+0000 7f788bfff700 1 -- 172.21.15.47:0/221764256 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f788c19c980 con 0x7f788c108500 2024-04-17T05:52:07.197 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:05.670+0000 7f788bfff700 1 --2- 172.21.15.47:0/221764256 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f788c108500 0x7f788c19c440 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f787c0117e0 tx=0x7f787c011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:52:07.198 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:05.671+0000 7f78897fa700 1 -- 172.21.15.47:0/221764256 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f787c003bf0 con 0x7f788c108500 2024-04-17T05:52:07.198 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:05.671+0000 7f78897fa700 1 -- 172.21.15.47:0/221764256 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f787c01a910 con 0x7f788c108500 2024-04-17T05:52:07.198 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:05.671+0000 7f7892484700 1 -- 172.21.15.47:0/221764256 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f788c19cc10 con 0x7f788c108500 2024-04-17T05:52:07.198 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:05.671+0000 7f78897fa700 1 -- 172.21.15.47:0/221764256 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f787c024970 con 0x7f788c108500 2024-04-17T05:52:07.198 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:05.671+0000 7f7892484700 1 -- 172.21.15.47:0/221764256 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f788c19d070 con 0x7f788c108500 2024-04-17T05:52:07.198 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:05.672+0000 7f7892484700 1 -- 172.21.15.47:0/221764256 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f788c0640b0 con 0x7f788c108500 2024-04-17T05:52:07.198 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:05.672+0000 7f78897fa700 1 -- 172.21.15.47:0/221764256 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f787c022020 con 0x7f788c108500 2024-04-17T05:52:07.198 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:05.672+0000 7f78897fa700 1 --2- 172.21.15.47:0/221764256 >> [v2:172.21.15.47:6800/3632157175,v1:172.21.15.47:6801/3632157175] conn(0x7f7874039600 0x7f787403bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:52:07.198 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:05.672+0000 7f78897fa700 1 -- 172.21.15.47:0/221764256 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f787c04da70 con 0x7f788c108500 2024-04-17T05:52:07.198 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:05.672+0000 7f788b7fe700 1 --2- 172.21.15.47:0/221764256 >> [v2:172.21.15.47:6800/3632157175,v1:172.21.15.47:6801/3632157175] conn(0x7f7874039600 0x7f787403bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:52:07.198 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:05.673+0000 7f788b7fe700 1 --2- 172.21.15.47:0/221764256 >> [v2:172.21.15.47:6800/3632157175,v1:172.21.15.47:6801/3632157175] conn(0x7f7874039600 0x7f787403bac0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f7880006fd0 tx=0x7f7880006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:52:07.198 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:05.678+0000 7f78897fa700 1 -- 172.21.15.47:0/221764256 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f787c00b3e0 con 0x7f788c108500 2024-04-17T05:52:07.199 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:05.952+0000 7f7892484700 1 -- 172.21.15.47:0/221764256 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f788c19fb00 con 0x7f788c108500 2024-04-17T05:52:07.199 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:06.578+0000 7f78897fa700 1 -- 172.21.15.47:0/221764256 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f787c020c10 con 0x7f788c108500 2024-04-17T05:52:07.199 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:06.580+0000 7f78897fa700 1 -- 172.21.15.47:0/221764256 <== mon.0 v2:172.21.15.47:3300/0 8 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f787c01aac0 con 0x7f788c108500 2024-04-17T05:52:07.199 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:06.581+0000 7f7892484700 1 -- 172.21.15.47:0/221764256 >> [v2:172.21.15.47:6800/3632157175,v1:172.21.15.47:6801/3632157175] conn(0x7f7874039600 msgr2=0x7f787403bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:52:07.199 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:06.581+0000 7f7892484700 1 --2- 172.21.15.47:0/221764256 >> [v2:172.21.15.47:6800/3632157175,v1:172.21.15.47:6801/3632157175] conn(0x7f7874039600 0x7f787403bac0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f7880006fd0 tx=0x7f7880006e40 comp rx=0 tx=0).stop 2024-04-17T05:52:07.199 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:06.581+0000 7f7892484700 1 -- 172.21.15.47:0/221764256 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f788c108500 msgr2=0x7f788c19c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:52:07.199 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:06.581+0000 7f7892484700 1 --2- 172.21.15.47:0/221764256 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f788c108500 0x7f788c19c440 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f787c0117e0 tx=0x7f787c011ba0 comp rx=0 tx=0).stop 2024-04-17T05:52:07.199 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:06.581+0000 7f7892484700 1 -- 172.21.15.47:0/221764256 shutdown_connections 2024-04-17T05:52:07.199 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:06.581+0000 7f7892484700 1 --2- 172.21.15.47:0/221764256 >> [v2:172.21.15.47:6800/3632157175,v1:172.21.15.47:6801/3632157175] conn(0x7f7874039600 0x7f787403bac0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:52:07.199 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:06.581+0000 7f7892484700 1 --2- 172.21.15.47:0/221764256 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f788c108500 0x7f788c19c440 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:52:07.199 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:06.581+0000 7f7892484700 1 -- 172.21.15.47:0/221764256 >> 172.21.15.47:0/221764256 conn(0x7f788c103b20 msgr2=0x7f788c104680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:52:07.199 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:06.582+0000 7f7892484700 1 -- 172.21.15.47:0/221764256 shutdown_connections 2024-04-17T05:52:07.199 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:06.582+0000 7f7892484700 1 -- 172.21.15.47:0/221764256 wait complete. 2024-04-17T05:52:07.263 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 05:52:07 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T05:52:07.164+0000 7f2a1fc8d200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-17T05:52:07.263 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 05:52:07 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T05:52:07.264+0000 7f2a1fc8d200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-17T05:52:07.869 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:07 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/221764256' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-04-17T05:52:07.869 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:07 smithi047 ceph-mon[32223]: mgrmap e9: x(active, since 20s) 2024-04-17T05:52:08.138 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 05:52:07 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T05:52:07.938+0000 7f2a1fc8d200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-17T05:52:08.139 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 05:52:08 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T05:52:08.033+0000 7f2a1fc8d200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-17T05:52:08.394 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout { 2024-04-17T05:52:08.394 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-04-17T05:52:08.394 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-17T05:52:08.394 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-17T05:52:08.394 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-17T05:52:08.394 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout } 2024-04-17T05:52:08.394 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:07.595+0000 7fb86dd5d700 1 Processor -- start 2024-04-17T05:52:08.394 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:07.595+0000 7fb86dd5d700 1 -- start start 2024-04-17T05:52:08.394 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:07.595+0000 7fb86dd5d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb86807de70 0x7fb86807c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:52:08.394 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:07.595+0000 7fb86dd5d700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb86807c8f0 con 0x7fb86807de70 2024-04-17T05:52:08.394 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:07.595+0000 7fb8677fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb86807de70 0x7fb86807c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:52:08.394 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:07.595+0000 7fb8677fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb86807de70 0x7fb86807c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:42188/0 (socket says 172.21.15.47:42188) 2024-04-17T05:52:08.395 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:07.595+0000 7fb8677fe700 1 -- 172.21.15.47:0/1975675591 learned_addr learned my addr 172.21.15.47:0/1975675591 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:52:08.395 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:07.595+0000 7fb8677fe700 1 -- 172.21.15.47:0/1975675591 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb86807ca70 con 0x7fb86807de70 2024-04-17T05:52:08.395 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:07.596+0000 7fb8677fe700 1 --2- 172.21.15.47:0/1975675591 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb86807de70 0x7fb86807c320 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb8580098a0 tx=0x7fb858009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c81b2042b95cb878 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:52:08.395 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:07.596+0000 7fb8667fc700 1 -- 172.21.15.47:0/1975675591 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb858004160 con 0x7fb86807de70 2024-04-17T05:52:08.395 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:07.596+0000 7fb8667fc700 1 -- 172.21.15.47:0/1975675591 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb858004300 con 0x7fb86807de70 2024-04-17T05:52:08.395 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:07.596+0000 7fb8667fc700 1 -- 172.21.15.47:0/1975675591 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb858011b20 con 0x7fb86807de70 2024-04-17T05:52:08.395 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:07.596+0000 7fb86dd5d700 1 -- 172.21.15.47:0/1975675591 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb86807de70 msgr2=0x7fb86807c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:52:08.395 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:07.596+0000 7fb86dd5d700 1 --2- 172.21.15.47:0/1975675591 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb86807de70 0x7fb86807c320 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb8580098a0 tx=0x7fb858009bb0 comp rx=0 tx=0).stop 2024-04-17T05:52:08.395 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:07.596+0000 7fb86dd5d700 1 -- 172.21.15.47:0/1975675591 shutdown_connections 2024-04-17T05:52:08.395 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:07.596+0000 7fb86dd5d700 1 --2- 172.21.15.47:0/1975675591 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb86807de70 0x7fb86807c320 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:52:08.395 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:07.597+0000 7fb86dd5d700 1 -- 172.21.15.47:0/1975675591 >> 172.21.15.47:0/1975675591 conn(0x7fb868103b80 msgr2=0x7fb868105fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:52:08.395 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:07.597+0000 7fb86dd5d700 1 -- 172.21.15.47:0/1975675591 shutdown_connections 2024-04-17T05:52:08.395 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:07.597+0000 7fb86dd5d700 1 -- 172.21.15.47:0/1975675591 wait complete. 2024-04-17T05:52:08.395 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:07.597+0000 7fb86dd5d700 1 Processor -- start 2024-04-17T05:52:08.396 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:07.597+0000 7fb86dd5d700 1 -- start start 2024-04-17T05:52:08.396 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:07.598+0000 7fb86dd5d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb86807de70 0x7fb86819e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:52:08.396 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:07.598+0000 7fb86dd5d700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb86810eb20 con 0x7fb86807de70 2024-04-17T05:52:08.396 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:07.598+0000 7fb8677fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb86807de70 0x7fb86819e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:52:08.396 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:07.598+0000 7fb8677fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb86807de70 0x7fb86819e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:42194/0 (socket says 172.21.15.47:42194) 2024-04-17T05:52:08.396 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:07.598+0000 7fb8677fe700 1 -- 172.21.15.47:0/383882342 learned_addr learned my addr 172.21.15.47:0/383882342 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:52:08.396 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:07.598+0000 7fb8677fe700 1 -- 172.21.15.47:0/383882342 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb86819eb30 con 0x7fb86807de70 2024-04-17T05:52:08.396 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:07.598+0000 7fb8677fe700 1 --2- 172.21.15.47:0/383882342 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb86807de70 0x7fb86819e5f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb8580117e0 tx=0x7fb858011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:52:08.396 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:07.598+0000 7fb864ff9700 1 -- 172.21.15.47:0/383882342 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb858003ba0 con 0x7fb86807de70 2024-04-17T05:52:08.396 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:07.598+0000 7fb864ff9700 1 -- 172.21.15.47:0/383882342 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb85801a910 con 0x7fb86807de70 2024-04-17T05:52:08.396 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:07.598+0000 7fb86dd5d700 1 -- 172.21.15.47:0/383882342 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb86819edc0 con 0x7fb86807de70 2024-04-17T05:52:08.396 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:07.598+0000 7fb864ff9700 1 -- 172.21.15.47:0/383882342 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb8580248f0 con 0x7fb86807de70 2024-04-17T05:52:08.396 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:07.598+0000 7fb86dd5d700 1 -- 172.21.15.47:0/383882342 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb86819f220 con 0x7fb86807de70 2024-04-17T05:52:08.396 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:07.599+0000 7fb86dd5d700 1 -- 172.21.15.47:0/383882342 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8680640b0 con 0x7fb86807de70 2024-04-17T05:52:08.397 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:07.599+0000 7fb864ff9700 1 -- 172.21.15.47:0/383882342 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7fb858022020 con 0x7fb86807de70 2024-04-17T05:52:08.397 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:07.600+0000 7fb864ff9700 1 --2- 172.21.15.47:0/383882342 >> [v2:172.21.15.47:6800/3632157175,v1:172.21.15.47:6801/3632157175] conn(0x7fb854039660 0x7fb85403bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:52:08.397 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:07.600+0000 7fb864ff9700 1 -- 172.21.15.47:0/383882342 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fb85804dcb0 con 0x7fb86807de70 2024-04-17T05:52:08.397 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:07.600+0000 7fb866ffd700 1 -- 172.21.15.47:0/383882342 >> [v2:172.21.15.47:6800/3632157175,v1:172.21.15.47:6801/3632157175] conn(0x7fb854039660 msgr2=0x7fb85403bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/3632157175 2024-04-17T05:52:08.397 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:07.600+0000 7fb866ffd700 1 --2- 172.21.15.47:0/383882342 >> [v2:172.21.15.47:6800/3632157175,v1:172.21.15.47:6801/3632157175] conn(0x7fb854039660 0x7fb85403bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-17T05:52:08.397 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:07.605+0000 7fb864ff9700 1 -- 172.21.15.47:0/383882342 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fb85800b3e0 con 0x7fb86807de70 2024-04-17T05:52:08.397 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:07.801+0000 7fb866ffd700 1 -- 172.21.15.47:0/383882342 >> [v2:172.21.15.47:6800/3632157175,v1:172.21.15.47:6801/3632157175] conn(0x7fb854039660 msgr2=0x7fb85403bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/3632157175 2024-04-17T05:52:08.397 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:07.801+0000 7fb866ffd700 1 --2- 172.21.15.47:0/383882342 >> [v2:172.21.15.47:6800/3632157175,v1:172.21.15.47:6801/3632157175] conn(0x7fb854039660 0x7fb85403bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-17T05:52:08.397 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:07.870+0000 7fb86dd5d700 1 -- 172.21.15.47:0/383882342 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fb8681a1c90 con 0x7fb86807de70 2024-04-17T05:52:08.397 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:07.871+0000 7fb864ff9700 1 -- 172.21.15.47:0/383882342 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7fb85801ab30 con 0x7fb86807de70 2024-04-17T05:52:08.397 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:07.873+0000 7fb86dd5d700 1 -- 172.21.15.47:0/383882342 >> [v2:172.21.15.47:6800/3632157175,v1:172.21.15.47:6801/3632157175] conn(0x7fb854039660 msgr2=0x7fb85403bb20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-17T05:52:08.397 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:07.873+0000 7fb86dd5d700 1 --2- 172.21.15.47:0/383882342 >> [v2:172.21.15.47:6800/3632157175,v1:172.21.15.47:6801/3632157175] conn(0x7fb854039660 0x7fb85403bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:52:08.397 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:07.873+0000 7fb86dd5d700 1 -- 172.21.15.47:0/383882342 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb86807de70 msgr2=0x7fb86819e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:52:08.397 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:07.873+0000 7fb86dd5d700 1 --2- 172.21.15.47:0/383882342 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb86807de70 0x7fb86819e5f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb8580117e0 tx=0x7fb858011ba0 comp rx=0 tx=0).stop 2024-04-17T05:52:08.398 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:07.873+0000 7fb86dd5d700 1 -- 172.21.15.47:0/383882342 shutdown_connections 2024-04-17T05:52:08.398 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:07.873+0000 7fb86dd5d700 1 --2- 172.21.15.47:0/383882342 >> [v2:172.21.15.47:6800/3632157175,v1:172.21.15.47:6801/3632157175] conn(0x7fb854039660 0x7fb85403bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:52:08.398 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:07.873+0000 7fb86dd5d700 1 --2- 172.21.15.47:0/383882342 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb86807de70 0x7fb86819e5f0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:52:08.398 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:07.873+0000 7fb86dd5d700 1 -- 172.21.15.47:0/383882342 >> 172.21.15.47:0/383882342 conn(0x7fb868103b80 msgr2=0x7fb868104600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:52:08.398 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:07.873+0000 7fb86dd5d700 1 -- 172.21.15.47:0/383882342 shutdown_connections 2024-04-17T05:52:08.398 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:07.873+0000 7fb86dd5d700 1 -- 172.21.15.47:0/383882342 wait complete. 2024-04-17T05:52:08.398 INFO:teuthology.orchestra.run.smithi047.stdout:Waiting for the mgr to restart... 2024-04-17T05:52:08.398 INFO:teuthology.orchestra.run.smithi047.stdout:Waiting for mgr epoch 9... 2024-04-17T05:52:08.511 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 05:52:08 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T05:52:08.181+0000 7f2a1fc8d200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-17T05:52:08.512 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 05:52:08 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T05:52:08.281+0000 7f2a1fc8d200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-17T05:52:08.512 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 05:52:08 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T05:52:08.366+0000 7f2a1fc8d200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-17T05:52:08.772 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:08 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/383882342' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-17T05:52:08.772 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 05:52:08 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T05:52:08.647+0000 7f2a1fc8d200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-17T05:52:08.772 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 05:52:08 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T05:52:08.762+0000 7f2a1fc8d200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-17T05:52:09.130 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 05:52:08 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T05:52:08.853+0000 7f2a1fc8d200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-17T05:52:09.130 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 05:52:08 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T05:52:08.951+0000 7f2a1fc8d200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-17T05:52:09.130 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 05:52:09 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T05:52:09.131+0000 7f2a1fc8d200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-17T05:52:10.389 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 05:52:10 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T05:52:10.122+0000 7f2a1fc8d200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-17T05:52:10.389 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 05:52:10 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T05:52:10.257+0000 7f2a1fc8d200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-17T05:52:10.888 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 05:52:10 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T05:52:10.598+0000 7f2a1fc8d200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-17T05:52:10.889 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 05:52:10 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T05:52:10.691+0000 7f2a1fc8d200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-17T05:52:11.383 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 05:52:10 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T05:52:10.895+0000 7f2a1fc8d200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-17T05:52:11.383 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 05:52:11 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T05:52:11.087+0000 7f2a1fc8d200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-17T05:52:11.639 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 05:52:11 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T05:52:11.383+0000 7f2a1fc8d200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-17T05:52:12.138 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 05:52:11 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T05:52:11.646+0000 7f2a1fc8d200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-17T05:52:12.139 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 05:52:11 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T05:52:11.894+0000 7f2a1fc8d200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-17T05:52:13.139 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 05:52:12 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T05:52:12.728+0000 7f2a1fc8d200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-17T05:52:13.639 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 05:52:13 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T05:52:13.315+0000 7f2a1fc8d200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-17T05:52:13.639 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 05:52:13 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T05:52:13.417+0000 7f2a1fc8d200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-17T05:52:14.139 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 05:52:13 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T05:52:13.643+0000 7f2a1fc8d200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-17T05:52:14.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:13 smithi047 ceph-mon[32223]: Active manager daemon x restarted 2024-04-17T05:52:14.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:13 smithi047 ceph-mon[32223]: Activating manager daemon x 2024-04-17T05:52:14.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:13 smithi047 ceph-mon[32223]: osdmap e3: 0 total, 0 up, 0 in 2024-04-17T05:52:14.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:13 smithi047 ceph-mon[32223]: mgrmap e10: x(active, starting, since 0.00340449s) 2024-04-17T05:52:14.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:13 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-17T05:52:14.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:13 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-17T05:52:14.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:13 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-17T05:52:14.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:13 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-17T05:52:14.140 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:13 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-17T05:52:14.140 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:13 smithi047 ceph-mon[32223]: Manager daemon x is now available 2024-04-17T05:52:14.140 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:13 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T05:52:14.140 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:13 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-17T05:52:14.954 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:14 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-17T05:52:14.954 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:14 smithi047 ceph-mon[32223]: [17/Apr/2024:05:52:14] ENGINE Bus STARTING 2024-04-17T05:52:14.954 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:14 smithi047 ceph-mon[32223]: [17/Apr/2024:05:52:14] ENGINE Serving on http://172.21.15.47:8765 2024-04-17T05:52:14.955 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:14 smithi047 ceph-mon[32223]: [17/Apr/2024:05:52:14] ENGINE Serving on https://172.21.15.47:7150 2024-04-17T05:52:14.955 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:14 smithi047 ceph-mon[32223]: [17/Apr/2024:05:52:14] ENGINE Bus STARTED 2024-04-17T05:52:14.955 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:14 smithi047 ceph-mon[32223]: mgrmap e11: x(active, since 1.0058s) 2024-04-17T05:52:15.379 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout { 2024-04-17T05:52:15.379 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-04-17T05:52:15.379 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-17T05:52:15.379 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout } 2024-04-17T05:52:15.379 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:08.791+0000 7fb9177b2700 1 Processor -- start 2024-04-17T05:52:15.379 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:08.792+0000 7fb9177b2700 1 -- start start 2024-04-17T05:52:15.380 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:08.792+0000 7fb9177b2700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb91007de90 0x7fb91007c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:52:15.380 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:08.792+0000 7fb9177b2700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb91007c910 con 0x7fb91007de90 2024-04-17T05:52:15.380 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:08.792+0000 7fb91554e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb91007de90 0x7fb91007c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:52:15.380 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:08.792+0000 7fb91554e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb91007de90 0x7fb91007c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:42200/0 (socket says 172.21.15.47:42200) 2024-04-17T05:52:15.380 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:08.792+0000 7fb91554e700 1 -- 172.21.15.47:0/3755644683 learned_addr learned my addr 172.21.15.47:0/3755644683 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:52:15.380 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:08.792+0000 7fb91554e700 1 -- 172.21.15.47:0/3755644683 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb91007ca90 con 0x7fb91007de90 2024-04-17T05:52:15.380 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:08.793+0000 7fb91554e700 1 --2- 172.21.15.47:0/3755644683 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb91007de90 0x7fb91007c340 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb90c0098a0 tx=0x7fb90c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5611ab87ebe38c32 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:52:15.380 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:08.793+0000 7fb903fff700 1 -- 172.21.15.47:0/3755644683 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb90c004160 con 0x7fb91007de90 2024-04-17T05:52:15.380 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:08.793+0000 7fb903fff700 1 -- 172.21.15.47:0/3755644683 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb90c004300 con 0x7fb91007de90 2024-04-17T05:52:15.380 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:08.793+0000 7fb903fff700 1 -- 172.21.15.47:0/3755644683 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb90c011b20 con 0x7fb91007de90 2024-04-17T05:52:15.380 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:08.793+0000 7fb9177b2700 1 -- 172.21.15.47:0/3755644683 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb91007de90 msgr2=0x7fb91007c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:52:15.380 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:08.793+0000 7fb9177b2700 1 --2- 172.21.15.47:0/3755644683 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb91007de90 0x7fb91007c340 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb90c0098a0 tx=0x7fb90c009bb0 comp rx=0 tx=0).stop 2024-04-17T05:52:15.380 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:08.794+0000 7fb9177b2700 1 -- 172.21.15.47:0/3755644683 shutdown_connections 2024-04-17T05:52:15.381 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:08.794+0000 7fb9177b2700 1 --2- 172.21.15.47:0/3755644683 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb91007de90 0x7fb91007c340 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:52:15.381 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:08.794+0000 7fb9177b2700 1 -- 172.21.15.47:0/3755644683 >> 172.21.15.47:0/3755644683 conn(0x7fb910103b50 msgr2=0x7fb910105f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:52:15.381 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:08.794+0000 7fb9177b2700 1 -- 172.21.15.47:0/3755644683 shutdown_connections 2024-04-17T05:52:15.381 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:08.794+0000 7fb9177b2700 1 -- 172.21.15.47:0/3755644683 wait complete. 2024-04-17T05:52:15.381 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:08.794+0000 7fb9177b2700 1 Processor -- start 2024-04-17T05:52:15.381 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:08.794+0000 7fb9177b2700 1 -- start start 2024-04-17T05:52:15.381 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:08.794+0000 7fb9177b2700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb91007de90 0x7fb91019e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:52:15.381 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:08.794+0000 7fb9177b2700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb91010eb10 con 0x7fb91007de90 2024-04-17T05:52:15.381 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:08.795+0000 7fb91554e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb91007de90 0x7fb91019e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:52:15.381 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:08.795+0000 7fb91554e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb91007de90 0x7fb91019e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:42208/0 (socket says 172.21.15.47:42208) 2024-04-17T05:52:15.381 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:08.795+0000 7fb91554e700 1 -- 172.21.15.47:0/2608433490 learned_addr learned my addr 172.21.15.47:0/2608433490 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:52:15.381 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:08.795+0000 7fb91554e700 1 -- 172.21.15.47:0/2608433490 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb91019ebc0 con 0x7fb91007de90 2024-04-17T05:52:15.381 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:08.795+0000 7fb91554e700 1 --2- 172.21.15.47:0/2608433490 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb91007de90 0x7fb91019e680 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb90c0116a0 tx=0x7fb90c011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:52:15.381 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:08.795+0000 7fb9027fc700 1 -- 172.21.15.47:0/2608433490 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb90c003a40 con 0x7fb91007de90 2024-04-17T05:52:15.382 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:08.795+0000 7fb9027fc700 1 -- 172.21.15.47:0/2608433490 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb90c003be0 con 0x7fb91007de90 2024-04-17T05:52:15.382 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:08.795+0000 7fb9177b2700 1 -- 172.21.15.47:0/2608433490 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb91019ee50 con 0x7fb91007de90 2024-04-17T05:52:15.382 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:08.795+0000 7fb9027fc700 1 -- 172.21.15.47:0/2608433490 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb90c01a750 con 0x7fb91007de90 2024-04-17T05:52:15.382 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:08.795+0000 7fb9177b2700 1 -- 172.21.15.47:0/2608433490 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb91019f2b0 con 0x7fb91007de90 2024-04-17T05:52:15.382 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:08.796+0000 7fb9027fc700 1 -- 172.21.15.47:0/2608433490 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7fb90c024430 con 0x7fb91007de90 2024-04-17T05:52:15.382 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:08.796+0000 7fb9027fc700 1 --2- 172.21.15.47:0/2608433490 >> [v2:172.21.15.47:6800/3632157175,v1:172.21.15.47:6801/3632157175] conn(0x7fb8fc0395b0 0x7fb8fc03ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:52:15.382 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:08.796+0000 7fb9027fc700 1 -- 172.21.15.47:0/2608433490 --> [v2:172.21.15.47:6800/3632157175,v1:172.21.15.47:6801/3632157175] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fb8fc03c180 con 0x7fb8fc0395b0 2024-04-17T05:52:15.382 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:08.796+0000 7fb914d4d700 1 -- 172.21.15.47:0/2608433490 >> [v2:172.21.15.47:6800/3632157175,v1:172.21.15.47:6801/3632157175] conn(0x7fb8fc0395b0 msgr2=0x7fb8fc03ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/3632157175 2024-04-17T05:52:15.382 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:08.796+0000 7fb9027fc700 1 -- 172.21.15.47:0/2608433490 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fb90c04ddc0 con 0x7fb91007de90 2024-04-17T05:52:15.382 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:08.796+0000 7fb914d4d700 1 --2- 172.21.15.47:0/2608433490 >> [v2:172.21.15.47:6800/3632157175,v1:172.21.15.47:6801/3632157175] conn(0x7fb8fc0395b0 0x7fb8fc03ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-17T05:52:15.382 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:08.997+0000 7fb914d4d700 1 -- 172.21.15.47:0/2608433490 >> [v2:172.21.15.47:6800/3632157175,v1:172.21.15.47:6801/3632157175] conn(0x7fb8fc0395b0 msgr2=0x7fb8fc03ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/3632157175 2024-04-17T05:52:15.382 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:08.997+0000 7fb914d4d700 1 --2- 172.21.15.47:0/2608433490 >> [v2:172.21.15.47:6800/3632157175,v1:172.21.15.47:6801/3632157175] conn(0x7fb8fc0395b0 0x7fb8fc03ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-17T05:52:15.382 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:09.398+0000 7fb914d4d700 1 -- 172.21.15.47:0/2608433490 >> [v2:172.21.15.47:6800/3632157175,v1:172.21.15.47:6801/3632157175] conn(0x7fb8fc0395b0 msgr2=0x7fb8fc03ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/3632157175 2024-04-17T05:52:15.383 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:09.398+0000 7fb914d4d700 1 --2- 172.21.15.47:0/2608433490 >> [v2:172.21.15.47:6800/3632157175,v1:172.21.15.47:6801/3632157175] conn(0x7fb8fc0395b0 0x7fb8fc03ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-17T05:52:15.383 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:10.199+0000 7fb914d4d700 1 -- 172.21.15.47:0/2608433490 >> [v2:172.21.15.47:6800/3632157175,v1:172.21.15.47:6801/3632157175] conn(0x7fb8fc0395b0 msgr2=0x7fb8fc03ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/3632157175 2024-04-17T05:52:15.383 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:10.199+0000 7fb914d4d700 1 --2- 172.21.15.47:0/2608433490 >> [v2:172.21.15.47:6800/3632157175,v1:172.21.15.47:6801/3632157175] conn(0x7fb8fc0395b0 0x7fb8fc03ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-17T05:52:15.383 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:11.800+0000 7fb914d4d700 1 -- 172.21.15.47:0/2608433490 >> [v2:172.21.15.47:6800/3632157175,v1:172.21.15.47:6801/3632157175] conn(0x7fb8fc0395b0 msgr2=0x7fb8fc03ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/3632157175 2024-04-17T05:52:15.383 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:11.800+0000 7fb914d4d700 1 --2- 172.21.15.47:0/2608433490 >> [v2:172.21.15.47:6800/3632157175,v1:172.21.15.47:6801/3632157175] conn(0x7fb8fc0395b0 0x7fb8fc03ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-17T05:52:15.383 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:13.646+0000 7fb9027fc700 1 -- 172.21.15.47:0/2608433490 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mgrmap(e 10) ==== 46338+0+0 (secure 0 0 0) 0x7fb90c02d3f0 con 0x7fb91007de90 2024-04-17T05:52:15.383 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:13.646+0000 7fb9027fc700 1 -- 172.21.15.47:0/2608433490 >> [v2:172.21.15.47:6800/3632157175,v1:172.21.15.47:6801/3632157175] conn(0x7fb8fc0395b0 msgr2=0x7fb8fc03ba70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-17T05:52:15.383 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:13.646+0000 7fb9027fc700 1 --2- 172.21.15.47:0/2608433490 >> [v2:172.21.15.47:6800/3632157175,v1:172.21.15.47:6801/3632157175] conn(0x7fb8fc0395b0 0x7fb8fc03ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:52:15.383 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:14.647+0000 7fb9027fc700 1 -- 172.21.15.47:0/2608433490 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mgrmap(e 11) ==== 46465+0+0 (secure 0 0 0) 0x7fb90c0165f0 con 0x7fb91007de90 2024-04-17T05:52:15.383 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:14.648+0000 7fb9027fc700 1 --2- 172.21.15.47:0/2608433490 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fb8fc03cfe0 0x7fb8fc03f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:52:15.383 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:14.648+0000 7fb9027fc700 1 -- 172.21.15.47:0/2608433490 --> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fb90c01fec0 con 0x7fb8fc03cfe0 2024-04-17T05:52:15.383 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:14.648+0000 7fb914d4d700 1 --2- 172.21.15.47:0/2608433490 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fb8fc03cfe0 0x7fb8fc03f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:52:15.383 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:14.649+0000 7fb914d4d700 1 --2- 172.21.15.47:0/2608433490 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fb8fc03cfe0 0x7fb8fc03f3d0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fb904003a10 tx=0x7fb9040092b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:52:15.384 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:14.649+0000 7fb9027fc700 1 -- 172.21.15.47:0/2608433490 <== mgr.14152 v2:172.21.15.47:6800/3157902806 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7fb90c01fec0 con 0x7fb8fc03cfe0 2024-04-17T05:52:15.384 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:14.656+0000 7fb9177b2700 1 -- 172.21.15.47:0/2608433490 --> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fb910069b80 con 0x7fb8fc03cfe0 2024-04-17T05:52:15.384 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:14.656+0000 7fb9027fc700 1 -- 172.21.15.47:0/2608433490 <== mgr.14152 v2:172.21.15.47:6800/3157902806 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7fb910069b80 con 0x7fb8fc03cfe0 2024-04-17T05:52:15.384 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:14.657+0000 7fb9177b2700 1 -- 172.21.15.47:0/2608433490 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fb8fc03cfe0 msgr2=0x7fb8fc03f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:52:15.384 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:14.657+0000 7fb9177b2700 1 --2- 172.21.15.47:0/2608433490 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fb8fc03cfe0 0x7fb8fc03f3d0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fb904003a10 tx=0x7fb9040092b0 comp rx=0 tx=0).stop 2024-04-17T05:52:15.384 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:14.657+0000 7fb9177b2700 1 -- 172.21.15.47:0/2608433490 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb91007de90 msgr2=0x7fb91019e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:52:15.384 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:14.657+0000 7fb9177b2700 1 --2- 172.21.15.47:0/2608433490 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb91007de90 0x7fb91019e680 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb90c0116a0 tx=0x7fb90c011ba0 comp rx=0 tx=0).stop 2024-04-17T05:52:15.384 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:14.657+0000 7fb9177b2700 1 -- 172.21.15.47:0/2608433490 shutdown_connections 2024-04-17T05:52:15.384 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:14.657+0000 7fb9177b2700 1 --2- 172.21.15.47:0/2608433490 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb91007de90 0x7fb91019e680 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:52:15.384 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:14.657+0000 7fb9177b2700 1 --2- 172.21.15.47:0/2608433490 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fb8fc03cfe0 0x7fb8fc03f3d0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:52:15.384 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:14.657+0000 7fb9177b2700 1 --2- 172.21.15.47:0/2608433490 >> [v2:172.21.15.47:6800/3632157175,v1:172.21.15.47:6801/3632157175] conn(0x7fb8fc0395b0 0x7fb8fc03ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:52:15.384 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:14.657+0000 7fb9177b2700 1 -- 172.21.15.47:0/2608433490 >> 172.21.15.47:0/2608433490 conn(0x7fb910103b50 msgr2=0x7fb9101046b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:52:15.384 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:14.657+0000 7fb9177b2700 1 -- 172.21.15.47:0/2608433490 shutdown_connections 2024-04-17T05:52:15.384 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:14.657+0000 7fb9177b2700 1 -- 172.21.15.47:0/2608433490 wait complete. 2024-04-17T05:52:15.385 INFO:teuthology.orchestra.run.smithi047.stdout:mgr epoch 9 is available 2024-04-17T05:52:15.385 INFO:teuthology.orchestra.run.smithi047.stdout:Generating a dashboard self-signed certificate... 2024-04-17T05:52:15.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:15 smithi047 ceph-mon[32223]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-17T05:52:15.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:15 smithi047 ceph-mon[32223]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-17T05:52:16.752 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:16 smithi047 ceph-mon[32223]: mgrmap e12: x(active, since 2s) 2024-04-17T05:52:16.752 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:16 smithi047 ceph-mon[32223]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T05:52:16.752 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:16 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:52:16.752 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:16 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:52:16.752 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:16 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:52:16.760 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-04-17T05:52:16.760 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:15.824+0000 7f83a5d22700 1 Processor -- start 2024-04-17T05:52:16.760 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:15.824+0000 7f83a5d22700 1 -- start start 2024-04-17T05:52:16.760 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:15.824+0000 7f83a5d22700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83a0109c20 0x7f83a010a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:52:16.760 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:15.824+0000 7f83a5d22700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f83a010a610 con 0x7f83a0109c20 2024-04-17T05:52:16.760 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:15.825+0000 7f839f7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83a0109c20 0x7f83a010a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:52:16.760 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:15.825+0000 7f839f7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83a0109c20 0x7f83a010a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48466/0 (socket says 172.21.15.47:48466) 2024-04-17T05:52:16.760 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:15.825+0000 7f839f7fe700 1 -- 172.21.15.47:0/2433738048 learned_addr learned my addr 172.21.15.47:0/2433738048 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:52:16.760 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:15.825+0000 7f839f7fe700 1 -- 172.21.15.47:0/2433738048 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83a0106470 con 0x7f83a0109c20 2024-04-17T05:52:16.761 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:15.825+0000 7f839f7fe700 1 --2- 172.21.15.47:0/2433738048 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83a0109c20 0x7f83a010a040 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f83900098a0 tx=0x7f8390009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee50c055b5c71fcd server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:52:16.761 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:15.825+0000 7f839e7fc700 1 -- 172.21.15.47:0/2433738048 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8390004160 con 0x7f83a0109c20 2024-04-17T05:52:16.761 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:15.825+0000 7f839e7fc700 1 -- 172.21.15.47:0/2433738048 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8390004300 con 0x7f83a0109c20 2024-04-17T05:52:16.761 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:15.825+0000 7f839e7fc700 1 -- 172.21.15.47:0/2433738048 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8390011b20 con 0x7f83a0109c20 2024-04-17T05:52:16.761 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:15.826+0000 7f83a5d22700 1 -- 172.21.15.47:0/2433738048 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83a0109c20 msgr2=0x7f83a010a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:52:16.761 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:15.826+0000 7f83a5d22700 1 --2- 172.21.15.47:0/2433738048 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83a0109c20 0x7f83a010a040 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f83900098a0 tx=0x7f8390009bb0 comp rx=0 tx=0).stop 2024-04-17T05:52:16.761 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:15.826+0000 7f83a5d22700 1 -- 172.21.15.47:0/2433738048 shutdown_connections 2024-04-17T05:52:16.761 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:15.826+0000 7f83a5d22700 1 --2- 172.21.15.47:0/2433738048 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83a0109c20 0x7f83a010a040 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:52:16.761 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:15.826+0000 7f83a5d22700 1 -- 172.21.15.47:0/2433738048 >> 172.21.15.47:0/2433738048 conn(0x7f83a0102190 msgr2=0x7f83a01045b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:52:16.761 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:15.826+0000 7f83a5d22700 1 -- 172.21.15.47:0/2433738048 shutdown_connections 2024-04-17T05:52:16.761 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:15.826+0000 7f83a5d22700 1 -- 172.21.15.47:0/2433738048 wait complete. 2024-04-17T05:52:16.761 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:15.827+0000 7f83a5d22700 1 Processor -- start 2024-04-17T05:52:16.761 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:15.827+0000 7f83a5d22700 1 -- start start 2024-04-17T05:52:16.761 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:15.827+0000 7f83a5d22700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83a0109c20 0x7f83a019c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:52:16.762 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:15.827+0000 7f83a5d22700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f83a010cb40 con 0x7f83a0109c20 2024-04-17T05:52:16.762 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:15.828+0000 7f839f7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83a0109c20 0x7f83a019c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:52:16.762 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:15.828+0000 7f839f7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83a0109c20 0x7f83a019c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48482/0 (socket says 172.21.15.47:48482) 2024-04-17T05:52:16.762 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:15.828+0000 7f839f7fe700 1 -- 172.21.15.47:0/1830748426 learned_addr learned my addr 172.21.15.47:0/1830748426 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:52:16.762 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:15.828+0000 7f839f7fe700 1 -- 172.21.15.47:0/1830748426 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83a019cb00 con 0x7f83a0109c20 2024-04-17T05:52:16.762 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:15.828+0000 7f839f7fe700 1 --2- 172.21.15.47:0/1830748426 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83a0109c20 0x7f83a019c5c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f83900117e0 tx=0x7f8390011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:52:16.762 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:15.828+0000 7f839cff9700 1 -- 172.21.15.47:0/1830748426 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8390003bf0 con 0x7f83a0109c20 2024-04-17T05:52:16.762 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:15.828+0000 7f839cff9700 1 -- 172.21.15.47:0/1830748426 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f839001a910 con 0x7f83a0109c20 2024-04-17T05:52:16.762 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:15.828+0000 7f83a5d22700 1 -- 172.21.15.47:0/1830748426 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83a019cd90 con 0x7f83a0109c20 2024-04-17T05:52:16.762 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:15.828+0000 7f839cff9700 1 -- 172.21.15.47:0/1830748426 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f83900249a0 con 0x7f83a0109c20 2024-04-17T05:52:16.762 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:15.828+0000 7f83a5d22700 1 -- 172.21.15.47:0/1830748426 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83a019d1f0 con 0x7f83a0109c20 2024-04-17T05:52:16.762 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:15.829+0000 7f839cff9700 1 -- 172.21.15.47:0/1830748426 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7f8390022020 con 0x7f83a0109c20 2024-04-17T05:52:16.762 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:15.829+0000 7f83a5d22700 1 -- 172.21.15.47:0/1830748426 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83a00640b0 con 0x7f83a0109c20 2024-04-17T05:52:16.763 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:15.829+0000 7f839cff9700 1 --2- 172.21.15.47:0/1830748426 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f83880395b0 0x7f838803ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:52:16.763 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:15.829+0000 7f839cff9700 1 -- 172.21.15.47:0/1830748426 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f839004db20 con 0x7f83a0109c20 2024-04-17T05:52:16.763 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:15.830+0000 7f839effd700 1 --2- 172.21.15.47:0/1830748426 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f83880395b0 0x7f838803ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:52:16.763 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:15.830+0000 7f839effd700 1 --2- 172.21.15.47:0/1830748426 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f83880395b0 0x7f838803ba70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8394006fd0 tx=0x7f8394006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:52:16.763 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:15.839+0000 7f839cff9700 1 -- 172.21.15.47:0/1830748426 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f8390011e20 con 0x7f83a0109c20 2024-04-17T05:52:16.763 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:16.068+0000 7f83a5d22700 1 -- 172.21.15.47:0/1830748426 --> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f83a0103360 con 0x7f83880395b0 2024-04-17T05:52:16.763 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:16.147+0000 7f839cff9700 1 -- 172.21.15.47:0/1830748426 <== mgr.14152 v2:172.21.15.47:6800/3157902806 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f83a0103360 con 0x7f83880395b0 2024-04-17T05:52:16.763 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:16.150+0000 7f83a5d22700 1 -- 172.21.15.47:0/1830748426 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f83880395b0 msgr2=0x7f838803ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:52:16.763 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:16.150+0000 7f83a5d22700 1 --2- 172.21.15.47:0/1830748426 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f83880395b0 0x7f838803ba70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8394006fd0 tx=0x7f8394006e40 comp rx=0 tx=0).stop 2024-04-17T05:52:16.763 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:16.150+0000 7f83a5d22700 1 -- 172.21.15.47:0/1830748426 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83a0109c20 msgr2=0x7f83a019c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:52:16.763 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:16.150+0000 7f83a5d22700 1 --2- 172.21.15.47:0/1830748426 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83a0109c20 0x7f83a019c5c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f83900117e0 tx=0x7f8390011c10 comp rx=0 tx=0).stop 2024-04-17T05:52:16.763 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:16.150+0000 7f83a5d22700 1 -- 172.21.15.47:0/1830748426 shutdown_connections 2024-04-17T05:52:16.763 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:16.150+0000 7f83a5d22700 1 --2- 172.21.15.47:0/1830748426 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f83880395b0 0x7f838803ba70 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:52:16.763 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:16.150+0000 7f83a5d22700 1 --2- 172.21.15.47:0/1830748426 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83a0109c20 0x7f83a019c5c0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:52:16.764 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:16.150+0000 7f83a5d22700 1 -- 172.21.15.47:0/1830748426 >> 172.21.15.47:0/1830748426 conn(0x7f83a0102190 msgr2=0x7f83a0102c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:52:16.764 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:16.150+0000 7f83a5d22700 1 -- 172.21.15.47:0/1830748426 shutdown_connections 2024-04-17T05:52:16.764 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:16.150+0000 7f83a5d22700 1 -- 172.21.15.47:0/1830748426 wait complete. 2024-04-17T05:52:16.764 INFO:teuthology.orchestra.run.smithi047.stdout:Creating initial admin user... 2024-04-17T05:52:18.184 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$F5.Xnsy8nq/QrfLK3bht1OU7HH/1XwmWkBUbfRHq/3eg/UdZpv1ai", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1713333137, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-04-17T05:52:18.184 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:17.182+0000 7f02c61da700 1 Processor -- start 2024-04-17T05:52:18.184 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:17.182+0000 7f02c61da700 1 -- start start 2024-04-17T05:52:18.184 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:17.183+0000 7f02c61da700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f02c0108fb0 0x7f02c010b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:52:18.184 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:17.183+0000 7f02c61da700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f02c007be90 con 0x7f02c0108fb0 2024-04-17T05:52:18.184 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:17.183+0000 7f02bf7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f02c0108fb0 0x7f02c010b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:52:18.184 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:17.183+0000 7f02bf7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f02c0108fb0 0x7f02c010b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48492/0 (socket says 172.21.15.47:48492) 2024-04-17T05:52:18.184 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:17.183+0000 7f02bf7fe700 1 -- 172.21.15.47:0/1910526113 learned_addr learned my addr 172.21.15.47:0/1910526113 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:52:18.185 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:17.183+0000 7f02bf7fe700 1 -- 172.21.15.47:0/1910526113 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02c010b8e0 con 0x7f02c0108fb0 2024-04-17T05:52:18.185 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:17.183+0000 7f02bf7fe700 1 --2- 172.21.15.47:0/1910526113 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f02c0108fb0 0x7f02c010b3a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f02b00098a0 tx=0x7f02b0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17bdbbba73e90d6a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:52:18.185 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:17.184+0000 7f02be7fc700 1 -- 172.21.15.47:0/1910526113 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f02b0004160 con 0x7f02c0108fb0 2024-04-17T05:52:18.185 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:17.184+0000 7f02be7fc700 1 -- 172.21.15.47:0/1910526113 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02b0004300 con 0x7f02c0108fb0 2024-04-17T05:52:18.185 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:17.184+0000 7f02be7fc700 1 -- 172.21.15.47:0/1910526113 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f02b0011b20 con 0x7f02c0108fb0 2024-04-17T05:52:18.185 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:17.184+0000 7f02c61da700 1 -- 172.21.15.47:0/1910526113 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f02c0108fb0 msgr2=0x7f02c010b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:52:18.185 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:17.184+0000 7f02c61da700 1 --2- 172.21.15.47:0/1910526113 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f02c0108fb0 0x7f02c010b3a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f02b00098a0 tx=0x7f02b0009bb0 comp rx=0 tx=0).stop 2024-04-17T05:52:18.185 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:17.184+0000 7f02c61da700 1 -- 172.21.15.47:0/1910526113 shutdown_connections 2024-04-17T05:52:18.185 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:17.184+0000 7f02c61da700 1 --2- 172.21.15.47:0/1910526113 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f02c0108fb0 0x7f02c010b3a0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:52:18.185 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:17.184+0000 7f02c61da700 1 -- 172.21.15.47:0/1910526113 >> 172.21.15.47:0/1910526113 conn(0x7f02c01029c0 msgr2=0x7f02c0104de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:52:18.185 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:17.184+0000 7f02c61da700 1 -- 172.21.15.47:0/1910526113 shutdown_connections 2024-04-17T05:52:18.185 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:17.184+0000 7f02c61da700 1 -- 172.21.15.47:0/1910526113 wait complete. 2024-04-17T05:52:18.185 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:17.185+0000 7f02c61da700 1 Processor -- start 2024-04-17T05:52:18.185 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:17.185+0000 7f02c61da700 1 -- start start 2024-04-17T05:52:18.186 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:17.185+0000 7f02c61da700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f02c0108fb0 0x7f02c019c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:52:18.186 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:17.185+0000 7f02c61da700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f02c010c940 con 0x7f02c0108fb0 2024-04-17T05:52:18.186 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:17.185+0000 7f02bf7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f02c0108fb0 0x7f02c019c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:52:18.186 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:17.185+0000 7f02bf7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f02c0108fb0 0x7f02c019c410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48506/0 (socket says 172.21.15.47:48506) 2024-04-17T05:52:18.186 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:17.185+0000 7f02bf7fe700 1 -- 172.21.15.47:0/1806698672 learned_addr learned my addr 172.21.15.47:0/1806698672 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:52:18.186 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:17.186+0000 7f02bf7fe700 1 -- 172.21.15.47:0/1806698672 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02c019c950 con 0x7f02c0108fb0 2024-04-17T05:52:18.186 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:17.186+0000 7f02bf7fe700 1 --2- 172.21.15.47:0/1806698672 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f02c0108fb0 0x7f02c019c410 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f02b00117e0 tx=0x7f02b0011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:52:18.186 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:17.186+0000 7f02bcff9700 1 -- 172.21.15.47:0/1806698672 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f02b0003ba0 con 0x7f02c0108fb0 2024-04-17T05:52:18.186 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:17.186+0000 7f02c61da700 1 -- 172.21.15.47:0/1806698672 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02c019cbe0 con 0x7f02c0108fb0 2024-04-17T05:52:18.186 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:17.186+0000 7f02bcff9700 1 -- 172.21.15.47:0/1806698672 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02b001a910 con 0x7f02c0108fb0 2024-04-17T05:52:18.186 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:17.186+0000 7f02bcff9700 1 -- 172.21.15.47:0/1806698672 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f02b00248f0 con 0x7f02c0108fb0 2024-04-17T05:52:18.187 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:17.186+0000 7f02c61da700 1 -- 172.21.15.47:0/1806698672 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02c019d040 con 0x7f02c0108fb0 2024-04-17T05:52:18.187 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:17.187+0000 7f02c61da700 1 -- 172.21.15.47:0/1806698672 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02c00640b0 con 0x7f02c0108fb0 2024-04-17T05:52:18.187 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:17.187+0000 7f02bcff9700 1 -- 172.21.15.47:0/1806698672 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7f02b0022020 con 0x7f02c0108fb0 2024-04-17T05:52:18.187 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:17.187+0000 7f02bcff9700 1 --2- 172.21.15.47:0/1806698672 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f02a8039600 0x7f02a803bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:52:18.187 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:17.187+0000 7f02bcff9700 1 -- 172.21.15.47:0/1806698672 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f02b004e1d0 con 0x7f02c0108fb0 2024-04-17T05:52:18.187 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:17.187+0000 7f02beffd700 1 --2- 172.21.15.47:0/1806698672 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f02a8039600 0x7f02a803bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:52:18.187 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:17.188+0000 7f02beffd700 1 --2- 172.21.15.47:0/1806698672 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f02a8039600 0x7f02a803bac0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f02b4006fd0 tx=0x7f02b4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:52:18.187 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:17.192+0000 7f02bcff9700 1 -- 172.21.15.47:0/1806698672 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f02b000b3e0 con 0x7f02c0108fb0 2024-04-17T05:52:18.187 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:17.417+0000 7f02c61da700 1 -- 172.21.15.47:0/1806698672 --> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f02c019fa70 con 0x7f02a8039600 2024-04-17T05:52:18.187 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:17.664+0000 7f02bcff9700 1 -- 172.21.15.47:0/1806698672 <== mgr.14152 v2:172.21.15.47:6800/3157902806 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f02c019fa70 con 0x7f02a8039600 2024-04-17T05:52:18.188 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:17.666+0000 7f02c61da700 1 -- 172.21.15.47:0/1806698672 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f02a8039600 msgr2=0x7f02a803bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:52:18.188 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:17.666+0000 7f02c61da700 1 --2- 172.21.15.47:0/1806698672 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f02a8039600 0x7f02a803bac0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f02b4006fd0 tx=0x7f02b4006e40 comp rx=0 tx=0).stop 2024-04-17T05:52:18.188 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:17.666+0000 7f02c61da700 1 -- 172.21.15.47:0/1806698672 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f02c0108fb0 msgr2=0x7f02c019c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:52:18.188 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:17.666+0000 7f02c61da700 1 --2- 172.21.15.47:0/1806698672 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f02c0108fb0 0x7f02c019c410 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f02b00117e0 tx=0x7f02b0011ba0 comp rx=0 tx=0).stop 2024-04-17T05:52:18.188 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:17.666+0000 7f02c61da700 1 -- 172.21.15.47:0/1806698672 shutdown_connections 2024-04-17T05:52:18.188 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:17.666+0000 7f02c61da700 1 --2- 172.21.15.47:0/1806698672 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f02a8039600 0x7f02a803bac0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:52:18.188 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:17.666+0000 7f02c61da700 1 --2- 172.21.15.47:0/1806698672 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f02c0108fb0 0x7f02c019c410 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:52:18.188 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:17.666+0000 7f02c61da700 1 -- 172.21.15.47:0/1806698672 >> 172.21.15.47:0/1806698672 conn(0x7f02c01029c0 msgr2=0x7f02c0103470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:52:18.188 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:17.666+0000 7f02c61da700 1 -- 172.21.15.47:0/1806698672 shutdown_connections 2024-04-17T05:52:18.188 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:17.666+0000 7f02c61da700 1 -- 172.21.15.47:0/1806698672 wait complete. 2024-04-17T05:52:18.188 INFO:teuthology.orchestra.run.smithi047.stdout:Fetching dashboard port number... 2024-04-17T05:52:18.545 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:18 smithi047 ceph-mon[32223]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T05:52:18.545 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:18 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:52:18.545 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:18 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:52:19.883 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:19 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/2404438543' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-04-17T05:52:20.055 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 8443 2024-04-17T05:52:20.055 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:18.578+0000 7fc97dadb700 1 Processor -- start 2024-04-17T05:52:20.055 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:18.578+0000 7fc97dadb700 1 -- start start 2024-04-17T05:52:20.055 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:18.579+0000 7fc97dadb700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc9781068d0 0x7fc978106cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:52:20.055 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:18.579+0000 7fc97dadb700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fc9781072c0 con 0x7fc9781068d0 2024-04-17T05:52:20.055 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:18.579+0000 7fc9777fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc9781068d0 0x7fc978106cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:52:20.055 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:18.579+0000 7fc9777fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc9781068d0 0x7fc978106cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48516/0 (socket says 172.21.15.47:48516) 2024-04-17T05:52:20.055 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:18.579+0000 7fc9777fe700 1 -- 172.21.15.47:0/4174958030 learned_addr learned my addr 172.21.15.47:0/4174958030 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:52:20.056 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:18.579+0000 7fc9777fe700 1 -- 172.21.15.47:0/4174958030 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc978109810 con 0x7fc9781068d0 2024-04-17T05:52:20.056 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:18.579+0000 7fc9777fe700 1 --2- 172.21.15.47:0/4174958030 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc9781068d0 0x7fc978106cf0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc9680098a0 tx=0x7fc968009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b19e79e47031ecef server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:52:20.056 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:18.579+0000 7fc9767fc700 1 -- 172.21.15.47:0/4174958030 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc968004160 con 0x7fc9781068d0 2024-04-17T05:52:20.056 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:18.580+0000 7fc9767fc700 1 -- 172.21.15.47:0/4174958030 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc968004300 con 0x7fc9781068d0 2024-04-17T05:52:20.056 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:18.580+0000 7fc9767fc700 1 -- 172.21.15.47:0/4174958030 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc968011b20 con 0x7fc9781068d0 2024-04-17T05:52:20.056 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:18.580+0000 7fc97dadb700 1 -- 172.21.15.47:0/4174958030 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc9781068d0 msgr2=0x7fc978106cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:52:20.056 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:18.580+0000 7fc97dadb700 1 --2- 172.21.15.47:0/4174958030 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc9781068d0 0x7fc978106cf0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc9680098a0 tx=0x7fc968009bb0 comp rx=0 tx=0).stop 2024-04-17T05:52:20.056 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:18.580+0000 7fc97dadb700 1 -- 172.21.15.47:0/4174958030 shutdown_connections 2024-04-17T05:52:20.056 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:18.580+0000 7fc97dadb700 1 --2- 172.21.15.47:0/4174958030 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc9781068d0 0x7fc978106cf0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:52:20.057 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:18.580+0000 7fc97dadb700 1 -- 172.21.15.47:0/4174958030 >> 172.21.15.47:0/4174958030 conn(0x7fc978102190 msgr2=0x7fc9781045b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:52:20.057 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:18.580+0000 7fc97dadb700 1 -- 172.21.15.47:0/4174958030 shutdown_connections 2024-04-17T05:52:20.057 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:18.581+0000 7fc97dadb700 1 -- 172.21.15.47:0/4174958030 wait complete. 2024-04-17T05:52:20.057 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:18.581+0000 7fc97dadb700 1 Processor -- start 2024-04-17T05:52:20.057 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:18.581+0000 7fc97dadb700 1 -- start start 2024-04-17T05:52:20.057 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:18.581+0000 7fc97dadb700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc9781068d0 0x7fc978109090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:52:20.057 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:18.581+0000 7fc97dadb700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fc97810cbc0 con 0x7fc9781068d0 2024-04-17T05:52:20.057 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:18.582+0000 7fc9777fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc9781068d0 0x7fc978109090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:52:20.057 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:18.582+0000 7fc9777fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc9781068d0 0x7fc978109090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48530/0 (socket says 172.21.15.47:48530) 2024-04-17T05:52:20.057 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:18.582+0000 7fc9777fe700 1 -- 172.21.15.47:0/2404438543 learned_addr learned my addr 172.21.15.47:0/2404438543 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:52:20.057 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:18.582+0000 7fc9777fe700 1 -- 172.21.15.47:0/2404438543 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9781095d0 con 0x7fc9781068d0 2024-04-17T05:52:20.057 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:18.582+0000 7fc9777fe700 1 --2- 172.21.15.47:0/2404438543 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc9781068d0 0x7fc978109090 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc9680117e0 tx=0x7fc968011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:52:20.057 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:18.582+0000 7fc974ff9700 1 -- 172.21.15.47:0/2404438543 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc968003bf0 con 0x7fc9781068d0 2024-04-17T05:52:20.058 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:18.582+0000 7fc974ff9700 1 -- 172.21.15.47:0/2404438543 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc96801a910 con 0x7fc9781068d0 2024-04-17T05:52:20.058 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:18.582+0000 7fc974ff9700 1 -- 172.21.15.47:0/2404438543 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc968024970 con 0x7fc9781068d0 2024-04-17T05:52:20.058 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:18.582+0000 7fc97dadb700 1 -- 172.21.15.47:0/2404438543 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9781077c0 con 0x7fc9781068d0 2024-04-17T05:52:20.058 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:18.582+0000 7fc97dadb700 1 -- 172.21.15.47:0/2404438543 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc978107c80 con 0x7fc9781068d0 2024-04-17T05:52:20.058 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:18.583+0000 7fc974ff9700 1 -- 172.21.15.47:0/2404438543 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7fc968022020 con 0x7fc9781068d0 2024-04-17T05:52:20.058 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:18.583+0000 7fc97dadb700 1 -- 172.21.15.47:0/2404438543 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9780640b0 con 0x7fc9781068d0 2024-04-17T05:52:20.058 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:18.584+0000 7fc974ff9700 1 --2- 172.21.15.47:0/2404438543 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fc960039660 0x7fc96003bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:52:20.058 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:18.584+0000 7fc974ff9700 1 -- 172.21.15.47:0/2404438543 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc96804dbe0 con 0x7fc9781068d0 2024-04-17T05:52:20.058 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:18.584+0000 7fc976ffd700 1 --2- 172.21.15.47:0/2404438543 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fc960039660 0x7fc96003bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:52:20.058 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:18.584+0000 7fc976ffd700 1 --2- 172.21.15.47:0/2404438543 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fc960039660 0x7fc96003bb20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc96c006fd0 tx=0x7fc96c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:52:20.058 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:18.589+0000 7fc974ff9700 1 -- 172.21.15.47:0/2404438543 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fc968020c10 con 0x7fc9781068d0 2024-04-17T05:52:20.058 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:18.815+0000 7fc97dadb700 1 -- 172.21.15.47:0/2404438543 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7fc9781082d0 con 0x7fc9781068d0 2024-04-17T05:52:20.058 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:18.815+0000 7fc974ff9700 1 -- 172.21.15.47:0/2404438543 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7fc96800b5a0 con 0x7fc9781068d0 2024-04-17T05:52:20.058 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:18.817+0000 7fc97dadb700 1 -- 172.21.15.47:0/2404438543 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fc960039660 msgr2=0x7fc96003bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:52:20.059 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:18.817+0000 7fc97dadb700 1 --2- 172.21.15.47:0/2404438543 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fc960039660 0x7fc96003bb20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc96c006fd0 tx=0x7fc96c006e40 comp rx=0 tx=0).stop 2024-04-17T05:52:20.059 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:18.817+0000 7fc97dadb700 1 -- 172.21.15.47:0/2404438543 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc9781068d0 msgr2=0x7fc978109090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:52:20.059 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:18.817+0000 7fc97dadb700 1 --2- 172.21.15.47:0/2404438543 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc9781068d0 0x7fc978109090 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc9680117e0 tx=0x7fc968011ba0 comp rx=0 tx=0).stop 2024-04-17T05:52:20.059 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:18.817+0000 7fc97dadb700 1 -- 172.21.15.47:0/2404438543 shutdown_connections 2024-04-17T05:52:20.059 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:18.817+0000 7fc97dadb700 1 --2- 172.21.15.47:0/2404438543 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fc960039660 0x7fc96003bb20 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:52:20.059 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:18.817+0000 7fc97dadb700 1 --2- 172.21.15.47:0/2404438543 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc9781068d0 0x7fc978109090 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:52:20.059 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:18.817+0000 7fc97dadb700 1 -- 172.21.15.47:0/2404438543 >> 172.21.15.47:0/2404438543 conn(0x7fc978102190 msgr2=0x7fc978102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:52:20.059 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:18.818+0000 7fc97dadb700 1 -- 172.21.15.47:0/2404438543 shutdown_connections 2024-04-17T05:52:20.059 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:18.818+0000 7fc97dadb700 1 -- 172.21.15.47:0/2404438543 wait complete. 2024-04-17T05:52:20.065 INFO:teuthology.orchestra.run.smithi047.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-17T05:52:20.065 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stdout disabled 2024-04-17T05:52:20.073 INFO:teuthology.orchestra.run.smithi047.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-17T05:52:20.073 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stdout inactive 2024-04-17T05:52:20.074 INFO:teuthology.orchestra.run.smithi047.stdout:firewalld.service is not enabled 2024-04-17T05:52:20.074 INFO:teuthology.orchestra.run.smithi047.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-04-17T05:52:20.075 INFO:teuthology.orchestra.run.smithi047.stdout:Ceph Dashboard is now available at: 2024-04-17T05:52:20.075 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-17T05:52:20.075 INFO:teuthology.orchestra.run.smithi047.stdout: URL: https://smithi047.front.sepia.ceph.com:8443/ 2024-04-17T05:52:20.075 INFO:teuthology.orchestra.run.smithi047.stdout: User: admin 2024-04-17T05:52:20.076 INFO:teuthology.orchestra.run.smithi047.stdout: Password: iurl8y0pbd 2024-04-17T05:52:20.076 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-17T05:52:20.076 INFO:teuthology.orchestra.run.smithi047.stdout:Saving cluster configuration to /var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/config directory 2024-04-17T05:52:20.076 INFO:teuthology.orchestra.run.smithi047.stdout:Enabling autotune for osd_memory_target 2024-04-17T05:52:21.353 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:20.542+0000 7f4da38b1700 1 Processor -- start 2024-04-17T05:52:21.353 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:20.542+0000 7f4da38b1700 1 -- start start 2024-04-17T05:52:21.354 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:20.543+0000 7f4da38b1700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d9c10a750 0x7f4d9c10ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:52:21.354 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:20.543+0000 7f4da38b1700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f4d9c10b140 con 0x7f4d9c10a750 2024-04-17T05:52:21.354 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:20.543+0000 7f4da164d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d9c10a750 0x7f4d9c10ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:52:21.354 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:20.543+0000 7f4da164d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d9c10a750 0x7f4d9c10ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48546/0 (socket says 172.21.15.47:48546) 2024-04-17T05:52:21.354 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:20.543+0000 7f4da164d700 1 -- 172.21.15.47:0/792825891 learned_addr learned my addr 172.21.15.47:0/792825891 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:52:21.354 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:20.543+0000 7f4da164d700 1 -- 172.21.15.47:0/792825891 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d9c10b900 con 0x7f4d9c10a750 2024-04-17T05:52:21.354 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:20.544+0000 7f4da164d700 1 --2- 172.21.15.47:0/792825891 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d9c10a750 0x7f4d9c10ab70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4d980098a0 tx=0x7f4d98009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7c4c9c07223308c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:52:21.355 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:20.544+0000 7f4d93fff700 1 -- 172.21.15.47:0/792825891 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4d98004160 con 0x7f4d9c10a750 2024-04-17T05:52:21.355 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:20.544+0000 7f4d93fff700 1 -- 172.21.15.47:0/792825891 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d98004300 con 0x7f4d9c10a750 2024-04-17T05:52:21.355 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:20.544+0000 7f4d93fff700 1 -- 172.21.15.47:0/792825891 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4d98011b20 con 0x7f4d9c10a750 2024-04-17T05:52:21.355 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:20.544+0000 7f4da38b1700 1 -- 172.21.15.47:0/792825891 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d9c10a750 msgr2=0x7f4d9c10ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:52:21.355 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:20.544+0000 7f4da38b1700 1 --2- 172.21.15.47:0/792825891 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d9c10a750 0x7f4d9c10ab70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4d980098a0 tx=0x7f4d98009bb0 comp rx=0 tx=0).stop 2024-04-17T05:52:21.355 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:20.545+0000 7f4da38b1700 1 -- 172.21.15.47:0/792825891 shutdown_connections 2024-04-17T05:52:21.355 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:20.545+0000 7f4da38b1700 1 --2- 172.21.15.47:0/792825891 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d9c10a750 0x7f4d9c10ab70 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:52:21.355 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:20.545+0000 7f4da38b1700 1 -- 172.21.15.47:0/792825891 >> 172.21.15.47:0/792825891 conn(0x7f4d9c07dc10 msgr2=0x7f4d9c1084f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:52:21.355 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:20.545+0000 7f4da38b1700 1 -- 172.21.15.47:0/792825891 shutdown_connections 2024-04-17T05:52:21.355 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:20.545+0000 7f4da38b1700 1 -- 172.21.15.47:0/792825891 wait complete. 2024-04-17T05:52:21.355 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:20.545+0000 7f4da38b1700 1 Processor -- start 2024-04-17T05:52:21.355 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:20.545+0000 7f4da38b1700 1 -- start start 2024-04-17T05:52:21.355 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:20.545+0000 7f4da38b1700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d9c10a750 0x7f4d9c19e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:52:21.356 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:20.545+0000 7f4da38b1700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f4d9c10ecb0 con 0x7f4d9c10a750 2024-04-17T05:52:21.356 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:20.546+0000 7f4da164d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d9c10a750 0x7f4d9c19e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:52:21.356 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:20.546+0000 7f4da164d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d9c10a750 0x7f4d9c19e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48556/0 (socket says 172.21.15.47:48556) 2024-04-17T05:52:21.356 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:20.546+0000 7f4da164d700 1 -- 172.21.15.47:0/28285355 learned_addr learned my addr 172.21.15.47:0/28285355 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:52:21.356 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:20.546+0000 7f4da164d700 1 -- 172.21.15.47:0/28285355 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d9c19eba0 con 0x7f4d9c10a750 2024-04-17T05:52:21.356 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:20.546+0000 7f4da164d700 1 --2- 172.21.15.47:0/28285355 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d9c10a750 0x7f4d9c19e660 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4d980117e0 tx=0x7f4d98011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:52:21.356 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:20.546+0000 7f4d927fc700 1 -- 172.21.15.47:0/28285355 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4d98003bf0 con 0x7f4d9c10a750 2024-04-17T05:52:21.356 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:20.546+0000 7f4da38b1700 1 -- 172.21.15.47:0/28285355 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d9c19ee30 con 0x7f4d9c10a750 2024-04-17T05:52:21.356 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:20.546+0000 7f4d927fc700 1 -- 172.21.15.47:0/28285355 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d9801a910 con 0x7f4d9c10a750 2024-04-17T05:52:21.356 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:20.546+0000 7f4d927fc700 1 -- 172.21.15.47:0/28285355 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4d98024970 con 0x7f4d9c10a750 2024-04-17T05:52:21.356 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:20.546+0000 7f4da38b1700 1 -- 172.21.15.47:0/28285355 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d9c19f290 con 0x7f4d9c10a750 2024-04-17T05:52:21.356 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:20.547+0000 7f4d927fc700 1 -- 172.21.15.47:0/28285355 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f4d98022020 con 0x7f4d9c10a750 2024-04-17T05:52:21.356 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:20.547+0000 7f4d927fc700 1 --2- 172.21.15.47:0/28285355 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f4d88039700 0x7f4d8803bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:52:21.356 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:20.547+0000 7f4d927fc700 1 -- 172.21.15.47:0/28285355 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f4d9804dc00 con 0x7f4d9c10a750 2024-04-17T05:52:21.357 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:20.547+0000 7f4da38b1700 1 -- 172.21.15.47:0/28285355 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d9c0640b0 con 0x7f4d9c10a750 2024-04-17T05:52:21.357 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:20.547+0000 7f4da0e4c700 1 --2- 172.21.15.47:0/28285355 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f4d88039700 0x7f4d8803bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:52:21.357 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:20.548+0000 7f4da0e4c700 1 --2- 172.21.15.47:0/28285355 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f4d88039700 0x7f4d8803bbc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4d8c006fd0 tx=0x7f4d8c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:52:21.357 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:20.553+0000 7f4d927fc700 1 -- 172.21.15.47:0/28285355 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f4d9800b3e0 con 0x7f4d9c10a750 2024-04-17T05:52:21.357 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:20.760+0000 7f4da38b1700 1 -- 172.21.15.47:0/28285355 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7f4d9c1a1d00 con 0x7f4d9c10a750 2024-04-17T05:52:21.357 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:20.761+0000 7f4d927fc700 1 -- 172.21.15.47:0/28285355 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7f4d980172b0 con 0x7f4d9c10a750 2024-04-17T05:52:21.357 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:20.763+0000 7f4da38b1700 1 -- 172.21.15.47:0/28285355 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f4d88039700 msgr2=0x7f4d8803bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:52:21.357 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:20.763+0000 7f4da38b1700 1 --2- 172.21.15.47:0/28285355 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f4d88039700 0x7f4d8803bbc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4d8c006fd0 tx=0x7f4d8c006e40 comp rx=0 tx=0).stop 2024-04-17T05:52:21.357 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:20.763+0000 7f4da38b1700 1 -- 172.21.15.47:0/28285355 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d9c10a750 msgr2=0x7f4d9c19e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:52:21.358 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:20.763+0000 7f4da38b1700 1 --2- 172.21.15.47:0/28285355 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d9c10a750 0x7f4d9c19e660 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4d980117e0 tx=0x7f4d98011ba0 comp rx=0 tx=0).stop 2024-04-17T05:52:21.358 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:20.763+0000 7f4da38b1700 1 -- 172.21.15.47:0/28285355 shutdown_connections 2024-04-17T05:52:21.358 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:20.763+0000 7f4da38b1700 1 --2- 172.21.15.47:0/28285355 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f4d88039700 0x7f4d8803bbc0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:52:21.358 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:20.763+0000 7f4da38b1700 1 --2- 172.21.15.47:0/28285355 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d9c10a750 0x7f4d9c19e660 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:52:21.358 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:20.763+0000 7f4da38b1700 1 -- 172.21.15.47:0/28285355 >> 172.21.15.47:0/28285355 conn(0x7f4d9c07dc10 msgr2=0x7f4d9c107c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:52:21.358 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:20.763+0000 7f4da38b1700 1 -- 172.21.15.47:0/28285355 shutdown_connections 2024-04-17T05:52:21.358 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:20.763+0000 7f4da38b1700 1 -- 172.21.15.47:0/28285355 wait complete. 2024-04-17T05:52:21.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:21 smithi047 ceph-mon[32223]: mgrmap e13: x(active, since 6s) 2024-04-17T05:52:22.634 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:21.775+0000 7fd1ff025700 1 Processor -- start 2024-04-17T05:52:22.634 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:21.775+0000 7fd1ff025700 1 -- start start 2024-04-17T05:52:22.634 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:21.775+0000 7fd1ff025700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd1f8107400 0x7fd1f8107820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:52:22.634 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:21.775+0000 7fd1ff025700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f8107df0 con 0x7fd1f8107400 2024-04-17T05:52:22.634 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:21.775+0000 7fd1fcdc1700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd1f8107400 0x7fd1f8107820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:52:22.635 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:21.776+0000 7fd1fcdc1700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd1f8107400 0x7fd1f8107820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48558/0 (socket says 172.21.15.47:48558) 2024-04-17T05:52:22.635 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:21.776+0000 7fd1fcdc1700 1 -- 172.21.15.47:0/701232836 learned_addr learned my addr 172.21.15.47:0/701232836 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:52:22.635 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:21.776+0000 7fd1fcdc1700 1 -- 172.21.15.47:0/701232836 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1f8108600 con 0x7fd1f8107400 2024-04-17T05:52:22.635 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:21.776+0000 7fd1fcdc1700 1 --2- 172.21.15.47:0/701232836 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd1f8107400 0x7fd1f8107820 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd1e80098a0 tx=0x7fd1e8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=978b13d8f95e2eb7 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:52:22.635 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:21.776+0000 7fd1f77fe700 1 -- 172.21.15.47:0/701232836 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd1e8004160 con 0x7fd1f8107400 2024-04-17T05:52:22.635 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:21.776+0000 7fd1f77fe700 1 -- 172.21.15.47:0/701232836 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1e8004300 con 0x7fd1f8107400 2024-04-17T05:52:22.635 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:21.776+0000 7fd1f77fe700 1 -- 172.21.15.47:0/701232836 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd1e8011b20 con 0x7fd1f8107400 2024-04-17T05:52:22.635 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:21.777+0000 7fd1ff025700 1 -- 172.21.15.47:0/701232836 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd1f8107400 msgr2=0x7fd1f8107820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:52:22.635 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:21.777+0000 7fd1ff025700 1 --2- 172.21.15.47:0/701232836 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd1f8107400 0x7fd1f8107820 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd1e80098a0 tx=0x7fd1e8009bb0 comp rx=0 tx=0).stop 2024-04-17T05:52:22.635 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:21.777+0000 7fd1ff025700 1 -- 172.21.15.47:0/701232836 shutdown_connections 2024-04-17T05:52:22.635 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:21.777+0000 7fd1ff025700 1 --2- 172.21.15.47:0/701232836 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd1f8107400 0x7fd1f8107820 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:52:22.635 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:21.777+0000 7fd1ff025700 1 -- 172.21.15.47:0/701232836 >> 172.21.15.47:0/701232836 conn(0x7fd1f8102a00 msgr2=0x7fd1f8104e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:52:22.635 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:21.777+0000 7fd1ff025700 1 -- 172.21.15.47:0/701232836 shutdown_connections 2024-04-17T05:52:22.636 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:21.777+0000 7fd1ff025700 1 -- 172.21.15.47:0/701232836 wait complete. 2024-04-17T05:52:22.636 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:21.777+0000 7fd1ff025700 1 Processor -- start 2024-04-17T05:52:22.636 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:21.777+0000 7fd1ff025700 1 -- start start 2024-04-17T05:52:22.636 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:21.778+0000 7fd1ff025700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd1f8107400 0x7fd1f807ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:52:22.636 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:21.778+0000 7fd1ff025700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f810b9b0 con 0x7fd1f8107400 2024-04-17T05:52:22.636 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:21.778+0000 7fd1fcdc1700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd1f8107400 0x7fd1f807ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:52:22.636 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:21.778+0000 7fd1fcdc1700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd1f8107400 0x7fd1f807ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48570/0 (socket says 172.21.15.47:48570) 2024-04-17T05:52:22.636 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:21.778+0000 7fd1fcdc1700 1 -- 172.21.15.47:0/1482364240 learned_addr learned my addr 172.21.15.47:0/1482364240 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:52:22.636 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:21.778+0000 7fd1fcdc1700 1 -- 172.21.15.47:0/1482364240 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1f819f240 con 0x7fd1f8107400 2024-04-17T05:52:22.636 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:21.778+0000 7fd1fcdc1700 1 --2- 172.21.15.47:0/1482364240 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd1f8107400 0x7fd1f807ba70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd1e80117e0 tx=0x7fd1e8011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:52:22.636 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:21.778+0000 7fd1f5ffb700 1 -- 172.21.15.47:0/1482364240 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd1e8003bf0 con 0x7fd1f8107400 2024-04-17T05:52:22.636 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:21.778+0000 7fd1f5ffb700 1 -- 172.21.15.47:0/1482364240 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1e801a910 con 0x7fd1f8107400 2024-04-17T05:52:22.636 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:21.778+0000 7fd1ff025700 1 -- 172.21.15.47:0/1482364240 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1f819f410 con 0x7fd1f8107400 2024-04-17T05:52:22.637 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:21.778+0000 7fd1f5ffb700 1 -- 172.21.15.47:0/1482364240 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd1e8024970 con 0x7fd1f8107400 2024-04-17T05:52:22.637 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:21.779+0000 7fd1ff025700 1 -- 172.21.15.47:0/1482364240 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1f819f7b0 con 0x7fd1f8107400 2024-04-17T05:52:22.637 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:21.779+0000 7fd1f5ffb700 1 -- 172.21.15.47:0/1482364240 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fd1e8022020 con 0x7fd1f8107400 2024-04-17T05:52:22.637 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:21.779+0000 7fd1ff025700 1 -- 172.21.15.47:0/1482364240 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1f80640b0 con 0x7fd1f8107400 2024-04-17T05:52:22.637 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:21.780+0000 7fd1f5ffb700 1 --2- 172.21.15.47:0/1482364240 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fd1e0039700 0x7fd1e003bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:52:22.637 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:21.780+0000 7fd1f5ffb700 1 -- 172.21.15.47:0/1482364240 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fd1e804ddb0 con 0x7fd1f8107400 2024-04-17T05:52:22.637 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:21.780+0000 7fd1f7fff700 1 --2- 172.21.15.47:0/1482364240 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fd1e0039700 0x7fd1e003bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:52:22.637 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:21.780+0000 7fd1f7fff700 1 --2- 172.21.15.47:0/1482364240 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fd1e0039700 0x7fd1e003bbc0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec006fd0 tx=0x7fd1ec006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:52:22.637 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:21.785+0000 7fd1f5ffb700 1 -- 172.21.15.47:0/1482364240 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fd1e800b3e0 con 0x7fd1f8107400 2024-04-17T05:52:22.637 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:22.104+0000 7fd1ff025700 1 -- 172.21.15.47:0/1482364240 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7fd1f819fb90 con 0x7fd1f8107400 2024-04-17T05:52:22.637 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:22.107+0000 7fd1f5ffb700 1 -- 172.21.15.47:0/1482364240 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7fd1e802cda0 con 0x7fd1f8107400 2024-04-17T05:52:22.637 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:22.109+0000 7fd1ff025700 1 -- 172.21.15.47:0/1482364240 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fd1e0039700 msgr2=0x7fd1e003bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:52:22.638 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:22.109+0000 7fd1ff025700 1 --2- 172.21.15.47:0/1482364240 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fd1e0039700 0x7fd1e003bbc0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec006fd0 tx=0x7fd1ec006e40 comp rx=0 tx=0).stop 2024-04-17T05:52:22.638 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:22.109+0000 7fd1ff025700 1 -- 172.21.15.47:0/1482364240 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd1f8107400 msgr2=0x7fd1f807ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:52:22.638 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:22.109+0000 7fd1ff025700 1 --2- 172.21.15.47:0/1482364240 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd1f8107400 0x7fd1f807ba70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd1e80117e0 tx=0x7fd1e8011ba0 comp rx=0 tx=0).stop 2024-04-17T05:52:22.638 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:22.109+0000 7fd1ff025700 1 -- 172.21.15.47:0/1482364240 shutdown_connections 2024-04-17T05:52:22.638 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:22.109+0000 7fd1ff025700 1 --2- 172.21.15.47:0/1482364240 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fd1e0039700 0x7fd1e003bbc0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:52:22.638 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:22.109+0000 7fd1ff025700 1 --2- 172.21.15.47:0/1482364240 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd1f8107400 0x7fd1f807ba70 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:52:22.638 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:22.109+0000 7fd1ff025700 1 -- 172.21.15.47:0/1482364240 >> 172.21.15.47:0/1482364240 conn(0x7fd1f8102a00 msgr2=0x7fd1f81035b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:52:22.638 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:22.109+0000 7fd1ff025700 1 -- 172.21.15.47:0/1482364240 shutdown_connections 2024-04-17T05:52:22.638 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-04-17T05:52:22.109+0000 7fd1ff025700 1 -- 172.21.15.47:0/1482364240 wait complete. 2024-04-17T05:52:22.638 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-04-17T05:52:22.638 INFO:teuthology.orchestra.run.smithi047.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-04-17T05:52:22.638 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-17T05:52:22.638 INFO:teuthology.orchestra.run.smithi047.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 565db2d8-fc7e-11ee-bc91-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-04-17T05:52:22.638 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-17T05:52:22.638 INFO:teuthology.orchestra.run.smithi047.stdout:Or, if you are only running a single cluster on this host: 2024-04-17T05:52:22.639 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-17T05:52:22.639 INFO:teuthology.orchestra.run.smithi047.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-04-17T05:52:22.639 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-17T05:52:22.639 INFO:teuthology.orchestra.run.smithi047.stdout:Please consider enabling telemetry to help improve Ceph: 2024-04-17T05:52:22.639 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-17T05:52:22.639 INFO:teuthology.orchestra.run.smithi047.stdout: ceph telemetry on 2024-04-17T05:52:22.639 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-17T05:52:22.639 INFO:teuthology.orchestra.run.smithi047.stdout:For more information see: 2024-04-17T05:52:22.639 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-17T05:52:22.639 INFO:teuthology.orchestra.run.smithi047.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-04-17T05:52:22.639 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-17T05:52:22.639 INFO:teuthology.orchestra.run.smithi047.stdout:Bootstrap complete. 2024-04-17T05:52:22.675 INFO:tasks.cephadm:Fetching config... 2024-04-17T05:52:22.676 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-17T05:52:22.676 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-04-17T05:52:22.692 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-04-17T05:52:22.692 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-17T05:52:22.692 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-04-17T05:52:22.747 INFO:tasks.cephadm:Fetching mon keyring... 2024-04-17T05:52:22.747 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-17T05:52:22.747 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/mon.a/keyring of=/dev/stdout 2024-04-17T05:52:22.815 INFO:tasks.cephadm:Fetching pub ssh key... 2024-04-17T05:52:22.815 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-17T05:52:22.815 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-04-17T05:52:22.872 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-04-17T05:52:22.872 DEBUG:teuthology.orchestra.run.smithi047:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDXDAtDHw0jii0KMV41FXBKNtkjOglzKcQhGLS0Kp5FMXfVIyR8qMDYWBlKAMPr+hpfasq1z8x1/zxJx9tq605jMqHIWcNqTuJFwZt6kJHJNW4tawoeHKyXr3QSYQwH7zcfX498XufYKhrMRt/vmQLLPvx1vcZ8QtKPaexF+/mVQRL/cvYqwwNmZgSpkGhwzG3gkNZML27yuh2bsGhu3VtY3NkzuonSxWXZMDeAGZVqE6cW0x/Mg7apTEPgWTgN09ng3I+bDj0+eN6jZwxNoh7qqLaVtcbmvPrNph4vgBGVzYGJaRsIThmjiN2E2jBv2IO9ku5ng/Hg88b2xDdB3F3VSKk53P/gwoDmqf+6Ah2Zflqeeebj63c4nFjRMXfi/kHRof2lr+OofeFYOyqFfXUKWJytd7CZztVhhMm0c1yVMhCM7Stvk7cfodJgIEC2RWP0+R9KgKqjI095xsmZS3ARk4RunLbf1D7vo1pFQjiIUkzKaNtsYYOFv0UO5gTjoLk= ceph-565db2d8-fc7e-11ee-bc91-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-17T05:52:22.955 INFO:teuthology.orchestra.run.smithi047.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDXDAtDHw0jii0KMV41FXBKNtkjOglzKcQhGLS0Kp5FMXfVIyR8qMDYWBlKAMPr+hpfasq1z8x1/zxJx9tq605jMqHIWcNqTuJFwZt6kJHJNW4tawoeHKyXr3QSYQwH7zcfX498XufYKhrMRt/vmQLLPvx1vcZ8QtKPaexF+/mVQRL/cvYqwwNmZgSpkGhwzG3gkNZML27yuh2bsGhu3VtY3NkzuonSxWXZMDeAGZVqE6cW0x/Mg7apTEPgWTgN09ng3I+bDj0+eN6jZwxNoh7qqLaVtcbmvPrNph4vgBGVzYGJaRsIThmjiN2E2jBv2IO9ku5ng/Hg88b2xDdB3F3VSKk53P/gwoDmqf+6Ah2Zflqeeebj63c4nFjRMXfi/kHRof2lr+OofeFYOyqFfXUKWJytd7CZztVhhMm0c1yVMhCM7Stvk7cfodJgIEC2RWP0+R9KgKqjI095xsmZS3ARk4RunLbf1D7vo1pFQjiIUkzKaNtsYYOFv0UO5gTjoLk= ceph-565db2d8-fc7e-11ee-bc91-c7b262605968 2024-04-17T05:52:22.971 DEBUG:teuthology.orchestra.run.smithi098:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDXDAtDHw0jii0KMV41FXBKNtkjOglzKcQhGLS0Kp5FMXfVIyR8qMDYWBlKAMPr+hpfasq1z8x1/zxJx9tq605jMqHIWcNqTuJFwZt6kJHJNW4tawoeHKyXr3QSYQwH7zcfX498XufYKhrMRt/vmQLLPvx1vcZ8QtKPaexF+/mVQRL/cvYqwwNmZgSpkGhwzG3gkNZML27yuh2bsGhu3VtY3NkzuonSxWXZMDeAGZVqE6cW0x/Mg7apTEPgWTgN09ng3I+bDj0+eN6jZwxNoh7qqLaVtcbmvPrNph4vgBGVzYGJaRsIThmjiN2E2jBv2IO9ku5ng/Hg88b2xDdB3F3VSKk53P/gwoDmqf+6Ah2Zflqeeebj63c4nFjRMXfi/kHRof2lr+OofeFYOyqFfXUKWJytd7CZztVhhMm0c1yVMhCM7Stvk7cfodJgIEC2RWP0+R9KgKqjI095xsmZS3ARk4RunLbf1D7vo1pFQjiIUkzKaNtsYYOFv0UO5gTjoLk= ceph-565db2d8-fc7e-11ee-bc91-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-17T05:52:23.016 INFO:teuthology.orchestra.run.smithi098.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDXDAtDHw0jii0KMV41FXBKNtkjOglzKcQhGLS0Kp5FMXfVIyR8qMDYWBlKAMPr+hpfasq1z8x1/zxJx9tq605jMqHIWcNqTuJFwZt6kJHJNW4tawoeHKyXr3QSYQwH7zcfX498XufYKhrMRt/vmQLLPvx1vcZ8QtKPaexF+/mVQRL/cvYqwwNmZgSpkGhwzG3gkNZML27yuh2bsGhu3VtY3NkzuonSxWXZMDeAGZVqE6cW0x/Mg7apTEPgWTgN09ng3I+bDj0+eN6jZwxNoh7qqLaVtcbmvPrNph4vgBGVzYGJaRsIThmjiN2E2jBv2IO9ku5ng/Hg88b2xDdB3F3VSKk53P/gwoDmqf+6Ah2Zflqeeebj63c4nFjRMXfi/kHRof2lr+OofeFYOyqFfXUKWJytd7CZztVhhMm0c1yVMhCM7Stvk7cfodJgIEC2RWP0+R9KgKqjI095xsmZS3ARk4RunLbf1D7vo1pFQjiIUkzKaNtsYYOFv0UO5gTjoLk= ceph-565db2d8-fc7e-11ee-bc91-c7b262605968 2024-04-17T05:52:23.031 DEBUG:teuthology.orchestra.run.smithi103:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDXDAtDHw0jii0KMV41FXBKNtkjOglzKcQhGLS0Kp5FMXfVIyR8qMDYWBlKAMPr+hpfasq1z8x1/zxJx9tq605jMqHIWcNqTuJFwZt6kJHJNW4tawoeHKyXr3QSYQwH7zcfX498XufYKhrMRt/vmQLLPvx1vcZ8QtKPaexF+/mVQRL/cvYqwwNmZgSpkGhwzG3gkNZML27yuh2bsGhu3VtY3NkzuonSxWXZMDeAGZVqE6cW0x/Mg7apTEPgWTgN09ng3I+bDj0+eN6jZwxNoh7qqLaVtcbmvPrNph4vgBGVzYGJaRsIThmjiN2E2jBv2IO9ku5ng/Hg88b2xDdB3F3VSKk53P/gwoDmqf+6Ah2Zflqeeebj63c4nFjRMXfi/kHRof2lr+OofeFYOyqFfXUKWJytd7CZztVhhMm0c1yVMhCM7Stvk7cfodJgIEC2RWP0+R9KgKqjI095xsmZS3ARk4RunLbf1D7vo1pFQjiIUkzKaNtsYYOFv0UO5gTjoLk= ceph-565db2d8-fc7e-11ee-bc91-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-17T05:52:23.077 INFO:teuthology.orchestra.run.smithi103.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDXDAtDHw0jii0KMV41FXBKNtkjOglzKcQhGLS0Kp5FMXfVIyR8qMDYWBlKAMPr+hpfasq1z8x1/zxJx9tq605jMqHIWcNqTuJFwZt6kJHJNW4tawoeHKyXr3QSYQwH7zcfX498XufYKhrMRt/vmQLLPvx1vcZ8QtKPaexF+/mVQRL/cvYqwwNmZgSpkGhwzG3gkNZML27yuh2bsGhu3VtY3NkzuonSxWXZMDeAGZVqE6cW0x/Mg7apTEPgWTgN09ng3I+bDj0+eN6jZwxNoh7qqLaVtcbmvPrNph4vgBGVzYGJaRsIThmjiN2E2jBv2IO9ku5ng/Hg88b2xDdB3F3VSKk53P/gwoDmqf+6Ah2Zflqeeebj63c4nFjRMXfi/kHRof2lr+OofeFYOyqFfXUKWJytd7CZztVhhMm0c1yVMhCM7Stvk7cfodJgIEC2RWP0+R9KgKqjI095xsmZS3ARk4RunLbf1D7vo1pFQjiIUkzKaNtsYYOFv0UO5gTjoLk= ceph-565db2d8-fc7e-11ee-bc91-c7b262605968 2024-04-17T05:52:23.092 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 565db2d8-fc7e-11ee-bc91-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-04-17T05:52:23.338 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T05:52:23.388 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:23 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/1482364240' entity='client.admin' 2024-04-17T05:52:25.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:25.647+0000 7f94b63b9700 1 -- 172.21.15.47:0/2864991794 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94b0102fb0 msgr2=0x7f94b01053a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:52:25.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:25.647+0000 7f94b63b9700 1 --2- 172.21.15.47:0/2864991794 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94b0102fb0 0x7f94b01053a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f94a0009910 tx=0x7f94a0009c20 comp rx=0 tx=0).stop 2024-04-17T05:52:25.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:25.647+0000 7f94b63b9700 1 -- 172.21.15.47:0/2864991794 shutdown_connections 2024-04-17T05:52:25.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:25.647+0000 7f94b63b9700 1 --2- 172.21.15.47:0/2864991794 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94b0102fb0 0x7f94b01053a0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:52:25.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:25.647+0000 7f94b63b9700 1 -- 172.21.15.47:0/2864991794 >> 172.21.15.47:0/2864991794 conn(0x7f94b00fc9a0 msgr2=0x7f94b00fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:52:25.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:25.647+0000 7f94b63b9700 1 -- 172.21.15.47:0/2864991794 shutdown_connections 2024-04-17T05:52:25.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:25.647+0000 7f94b63b9700 1 -- 172.21.15.47:0/2864991794 wait complete. 2024-04-17T05:52:25.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:25.648+0000 7f94b63b9700 1 Processor -- start 2024-04-17T05:52:25.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:25.648+0000 7f94b63b9700 1 -- start start 2024-04-17T05:52:25.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:25.648+0000 7f94b63b9700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94b0102fb0 0x7f94b0102830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:52:25.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:25.648+0000 7f94b63b9700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f94b0108560 con 0x7f94b0102fb0 2024-04-17T05:52:25.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:25.648+0000 7f94affff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94b0102fb0 0x7f94b0102830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:52:25.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:25.648+0000 7f94affff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94b0102fb0 0x7f94b0102830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41998/0 (socket says 172.21.15.47:41998) 2024-04-17T05:52:25.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:25.648+0000 7f94affff700 1 -- 172.21.15.47:0/649096290 learned_addr learned my addr 172.21.15.47:0/649096290 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:52:25.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:25.648+0000 7f94affff700 1 -- 172.21.15.47:0/649096290 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94b0102d70 con 0x7f94b0102fb0 2024-04-17T05:52:25.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:25.649+0000 7f94affff700 1 --2- 172.21.15.47:0/649096290 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94b0102fb0 0x7f94b0102830 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f94a0005950 tx=0x7f94a0005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:52:25.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:25.649+0000 7f94adffb700 1 -- 172.21.15.47:0/649096290 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f94a0026070 con 0x7f94b0102fb0 2024-04-17T05:52:25.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:25.649+0000 7f94adffb700 1 -- 172.21.15.47:0/649096290 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94a000bc30 con 0x7f94b0102fb0 2024-04-17T05:52:25.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:25.649+0000 7f94b63b9700 1 -- 172.21.15.47:0/649096290 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94b0100f60 con 0x7f94b0102fb0 2024-04-17T05:52:25.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:25.649+0000 7f94b63b9700 1 -- 172.21.15.47:0/649096290 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94b01014a0 con 0x7f94b0102fb0 2024-04-17T05:52:25.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:25.649+0000 7f94adffb700 1 -- 172.21.15.47:0/649096290 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f94a0003b30 con 0x7f94b0102fb0 2024-04-17T05:52:25.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:25.650+0000 7f94adffb700 1 -- 172.21.15.47:0/649096290 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f94a0029070 con 0x7f94b0102fb0 2024-04-17T05:52:25.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:25.650+0000 7f94b63b9700 1 -- 172.21.15.47:0/649096290 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94b0108ce0 con 0x7f94b0102fb0 2024-04-17T05:52:25.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:25.650+0000 7f94adffb700 1 --2- 172.21.15.47:0/649096290 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f949c039650 0x7f949c03bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:52:25.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:25.650+0000 7f94adffb700 1 -- 172.21.15.47:0/649096290 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f94a001e070 con 0x7f94b0102fb0 2024-04-17T05:52:25.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:25.650+0000 7f94a7fff700 1 --2- 172.21.15.47:0/649096290 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f949c039650 0x7f949c03bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:52:25.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:25.651+0000 7f94a7fff700 1 --2- 172.21.15.47:0/649096290 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f949c039650 0x7f949c03bb10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9498006fd0 tx=0x7f9498006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:52:25.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:25.655+0000 7f94adffb700 1 -- 172.21.15.47:0/649096290 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f94a0013070 con 0x7f94b0102fb0 2024-04-17T05:52:25.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:25.869+0000 7f94b63b9700 1 -- 172.21.15.47:0/649096290 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f94b0190fe0 con 0x7f94b0102fb0 2024-04-17T05:52:25.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:25.871+0000 7f94adffb700 1 -- 172.21.15.47:0/649096290 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f94a0030020 con 0x7f94b0102fb0 2024-04-17T05:52:25.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:25.874+0000 7f94b63b9700 1 -- 172.21.15.47:0/649096290 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f949c039650 msgr2=0x7f949c03bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:52:25.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:25.874+0000 7f94b63b9700 1 --2- 172.21.15.47:0/649096290 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f949c039650 0x7f949c03bb10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9498006fd0 tx=0x7f9498006e40 comp rx=0 tx=0).stop 2024-04-17T05:52:25.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:25.874+0000 7f94b63b9700 1 -- 172.21.15.47:0/649096290 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94b0102fb0 msgr2=0x7f94b0102830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:52:25.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:25.874+0000 7f94b63b9700 1 --2- 172.21.15.47:0/649096290 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94b0102fb0 0x7f94b0102830 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f94a0005950 tx=0x7f94a0005030 comp rx=0 tx=0).stop 2024-04-17T05:52:25.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:25.874+0000 7f94b63b9700 1 -- 172.21.15.47:0/649096290 shutdown_connections 2024-04-17T05:52:25.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:25.874+0000 7f94b63b9700 1 --2- 172.21.15.47:0/649096290 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f949c039650 0x7f949c03bb10 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:52:25.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:25.874+0000 7f94b63b9700 1 --2- 172.21.15.47:0/649096290 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94b0102fb0 0x7f94b0102830 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:52:25.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:25.874+0000 7f94b63b9700 1 -- 172.21.15.47:0/649096290 >> 172.21.15.47:0/649096290 conn(0x7f94b00fc9a0 msgr2=0x7f94b00fd380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:52:25.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:25.874+0000 7f94b63b9700 1 -- 172.21.15.47:0/649096290 shutdown_connections 2024-04-17T05:52:25.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:25.874+0000 7f94b63b9700 1 -- 172.21.15.47:0/649096290 wait complete. 2024-04-17T05:52:26.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:26 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:52:26.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:26 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:52:26.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:26 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi047", "name": "osd_memory_target"}]: dispatch 2024-04-17T05:52:26.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:26 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:52:26.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:26 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T05:52:26.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:26 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:52:26.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:26 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/649096290' entity='client.admin' 2024-04-17T05:52:26.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:26 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T05:52:26.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:26 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:52:26.537 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-04-17T05:52:26.537 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 565db2d8-fc7e-11ee-bc91-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-04-17T05:52:27.129 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T05:52:28.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:28.635+0000 7f6870bd5700 1 -- 172.21.15.47:0/2738393766 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f686c102f50 msgr2=0x7f686c105340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:52:28.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:28.635+0000 7f6870bd5700 1 --2- 172.21.15.47:0/2738393766 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f686c102f50 0x7f686c105340 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f685c009910 tx=0x7f685c009c20 comp rx=0 tx=0).stop 2024-04-17T05:52:28.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:28.635+0000 7f6870bd5700 1 -- 172.21.15.47:0/2738393766 shutdown_connections 2024-04-17T05:52:28.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:28.635+0000 7f6870bd5700 1 --2- 172.21.15.47:0/2738393766 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f686c102f50 0x7f686c105340 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:52:28.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:28.635+0000 7f6870bd5700 1 -- 172.21.15.47:0/2738393766 >> 172.21.15.47:0/2738393766 conn(0x7f686c0fc9a0 msgr2=0x7f686c0fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:52:28.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:28.635+0000 7f6870bd5700 1 -- 172.21.15.47:0/2738393766 shutdown_connections 2024-04-17T05:52:28.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:28.635+0000 7f6870bd5700 1 -- 172.21.15.47:0/2738393766 wait complete. 2024-04-17T05:52:28.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:28.635+0000 7f6870bd5700 1 Processor -- start 2024-04-17T05:52:28.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:28.636+0000 7f6870bd5700 1 -- start start 2024-04-17T05:52:28.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:28.636+0000 7f6870bd5700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f686c102f50 0x7f686c197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:52:28.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:28.636+0000 7f6870bd5700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f686c1084f0 con 0x7f686c102f50 2024-04-17T05:52:28.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:28.636+0000 7f686a59c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f686c102f50 0x7f686c197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:52:28.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:28.636+0000 7f686a59c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f686c102f50 0x7f686c197b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:42020/0 (socket says 172.21.15.47:42020) 2024-04-17T05:52:28.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:28.636+0000 7f686a59c700 1 -- 172.21.15.47:0/3411244415 learned_addr learned my addr 172.21.15.47:0/3411244415 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:52:28.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:28.636+0000 7f686a59c700 1 -- 172.21.15.47:0/3411244415 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f686c198080 con 0x7f686c102f50 2024-04-17T05:52:28.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:28.636+0000 7f686a59c700 1 --2- 172.21.15.47:0/3411244415 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f686c102f50 0x7f686c197b40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f685c004fa0 tx=0x7f685c005080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:52:28.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:28.636+0000 7f6863fff700 1 -- 172.21.15.47:0/3411244415 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f685c027070 con 0x7f686c102f50 2024-04-17T05:52:28.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:28.637+0000 7f6870bd5700 1 -- 172.21.15.47:0/3411244415 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f686c198370 con 0x7f686c102f50 2024-04-17T05:52:28.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:28.637+0000 7f6863fff700 1 -- 172.21.15.47:0/3411244415 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f685c00be40 con 0x7f686c102f50 2024-04-17T05:52:28.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:28.637+0000 7f6863fff700 1 -- 172.21.15.47:0/3411244415 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f685c00f460 con 0x7f686c102f50 2024-04-17T05:52:28.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:28.637+0000 7f6870bd5700 1 -- 172.21.15.47:0/3411244415 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f686c19b000 con 0x7f686c102f50 2024-04-17T05:52:28.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:28.638+0000 7f6863fff700 1 -- 172.21.15.47:0/3411244415 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f685c02a070 con 0x7f686c102f50 2024-04-17T05:52:28.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:28.638+0000 7f6870bd5700 1 -- 172.21.15.47:0/3411244415 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f686c108c70 con 0x7f686c102f50 2024-04-17T05:52:28.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:28.638+0000 7f6863fff700 1 --2- 172.21.15.47:0/3411244415 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f6858039650 0x7f685803bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:52:28.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:28.638+0000 7f6863fff700 1 -- 172.21.15.47:0/3411244415 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f685c01e030 con 0x7f686c102f50 2024-04-17T05:52:28.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:28.638+0000 7f6861d9b700 1 --2- 172.21.15.47:0/3411244415 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f6858039650 0x7f685803bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:52:28.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:28.638+0000 7f6861d9b700 1 --2- 172.21.15.47:0/3411244415 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f6858039650 0x7f685803bb10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f6854006fd0 tx=0x7f6854006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:52:28.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:28.643+0000 7f6863fff700 1 -- 172.21.15.47:0/3411244415 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f685c013070 con 0x7f686c102f50 2024-04-17T05:52:28.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:28.855+0000 7f6870bd5700 1 -- 172.21.15.47:0/3411244415 --> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f686c02e180 con 0x7f6858039650 2024-04-17T05:52:28.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:28.858+0000 7f6863fff700 1 -- 172.21.15.47:0/3411244415 <== mgr.14152 v2:172.21.15.47:6800/3157902806 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f686c02e180 con 0x7f6858039650 2024-04-17T05:52:28.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:28.860+0000 7f6870bd5700 1 -- 172.21.15.47:0/3411244415 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f6858039650 msgr2=0x7f685803bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:52:28.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:28.860+0000 7f6870bd5700 1 --2- 172.21.15.47:0/3411244415 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f6858039650 0x7f685803bb10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f6854006fd0 tx=0x7f6854006e40 comp rx=0 tx=0).stop 2024-04-17T05:52:28.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:28.860+0000 7f6870bd5700 1 -- 172.21.15.47:0/3411244415 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f686c102f50 msgr2=0x7f686c197b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:52:28.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:28.860+0000 7f6870bd5700 1 --2- 172.21.15.47:0/3411244415 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f686c102f50 0x7f686c197b40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f685c004fa0 tx=0x7f685c005080 comp rx=0 tx=0).stop 2024-04-17T05:52:28.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:28.861+0000 7f6870bd5700 1 -- 172.21.15.47:0/3411244415 shutdown_connections 2024-04-17T05:52:28.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:28.861+0000 7f6870bd5700 1 --2- 172.21.15.47:0/3411244415 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f6858039650 0x7f685803bb10 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:52:28.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:28.861+0000 7f6870bd5700 1 --2- 172.21.15.47:0/3411244415 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f686c102f50 0x7f686c197b40 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:52:28.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:28.861+0000 7f6870bd5700 1 -- 172.21.15.47:0/3411244415 >> 172.21.15.47:0/3411244415 conn(0x7f686c0fc9a0 msgr2=0x7f686c0fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:52:28.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:28.861+0000 7f6870bd5700 1 -- 172.21.15.47:0/3411244415 shutdown_connections 2024-04-17T05:52:28.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:28.861+0000 7f6870bd5700 1 -- 172.21.15.47:0/3411244415 wait complete. 2024-04-17T05:52:29.614 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi098 2024-04-17T05:52:29.614 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-04-17T05:52:29.614 DEBUG:teuthology.orchestra.run.smithi098:> dd of=/etc/ceph/ceph.conf 2024-04-17T05:52:29.634 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-04-17T05:52:29.634 DEBUG:teuthology.orchestra.run.smithi098:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-17T05:52:29.691 INFO:tasks.cephadm:Adding host smithi098 to orchestrator... 2024-04-17T05:52:29.691 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 565db2d8-fc7e-11ee-bc91-c7b262605968 -- ceph orch host add smithi098 2024-04-17T05:52:29.935 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T05:52:30.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:29 smithi047 ceph-mon[32223]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T05:52:30.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:29 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:52:30.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:29 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T05:52:30.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:29 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:52:30.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:29 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T05:52:30.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:29 smithi047 ceph-mon[32223]: Updating smithi047:/etc/ceph/ceph.conf 2024-04-17T05:52:30.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:29 smithi047 ceph-mon[32223]: Updating smithi047:/var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/config/ceph.conf 2024-04-17T05:52:30.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:29 smithi047 ceph-mon[32223]: Updating smithi047:/etc/ceph/ceph.client.admin.keyring 2024-04-17T05:52:30.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:29 smithi047 ceph-mon[32223]: Updating smithi047:/var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/config/ceph.client.admin.keyring 2024-04-17T05:52:30.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:29 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:52:30.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:29 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:52:30.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:29 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:52:31.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:31.405+0000 7fdefee32700 1 -- 172.21.15.47:0/2167626091 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdef81015a0 msgr2=0x7fdef8101980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:52:31.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:31.405+0000 7fdefee32700 1 --2- 172.21.15.47:0/2167626091 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdef81015a0 0x7fdef8101980 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fdeec009910 tx=0x7fdeec009c20 comp rx=0 tx=0).stop 2024-04-17T05:52:31.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:31.405+0000 7fdefee32700 1 -- 172.21.15.47:0/2167626091 shutdown_connections 2024-04-17T05:52:31.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:31.405+0000 7fdefee32700 1 --2- 172.21.15.47:0/2167626091 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdef81015a0 0x7fdef8101980 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:52:31.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:31.405+0000 7fdefee32700 1 -- 172.21.15.47:0/2167626091 >> 172.21.15.47:0/2167626091 conn(0x7fdef80fce50 msgr2=0x7fdef80ff270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:52:31.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:31.405+0000 7fdefee32700 1 -- 172.21.15.47:0/2167626091 shutdown_connections 2024-04-17T05:52:31.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:31.405+0000 7fdefee32700 1 -- 172.21.15.47:0/2167626091 wait complete. 2024-04-17T05:52:31.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:31.406+0000 7fdefee32700 1 Processor -- start 2024-04-17T05:52:31.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:31.406+0000 7fdefee32700 1 -- start start 2024-04-17T05:52:31.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:31.406+0000 7fdefee32700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdef81015a0 0x7fdef8197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:52:31.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:31.406+0000 7fdefee32700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fdef810b970 con 0x7fdef81015a0 2024-04-17T05:52:31.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:31.406+0000 7fdefcbce700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdef81015a0 0x7fdef8197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:52:31.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:31.406+0000 7fdefcbce700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdef81015a0 0x7fdef8197fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:42044/0 (socket says 172.21.15.47:42044) 2024-04-17T05:52:31.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:31.406+0000 7fdefcbce700 1 -- 172.21.15.47:0/3386059608 learned_addr learned my addr 172.21.15.47:0/3386059608 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:52:31.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:31.406+0000 7fdefcbce700 1 -- 172.21.15.47:0/3386059608 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdef8198510 con 0x7fdef81015a0 2024-04-17T05:52:31.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:31.407+0000 7fdefcbce700 1 --2- 172.21.15.47:0/3386059608 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdef81015a0 0x7fdef8197fd0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fdeec005850 tx=0x7fdeec005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:52:31.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:31.407+0000 7fdef5ffb700 1 -- 172.21.15.47:0/3386059608 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdeec026070 con 0x7fdef81015a0 2024-04-17T05:52:31.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:31.407+0000 7fdef5ffb700 1 -- 172.21.15.47:0/3386059608 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdeec00bbd0 con 0x7fdef81015a0 2024-04-17T05:52:31.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:31.407+0000 7fdefee32700 1 -- 172.21.15.47:0/3386059608 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdef8198800 con 0x7fdef81015a0 2024-04-17T05:52:31.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:31.407+0000 7fdef5ffb700 1 -- 172.21.15.47:0/3386059608 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdeec003970 con 0x7fdef81015a0 2024-04-17T05:52:31.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:31.407+0000 7fdefee32700 1 -- 172.21.15.47:0/3386059608 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdef819c8a0 con 0x7fdef81015a0 2024-04-17T05:52:31.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:31.408+0000 7fdef5ffb700 1 -- 172.21.15.47:0/3386059608 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fdeec029070 con 0x7fdef81015a0 2024-04-17T05:52:31.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:31.408+0000 7fdefee32700 1 -- 172.21.15.47:0/3386059608 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdef8102a80 con 0x7fdef81015a0 2024-04-17T05:52:31.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:31.408+0000 7fdef5ffb700 1 --2- 172.21.15.47:0/3386059608 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fdee80396b0 0x7fdee803bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:52:31.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:31.408+0000 7fdef5ffb700 1 -- 172.21.15.47:0/3386059608 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fdeec01e070 con 0x7fdef81015a0 2024-04-17T05:52:31.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:31.408+0000 7fdef7fff700 1 --2- 172.21.15.47:0/3386059608 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fdee80396b0 0x7fdee803bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:52:31.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:31.409+0000 7fdef7fff700 1 --2- 172.21.15.47:0/3386059608 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fdee80396b0 0x7fdee803bb70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fdee4006fd0 tx=0x7fdee4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:52:31.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:31.413+0000 7fdef5ffb700 1 -- 172.21.15.47:0/3386059608 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fdeec013070 con 0x7fdef81015a0 2024-04-17T05:52:31.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:31.625+0000 7fdefee32700 1 -- 172.21.15.47:0/3386059608 --> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi098", "target": ["mon-mgr", ""]}) -- 0x7fdef80640b0 con 0x7fdee80396b0 2024-04-17T05:52:32.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:31 smithi047 ceph-mon[32223]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi098", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T05:52:33.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:32 smithi047 ceph-mon[32223]: Deploying cephadm binary to smithi098 2024-04-17T05:52:34.958 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:34 smithi047 ceph-mon[32223]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T05:52:35.964 INFO:teuthology.orchestra.run.smithi047.stdout:Added host 'smithi098' with addr '172.21.15.98' 2024-04-17T05:52:35.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:35.966+0000 7fdef5ffb700 1 -- 172.21.15.47:0/3386059608 <== mgr.14152 v2:172.21.15.47:6800/3157902806 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7fdef80640b0 con 0x7fdee80396b0 2024-04-17T05:52:35.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:35.968+0000 7fdefee32700 1 -- 172.21.15.47:0/3386059608 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fdee80396b0 msgr2=0x7fdee803bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:52:35.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:35.968+0000 7fdefee32700 1 --2- 172.21.15.47:0/3386059608 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fdee80396b0 0x7fdee803bb70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fdee4006fd0 tx=0x7fdee4006e40 comp rx=0 tx=0).stop 2024-04-17T05:52:35.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:35.969+0000 7fdefee32700 1 -- 172.21.15.47:0/3386059608 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdef81015a0 msgr2=0x7fdef8197fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:52:35.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:35.969+0000 7fdefee32700 1 --2- 172.21.15.47:0/3386059608 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdef81015a0 0x7fdef8197fd0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fdeec005850 tx=0x7fdeec005030 comp rx=0 tx=0).stop 2024-04-17T05:52:35.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:35.969+0000 7fdefee32700 1 -- 172.21.15.47:0/3386059608 shutdown_connections 2024-04-17T05:52:35.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:35.969+0000 7fdefee32700 1 --2- 172.21.15.47:0/3386059608 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fdee80396b0 0x7fdee803bb70 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:52:35.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:35.969+0000 7fdefee32700 1 --2- 172.21.15.47:0/3386059608 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdef81015a0 0x7fdef8197fd0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:52:35.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:35.969+0000 7fdefee32700 1 -- 172.21.15.47:0/3386059608 >> 172.21.15.47:0/3386059608 conn(0x7fdef80fce50 msgr2=0x7fdef806a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:52:35.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:35.969+0000 7fdefee32700 1 -- 172.21.15.47:0/3386059608 shutdown_connections 2024-04-17T05:52:35.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:35.969+0000 7fdefee32700 1 -- 172.21.15.47:0/3386059608 wait complete. 2024-04-17T05:52:36.546 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 565db2d8-fc7e-11ee-bc91-c7b262605968 -- ceph orch host ls --format=json 2024-04-17T05:52:36.792 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T05:52:37.168 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:36 smithi047 ceph-mon[32223]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T05:52:37.168 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:36 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:52:37.168 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:36 smithi047 ceph-mon[32223]: Added host smithi098 2024-04-17T05:52:37.168 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:36 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T05:52:37.168 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:36 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:52:38.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:38.164+0000 7fb6b9167700 1 -- 172.21.15.47:0/3322218994 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6b41015c0 msgr2=0x7fb6b41019a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:52:38.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:38.164+0000 7fb6b9167700 1 --2- 172.21.15.47:0/3322218994 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6b41015c0 0x7fb6b41019a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb6a4009910 tx=0x7fb6a4009c20 comp rx=0 tx=0).stop 2024-04-17T05:52:38.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:38.164+0000 7fb6b9167700 1 -- 172.21.15.47:0/3322218994 shutdown_connections 2024-04-17T05:52:38.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:38.164+0000 7fb6b9167700 1 --2- 172.21.15.47:0/3322218994 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6b41015c0 0x7fb6b41019a0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:52:38.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:38.164+0000 7fb6b9167700 1 -- 172.21.15.47:0/3322218994 >> 172.21.15.47:0/3322218994 conn(0x7fb6b40fce50 msgr2=0x7fb6b40ff270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:52:38.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:38.164+0000 7fb6b9167700 1 -- 172.21.15.47:0/3322218994 shutdown_connections 2024-04-17T05:52:38.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:38.164+0000 7fb6b9167700 1 -- 172.21.15.47:0/3322218994 wait complete. 2024-04-17T05:52:38.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:38.165+0000 7fb6b9167700 1 Processor -- start 2024-04-17T05:52:38.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:38.165+0000 7fb6b9167700 1 -- start start 2024-04-17T05:52:38.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:38.165+0000 7fb6b9167700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6b41015c0 0x7fb6b4198e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:52:38.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:38.165+0000 7fb6b9167700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb6b410b990 con 0x7fb6b41015c0 2024-04-17T05:52:38.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:38.165+0000 7fb6b2d9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6b41015c0 0x7fb6b4198e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:52:38.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:38.165+0000 7fb6b2d9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6b41015c0 0x7fb6b4198e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55294/0 (socket says 172.21.15.47:55294) 2024-04-17T05:52:38.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:38.165+0000 7fb6b2d9d700 1 -- 172.21.15.47:0/112035554 learned_addr learned my addr 172.21.15.47:0/112035554 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:52:38.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:38.165+0000 7fb6b2d9d700 1 -- 172.21.15.47:0/112035554 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6b4199350 con 0x7fb6b41015c0 2024-04-17T05:52:38.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:38.165+0000 7fb6b2d9d700 1 --2- 172.21.15.47:0/112035554 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6b41015c0 0x7fb6b4198e10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb6a4005850 tx=0x7fb6a4005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:52:38.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:38.166+0000 7fb6b0d99700 1 -- 172.21.15.47:0/112035554 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb6a4026070 con 0x7fb6b41015c0 2024-04-17T05:52:38.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:38.166+0000 7fb6b0d99700 1 -- 172.21.15.47:0/112035554 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6a400bbd0 con 0x7fb6b41015c0 2024-04-17T05:52:38.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:38.166+0000 7fb6b9167700 1 -- 172.21.15.47:0/112035554 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6b4199640 con 0x7fb6b41015c0 2024-04-17T05:52:38.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:38.166+0000 7fb6b0d99700 1 -- 172.21.15.47:0/112035554 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb6a4003970 con 0x7fb6b41015c0 2024-04-17T05:52:38.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:38.166+0000 7fb6b9167700 1 -- 172.21.15.47:0/112035554 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6b4199b00 con 0x7fb6b41015c0 2024-04-17T05:52:38.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:38.167+0000 7fb6b9167700 1 -- 172.21.15.47:0/112035554 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6b4102aa0 con 0x7fb6b41015c0 2024-04-17T05:52:38.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:38.167+0000 7fb6b0d99700 1 -- 172.21.15.47:0/112035554 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fb6a4029070 con 0x7fb6b41015c0 2024-04-17T05:52:38.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:38.167+0000 7fb6b0d99700 1 --2- 172.21.15.47:0/112035554 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fb6a00396b0 0x7fb6a003bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:52:38.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:38.167+0000 7fb6b0d99700 1 -- 172.21.15.47:0/112035554 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb6a401e070 con 0x7fb6b41015c0 2024-04-17T05:52:38.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:38.167+0000 7fb6aa59c700 1 --2- 172.21.15.47:0/112035554 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fb6a00396b0 0x7fb6a003bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:52:38.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:38.167+0000 7fb6aa59c700 1 --2- 172.21.15.47:0/112035554 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fb6a00396b0 0x7fb6a003bb70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb69c006fd0 tx=0x7fb69c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:52:38.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:38.172+0000 7fb6b0d99700 1 -- 172.21.15.47:0/112035554 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fb6a4013070 con 0x7fb6b41015c0 2024-04-17T05:52:38.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:38.384+0000 7fb6b9167700 1 -- 172.21.15.47:0/112035554 --> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb6b406aeb0 con 0x7fb6a00396b0 2024-04-17T05:52:38.383 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:38 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:52:38.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:38.386+0000 7fb6b0d99700 1 -- 172.21.15.47:0/112035554 <== mgr.14152 v2:172.21.15.47:6800/3157902806 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+159 (secure 0 0 0) 0x7fb6b406aeb0 con 0x7fb6a00396b0 2024-04-17T05:52:38.384 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-17T05:52:38.384 INFO:teuthology.orchestra.run.smithi047.stdout:[{"addr": "172.21.15.47", "hostname": "smithi047", "labels": [], "status": ""}, {"addr": "172.21.15.98", "hostname": "smithi098", "labels": [], "status": ""}] 2024-04-17T05:52:38.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:38.388+0000 7fb6b9167700 1 -- 172.21.15.47:0/112035554 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fb6a00396b0 msgr2=0x7fb6a003bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:52:38.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:38.388+0000 7fb6b9167700 1 --2- 172.21.15.47:0/112035554 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fb6a00396b0 0x7fb6a003bb70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb69c006fd0 tx=0x7fb69c006e40 comp rx=0 tx=0).stop 2024-04-17T05:52:38.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:38.388+0000 7fb6b9167700 1 -- 172.21.15.47:0/112035554 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6b41015c0 msgr2=0x7fb6b4198e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:52:38.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:38.388+0000 7fb6b9167700 1 --2- 172.21.15.47:0/112035554 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6b41015c0 0x7fb6b4198e10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb6a4005850 tx=0x7fb6a4005030 comp rx=0 tx=0).stop 2024-04-17T05:52:38.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:38.388+0000 7fb6b9167700 1 -- 172.21.15.47:0/112035554 shutdown_connections 2024-04-17T05:52:38.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:38.388+0000 7fb6b9167700 1 --2- 172.21.15.47:0/112035554 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fb6a00396b0 0x7fb6a003bb70 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:52:38.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:38.388+0000 7fb6b9167700 1 --2- 172.21.15.47:0/112035554 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6b41015c0 0x7fb6b4198e10 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:52:38.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:38.388+0000 7fb6b9167700 1 -- 172.21.15.47:0/112035554 >> 172.21.15.47:0/112035554 conn(0x7fb6b40fce50 msgr2=0x7fb6b4102f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:52:38.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:38.388+0000 7fb6b9167700 1 -- 172.21.15.47:0/112035554 shutdown_connections 2024-04-17T05:52:38.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:38.388+0000 7fb6b9167700 1 -- 172.21.15.47:0/112035554 wait complete. 2024-04-17T05:52:38.982 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi103 2024-04-17T05:52:38.982 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-04-17T05:52:38.982 DEBUG:teuthology.orchestra.run.smithi103:> dd of=/etc/ceph/ceph.conf 2024-04-17T05:52:39.001 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-04-17T05:52:39.001 DEBUG:teuthology.orchestra.run.smithi103:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-17T05:52:39.057 INFO:tasks.cephadm:Adding host smithi103 to orchestrator... 2024-04-17T05:52:39.057 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 565db2d8-fc7e-11ee-bc91-c7b262605968 -- ceph orch host add smithi103 2024-04-17T05:52:39.302 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T05:52:39.729 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:39 smithi047 ceph-mon[32223]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T05:52:39.730 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:39 smithi047 ceph-mon[32223]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-17T05:52:39.730 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:39 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:52:40.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:40.797+0000 7ffa589f5700 1 -- 172.21.15.47:0/3201938515 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa541015a0 msgr2=0x7ffa54101980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:52:40.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:40.797+0000 7ffa589f5700 1 --2- 172.21.15.47:0/3201938515 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa541015a0 0x7ffa54101980 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ffa4400b0a0 tx=0x7ffa4400b3b0 comp rx=0 tx=0).stop 2024-04-17T05:52:40.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:40.797+0000 7ffa589f5700 1 -- 172.21.15.47:0/3201938515 shutdown_connections 2024-04-17T05:52:40.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:40.797+0000 7ffa589f5700 1 --2- 172.21.15.47:0/3201938515 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa541015a0 0x7ffa54101980 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:52:40.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:40.797+0000 7ffa589f5700 1 -- 172.21.15.47:0/3201938515 >> 172.21.15.47:0/3201938515 conn(0x7ffa540fce50 msgr2=0x7ffa540ff270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:52:40.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:40.797+0000 7ffa589f5700 1 -- 172.21.15.47:0/3201938515 shutdown_connections 2024-04-17T05:52:40.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:40.797+0000 7ffa589f5700 1 -- 172.21.15.47:0/3201938515 wait complete. 2024-04-17T05:52:40.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:40.798+0000 7ffa589f5700 1 Processor -- start 2024-04-17T05:52:40.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:40.798+0000 7ffa589f5700 1 -- start start 2024-04-17T05:52:40.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:40.798+0000 7ffa589f5700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa541015a0 0x7ffa54103040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:52:40.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:40.798+0000 7ffa589f5700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ffa5410b970 con 0x7ffa541015a0 2024-04-17T05:52:40.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:40.798+0000 7ffa5259c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa541015a0 0x7ffa54103040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:52:40.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:40.798+0000 7ffa5259c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa541015a0 0x7ffa54103040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55312/0 (socket says 172.21.15.47:55312) 2024-04-17T05:52:40.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:40.798+0000 7ffa5259c700 1 -- 172.21.15.47:0/3788269796 learned_addr learned my addr 172.21.15.47:0/3788269796 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:52:40.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:40.798+0000 7ffa5259c700 1 -- 172.21.15.47:0/3788269796 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa54104f30 con 0x7ffa541015a0 2024-04-17T05:52:40.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:40.799+0000 7ffa5259c700 1 --2- 172.21.15.47:0/3788269796 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa541015a0 0x7ffa54103040 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ffa44015040 tx=0x7ffa44003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:52:40.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:40.799+0000 7ffa4b7fe700 1 -- 172.21.15.47:0/3788269796 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa44017040 con 0x7ffa541015a0 2024-04-17T05:52:40.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:40.799+0000 7ffa4b7fe700 1 -- 172.21.15.47:0/3788269796 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa44022070 con 0x7ffa541015a0 2024-04-17T05:52:40.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:40.799+0000 7ffa589f5700 1 -- 172.21.15.47:0/3788269796 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa541035e0 con 0x7ffa541015a0 2024-04-17T05:52:40.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:40.799+0000 7ffa4b7fe700 1 -- 172.21.15.47:0/3788269796 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa4401d5d0 con 0x7ffa541015a0 2024-04-17T05:52:40.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:40.799+0000 7ffa589f5700 1 -- 172.21.15.47:0/3788269796 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa54103aa0 con 0x7ffa541015a0 2024-04-17T05:52:40.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:40.800+0000 7ffa4b7fe700 1 -- 172.21.15.47:0/3788269796 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7ffa4401d880 con 0x7ffa541015a0 2024-04-17T05:52:40.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:40.800+0000 7ffa589f5700 1 -- 172.21.15.47:0/3788269796 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa54102a80 con 0x7ffa541015a0 2024-04-17T05:52:40.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:40.800+0000 7ffa4b7fe700 1 --2- 172.21.15.47:0/3788269796 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7ffa400396b0 0x7ffa4003bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:52:40.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:40.800+0000 7ffa4b7fe700 1 -- 172.21.15.47:0/3788269796 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ffa4400f050 con 0x7ffa541015a0 2024-04-17T05:52:40.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:40.800+0000 7ffa51d9b700 1 --2- 172.21.15.47:0/3788269796 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7ffa400396b0 0x7ffa4003bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:52:40.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:40.800+0000 7ffa51d9b700 1 --2- 172.21.15.47:0/3788269796 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7ffa400396b0 0x7ffa4003bb70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c006fd0 tx=0x7ffa3c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:52:40.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:40.805+0000 7ffa4b7fe700 1 -- 172.21.15.47:0/3788269796 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7ffa44007e20 con 0x7ffa541015a0 2024-04-17T05:52:41.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:41.018+0000 7ffa589f5700 1 -- 172.21.15.47:0/3788269796 --> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi103", "target": ["mon-mgr", ""]}) -- 0x7ffa54103cd0 con 0x7ffa400396b0 2024-04-17T05:52:41.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:41 smithi047 ceph-mon[32223]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T05:52:42.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:42 smithi047 ceph-mon[32223]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi103", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T05:52:42.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:42 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:52:42.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:42 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:52:42.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:42 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:52:42.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:42 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:52:42.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:42 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi098", "name": "osd_memory_target"}]: dispatch 2024-04-17T05:52:42.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:42 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:52:42.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:42 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T05:52:42.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:42 smithi047 ceph-mon[32223]: Updating smithi098:/etc/ceph/ceph.conf 2024-04-17T05:52:42.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:42 smithi047 ceph-mon[32223]: Deploying cephadm binary to smithi103 2024-04-17T05:52:43.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:43 smithi047 ceph-mon[32223]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T05:52:43.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:43 smithi047 ceph-mon[32223]: Updating smithi098:/var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/config/ceph.conf 2024-04-17T05:52:43.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:43 smithi047 ceph-mon[32223]: Updating smithi098:/etc/ceph/ceph.client.admin.keyring 2024-04-17T05:52:45.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:45.329+0000 7ffa4b7fe700 1 -- 172.21.15.47:0/3788269796 <== mgr.14152 v2:172.21.15.47:6800/3157902806 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7ffa54103cd0 con 0x7ffa400396b0 2024-04-17T05:52:45.328 INFO:teuthology.orchestra.run.smithi047.stdout:Added host 'smithi103' with addr '172.21.15.103' 2024-04-17T05:52:45.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:45.332+0000 7ffa589f5700 1 -- 172.21.15.47:0/3788269796 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7ffa400396b0 msgr2=0x7ffa4003bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:52:45.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:45.332+0000 7ffa589f5700 1 --2- 172.21.15.47:0/3788269796 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7ffa400396b0 0x7ffa4003bb70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c006fd0 tx=0x7ffa3c006e40 comp rx=0 tx=0).stop 2024-04-17T05:52:45.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:45.332+0000 7ffa589f5700 1 -- 172.21.15.47:0/3788269796 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa541015a0 msgr2=0x7ffa54103040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:52:45.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:45.332+0000 7ffa589f5700 1 --2- 172.21.15.47:0/3788269796 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa541015a0 0x7ffa54103040 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ffa44015040 tx=0x7ffa44003d50 comp rx=0 tx=0).stop 2024-04-17T05:52:45.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:45.332+0000 7ffa589f5700 1 -- 172.21.15.47:0/3788269796 shutdown_connections 2024-04-17T05:52:45.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:45.332+0000 7ffa589f5700 1 --2- 172.21.15.47:0/3788269796 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7ffa400396b0 0x7ffa4003bb70 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:52:45.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:45.332+0000 7ffa589f5700 1 --2- 172.21.15.47:0/3788269796 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa541015a0 0x7ffa54103040 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:52:45.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:45.332+0000 7ffa589f5700 1 -- 172.21.15.47:0/3788269796 >> 172.21.15.47:0/3788269796 conn(0x7ffa540fce50 msgr2=0x7ffa5406a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:52:45.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:45.332+0000 7ffa589f5700 1 -- 172.21.15.47:0/3788269796 shutdown_connections 2024-04-17T05:52:45.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:45.332+0000 7ffa589f5700 1 -- 172.21.15.47:0/3788269796 wait complete. 2024-04-17T05:52:45.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:45 smithi047 ceph-mon[32223]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T05:52:45.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:45 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:52:45.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:45 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:52:45.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:45 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:52:45.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:45 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:52:45.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:45 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T05:52:45.944 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 565db2d8-fc7e-11ee-bc91-c7b262605968 -- ceph orch host ls --format=json 2024-04-17T05:52:46.186 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T05:52:46.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:46 smithi047 ceph-mon[32223]: Updating smithi098:/var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/config/ceph.client.admin.keyring 2024-04-17T05:52:46.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:46 smithi047 ceph-mon[32223]: Added host smithi103 2024-04-17T05:52:46.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:46 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:52:46.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:46 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:52:47.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:47.657+0000 7f8985aed700 1 -- 172.21.15.47:0/322663291 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8980101600 msgr2=0x7f89801019e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:52:47.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:47.657+0000 7f8985aed700 1 --2- 172.21.15.47:0/322663291 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8980101600 0x7f89801019e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8970009910 tx=0x7f8970009c20 comp rx=0 tx=0).stop 2024-04-17T05:52:47.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:47.657+0000 7f8985aed700 1 -- 172.21.15.47:0/322663291 shutdown_connections 2024-04-17T05:52:47.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:47.657+0000 7f8985aed700 1 --2- 172.21.15.47:0/322663291 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8980101600 0x7f89801019e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:52:47.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:47.657+0000 7f8985aed700 1 -- 172.21.15.47:0/322663291 >> 172.21.15.47:0/322663291 conn(0x7f8980076eb0 msgr2=0x7f89800772c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:52:47.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:47.657+0000 7f8985aed700 1 -- 172.21.15.47:0/322663291 shutdown_connections 2024-04-17T05:52:47.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:47.657+0000 7f8985aed700 1 -- 172.21.15.47:0/322663291 wait complete. 2024-04-17T05:52:47.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:47.658+0000 7f8985aed700 1 Processor -- start 2024-04-17T05:52:47.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:47.658+0000 7f8985aed700 1 -- start start 2024-04-17T05:52:47.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:47.658+0000 7f8985aed700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8980101600 0x7f8980196c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:52:47.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:47.658+0000 7f8985aed700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f898010b980 con 0x7f8980101600 2024-04-17T05:52:47.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:47.658+0000 7f897f7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8980101600 0x7f8980196c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:52:47.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:47.658+0000 7f897f7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8980101600 0x7f8980196c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44496/0 (socket says 172.21.15.47:44496) 2024-04-17T05:52:47.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:47.658+0000 7f897f7fe700 1 -- 172.21.15.47:0/956390693 learned_addr learned my addr 172.21.15.47:0/956390693 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:52:47.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:47.658+0000 7f897f7fe700 1 -- 172.21.15.47:0/956390693 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f898006ac70 con 0x7f8980101600 2024-04-17T05:52:47.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:47.658+0000 7f897f7fe700 1 --2- 172.21.15.47:0/956390693 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8980101600 0x7f8980196c80 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8970006010 tx=0x7f8970005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:52:47.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:47.659+0000 7f897d7fa700 1 -- 172.21.15.47:0/956390693 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8970026070 con 0x7f8980101600 2024-04-17T05:52:47.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:47.659+0000 7f8985aed700 1 -- 172.21.15.47:0/956390693 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89801971c0 con 0x7f8980101600 2024-04-17T05:52:47.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:47.659+0000 7f897d7fa700 1 -- 172.21.15.47:0/956390693 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f897000bbd0 con 0x7f8980101600 2024-04-17T05:52:47.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:47.659+0000 7f8985aed700 1 -- 172.21.15.47:0/956390693 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8980197680 con 0x7f8980101600 2024-04-17T05:52:47.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:47.659+0000 7f897d7fa700 1 -- 172.21.15.47:0/956390693 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8970003910 con 0x7f8980101600 2024-04-17T05:52:47.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:47.660+0000 7f897d7fa700 1 -- 172.21.15.47:0/956390693 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f8970029070 con 0x7f8980101600 2024-04-17T05:52:47.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:47.660+0000 7f897d7fa700 1 --2- 172.21.15.47:0/956390693 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f896c0396b0 0x7f896c03bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:52:47.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:47.660+0000 7f8985aed700 1 -- 172.21.15.47:0/956390693 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8960005330 con 0x7f8980101600 2024-04-17T05:52:47.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:47.660+0000 7f897d7fa700 1 -- 172.21.15.47:0/956390693 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f897001e070 con 0x7f8980101600 2024-04-17T05:52:47.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:47.660+0000 7f8977fff700 1 --2- 172.21.15.47:0/956390693 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f896c0396b0 0x7f896c03bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:52:47.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:47.660+0000 7f8977fff700 1 --2- 172.21.15.47:0/956390693 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f896c0396b0 0x7f896c03bb70 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f8968006fd0 tx=0x7f8968006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:52:47.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:47.665+0000 7f897d7fa700 1 -- 172.21.15.47:0/956390693 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f8970013070 con 0x7f8980101600 2024-04-17T05:52:47.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:47.881+0000 7f8985aed700 1 -- 172.21.15.47:0/956390693 --> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8960000bc0 con 0x7f896c0396b0 2024-04-17T05:52:47.879 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:47 smithi047 ceph-mon[32223]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T05:52:47.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:47.882+0000 7f897d7fa700 1 -- 172.21.15.47:0/956390693 <== mgr.14152 v2:172.21.15.47:6800/3157902806 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+239 (secure 0 0 0) 0x7f8960000bc0 con 0x7f896c0396b0 2024-04-17T05:52:47.880 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-17T05:52:47.880 INFO:teuthology.orchestra.run.smithi047.stdout:[{"addr": "172.21.15.47", "hostname": "smithi047", "labels": [], "status": ""}, {"addr": "172.21.15.98", "hostname": "smithi098", "labels": [], "status": ""}, {"addr": "172.21.15.103", "hostname": "smithi103", "labels": [], "status": ""}] 2024-04-17T05:52:47.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:47.885+0000 7f8985aed700 1 -- 172.21.15.47:0/956390693 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f896c0396b0 msgr2=0x7f896c03bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:52:47.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:47.885+0000 7f8985aed700 1 --2- 172.21.15.47:0/956390693 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f896c0396b0 0x7f896c03bb70 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f8968006fd0 tx=0x7f8968006e40 comp rx=0 tx=0).stop 2024-04-17T05:52:47.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:47.885+0000 7f8985aed700 1 -- 172.21.15.47:0/956390693 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8980101600 msgr2=0x7f8980196c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:52:47.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:47.885+0000 7f8985aed700 1 --2- 172.21.15.47:0/956390693 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8980101600 0x7f8980196c80 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8970006010 tx=0x7f8970005030 comp rx=0 tx=0).stop 2024-04-17T05:52:47.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:47.885+0000 7f8985aed700 1 -- 172.21.15.47:0/956390693 shutdown_connections 2024-04-17T05:52:47.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:47.885+0000 7f8985aed700 1 --2- 172.21.15.47:0/956390693 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f896c0396b0 0x7f896c03bb70 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:52:47.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:47.885+0000 7f8985aed700 1 --2- 172.21.15.47:0/956390693 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8980101600 0x7f8980196c80 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:52:47.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:47.885+0000 7f8985aed700 1 -- 172.21.15.47:0/956390693 >> 172.21.15.47:0/956390693 conn(0x7f8980076eb0 msgr2=0x7f89801043e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:52:47.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:47.885+0000 7f8985aed700 1 -- 172.21.15.47:0/956390693 shutdown_connections 2024-04-17T05:52:47.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:47.885+0000 7f8985aed700 1 -- 172.21.15.47:0/956390693 wait complete. 2024-04-17T05:52:48.462 INFO:tasks.cephadm:Setting crush tunables to default 2024-04-17T05:52:48.462 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 565db2d8-fc7e-11ee-bc91-c7b262605968 -- ceph osd crush tunables default 2024-04-17T05:52:48.710 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T05:52:49.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:48 smithi047 ceph-mon[32223]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T05:52:49.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:48 smithi047 ceph-mon[32223]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-17T05:52:49.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:48 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:52:50.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:50.117+0000 7f88973d3700 1 -- 172.21.15.47:0/2426915910 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8890101600 msgr2=0x7f88901019e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:52:50.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:50.117+0000 7f88973d3700 1 --2- 172.21.15.47:0/2426915910 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8890101600 0x7f88901019e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f888c009910 tx=0x7f888c009c20 comp rx=0 tx=0).stop 2024-04-17T05:52:50.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:50.118+0000 7f88973d3700 1 -- 172.21.15.47:0/2426915910 shutdown_connections 2024-04-17T05:52:50.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:50.118+0000 7f88973d3700 1 --2- 172.21.15.47:0/2426915910 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8890101600 0x7f88901019e0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:52:50.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:50.118+0000 7f88973d3700 1 -- 172.21.15.47:0/2426915910 >> 172.21.15.47:0/2426915910 conn(0x7f8890076eb0 msgr2=0x7f88900772c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:52:50.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:50.118+0000 7f88973d3700 1 -- 172.21.15.47:0/2426915910 shutdown_connections 2024-04-17T05:52:50.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:50.118+0000 7f88973d3700 1 -- 172.21.15.47:0/2426915910 wait complete. 2024-04-17T05:52:50.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:50.118+0000 7f88973d3700 1 Processor -- start 2024-04-17T05:52:50.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:50.118+0000 7f88973d3700 1 -- start start 2024-04-17T05:52:50.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:50.119+0000 7f88973d3700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8890101600 0x7f8890103050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:52:50.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:50.119+0000 7f88973d3700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f889010b980 con 0x7f8890101600 2024-04-17T05:52:50.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:50.119+0000 7f889516f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8890101600 0x7f8890103050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:52:50.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:50.119+0000 7f889516f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8890101600 0x7f8890103050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44518/0 (socket says 172.21.15.47:44518) 2024-04-17T05:52:50.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:50.119+0000 7f889516f700 1 -- 172.21.15.47:0/3349535563 learned_addr learned my addr 172.21.15.47:0/3349535563 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:52:50.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:50.119+0000 7f889516f700 1 -- 172.21.15.47:0/3349535563 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8890104f40 con 0x7f8890101600 2024-04-17T05:52:50.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:50.119+0000 7f889516f700 1 --2- 172.21.15.47:0/3349535563 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8890101600 0x7f8890103050 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f888c005850 tx=0x7f888c005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:52:50.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:50.119+0000 7f88867fc700 1 -- 172.21.15.47:0/3349535563 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f888c026070 con 0x7f8890101600 2024-04-17T05:52:50.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:50.119+0000 7f88867fc700 1 -- 172.21.15.47:0/3349535563 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f888c00bc30 con 0x7f8890101600 2024-04-17T05:52:50.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:50.119+0000 7f88973d3700 1 -- 172.21.15.47:0/3349535563 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88901035f0 con 0x7f8890101600 2024-04-17T05:52:50.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:50.119+0000 7f88867fc700 1 -- 172.21.15.47:0/3349535563 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f888c003a20 con 0x7f8890101600 2024-04-17T05:52:50.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:50.119+0000 7f88973d3700 1 -- 172.21.15.47:0/3349535563 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8890103ab0 con 0x7f8890101600 2024-04-17T05:52:50.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:50.120+0000 7f88867fc700 1 -- 172.21.15.47:0/3349535563 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f888c029070 con 0x7f8890101600 2024-04-17T05:52:50.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:50.120+0000 7f88973d3700 1 -- 172.21.15.47:0/3349535563 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8890102a90 con 0x7f8890101600 2024-04-17T05:52:50.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:50.120+0000 7f88867fc700 1 --2- 172.21.15.47:0/3349535563 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f887c0396b0 0x7f887c03bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:52:50.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:50.120+0000 7f88867fc700 1 -- 172.21.15.47:0/3349535563 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f888c01e070 con 0x7f8890101600 2024-04-17T05:52:50.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:50.120+0000 7f889496e700 1 --2- 172.21.15.47:0/3349535563 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f887c0396b0 0x7f887c03bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:52:50.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:50.121+0000 7f889496e700 1 --2- 172.21.15.47:0/3349535563 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f887c0396b0 0x7f887c03bb70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f8880006fd0 tx=0x7f8880006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:52:50.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:50.126+0000 7f88867fc700 1 -- 172.21.15.47:0/3349535563 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f888c013070 con 0x7f8890101600 2024-04-17T05:52:50.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:50.339+0000 7f88973d3700 1 -- 172.21.15.47:0/3349535563 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f8890104620 con 0x7f8890101600 2024-04-17T05:52:50.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:50.996+0000 7f88867fc700 1 -- 172.21.15.47:0/3349535563 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f888c004390 con 0x7f8890101600 2024-04-17T05:52:50.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:50.999+0000 7f88973d3700 1 -- 172.21.15.47:0/3349535563 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f887c0396b0 msgr2=0x7f887c03bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:52:50.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:50.999+0000 7f88973d3700 1 --2- 172.21.15.47:0/3349535563 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f887c0396b0 0x7f887c03bb70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f8880006fd0 tx=0x7f8880006e40 comp rx=0 tx=0).stop 2024-04-17T05:52:50.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:50.999+0000 7f88973d3700 1 -- 172.21.15.47:0/3349535563 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8890101600 msgr2=0x7f8890103050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:52:50.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:50.999+0000 7f88973d3700 1 --2- 172.21.15.47:0/3349535563 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8890101600 0x7f8890103050 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f888c005850 tx=0x7f888c005030 comp rx=0 tx=0).stop 2024-04-17T05:52:50.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:50.999+0000 7f88973d3700 1 -- 172.21.15.47:0/3349535563 shutdown_connections 2024-04-17T05:52:50.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:50.999+0000 7f88973d3700 1 --2- 172.21.15.47:0/3349535563 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f887c0396b0 0x7f887c03bb70 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:52:50.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:50.999+0000 7f88973d3700 1 --2- 172.21.15.47:0/3349535563 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8890101600 0x7f8890103050 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:52:50.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:50.999+0000 7f88973d3700 1 -- 172.21.15.47:0/3349535563 >> 172.21.15.47:0/3349535563 conn(0x7f8890076eb0 msgr2=0x7f8890069e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:52:50.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:50.999+0000 7f88973d3700 1 -- 172.21.15.47:0/3349535563 shutdown_connections 2024-04-17T05:52:50.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:52:50.999+0000 7f88973d3700 1 -- 172.21.15.47:0/3349535563 wait complete. 2024-04-17T05:52:50.998 INFO:teuthology.orchestra.run.smithi047.stderr:adjusted tunables profile to default 2024-04-17T05:52:51.296 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:50 smithi047 ceph-mon[32223]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T05:52:51.297 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:50 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/3349535563' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-17T05:52:51.604 INFO:tasks.cephadm:Adding mon.a on smithi047 2024-04-17T05:52:51.605 INFO:tasks.cephadm:Adding mon.b on smithi098 2024-04-17T05:52:51.605 INFO:tasks.cephadm:Adding mon.c on smithi103 2024-04-17T05:52:51.605 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 565db2d8-fc7e-11ee-bc91-c7b262605968 -- ceph orch apply mon '3;smithi047:172.21.15.47=a;smithi098:172.21.15.98=b;smithi103:172.21.15.103=c' 2024-04-17T05:52:51.839 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/config/ceph.conf 2024-04-17T05:52:52.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:51 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/3349535563' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-17T05:52:52.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:51 smithi047 ceph-mon[32223]: osdmap e4: 0 total, 0 up, 0 in 2024-04-17T05:52:52.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:51 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:52:52.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:51 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:52:52.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:51 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:52:52.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:51 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:52:52.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:51 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi103", "name": "osd_memory_target"}]: dispatch 2024-04-17T05:52:52.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:51 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:52:52.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:51 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T05:52:52.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:51 smithi047 ceph-mon[32223]: Updating smithi103:/etc/ceph/ceph.conf 2024-04-17T05:52:52.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:51 smithi047 ceph-mon[32223]: Updating smithi103:/var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/config/ceph.conf 2024-04-17T05:52:53.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:52:53.362+0000 7f0c15e88700 1 -- 172.21.15.103:0/2223301260 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c10108560 msgr2=0x7f0c10108940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:52:53.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:52:53.362+0000 7f0c15e88700 1 --2- 172.21.15.103:0/2223301260 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c10108560 0x7f0c10108940 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0c00009910 tx=0x7f0c00009c20 comp rx=0 tx=0).stop 2024-04-17T05:52:53.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:52:53.362+0000 7f0c15e88700 1 -- 172.21.15.103:0/2223301260 shutdown_connections 2024-04-17T05:52:53.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:52:53.362+0000 7f0c15e88700 1 --2- 172.21.15.103:0/2223301260 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c10108560 0x7f0c10108940 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:52:53.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:52:53.362+0000 7f0c15e88700 1 -- 172.21.15.103:0/2223301260 >> 172.21.15.103:0/2223301260 conn(0x7f0c10077ad0 msgr2=0x7f0c10077ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:52:53.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:52:53.362+0000 7f0c15e88700 1 -- 172.21.15.103:0/2223301260 shutdown_connections 2024-04-17T05:52:53.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:52:53.362+0000 7f0c15e88700 1 -- 172.21.15.103:0/2223301260 wait complete. 2024-04-17T05:52:53.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:52:53.363+0000 7f0c15e88700 1 Processor -- start 2024-04-17T05:52:53.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:52:53.363+0000 7f0c15e88700 1 -- start start 2024-04-17T05:52:53.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:52:53.363+0000 7f0c15e88700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c10108560 0x7f0c1019a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:52:53.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:52:53.363+0000 7f0c15e88700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f0c1010cb20 con 0x7f0c10108560 2024-04-17T05:52:53.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:52:53.364+0000 7f0c0f7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c10108560 0x7f0c1019a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:52:53.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:52:53.364+0000 7f0c0f7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c10108560 0x7f0c1019a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.103:34814/0 (socket says 172.21.15.103:34814) 2024-04-17T05:52:53.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:52:53.364+0000 7f0c0f7fe700 1 -- 172.21.15.103:0/832606683 learned_addr learned my addr 172.21.15.103:0/832606683 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-04-17T05:52:53.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:52:53.364+0000 7f0c0f7fe700 1 -- 172.21.15.103:0/832606683 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c1019a640 con 0x7f0c10108560 2024-04-17T05:52:53.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:52:53.364+0000 7f0c0f7fe700 1 --2- 172.21.15.103:0/832606683 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c10108560 0x7f0c1019a100 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0c00005850 tx=0x7f0c00005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:52:53.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:52:53.364+0000 7f0c0cff9700 1 -- 172.21.15.103:0/832606683 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0c00026070 con 0x7f0c10108560 2024-04-17T05:52:53.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:52:53.364+0000 7f0c15e88700 1 -- 172.21.15.103:0/832606683 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c1019a8d0 con 0x7f0c10108560 2024-04-17T05:52:53.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:52:53.364+0000 7f0c0cff9700 1 -- 172.21.15.103:0/832606683 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c0000bc30 con 0x7f0c10108560 2024-04-17T05:52:53.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:52:53.364+0000 7f0c0cff9700 1 -- 172.21.15.103:0/832606683 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0c00003a20 con 0x7f0c10108560 2024-04-17T05:52:53.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:52:53.364+0000 7f0c15e88700 1 -- 172.21.15.103:0/832606683 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c1019e9d0 con 0x7f0c10108560 2024-04-17T05:52:53.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:52:53.365+0000 7f0c0cff9700 1 -- 172.21.15.103:0/832606683 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f0c00029070 con 0x7f0c10108560 2024-04-17T05:52:53.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:52:53.365+0000 7f0c15e88700 1 -- 172.21.15.103:0/832606683 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c10103d00 con 0x7f0c10108560 2024-04-17T05:52:53.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:52:53.365+0000 7f0c0cff9700 1 --2- 172.21.15.103:0/832606683 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f0bf8039650 0x7f0bf803bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:52:53.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:52:53.365+0000 7f0c0cff9700 1 -- 172.21.15.103:0/832606683 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f0c0001e070 con 0x7f0c10108560 2024-04-17T05:52:53.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:52:53.366+0000 7f0c0effd700 1 --2- 172.21.15.103:0/832606683 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f0bf8039650 0x7f0bf803bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:52:53.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:52:53.366+0000 7f0c0effd700 1 --2- 172.21.15.103:0/832606683 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f0bf8039650 0x7f0bf803bb10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0c04006fd0 tx=0x7f0c04006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:52:53.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:52:53.371+0000 7f0c0cff9700 1 -- 172.21.15.103:0/832606683 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f0c00013070 con 0x7f0c10108560 2024-04-17T05:52:53.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:52:53.581+0000 7f0c15e88700 1 -- 172.21.15.103:0/832606683 --> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi047:172.21.15.47=a;smithi098:172.21.15.98=b;smithi103:172.21.15.103=c", "target": ["mon-mgr", ""]}) -- 0x7f0c1019b460 con 0x7f0bf8039650 2024-04-17T05:52:53.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:52:53.588+0000 7f0c0cff9700 1 -- 172.21.15.103:0/832606683 <== mgr.14152 v2:172.21.15.47:6800/3157902806 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f0c1019b460 con 0x7f0bf8039650 2024-04-17T05:52:53.585 INFO:teuthology.orchestra.run.smithi103.stdout:Scheduled mon update... 2024-04-17T05:52:53.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:52:53.590+0000 7f0c15e88700 1 -- 172.21.15.103:0/832606683 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f0bf8039650 msgr2=0x7f0bf803bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:52:53.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:52:53.590+0000 7f0c15e88700 1 --2- 172.21.15.103:0/832606683 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f0bf8039650 0x7f0bf803bb10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0c04006fd0 tx=0x7f0c04006e40 comp rx=0 tx=0).stop 2024-04-17T05:52:53.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:52:53.590+0000 7f0c15e88700 1 -- 172.21.15.103:0/832606683 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c10108560 msgr2=0x7f0c1019a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:52:53.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:52:53.590+0000 7f0c15e88700 1 --2- 172.21.15.103:0/832606683 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c10108560 0x7f0c1019a100 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0c00005850 tx=0x7f0c00005030 comp rx=0 tx=0).stop 2024-04-17T05:52:53.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:52:53.590+0000 7f0c15e88700 1 -- 172.21.15.103:0/832606683 shutdown_connections 2024-04-17T05:52:53.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:52:53.590+0000 7f0c15e88700 1 --2- 172.21.15.103:0/832606683 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f0bf8039650 0x7f0bf803bb10 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:52:53.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:52:53.590+0000 7f0c15e88700 1 --2- 172.21.15.103:0/832606683 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c10108560 0x7f0c1019a100 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:52:53.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:52:53.590+0000 7f0c15e88700 1 -- 172.21.15.103:0/832606683 >> 172.21.15.103:0/832606683 conn(0x7f0c10077ad0 msgr2=0x7f0c1010aaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:52:53.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:52:53.590+0000 7f0c15e88700 1 -- 172.21.15.103:0/832606683 shutdown_connections 2024-04-17T05:52:53.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:52:53.590+0000 7f0c15e88700 1 -- 172.21.15.103:0/832606683 wait complete. 2024-04-17T05:52:53.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:53 smithi047 ceph-mon[32223]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T05:52:53.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:53 smithi047 ceph-mon[32223]: Updating smithi103:/etc/ceph/ceph.client.admin.keyring 2024-04-17T05:52:53.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:53 smithi047 ceph-mon[32223]: Updating smithi103:/var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/config/ceph.client.admin.keyring 2024-04-17T05:52:53.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:53 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:52:53.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:53 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:52:53.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:53 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:52:54.182 DEBUG:teuthology.orchestra.run.smithi098:mon.b> sudo journalctl -f -n 0 -u ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@mon.b.service 2024-04-17T05:52:54.185 DEBUG:teuthology.orchestra.run.smithi103:mon.c> sudo journalctl -f -n 0 -u ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@mon.c.service 2024-04-17T05:52:54.187 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-17T05:52:54.187 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 565db2d8-fc7e-11ee-bc91-c7b262605968 -- ceph mon dump -f json 2024-04-17T05:52:54.812 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/config/ceph.conf 2024-04-17T05:52:54.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:54 smithi047 ceph-mon[32223]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi047:172.21.15.47=a;smithi098:172.21.15.98=b;smithi103:172.21.15.103=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T05:52:54.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:54 smithi047 ceph-mon[32223]: Saving service mon spec with placement smithi047:172.21.15.47=a;smithi098:172.21.15.98=b;smithi103:172.21.15.103=c;count:3 2024-04-17T05:52:54.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:54 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:52:54.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:54 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T05:52:54.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:54 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:52:54.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:54 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T05:52:54.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:54 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:52:54.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:54 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-17T05:52:54.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:54 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:52:54.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:54 smithi047 ceph-mon[32223]: Deploying daemon mon.c on smithi103 2024-04-17T05:52:55.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:55 smithi047 ceph-mon[32223]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T05:52:57.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:52:57.886+0000 7f3325059700 1 -- 172.21.15.103:0/64444250 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33201015c0 msgr2=0x7f33201019a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:52:57.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:52:57.886+0000 7f3325059700 1 --2- 172.21.15.103:0/64444250 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33201015c0 0x7f33201019a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3314009910 tx=0x7f3314009c20 comp rx=0 tx=0).stop 2024-04-17T05:52:57.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:52:57.886+0000 7f3325059700 1 -- 172.21.15.103:0/64444250 shutdown_connections 2024-04-17T05:52:57.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:52:57.886+0000 7f3325059700 1 --2- 172.21.15.103:0/64444250 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33201015c0 0x7f33201019a0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:52:57.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:52:57.886+0000 7f3325059700 1 -- 172.21.15.103:0/64444250 >> 172.21.15.103:0/64444250 conn(0x7f33200fce70 msgr2=0x7f33200ff290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:52:57.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:52:57.886+0000 7f3325059700 1 -- 172.21.15.103:0/64444250 shutdown_connections 2024-04-17T05:52:57.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:52:57.886+0000 7f3325059700 1 -- 172.21.15.103:0/64444250 wait complete. 2024-04-17T05:52:57.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:52:57.886+0000 7f3325059700 1 Processor -- start 2024-04-17T05:52:57.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:52:57.887+0000 7f3325059700 1 -- start start 2024-04-17T05:52:57.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:52:57.887+0000 7f3325059700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33201015c0 0x7f3320197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:52:57.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:52:57.887+0000 7f3325059700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f332010b990 con 0x7f33201015c0 2024-04-17T05:52:57.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:52:57.887+0000 7f331ed9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33201015c0 0x7f3320197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:52:57.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:52:57.887+0000 7f331ed9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33201015c0 0x7f3320197fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.103:38216/0 (socket says 172.21.15.103:38216) 2024-04-17T05:52:57.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:52:57.887+0000 7f331ed9d700 1 -- 172.21.15.103:0/525476473 learned_addr learned my addr 172.21.15.103:0/525476473 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-04-17T05:52:57.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:52:57.887+0000 7f331ed9d700 1 -- 172.21.15.103:0/525476473 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3320198520 con 0x7f33201015c0 2024-04-17T05:52:57.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:52:57.888+0000 7f331ed9d700 1 --2- 172.21.15.103:0/525476473 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33201015c0 0x7f3320197fe0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3314005850 tx=0x7f3314005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:52:57.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:52:57.888+0000 7f3307fff700 1 -- 172.21.15.103:0/525476473 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3314026070 con 0x7f33201015c0 2024-04-17T05:52:57.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:52:57.888+0000 7f3325059700 1 -- 172.21.15.103:0/525476473 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3320198810 con 0x7f33201015c0 2024-04-17T05:52:57.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:52:57.888+0000 7f3307fff700 1 -- 172.21.15.103:0/525476473 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f331400bbd0 con 0x7f33201015c0 2024-04-17T05:52:57.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:52:57.888+0000 7f3325059700 1 -- 172.21.15.103:0/525476473 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f332019c8b0 con 0x7f33201015c0 2024-04-17T05:52:57.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:52:57.888+0000 7f3307fff700 1 -- 172.21.15.103:0/525476473 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3314003970 con 0x7f33201015c0 2024-04-17T05:52:57.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:52:57.889+0000 7f3307fff700 1 -- 172.21.15.103:0/525476473 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f3314029070 con 0x7f33201015c0 2024-04-17T05:52:57.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:52:57.889+0000 7f3325059700 1 -- 172.21.15.103:0/525476473 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3320102aa0 con 0x7f33201015c0 2024-04-17T05:52:57.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:52:57 smithi047 ceph-mon[32223]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T05:52:57.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:52:57.889+0000 7f3307fff700 1 --2- 172.21.15.103:0/525476473 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f3308039650 0x7f330803bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:52:57.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:52:57.889+0000 7f3307fff700 1 -- 172.21.15.103:0/525476473 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f331401e070 con 0x7f33201015c0 2024-04-17T05:52:57.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:52:57.889+0000 7f331e59c700 1 --2- 172.21.15.103:0/525476473 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f3308039650 0x7f330803bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:52:57.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:52:57.890+0000 7f331e59c700 1 --2- 172.21.15.103:0/525476473 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f3308039650 0x7f330803bb10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f3310006fd0 tx=0x7f3310006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:52:57.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:52:57.895+0000 7f3307fff700 1 -- 172.21.15.103:0/525476473 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f3314013070 con 0x7f33201015c0 2024-04-17T05:52:58.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:52:58.186+0000 7f3325059700 1 -- 172.21.15.103:0/525476473 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f33200640b0 con 0x7f33201015c0 2024-04-17T05:52:58.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:52:58.187+0000 7f3307fff700 1 -- 172.21.15.103:0/525476473 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7f3314030090 con 0x7f33201015c0 2024-04-17T05:52:58.186 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-04-17T05:52:58.186 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":1,"fsid":"565db2d8-fc7e-11ee-bc91-c7b262605968","modified":"2024-04-17T05:51:16.462437Z","created":"2024-04-17T05:51:16.462437Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3300","nonce":0},{"type":"v1","addr":"172.21.15.47:6789","nonce":0}]},"addr":"172.21.15.47:6789/0","public_addr":"172.21.15.47:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-17T05:52:58.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:52:58.189+0000 7f3325059700 1 -- 172.21.15.103:0/525476473 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f3308039650 msgr2=0x7f330803bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:52:58.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:52:58.189+0000 7f3325059700 1 --2- 172.21.15.103:0/525476473 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f3308039650 0x7f330803bb10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f3310006fd0 tx=0x7f3310006e40 comp rx=0 tx=0).stop 2024-04-17T05:52:58.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:52:58.189+0000 7f3325059700 1 -- 172.21.15.103:0/525476473 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33201015c0 msgr2=0x7f3320197fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:52:58.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:52:58.189+0000 7f3325059700 1 --2- 172.21.15.103:0/525476473 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33201015c0 0x7f3320197fe0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3314005850 tx=0x7f3314005030 comp rx=0 tx=0).stop 2024-04-17T05:52:58.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:52:58.189+0000 7f3325059700 1 -- 172.21.15.103:0/525476473 shutdown_connections 2024-04-17T05:52:58.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:52:58.189+0000 7f3325059700 1 --2- 172.21.15.103:0/525476473 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f3308039650 0x7f330803bb10 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:52:58.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:52:58.189+0000 7f3325059700 1 --2- 172.21.15.103:0/525476473 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33201015c0 0x7f3320197fe0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:52:58.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:52:58.189+0000 7f3325059700 1 -- 172.21.15.103:0/525476473 >> 172.21.15.103:0/525476473 conn(0x7f33200fce70 msgr2=0x7f332006a8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:52:58.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:52:58.189+0000 7f3325059700 1 -- 172.21.15.103:0/525476473 shutdown_connections 2024-04-17T05:52:58.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:52:58.189+0000 7f3325059700 1 -- 172.21.15.103:0/525476473 wait complete. 2024-04-17T05:52:58.189 INFO:teuthology.orchestra.run.smithi103.stderr:dumped monmap epoch 1 2024-04-17T05:52:58.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:52:58 smithi103 systemd[1]: Started Ceph mon.c for 565db2d8-fc7e-11ee-bc91-c7b262605968. 2024-04-17T05:52:59.935 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-17T05:52:59.936 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 565db2d8-fc7e-11ee-bc91-c7b262605968 -- ceph mon dump -f json 2024-04-17T05:53:00.182 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/mon.c/config 2024-04-17T05:53:03.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:03 smithi047 ceph-mon[32223]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T05:53:03.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:03 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-17T05:53:03.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:03 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-17T05:53:03.640 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:03 smithi047 ceph-mon[32223]: mon.a calling monitor election 2024-04-17T05:53:03.640 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:03 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-17T05:53:03.640 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:03 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-17T05:53:03.640 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:03 smithi047 ceph-mon[32223]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T05:53:03.640 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:03 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-17T05:53:03.640 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:03 smithi047 ceph-mon[32223]: mon.c calling monitor election 2024-04-17T05:53:03.640 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:03 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-17T05:53:03.640 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:03 smithi047 ceph-mon[32223]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T05:53:03.640 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:03 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-17T05:53:03.640 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:03 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-17T05:53:03.640 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:03 smithi047 ceph-mon[32223]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-17T05:53:03.641 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:03 smithi047 ceph-mon[32223]: monmap epoch 2 2024-04-17T05:53:03.641 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:03 smithi047 ceph-mon[32223]: fsid 565db2d8-fc7e-11ee-bc91-c7b262605968 2024-04-17T05:53:03.641 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:03 smithi047 ceph-mon[32223]: last_changed 2024-04-17T05:52:58.281719+0000 2024-04-17T05:53:03.641 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:03 smithi047 ceph-mon[32223]: created 2024-04-17T05:51:16.462437+0000 2024-04-17T05:53:03.641 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:03 smithi047 ceph-mon[32223]: min_mon_release 19 (squid) 2024-04-17T05:53:03.641 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:03 smithi047 ceph-mon[32223]: election_strategy: 1 2024-04-17T05:53:03.641 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:03 smithi047 ceph-mon[32223]: 0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-17T05:53:03.641 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:03 smithi047 ceph-mon[32223]: 1: [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] mon.c 2024-04-17T05:53:03.641 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:03 smithi047 ceph-mon[32223]: fsmap 2024-04-17T05:53:03.641 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:03 smithi047 ceph-mon[32223]: osdmap e4: 0 total, 0 up, 0 in 2024-04-17T05:53:03.641 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:03 smithi047 ceph-mon[32223]: mgrmap e13: x(active, since 49s) 2024-04-17T05:53:03.641 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:03 smithi047 ceph-mon[32223]: overall HEALTH_OK 2024-04-17T05:53:03.641 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:03 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:03.642 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:03 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:03.642 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:03 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:03.642 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:03 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-17T05:53:03.642 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:03 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:53:03.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:03 smithi103 ceph-mon[36243]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T05:53:03.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:03 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-17T05:53:03.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:03 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-17T05:53:03.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:03 smithi103 ceph-mon[36243]: mon.a calling monitor election 2024-04-17T05:53:03.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:03 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-17T05:53:03.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:03 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-17T05:53:03.785 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:03 smithi103 ceph-mon[36243]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T05:53:03.785 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:03 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-17T05:53:03.785 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:03 smithi103 ceph-mon[36243]: mon.c calling monitor election 2024-04-17T05:53:03.785 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:03 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-17T05:53:03.785 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:03 smithi103 ceph-mon[36243]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T05:53:03.785 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:03 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-17T05:53:03.785 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:03 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-17T05:53:03.785 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:03 smithi103 ceph-mon[36243]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-17T05:53:03.785 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:03 smithi103 ceph-mon[36243]: monmap epoch 2 2024-04-17T05:53:03.785 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:03 smithi103 ceph-mon[36243]: fsid 565db2d8-fc7e-11ee-bc91-c7b262605968 2024-04-17T05:53:03.785 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:03 smithi103 ceph-mon[36243]: last_changed 2024-04-17T05:52:58.281719+0000 2024-04-17T05:53:03.785 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:03 smithi103 ceph-mon[36243]: created 2024-04-17T05:51:16.462437+0000 2024-04-17T05:53:03.785 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:03 smithi103 ceph-mon[36243]: min_mon_release 19 (squid) 2024-04-17T05:53:03.786 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:03 smithi103 ceph-mon[36243]: election_strategy: 1 2024-04-17T05:53:03.786 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:03 smithi103 ceph-mon[36243]: 0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-17T05:53:03.786 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:03 smithi103 ceph-mon[36243]: 1: [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] mon.c 2024-04-17T05:53:03.786 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:03 smithi103 ceph-mon[36243]: fsmap 2024-04-17T05:53:03.786 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:03 smithi103 ceph-mon[36243]: osdmap e4: 0 total, 0 up, 0 in 2024-04-17T05:53:03.786 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:03 smithi103 ceph-mon[36243]: mgrmap e13: x(active, since 49s) 2024-04-17T05:53:03.786 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:03 smithi103 ceph-mon[36243]: overall HEALTH_OK 2024-04-17T05:53:03.786 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:03 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:03.786 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:03 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:03.786 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:03 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:03.786 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:03 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-17T05:53:03.786 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:03 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:53:04.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:04 smithi047 ceph-mon[32223]: Deploying daemon mon.b on smithi098 2024-04-17T05:53:04.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:04 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-17T05:53:04.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:04.702+0000 7fbfd2750700 1 -- 172.21.15.103:0/1152846068 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbfb40035c0 msgr2=0x7fbfb4005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:53:04.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:04.702+0000 7fbfd2750700 1 --2- 172.21.15.103:0/1152846068 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbfb40035c0 0x7fbfb4005a50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc101390 tx=0x7fbfbc00feb0 comp rx=0 tx=0).stop 2024-04-17T05:53:04.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:04.702+0000 7fbfd2750700 1 -- 172.21.15.103:0/1152846068 shutdown_connections 2024-04-17T05:53:04.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:04.703+0000 7fbfd2750700 1 --2- 172.21.15.103:0/1152846068 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbfb40035c0 0x7fbfb4005a50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:53:04.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:04.703+0000 7fbfd2750700 1 -- 172.21.15.103:0/1152846068 >> 172.21.15.103:0/1152846068 conn(0x7fbfcc0fcf90 msgr2=0x7fbfcc0ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:53:04.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:04.703+0000 7fbfd2750700 1 -- 172.21.15.103:0/1152846068 shutdown_connections 2024-04-17T05:53:04.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:04.703+0000 7fbfd2750700 1 -- 172.21.15.103:0/1152846068 wait complete. 2024-04-17T05:53:04.702 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:04 smithi103 ceph-mon[36243]: Deploying daemon mon.b on smithi098 2024-04-17T05:53:04.702 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:04 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-17T05:53:04.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:04.703+0000 7fbfd2750700 1 Processor -- start 2024-04-17T05:53:04.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:04.703+0000 7fbfd2750700 1 -- start start 2024-04-17T05:53:04.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:04.704+0000 7fbfd2750700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbfb40035c0 0x7fbfcc1030c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:53:04.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:04.704+0000 7fbfd2750700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbfcc103600 0x7fbfcc103a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:53:04.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:04.704+0000 7fbfd2750700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fbfcc109500 con 0x7fbfb40035c0 2024-04-17T05:53:04.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:04.704+0000 7fbfd2750700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fbfcc109380 con 0x7fbfcc103600 2024-04-17T05:53:04.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:04.704+0000 7fbfcb7fe700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbfcc103600 0x7fbfcc103a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:53:04.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:04.704+0000 7fbfcb7fe700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbfcc103600 0x7fbfcc103a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60508/0 (socket says 172.21.15.103:60508) 2024-04-17T05:53:04.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:04.704+0000 7fbfcb7fe700 1 -- 172.21.15.103:0/2354423693 learned_addr learned my addr 172.21.15.103:0/2354423693 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-04-17T05:53:04.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:04.704+0000 7fbfcb7fe700 1 -- 172.21.15.103:0/2354423693 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbfcc103600 msgr2=0x7fbfcc103a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-17T05:53:04.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:04.704+0000 7fbfcb7fe700 1 -- 172.21.15.103:0/2354423693 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbfcc103600 msgr2=0x7fbfcc103a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-17T05:53:04.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:04.704+0000 7fbfcb7fe700 1 --2- 172.21.15.103:0/2354423693 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbfcc103600 0x7fbfcc103a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-17T05:53:04.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:04.704+0000 7fbfcbfff700 1 --2- 172.21.15.103:0/2354423693 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbfb40035c0 0x7fbfcc1030c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:53:04.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:04.704+0000 7fbfcb7fe700 1 --2- 172.21.15.103:0/2354423693 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbfcc103600 0x7fbfcc103a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-17T05:53:04.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:04.705+0000 7fbfcbfff700 1 -- 172.21.15.103:0/2354423693 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbfcc103600 msgr2=0x7fbfcc103a80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-17T05:53:04.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:04.705+0000 7fbfcbfff700 1 --2- 172.21.15.103:0/2354423693 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbfcc103600 0x7fbfcc103a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:53:04.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:04.705+0000 7fbfcbfff700 1 -- 172.21.15.103:0/2354423693 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfcc104fe0 con 0x7fbfb40035c0 2024-04-17T05:53:04.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:04.705+0000 7fbfcbfff700 1 --2- 172.21.15.103:0/2354423693 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbfb40035c0 0x7fbfcc1030c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc010010 tx=0x7fbfbc0046f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:53:04.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:04.705+0000 7fbfc97fa700 1 -- 172.21.15.103:0/2354423693 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fbfbc013b00 con 0x7fbfb40035c0 2024-04-17T05:53:04.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:04.705+0000 7fbfc97fa700 1 -- 172.21.15.103:0/2354423693 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbfbc004510 con 0x7fbfb40035c0 2024-04-17T05:53:04.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:04.705+0000 7fbfd2750700 1 -- 172.21.15.103:0/2354423693 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfcc1a4380 con 0x7fbfb40035c0 2024-04-17T05:53:04.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:04.705+0000 7fbfc97fa700 1 -- 172.21.15.103:0/2354423693 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fbfbc003d80 con 0x7fbfb40035c0 2024-04-17T05:53:04.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:04.705+0000 7fbfd2750700 1 -- 172.21.15.103:0/2354423693 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfcc1a48c0 con 0x7fbfb40035c0 2024-04-17T05:53:04.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:04.706+0000 7fbfc97fa700 1 -- 172.21.15.103:0/2354423693 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fbfbc023070 con 0x7fbfb40035c0 2024-04-17T05:53:04.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:04.707+0000 7fbfc97fa700 1 --2- 172.21.15.103:0/2354423693 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fbfb8039440 0x7fbfb803b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:53:04.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:04.707+0000 7fbfc97fa700 1 -- 172.21.15.103:0/2354423693 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fbfbc020050 con 0x7fbfb40035c0 2024-04-17T05:53:04.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:04.707+0000 7fbfcb7fe700 1 --2- 172.21.15.103:0/2354423693 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fbfb8039440 0x7fbfb803b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:53:04.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:04.707+0000 7fbfd2750700 1 -- 172.21.15.103:0/2354423693 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfcc102ba0 con 0x7fbfb40035c0 2024-04-17T05:53:04.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:04.707+0000 7fbfcb7fe700 1 --2- 172.21.15.103:0/2354423693 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fbfb8039440 0x7fbfb803b900 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fbfc0009180 tx=0x7fbfc0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:53:04.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:04.713+0000 7fbfc97fa700 1 -- 172.21.15.103:0/2354423693 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fbfbc028020 con 0x7fbfb40035c0 2024-04-17T05:53:05.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:05.000+0000 7fbfd2750700 1 -- 172.21.15.103:0/2354423693 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fbfcc104770 con 0x7fbfb40035c0 2024-04-17T05:53:05.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:05.000+0000 7fbfc97fa700 1 -- 172.21.15.103:0/2354423693 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7fbfbc02adc0 con 0x7fbfb40035c0 2024-04-17T05:53:05.003 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-04-17T05:53:05.003 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":2,"fsid":"565db2d8-fc7e-11ee-bc91-c7b262605968","modified":"2024-04-17T05:52:58.281719Z","created":"2024-04-17T05:51:16.462437Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3300","nonce":0},{"type":"v1","addr":"172.21.15.47:6789","nonce":0}]},"addr":"172.21.15.47:6789/0","public_addr":"172.21.15.47:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:3300","nonce":0},{"type":"v1","addr":"172.21.15.103:6789","nonce":0}]},"addr":"172.21.15.103:6789/0","public_addr":"172.21.15.103:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-17T05:53:05.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:05.003+0000 7fbfd2750700 1 -- 172.21.15.103:0/2354423693 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fbfb8039440 msgr2=0x7fbfb803b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:53:05.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:05.003+0000 7fbfd2750700 1 --2- 172.21.15.103:0/2354423693 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fbfb8039440 0x7fbfb803b900 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fbfc0009180 tx=0x7fbfc0008040 comp rx=0 tx=0).stop 2024-04-17T05:53:05.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:05.003+0000 7fbfd2750700 1 -- 172.21.15.103:0/2354423693 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbfb40035c0 msgr2=0x7fbfcc1030c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:53:05.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:05.003+0000 7fbfd2750700 1 --2- 172.21.15.103:0/2354423693 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbfb40035c0 0x7fbfcc1030c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc010010 tx=0x7fbfbc0046f0 comp rx=0 tx=0).stop 2024-04-17T05:53:05.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:05.003+0000 7fbfd2750700 1 -- 172.21.15.103:0/2354423693 shutdown_connections 2024-04-17T05:53:05.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:05.003+0000 7fbfd2750700 1 --2- 172.21.15.103:0/2354423693 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fbfb8039440 0x7fbfb803b900 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:53:05.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:05.003+0000 7fbfd2750700 1 --2- 172.21.15.103:0/2354423693 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbfb40035c0 0x7fbfcc1030c0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:53:05.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:05.003+0000 7fbfd2750700 1 --2- 172.21.15.103:0/2354423693 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbfcc103600 0x7fbfcc103a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:53:05.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:05.003+0000 7fbfd2750700 1 -- 172.21.15.103:0/2354423693 >> 172.21.15.103:0/2354423693 conn(0x7fbfcc0fcf90 msgr2=0x7fbfcc06b110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:53:05.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:05.003+0000 7fbfd2750700 1 -- 172.21.15.103:0/2354423693 shutdown_connections 2024-04-17T05:53:05.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:05.003+0000 7fbfd2750700 1 -- 172.21.15.103:0/2354423693 wait complete. 2024-04-17T05:53:05.005 INFO:teuthology.orchestra.run.smithi103.stderr:dumped monmap epoch 2 2024-04-17T05:53:05.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:05 smithi047 ceph-mon[32223]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T05:53:05.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:05 smithi047 ceph-mon[32223]: from='client.? 172.21.15.103:0/2354423693' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-17T05:53:05.639 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 05:53:05 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T05:53:05.280+0000 7f29d5e2a700 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-17T05:53:05.783 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:05 smithi103 ceph-mon[36243]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T05:53:05.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:05 smithi103 ceph-mon[36243]: from='client.? 172.21.15.103:0/2354423693' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-17T05:53:06.900 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-17T05:53:06.900 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 565db2d8-fc7e-11ee-bc91-c7b262605968 -- ceph mon dump -f json 2024-04-17T05:53:07.144 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/mon.c/config 2024-04-17T05:53:07.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:07 smithi098 systemd[1]: Started Ceph mon.b for 565db2d8-fc7e-11ee-bc91-c7b262605968. 2024-04-17T05:53:12.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:12.357+0000 7fc3d95f1700 1 -- 172.21.15.103:0/3063167261 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc3bc0035c0 msgr2=0x7fc3bc005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:53:12.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:12.357+0000 7fc3d95f1700 1 --2- 172.21.15.103:0/3063167261 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc3bc0035c0 0x7fc3bc005a50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc3d4074d20 tx=0x7fc3c4014ec0 comp rx=0 tx=0).stop 2024-04-17T05:53:12.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:12.357+0000 7fc3d95f1700 1 -- 172.21.15.103:0/3063167261 shutdown_connections 2024-04-17T05:53:12.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:12.357+0000 7fc3d95f1700 1 --2- 172.21.15.103:0/3063167261 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc3bc0035c0 0x7fc3bc005a50 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:53:12.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:12.357+0000 7fc3d95f1700 1 -- 172.21.15.103:0/3063167261 >> 172.21.15.103:0/3063167261 conn(0x7fc3d40fe060 msgr2=0x7fc3d4100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:53:12.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:12.357+0000 7fc3d95f1700 1 -- 172.21.15.103:0/3063167261 shutdown_connections 2024-04-17T05:53:12.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:12.357+0000 7fc3d95f1700 1 -- 172.21.15.103:0/3063167261 wait complete. 2024-04-17T05:53:12.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:12.358+0000 7fc3d95f1700 1 Processor -- start 2024-04-17T05:53:12.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:12.359+0000 7fc3d95f1700 1 -- start start 2024-04-17T05:53:12.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:12.359+0000 7fc3d95f1700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc3bc0035c0 0x7fc3d41a45a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:53:12.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:12.359+0000 7fc3d95f1700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc3d41a4ae0 0x7fc3d419e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:53:12.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:12.359+0000 7fc3d95f1700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc3d419ebb0 0x7fc3d419f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:53:12.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:12.359+0000 7fc3d95f1700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fc3d4110bc0 con 0x7fc3bc0035c0 2024-04-17T05:53:12.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:12.359+0000 7fc3d95f1700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc3d4110a40 con 0x7fc3d41a4ae0 2024-04-17T05:53:12.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:12.359+0000 7fc3d95f1700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc3d4110d40 con 0x7fc3d419ebb0 2024-04-17T05:53:12.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:12.359+0000 7fc3d2ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc3bc0035c0 0x7fc3d41a45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:53:12.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:12.359+0000 7fc3d37fe700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc3d419ebb0 0x7fc3d419f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:53:12.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:12.359+0000 7fc3d37fe700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc3d419ebb0 0x7fc3d419f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60524/0 (socket says 172.21.15.103:60524) 2024-04-17T05:53:12.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:12.359+0000 7fc3d2ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc3bc0035c0 0x7fc3d41a45a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.103:56392/0 (socket says 172.21.15.103:56392) 2024-04-17T05:53:12.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:12.359+0000 7fc3d27fc700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc3d41a4ae0 0x7fc3d419e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:53:12.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:12.359+0000 7fc3d37fe700 1 -- 172.21.15.103:0/2247768442 learned_addr learned my addr 172.21.15.103:0/2247768442 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-04-17T05:53:12.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:12.359+0000 7fc3d27fc700 1 -- 172.21.15.103:0/2247768442 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc3d41a4ae0 msgr2=0x7fc3d419e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-17T05:53:12.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:12.359+0000 7fc3d27fc700 1 -- 172.21.15.103:0/2247768442 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc3d41a4ae0 msgr2=0x7fc3d419e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-17T05:53:12.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:12.359+0000 7fc3d37fe700 1 -- 172.21.15.103:0/2247768442 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc3d419ebb0 msgr2=0x7fc3d419f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-04-17T05:53:12.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:12.359+0000 7fc3d27fc700 1 --2- 172.21.15.103:0/2247768442 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc3d41a4ae0 0x7fc3d419e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-17T05:53:12.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:12.359+0000 7fc3d37fe700 1 -- 172.21.15.103:0/2247768442 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc3d419ebb0 msgr2=0x7fc3d419f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-17T05:53:12.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:12.359+0000 7fc3d37fe700 1 --2- 172.21.15.103:0/2247768442 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc3d419ebb0 0x7fc3d419f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-17T05:53:12.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:12.359+0000 7fc3d27fc700 1 --2- 172.21.15.103:0/2247768442 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc3d41a4ae0 0x7fc3d419e670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-17T05:53:12.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:12.359+0000 7fc3d37fe700 1 --2- 172.21.15.103:0/2247768442 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc3d419ebb0 0x7fc3d419f060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-17T05:53:12.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:12.359+0000 7fc3d2ffd700 1 -- 172.21.15.103:0/2247768442 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc3d419ebb0 msgr2=0x7fc3d419f060 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-17T05:53:12.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:12.359+0000 7fc3d2ffd700 1 --2- 172.21.15.103:0/2247768442 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc3d419ebb0 0x7fc3d419f060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:53:12.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:12.359+0000 7fc3d2ffd700 1 -- 172.21.15.103:0/2247768442 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc3d41a4ae0 msgr2=0x7fc3d419e670 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-17T05:53:12.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:12.359+0000 7fc3d2ffd700 1 --2- 172.21.15.103:0/2247768442 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc3d41a4ae0 0x7fc3d419e670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:53:12.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:12.359+0000 7fc3d2ffd700 1 -- 172.21.15.103:0/2247768442 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3d41ab350 con 0x7fc3bc0035c0 2024-04-17T05:53:12.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:12.360+0000 7fc3d2ffd700 1 --2- 172.21.15.103:0/2247768442 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc3bc0035c0 0x7fc3d41a45a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc3c401d000 tx=0x7fc3c4005160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:53:12.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:12.360+0000 7fc3bbfff700 1 -- 172.21.15.103:0/2247768442 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3c4004900 con 0x7fc3bc0035c0 2024-04-17T05:53:12.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:12.360+0000 7fc3bbfff700 1 -- 172.21.15.103:0/2247768442 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3c4004aa0 con 0x7fc3bc0035c0 2024-04-17T05:53:12.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:12.360+0000 7fc3d95f1700 1 -- 172.21.15.103:0/2247768442 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3d41ab580 con 0x7fc3bc0035c0 2024-04-17T05:53:12.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:12.360+0000 7fc3bbfff700 1 -- 172.21.15.103:0/2247768442 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3c401ee00 con 0x7fc3bc0035c0 2024-04-17T05:53:12.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:12.360+0000 7fc3d95f1700 1 -- 172.21.15.103:0/2247768442 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3d41abb10 con 0x7fc3bc0035c0 2024-04-17T05:53:12.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:12.361+0000 7fc3bbfff700 1 -- 172.21.15.103:0/2247768442 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fc3c4012450 con 0x7fc3bc0035c0 2024-04-17T05:53:12.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:12.361+0000 7fc3bbfff700 1 --2- 172.21.15.103:0/2247768442 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fc3c0039ae0 0x7fc3c003bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:53:12.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:12.361+0000 7fc3d95f1700 1 -- 172.21.15.103:0/2247768442 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3d4076530 con 0x7fc3bc0035c0 2024-04-17T05:53:12.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:12.361+0000 7fc3bbfff700 1 -- 172.21.15.103:0/2247768442 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fc3c401e730 con 0x7fc3bc0035c0 2024-04-17T05:53:12.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:12.361+0000 7fc3d27fc700 1 --2- 172.21.15.103:0/2247768442 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fc3c0039ae0 0x7fc3c003bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:53:12.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:12.362+0000 7fc3d27fc700 1 --2- 172.21.15.103:0/2247768442 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fc3c0039ae0 0x7fc3c003bfa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8009850 tx=0x7fc3c8008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:53:12.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:12.367+0000 7fc3bbfff700 1 -- 172.21.15.103:0/2247768442 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fc3c401e930 con 0x7fc3bc0035c0 2024-04-17T05:53:12.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:12 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-17T05:53:12.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:12 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-17T05:53:12.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:12 smithi098 ceph-mon[36516]: mon.a calling monitor election 2024-04-17T05:53:12.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:12 smithi098 ceph-mon[36516]: mon.c calling monitor election 2024-04-17T05:53:12.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:12 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-17T05:53:12.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:12 smithi098 ceph-mon[36516]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T05:53:12.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:12 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-17T05:53:12.634 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:12 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-17T05:53:12.634 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:12 smithi098 ceph-mon[36516]: mon.b calling monitor election 2024-04-17T05:53:12.634 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:12 smithi098 ceph-mon[36516]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T05:53:12.634 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:12 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-17T05:53:12.634 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:12 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-17T05:53:12.634 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:12 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-17T05:53:12.634 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:12 smithi098 ceph-mon[36516]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-17T05:53:12.634 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:12 smithi098 ceph-mon[36516]: monmap epoch 3 2024-04-17T05:53:12.634 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:12 smithi098 ceph-mon[36516]: fsid 565db2d8-fc7e-11ee-bc91-c7b262605968 2024-04-17T05:53:12.634 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:12 smithi098 ceph-mon[36516]: last_changed 2024-04-17T05:53:07.213043+0000 2024-04-17T05:53:12.634 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:12 smithi098 ceph-mon[36516]: created 2024-04-17T05:51:16.462437+0000 2024-04-17T05:53:12.634 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:12 smithi098 ceph-mon[36516]: min_mon_release 19 (squid) 2024-04-17T05:53:12.634 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:12 smithi098 ceph-mon[36516]: election_strategy: 1 2024-04-17T05:53:12.635 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:12 smithi098 ceph-mon[36516]: 0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-17T05:53:12.635 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:12 smithi098 ceph-mon[36516]: 1: [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] mon.c 2024-04-17T05:53:12.635 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:12 smithi098 ceph-mon[36516]: 2: [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] mon.b 2024-04-17T05:53:12.635 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:12 smithi098 ceph-mon[36516]: fsmap 2024-04-17T05:53:12.635 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:12 smithi098 ceph-mon[36516]: osdmap e4: 0 total, 0 up, 0 in 2024-04-17T05:53:12.635 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:12 smithi098 ceph-mon[36516]: mgrmap e13: x(active, since 58s) 2024-04-17T05:53:12.635 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:12 smithi098 ceph-mon[36516]: overall HEALTH_OK 2024-04-17T05:53:12.635 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:12 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:12.635 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:12 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:12.635 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:12 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:12.635 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:12 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:12.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:12 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-17T05:53:12.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:12 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-17T05:53:12.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:12 smithi047 ceph-mon[32223]: mon.a calling monitor election 2024-04-17T05:53:12.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:12 smithi047 ceph-mon[32223]: mon.c calling monitor election 2024-04-17T05:53:12.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:12 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-17T05:53:12.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:12 smithi047 ceph-mon[32223]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T05:53:12.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:12 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-17T05:53:12.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:12 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-17T05:53:12.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:12 smithi047 ceph-mon[32223]: mon.b calling monitor election 2024-04-17T05:53:12.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:12 smithi047 ceph-mon[32223]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T05:53:12.640 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:12 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-17T05:53:12.640 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:12 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-17T05:53:12.640 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:12 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-17T05:53:12.640 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:12 smithi047 ceph-mon[32223]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-17T05:53:12.640 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:12 smithi047 ceph-mon[32223]: monmap epoch 3 2024-04-17T05:53:12.640 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:12 smithi047 ceph-mon[32223]: fsid 565db2d8-fc7e-11ee-bc91-c7b262605968 2024-04-17T05:53:12.640 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:12 smithi047 ceph-mon[32223]: last_changed 2024-04-17T05:53:07.213043+0000 2024-04-17T05:53:12.640 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:12 smithi047 ceph-mon[32223]: created 2024-04-17T05:51:16.462437+0000 2024-04-17T05:53:12.640 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:12 smithi047 ceph-mon[32223]: min_mon_release 19 (squid) 2024-04-17T05:53:12.640 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:12 smithi047 ceph-mon[32223]: election_strategy: 1 2024-04-17T05:53:12.640 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:12 smithi047 ceph-mon[32223]: 0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-17T05:53:12.640 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:12 smithi047 ceph-mon[32223]: 1: [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] mon.c 2024-04-17T05:53:12.640 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:12 smithi047 ceph-mon[32223]: 2: [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] mon.b 2024-04-17T05:53:12.641 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:12 smithi047 ceph-mon[32223]: fsmap 2024-04-17T05:53:12.641 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:12 smithi047 ceph-mon[32223]: osdmap e4: 0 total, 0 up, 0 in 2024-04-17T05:53:12.641 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:12 smithi047 ceph-mon[32223]: mgrmap e13: x(active, since 58s) 2024-04-17T05:53:12.641 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:12 smithi047 ceph-mon[32223]: overall HEALTH_OK 2024-04-17T05:53:12.641 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:12 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:12.641 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:12 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:12.641 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:12 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:12.641 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:12 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:12.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:12.673+0000 7fc3d95f1700 1 -- 172.21.15.103:0/2247768442 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fc3d41abda0 con 0x7fc3bc0035c0 2024-04-17T05:53:12.672 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:12 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-17T05:53:12.672 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:12 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-17T05:53:12.673 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:12 smithi103 ceph-mon[36243]: mon.a calling monitor election 2024-04-17T05:53:12.673 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:12 smithi103 ceph-mon[36243]: mon.c calling monitor election 2024-04-17T05:53:12.673 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:12 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-17T05:53:12.673 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:12 smithi103 ceph-mon[36243]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T05:53:12.673 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:12 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-17T05:53:12.673 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:12 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-17T05:53:12.673 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:12 smithi103 ceph-mon[36243]: mon.b calling monitor election 2024-04-17T05:53:12.673 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:12 smithi103 ceph-mon[36243]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T05:53:12.673 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:12 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-17T05:53:12.673 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:12 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-17T05:53:12.673 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:12 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-17T05:53:12.673 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:12 smithi103 ceph-mon[36243]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-17T05:53:12.673 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:12 smithi103 ceph-mon[36243]: monmap epoch 3 2024-04-17T05:53:12.673 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:12 smithi103 ceph-mon[36243]: fsid 565db2d8-fc7e-11ee-bc91-c7b262605968 2024-04-17T05:53:12.674 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:12 smithi103 ceph-mon[36243]: last_changed 2024-04-17T05:53:07.213043+0000 2024-04-17T05:53:12.674 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:12 smithi103 ceph-mon[36243]: created 2024-04-17T05:51:16.462437+0000 2024-04-17T05:53:12.674 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:12 smithi103 ceph-mon[36243]: min_mon_release 19 (squid) 2024-04-17T05:53:12.674 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:12 smithi103 ceph-mon[36243]: election_strategy: 1 2024-04-17T05:53:12.674 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:12 smithi103 ceph-mon[36243]: 0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-17T05:53:12.674 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:12 smithi103 ceph-mon[36243]: 1: [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] mon.c 2024-04-17T05:53:12.674 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:12 smithi103 ceph-mon[36243]: 2: [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] mon.b 2024-04-17T05:53:12.674 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:12 smithi103 ceph-mon[36243]: fsmap 2024-04-17T05:53:12.674 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:12 smithi103 ceph-mon[36243]: osdmap e4: 0 total, 0 up, 0 in 2024-04-17T05:53:12.674 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:12 smithi103 ceph-mon[36243]: mgrmap e13: x(active, since 58s) 2024-04-17T05:53:12.674 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:12 smithi103 ceph-mon[36243]: overall HEALTH_OK 2024-04-17T05:53:12.674 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:12 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:12.674 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:12 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:12.674 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:12 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:12.674 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:12 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:12.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:12.674+0000 7fc3bbfff700 1 -- 172.21.15.103:0/2247768442 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1281 (secure 0 0 0) 0x7fc3c4037020 con 0x7fc3bc0035c0 2024-04-17T05:53:12.675 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-04-17T05:53:12.675 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":3,"fsid":"565db2d8-fc7e-11ee-bc91-c7b262605968","modified":"2024-04-17T05:53:07.213043Z","created":"2024-04-17T05:51:16.462437Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3300","nonce":0},{"type":"v1","addr":"172.21.15.47:6789","nonce":0}]},"addr":"172.21.15.47:6789/0","public_addr":"172.21.15.47:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:3300","nonce":0},{"type":"v1","addr":"172.21.15.103:6789","nonce":0}]},"addr":"172.21.15.103:6789/0","public_addr":"172.21.15.103:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:3300","nonce":0},{"type":"v1","addr":"172.21.15.98:6789","nonce":0}]},"addr":"172.21.15.98:6789/0","public_addr":"172.21.15.98:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-04-17T05:53:12.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:12.676+0000 7fc3d95f1700 1 -- 172.21.15.103:0/2247768442 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fc3c0039ae0 msgr2=0x7fc3c003bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:53:12.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:12.676+0000 7fc3d95f1700 1 --2- 172.21.15.103:0/2247768442 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fc3c0039ae0 0x7fc3c003bfa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8009850 tx=0x7fc3c8008040 comp rx=0 tx=0).stop 2024-04-17T05:53:12.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:12.676+0000 7fc3d95f1700 1 -- 172.21.15.103:0/2247768442 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc3bc0035c0 msgr2=0x7fc3d41a45a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:53:12.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:12.676+0000 7fc3d95f1700 1 --2- 172.21.15.103:0/2247768442 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc3bc0035c0 0x7fc3d41a45a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc3c401d000 tx=0x7fc3c4005160 comp rx=0 tx=0).stop 2024-04-17T05:53:12.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:12.676+0000 7fc3d95f1700 1 -- 172.21.15.103:0/2247768442 shutdown_connections 2024-04-17T05:53:12.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:12.676+0000 7fc3d95f1700 1 --2- 172.21.15.103:0/2247768442 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fc3c0039ae0 0x7fc3c003bfa0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:53:12.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:12.676+0000 7fc3d95f1700 1 --2- 172.21.15.103:0/2247768442 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc3d419ebb0 0x7fc3d419f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:53:12.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:12.676+0000 7fc3d95f1700 1 --2- 172.21.15.103:0/2247768442 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc3bc0035c0 0x7fc3d41a45a0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:53:12.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:12.676+0000 7fc3d95f1700 1 --2- 172.21.15.103:0/2247768442 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc3d41a4ae0 0x7fc3d419e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:53:12.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:12.676+0000 7fc3d95f1700 1 -- 172.21.15.103:0/2247768442 >> 172.21.15.103:0/2247768442 conn(0x7fc3d40fe060 msgr2=0x7fc3d4197240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:53:12.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:12.676+0000 7fc3d95f1700 1 -- 172.21.15.103:0/2247768442 shutdown_connections 2024-04-17T05:53:12.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:12.676+0000 7fc3d95f1700 1 -- 172.21.15.103:0/2247768442 wait complete. 2024-04-17T05:53:12.677 INFO:teuthology.orchestra.run.smithi103.stderr:dumped monmap epoch 3 2024-04-17T05:53:13.218 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-04-17T05:53:13.219 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 565db2d8-fc7e-11ee-bc91-c7b262605968 -- ceph config generate-minimal-conf 2024-04-17T05:53:13.474 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T05:53:13.533 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:13 smithi103 ceph-mon[36243]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T05:53:13.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:13 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T05:53:13.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:13 smithi103 ceph-mon[36243]: from='client.? 172.21.15.103:0/2247768442' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-17T05:53:13.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:13 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-17T05:53:13.595 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:13 smithi047 ceph-mon[32223]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T05:53:13.596 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:13 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T05:53:13.596 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:13 smithi047 ceph-mon[32223]: from='client.? 172.21.15.103:0/2247768442' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-17T05:53:13.596 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:13 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-17T05:53:13.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:13 smithi098 ceph-mon[36516]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T05:53:13.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:13 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T05:53:13.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:13 smithi098 ceph-mon[36516]: from='client.? 172.21.15.103:0/2247768442' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-17T05:53:13.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:13 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-17T05:53:14.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:14 smithi047 ceph-mon[32223]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T05:53:14.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:14 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:14.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:14 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:14.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:14 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:14.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:14 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:14.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:14 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:14.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:14 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:53:14.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:14 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T05:53:14.890 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:14 smithi047 ceph-mon[32223]: Updating smithi047:/etc/ceph/ceph.conf 2024-04-17T05:53:14.890 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:14 smithi047 ceph-mon[32223]: Updating smithi098:/etc/ceph/ceph.conf 2024-04-17T05:53:14.890 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:14 smithi047 ceph-mon[32223]: Updating smithi103:/etc/ceph/ceph.conf 2024-04-17T05:53:14.890 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:14 smithi047 ceph-mon[32223]: Updating smithi047:/var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/config/ceph.conf 2024-04-17T05:53:14.890 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:14 smithi047 ceph-mon[32223]: Updating smithi103:/var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/config/ceph.conf 2024-04-17T05:53:14.890 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:14 smithi047 ceph-mon[32223]: Updating smithi098:/var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/config/ceph.conf 2024-04-17T05:53:14.890 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:14 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:14.890 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:14 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:14.890 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:14 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:14.890 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:14 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:14.890 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:14 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:14.890 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:14 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:14.890 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:14 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:14.890 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:14 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:14.891 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:14 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:14.891 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:14 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:14.891 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:14 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:14.891 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:14 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-17T05:53:14.891 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:14 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-17T05:53:14.891 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:14 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:53:15.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:14 smithi103 ceph-mon[36243]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T05:53:15.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:14 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:15.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:14 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:15.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:14 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:15.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:14 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:15.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:14 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:15.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:14 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:53:15.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:14 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T05:53:15.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:14 smithi103 ceph-mon[36243]: Updating smithi047:/etc/ceph/ceph.conf 2024-04-17T05:53:15.035 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:14 smithi103 ceph-mon[36243]: Updating smithi098:/etc/ceph/ceph.conf 2024-04-17T05:53:15.035 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:14 smithi103 ceph-mon[36243]: Updating smithi103:/etc/ceph/ceph.conf 2024-04-17T05:53:15.035 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:14 smithi103 ceph-mon[36243]: Updating smithi047:/var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/config/ceph.conf 2024-04-17T05:53:15.035 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:14 smithi103 ceph-mon[36243]: Updating smithi103:/var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/config/ceph.conf 2024-04-17T05:53:15.035 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:14 smithi103 ceph-mon[36243]: Updating smithi098:/var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/config/ceph.conf 2024-04-17T05:53:15.035 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:14 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:15.035 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:14 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:15.035 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:14 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:15.035 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:14 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:15.035 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:14 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:15.035 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:14 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:15.035 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:14 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:15.035 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:14 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:15.035 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:14 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:15.035 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:14 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:15.036 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:14 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:15.036 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:14 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-17T05:53:15.036 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:14 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-17T05:53:15.036 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:14 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:53:15.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:53:15.124+0000 7fa66e60b700 1 -- 172.21.15.47:0/2537643655 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa668101600 msgr2=0x7fa6681019e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:53:15.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:53:15.124+0000 7fa66e60b700 1 --2- 172.21.15.47:0/2537643655 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa668101600 0x7fa6681019e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa658009910 tx=0x7fa658009c20 comp rx=0 tx=0).stop 2024-04-17T05:53:15.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:53:15.124+0000 7fa66e60b700 1 -- 172.21.15.47:0/2537643655 shutdown_connections 2024-04-17T05:53:15.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:53:15.124+0000 7fa66e60b700 1 --2- 172.21.15.47:0/2537643655 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa668101600 0x7fa6681019e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:53:15.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:53:15.124+0000 7fa66e60b700 1 -- 172.21.15.47:0/2537643655 >> 172.21.15.47:0/2537643655 conn(0x7fa668076eb0 msgr2=0x7fa6680772c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:53:15.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:53:15.124+0000 7fa66e60b700 1 -- 172.21.15.47:0/2537643655 shutdown_connections 2024-04-17T05:53:15.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:53:15.124+0000 7fa66e60b700 1 -- 172.21.15.47:0/2537643655 wait complete. 2024-04-17T05:53:15.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:53:15.124+0000 7fa66e60b700 1 Processor -- start 2024-04-17T05:53:15.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:53:15.124+0000 7fa66e60b700 1 -- start start 2024-04-17T05:53:15.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:53:15.125+0000 7fa66e60b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa668101600 0x7fa668103050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:53:15.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:53:15.125+0000 7fa66e60b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa668103590 0x7fa668103a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:53:15.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:53:15.125+0000 7fa66e60b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6681a43d0 0x7fa6681a67c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:53:15.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:53:15.125+0000 7fa66e60b700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa66810b980 con 0x7fa6681a43d0 2024-04-17T05:53:15.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:53:15.125+0000 7fa66e60b700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa66810b800 con 0x7fa668103590 2024-04-17T05:53:15.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:53:15.125+0000 7fa66e60b700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa66810bb00 con 0x7fa668101600 2024-04-17T05:53:15.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:53:15.125+0000 7fa66cba8700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6681a43d0 0x7fa6681a67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:53:15.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:53:15.125+0000 7fa66cba8700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6681a43d0 0x7fa6681a67c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54702/0 (socket says 172.21.15.47:54702) 2024-04-17T05:53:15.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:53:15.125+0000 7fa66cba8700 1 -- 172.21.15.47:0/2616444493 learned_addr learned my addr 172.21.15.47:0/2616444493 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:53:15.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:53:15.125+0000 7fa667fff700 1 --2- 172.21.15.47:0/2616444493 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa668101600 0x7fa668103050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:53:15.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:53:15.125+0000 7fa6677fe700 1 --2- 172.21.15.47:0/2616444493 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa668103590 0x7fa668103a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:53:15.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:53:15.125+0000 7fa66cba8700 1 -- 172.21.15.47:0/2616444493 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa668101600 msgr2=0x7fa668103050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:53:15.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:53:15.125+0000 7fa66cba8700 1 --2- 172.21.15.47:0/2616444493 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa668101600 0x7fa668103050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:53:15.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:53:15.125+0000 7fa66cba8700 1 -- 172.21.15.47:0/2616444493 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa668103590 msgr2=0x7fa668103a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:53:15.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:53:15.125+0000 7fa66cba8700 1 --2- 172.21.15.47:0/2616444493 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa668103590 0x7fa668103a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:53:15.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:53:15.125+0000 7fa66cba8700 1 -- 172.21.15.47:0/2616444493 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6681a6ef0 con 0x7fa6681a43d0 2024-04-17T05:53:15.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:53:15.125+0000 7fa6677fe700 1 --2- 172.21.15.47:0/2616444493 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa668103590 0x7fa668103a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T05:53:15.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:53:15.126+0000 7fa66cba8700 1 --2- 172.21.15.47:0/2616444493 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6681a43d0 0x7fa6681a67c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa66000ba50 tx=0x7fa66000bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:53:15.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:53:15.126+0000 7fa6657fa700 1 -- 172.21.15.47:0/2616444493 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa660019070 con 0x7fa6681a43d0 2024-04-17T05:53:15.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:53:15.126+0000 7fa6657fa700 1 -- 172.21.15.47:0/2616444493 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa660014440 con 0x7fa6681a43d0 2024-04-17T05:53:15.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:53:15.126+0000 7fa66e60b700 1 -- 172.21.15.47:0/2616444493 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6681a71e0 con 0x7fa6681a43d0 2024-04-17T05:53:15.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:53:15.126+0000 7fa6657fa700 1 -- 172.21.15.47:0/2616444493 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6600075b0 con 0x7fa6681a43d0 2024-04-17T05:53:15.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:53:15.126+0000 7fa66e60b700 1 -- 172.21.15.47:0/2616444493 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6681a7720 con 0x7fa6681a43d0 2024-04-17T05:53:15.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:53:15.127+0000 7fa6657fa700 1 -- 172.21.15.47:0/2616444493 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fa66001c070 con 0x7fa6681a43d0 2024-04-17T05:53:15.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:53:15.127+0000 7fa66e60b700 1 -- 172.21.15.47:0/2616444493 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6680640b0 con 0x7fa6681a43d0 2024-04-17T05:53:15.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:53:15.127+0000 7fa6657fa700 1 --2- 172.21.15.47:0/2616444493 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fa650039ae0 0x7fa65003bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:53:15.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:53:15.127+0000 7fa6657fa700 1 -- 172.21.15.47:0/2616444493 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fa66004be10 con 0x7fa6681a43d0 2024-04-17T05:53:15.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:53:15.127+0000 7fa667fff700 1 --2- 172.21.15.47:0/2616444493 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fa650039ae0 0x7fa65003bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:53:15.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:53:15.128+0000 7fa667fff700 1 --2- 172.21.15.47:0/2616444493 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fa650039ae0 0x7fa65003bfa0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa658006010 tx=0x7fa6580058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:53:15.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:53:15.132+0000 7fa6657fa700 1 -- 172.21.15.47:0/2616444493 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fa660012740 con 0x7fa6681a43d0 2024-04-17T05:53:15.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:14 smithi098 ceph-mon[36516]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T05:53:15.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:14 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:15.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:14 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:15.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:14 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:15.134 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:14 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:15.134 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:14 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:15.134 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:14 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:53:15.134 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:14 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T05:53:15.134 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:14 smithi098 ceph-mon[36516]: Updating smithi047:/etc/ceph/ceph.conf 2024-04-17T05:53:15.134 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:14 smithi098 ceph-mon[36516]: Updating smithi098:/etc/ceph/ceph.conf 2024-04-17T05:53:15.134 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:14 smithi098 ceph-mon[36516]: Updating smithi103:/etc/ceph/ceph.conf 2024-04-17T05:53:15.134 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:14 smithi098 ceph-mon[36516]: Updating smithi047:/var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/config/ceph.conf 2024-04-17T05:53:15.135 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:14 smithi098 ceph-mon[36516]: Updating smithi103:/var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/config/ceph.conf 2024-04-17T05:53:15.135 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:14 smithi098 ceph-mon[36516]: Updating smithi098:/var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/config/ceph.conf 2024-04-17T05:53:15.135 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:14 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:15.135 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:14 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:15.135 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:14 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:15.135 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:14 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:15.135 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:14 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:15.135 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:14 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:15.135 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:14 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:15.136 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:14 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:15.136 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:14 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:15.136 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:14 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:15.136 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:14 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:15.136 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:14 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-17T05:53:15.136 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:14 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-17T05:53:15.136 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:14 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:53:15.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:53:15.348+0000 7fa66e60b700 1 -- 172.21.15.47:0/2616444493 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fa6681a79e0 con 0x7fa6681a43d0 2024-04-17T05:53:15.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:53:15.348+0000 7fa6657fa700 1 -- 172.21.15.47:0/2616444493 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+273 (secure 0 0 0) 0x7fa66000e090 con 0x7fa6681a43d0 2024-04-17T05:53:15.347 INFO:teuthology.orchestra.run.smithi047.stdout:# minimal ceph.conf for 565db2d8-fc7e-11ee-bc91-c7b262605968 2024-04-17T05:53:15.347 INFO:teuthology.orchestra.run.smithi047.stdout:[global] 2024-04-17T05:53:15.347 INFO:teuthology.orchestra.run.smithi047.stdout: fsid = 565db2d8-fc7e-11ee-bc91-c7b262605968 2024-04-17T05:53:15.347 INFO:teuthology.orchestra.run.smithi047.stdout: mon_host = [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] 2024-04-17T05:53:15.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:53:15.350+0000 7fa66e60b700 1 -- 172.21.15.47:0/2616444493 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fa650039ae0 msgr2=0x7fa65003bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:53:15.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:53:15.350+0000 7fa66e60b700 1 --2- 172.21.15.47:0/2616444493 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fa650039ae0 0x7fa65003bfa0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa658006010 tx=0x7fa6580058e0 comp rx=0 tx=0).stop 2024-04-17T05:53:15.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:53:15.350+0000 7fa66e60b700 1 -- 172.21.15.47:0/2616444493 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6681a43d0 msgr2=0x7fa6681a67c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:53:15.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:53:15.350+0000 7fa66e60b700 1 --2- 172.21.15.47:0/2616444493 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6681a43d0 0x7fa6681a67c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa66000ba50 tx=0x7fa66000bd60 comp rx=0 tx=0).stop 2024-04-17T05:53:15.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:53:15.350+0000 7fa66e60b700 1 -- 172.21.15.47:0/2616444493 shutdown_connections 2024-04-17T05:53:15.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:53:15.351+0000 7fa66e60b700 1 --2- 172.21.15.47:0/2616444493 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fa650039ae0 0x7fa65003bfa0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:53:15.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:53:15.351+0000 7fa66e60b700 1 --2- 172.21.15.47:0/2616444493 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6681a43d0 0x7fa6681a67c0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:53:15.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:53:15.351+0000 7fa66e60b700 1 --2- 172.21.15.47:0/2616444493 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa668101600 0x7fa668103050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:53:15.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:53:15.351+0000 7fa66e60b700 1 --2- 172.21.15.47:0/2616444493 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa668103590 0x7fa668103a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:53:15.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:53:15.351+0000 7fa66e60b700 1 -- 172.21.15.47:0/2616444493 >> 172.21.15.47:0/2616444493 conn(0x7fa668076eb0 msgr2=0x7fa668069e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:53:15.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:53:15.351+0000 7fa66e60b700 1 -- 172.21.15.47:0/2616444493 shutdown_connections 2024-04-17T05:53:15.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:53:15.351+0000 7fa66e60b700 1 -- 172.21.15.47:0/2616444493 wait complete. 2024-04-17T05:53:15.980 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:15 smithi047 ceph-mon[32223]: Reconfiguring mon.a (unknown last config time)... 2024-04-17T05:53:15.980 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:15 smithi047 ceph-mon[32223]: Reconfiguring daemon mon.a on smithi047 2024-04-17T05:53:15.980 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:15 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/2616444493' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:53:16.007 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-04-17T05:53:16.008 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-17T05:53:16.008 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/etc/ceph/ceph.conf 2024-04-17T05:53:16.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:15 smithi103 ceph-mon[36243]: Reconfiguring mon.a (unknown last config time)... 2024-04-17T05:53:16.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:15 smithi103 ceph-mon[36243]: Reconfiguring daemon mon.a on smithi047 2024-04-17T05:53:16.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:15 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/2616444493' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:53:16.039 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-17T05:53:16.039 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-17T05:53:16.052 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:15 smithi098 ceph-mon[36516]: Reconfiguring mon.a (unknown last config time)... 2024-04-17T05:53:16.052 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:15 smithi098 ceph-mon[36516]: Reconfiguring daemon mon.a on smithi047 2024-04-17T05:53:16.052 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:15 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/2616444493' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:53:16.109 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-04-17T05:53:16.109 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/etc/ceph/ceph.conf 2024-04-17T05:53:16.139 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-04-17T05:53:16.139 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-17T05:53:16.208 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-04-17T05:53:16.208 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/etc/ceph/ceph.conf 2024-04-17T05:53:16.242 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-04-17T05:53:16.242 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-17T05:53:16.313 INFO:tasks.cephadm:Adding mgr.x on smithi047 2024-04-17T05:53:16.314 INFO:tasks.cephadm:Adding mgr.y on smithi098 2024-04-17T05:53:16.314 INFO:tasks.cephadm:Adding mgr.z on smithi103 2024-04-17T05:53:16.314 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 565db2d8-fc7e-11ee-bc91-c7b262605968 -- ceph orch apply mgr '3;smithi047=x;smithi098=y;smithi103=z' 2024-04-17T05:53:16.590 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/mon.c/config 2024-04-17T05:53:16.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:16 smithi047 ceph-mon[32223]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T05:53:16.982 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:16 smithi103 ceph-mon[36243]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T05:53:17.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:16 smithi098 ceph-mon[36516]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T05:53:17.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:17.974+0000 7f0a6de8b700 1 -- 172.21.15.103:0/4060313690 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a68103060 msgr2=0x7f0a68105450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:53:17.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:17.974+0000 7f0a6de8b700 1 --2- 172.21.15.103:0/4060313690 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a68103060 0x7f0a68105450 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0a58009910 tx=0x7f0a58009c20 comp rx=0 tx=0).stop 2024-04-17T05:53:17.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:17.975+0000 7f0a6de8b700 1 -- 172.21.15.103:0/4060313690 shutdown_connections 2024-04-17T05:53:17.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:17.975+0000 7f0a6de8b700 1 --2- 172.21.15.103:0/4060313690 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a68103060 0x7f0a68105450 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:53:17.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:17.975+0000 7f0a6de8b700 1 -- 172.21.15.103:0/4060313690 >> 172.21.15.103:0/4060313690 conn(0x7f0a680fca90 msgr2=0x7f0a680feeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:53:17.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:17.975+0000 7f0a6de8b700 1 -- 172.21.15.103:0/4060313690 shutdown_connections 2024-04-17T05:53:17.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:17.975+0000 7f0a6de8b700 1 -- 172.21.15.103:0/4060313690 wait complete. 2024-04-17T05:53:17.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:17.975+0000 7f0a6de8b700 1 Processor -- start 2024-04-17T05:53:17.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:17.975+0000 7f0a6de8b700 1 -- start start 2024-04-17T05:53:17.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:17.975+0000 7f0a6de8b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0a68103060 0x7f0a68197d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:53:17.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:17.975+0000 7f0a6de8b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a68198270 0x7f0a6819d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:53:17.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:17.975+0000 7f0a6de8b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a681986f0 0x7f0a6819f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:53:17.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:17.975+0000 7f0a6de8b700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f0a68108600 con 0x7f0a68198270 2024-04-17T05:53:17.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:17.975+0000 7f0a6de8b700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0a68108480 con 0x7f0a68103060 2024-04-17T05:53:17.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:17.975+0000 7f0a6de8b700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0a68108780 con 0x7f0a681986f0 2024-04-17T05:53:17.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:17.976+0000 7f0a67fff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a681986f0 0x7f0a6819f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:53:17.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:17.976+0000 7f0a5effd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a68198270 0x7f0a6819d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:53:17.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:17.976+0000 7f0a67fff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a681986f0 0x7f0a6819f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37734/0 (socket says 172.21.15.103:37734) 2024-04-17T05:53:17.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:17.976+0000 7f0a67fff700 1 -- 172.21.15.103:0/1837535421 learned_addr learned my addr 172.21.15.103:0/1837535421 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-04-17T05:53:17.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:17.976+0000 7f0a677fe700 1 --2- 172.21.15.103:0/1837535421 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0a68103060 0x7f0a68197d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:53:17.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:17.976+0000 7f0a67fff700 1 -- 172.21.15.103:0/1837535421 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0a68103060 msgr2=0x7f0a68197d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:53:17.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:17.976+0000 7f0a67fff700 1 --2- 172.21.15.103:0/1837535421 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0a68103060 0x7f0a68197d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:53:17.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:17.976+0000 7f0a67fff700 1 -- 172.21.15.103:0/1837535421 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a68198270 msgr2=0x7f0a6819d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:53:17.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:17.976+0000 7f0a67fff700 1 --2- 172.21.15.103:0/1837535421 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a68198270 0x7f0a6819d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:53:17.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:17.976+0000 7f0a67fff700 1 -- 172.21.15.103:0/1837535421 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a681a0140 con 0x7f0a681986f0 2024-04-17T05:53:17.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:17.976+0000 7f0a67fff700 1 --2- 172.21.15.103:0/1837535421 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a681986f0 0x7f0a6819f870 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0a6000ba50 tx=0x7f0a6000bd60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:53:17.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:17.976+0000 7f0a657fa700 1 -- 172.21.15.103:0/1837535421 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a60019070 con 0x7f0a681986f0 2024-04-17T05:53:17.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:17.976+0000 7f0a6de8b700 1 -- 172.21.15.103:0/1837535421 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a681a0430 con 0x7f0a681986f0 2024-04-17T05:53:17.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:17.976+0000 7f0a657fa700 1 -- 172.21.15.103:0/1837535421 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a60014440 con 0x7f0a681986f0 2024-04-17T05:53:17.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:17.976+0000 7f0a657fa700 1 -- 172.21.15.103:0/1837535421 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a600075b0 con 0x7f0a681986f0 2024-04-17T05:53:17.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:17.976+0000 7f0a6de8b700 1 -- 172.21.15.103:0/1837535421 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a681a0a40 con 0x7f0a681986f0 2024-04-17T05:53:17.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:17.977+0000 7f0a657fa700 1 -- 172.21.15.103:0/1837535421 <== mon.1 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f0a6001c070 con 0x7f0a681986f0 2024-04-17T05:53:17.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:17.977+0000 7f0a6de8b700 1 -- 172.21.15.103:0/1837535421 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a680640b0 con 0x7f0a681986f0 2024-04-17T05:53:17.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:17.977+0000 7f0a657fa700 1 --2- 172.21.15.103:0/1837535421 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f0a54039ae0 0x7f0a5403bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:53:17.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:17.977+0000 7f0a657fa700 1 -- 172.21.15.103:0/1837535421 <== mon.1 v2:172.21.15.103:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f0a6004bdf0 con 0x7f0a681986f0 2024-04-17T05:53:17.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:17.978+0000 7f0a677fe700 1 --2- 172.21.15.103:0/1837535421 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f0a54039ae0 0x7f0a5403bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:53:17.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:17.978+0000 7f0a677fe700 1 --2- 172.21.15.103:0/1837535421 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f0a54039ae0 0x7f0a5403bfa0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0a5800b5c0 tx=0x7f0a5801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:53:17.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:17.983+0000 7f0a657fa700 1 -- 172.21.15.103:0/1837535421 <== mon.1 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f0a60014980 con 0x7f0a681986f0 2024-04-17T05:53:18.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:17 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:18.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:17 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:18.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:17 smithi098 ceph-mon[36516]: Reconfiguring mon.b (monmap changed)... 2024-04-17T05:53:18.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:17 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-17T05:53:18.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:17 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-17T05:53:18.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:17 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:53:18.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:17 smithi098 ceph-mon[36516]: Reconfiguring daemon mon.b on smithi098 2024-04-17T05:53:18.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:17 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:18.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:17 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:18.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:17 smithi047 ceph-mon[32223]: Reconfiguring mon.b (monmap changed)... 2024-04-17T05:53:18.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:17 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-17T05:53:18.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:17 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-17T05:53:18.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:17 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:53:18.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:17 smithi047 ceph-mon[32223]: Reconfiguring daemon mon.b on smithi098 2024-04-17T05:53:18.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:18.191+0000 7f0a6de8b700 1 -- 172.21.15.103:0/1837535421 --> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi047=x;smithi098=y;smithi103=z", "target": ["mon-mgr", ""]}) -- 0x7f0a68062f20 con 0x7f0a54039ae0 2024-04-17T05:53:18.190 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:17 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:18.190 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:17 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:18.191 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:17 smithi103 ceph-mon[36243]: Reconfiguring mon.b (monmap changed)... 2024-04-17T05:53:18.191 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:17 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-17T05:53:18.191 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:17 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-17T05:53:18.191 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:17 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:53:18.191 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:17 smithi103 ceph-mon[36243]: Reconfiguring daemon mon.b on smithi098 2024-04-17T05:53:18.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:18.197+0000 7f0a657fa700 1 -- 172.21.15.103:0/1837535421 <== mgr.14152 v2:172.21.15.47:6800/3157902806 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f0a68062f20 con 0x7f0a54039ae0 2024-04-17T05:53:18.196 INFO:teuthology.orchestra.run.smithi103.stdout:Scheduled mgr update... 2024-04-17T05:53:18.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:18.200+0000 7f0a6de8b700 1 -- 172.21.15.103:0/1837535421 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f0a54039ae0 msgr2=0x7f0a5403bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:53:18.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:18.200+0000 7f0a6de8b700 1 --2- 172.21.15.103:0/1837535421 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f0a54039ae0 0x7f0a5403bfa0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0a5800b5c0 tx=0x7f0a5801a040 comp rx=0 tx=0).stop 2024-04-17T05:53:18.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:18.200+0000 7f0a6de8b700 1 -- 172.21.15.103:0/1837535421 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a681986f0 msgr2=0x7f0a6819f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:53:18.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:18.200+0000 7f0a6de8b700 1 --2- 172.21.15.103:0/1837535421 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a681986f0 0x7f0a6819f870 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0a6000ba50 tx=0x7f0a6000bd60 comp rx=0 tx=0).stop 2024-04-17T05:53:18.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:18.200+0000 7f0a6de8b700 1 -- 172.21.15.103:0/1837535421 shutdown_connections 2024-04-17T05:53:18.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:18.200+0000 7f0a6de8b700 1 --2- 172.21.15.103:0/1837535421 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f0a54039ae0 0x7f0a5403bfa0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:53:18.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:18.200+0000 7f0a6de8b700 1 --2- 172.21.15.103:0/1837535421 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a681986f0 0x7f0a6819f870 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:53:18.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:18.200+0000 7f0a6de8b700 1 --2- 172.21.15.103:0/1837535421 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0a68103060 0x7f0a68197d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:53:18.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:18.200+0000 7f0a6de8b700 1 --2- 172.21.15.103:0/1837535421 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a68198270 0x7f0a6819d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:53:18.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:18.200+0000 7f0a6de8b700 1 -- 172.21.15.103:0/1837535421 >> 172.21.15.103:0/1837535421 conn(0x7f0a680fca90 msgr2=0x7f0a680fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:53:18.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:18.200+0000 7f0a6de8b700 1 -- 172.21.15.103:0/1837535421 shutdown_connections 2024-04-17T05:53:18.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:53:18.200+0000 7f0a6de8b700 1 -- 172.21.15.103:0/1837535421 wait complete. 2024-04-17T05:53:18.786 DEBUG:teuthology.orchestra.run.smithi098:mgr.y> sudo journalctl -f -n 0 -u ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@mgr.y.service 2024-04-17T05:53:18.789 DEBUG:teuthology.orchestra.run.smithi103:mgr.z> sudo journalctl -f -n 0 -u ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@mgr.z.service 2024-04-17T05:53:18.791 INFO:tasks.cephadm:Deploying OSDs... 2024-04-17T05:53:18.791 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-17T05:53:18.791 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/scratch_devs of=/dev/stdout 2024-04-17T05:53:18.811 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-17T05:53:18.811 DEBUG:teuthology.orchestra.run.smithi047:> stat /dev/vg_nvme/lv_1 2024-04-17T05:53:18.869 INFO:teuthology.orchestra.run.smithi047.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-17T05:53:18.869 INFO:teuthology.orchestra.run.smithi047.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-17T05:53:18.869 INFO:teuthology.orchestra.run.smithi047.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-04-17T05:53:18.870 INFO:teuthology.orchestra.run.smithi047.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-17T05:53:18.870 INFO:teuthology.orchestra.run.smithi047.stdout:Context: system_u:object_r:device_t:s0 2024-04-17T05:53:18.870 INFO:teuthology.orchestra.run.smithi047.stdout:Access: 2024-04-17 05:52:26.158577612 +0000 2024-04-17T05:53:18.870 INFO:teuthology.orchestra.run.smithi047.stdout:Modify: 2024-04-17 05:47:25.775339416 +0000 2024-04-17T05:53:18.870 INFO:teuthology.orchestra.run.smithi047.stdout:Change: 2024-04-17 05:47:25.775339416 +0000 2024-04-17T05:53:18.870 INFO:teuthology.orchestra.run.smithi047.stdout: Birth: 2024-04-17 05:47:25.775339416 +0000 2024-04-17T05:53:18.870 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-17T05:53:18.941 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records in 2024-04-17T05:53:18.941 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records out 2024-04-17T05:53:18.941 INFO:teuthology.orchestra.run.smithi047.stderr:512 bytes copied, 0.000123276 s, 4.2 MB/s 2024-04-17T05:53:18.942 DEBUG:teuthology.orchestra.run.smithi047:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-17T05:53:19.000 DEBUG:teuthology.orchestra.run.smithi047:> stat /dev/vg_nvme/lv_2 2024-04-17T05:53:19.057 INFO:teuthology.orchestra.run.smithi047.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-17T05:53:19.058 INFO:teuthology.orchestra.run.smithi047.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-17T05:53:19.058 INFO:teuthology.orchestra.run.smithi047.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-04-17T05:53:19.058 INFO:teuthology.orchestra.run.smithi047.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-17T05:53:19.058 INFO:teuthology.orchestra.run.smithi047.stdout:Context: system_u:object_r:device_t:s0 2024-04-17T05:53:19.058 INFO:teuthology.orchestra.run.smithi047.stdout:Access: 2024-04-17 05:52:26.186577013 +0000 2024-04-17T05:53:19.058 INFO:teuthology.orchestra.run.smithi047.stdout:Modify: 2024-04-17 05:47:26.122331877 +0000 2024-04-17T05:53:19.058 INFO:teuthology.orchestra.run.smithi047.stdout:Change: 2024-04-17 05:47:26.122331877 +0000 2024-04-17T05:53:19.058 INFO:teuthology.orchestra.run.smithi047.stdout: Birth: 2024-04-17 05:47:26.122331877 +0000 2024-04-17T05:53:19.058 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-17T05:53:19.126 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records in 2024-04-17T05:53:19.126 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records out 2024-04-17T05:53:19.126 INFO:teuthology.orchestra.run.smithi047.stderr:512 bytes copied, 0.000113901 s, 4.5 MB/s 2024-04-17T05:53:19.128 DEBUG:teuthology.orchestra.run.smithi047:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-17T05:53:19.185 DEBUG:teuthology.orchestra.run.smithi047:> stat /dev/vg_nvme/lv_3 2024-04-17T05:53:19.242 INFO:teuthology.orchestra.run.smithi047.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-17T05:53:19.242 INFO:teuthology.orchestra.run.smithi047.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-17T05:53:19.242 INFO:teuthology.orchestra.run.smithi047.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-04-17T05:53:19.242 INFO:teuthology.orchestra.run.smithi047.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-17T05:53:19.242 INFO:teuthology.orchestra.run.smithi047.stdout:Context: system_u:object_r:device_t:s0 2024-04-17T05:53:19.242 INFO:teuthology.orchestra.run.smithi047.stdout:Access: 2024-04-17 05:52:26.213576434 +0000 2024-04-17T05:53:19.242 INFO:teuthology.orchestra.run.smithi047.stdout:Modify: 2024-04-17 05:47:26.464324447 +0000 2024-04-17T05:53:19.242 INFO:teuthology.orchestra.run.smithi047.stdout:Change: 2024-04-17 05:47:26.464324447 +0000 2024-04-17T05:53:19.242 INFO:teuthology.orchestra.run.smithi047.stdout: Birth: 2024-04-17 05:47:26.464324447 +0000 2024-04-17T05:53:19.243 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-17T05:53:19.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:19 smithi103 ceph-mon[36243]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T05:53:19.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:19 smithi103 ceph-mon[36243]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi047=x;smithi098=y;smithi103=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T05:53:19.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:19 smithi103 ceph-mon[36243]: Saving service mgr spec with placement smithi047=x;smithi098=y;smithi103=z;count:3 2024-04-17T05:53:19.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:19 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:19.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:19 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:19.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:19 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:19.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:19 smithi103 ceph-mon[36243]: Reconfiguring mon.c (monmap changed)... 2024-04-17T05:53:19.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:19 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-17T05:53:19.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:19 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-17T05:53:19.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:19 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:53:19.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:19 smithi103 ceph-mon[36243]: Reconfiguring daemon mon.c on smithi103 2024-04-17T05:53:19.309 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records in 2024-04-17T05:53:19.309 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records out 2024-04-17T05:53:19.309 INFO:teuthology.orchestra.run.smithi047.stderr:512 bytes copied, 0.000107538 s, 4.8 MB/s 2024-04-17T05:53:19.310 DEBUG:teuthology.orchestra.run.smithi047:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-17T05:53:19.367 DEBUG:teuthology.orchestra.run.smithi047:> stat /dev/vg_nvme/lv_4 2024-04-17T05:53:19.422 INFO:teuthology.orchestra.run.smithi047.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-17T05:53:19.422 INFO:teuthology.orchestra.run.smithi047.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-17T05:53:19.422 INFO:teuthology.orchestra.run.smithi047.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-04-17T05:53:19.422 INFO:teuthology.orchestra.run.smithi047.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-17T05:53:19.422 INFO:teuthology.orchestra.run.smithi047.stdout:Context: system_u:object_r:device_t:s0 2024-04-17T05:53:19.422 INFO:teuthology.orchestra.run.smithi047.stdout:Access: 2024-04-17 05:52:26.240575856 +0000 2024-04-17T05:53:19.422 INFO:teuthology.orchestra.run.smithi047.stdout:Modify: 2024-04-17 05:47:26.793317299 +0000 2024-04-17T05:53:19.422 INFO:teuthology.orchestra.run.smithi047.stdout:Change: 2024-04-17 05:47:26.793317299 +0000 2024-04-17T05:53:19.422 INFO:teuthology.orchestra.run.smithi047.stdout: Birth: 2024-04-17 05:47:26.793317299 +0000 2024-04-17T05:53:19.422 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-17T05:53:19.486 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:19 smithi047 ceph-mon[32223]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T05:53:19.486 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:19 smithi047 ceph-mon[32223]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi047=x;smithi098=y;smithi103=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T05:53:19.486 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:19 smithi047 ceph-mon[32223]: Saving service mgr spec with placement smithi047=x;smithi098=y;smithi103=z;count:3 2024-04-17T05:53:19.486 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:19 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:19.487 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:19 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:19.487 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:19 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:19.487 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:19 smithi047 ceph-mon[32223]: Reconfiguring mon.c (monmap changed)... 2024-04-17T05:53:19.487 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:19 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-17T05:53:19.487 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:19 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-17T05:53:19.487 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:19 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:53:19.487 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:19 smithi047 ceph-mon[32223]: Reconfiguring daemon mon.c on smithi103 2024-04-17T05:53:19.488 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records in 2024-04-17T05:53:19.488 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records out 2024-04-17T05:53:19.488 INFO:teuthology.orchestra.run.smithi047.stderr:512 bytes copied, 9.795e-05 s, 5.2 MB/s 2024-04-17T05:53:19.490 DEBUG:teuthology.orchestra.run.smithi047:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-17T05:53:19.547 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-04-17T05:53:19.547 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/scratch_devs of=/dev/stdout 2024-04-17T05:53:19.565 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-17T05:53:19.566 DEBUG:teuthology.orchestra.run.smithi098:> stat /dev/vg_nvme/lv_1 2024-04-17T05:53:19.623 INFO:teuthology.orchestra.run.smithi098.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-17T05:53:19.623 INFO:teuthology.orchestra.run.smithi098.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-17T05:53:19.623 INFO:teuthology.orchestra.run.smithi098.stdout:Device: 5h/5d Inode: 663 Links: 1 2024-04-17T05:53:19.624 INFO:teuthology.orchestra.run.smithi098.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-17T05:53:19.624 INFO:teuthology.orchestra.run.smithi098.stdout:Context: system_u:object_r:device_t:s0 2024-04-17T05:53:19.624 INFO:teuthology.orchestra.run.smithi098.stdout:Access: 2024-04-17 05:53:06.679372429 +0000 2024-04-17T05:53:19.624 INFO:teuthology.orchestra.run.smithi098.stdout:Modify: 2024-04-17 05:47:44.584465963 +0000 2024-04-17T05:53:19.624 INFO:teuthology.orchestra.run.smithi098.stdout:Change: 2024-04-17 05:47:44.584465963 +0000 2024-04-17T05:53:19.624 INFO:teuthology.orchestra.run.smithi098.stdout: Birth: 2024-04-17 05:47:44.584465963 +0000 2024-04-17T05:53:19.624 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-17T05:53:19.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:19 smithi098 ceph-mon[36516]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T05:53:19.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:19 smithi098 ceph-mon[36516]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi047=x;smithi098=y;smithi103=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T05:53:19.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:19 smithi098 ceph-mon[36516]: Saving service mgr spec with placement smithi047=x;smithi098=y;smithi103=z;count:3 2024-04-17T05:53:19.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:19 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:19.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:19 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:19.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:19 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:19.634 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:19 smithi098 ceph-mon[36516]: Reconfiguring mon.c (monmap changed)... 2024-04-17T05:53:19.634 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:19 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-17T05:53:19.634 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:19 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-17T05:53:19.634 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:19 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:53:19.634 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:19 smithi098 ceph-mon[36516]: Reconfiguring daemon mon.c on smithi103 2024-04-17T05:53:19.662 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records in 2024-04-17T05:53:19.696 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records out 2024-04-17T05:53:19.696 INFO:teuthology.orchestra.run.smithi098.stderr:512 bytes copied, 0.000221868 s, 2.3 MB/s 2024-04-17T05:53:19.697 DEBUG:teuthology.orchestra.run.smithi098:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-17T05:53:19.755 DEBUG:teuthology.orchestra.run.smithi098:> stat /dev/vg_nvme/lv_2 2024-04-17T05:53:19.812 INFO:teuthology.orchestra.run.smithi098.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-17T05:53:19.812 INFO:teuthology.orchestra.run.smithi098.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-17T05:53:19.812 INFO:teuthology.orchestra.run.smithi098.stdout:Device: 5h/5d Inode: 682 Links: 1 2024-04-17T05:53:19.812 INFO:teuthology.orchestra.run.smithi098.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-17T05:53:19.812 INFO:teuthology.orchestra.run.smithi098.stdout:Context: system_u:object_r:device_t:s0 2024-04-17T05:53:19.812 INFO:teuthology.orchestra.run.smithi098.stdout:Access: 2024-04-17 05:53:06.680372416 +0000 2024-04-17T05:53:19.812 INFO:teuthology.orchestra.run.smithi098.stdout:Modify: 2024-04-17 05:47:44.919461196 +0000 2024-04-17T05:53:19.813 INFO:teuthology.orchestra.run.smithi098.stdout:Change: 2024-04-17 05:47:44.919461196 +0000 2024-04-17T05:53:19.813 INFO:teuthology.orchestra.run.smithi098.stdout: Birth: 2024-04-17 05:47:44.919461196 +0000 2024-04-17T05:53:19.813 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-17T05:53:19.879 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records in 2024-04-17T05:53:19.879 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records out 2024-04-17T05:53:19.880 INFO:teuthology.orchestra.run.smithi098.stderr:512 bytes copied, 0.00022237 s, 2.3 MB/s 2024-04-17T05:53:19.881 DEBUG:teuthology.orchestra.run.smithi098:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-17T05:53:19.938 DEBUG:teuthology.orchestra.run.smithi098:> stat /dev/vg_nvme/lv_3 2024-04-17T05:53:19.993 INFO:teuthology.orchestra.run.smithi098.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-17T05:53:19.993 INFO:teuthology.orchestra.run.smithi098.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-17T05:53:19.993 INFO:teuthology.orchestra.run.smithi098.stdout:Device: 5h/5d Inode: 701 Links: 1 2024-04-17T05:53:19.993 INFO:teuthology.orchestra.run.smithi098.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-17T05:53:19.994 INFO:teuthology.orchestra.run.smithi098.stdout:Context: system_u:object_r:device_t:s0 2024-04-17T05:53:19.994 INFO:teuthology.orchestra.run.smithi098.stdout:Access: 2024-04-17 05:53:06.680372416 +0000 2024-04-17T05:53:19.994 INFO:teuthology.orchestra.run.smithi098.stdout:Modify: 2024-04-17 05:47:45.263456302 +0000 2024-04-17T05:53:19.994 INFO:teuthology.orchestra.run.smithi098.stdout:Change: 2024-04-17 05:47:45.263456302 +0000 2024-04-17T05:53:19.994 INFO:teuthology.orchestra.run.smithi098.stdout: Birth: 2024-04-17 05:47:45.263456302 +0000 2024-04-17T05:53:19.994 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-17T05:53:20.060 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records in 2024-04-17T05:53:20.061 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records out 2024-04-17T05:53:20.061 INFO:teuthology.orchestra.run.smithi098.stderr:512 bytes copied, 0.000179223 s, 2.9 MB/s 2024-04-17T05:53:20.062 DEBUG:teuthology.orchestra.run.smithi098:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-17T05:53:20.118 DEBUG:teuthology.orchestra.run.smithi098:> stat /dev/vg_nvme/lv_4 2024-04-17T05:53:20.173 INFO:teuthology.orchestra.run.smithi098.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-17T05:53:20.173 INFO:teuthology.orchestra.run.smithi098.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-17T05:53:20.173 INFO:teuthology.orchestra.run.smithi098.stdout:Device: 5h/5d Inode: 718 Links: 1 2024-04-17T05:53:20.173 INFO:teuthology.orchestra.run.smithi098.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-17T05:53:20.173 INFO:teuthology.orchestra.run.smithi098.stdout:Context: system_u:object_r:device_t:s0 2024-04-17T05:53:20.173 INFO:teuthology.orchestra.run.smithi098.stdout:Access: 2024-04-17 05:53:06.681372402 +0000 2024-04-17T05:53:20.173 INFO:teuthology.orchestra.run.smithi098.stdout:Modify: 2024-04-17 05:47:45.609451379 +0000 2024-04-17T05:53:20.173 INFO:teuthology.orchestra.run.smithi098.stdout:Change: 2024-04-17 05:47:45.609451379 +0000 2024-04-17T05:53:20.173 INFO:teuthology.orchestra.run.smithi098.stdout: Birth: 2024-04-17 05:47:45.609451379 +0000 2024-04-17T05:53:20.173 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-17T05:53:20.239 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records in 2024-04-17T05:53:20.239 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records out 2024-04-17T05:53:20.239 INFO:teuthology.orchestra.run.smithi098.stderr:512 bytes copied, 0.000200454 s, 2.6 MB/s 2024-04-17T05:53:20.240 DEBUG:teuthology.orchestra.run.smithi098:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-17T05:53:20.297 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-04-17T05:53:20.297 DEBUG:teuthology.orchestra.run.smithi103:> dd if=/scratch_devs of=/dev/stdout 2024-04-17T05:53:20.316 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-17T05:53:20.316 DEBUG:teuthology.orchestra.run.smithi103:> stat /dev/vg_nvme/lv_1 2024-04-17T05:53:20.373 INFO:teuthology.orchestra.run.smithi103.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-17T05:53:20.373 INFO:teuthology.orchestra.run.smithi103.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-17T05:53:20.373 INFO:teuthology.orchestra.run.smithi103.stdout:Device: 5h/5d Inode: 662 Links: 1 2024-04-17T05:53:20.373 INFO:teuthology.orchestra.run.smithi103.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-17T05:53:20.373 INFO:teuthology.orchestra.run.smithi103.stdout:Context: system_u:object_r:device_t:s0 2024-04-17T05:53:20.373 INFO:teuthology.orchestra.run.smithi103.stdout:Access: 2024-04-17 05:52:57.480931079 +0000 2024-04-17T05:53:20.373 INFO:teuthology.orchestra.run.smithi103.stdout:Modify: 2024-04-17 05:47:44.885291492 +0000 2024-04-17T05:53:20.373 INFO:teuthology.orchestra.run.smithi103.stdout:Change: 2024-04-17 05:47:44.885291492 +0000 2024-04-17T05:53:20.373 INFO:teuthology.orchestra.run.smithi103.stdout: Birth: 2024-04-17 05:47:44.885291492 +0000 2024-04-17T05:53:20.373 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-17T05:53:20.443 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records in 2024-04-17T05:53:20.443 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records out 2024-04-17T05:53:20.443 INFO:teuthology.orchestra.run.smithi103.stderr:512 bytes copied, 0.000285822 s, 1.8 MB/s 2024-04-17T05:53:20.444 DEBUG:teuthology.orchestra.run.smithi103:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-17T05:53:20.502 DEBUG:teuthology.orchestra.run.smithi103:> stat /dev/vg_nvme/lv_2 2024-04-17T05:53:20.559 INFO:teuthology.orchestra.run.smithi103.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-17T05:53:20.559 INFO:teuthology.orchestra.run.smithi103.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-17T05:53:20.559 INFO:teuthology.orchestra.run.smithi103.stdout:Device: 5h/5d Inode: 680 Links: 1 2024-04-17T05:53:20.559 INFO:teuthology.orchestra.run.smithi103.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-17T05:53:20.560 INFO:teuthology.orchestra.run.smithi103.stdout:Context: system_u:object_r:device_t:s0 2024-04-17T05:53:20.560 INFO:teuthology.orchestra.run.smithi103.stdout:Access: 2024-04-17 05:52:57.480931079 +0000 2024-04-17T05:53:20.560 INFO:teuthology.orchestra.run.smithi103.stdout:Modify: 2024-04-17 05:47:45.235284721 +0000 2024-04-17T05:53:20.560 INFO:teuthology.orchestra.run.smithi103.stdout:Change: 2024-04-17 05:47:45.235284721 +0000 2024-04-17T05:53:20.560 INFO:teuthology.orchestra.run.smithi103.stdout: Birth: 2024-04-17 05:47:45.235284721 +0000 2024-04-17T05:53:20.561 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-17T05:53:20.628 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records in 2024-04-17T05:53:20.628 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records out 2024-04-17T05:53:20.628 INFO:teuthology.orchestra.run.smithi103.stderr:512 bytes copied, 0.000223297 s, 2.3 MB/s 2024-04-17T05:53:20.629 DEBUG:teuthology.orchestra.run.smithi103:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-17T05:53:20.685 DEBUG:teuthology.orchestra.run.smithi103:> stat /dev/vg_nvme/lv_3 2024-04-17T05:53:20.740 INFO:teuthology.orchestra.run.smithi103.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-17T05:53:20.741 INFO:teuthology.orchestra.run.smithi103.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-17T05:53:20.741 INFO:teuthology.orchestra.run.smithi103.stdout:Device: 5h/5d Inode: 700 Links: 1 2024-04-17T05:53:20.741 INFO:teuthology.orchestra.run.smithi103.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-17T05:53:20.741 INFO:teuthology.orchestra.run.smithi103.stdout:Context: system_u:object_r:device_t:s0 2024-04-17T05:53:20.741 INFO:teuthology.orchestra.run.smithi103.stdout:Access: 2024-04-17 05:52:57.481931060 +0000 2024-04-17T05:53:20.741 INFO:teuthology.orchestra.run.smithi103.stdout:Modify: 2024-04-17 05:47:45.556278511 +0000 2024-04-17T05:53:20.741 INFO:teuthology.orchestra.run.smithi103.stdout:Change: 2024-04-17 05:47:45.556278511 +0000 2024-04-17T05:53:20.741 INFO:teuthology.orchestra.run.smithi103.stdout: Birth: 2024-04-17 05:47:45.556278511 +0000 2024-04-17T05:53:20.741 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-17T05:53:20.806 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records in 2024-04-17T05:53:20.807 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records out 2024-04-17T05:53:20.807 INFO:teuthology.orchestra.run.smithi103.stderr:512 bytes copied, 0.000253808 s, 2.0 MB/s 2024-04-17T05:53:20.808 DEBUG:teuthology.orchestra.run.smithi103:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-17T05:53:20.864 DEBUG:teuthology.orchestra.run.smithi103:> stat /dev/vg_nvme/lv_4 2024-04-17T05:53:20.919 INFO:teuthology.orchestra.run.smithi103.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-17T05:53:20.919 INFO:teuthology.orchestra.run.smithi103.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-17T05:53:20.920 INFO:teuthology.orchestra.run.smithi103.stdout:Device: 5h/5d Inode: 719 Links: 1 2024-04-17T05:53:20.920 INFO:teuthology.orchestra.run.smithi103.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-17T05:53:20.920 INFO:teuthology.orchestra.run.smithi103.stdout:Context: system_u:object_r:device_t:s0 2024-04-17T05:53:20.920 INFO:teuthology.orchestra.run.smithi103.stdout:Access: 2024-04-17 05:52:57.481931060 +0000 2024-04-17T05:53:20.920 INFO:teuthology.orchestra.run.smithi103.stdout:Modify: 2024-04-17 05:47:45.903271798 +0000 2024-04-17T05:53:20.920 INFO:teuthology.orchestra.run.smithi103.stdout:Change: 2024-04-17 05:47:45.903271798 +0000 2024-04-17T05:53:20.920 INFO:teuthology.orchestra.run.smithi103.stdout: Birth: 2024-04-17 05:47:45.903271798 +0000 2024-04-17T05:53:20.920 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-17T05:53:20.985 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records in 2024-04-17T05:53:20.985 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records out 2024-04-17T05:53:20.985 INFO:teuthology.orchestra.run.smithi103.stderr:512 bytes copied, 0.000213859 s, 2.4 MB/s 2024-04-17T05:53:20.986 DEBUG:teuthology.orchestra.run.smithi103:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-17T05:53:21.042 INFO:tasks.cephadm:Deploying osd.0 on smithi047 with /dev/vg_nvme/lv_4... 2024-04-17T05:53:21.042 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 565db2d8-fc7e-11ee-bc91-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-17T05:53:21.216 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:20 smithi103 ceph-mon[36243]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T05:53:21.216 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:20 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:21.216 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:20 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:21.217 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:20 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T05:53:21.217 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:20 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:53:21.217 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:20 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T05:53:21.217 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:20 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:21.217 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:20 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-17T05:53:21.217 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:20 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-17T05:53:21.217 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:20 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-17T05:53:21.217 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:20 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:53:21.217 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:20 smithi103 ceph-mon[36243]: Deploying daemon mgr.z on smithi103 2024-04-17T05:53:21.287 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T05:53:21.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:20 smithi098 ceph-mon[36516]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T05:53:21.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:20 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:21.384 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:20 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:21.384 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:20 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T05:53:21.384 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:20 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:53:21.384 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:20 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T05:53:21.384 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:20 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:21.384 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:20 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-17T05:53:21.385 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:20 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-17T05:53:21.385 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:20 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-17T05:53:21.385 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:20 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:53:21.385 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:20 smithi098 ceph-mon[36516]: Deploying daemon mgr.z on smithi103 2024-04-17T05:53:21.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:20 smithi047 ceph-mon[32223]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T05:53:21.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:20 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:21.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:20 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:21.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:20 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T05:53:21.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:20 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:53:21.390 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:20 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T05:53:21.390 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:20 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:21.390 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:20 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-17T05:53:21.390 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:20 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-17T05:53:21.390 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:20 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-17T05:53:21.390 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:20 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:53:21.391 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:20 smithi047 ceph-mon[32223]: Deploying daemon mgr.z on smithi103 2024-04-17T05:53:22.723 INFO:journalctl@ceph.mgr.z.smithi103.stdout:Apr 17 05:53:22 smithi103 systemd[1]: Starting Ceph mgr.z for 565db2d8-fc7e-11ee-bc91-c7b262605968... 2024-04-17T05:53:23.009 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:22 smithi103 ceph-mon[36243]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T05:53:23.010 INFO:journalctl@ceph.mgr.z.smithi103.stdout:Apr 17 05:53:22 smithi103 podman[38077]: 2024-04-17 05:53:22.628474133 +0000 UTC m=+0.017574101 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-17T05:53:23.010 INFO:journalctl@ceph.mgr.z.smithi103.stdout:Apr 17 05:53:22 smithi103 podman[38077]: 2024-04-17 05:53:22.726192804 +0000 UTC m=+0.115292755 container create 6af7082b7f514ef604e356ecf1c23d76aa639607f98383bf74b6e2f1ce3084aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-z, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-17T05:53:23.010 INFO:journalctl@ceph.mgr.z.smithi103.stdout:Apr 17 05:53:22 smithi103 podman[38077]: 2024-04-17 05:53:22.937421163 +0000 UTC m=+0.326521124 container init 6af7082b7f514ef604e356ecf1c23d76aa639607f98383bf74b6e2f1ce3084aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-z, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-17T05:53:23.010 INFO:journalctl@ceph.mgr.z.smithi103.stdout:Apr 17 05:53:22 smithi103 podman[38077]: 2024-04-17 05:53:22.939948808 +0000 UTC m=+0.329048778 container start 6af7082b7f514ef604e356ecf1c23d76aa639607f98383bf74b6e2f1ce3084aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-z, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=) 2024-04-17T05:53:23.011 INFO:journalctl@ceph.mgr.z.smithi103.stdout:Apr 17 05:53:22 smithi103 ceph-mgr[38095]: -- 172.21.15.103:0/3705724738 <== mon.1 v2:172.21.15.103:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562fc538a5a0 con 0x562fc535ec00 2024-04-17T05:53:23.011 INFO:journalctl@ceph.mgr.z.smithi103.stdout:Apr 17 05:53:23 smithi103 bash[38077]: 6af7082b7f514ef604e356ecf1c23d76aa639607f98383bf74b6e2f1ce3084aa 2024-04-17T05:53:23.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:22 smithi047 ceph-mon[32223]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T05:53:23.197 INFO:teuthology.orchestra.run.smithi047.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-17T05:53:23.198 INFO:teuthology.orchestra.run.smithi047.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-17T05:53:23.198 INFO:teuthology.orchestra.run.smithi047.stderr: stderr: 10+0 records in 2024-04-17T05:53:23.198 INFO:teuthology.orchestra.run.smithi047.stderr:10+0 records out 2024-04-17T05:53:23.198 INFO:teuthology.orchestra.run.smithi047.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00832063 s, 1.3 GB/s 2024-04-17T05:53:23.198 INFO:teuthology.orchestra.run.smithi047.stderr:--> Zapping successful for: 2024-04-17T05:53:23.283 INFO:journalctl@ceph.mgr.z.smithi103.stdout:Apr 17 05:53:23 smithi103 systemd[1]: Started Ceph mgr.z for 565db2d8-fc7e-11ee-bc91-c7b262605968. 2024-04-17T05:53:23.382 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:22 smithi098 ceph-mon[36516]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T05:53:23.783 INFO:journalctl@ceph.mgr.z.smithi103.stdout:Apr 17 05:53:23 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-z[38091]: 2024-04-17T05:53:23.485+0000 7fb119768200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-17T05:53:23.784 INFO:journalctl@ceph.mgr.z.smithi103.stdout:Apr 17 05:53:23 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-z[38091]: 2024-04-17T05:53:23.578+0000 7fb119768200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-17T05:53:23.811 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 565db2d8-fc7e-11ee-bc91-c7b262605968 -- ceph orch daemon add osd smithi047:vg_nvme/lv_4 2024-04-17T05:53:24.057 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T05:53:24.330 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:24 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:24.330 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:24 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:24.330 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:24 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:24.330 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:24 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-17T05:53:24.331 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:24 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-17T05:53:24.331 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:24 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-17T05:53:24.331 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:24 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:53:24.331 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:24 smithi047 ceph-mon[32223]: Deploying daemon mgr.y on smithi098 2024-04-17T05:53:24.345 INFO:journalctl@ceph.mgr.z.smithi103.stdout:Apr 17 05:53:24 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-z[38091]: 2024-04-17T05:53:24.224+0000 7fb119768200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-17T05:53:24.345 INFO:journalctl@ceph.mgr.z.smithi103.stdout:Apr 17 05:53:24 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-z[38091]: 2024-04-17T05:53:24.317+0000 7fb119768200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-17T05:53:24.345 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:24 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:24.345 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:24 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:24.345 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:24 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:24.345 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:24 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-17T05:53:24.345 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:24 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-17T05:53:24.345 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:24 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-17T05:53:24.345 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:24 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:53:24.345 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:24 smithi103 ceph-mon[36243]: Deploying daemon mgr.y on smithi098 2024-04-17T05:53:24.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:24 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:24.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:24 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:24.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:24 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:24.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:24 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-17T05:53:24.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:24 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-17T05:53:24.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:24 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-17T05:53:24.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:24 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:53:24.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:24 smithi098 ceph-mon[36516]: Deploying daemon mgr.y on smithi098 2024-04-17T05:53:24.647 INFO:journalctl@ceph.mgr.z.smithi103.stdout:Apr 17 05:53:24 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-z[38091]: 2024-04-17T05:53:24.465+0000 7fb119768200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-17T05:53:24.647 INFO:journalctl@ceph.mgr.z.smithi103.stdout:Apr 17 05:53:24 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-z[38091]: 2024-04-17T05:53:24.563+0000 7fb119768200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-17T05:53:24.647 INFO:journalctl@ceph.mgr.z.smithi103.stdout:Apr 17 05:53:24 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-z[38091]: 2024-04-17T05:53:24.648+0000 7fb119768200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-17T05:53:25.214 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:25 smithi103 ceph-mon[36243]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T05:53:25.215 INFO:journalctl@ceph.mgr.z.smithi103.stdout:Apr 17 05:53:24 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-z[38091]: 2024-04-17T05:53:24.920+0000 7fb119768200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-17T05:53:25.215 INFO:journalctl@ceph.mgr.z.smithi103.stdout:Apr 17 05:53:25 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-z[38091]: 2024-04-17T05:53:25.031+0000 7fb119768200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-17T05:53:25.215 INFO:journalctl@ceph.mgr.z.smithi103.stdout:Apr 17 05:53:25 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-z[38091]: 2024-04-17T05:53:25.119+0000 7fb119768200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-17T05:53:25.215 INFO:journalctl@ceph.mgr.z.smithi103.stdout:Apr 17 05:53:25 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-z[38091]: 2024-04-17T05:53:25.216+0000 7fb119768200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-17T05:53:25.277 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:25 smithi098 ceph-mon[36516]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T05:53:25.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:25 smithi047 ceph-mon[32223]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T05:53:25.533 INFO:journalctl@ceph.mgr.z.smithi103.stdout:Apr 17 05:53:25 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-z[38091]: 2024-04-17T05:53:25.395+0000 7fb119768200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-17T05:53:25.573 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 17 05:53:25 smithi098 ceph-mgr[37900]: -- 172.21.15.98:0/2705390195 <== mon.2 v2:172.21.15.98:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558dd14ba5a0 con 0x558dd148f000 2024-04-17T05:53:25.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:53:25.675+0000 7f004ae5b700 1 -- 172.21.15.47:0/1139112519 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0044104690 msgr2=0x7f004410af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:53:25.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:53:25.675+0000 7f004ae5b700 1 --2- 172.21.15.47:0/1139112519 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0044104690 0x7f004410af40 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f003c009f90 tx=0x7f003c00b330 comp rx=0 tx=0).stop 2024-04-17T05:53:25.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:53:25.675+0000 7f004ae5b700 1 -- 172.21.15.47:0/1139112519 shutdown_connections 2024-04-17T05:53:25.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:53:25.675+0000 7f004ae5b700 1 --2- 172.21.15.47:0/1139112519 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0044104690 0x7f004410af40 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:53:25.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:53:25.675+0000 7f004ae5b700 1 --2- 172.21.15.47:0/1139112519 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0044102af0 0x7f0044102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:53:25.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:53:25.675+0000 7f004ae5b700 1 --2- 172.21.15.47:0/1139112519 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0044103cf0 0x7f0044104150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:53:25.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:53:25.675+0000 7f004ae5b700 1 -- 172.21.15.47:0/1139112519 >> 172.21.15.47:0/1139112519 conn(0x7f00440fe090 msgr2=0x7f00441004d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:53:25.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:53:25.675+0000 7f004ae5b700 1 -- 172.21.15.47:0/1139112519 shutdown_connections 2024-04-17T05:53:25.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:53:25.675+0000 7f004ae5b700 1 -- 172.21.15.47:0/1139112519 wait complete. 2024-04-17T05:53:25.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:53:25.675+0000 7f004ae5b700 1 Processor -- start 2024-04-17T05:53:25.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:53:25.676+0000 7f004ae5b700 1 -- start start 2024-04-17T05:53:25.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:53:25.676+0000 7f004ae5b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0044102af0 0x7f00440767f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:53:25.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:53:25.676+0000 7f004ae5b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0044103cf0 0x7f0044074e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:53:25.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:53:25.676+0000 7f004ae5b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0044104690 0x7f0044075380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:53:25.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:53:25.676+0000 7f004ae5b700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f004410e6c0 con 0x7f0044102af0 2024-04-17T05:53:25.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:53:25.676+0000 7f004ae5b700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f004410e540 con 0x7f0044103cf0 2024-04-17T05:53:25.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:53:25.676+0000 7f004ae5b700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f004410e840 con 0x7f0044104690 2024-04-17T05:53:25.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:53:25.676+0000 7f0043fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0044103cf0 0x7f0044074e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:53:25.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:53:25.676+0000 7f0048bf7700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0044102af0 0x7f00440767f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:53:25.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:53:25.676+0000 7f0043fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0044103cf0 0x7f0044074e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.47:40786/0 (socket says 172.21.15.47:40786) 2024-04-17T05:53:25.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:53:25.676+0000 7f0048bf7700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0044102af0 0x7f00440767f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:42852/0 (socket says 172.21.15.47:42852) 2024-04-17T05:53:25.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:53:25.676+0000 7f0048bf7700 1 -- 172.21.15.47:0/3058403843 learned_addr learned my addr 172.21.15.47:0/3058403843 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:53:25.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:53:25.676+0000 7f00493f8700 1 --2- 172.21.15.47:0/3058403843 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0044104690 0x7f0044075380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:53:25.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:53:25.676+0000 7f0043fff700 1 -- 172.21.15.47:0/3058403843 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0044104690 msgr2=0x7f0044075380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:53:25.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:53:25.676+0000 7f0043fff700 1 --2- 172.21.15.47:0/3058403843 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0044104690 0x7f0044075380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:53:25.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:53:25.676+0000 7f0043fff700 1 -- 172.21.15.47:0/3058403843 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0044102af0 msgr2=0x7f00440767f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:53:25.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:53:25.676+0000 7f0043fff700 1 --2- 172.21.15.47:0/3058403843 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0044102af0 0x7f00440767f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:53:25.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:53:25.676+0000 7f0043fff700 1 -- 172.21.15.47:0/3058403843 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0044075c30 con 0x7f0044103cf0 2024-04-17T05:53:25.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:53:25.676+0000 7f00493f8700 1 --2- 172.21.15.47:0/3058403843 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0044104690 0x7f0044075380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T05:53:25.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:53:25.677+0000 7f0043fff700 1 --2- 172.21.15.47:0/3058403843 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0044103cf0 0x7f0044074e40 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f003000e9f0 tx=0x7f003000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:53:25.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:53:25.677+0000 7f0041ffb700 1 -- 172.21.15.47:0/3058403843 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f003000c9c0 con 0x7f0044103cf0 2024-04-17T05:53:25.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:53:25.677+0000 7f004ae5b700 1 -- 172.21.15.47:0/3058403843 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00441a2da0 con 0x7f0044103cf0 2024-04-17T05:53:25.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:53:25.677+0000 7f0041ffb700 1 -- 172.21.15.47:0/3058403843 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f003000cb60 con 0x7f0044103cf0 2024-04-17T05:53:25.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:53:25.677+0000 7f0041ffb700 1 -- 172.21.15.47:0/3058403843 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0030018680 con 0x7f0044103cf0 2024-04-17T05:53:25.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:53:25.677+0000 7f004ae5b700 1 -- 172.21.15.47:0/3058403843 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00441a3250 con 0x7f0044103cf0 2024-04-17T05:53:25.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:53:25.678+0000 7f0041ffb700 1 -- 172.21.15.47:0/3058403843 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f0030018820 con 0x7f0044103cf0 2024-04-17T05:53:25.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:53:25.678+0000 7f004ae5b700 1 -- 172.21.15.47:0/3058403843 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f004402df90 con 0x7f0044103cf0 2024-04-17T05:53:25.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:53:25.678+0000 7f0041ffb700 1 --2- 172.21.15.47:0/3058403843 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f0034039890 0x7f003403bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:53:25.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:53:25.678+0000 7f0041ffb700 1 -- 172.21.15.47:0/3058403843 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f003004cb90 con 0x7f0044103cf0 2024-04-17T05:53:25.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:53:25.678+0000 7f0048bf7700 1 --2- 172.21.15.47:0/3058403843 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f0034039890 0x7f003403bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:53:25.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:53:25.679+0000 7f0048bf7700 1 --2- 172.21.15.47:0/3058403843 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f0034039890 0x7f003403bd50 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0038009750 tx=0x7f0038009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:53:25.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:53:25.688+0000 7f0041ffb700 1 -- 172.21.15.47:0/3058403843 <== mon.2 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f00300071c0 con 0x7f0044103cf0 2024-04-17T05:53:25.882 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 17 05:53:25 smithi098 bash[37874]: a400e56efbbfad06eec1931950d736347e830916175e8e61fe9389b80f60ebb7 2024-04-17T05:53:25.883 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 17 05:53:25 smithi098 systemd[1]: Started Ceph mgr.y for 565db2d8-fc7e-11ee-bc91-c7b262605968. 2024-04-17T05:53:25.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:53:25.904+0000 7f004ae5b700 1 -- 172.21.15.47:0/3058403843 --> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f0044062e90 con 0x7f0034039890 2024-04-17T05:53:26.383 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 17 05:53:26 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-y[37889]: 2024-04-17T05:53:26.053+0000 7f167f5eb200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-17T05:53:26.383 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 17 05:53:26 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-y[37889]: 2024-04-17T05:53:26.150+0000 7f167f5eb200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-17T05:53:26.643 INFO:journalctl@ceph.mgr.z.smithi103.stdout:Apr 17 05:53:26 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-z[38091]: 2024-04-17T05:53:26.393+0000 7fb119768200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-17T05:53:26.643 INFO:journalctl@ceph.mgr.z.smithi103.stdout:Apr 17 05:53:26 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-z[38091]: 2024-04-17T05:53:26.531+0000 7fb119768200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-17T05:53:26.825 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:26 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:26.826 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:26 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:26.826 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:26 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:26.826 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:26 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:26.826 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:26 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T05:53:26.826 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:26 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-17T05:53:26.826 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:26 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-17T05:53:26.826 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:26 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:53:26.937 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 17 05:53:26 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-y[37889]: 2024-04-17T05:53:26.846+0000 7f167f5eb200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-17T05:53:26.937 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 17 05:53:26 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-y[37889]: 2024-04-17T05:53:26.938+0000 7f167f5eb200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-17T05:53:26.937 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:26 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:26.937 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:26 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:26.937 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:26 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:26.938 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:26 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:26.938 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:26 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T05:53:26.938 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:26 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-17T05:53:26.938 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:26 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-17T05:53:26.938 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:26 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:53:26.957 INFO:journalctl@ceph.mgr.z.smithi103.stdout:Apr 17 05:53:26 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-z[38091]: 2024-04-17T05:53:26.871+0000 7fb119768200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-17T05:53:26.957 INFO:journalctl@ceph.mgr.z.smithi103.stdout:Apr 17 05:53:26 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-z[38091]: 2024-04-17T05:53:26.959+0000 7fb119768200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-17T05:53:26.958 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:26 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:26.958 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:26 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:26.958 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:26 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:26.958 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:26 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:26.958 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:26 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T05:53:26.958 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:26 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-17T05:53:26.958 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:26 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-17T05:53:26.958 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:26 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:53:27.203 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 17 05:53:27 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-y[37889]: 2024-04-17T05:53:27.091+0000 7f167f5eb200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-17T05:53:27.203 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 17 05:53:27 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-y[37889]: 2024-04-17T05:53:27.204+0000 7f167f5eb200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-17T05:53:27.285 INFO:journalctl@ceph.mgr.z.smithi103.stdout:Apr 17 05:53:27 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-z[38091]: 2024-04-17T05:53:27.157+0000 7fb119768200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-17T05:53:27.551 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 17 05:53:27 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-y[37889]: 2024-04-17T05:53:27.287+0000 7f167f5eb200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-17T05:53:27.551 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 17 05:53:27 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-y[37889]: 2024-04-17T05:53:27.553+0000 7f167f5eb200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-17T05:53:27.628 INFO:journalctl@ceph.mgr.z.smithi103.stdout:Apr 17 05:53:27 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-z[38091]: 2024-04-17T05:53:27.347+0000 7fb119768200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-17T05:53:27.781 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:27 smithi047 ceph-mon[32223]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T05:53:27.781 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:27 smithi047 ceph-mon[32223]: from='client.24119 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T05:53:27.781 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:27 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:27.781 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:27 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:27.781 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:27 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:27.781 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:27 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:27.782 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:27 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:53:27.782 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:27 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T05:53:27.782 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:27 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:27.782 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:27 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-17T05:53:27.782 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:27 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-17T05:53:27.782 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:27 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:53:27.840 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 17 05:53:27 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-y[37889]: 2024-04-17T05:53:27.662+0000 7f167f5eb200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-17T05:53:27.841 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 17 05:53:27 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-y[37889]: 2024-04-17T05:53:27.748+0000 7f167f5eb200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-17T05:53:27.841 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 17 05:53:27 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-y[37889]: 2024-04-17T05:53:27.842+0000 7f167f5eb200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-17T05:53:27.841 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:27 smithi098 ceph-mon[36516]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T05:53:27.841 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:27 smithi098 ceph-mon[36516]: from='client.24119 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T05:53:27.841 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:27 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:27.841 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:27 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:27.841 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:27 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:27.842 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:27 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:27.842 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:27 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:53:27.842 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:27 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T05:53:27.842 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:27 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:27.842 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:27 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-17T05:53:27.842 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:27 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-17T05:53:27.842 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:27 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:53:27.885 INFO:journalctl@ceph.mgr.z.smithi103.stdout:Apr 17 05:53:27 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-z[38091]: 2024-04-17T05:53:27.630+0000 7fb119768200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-17T05:53:27.885 INFO:journalctl@ceph.mgr.z.smithi103.stdout:Apr 17 05:53:27 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-z[38091]: 2024-04-17T05:53:27.887+0000 7fb119768200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-17T05:53:27.885 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:27 smithi103 ceph-mon[36243]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T05:53:27.885 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:27 smithi103 ceph-mon[36243]: from='client.24119 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T05:53:27.885 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:27 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:27.885 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:27 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:27.885 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:27 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:27.885 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:27 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:27.885 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:27 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:53:27.886 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:27 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T05:53:27.886 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:27 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:27.886 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:27 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-17T05:53:27.886 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:27 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-17T05:53:27.886 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:27 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:53:28.132 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 17 05:53:28 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-y[37889]: 2024-04-17T05:53:28.021+0000 7f167f5eb200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-17T05:53:28.283 INFO:journalctl@ceph.mgr.z.smithi103.stdout:Apr 17 05:53:28 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-z[38091]: 2024-04-17T05:53:28.120+0000 7fb119768200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-17T05:53:28.912 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:28 smithi103 ceph-mon[36243]: Reconfiguring mgr.x (unknown last config time)... 2024-04-17T05:53:28.912 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:28 smithi103 ceph-mon[36243]: Reconfiguring daemon mgr.x on smithi047 2024-04-17T05:53:28.912 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:28 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/2349699536' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8453f300-533d-4736-a343-55619c08e966"}]: dispatch 2024-04-17T05:53:28.912 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:28 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/2349699536' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8453f300-533d-4736-a343-55619c08e966"}]': finished 2024-04-17T05:53:28.978 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:28 smithi047 ceph-mon[32223]: Reconfiguring mgr.x (unknown last config time)... 2024-04-17T05:53:28.978 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:28 smithi047 ceph-mon[32223]: Reconfiguring daemon mgr.x on smithi047 2024-04-17T05:53:28.978 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:28 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/2349699536' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8453f300-533d-4736-a343-55619c08e966"}]: dispatch 2024-04-17T05:53:28.978 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:28 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/2349699536' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8453f300-533d-4736-a343-55619c08e966"}]': finished 2024-04-17T05:53:28.986 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:28 smithi098 ceph-mon[36516]: Reconfiguring mgr.x (unknown last config time)... 2024-04-17T05:53:28.986 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:28 smithi098 ceph-mon[36516]: Reconfiguring daemon mgr.x on smithi047 2024-04-17T05:53:28.986 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:28 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/2349699536' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8453f300-533d-4736-a343-55619c08e966"}]: dispatch 2024-04-17T05:53:28.986 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:28 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/2349699536' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8453f300-533d-4736-a343-55619c08e966"}]': finished 2024-04-17T05:53:29.284 INFO:journalctl@ceph.mgr.z.smithi103.stdout:Apr 17 05:53:28 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-z[38091]: 2024-04-17T05:53:28.914+0000 7fb119768200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-17T05:53:29.382 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 17 05:53:28 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-y[37889]: 2024-04-17T05:53:28.988+0000 7f167f5eb200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-17T05:53:29.383 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 17 05:53:29 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-y[37889]: 2024-04-17T05:53:29.117+0000 7f167f5eb200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-17T05:53:29.728 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 17 05:53:29 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-y[37889]: 2024-04-17T05:53:29.447+0000 7f167f5eb200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-17T05:53:29.728 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 17 05:53:29 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-y[37889]: 2024-04-17T05:53:29.535+0000 7f167f5eb200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-17T05:53:29.728 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 17 05:53:29 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-y[37889]: 2024-04-17T05:53:29.729+0000 7f167f5eb200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-17T05:53:29.728 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:29 smithi098 ceph-mon[36516]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T05:53:29.728 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:29 smithi098 ceph-mon[36516]: osdmap e5: 1 total, 0 up, 1 in 2024-04-17T05:53:29.728 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:29 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-17T05:53:29.728 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:29 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:29.728 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:29 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/265614496' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-17T05:53:29.784 INFO:journalctl@ceph.mgr.z.smithi103.stdout:Apr 17 05:53:29 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-z[38091]: 2024-04-17T05:53:29.478+0000 7fb119768200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-17T05:53:29.784 INFO:journalctl@ceph.mgr.z.smithi103.stdout:Apr 17 05:53:29 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-z[38091]: 2024-04-17T05:53:29.574+0000 7fb119768200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-17T05:53:29.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:29 smithi103 ceph-mon[36243]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T05:53:29.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:29 smithi103 ceph-mon[36243]: osdmap e5: 1 total, 0 up, 1 in 2024-04-17T05:53:29.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:29 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-17T05:53:29.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:29 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:29.785 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:29 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/265614496' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-17T05:53:30.132 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 17 05:53:29 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-y[37889]: 2024-04-17T05:53:29.913+0000 7f167f5eb200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-17T05:53:30.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:29 smithi047 ceph-mon[32223]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T05:53:30.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:29 smithi047 ceph-mon[32223]: osdmap e5: 1 total, 0 up, 1 in 2024-04-17T05:53:30.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:29 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-17T05:53:30.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:29 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:30.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:29 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/265614496' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-17T05:53:30.283 INFO:journalctl@ceph.mgr.z.smithi103.stdout:Apr 17 05:53:29 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-z[38091]: 2024-04-17T05:53:29.789+0000 7fb119768200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-17T05:53:30.448 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 17 05:53:30 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-y[37889]: 2024-04-17T05:53:30.193+0000 7f167f5eb200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-17T05:53:30.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:53:30.663+0000 7f0041ffb700 1 -- 172.21.15.47:0/3058403843 <== mon.2 v2:172.21.15.98:3300/0 7 ==== mgrmap(e 14) ==== 92602+0+0 (secure 0 0 0) 0x7f00300178f0 con 0x7f0044103cf0 2024-04-17T05:53:30.883 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 17 05:53:30 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-y[37889]: 2024-04-17T05:53:30.450+0000 7f167f5eb200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-17T05:53:30.883 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 17 05:53:30 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-y[37889]: 2024-04-17T05:53:30.686+0000 7f167f5eb200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-17T05:53:30.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:30 smithi098 ceph-mon[36516]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T05:53:30.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:30 smithi098 ceph-mon[36516]: Standby manager daemon z started 2024-04-17T05:53:30.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:30 smithi098 ceph-mon[36516]: from='mgr.? 172.21.15.103:0/2815214803' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-17T05:53:30.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:30 smithi098 ceph-mon[36516]: from='mgr.? 172.21.15.103:0/2815214803' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-17T05:53:30.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:30 smithi098 ceph-mon[36516]: from='mgr.? 172.21.15.103:0/2815214803' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-17T05:53:30.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:30 smithi098 ceph-mon[36516]: from='mgr.? 172.21.15.103:0/2815214803' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-17T05:53:31.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:30 smithi103 ceph-mon[36243]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T05:53:31.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:30 smithi103 ceph-mon[36243]: Standby manager daemon z started 2024-04-17T05:53:31.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:30 smithi103 ceph-mon[36243]: from='mgr.? 172.21.15.103:0/2815214803' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-17T05:53:31.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:30 smithi103 ceph-mon[36243]: from='mgr.? 172.21.15.103:0/2815214803' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-17T05:53:31.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:30 smithi103 ceph-mon[36243]: from='mgr.? 172.21.15.103:0/2815214803' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-17T05:53:31.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:30 smithi103 ceph-mon[36243]: from='mgr.? 172.21.15.103:0/2815214803' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-17T05:53:31.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:30 smithi047 ceph-mon[32223]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T05:53:31.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:30 smithi047 ceph-mon[32223]: Standby manager daemon z started 2024-04-17T05:53:31.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:30 smithi047 ceph-mon[32223]: from='mgr.? 172.21.15.103:0/2815214803' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-17T05:53:31.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:30 smithi047 ceph-mon[32223]: from='mgr.? 172.21.15.103:0/2815214803' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-17T05:53:31.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:30 smithi047 ceph-mon[32223]: from='mgr.? 172.21.15.103:0/2815214803' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-17T05:53:31.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:30 smithi047 ceph-mon[32223]: from='mgr.? 172.21.15.103:0/2815214803' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-17T05:53:31.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:31 smithi098 ceph-mon[36516]: mgrmap e14: x(active, since 77s), standbys: z 2024-04-17T05:53:31.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:31 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-17T05:53:31.883 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 17 05:53:31 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-y[37889]: 2024-04-17T05:53:31.485+0000 7f167f5eb200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-17T05:53:32.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:31 smithi103 ceph-mon[36243]: mgrmap e14: x(active, since 77s), standbys: z 2024-04-17T05:53:32.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:31 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-17T05:53:32.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:31 smithi047 ceph-mon[32223]: mgrmap e14: x(active, since 77s), standbys: z 2024-04-17T05:53:32.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:31 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-17T05:53:32.354 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 17 05:53:32 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-y[37889]: 2024-04-17T05:53:32.049+0000 7f167f5eb200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-17T05:53:32.355 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 17 05:53:32 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-y[37889]: 2024-04-17T05:53:32.144+0000 7f167f5eb200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-17T05:53:32.355 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 17 05:53:32 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-y[37889]: 2024-04-17T05:53:32.356+0000 7f167f5eb200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-17T05:53:32.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:53:32.722+0000 7f0041ffb700 1 -- 172.21.15.47:0/3058403843 <== mon.2 v2:172.21.15.98:3300/0 8 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f003000fca0 con 0x7f0044103cf0 2024-04-17T05:53:33.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:32 smithi103 ceph-mon[36243]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T05:53:33.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:32 smithi103 ceph-mon[36243]: Standby manager daemon y started 2024-04-17T05:53:33.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:32 smithi103 ceph-mon[36243]: from='mgr.? 172.21.15.98:0/2958769170' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-17T05:53:33.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:32 smithi103 ceph-mon[36243]: from='mgr.? 172.21.15.98:0/2958769170' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-17T05:53:33.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:32 smithi103 ceph-mon[36243]: from='mgr.? 172.21.15.98:0/2958769170' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-17T05:53:33.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:32 smithi103 ceph-mon[36243]: from='mgr.? 172.21.15.98:0/2958769170' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-17T05:53:33.079 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:32 smithi047 ceph-mon[32223]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T05:53:33.079 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:32 smithi047 ceph-mon[32223]: Standby manager daemon y started 2024-04-17T05:53:33.079 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:32 smithi047 ceph-mon[32223]: from='mgr.? 172.21.15.98:0/2958769170' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-17T05:53:33.079 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:32 smithi047 ceph-mon[32223]: from='mgr.? 172.21.15.98:0/2958769170' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-17T05:53:33.079 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:32 smithi047 ceph-mon[32223]: from='mgr.? 172.21.15.98:0/2958769170' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-17T05:53:33.079 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:32 smithi047 ceph-mon[32223]: from='mgr.? 172.21.15.98:0/2958769170' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-17T05:53:33.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:32 smithi098 ceph-mon[36516]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T05:53:33.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:32 smithi098 ceph-mon[36516]: Standby manager daemon y started 2024-04-17T05:53:33.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:32 smithi098 ceph-mon[36516]: from='mgr.? 172.21.15.98:0/2958769170' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-17T05:53:33.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:32 smithi098 ceph-mon[36516]: from='mgr.? 172.21.15.98:0/2958769170' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-17T05:53:33.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:32 smithi098 ceph-mon[36516]: from='mgr.? 172.21.15.98:0/2958769170' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-17T05:53:33.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:32 smithi098 ceph-mon[36516]: from='mgr.? 172.21.15.98:0/2958769170' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-17T05:53:34.026 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:33 smithi047 ceph-mon[32223]: mgrmap e15: x(active, since 79s), standbys: z, y 2024-04-17T05:53:34.026 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:33 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-17T05:53:34.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:33 smithi103 ceph-mon[36243]: mgrmap e15: x(active, since 79s), standbys: z, y 2024-04-17T05:53:34.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:33 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-17T05:53:34.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:33 smithi098 ceph-mon[36516]: mgrmap e15: x(active, since 79s), standbys: z, y 2024-04-17T05:53:34.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:33 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-17T05:53:35.483 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:35 smithi047 ceph-mon[32223]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T05:53:35.483 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:35 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:35.483 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:35 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:35.483 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:35 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T05:53:35.483 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:35 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:53:35.483 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:35 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T05:53:35.483 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:35 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:35.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:35 smithi098 ceph-mon[36516]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T05:53:35.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:35 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:35.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:35 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:35.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:35 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T05:53:35.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:35 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:53:35.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:35 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T05:53:35.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:35 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:35.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:35 smithi103 ceph-mon[36243]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T05:53:35.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:35 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:35.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:35 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:35.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:35 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T05:53:35.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:35 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:53:35.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:35 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T05:53:35.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:35 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:37.638 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:37 smithi047 ceph-mon[32223]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T05:53:37.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:37 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-17T05:53:37.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:37 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:53:37.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:37 smithi103 ceph-mon[36243]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T05:53:37.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:37 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-17T05:53:37.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:37 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:53:37.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:37 smithi098 ceph-mon[36516]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T05:53:37.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:37 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-17T05:53:37.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:37 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:53:38.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:38 smithi047 ceph-mon[32223]: Deploying daemon osd.0 on smithi047 2024-04-17T05:53:38.783 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:38 smithi103 ceph-mon[36243]: Deploying daemon osd.0 on smithi047 2024-04-17T05:53:38.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:38 smithi098 ceph-mon[36516]: Deploying daemon osd.0 on smithi047 2024-04-17T05:53:39.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:39 smithi047 ceph-mon[32223]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T05:53:39.783 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:39 smithi103 ceph-mon[36243]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T05:53:39.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:39 smithi098 ceph-mon[36516]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T05:53:41.620 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:41 smithi047 ceph-mon[32223]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T05:53:41.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:41 smithi103 ceph-mon[36243]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T05:53:41.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:41 smithi098 ceph-mon[36516]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T05:53:43.687 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:43 smithi047 ceph-mon[32223]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T05:53:43.688 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:43 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:43.688 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:43 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:43.688 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:43 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T05:53:43.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:43 smithi103 ceph-mon[36243]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T05:53:43.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:43 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:43.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:43 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:43.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:43 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T05:53:43.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:43 smithi098 ceph-mon[36516]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T05:53:43.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:43 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:43.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:43 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:43.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:43 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T05:53:45.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:45 smithi103 ceph-mon[36243]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T05:53:45.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:45 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:45.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:45 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:45.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:45 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:53:45.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:45 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T05:53:45.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:45 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:45.873 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:45 smithi047 ceph-mon[32223]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T05:53:45.873 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:45 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:45.874 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:45 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:45.874 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:45 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:53:45.874 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:45 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T05:53:45.874 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:45 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:45.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:45 smithi098 ceph-mon[36516]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T05:53:45.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:45 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:45.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:45 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:45.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:45 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:53:45.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:45 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T05:53:45.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:45 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:46.187 INFO:teuthology.orchestra.run.smithi047.stdout:Created osd(s) 0 on host 'smithi047' 2024-04-17T05:53:46.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:53:46.188+0000 7f0041ffb700 1 -- 172.21.15.47:0/3058403843 <== mgr.14152 v2:172.21.15.47:6800/3157902806 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f0044062e90 con 0x7f0034039890 2024-04-17T05:53:46.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:53:46.190+0000 7f004ae5b700 1 -- 172.21.15.47:0/3058403843 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f0034039890 msgr2=0x7f003403bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:53:46.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:53:46.190+0000 7f004ae5b700 1 --2- 172.21.15.47:0/3058403843 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f0034039890 0x7f003403bd50 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0038009750 tx=0x7f0038009310 comp rx=0 tx=0).stop 2024-04-17T05:53:46.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:53:46.190+0000 7f004ae5b700 1 -- 172.21.15.47:0/3058403843 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0044103cf0 msgr2=0x7f0044074e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:53:46.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:53:46.190+0000 7f004ae5b700 1 --2- 172.21.15.47:0/3058403843 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0044103cf0 0x7f0044074e40 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f003000e9f0 tx=0x7f003000ed00 comp rx=0 tx=0).stop 2024-04-17T05:53:46.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:53:46.190+0000 7f004ae5b700 1 -- 172.21.15.47:0/3058403843 shutdown_connections 2024-04-17T05:53:46.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:53:46.190+0000 7f004ae5b700 1 --2- 172.21.15.47:0/3058403843 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f0034039890 0x7f003403bd50 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:53:46.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:53:46.190+0000 7f004ae5b700 1 --2- 172.21.15.47:0/3058403843 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0044104690 0x7f0044075380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:53:46.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:53:46.190+0000 7f004ae5b700 1 --2- 172.21.15.47:0/3058403843 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0044102af0 0x7f00440767f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:53:46.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:53:46.190+0000 7f004ae5b700 1 --2- 172.21.15.47:0/3058403843 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0044103cf0 0x7f0044074e40 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:53:46.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:53:46.190+0000 7f004ae5b700 1 -- 172.21.15.47:0/3058403843 >> 172.21.15.47:0/3058403843 conn(0x7f00440fe090 msgr2=0x7f0044106f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:53:46.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:53:46.190+0000 7f004ae5b700 1 -- 172.21.15.47:0/3058403843 shutdown_connections 2024-04-17T05:53:46.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:53:46.190+0000 7f004ae5b700 1 -- 172.21.15.47:0/3058403843 wait complete. 2024-04-17T05:53:46.837 DEBUG:teuthology.orchestra.run.smithi047:osd.0> sudo journalctl -f -n 0 -u ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@osd.0.service 2024-04-17T05:53:46.839 INFO:tasks.cephadm:Deploying osd.1 on smithi098 with /dev/vg_nvme/lv_4... 2024-04-17T05:53:46.839 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 565db2d8-fc7e-11ee-bc91-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-17T05:53:47.084 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/mon.b/config 2024-04-17T05:53:47.471 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:47 smithi098 ceph-mon[36516]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T05:53:47.471 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:47 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:47.471 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:47 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:47.471 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:47 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T05:53:47.471 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:47 smithi098 ceph-mon[36516]: from='osd.0 [v2:172.21.15.47:6802/1290127544,v1:172.21.15.47:6803/1290127544]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-17T05:53:47.533 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:47 smithi103 ceph-mon[36243]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T05:53:47.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:47 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:47.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:47 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:47.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:47 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T05:53:47.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:47 smithi103 ceph-mon[36243]: from='osd.0 [v2:172.21.15.47:6802/1290127544,v1:172.21.15.47:6803/1290127544]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-17T05:53:47.638 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:47 smithi047 ceph-mon[32223]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T05:53:47.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:47 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:47.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:47 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:47.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:47 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T05:53:47.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:47 smithi047 ceph-mon[32223]: from='osd.0 [v2:172.21.15.47:6802/1290127544,v1:172.21.15.47:6803/1290127544]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-17T05:53:48.353 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:48 smithi098 ceph-mon[36516]: from='osd.0 [v2:172.21.15.47:6802/1290127544,v1:172.21.15.47:6803/1290127544]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-17T05:53:48.354 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:48 smithi098 ceph-mon[36516]: osdmap e6: 1 total, 0 up, 1 in 2024-04-17T05:53:48.354 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:48 smithi098 ceph-mon[36516]: from='osd.0 [v2:172.21.15.47:6802/1290127544,v1:172.21.15.47:6803/1290127544]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-04-17T05:53:48.354 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:48 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-17T05:53:48.367 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:48 smithi047 ceph-mon[32223]: from='osd.0 [v2:172.21.15.47:6802/1290127544,v1:172.21.15.47:6803/1290127544]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-17T05:53:48.367 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:48 smithi047 ceph-mon[32223]: osdmap e6: 1 total, 0 up, 1 in 2024-04-17T05:53:48.367 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:48 smithi047 ceph-mon[32223]: from='osd.0 [v2:172.21.15.47:6802/1290127544,v1:172.21.15.47:6803/1290127544]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-04-17T05:53:48.367 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:48 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-17T05:53:48.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:48 smithi103 ceph-mon[36243]: from='osd.0 [v2:172.21.15.47:6802/1290127544,v1:172.21.15.47:6803/1290127544]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-17T05:53:48.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:48 smithi103 ceph-mon[36243]: osdmap e6: 1 total, 0 up, 1 in 2024-04-17T05:53:48.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:48 smithi103 ceph-mon[36243]: from='osd.0 [v2:172.21.15.47:6802/1290127544,v1:172.21.15.47:6803/1290127544]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-04-17T05:53:48.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:48 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-17T05:53:48.907 INFO:teuthology.orchestra.run.smithi098.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-17T05:53:48.907 INFO:teuthology.orchestra.run.smithi098.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-17T05:53:48.907 INFO:teuthology.orchestra.run.smithi098.stderr: stderr: 10+0 records in 2024-04-17T05:53:48.907 INFO:teuthology.orchestra.run.smithi098.stderr:10+0 records out 2024-04-17T05:53:48.907 INFO:teuthology.orchestra.run.smithi098.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.011275 s, 930 MB/s 2024-04-17T05:53:48.907 INFO:teuthology.orchestra.run.smithi098.stderr:--> Zapping successful for: 2024-04-17T05:53:49.139 INFO:journalctl@ceph.osd.0.smithi047.stdout:Apr 17 05:53:48 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-0[46923]: 2024-04-17T05:53:48.655+0000 7f4c87d15700 -1 osd.0 0 waiting for initial osdmap 2024-04-17T05:53:49.139 INFO:journalctl@ceph.osd.0.smithi047.stdout:Apr 17 05:53:48 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-0[46923]: 2024-04-17T05:53:48.658+0000 7f4c7fb00700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-17T05:53:49.481 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:49 smithi098 ceph-mon[36516]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T05:53:49.482 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:49 smithi098 ceph-mon[36516]: from='osd.0 [v2:172.21.15.47:6802/1290127544,v1:172.21.15.47:6803/1290127544]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]': finished 2024-04-17T05:53:49.482 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:49 smithi098 ceph-mon[36516]: osdmap e7: 1 total, 0 up, 1 in 2024-04-17T05:53:49.482 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:49 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-17T05:53:49.482 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:49 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-17T05:53:49.482 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:49 smithi098 ceph-mon[36516]: from='osd.0 [v2:172.21.15.47:6802/1290127544,v1:172.21.15.47:6803/1290127544]' entity='osd.0' 2024-04-17T05:53:49.508 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 565db2d8-fc7e-11ee-bc91-c7b262605968 -- ceph orch daemon add osd smithi098:vg_nvme/lv_4 2024-04-17T05:53:49.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:49 smithi103 ceph-mon[36243]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T05:53:49.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:49 smithi103 ceph-mon[36243]: from='osd.0 [v2:172.21.15.47:6802/1290127544,v1:172.21.15.47:6803/1290127544]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]': finished 2024-04-17T05:53:49.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:49 smithi103 ceph-mon[36243]: osdmap e7: 1 total, 0 up, 1 in 2024-04-17T05:53:49.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:49 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-17T05:53:49.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:49 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-17T05:53:49.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:49 smithi103 ceph-mon[36243]: from='osd.0 [v2:172.21.15.47:6802/1290127544,v1:172.21.15.47:6803/1290127544]' entity='osd.0' 2024-04-17T05:53:49.607 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:49 smithi047 ceph-mon[32223]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T05:53:49.607 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:49 smithi047 ceph-mon[32223]: from='osd.0 [v2:172.21.15.47:6802/1290127544,v1:172.21.15.47:6803/1290127544]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]': finished 2024-04-17T05:53:49.607 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:49 smithi047 ceph-mon[32223]: osdmap e7: 1 total, 0 up, 1 in 2024-04-17T05:53:49.608 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:49 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-17T05:53:49.608 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:49 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-17T05:53:49.608 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:49 smithi047 ceph-mon[32223]: from='osd.0 [v2:172.21.15.47:6802/1290127544,v1:172.21.15.47:6803/1290127544]' entity='osd.0' 2024-04-17T05:53:49.750 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/mon.b/config 2024-04-17T05:53:50.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:50 smithi098 ceph-mon[36516]: purged_snaps scrub starts 2024-04-17T05:53:50.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:50 smithi098 ceph-mon[36516]: purged_snaps scrub ok 2024-04-17T05:53:50.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:50 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-17T05:53:50.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:50 smithi098 ceph-mon[36516]: osd.0 [v2:172.21.15.47:6802/1290127544,v1:172.21.15.47:6803/1290127544] boot 2024-04-17T05:53:50.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:50 smithi098 ceph-mon[36516]: osdmap e8: 1 total, 1 up, 1 in 2024-04-17T05:53:50.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:50 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-17T05:53:50.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:50 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:50.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:50 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:50.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:50 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-17T05:53:50.384 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:50 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:50.384 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:50 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:53:50.384 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:50 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T05:53:50.384 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:50 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:50.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:50 smithi103 ceph-mon[36243]: purged_snaps scrub starts 2024-04-17T05:53:50.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:50 smithi103 ceph-mon[36243]: purged_snaps scrub ok 2024-04-17T05:53:50.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:50 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-17T05:53:50.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:50 smithi103 ceph-mon[36243]: osd.0 [v2:172.21.15.47:6802/1290127544,v1:172.21.15.47:6803/1290127544] boot 2024-04-17T05:53:50.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:50 smithi103 ceph-mon[36243]: osdmap e8: 1 total, 1 up, 1 in 2024-04-17T05:53:50.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:50 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-17T05:53:50.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:50 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:50.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:50 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:50.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:50 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-17T05:53:50.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:50 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:50.535 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:50 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:53:50.535 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:50 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T05:53:50.535 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:50 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:50.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:50 smithi047 ceph-mon[32223]: purged_snaps scrub starts 2024-04-17T05:53:50.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:50 smithi047 ceph-mon[32223]: purged_snaps scrub ok 2024-04-17T05:53:50.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:50 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-17T05:53:50.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:50 smithi047 ceph-mon[32223]: osd.0 [v2:172.21.15.47:6802/1290127544,v1:172.21.15.47:6803/1290127544] boot 2024-04-17T05:53:50.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:50 smithi047 ceph-mon[32223]: osdmap e8: 1 total, 1 up, 1 in 2024-04-17T05:53:50.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:50 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-17T05:53:50.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:50 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:50.640 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:50 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:50.640 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:50 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-17T05:53:50.640 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:50 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:50.640 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:50 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:53:50.640 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:50 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T05:53:50.640 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:50 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:53:51.194 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:53:51.195+0000 7f7a0ba20700 1 -- 172.21.15.98:0/898344441 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7a041025b0 msgr2=0x7f7a04109e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:53:51.194 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:53:51.195+0000 7f7a0ba20700 1 --2- 172.21.15.98:0/898344441 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7a041025b0 0x7f7a04109e50 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f79fc00b0a0 tx=0x7f79fc00b3b0 comp rx=0 tx=0).stop 2024-04-17T05:53:51.194 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:53:51.195+0000 7f7a0ba20700 1 -- 172.21.15.98:0/898344441 shutdown_connections 2024-04-17T05:53:51.194 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:53:51.195+0000 7f7a0ba20700 1 --2- 172.21.15.98:0/898344441 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7a041025b0 0x7f7a04109e50 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:53:51.194 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:53:51.195+0000 7f7a0ba20700 1 --2- 172.21.15.98:0/898344441 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a04103020 0x7f7a04103440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:53:51.194 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:53:51.195+0000 7f7a0ba20700 1 --2- 172.21.15.98:0/898344441 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7a04101bf0 0x7f7a04102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:53:51.194 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:53:51.195+0000 7f7a0ba20700 1 -- 172.21.15.98:0/898344441 >> 172.21.15.98:0/898344441 conn(0x7f7a040fc6e0 msgr2=0x7f7a040feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:53:51.195 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:53:51.195+0000 7f7a0ba20700 1 -- 172.21.15.98:0/898344441 shutdown_connections 2024-04-17T05:53:51.195 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:53:51.195+0000 7f7a0ba20700 1 -- 172.21.15.98:0/898344441 wait complete. 2024-04-17T05:53:51.197 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:53:51.195+0000 7f7a0ba20700 1 Processor -- start 2024-04-17T05:53:51.197 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:53:51.196+0000 7f7a0ba20700 1 -- start start 2024-04-17T05:53:51.197 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:53:51.196+0000 7f7a0ba20700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7a04101bf0 0x7f7a04197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:53:51.197 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:53:51.196+0000 7f7a0ba20700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a041025b0 0x7f7a04198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:53:51.197 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:53:51.196+0000 7f7a0ba20700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7a04103020 0x7f7a0419f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:53:51.197 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:53:51.196+0000 7f7a0ba20700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f7a0410d590 con 0x7f7a041025b0 2024-04-17T05:53:51.197 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:53:51.196+0000 7f7a0ba20700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f7a0410d410 con 0x7f7a04103020 2024-04-17T05:53:51.197 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:53:51.196+0000 7f7a0ba20700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7a0410d710 con 0x7f7a04101bf0 2024-04-17T05:53:51.197 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:53:51.196+0000 7f7a09fbd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7a04103020 0x7f7a0419f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:53:51.197 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:53:51.196+0000 7f7a09fbd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7a04103020 0x7f7a0419f0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41210/0 (socket says 172.21.15.98:41210) 2024-04-17T05:53:51.197 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:53:51.196+0000 7f7a09fbd700 1 -- 172.21.15.98:0/3754825932 learned_addr learned my addr 172.21.15.98:0/3754825932 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-04-17T05:53:51.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:53:51.196+0000 7f7a08fbb700 1 --2- 172.21.15.98:0/3754825932 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a041025b0 0x7f7a04198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:53:51.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:53:51.196+0000 7f7a097bc700 1 --2- 172.21.15.98:0/3754825932 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7a04101bf0 0x7f7a04197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:53:51.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:53:51.196+0000 7f7a09fbd700 1 -- 172.21.15.98:0/3754825932 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7a04101bf0 msgr2=0x7f7a04197b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:53:51.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:53:51.196+0000 7f7a09fbd700 1 --2- 172.21.15.98:0/3754825932 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7a04101bf0 0x7f7a04197b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:53:51.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:53:51.196+0000 7f7a09fbd700 1 -- 172.21.15.98:0/3754825932 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a041025b0 msgr2=0x7f7a04198060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:53:51.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:53:51.196+0000 7f7a09fbd700 1 --2- 172.21.15.98:0/3754825932 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a041025b0 0x7f7a04198060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:53:51.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:53:51.196+0000 7f7a09fbd700 1 -- 172.21.15.98:0/3754825932 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a0419f780 con 0x7f7a04103020 2024-04-17T05:53:51.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:53:51.196+0000 7f7a09fbd700 1 --2- 172.21.15.98:0/3754825932 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7a04103020 0x7f7a0419f0e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f79fc007f00 tx=0x7f79fc004dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:53:51.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:53:51.196+0000 7f79fa7fc700 1 -- 172.21.15.98:0/3754825932 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79fc021070 con 0x7f7a04103020 2024-04-17T05:53:51.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:53:51.197+0000 7f79fa7fc700 1 -- 172.21.15.98:0/3754825932 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79fc007b80 con 0x7f7a04103020 2024-04-17T05:53:51.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:53:51.197+0000 7f7a0ba20700 1 -- 172.21.15.98:0/3754825932 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a04100ae0 con 0x7f7a04103020 2024-04-17T05:53:51.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:53:51.197+0000 7f79fa7fc700 1 -- 172.21.15.98:0/3754825932 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79fc01d450 con 0x7f7a04103020 2024-04-17T05:53:51.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:53:51.197+0000 7f7a0ba20700 1 -- 172.21.15.98:0/3754825932 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a04101070 con 0x7f7a04103020 2024-04-17T05:53:51.199 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:53:51.198+0000 7f7a0ba20700 1 -- 172.21.15.98:0/3754825932 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a04069740 con 0x7f7a04103020 2024-04-17T05:53:51.199 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:53:51.199+0000 7f79fa7fc700 1 -- 172.21.15.98:0/3754825932 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f79fc004140 con 0x7f7a04103020 2024-04-17T05:53:51.199 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:53:51.199+0000 7f79fa7fc700 1 --2- 172.21.15.98:0/3754825932 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f79f00a3c30 0x7f79f00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:53:51.199 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:53:51.199+0000 7f79fa7fc700 1 -- 172.21.15.98:0/3754825932 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7f79fc00f020 con 0x7f7a04103020 2024-04-17T05:53:51.200 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:53:51.199+0000 7f7a097bc700 1 --2- 172.21.15.98:0/3754825932 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f79f00a3c30 0x7f79f00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:53:51.200 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:53:51.200+0000 7f7a097bc700 1 --2- 172.21.15.98:0/3754825932 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f79f00a3c30 0x7f79f00a60f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7a00005950 tx=0x7f7a0000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:53:51.203 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:53:51.204+0000 7f79fa7fc700 1 -- 172.21.15.98:0/3754825932 <== mon.2 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f79fc02a030 con 0x7f7a04103020 2024-04-17T05:53:51.382 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:51 smithi098 ceph-mon[36516]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T05:53:51.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:51 smithi098 ceph-mon[36516]: Detected new or changed devices on smithi047 2024-04-17T05:53:51.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:51 smithi098 ceph-mon[36516]: Adjusting osd_memory_target on smithi047 to 17093M 2024-04-17T05:53:51.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:53:51.411+0000 7f7a0ba20700 1 -- 172.21.15.98:0/3754825932 --> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f7a04101390 con 0x7f79f00a3c30 2024-04-17T05:53:51.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:51 smithi103 ceph-mon[36243]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T05:53:51.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:51 smithi103 ceph-mon[36243]: Detected new or changed devices on smithi047 2024-04-17T05:53:51.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:51 smithi103 ceph-mon[36243]: Adjusting osd_memory_target on smithi047 to 17093M 2024-04-17T05:53:51.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:51 smithi047 ceph-mon[32223]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T05:53:51.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:51 smithi047 ceph-mon[32223]: Detected new or changed devices on smithi047 2024-04-17T05:53:51.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:51 smithi047 ceph-mon[32223]: Adjusting osd_memory_target on smithi047 to 17093M 2024-04-17T05:53:52.295 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:52 smithi098 ceph-mon[36516]: osdmap e9: 1 total, 1 up, 1 in 2024-04-17T05:53:52.295 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:52 smithi098 ceph-mon[36516]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T05:53:52.296 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:52 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-17T05:53:52.296 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:52 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-17T05:53:52.296 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:52 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:53:52.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:52 smithi103 ceph-mon[36243]: osdmap e9: 1 total, 1 up, 1 in 2024-04-17T05:53:52.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:52 smithi103 ceph-mon[36243]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T05:53:52.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:52 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-17T05:53:52.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:52 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-17T05:53:52.535 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:52 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:53:52.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:52 smithi047 ceph-mon[32223]: osdmap e9: 1 total, 1 up, 1 in 2024-04-17T05:53:52.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:52 smithi047 ceph-mon[32223]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T05:53:52.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:52 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-17T05:53:52.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:52 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-17T05:53:52.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:52 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:53:53.533 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:53 smithi103 ceph-mon[36243]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-17T05:53:53.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:53 smithi098 ceph-mon[36516]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-17T05:53:53.638 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:53 smithi047 ceph-mon[32223]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-17T05:53:54.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:54 smithi103 ceph-mon[36243]: from='client.? 172.21.15.98:0/1507968357' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "80b0f03e-c1bc-4658-8028-1e6b8eedcc47"}]: dispatch 2024-04-17T05:53:54.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:54 smithi103 ceph-mon[36243]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "80b0f03e-c1bc-4658-8028-1e6b8eedcc47"}]: dispatch 2024-04-17T05:53:54.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:54 smithi103 ceph-mon[36243]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "80b0f03e-c1bc-4658-8028-1e6b8eedcc47"}]': finished 2024-04-17T05:53:54.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:54 smithi103 ceph-mon[36243]: osdmap e10: 2 total, 1 up, 2 in 2024-04-17T05:53:54.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:54 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-17T05:53:54.534 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:54 smithi098 ceph-mon[36516]: from='client.? 172.21.15.98:0/1507968357' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "80b0f03e-c1bc-4658-8028-1e6b8eedcc47"}]: dispatch 2024-04-17T05:53:54.534 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:54 smithi098 ceph-mon[36516]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "80b0f03e-c1bc-4658-8028-1e6b8eedcc47"}]: dispatch 2024-04-17T05:53:54.535 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:54 smithi098 ceph-mon[36516]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "80b0f03e-c1bc-4658-8028-1e6b8eedcc47"}]': finished 2024-04-17T05:53:54.535 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:54 smithi098 ceph-mon[36516]: osdmap e10: 2 total, 1 up, 2 in 2024-04-17T05:53:54.535 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:54 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-17T05:53:54.638 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:54 smithi047 ceph-mon[32223]: from='client.? 172.21.15.98:0/1507968357' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "80b0f03e-c1bc-4658-8028-1e6b8eedcc47"}]: dispatch 2024-04-17T05:53:54.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:54 smithi047 ceph-mon[32223]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "80b0f03e-c1bc-4658-8028-1e6b8eedcc47"}]: dispatch 2024-04-17T05:53:54.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:54 smithi047 ceph-mon[32223]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "80b0f03e-c1bc-4658-8028-1e6b8eedcc47"}]': finished 2024-04-17T05:53:54.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:54 smithi047 ceph-mon[32223]: osdmap e10: 2 total, 1 up, 2 in 2024-04-17T05:53:54.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:54 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-17T05:53:55.533 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:55 smithi103 ceph-mon[36243]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-17T05:53:55.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:55 smithi103 ceph-mon[36243]: from='client.? 172.21.15.98:0/2040593248' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-17T05:53:55.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:55 smithi098 ceph-mon[36516]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-17T05:53:55.543 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:55 smithi098 ceph-mon[36516]: from='client.? 172.21.15.98:0/2040593248' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-17T05:53:55.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:55 smithi047 ceph-mon[32223]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-17T05:53:55.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:55 smithi047 ceph-mon[32223]: from='client.? 172.21.15.98:0/2040593248' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-17T05:53:57.533 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:57 smithi103 ceph-mon[36243]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-17T05:53:57.570 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:57 smithi098 ceph-mon[36516]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-17T05:53:57.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:57 smithi047 ceph-mon[32223]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-17T05:53:59.380 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:53:59 smithi098 ceph-mon[36516]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-17T05:53:59.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:53:59 smithi103 ceph-mon[36243]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-17T05:53:59.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:53:59 smithi047 ceph-mon[32223]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-17T05:54:01.533 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:01 smithi103 ceph-mon[36243]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-17T05:54:01.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:01 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-17T05:54:01.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:01 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:54:01.537 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:01 smithi098 ceph-mon[36516]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-17T05:54:01.537 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:01 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-17T05:54:01.537 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:01 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:54:01.638 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:01 smithi047 ceph-mon[32223]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-17T05:54:01.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:01 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-17T05:54:01.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:01 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:54:02.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:02 smithi103 ceph-mon[36243]: Deploying daemon osd.1 on smithi098 2024-04-17T05:54:02.534 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:02 smithi098 ceph-mon[36516]: Deploying daemon osd.1 on smithi098 2024-04-17T05:54:02.638 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:02 smithi047 ceph-mon[32223]: Deploying daemon osd.1 on smithi098 2024-04-17T05:54:03.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:03 smithi098 ceph-mon[36516]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-17T05:54:03.533 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:03 smithi103 ceph-mon[36243]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-17T05:54:03.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:03 smithi047 ceph-mon[32223]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-17T05:54:05.443 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:05 smithi098 ceph-mon[36516]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-17T05:54:05.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:05 smithi103 ceph-mon[36243]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-17T05:54:05.638 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:05 smithi047 ceph-mon[32223]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-17T05:54:07.245 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:07 smithi098 ceph-mon[36516]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-17T05:54:07.245 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:07 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:54:07.245 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:07 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T05:54:07.245 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:07 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:54:07.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:07 smithi047 ceph-mon[32223]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-17T05:54:07.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:07 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:54:07.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:07 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T05:54:07.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:07 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:54:07.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:07 smithi103 ceph-mon[36243]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-17T05:54:07.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:07 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:54:07.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:07 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T05:54:07.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:07 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:54:09.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:09 smithi098 ceph-mon[36516]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-17T05:54:09.174 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:54:09.175+0000 7f79fa7fc700 1 -- 172.21.15.98:0/3754825932 <== mgr.14152 v2:172.21.15.47:6800/3157902806 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f7a04101390 con 0x7f79f00a3c30 2024-04-17T05:54:09.174 INFO:teuthology.orchestra.run.smithi098.stdout:Created osd(s) 1 on host 'smithi098' 2024-04-17T05:54:09.176 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:54:09.178+0000 7f7a0ba20700 1 -- 172.21.15.98:0/3754825932 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f79f00a3c30 msgr2=0x7f79f00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:54:09.176 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:54:09.178+0000 7f7a0ba20700 1 --2- 172.21.15.98:0/3754825932 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f79f00a3c30 0x7f79f00a60f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7a00005950 tx=0x7f7a0000b410 comp rx=0 tx=0).stop 2024-04-17T05:54:09.177 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:54:09.178+0000 7f7a0ba20700 1 -- 172.21.15.98:0/3754825932 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7a04103020 msgr2=0x7f7a0419f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:54:09.177 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:54:09.178+0000 7f7a0ba20700 1 --2- 172.21.15.98:0/3754825932 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7a04103020 0x7f7a0419f0e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f79fc007f00 tx=0x7f79fc004dd0 comp rx=0 tx=0).stop 2024-04-17T05:54:09.177 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:54:09.178+0000 7f7a0ba20700 1 -- 172.21.15.98:0/3754825932 shutdown_connections 2024-04-17T05:54:09.177 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:54:09.178+0000 7f7a0ba20700 1 --2- 172.21.15.98:0/3754825932 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f79f00a3c30 0x7f79f00a60f0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:54:09.177 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:54:09.178+0000 7f7a0ba20700 1 --2- 172.21.15.98:0/3754825932 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7a04103020 0x7f7a0419f0e0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:54:09.177 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:54:09.178+0000 7f7a0ba20700 1 --2- 172.21.15.98:0/3754825932 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7a04101bf0 0x7f7a04197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:54:09.177 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:54:09.178+0000 7f7a0ba20700 1 --2- 172.21.15.98:0/3754825932 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a041025b0 0x7f7a04198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:54:09.177 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:54:09.178+0000 7f7a0ba20700 1 -- 172.21.15.98:0/3754825932 >> 172.21.15.98:0/3754825932 conn(0x7f7a040fc6e0 msgr2=0x7f7a04105ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:54:09.177 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:54:09.179+0000 7f7a0ba20700 1 -- 172.21.15.98:0/3754825932 shutdown_connections 2024-04-17T05:54:09.177 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:54:09.179+0000 7f7a0ba20700 1 -- 172.21.15.98:0/3754825932 wait complete. 2024-04-17T05:54:09.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:09 smithi047 ceph-mon[32223]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-17T05:54:09.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:09 smithi103 ceph-mon[36243]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-17T05:54:09.757 DEBUG:teuthology.orchestra.run.smithi098:osd.1> sudo journalctl -f -n 0 -u ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@osd.1.service 2024-04-17T05:54:09.759 INFO:tasks.cephadm:Deploying osd.2 on smithi103 with /dev/vg_nvme/lv_4... 2024-04-17T05:54:09.759 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 565db2d8-fc7e-11ee-bc91-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-17T05:54:10.013 INFO:journalctl@ceph.osd.1.smithi098.stdout:Apr 17 05:54:09 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-1[41120]: 2024-04-17T05:54:09.909+0000 7fbe893657c0 -1 osd.1 0 log_to_monitors true 2024-04-17T05:54:10.019 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/mon.c/config 2024-04-17T05:54:10.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:10 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:54:10.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:10 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:54:10.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:10 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:54:10.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:10 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:54:10.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:10 smithi098 ceph-mon[36516]: from='osd.1 [v2:172.21.15.98:6800/1180430954,v1:172.21.15.98:6801/1180430954]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-17T05:54:10.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:10 smithi098 ceph-mon[36516]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-17T05:54:10.388 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:10 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:54:10.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:10 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:54:10.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:10 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:54:10.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:10 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:54:10.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:10 smithi047 ceph-mon[32223]: from='osd.1 [v2:172.21.15.98:6800/1180430954,v1:172.21.15.98:6801/1180430954]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-17T05:54:10.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:10 smithi047 ceph-mon[32223]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-17T05:54:10.422 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:10 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:54:10.422 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:10 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:54:10.422 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:10 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:54:10.422 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:10 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:54:10.422 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:10 smithi103 ceph-mon[36243]: from='osd.1 [v2:172.21.15.98:6800/1180430954,v1:172.21.15.98:6801/1180430954]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-17T05:54:10.422 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:10 smithi103 ceph-mon[36243]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-17T05:54:11.333 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:11 smithi103 ceph-mon[36243]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-17T05:54:11.334 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:11 smithi103 ceph-mon[36243]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-17T05:54:11.334 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:11 smithi103 ceph-mon[36243]: osdmap e11: 2 total, 1 up, 2 in 2024-04-17T05:54:11.334 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:11 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-17T05:54:11.334 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:11 smithi103 ceph-mon[36243]: from='osd.1 [v2:172.21.15.98:6800/1180430954,v1:172.21.15.98:6801/1180430954]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2024-04-17T05:54:11.334 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:11 smithi103 ceph-mon[36243]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2024-04-17T05:54:11.633 INFO:journalctl@ceph.osd.1.smithi098.stdout:Apr 17 05:54:11 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-1[41120]: 2024-04-17T05:54:11.243+0000 7fbe7e9ca700 -1 osd.1 0 waiting for initial osdmap 2024-04-17T05:54:11.633 INFO:journalctl@ceph.osd.1.smithi098.stdout:Apr 17 05:54:11 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-1[41120]: 2024-04-17T05:54:11.247+0000 7fbe7a7bd700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-17T05:54:11.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:11 smithi098 ceph-mon[36516]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-17T05:54:11.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:11 smithi098 ceph-mon[36516]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-17T05:54:11.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:11 smithi098 ceph-mon[36516]: osdmap e11: 2 total, 1 up, 2 in 2024-04-17T05:54:11.634 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:11 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-17T05:54:11.634 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:11 smithi098 ceph-mon[36516]: from='osd.1 [v2:172.21.15.98:6800/1180430954,v1:172.21.15.98:6801/1180430954]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2024-04-17T05:54:11.634 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:11 smithi098 ceph-mon[36516]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2024-04-17T05:54:11.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:11 smithi047 ceph-mon[32223]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-17T05:54:11.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:11 smithi047 ceph-mon[32223]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-17T05:54:11.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:11 smithi047 ceph-mon[32223]: osdmap e11: 2 total, 1 up, 2 in 2024-04-17T05:54:11.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:11 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-17T05:54:11.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:11 smithi047 ceph-mon[32223]: from='osd.1 [v2:172.21.15.98:6800/1180430954,v1:172.21.15.98:6801/1180430954]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2024-04-17T05:54:11.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:11 smithi047 ceph-mon[32223]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2024-04-17T05:54:11.914 INFO:teuthology.orchestra.run.smithi103.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-17T05:54:11.914 INFO:teuthology.orchestra.run.smithi103.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-17T05:54:11.914 INFO:teuthology.orchestra.run.smithi103.stderr: stderr: 10+0 records in 2024-04-17T05:54:11.914 INFO:teuthology.orchestra.run.smithi103.stderr:10+0 records out 2024-04-17T05:54:11.914 INFO:teuthology.orchestra.run.smithi103.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113128 s, 927 MB/s 2024-04-17T05:54:11.914 INFO:teuthology.orchestra.run.smithi103.stderr:--> Zapping successful for: 2024-04-17T05:54:12.500 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:12 smithi103 ceph-mon[36243]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]': finished 2024-04-17T05:54:12.500 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:12 smithi103 ceph-mon[36243]: osdmap e12: 2 total, 1 up, 2 in 2024-04-17T05:54:12.500 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:12 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-17T05:54:12.500 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:12 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-17T05:54:12.502 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 565db2d8-fc7e-11ee-bc91-c7b262605968 -- ceph orch daemon add osd smithi103:vg_nvme/lv_4 2024-04-17T05:54:12.608 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:12 smithi098 ceph-mon[36516]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]': finished 2024-04-17T05:54:12.608 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:12 smithi098 ceph-mon[36516]: osdmap e12: 2 total, 1 up, 2 in 2024-04-17T05:54:12.608 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:12 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-17T05:54:12.608 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:12 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-17T05:54:12.638 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:12 smithi047 ceph-mon[32223]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]': finished 2024-04-17T05:54:12.638 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:12 smithi047 ceph-mon[32223]: osdmap e12: 2 total, 1 up, 2 in 2024-04-17T05:54:12.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:12 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-17T05:54:12.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:12 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-17T05:54:12.744 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/mon.c/config 2024-04-17T05:54:13.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:13 smithi103 ceph-mon[36243]: purged_snaps scrub starts 2024-04-17T05:54:13.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:13 smithi103 ceph-mon[36243]: purged_snaps scrub ok 2024-04-17T05:54:13.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:13 smithi103 ceph-mon[36243]: pgmap v61: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-17T05:54:13.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:13 smithi103 ceph-mon[36243]: osd.1 [v2:172.21.15.98:6800/1180430954,v1:172.21.15.98:6801/1180430954] boot 2024-04-17T05:54:13.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:13 smithi103 ceph-mon[36243]: osdmap e13: 2 total, 2 up, 2 in 2024-04-17T05:54:13.535 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:13 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-17T05:54:13.535 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:13 smithi103 ceph-mon[36243]: Detected new or changed devices on smithi098 2024-04-17T05:54:13.535 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:13 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:54:13.535 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:13 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:54:13.535 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:13 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-17T05:54:13.535 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:13 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:54:13.535 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:13 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:54:13.535 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:13 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T05:54:13.535 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:13 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:54:13.535 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:13 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T05:54:13.535 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:13 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:54:13.535 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:13 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T05:54:13.535 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:13 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:54:13.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:13 smithi098 ceph-mon[36516]: purged_snaps scrub starts 2024-04-17T05:54:13.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:13 smithi098 ceph-mon[36516]: purged_snaps scrub ok 2024-04-17T05:54:13.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:13 smithi098 ceph-mon[36516]: pgmap v61: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-17T05:54:13.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:13 smithi098 ceph-mon[36516]: osd.1 [v2:172.21.15.98:6800/1180430954,v1:172.21.15.98:6801/1180430954] boot 2024-04-17T05:54:13.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:13 smithi098 ceph-mon[36516]: osdmap e13: 2 total, 2 up, 2 in 2024-04-17T05:54:13.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:13 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-17T05:54:13.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:13 smithi098 ceph-mon[36516]: Detected new or changed devices on smithi098 2024-04-17T05:54:13.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:13 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:54:13.634 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:13 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:54:13.634 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:13 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-17T05:54:13.634 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:13 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:54:13.634 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:13 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:54:13.634 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:13 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T05:54:13.634 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:13 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:54:13.634 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:13 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T05:54:13.634 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:13 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:54:13.634 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:13 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T05:54:13.634 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:13 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:54:13.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:13 smithi047 ceph-mon[32223]: purged_snaps scrub starts 2024-04-17T05:54:13.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:13 smithi047 ceph-mon[32223]: purged_snaps scrub ok 2024-04-17T05:54:13.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:13 smithi047 ceph-mon[32223]: pgmap v61: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-17T05:54:13.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:13 smithi047 ceph-mon[32223]: osd.1 [v2:172.21.15.98:6800/1180430954,v1:172.21.15.98:6801/1180430954] boot 2024-04-17T05:54:13.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:13 smithi047 ceph-mon[32223]: osdmap e13: 2 total, 2 up, 2 in 2024-04-17T05:54:13.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:13 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-17T05:54:13.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:13 smithi047 ceph-mon[32223]: Detected new or changed devices on smithi098 2024-04-17T05:54:13.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:13 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:54:13.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:13 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:54:13.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:13 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-17T05:54:13.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:13 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:54:13.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:13 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:54:13.640 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:13 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T05:54:13.640 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:13 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:54:13.640 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:13 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T05:54:13.640 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:13 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:54:13.640 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:13 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T05:54:13.640 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:13 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:54:14.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:54:14.249+0000 7f0bc29d8700 1 -- 172.21.15.103:0/2062923375 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0bbc102ab0 msgr2=0x7f0bbc102ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:54:14.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:54:14.249+0000 7f0bc29d8700 1 --2- 172.21.15.103:0/2062923375 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0bbc102ab0 0x7f0bbc102ed0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0bac009990 tx=0x7f0bac009ca0 comp rx=0 tx=0).stop 2024-04-17T05:54:14.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:54:14.249+0000 7f0bc29d8700 1 -- 172.21.15.103:0/2062923375 shutdown_connections 2024-04-17T05:54:14.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:54:14.249+0000 7f0bc29d8700 1 --2- 172.21.15.103:0/2062923375 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0bbc104650 0x7f0bbc10af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:54:14.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:54:14.249+0000 7f0bc29d8700 1 --2- 172.21.15.103:0/2062923375 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0bbc102ab0 0x7f0bbc102ed0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:54:14.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:54:14.249+0000 7f0bc29d8700 1 --2- 172.21.15.103:0/2062923375 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0bbc103cb0 0x7f0bbc104110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:54:14.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:54:14.249+0000 7f0bc29d8700 1 -- 172.21.15.103:0/2062923375 >> 172.21.15.103:0/2062923375 conn(0x7f0bbc0fe050 msgr2=0x7f0bbc100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:54:14.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:54:14.249+0000 7f0bc29d8700 1 -- 172.21.15.103:0/2062923375 shutdown_connections 2024-04-17T05:54:14.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:54:14.249+0000 7f0bc29d8700 1 -- 172.21.15.103:0/2062923375 wait complete. 2024-04-17T05:54:14.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:54:14.249+0000 7f0bc29d8700 1 Processor -- start 2024-04-17T05:54:14.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:54:14.249+0000 7f0bc29d8700 1 -- start start 2024-04-17T05:54:14.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:54:14.250+0000 7f0bc29d8700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0bbc102ab0 0x7f0bbc0767f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:54:14.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:54:14.250+0000 7f0bc29d8700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0bbc103cb0 0x7f0bbc074e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:54:14.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:54:14.250+0000 7f0bc29d8700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0bbc104650 0x7f0bbc075380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:54:14.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:54:14.250+0000 7f0bc29d8700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f0bbc10e690 con 0x7f0bbc102ab0 2024-04-17T05:54:14.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:54:14.250+0000 7f0bc29d8700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0bbc10e510 con 0x7f0bbc103cb0 2024-04-17T05:54:14.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:54:14.250+0000 7f0bc29d8700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0bbc10e810 con 0x7f0bbc104650 2024-04-17T05:54:14.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:54:14.250+0000 7f0bc0f75700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0bbc104650 0x7f0bbc075380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:54:14.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:54:14.250+0000 7f0bc0f75700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0bbc104650 0x7f0bbc075380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48168/0 (socket says 172.21.15.103:48168) 2024-04-17T05:54:14.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:54:14.250+0000 7f0bc0f75700 1 -- 172.21.15.103:0/4073579820 learned_addr learned my addr 172.21.15.103:0/4073579820 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-04-17T05:54:14.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:54:14.250+0000 7f0bbb7fe700 1 --2- 172.21.15.103:0/4073579820 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0bbc103cb0 0x7f0bbc074e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:54:14.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:54:14.250+0000 7f0bbbfff700 1 --2- 172.21.15.103:0/4073579820 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0bbc102ab0 0x7f0bbc0767f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:54:14.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:54:14.250+0000 7f0bc0f75700 1 -- 172.21.15.103:0/4073579820 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0bbc103cb0 msgr2=0x7f0bbc074e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:54:14.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:54:14.250+0000 7f0bc0f75700 1 --2- 172.21.15.103:0/4073579820 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0bbc103cb0 0x7f0bbc074e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:54:14.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:54:14.250+0000 7f0bc0f75700 1 -- 172.21.15.103:0/4073579820 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0bbc102ab0 msgr2=0x7f0bbc0767f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:54:14.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:54:14.250+0000 7f0bc0f75700 1 --2- 172.21.15.103:0/4073579820 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0bbc102ab0 0x7f0bbc0767f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:54:14.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:54:14.250+0000 7f0bc0f75700 1 -- 172.21.15.103:0/4073579820 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bbc075c30 con 0x7f0bbc104650 2024-04-17T05:54:14.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:54:14.251+0000 7f0bc0f75700 1 --2- 172.21.15.103:0/4073579820 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0bbc104650 0x7f0bbc075380 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0bb4007ee0 tx=0x7f0bb400d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:54:14.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:54:14.252+0000 7f0bb97fa700 1 -- 172.21.15.103:0/4073579820 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb400dd20 con 0x7f0bbc104650 2024-04-17T05:54:14.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:54:14.252+0000 7f0bb97fa700 1 -- 172.21.15.103:0/4073579820 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0bb4004020 con 0x7f0bbc104650 2024-04-17T05:54:14.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:54:14.252+0000 7f0bc29d8700 1 -- 172.21.15.103:0/4073579820 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bbc1a2e90 con 0x7f0bbc104650 2024-04-17T05:54:14.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:54:14.252+0000 7f0bb97fa700 1 -- 172.21.15.103:0/4073579820 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb4015660 con 0x7f0bbc104650 2024-04-17T05:54:14.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:54:14.252+0000 7f0bc29d8700 1 -- 172.21.15.103:0/4073579820 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bbc1a32f0 con 0x7f0bbc104650 2024-04-17T05:54:14.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:54:14.253+0000 7f0bc29d8700 1 -- 172.21.15.103:0/4073579820 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bbc069740 con 0x7f0bbc104650 2024-04-17T05:54:14.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:54:14.254+0000 7f0bb97fa700 1 -- 172.21.15.103:0/4073579820 <== mon.1 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f0bb40041d0 con 0x7f0bbc104650 2024-04-17T05:54:14.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:54:14.255+0000 7f0bb97fa700 1 --2- 172.21.15.103:0/4073579820 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f0ba40a3c30 0x7f0ba40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:54:14.254 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:14 smithi103 ceph-mon[36243]: Adjusting osd_memory_target on smithi098 to 17080M 2024-04-17T05:54:14.254 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:14 smithi103 ceph-mon[36243]: osdmap e14: 2 total, 2 up, 2 in 2024-04-17T05:54:14.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:54:14.255+0000 7f0bb97fa700 1 -- 172.21.15.103:0/4073579820 <== mon.1 v2:172.21.15.103:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7f0bb40ce4b0 con 0x7f0bbc104650 2024-04-17T05:54:14.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:54:14.255+0000 7f0bbbfff700 1 --2- 172.21.15.103:0/4073579820 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f0ba40a3c30 0x7f0ba40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:54:14.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:54:14.255+0000 7f0bbbfff700 1 --2- 172.21.15.103:0/4073579820 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f0ba40a3c30 0x7f0ba40a60f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0bac00b820 tx=0x7f0bac005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:54:14.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:54:14.260+0000 7f0bb97fa700 1 -- 172.21.15.103:0/4073579820 <== mon.1 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f0bb4010070 con 0x7f0bbc104650 2024-04-17T05:54:14.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:54:14.468+0000 7f0bc29d8700 1 -- 172.21.15.103:0/4073579820 --> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f0bbc062e90 con 0x7f0ba40a3c30 2024-04-17T05:54:14.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:14 smithi098 ceph-mon[36516]: Adjusting osd_memory_target on smithi098 to 17080M 2024-04-17T05:54:14.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:14 smithi098 ceph-mon[36516]: osdmap e14: 2 total, 2 up, 2 in 2024-04-17T05:54:14.638 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:14 smithi047 ceph-mon[32223]: Adjusting osd_memory_target on smithi098 to 17080M 2024-04-17T05:54:14.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:14 smithi047 ceph-mon[32223]: osdmap e14: 2 total, 2 up, 2 in 2024-04-17T05:54:15.344 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:15 smithi103 ceph-mon[36243]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-17T05:54:15.345 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:15 smithi103 ceph-mon[36243]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T05:54:15.345 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:15 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-17T05:54:15.345 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:15 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-17T05:54:15.345 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:15 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:54:15.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:15 smithi047 ceph-mon[32223]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-17T05:54:15.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:15 smithi047 ceph-mon[32223]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T05:54:15.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:15 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-17T05:54:15.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:15 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-17T05:54:15.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:15 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:54:15.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:15 smithi098 ceph-mon[36516]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-17T05:54:15.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:15 smithi098 ceph-mon[36516]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T05:54:15.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:15 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-17T05:54:15.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:15 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-17T05:54:15.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:15 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:54:17.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:17 smithi103 ceph-mon[36243]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-17T05:54:17.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:17 smithi103 ceph-mon[36243]: from='client.? 172.21.15.103:0/2397669445' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0fc9d383-05b4-498d-b530-fccf7aeddd7c"}]: dispatch 2024-04-17T05:54:17.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:17 smithi103 ceph-mon[36243]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0fc9d383-05b4-498d-b530-fccf7aeddd7c"}]: dispatch 2024-04-17T05:54:17.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:17 smithi103 ceph-mon[36243]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0fc9d383-05b4-498d-b530-fccf7aeddd7c"}]': finished 2024-04-17T05:54:17.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:17 smithi103 ceph-mon[36243]: osdmap e15: 3 total, 2 up, 3 in 2024-04-17T05:54:17.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:17 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-17T05:54:17.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:17 smithi098 ceph-mon[36516]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-17T05:54:17.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:17 smithi098 ceph-mon[36516]: from='client.? 172.21.15.103:0/2397669445' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0fc9d383-05b4-498d-b530-fccf7aeddd7c"}]: dispatch 2024-04-17T05:54:17.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:17 smithi098 ceph-mon[36516]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0fc9d383-05b4-498d-b530-fccf7aeddd7c"}]: dispatch 2024-04-17T05:54:17.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:17 smithi098 ceph-mon[36516]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0fc9d383-05b4-498d-b530-fccf7aeddd7c"}]': finished 2024-04-17T05:54:17.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:17 smithi098 ceph-mon[36516]: osdmap e15: 3 total, 2 up, 3 in 2024-04-17T05:54:17.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:17 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-17T05:54:17.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:17 smithi047 ceph-mon[32223]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-17T05:54:17.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:17 smithi047 ceph-mon[32223]: from='client.? 172.21.15.103:0/2397669445' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0fc9d383-05b4-498d-b530-fccf7aeddd7c"}]: dispatch 2024-04-17T05:54:17.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:17 smithi047 ceph-mon[32223]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0fc9d383-05b4-498d-b530-fccf7aeddd7c"}]: dispatch 2024-04-17T05:54:17.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:17 smithi047 ceph-mon[32223]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0fc9d383-05b4-498d-b530-fccf7aeddd7c"}]': finished 2024-04-17T05:54:17.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:17 smithi047 ceph-mon[32223]: osdmap e15: 3 total, 2 up, 3 in 2024-04-17T05:54:17.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:17 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-17T05:54:18.533 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:18 smithi103 ceph-mon[36243]: from='client.? 172.21.15.103:0/1496109910' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-17T05:54:18.632 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:18 smithi098 ceph-mon[36516]: from='client.? 172.21.15.103:0/1496109910' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-17T05:54:18.638 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:18 smithi047 ceph-mon[32223]: from='client.? 172.21.15.103:0/1496109910' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-17T05:54:19.533 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:19 smithi103 ceph-mon[36243]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-17T05:54:19.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:19 smithi098 ceph-mon[36516]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-17T05:54:19.638 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:19 smithi047 ceph-mon[32223]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-17T05:54:21.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:21 smithi103 ceph-mon[36243]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-17T05:54:21.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:21 smithi098 ceph-mon[36516]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-17T05:54:21.638 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:21 smithi047 ceph-mon[32223]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-17T05:54:23.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:23 smithi103 ceph-mon[36243]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-17T05:54:23.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:23 smithi098 ceph-mon[36516]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-17T05:54:23.638 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:23 smithi047 ceph-mon[32223]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-17T05:54:24.533 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:24 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-17T05:54:24.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:24 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:54:24.534 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:24 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-17T05:54:24.534 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:24 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:54:24.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:24 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-17T05:54:24.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:24 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:54:25.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:25 smithi103 ceph-mon[36243]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-17T05:54:25.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:25 smithi103 ceph-mon[36243]: Deploying daemon osd.2 on smithi103 2024-04-17T05:54:25.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:25 smithi098 ceph-mon[36516]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-17T05:54:25.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:25 smithi098 ceph-mon[36516]: Deploying daemon osd.2 on smithi103 2024-04-17T05:54:25.638 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:25 smithi047 ceph-mon[32223]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-17T05:54:25.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:25 smithi047 ceph-mon[32223]: Deploying daemon osd.2 on smithi103 2024-04-17T05:54:27.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:27 smithi103 ceph-mon[36243]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-17T05:54:27.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:27 smithi098 ceph-mon[36516]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-17T05:54:27.638 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:27 smithi047 ceph-mon[32223]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-17T05:54:29.354 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:29 smithi103 ceph-mon[36243]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-17T05:54:29.632 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:29 smithi098 ceph-mon[36516]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-17T05:54:29.638 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:29 smithi047 ceph-mon[32223]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-17T05:54:30.692 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:30 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:54:30.692 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:30 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T05:54:30.692 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:30 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:54:30.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:30 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:54:30.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:30 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T05:54:30.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:30 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:54:30.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:30 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:54:30.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:30 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T05:54:30.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:30 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:54:31.533 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:31 smithi103 ceph-mon[36243]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-17T05:54:31.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:31 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:54:31.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:31 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:54:31.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:31 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:54:31.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:31 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T05:54:31.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:31 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:54:31.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:31 smithi098 ceph-mon[36516]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-17T05:54:31.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:31 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:54:31.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:31 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:54:31.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:31 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:54:31.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:31 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T05:54:31.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:31 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:54:31.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:31 smithi047 ceph-mon[32223]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-17T05:54:31.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:31 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:54:31.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:31 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:54:31.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:31 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:54:31.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:31 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T05:54:31.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:31 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:54:32.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:54:32.931+0000 7f0bb97fa700 1 -- 172.21.15.103:0/4073579820 <== mgr.14152 v2:172.21.15.47:6800/3157902806 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f0bbc062e90 con 0x7f0ba40a3c30 2024-04-17T05:54:32.930 INFO:teuthology.orchestra.run.smithi103.stdout:Created osd(s) 2 on host 'smithi103' 2024-04-17T05:54:32.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:54:32.934+0000 7f0bc29d8700 1 -- 172.21.15.103:0/4073579820 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f0ba40a3c30 msgr2=0x7f0ba40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:54:32.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:54:32.934+0000 7f0bc29d8700 1 --2- 172.21.15.103:0/4073579820 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f0ba40a3c30 0x7f0ba40a60f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0bac00b820 tx=0x7f0bac005fd0 comp rx=0 tx=0).stop 2024-04-17T05:54:32.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:54:32.934+0000 7f0bc29d8700 1 -- 172.21.15.103:0/4073579820 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0bbc104650 msgr2=0x7f0bbc075380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:54:32.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:54:32.934+0000 7f0bc29d8700 1 --2- 172.21.15.103:0/4073579820 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0bbc104650 0x7f0bbc075380 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0bb4007ee0 tx=0x7f0bb400d3b0 comp rx=0 tx=0).stop 2024-04-17T05:54:32.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:54:32.934+0000 7f0bc29d8700 1 -- 172.21.15.103:0/4073579820 shutdown_connections 2024-04-17T05:54:32.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:54:32.934+0000 7f0bc29d8700 1 --2- 172.21.15.103:0/4073579820 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f0ba40a3c30 0x7f0ba40a60f0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:54:32.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:54:32.934+0000 7f0bc29d8700 1 --2- 172.21.15.103:0/4073579820 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0bbc104650 0x7f0bbc075380 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:54:32.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:54:32.934+0000 7f0bc29d8700 1 --2- 172.21.15.103:0/4073579820 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0bbc102ab0 0x7f0bbc0767f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:54:32.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:54:32.934+0000 7f0bc29d8700 1 --2- 172.21.15.103:0/4073579820 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0bbc103cb0 0x7f0bbc074e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:54:32.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:54:32.934+0000 7f0bc29d8700 1 -- 172.21.15.103:0/4073579820 >> 172.21.15.103:0/4073579820 conn(0x7f0bbc0fe050 msgr2=0x7f0bbc107030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:54:32.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:54:32.934+0000 7f0bc29d8700 1 -- 172.21.15.103:0/4073579820 shutdown_connections 2024-04-17T05:54:32.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:54:32.934+0000 7f0bc29d8700 1 -- 172.21.15.103:0/4073579820 wait complete. 2024-04-17T05:54:33.501 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:33 smithi103 ceph-mon[36243]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-17T05:54:33.501 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:33 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:54:33.501 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:33 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:54:33.501 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:33 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T05:54:33.501 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:33 smithi103 ceph-mon[36243]: from='osd.2 [v2:172.21.15.103:6800/494676216,v1:172.21.15.103:6801/494676216]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-17T05:54:33.501 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:33 smithi103 ceph-mon[36243]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-17T05:54:33.526 DEBUG:teuthology.orchestra.run.smithi103:osd.2> sudo journalctl -f -n 0 -u ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@osd.2.service 2024-04-17T05:54:33.529 INFO:tasks.cephadm:Deploying osd.3 on smithi047 with /dev/vg_nvme/lv_3... 2024-04-17T05:54:33.529 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 565db2d8-fc7e-11ee-bc91-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-17T05:54:33.787 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T05:54:33.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:33 smithi098 ceph-mon[36516]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-17T05:54:33.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:33 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:54:33.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:33 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:54:33.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:33 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T05:54:33.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:33 smithi098 ceph-mon[36516]: from='osd.2 [v2:172.21.15.103:6800/494676216,v1:172.21.15.103:6801/494676216]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-17T05:54:33.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:33 smithi098 ceph-mon[36516]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-17T05:54:33.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:33 smithi047 ceph-mon[32223]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-17T05:54:33.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:33 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:54:33.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:33 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:54:33.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:33 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T05:54:33.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:33 smithi047 ceph-mon[32223]: from='osd.2 [v2:172.21.15.103:6800/494676216,v1:172.21.15.103:6801/494676216]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-17T05:54:33.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:33 smithi047 ceph-mon[32223]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-17T05:54:35.004 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:34 smithi103 ceph-mon[36243]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-17T05:54:35.004 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:34 smithi103 ceph-mon[36243]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-17T05:54:35.004 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:34 smithi103 ceph-mon[36243]: osdmap e16: 3 total, 2 up, 3 in 2024-04-17T05:54:35.004 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:34 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-17T05:54:35.004 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:34 smithi103 ceph-mon[36243]: from='osd.2 [v2:172.21.15.103:6800/494676216,v1:172.21.15.103:6801/494676216]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-04-17T05:54:35.004 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:34 smithi103 ceph-mon[36243]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-04-17T05:54:35.004 INFO:journalctl@ceph.osd.2.smithi103.stdout:Apr 17 05:54:34 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-2[41376]: 2024-04-17T05:54:34.908+0000 7fcce0651700 -1 osd.2 0 waiting for initial osdmap 2024-04-17T05:54:35.004 INFO:journalctl@ceph.osd.2.smithi103.stdout:Apr 17 05:54:34 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-2[41376]: 2024-04-17T05:54:34.911+0000 7fccdc444700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-17T05:54:35.172 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:34 smithi047 ceph-mon[32223]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-17T05:54:35.172 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:34 smithi047 ceph-mon[32223]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-17T05:54:35.172 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:34 smithi047 ceph-mon[32223]: osdmap e16: 3 total, 2 up, 3 in 2024-04-17T05:54:35.172 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:34 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-17T05:54:35.172 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:34 smithi047 ceph-mon[32223]: from='osd.2 [v2:172.21.15.103:6800/494676216,v1:172.21.15.103:6801/494676216]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-04-17T05:54:35.172 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:34 smithi047 ceph-mon[32223]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-04-17T05:54:35.232 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:34 smithi098 ceph-mon[36516]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-17T05:54:35.233 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:34 smithi098 ceph-mon[36516]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-17T05:54:35.233 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:34 smithi098 ceph-mon[36516]: osdmap e16: 3 total, 2 up, 3 in 2024-04-17T05:54:35.233 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:34 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-17T05:54:35.233 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:34 smithi098 ceph-mon[36516]: from='osd.2 [v2:172.21.15.103:6800/494676216,v1:172.21.15.103:6801/494676216]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-04-17T05:54:35.233 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:34 smithi098 ceph-mon[36516]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-04-17T05:54:35.732 INFO:teuthology.orchestra.run.smithi047.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-17T05:54:35.732 INFO:teuthology.orchestra.run.smithi047.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-17T05:54:35.732 INFO:teuthology.orchestra.run.smithi047.stderr: stderr: 10+0 records in 2024-04-17T05:54:35.733 INFO:teuthology.orchestra.run.smithi047.stderr:10+0 records out 2024-04-17T05:54:35.733 INFO:teuthology.orchestra.run.smithi047.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00836199 s, 1.3 GB/s 2024-04-17T05:54:35.733 INFO:teuthology.orchestra.run.smithi047.stderr:--> Zapping successful for: 2024-04-17T05:54:36.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:35 smithi098 ceph-mon[36516]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-04-17T05:54:36.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:35 smithi098 ceph-mon[36516]: osdmap e17: 3 total, 2 up, 3 in 2024-04-17T05:54:36.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:35 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-17T05:54:36.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:35 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-17T05:54:36.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:35 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-17T05:54:36.190 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:35 smithi103 ceph-mon[36243]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-04-17T05:54:36.190 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:35 smithi103 ceph-mon[36243]: osdmap e17: 3 total, 2 up, 3 in 2024-04-17T05:54:36.191 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:35 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-17T05:54:36.191 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:35 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-17T05:54:36.191 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:35 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-17T05:54:36.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:35 smithi047 ceph-mon[32223]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-04-17T05:54:36.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:35 smithi047 ceph-mon[32223]: osdmap e17: 3 total, 2 up, 3 in 2024-04-17T05:54:36.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:35 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-17T05:54:36.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:35 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-17T05:54:36.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:35 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-17T05:54:36.589 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 565db2d8-fc7e-11ee-bc91-c7b262605968 -- ceph orch daemon add osd smithi047:vg_nvme/lv_3 2024-04-17T05:54:36.854 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T05:54:37.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:36 smithi047 ceph-mon[32223]: purged_snaps scrub starts 2024-04-17T05:54:37.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:36 smithi047 ceph-mon[32223]: purged_snaps scrub ok 2024-04-17T05:54:37.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:36 smithi047 ceph-mon[32223]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-17T05:54:37.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:36 smithi047 ceph-mon[32223]: osd.2 [v2:172.21.15.103:6800/494676216,v1:172.21.15.103:6801/494676216] boot 2024-04-17T05:54:37.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:36 smithi047 ceph-mon[32223]: osdmap e18: 3 total, 3 up, 3 in 2024-04-17T05:54:37.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:36 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-17T05:54:37.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:36 smithi047 ceph-mon[32223]: Detected new or changed devices on smithi103 2024-04-17T05:54:37.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:36 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:54:37.140 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:36 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:54:37.140 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:36 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-17T05:54:37.140 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:36 smithi047 ceph-mon[32223]: Adjusting osd_memory_target on smithi103 to 17080M 2024-04-17T05:54:37.140 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:36 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:54:37.140 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:36 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:54:37.140 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:36 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T05:54:37.140 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:36 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:54:37.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:36 smithi103 ceph-mon[36243]: purged_snaps scrub starts 2024-04-17T05:54:37.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:36 smithi103 ceph-mon[36243]: purged_snaps scrub ok 2024-04-17T05:54:37.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:36 smithi103 ceph-mon[36243]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-17T05:54:37.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:36 smithi103 ceph-mon[36243]: osd.2 [v2:172.21.15.103:6800/494676216,v1:172.21.15.103:6801/494676216] boot 2024-04-17T05:54:37.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:36 smithi103 ceph-mon[36243]: osdmap e18: 3 total, 3 up, 3 in 2024-04-17T05:54:37.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:36 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-17T05:54:37.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:36 smithi103 ceph-mon[36243]: Detected new or changed devices on smithi103 2024-04-17T05:54:37.285 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:36 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:54:37.285 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:36 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:54:37.285 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:36 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-17T05:54:37.285 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:36 smithi103 ceph-mon[36243]: Adjusting osd_memory_target on smithi103 to 17080M 2024-04-17T05:54:37.285 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:36 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:54:37.285 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:36 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:54:37.285 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:36 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T05:54:37.285 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:36 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:54:37.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:36 smithi098 ceph-mon[36516]: purged_snaps scrub starts 2024-04-17T05:54:37.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:36 smithi098 ceph-mon[36516]: purged_snaps scrub ok 2024-04-17T05:54:37.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:36 smithi098 ceph-mon[36516]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-17T05:54:37.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:36 smithi098 ceph-mon[36516]: osd.2 [v2:172.21.15.103:6800/494676216,v1:172.21.15.103:6801/494676216] boot 2024-04-17T05:54:37.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:36 smithi098 ceph-mon[36516]: osdmap e18: 3 total, 3 up, 3 in 2024-04-17T05:54:37.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:36 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-17T05:54:37.384 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:36 smithi098 ceph-mon[36516]: Detected new or changed devices on smithi103 2024-04-17T05:54:37.384 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:36 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:54:37.384 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:36 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:54:37.384 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:36 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-17T05:54:37.384 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:36 smithi098 ceph-mon[36516]: Adjusting osd_memory_target on smithi103 to 17080M 2024-04-17T05:54:37.384 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:36 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:54:37.384 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:36 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:54:37.384 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:36 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T05:54:37.384 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:36 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:54:38.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:37 smithi047 ceph-mon[32223]: osdmap e19: 3 total, 3 up, 3 in 2024-04-17T05:54:38.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:37 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-17T05:54:38.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:37 smithi103 ceph-mon[36243]: osdmap e19: 3 total, 3 up, 3 in 2024-04-17T05:54:38.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:37 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-17T05:54:38.382 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:37 smithi098 ceph-mon[36516]: osdmap e19: 3 total, 3 up, 3 in 2024-04-17T05:54:38.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:37 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-17T05:54:38.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:54:38.603+0000 7f0ed4584700 1 -- 172.21.15.47:0/2015284810 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0ecc104650 msgr2=0x7f0ecc10af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:54:38.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:54:38.603+0000 7f0ed4584700 1 --2- 172.21.15.47:0/2015284810 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0ecc104650 0x7f0ecc10af00 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0ec400b0d0 tx=0x7f0ec400b3e0 comp rx=0 tx=0).stop 2024-04-17T05:54:38.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:54:38.604+0000 7f0ed4584700 1 -- 172.21.15.47:0/2015284810 shutdown_connections 2024-04-17T05:54:38.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:54:38.604+0000 7f0ed4584700 1 --2- 172.21.15.47:0/2015284810 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0ecc104650 0x7f0ecc10af00 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:54:38.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:54:38.604+0000 7f0ed4584700 1 --2- 172.21.15.47:0/2015284810 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ecc102ab0 0x7f0ecc102ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:54:38.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:54:38.604+0000 7f0ed4584700 1 --2- 172.21.15.47:0/2015284810 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ecc103cb0 0x7f0ecc104110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:54:38.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:54:38.604+0000 7f0ed4584700 1 -- 172.21.15.47:0/2015284810 >> 172.21.15.47:0/2015284810 conn(0x7f0ecc0fe050 msgr2=0x7f0ecc100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:54:38.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:54:38.604+0000 7f0ed4584700 1 -- 172.21.15.47:0/2015284810 shutdown_connections 2024-04-17T05:54:38.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:54:38.604+0000 7f0ed4584700 1 -- 172.21.15.47:0/2015284810 wait complete. 2024-04-17T05:54:38.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:54:38.604+0000 7f0ed4584700 1 Processor -- start 2024-04-17T05:54:38.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:54:38.604+0000 7f0ed4584700 1 -- start start 2024-04-17T05:54:38.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:54:38.604+0000 7f0ed4584700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ecc102ab0 0x7f0ecc0767f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:54:38.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:54:38.604+0000 7f0ed4584700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0ecc103cb0 0x7f0ecc074e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:54:38.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:54:38.604+0000 7f0ed4584700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ecc104650 0x7f0ecc075380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:54:38.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:54:38.604+0000 7f0ed4584700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f0ecc10e710 con 0x7f0ecc103cb0 2024-04-17T05:54:38.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:54:38.604+0000 7f0ed4584700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0ecc10e590 con 0x7f0ecc104650 2024-04-17T05:54:38.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:54:38.604+0000 7f0ed4584700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0ecc10e890 con 0x7f0ecc102ab0 2024-04-17T05:54:38.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:54:38.605+0000 7f0ed1b1f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0ecc103cb0 0x7f0ecc074e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:54:38.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:54:38.605+0000 7f0ed1b1f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0ecc103cb0 0x7f0ecc074e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58928/0 (socket says 172.21.15.47:58928) 2024-04-17T05:54:38.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:54:38.605+0000 7f0ed1b1f700 1 -- 172.21.15.47:0/1901745096 learned_addr learned my addr 172.21.15.47:0/1901745096 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:54:38.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:54:38.605+0000 7f0ed2320700 1 --2- 172.21.15.47:0/1901745096 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ecc102ab0 0x7f0ecc0767f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:54:38.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:54:38.605+0000 7f0ed2b21700 1 --2- 172.21.15.47:0/1901745096 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ecc104650 0x7f0ecc075380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:54:38.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:54:38.605+0000 7f0ed1b1f700 1 -- 172.21.15.47:0/1901745096 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ecc102ab0 msgr2=0x7f0ecc0767f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:54:38.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:54:38.605+0000 7f0ed1b1f700 1 --2- 172.21.15.47:0/1901745096 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ecc102ab0 0x7f0ecc0767f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:54:38.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:54:38.605+0000 7f0ed1b1f700 1 -- 172.21.15.47:0/1901745096 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ecc104650 msgr2=0x7f0ecc075380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:54:38.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:54:38.605+0000 7f0ed1b1f700 1 --2- 172.21.15.47:0/1901745096 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ecc104650 0x7f0ecc075380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:54:38.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:54:38.605+0000 7f0ed1b1f700 1 -- 172.21.15.47:0/1901745096 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ecc075c30 con 0x7f0ecc103cb0 2024-04-17T05:54:38.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:54:38.605+0000 7f0ed2b21700 1 --2- 172.21.15.47:0/1901745096 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ecc104650 0x7f0ecc075380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T05:54:38.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:54:38.605+0000 7f0ed1b1f700 1 --2- 172.21.15.47:0/1901745096 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0ecc103cb0 0x7f0ecc074e40 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc00d950 tx=0x7f0ebc00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:54:38.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:54:38.605+0000 7f0ec37fe700 1 -- 172.21.15.47:0/1901745096 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ebc016070 con 0x7f0ecc103cb0 2024-04-17T05:54:38.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:54:38.605+0000 7f0ed4584700 1 -- 172.21.15.47:0/1901745096 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ecc1a2e20 con 0x7f0ecc103cb0 2024-04-17T05:54:38.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:54:38.605+0000 7f0ec37fe700 1 -- 172.21.15.47:0/1901745096 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ebc009e20 con 0x7f0ecc103cb0 2024-04-17T05:54:38.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:54:38.605+0000 7f0ec37fe700 1 -- 172.21.15.47:0/1901745096 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ebc00f620 con 0x7f0ecc103cb0 2024-04-17T05:54:38.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:54:38.605+0000 7f0ed4584700 1 -- 172.21.15.47:0/1901745096 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ecc1a3300 con 0x7f0ecc103cb0 2024-04-17T05:54:38.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:54:38.606+0000 7f0ed4584700 1 -- 172.21.15.47:0/1901745096 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ecc069740 con 0x7f0ecc103cb0 2024-04-17T05:54:38.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:54:38.608+0000 7f0ec37fe700 1 -- 172.21.15.47:0/1901745096 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f0ebc019070 con 0x7f0ecc103cb0 2024-04-17T05:54:38.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:54:38.609+0000 7f0ec37fe700 1 --2- 172.21.15.47:0/1901745096 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f0eb80a3bb0 0x7f0eb80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:54:38.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:54:38.609+0000 7f0ed2320700 1 --2- 172.21.15.47:0/1901745096 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f0eb80a3bb0 0x7f0eb80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:54:38.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:54:38.609+0000 7f0ec37fe700 1 -- 172.21.15.47:0/1901745096 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 3226+0+0 (secure 0 0 0) 0x7f0ebc0ceb40 con 0x7f0ecc103cb0 2024-04-17T05:54:38.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:54:38.609+0000 7f0ed2320700 1 --2- 172.21.15.47:0/1901745096 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f0eb80a3bb0 0x7f0eb80a6070 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc103b10 tx=0x7f0ec8009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:54:38.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:54:38.613+0000 7f0ec37fe700 1 -- 172.21.15.47:0/1901745096 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f0ebc09c940 con 0x7f0ecc103cb0 2024-04-17T05:54:38.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:54:38.830+0000 7f0ed4584700 1 -- 172.21.15.47:0/1901745096 --> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f0ecc062e90 con 0x7f0eb80a3bb0 2024-04-17T05:54:39.139 INFO:journalctl@ceph.osd.0.smithi047.stdout:Apr 17 05:54:39 smithi047 sudo[51711]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-17T05:54:39.139 INFO:journalctl@ceph.osd.0.smithi047.stdout:Apr 17 05:54:39 smithi047 sudo[51711]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-17T05:54:39.139 INFO:journalctl@ceph.osd.0.smithi047.stdout:Apr 17 05:54:39 smithi047 sudo[51711]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-17T05:54:39.139 INFO:journalctl@ceph.osd.0.smithi047.stdout:Apr 17 05:54:39 smithi047 sudo[51711]: pam_unix(sudo:session): session closed for user root 2024-04-17T05:54:39.139 INFO:journalctl@ceph.osd.0.smithi047.stdout:Apr 17 05:54:39 smithi047 sudo[51714]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-17T05:54:39.139 INFO:journalctl@ceph.osd.0.smithi047.stdout:Apr 17 05:54:39 smithi047 sudo[51714]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-17T05:54:39.139 INFO:journalctl@ceph.osd.0.smithi047.stdout:Apr 17 05:54:39 smithi047 sudo[51714]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-17T05:54:39.139 INFO:journalctl@ceph.osd.0.smithi047.stdout:Apr 17 05:54:39 smithi047 sudo[51714]: pam_unix(sudo:session): session closed for user root 2024-04-17T05:54:39.140 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:38 smithi047 ceph-mon[32223]: pgmap v81: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-17T05:54:39.140 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:38 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-17T05:54:39.140 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:38 smithi047 ceph-mon[32223]: osdmap e20: 3 total, 3 up, 3 in 2024-04-17T05:54:39.140 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:38 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-17T05:54:39.140 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:38 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-17T05:54:39.140 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:38 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-17T05:54:39.140 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:38 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:54:39.182 INFO:journalctl@ceph.osd.2.smithi103.stdout:Apr 17 05:54:39 smithi103 sudo[45451]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-17T05:54:39.182 INFO:journalctl@ceph.osd.2.smithi103.stdout:Apr 17 05:54:39 smithi103 sudo[45451]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-17T05:54:39.182 INFO:journalctl@ceph.osd.2.smithi103.stdout:Apr 17 05:54:39 smithi103 sudo[45451]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-17T05:54:39.182 INFO:journalctl@ceph.osd.2.smithi103.stdout:Apr 17 05:54:39 smithi103 sudo[45451]: pam_unix(sudo:session): session closed for user root 2024-04-17T05:54:39.183 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:38 smithi103 ceph-mon[36243]: pgmap v81: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-17T05:54:39.183 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:38 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-17T05:54:39.183 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:38 smithi103 ceph-mon[36243]: osdmap e20: 3 total, 3 up, 3 in 2024-04-17T05:54:39.183 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:38 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-17T05:54:39.183 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:38 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-17T05:54:39.183 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:38 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-17T05:54:39.183 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:38 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:54:39.290 INFO:journalctl@ceph.osd.1.smithi098.stdout:Apr 17 05:54:39 smithi098 sudo[45273]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-17T05:54:39.291 INFO:journalctl@ceph.osd.1.smithi098.stdout:Apr 17 05:54:39 smithi098 sudo[45273]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-17T05:54:39.291 INFO:journalctl@ceph.osd.1.smithi098.stdout:Apr 17 05:54:39 smithi098 sudo[45273]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-17T05:54:39.291 INFO:journalctl@ceph.osd.1.smithi098.stdout:Apr 17 05:54:39 smithi098 sudo[45273]: pam_unix(sudo:session): session closed for user root 2024-04-17T05:54:39.291 INFO:journalctl@ceph.osd.1.smithi098.stdout:Apr 17 05:54:39 smithi098 sudo[45276]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-17T05:54:39.291 INFO:journalctl@ceph.osd.1.smithi098.stdout:Apr 17 05:54:39 smithi098 sudo[45276]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-17T05:54:39.291 INFO:journalctl@ceph.osd.1.smithi098.stdout:Apr 17 05:54:39 smithi098 sudo[45276]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-17T05:54:39.291 INFO:journalctl@ceph.osd.1.smithi098.stdout:Apr 17 05:54:39 smithi098 sudo[45276]: pam_unix(sudo:session): session closed for user root 2024-04-17T05:54:39.291 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:38 smithi098 ceph-mon[36516]: pgmap v81: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-17T05:54:39.291 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:38 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-17T05:54:39.292 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:38 smithi098 ceph-mon[36516]: osdmap e20: 3 total, 3 up, 3 in 2024-04-17T05:54:39.292 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:38 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-17T05:54:39.292 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:38 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-17T05:54:39.292 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:38 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-17T05:54:39.292 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:38 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:54:39.493 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:39 smithi047 sudo[51717]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-17T05:54:39.493 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:39 smithi047 sudo[51717]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-17T05:54:39.493 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:39 smithi047 sudo[51717]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-17T05:54:39.493 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:39 smithi047 sudo[51717]: pam_unix(sudo:session): session closed for user root 2024-04-17T05:54:39.493 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:39 smithi047 sudo[51720]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-17T05:54:39.493 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:39 smithi047 sudo[51720]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-17T05:54:39.493 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:39 smithi047 sudo[51720]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-17T05:54:39.493 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:39 smithi047 sudo[51720]: pam_unix(sudo:session): session closed for user root 2024-04-17T05:54:39.534 INFO:journalctl@ceph.osd.2.smithi103.stdout:Apr 17 05:54:39 smithi103 sudo[45454]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-17T05:54:39.534 INFO:journalctl@ceph.osd.2.smithi103.stdout:Apr 17 05:54:39 smithi103 sudo[45454]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-17T05:54:39.534 INFO:journalctl@ceph.osd.2.smithi103.stdout:Apr 17 05:54:39 smithi103 sudo[45454]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-17T05:54:39.534 INFO:journalctl@ceph.osd.2.smithi103.stdout:Apr 17 05:54:39 smithi103 sudo[45454]: pam_unix(sudo:session): session closed for user root 2024-04-17T05:54:39.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:39 smithi103 sudo[45457]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-17T05:54:39.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:39 smithi103 sudo[45457]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-17T05:54:39.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:39 smithi103 sudo[45457]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-17T05:54:39.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:39 smithi103 sudo[45457]: pam_unix(sudo:session): session closed for user root 2024-04-17T05:54:39.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:39 smithi103 sudo[45460]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-17T05:54:39.535 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:39 smithi103 sudo[45460]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-17T05:54:39.535 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:39 smithi103 sudo[45460]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-17T05:54:39.535 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:39 smithi103 sudo[45460]: pam_unix(sudo:session): session closed for user root 2024-04-17T05:54:39.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:39 smithi098 sudo[45279]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-17T05:54:39.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:39 smithi098 sudo[45279]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-17T05:54:39.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:39 smithi098 sudo[45279]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-17T05:54:39.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:39 smithi098 sudo[45279]: pam_unix(sudo:session): session closed for user root 2024-04-17T05:54:39.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:39 smithi098 sudo[45282]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-17T05:54:39.634 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:39 smithi098 sudo[45282]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-17T05:54:39.634 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:39 smithi098 sudo[45282]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-17T05:54:39.634 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:39 smithi098 sudo[45282]: pam_unix(sudo:session): session closed for user root 2024-04-17T05:54:40.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:39 smithi047 ceph-mon[32223]: from='client.14304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T05:54:40.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:39 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-17T05:54:40.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:39 smithi047 ceph-mon[32223]: osdmap e21: 3 total, 3 up, 3 in 2024-04-17T05:54:40.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:39 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:54:40.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:39 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:54:40.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:39 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:54:40.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:39 smithi047 ceph-mon[32223]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-17T05:54:40.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:39 smithi047 ceph-mon[32223]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-17T05:54:40.140 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:39 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-17T05:54:40.140 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:39 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-17T05:54:40.140 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:39 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-17T05:54:40.140 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:39 smithi047 ceph-mon[32223]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-17T05:54:40.140 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:39 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-17T05:54:40.140 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:39 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-17T05:54:40.140 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:39 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-17T05:54:40.140 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:39 smithi047 ceph-mon[32223]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-17T05:54:40.140 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:39 smithi047 ceph-mon[32223]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-17T05:54:40.140 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:39 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-17T05:54:40.140 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:39 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-17T05:54:40.140 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:39 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-17T05:54:40.140 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:39 smithi047 ceph-mon[32223]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-17T05:54:40.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:54:40.219+0000 7f0ec37fe700 1 -- 172.21.15.47:0/1901745096 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f0ebc010460 con 0x7f0ecc103cb0 2024-04-17T05:54:40.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:39 smithi103 ceph-mon[36243]: from='client.14304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T05:54:40.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:39 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-17T05:54:40.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:39 smithi103 ceph-mon[36243]: osdmap e21: 3 total, 3 up, 3 in 2024-04-17T05:54:40.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:39 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:54:40.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:39 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:54:40.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:39 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:54:40.285 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:39 smithi103 ceph-mon[36243]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-17T05:54:40.285 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:39 smithi103 ceph-mon[36243]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-17T05:54:40.285 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:39 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-17T05:54:40.285 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:39 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-17T05:54:40.285 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:39 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-17T05:54:40.285 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:39 smithi103 ceph-mon[36243]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-17T05:54:40.285 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:39 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-17T05:54:40.285 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:39 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-17T05:54:40.285 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:39 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-17T05:54:40.285 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:39 smithi103 ceph-mon[36243]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-17T05:54:40.285 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:39 smithi103 ceph-mon[36243]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-17T05:54:40.285 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:39 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-17T05:54:40.286 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:39 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-17T05:54:40.286 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:39 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-17T05:54:40.286 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:39 smithi103 ceph-mon[36243]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-17T05:54:40.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:39 smithi098 ceph-mon[36516]: from='client.14304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T05:54:40.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:39 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-17T05:54:40.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:39 smithi098 ceph-mon[36516]: osdmap e21: 3 total, 3 up, 3 in 2024-04-17T05:54:40.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:39 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:54:40.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:39 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:54:40.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:39 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:54:40.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:39 smithi098 ceph-mon[36516]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-17T05:54:40.384 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:39 smithi098 ceph-mon[36516]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-17T05:54:40.384 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:39 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-17T05:54:40.384 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:39 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-17T05:54:40.384 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:39 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-17T05:54:40.384 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:39 smithi098 ceph-mon[36516]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-17T05:54:40.384 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:39 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-17T05:54:40.384 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:39 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-17T05:54:40.384 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:39 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-17T05:54:40.384 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:39 smithi098 ceph-mon[36516]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-17T05:54:40.384 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:39 smithi098 ceph-mon[36516]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-17T05:54:40.384 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:39 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-17T05:54:40.384 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:39 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-17T05:54:40.384 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:39 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-17T05:54:40.384 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:39 smithi098 ceph-mon[36516]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-17T05:54:41.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:40 smithi047 ceph-mon[32223]: pgmap v84: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-17T05:54:41.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:40 smithi047 ceph-mon[32223]: osdmap e22: 3 total, 3 up, 3 in 2024-04-17T05:54:41.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:40 smithi047 ceph-mon[32223]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-17T05:54:41.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:40 smithi103 ceph-mon[36243]: pgmap v84: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-17T05:54:41.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:40 smithi103 ceph-mon[36243]: osdmap e22: 3 total, 3 up, 3 in 2024-04-17T05:54:41.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:40 smithi103 ceph-mon[36243]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-17T05:54:41.382 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:40 smithi098 ceph-mon[36516]: pgmap v84: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-17T05:54:41.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:40 smithi098 ceph-mon[36516]: osdmap e22: 3 total, 3 up, 3 in 2024-04-17T05:54:41.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:40 smithi098 ceph-mon[36516]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-17T05:54:42.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:41 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/515673548' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f6e0b0d5-0fb3-42d0-9e84-f7cfef21a998"}]: dispatch 2024-04-17T05:54:42.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:41 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/515673548' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f6e0b0d5-0fb3-42d0-9e84-f7cfef21a998"}]': finished 2024-04-17T05:54:42.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:41 smithi103 ceph-mon[36243]: osdmap e23: 4 total, 3 up, 4 in 2024-04-17T05:54:42.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:41 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-17T05:54:42.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:41 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/515673548' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f6e0b0d5-0fb3-42d0-9e84-f7cfef21a998"}]: dispatch 2024-04-17T05:54:42.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:41 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/515673548' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f6e0b0d5-0fb3-42d0-9e84-f7cfef21a998"}]': finished 2024-04-17T05:54:42.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:41 smithi098 ceph-mon[36516]: osdmap e23: 4 total, 3 up, 4 in 2024-04-17T05:54:42.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:41 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-17T05:54:42.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:41 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/515673548' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f6e0b0d5-0fb3-42d0-9e84-f7cfef21a998"}]: dispatch 2024-04-17T05:54:42.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:41 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/515673548' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f6e0b0d5-0fb3-42d0-9e84-f7cfef21a998"}]': finished 2024-04-17T05:54:42.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:41 smithi047 ceph-mon[32223]: osdmap e23: 4 total, 3 up, 4 in 2024-04-17T05:54:42.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:41 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-17T05:54:43.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:42 smithi103 ceph-mon[36243]: pgmap v86: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-17T05:54:43.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:42 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/1522599097' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-17T05:54:43.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:42 smithi098 ceph-mon[36516]: pgmap v86: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-17T05:54:43.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:42 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/1522599097' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-17T05:54:43.388 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:42 smithi047 ceph-mon[32223]: pgmap v86: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-17T05:54:43.388 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:42 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/1522599097' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-17T05:54:45.201 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:44 smithi047 ceph-mon[32223]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-17T05:54:45.286 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:44 smithi103 ceph-mon[36243]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-17T05:54:45.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:44 smithi098 ceph-mon[36516]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-17T05:54:47.050 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:46 smithi047 ceph-mon[32223]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-17T05:54:47.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:46 smithi103 ceph-mon[36243]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-17T05:54:47.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:46 smithi098 ceph-mon[36516]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-17T05:54:49.036 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:48 smithi047 ceph-mon[32223]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-17T05:54:49.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:48 smithi103 ceph-mon[36243]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-17T05:54:49.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:48 smithi098 ceph-mon[36516]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-17T05:54:50.014 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:49 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-17T05:54:50.014 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:49 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:54:50.014 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:49 smithi047 ceph-mon[32223]: Deploying daemon osd.3 on smithi047 2024-04-17T05:54:50.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:49 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-17T05:54:50.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:49 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:54:50.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:49 smithi103 ceph-mon[36243]: Deploying daemon osd.3 on smithi047 2024-04-17T05:54:50.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:49 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-17T05:54:50.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:49 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:54:50.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:49 smithi098 ceph-mon[36516]: Deploying daemon osd.3 on smithi047 2024-04-17T05:54:51.230 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:50 smithi047 ceph-mon[32223]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-17T05:54:51.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:50 smithi103 ceph-mon[36243]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-17T05:54:51.382 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:50 smithi098 ceph-mon[36516]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-17T05:54:53.121 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:52 smithi047 ceph-mon[32223]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-17T05:54:53.286 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:52 smithi103 ceph-mon[36243]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-17T05:54:53.382 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:52 smithi098 ceph-mon[36516]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-17T05:54:55.229 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:54 smithi047 ceph-mon[32223]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-17T05:54:55.229 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:54 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:54:55.229 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:54 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:54:55.229 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:54 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T05:54:55.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:54 smithi103 ceph-mon[36243]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-17T05:54:55.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:54 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:54:55.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:54 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:54:55.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:54 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T05:54:55.382 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:54 smithi098 ceph-mon[36516]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-17T05:54:55.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:54 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:54:55.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:54 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:54:55.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:54 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T05:54:57.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:56 smithi047 ceph-mon[32223]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-17T05:54:57.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:56 smithi103 ceph-mon[36243]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-17T05:54:57.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:56 smithi098 ceph-mon[36516]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-17T05:54:58.438 INFO:teuthology.orchestra.run.smithi047.stdout:Created osd(s) 3 on host 'smithi047' 2024-04-17T05:54:58.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:54:58.439+0000 7f0ec37fe700 1 -- 172.21.15.47:0/1901745096 <== mgr.14152 v2:172.21.15.47:6800/3157902806 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f0ecc062e90 con 0x7f0eb80a3bb0 2024-04-17T05:54:58.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:54:58.443+0000 7f0ed4584700 1 -- 172.21.15.47:0/1901745096 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f0eb80a3bb0 msgr2=0x7f0eb80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:54:58.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:54:58.443+0000 7f0ed4584700 1 --2- 172.21.15.47:0/1901745096 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f0eb80a3bb0 0x7f0eb80a6070 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc103b10 tx=0x7f0ec8009190 comp rx=0 tx=0).stop 2024-04-17T05:54:58.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:54:58.443+0000 7f0ed4584700 1 -- 172.21.15.47:0/1901745096 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0ecc103cb0 msgr2=0x7f0ecc074e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:54:58.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:54:58.443+0000 7f0ed4584700 1 --2- 172.21.15.47:0/1901745096 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0ecc103cb0 0x7f0ecc074e40 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc00d950 tx=0x7f0ebc00dc60 comp rx=0 tx=0).stop 2024-04-17T05:54:58.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:54:58.443+0000 7f0ed4584700 1 -- 172.21.15.47:0/1901745096 shutdown_connections 2024-04-17T05:54:58.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:54:58.443+0000 7f0ed4584700 1 --2- 172.21.15.47:0/1901745096 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f0eb80a3bb0 0x7f0eb80a6070 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:54:58.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:54:58.443+0000 7f0ed4584700 1 --2- 172.21.15.47:0/1901745096 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ecc104650 0x7f0ecc075380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:54:58.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:54:58.443+0000 7f0ed4584700 1 --2- 172.21.15.47:0/1901745096 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ecc102ab0 0x7f0ecc0767f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:54:58.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:54:58.443+0000 7f0ed4584700 1 --2- 172.21.15.47:0/1901745096 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0ecc103cb0 0x7f0ecc074e40 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:54:58.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:54:58.443+0000 7f0ed4584700 1 -- 172.21.15.47:0/1901745096 >> 172.21.15.47:0/1901745096 conn(0x7f0ecc0fe050 msgr2=0x7f0ecc106f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:54:58.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:54:58.443+0000 7f0ed4584700 1 -- 172.21.15.47:0/1901745096 shutdown_connections 2024-04-17T05:54:58.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:54:58.443+0000 7f0ed4584700 1 -- 172.21.15.47:0/1901745096 wait complete. 2024-04-17T05:54:59.128 DEBUG:teuthology.orchestra.run.smithi047:osd.3> sudo journalctl -f -n 0 -u ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@osd.3.service 2024-04-17T05:54:59.131 INFO:tasks.cephadm:Deploying osd.4 on smithi098 with /dev/vg_nvme/lv_3... 2024-04-17T05:54:59.131 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 565db2d8-fc7e-11ee-bc91-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-17T05:54:59.375 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/mon.b/config 2024-04-17T05:54:59.497 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:59 smithi098 ceph-mon[36516]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-17T05:54:59.497 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:59 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:54:59.497 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:59 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:54:59.497 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:59 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:54:59.497 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:59 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:54:59.497 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:54:59 smithi098 ceph-mon[36516]: from='osd.3 [v2:172.21.15.47:6810/1998369447,v1:172.21.15.47:6811/1998369447]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-17T05:54:59.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:59 smithi047 ceph-mon[32223]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-17T05:54:59.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:59 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:54:59.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:59 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:54:59.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:59 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:54:59.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:59 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:54:59.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:54:59 smithi047 ceph-mon[32223]: from='osd.3 [v2:172.21.15.47:6810/1998369447,v1:172.21.15.47:6811/1998369447]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-17T05:54:59.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:59 smithi103 ceph-mon[36243]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-17T05:54:59.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:59 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:54:59.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:59 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:54:59.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:59 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:54:59.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:59 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:54:59.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:54:59 smithi103 ceph-mon[36243]: from='osd.3 [v2:172.21.15.47:6810/1998369447,v1:172.21.15.47:6811/1998369447]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-17T05:55:00.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:00 smithi047 ceph-mon[32223]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-17T05:55:00.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:00 smithi047 ceph-mon[32223]: from='osd.3 [v2:172.21.15.47:6810/1998369447,v1:172.21.15.47:6811/1998369447]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-17T05:55:00.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:00 smithi047 ceph-mon[32223]: osdmap e24: 4 total, 3 up, 4 in 2024-04-17T05:55:00.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:00 smithi047 ceph-mon[32223]: from='osd.3 [v2:172.21.15.47:6810/1998369447,v1:172.21.15.47:6811/1998369447]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-04-17T05:55:00.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:00 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-17T05:55:00.890 INFO:journalctl@ceph.osd.3.smithi047.stdout:Apr 17 05:55:00 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-3[53909]: 2024-04-17T05:55:00.720+0000 7f62fb1c5700 -1 osd.3 0 waiting for initial osdmap 2024-04-17T05:55:00.890 INFO:journalctl@ceph.osd.3.smithi047.stdout:Apr 17 05:55:00 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-3[53909]: 2024-04-17T05:55:00.725+0000 7f62f6fb8700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-17T05:55:01.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:00 smithi103 ceph-mon[36243]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-17T05:55:01.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:00 smithi103 ceph-mon[36243]: from='osd.3 [v2:172.21.15.47:6810/1998369447,v1:172.21.15.47:6811/1998369447]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-17T05:55:01.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:00 smithi103 ceph-mon[36243]: osdmap e24: 4 total, 3 up, 4 in 2024-04-17T05:55:01.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:00 smithi103 ceph-mon[36243]: from='osd.3 [v2:172.21.15.47:6810/1998369447,v1:172.21.15.47:6811/1998369447]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-04-17T05:55:01.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:00 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-17T05:55:01.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:00 smithi098 ceph-mon[36516]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-17T05:55:01.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:00 smithi098 ceph-mon[36516]: from='osd.3 [v2:172.21.15.47:6810/1998369447,v1:172.21.15.47:6811/1998369447]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-17T05:55:01.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:00 smithi098 ceph-mon[36516]: osdmap e24: 4 total, 3 up, 4 in 2024-04-17T05:55:01.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:00 smithi098 ceph-mon[36516]: from='osd.3 [v2:172.21.15.47:6810/1998369447,v1:172.21.15.47:6811/1998369447]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-04-17T05:55:01.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:00 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-17T05:55:01.250 INFO:teuthology.orchestra.run.smithi098.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-17T05:55:01.250 INFO:teuthology.orchestra.run.smithi098.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-17T05:55:01.250 INFO:teuthology.orchestra.run.smithi098.stderr: stderr: 10+0 records in 2024-04-17T05:55:01.250 INFO:teuthology.orchestra.run.smithi098.stderr:10+0 records out 2024-04-17T05:55:01.250 INFO:teuthology.orchestra.run.smithi098.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.011259 s, 931 MB/s 2024-04-17T05:55:01.250 INFO:teuthology.orchestra.run.smithi098.stderr: stderr: 2024-04-17T05:55:01.251 INFO:teuthology.orchestra.run.smithi098.stderr:--> Zapping successful for: 2024-04-17T05:55:01.838 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:01 smithi098 ceph-mon[36516]: from='osd.3 [v2:172.21.15.47:6810/1998369447,v1:172.21.15.47:6811/1998369447]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]': finished 2024-04-17T05:55:01.838 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:01 smithi098 ceph-mon[36516]: osdmap e25: 4 total, 3 up, 4 in 2024-04-17T05:55:01.838 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:01 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-17T05:55:01.839 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:01 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-17T05:55:01.866 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 565db2d8-fc7e-11ee-bc91-c7b262605968 -- ceph orch daemon add osd smithi098:vg_nvme/lv_3 2024-04-17T05:55:02.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:01 smithi103 ceph-mon[36243]: from='osd.3 [v2:172.21.15.47:6810/1998369447,v1:172.21.15.47:6811/1998369447]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]': finished 2024-04-17T05:55:02.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:01 smithi103 ceph-mon[36243]: osdmap e25: 4 total, 3 up, 4 in 2024-04-17T05:55:02.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:01 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-17T05:55:02.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:01 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-17T05:55:02.041 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:01 smithi047 ceph-mon[32223]: from='osd.3 [v2:172.21.15.47:6810/1998369447,v1:172.21.15.47:6811/1998369447]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]': finished 2024-04-17T05:55:02.041 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:01 smithi047 ceph-mon[32223]: osdmap e25: 4 total, 3 up, 4 in 2024-04-17T05:55:02.041 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:01 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-17T05:55:02.041 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:01 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-17T05:55:02.112 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/mon.b/config 2024-04-17T05:55:02.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:02 smithi098 ceph-mon[36516]: purged_snaps scrub starts 2024-04-17T05:55:02.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:02 smithi098 ceph-mon[36516]: purged_snaps scrub ok 2024-04-17T05:55:02.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:02 smithi098 ceph-mon[36516]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-17T05:55:02.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:02 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-17T05:55:02.884 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:02 smithi098 ceph-mon[36516]: osd.3 [v2:172.21.15.47:6810/1998369447,v1:172.21.15.47:6811/1998369447] boot 2024-04-17T05:55:02.884 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:02 smithi098 ceph-mon[36516]: osdmap e26: 4 total, 4 up, 4 in 2024-04-17T05:55:02.884 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:02 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-17T05:55:02.884 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:02 smithi098 ceph-mon[36516]: Detected new or changed devices on smithi047 2024-04-17T05:55:02.884 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:02 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:55:02.884 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:02 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:55:02.884 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:02 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-17T05:55:02.884 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:02 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-17T05:55:02.884 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:02 smithi098 ceph-mon[36516]: Adjusting osd_memory_target on smithi047 to 8546M 2024-04-17T05:55:02.884 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:02 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:55:02.884 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:02 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:55:02.884 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:02 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T05:55:02.884 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:02 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:55:02.884 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:02 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T05:55:02.885 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:02 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:55:02.885 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:02 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T05:55:02.885 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:02 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:55:03.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:02 smithi103 ceph-mon[36243]: purged_snaps scrub starts 2024-04-17T05:55:03.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:02 smithi103 ceph-mon[36243]: purged_snaps scrub ok 2024-04-17T05:55:03.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:02 smithi103 ceph-mon[36243]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-17T05:55:03.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:02 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-17T05:55:03.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:02 smithi103 ceph-mon[36243]: osd.3 [v2:172.21.15.47:6810/1998369447,v1:172.21.15.47:6811/1998369447] boot 2024-04-17T05:55:03.035 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:02 smithi103 ceph-mon[36243]: osdmap e26: 4 total, 4 up, 4 in 2024-04-17T05:55:03.035 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:02 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-17T05:55:03.035 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:02 smithi103 ceph-mon[36243]: Detected new or changed devices on smithi047 2024-04-17T05:55:03.035 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:02 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:55:03.035 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:02 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:55:03.035 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:02 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-17T05:55:03.035 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:02 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-17T05:55:03.035 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:02 smithi103 ceph-mon[36243]: Adjusting osd_memory_target on smithi047 to 8546M 2024-04-17T05:55:03.035 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:02 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:55:03.035 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:02 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:55:03.035 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:02 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T05:55:03.036 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:02 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:55:03.036 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:02 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T05:55:03.036 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:02 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:55:03.036 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:02 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T05:55:03.036 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:02 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:55:03.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:02 smithi047 ceph-mon[32223]: purged_snaps scrub starts 2024-04-17T05:55:03.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:02 smithi047 ceph-mon[32223]: purged_snaps scrub ok 2024-04-17T05:55:03.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:02 smithi047 ceph-mon[32223]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-17T05:55:03.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:02 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-17T05:55:03.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:02 smithi047 ceph-mon[32223]: osd.3 [v2:172.21.15.47:6810/1998369447,v1:172.21.15.47:6811/1998369447] boot 2024-04-17T05:55:03.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:02 smithi047 ceph-mon[32223]: osdmap e26: 4 total, 4 up, 4 in 2024-04-17T05:55:03.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:02 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-17T05:55:03.140 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:02 smithi047 ceph-mon[32223]: Detected new or changed devices on smithi047 2024-04-17T05:55:03.140 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:02 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:55:03.140 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:02 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:55:03.140 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:02 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-17T05:55:03.140 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:02 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-17T05:55:03.140 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:02 smithi047 ceph-mon[32223]: Adjusting osd_memory_target on smithi047 to 8546M 2024-04-17T05:55:03.140 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:02 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:55:03.140 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:02 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:55:03.140 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:02 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T05:55:03.140 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:02 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:55:03.140 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:02 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T05:55:03.140 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:02 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:55:03.140 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:02 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T05:55:03.140 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:02 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:55:03.713 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:55:03.712+0000 7f3362cb6700 1 -- 172.21.15.98:0/1460581419 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f335c102ab0 msgr2=0x7f335c102ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:55:03.713 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:55:03.712+0000 7f3362cb6700 1 --2- 172.21.15.98:0/1460581419 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f335c102ab0 0x7f335c102ed0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3350009920 tx=0x7f3350009c30 comp rx=0 tx=0).stop 2024-04-17T05:55:03.713 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:55:03.712+0000 7f3362cb6700 1 -- 172.21.15.98:0/1460581419 shutdown_connections 2024-04-17T05:55:03.714 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:55:03.712+0000 7f3362cb6700 1 --2- 172.21.15.98:0/1460581419 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f335c104650 0x7f335c10af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:55:03.714 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:55:03.712+0000 7f3362cb6700 1 --2- 172.21.15.98:0/1460581419 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f335c102ab0 0x7f335c102ed0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:55:03.714 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:55:03.712+0000 7f3362cb6700 1 --2- 172.21.15.98:0/1460581419 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f335c103cb0 0x7f335c104110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:55:03.714 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:55:03.712+0000 7f3362cb6700 1 -- 172.21.15.98:0/1460581419 >> 172.21.15.98:0/1460581419 conn(0x7f335c0fe050 msgr2=0x7f335c100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:55:03.714 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:55:03.712+0000 7f3362cb6700 1 -- 172.21.15.98:0/1460581419 shutdown_connections 2024-04-17T05:55:03.714 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:55:03.712+0000 7f3362cb6700 1 -- 172.21.15.98:0/1460581419 wait complete. 2024-04-17T05:55:03.714 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:55:03.712+0000 7f3362cb6700 1 Processor -- start 2024-04-17T05:55:03.714 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:55:03.712+0000 7f3362cb6700 1 -- start start 2024-04-17T05:55:03.714 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:55:03.712+0000 7f3362cb6700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f335c102ab0 0x7f335c198d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:55:03.714 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:55:03.712+0000 7f3362cb6700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f335c103cb0 0x7f335c199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:55:03.714 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:55:03.712+0000 7f3362cb6700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f335c104650 0x7f335c1a0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:55:03.715 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:55:03.712+0000 7f3362cb6700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f335c10e690 con 0x7f335c103cb0 2024-04-17T05:55:03.715 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:55:03.712+0000 7f3362cb6700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f335c10e510 con 0x7f335c104650 2024-04-17T05:55:03.715 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:55:03.712+0000 7f3362cb6700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f335c10e810 con 0x7f335c102ab0 2024-04-17T05:55:03.715 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:55:03.713+0000 7f3361253700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f335c104650 0x7f335c1a0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:55:03.715 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:55:03.713+0000 7f3361253700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f335c104650 0x7f335c1a0300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34702/0 (socket says 172.21.15.98:34702) 2024-04-17T05:55:03.715 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:55:03.713+0000 7f3361253700 1 -- 172.21.15.98:0/3068234145 learned_addr learned my addr 172.21.15.98:0/3068234145 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-04-17T05:55:03.715 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:55:03.713+0000 7f335bfff700 1 --2- 172.21.15.98:0/3068234145 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f335c103cb0 0x7f335c199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:55:03.715 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:55:03.713+0000 7f3360a52700 1 --2- 172.21.15.98:0/3068234145 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f335c102ab0 0x7f335c198d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:55:03.715 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:55:03.713+0000 7f3361253700 1 -- 172.21.15.98:0/3068234145 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f335c102ab0 msgr2=0x7f335c198d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:55:03.715 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:55:03.713+0000 7f3361253700 1 --2- 172.21.15.98:0/3068234145 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f335c102ab0 0x7f335c198d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:55:03.715 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:55:03.713+0000 7f3361253700 1 -- 172.21.15.98:0/3068234145 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f335c103cb0 msgr2=0x7f335c199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:55:03.715 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:55:03.713+0000 7f3361253700 1 --2- 172.21.15.98:0/3068234145 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f335c103cb0 0x7f335c199280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:55:03.715 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:55:03.713+0000 7f3361253700 1 -- 172.21.15.98:0/3068234145 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f335c1a09a0 con 0x7f335c104650 2024-04-17T05:55:03.715 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:55:03.713+0000 7f335bfff700 1 --2- 172.21.15.98:0/3068234145 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f335c103cb0 0x7f335c199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T05:55:03.716 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:55:03.713+0000 7f3361253700 1 --2- 172.21.15.98:0/3068234145 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f335c104650 0x7f335c1a0300 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f335400ef30 tx=0x7f335400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:55:03.716 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:55:03.713+0000 7f3359ffb700 1 -- 172.21.15.98:0/3068234145 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33540092e0 con 0x7f335c104650 2024-04-17T05:55:03.716 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:55:03.713+0000 7f3359ffb700 1 -- 172.21.15.98:0/3068234145 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3354004bb0 con 0x7f335c104650 2024-04-17T05:55:03.716 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:55:03.713+0000 7f3359ffb700 1 -- 172.21.15.98:0/3068234145 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3354004280 con 0x7f335c104650 2024-04-17T05:55:03.716 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:55:03.713+0000 7f3362cb6700 1 -- 172.21.15.98:0/3068234145 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f335c1a0c30 con 0x7f335c104650 2024-04-17T05:55:03.716 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:55:03.714+0000 7f3362cb6700 1 -- 172.21.15.98:0/3068234145 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f335c1a1170 con 0x7f335c104650 2024-04-17T05:55:03.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:55:03.715+0000 7f3362cb6700 1 -- 172.21.15.98:0/3068234145 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f335c069740 con 0x7f335c104650 2024-04-17T05:55:03.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:55:03.716+0000 7f3359ffb700 1 -- 172.21.15.98:0/3068234145 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f335401b030 con 0x7f335c104650 2024-04-17T05:55:03.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:55:03.716+0000 7f3359ffb700 1 --2- 172.21.15.98:0/3068234145 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f334c0a3c40 0x7f334c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:55:03.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:55:03.716+0000 7f3359ffb700 1 -- 172.21.15.98:0/3068234145 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 3698+0+0 (secure 0 0 0) 0x7f33540cec40 con 0x7f335c104650 2024-04-17T05:55:03.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:55:03.716+0000 7f3360a52700 1 --2- 172.21.15.98:0/3068234145 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f334c0a3c40 0x7f334c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:55:03.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:55:03.717+0000 7f3360a52700 1 --2- 172.21.15.98:0/3068234145 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f334c0a3c40 0x7f334c0a6100 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f335000c010 tx=0x7f3350019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:55:03.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:55:03.721+0000 7f3359ffb700 1 -- 172.21.15.98:0/3068234145 <== mon.2 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f3354022020 con 0x7f335c104650 2024-04-17T05:55:03.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:03 smithi098 ceph-mon[36516]: osdmap e27: 4 total, 4 up, 4 in 2024-04-17T05:55:03.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:55:03.933+0000 7f3362cb6700 1 -- 172.21.15.98:0/3068234145 --> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f335c062e90 con 0x7f334c0a3c40 2024-04-17T05:55:04.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:03 smithi103 ceph-mon[36243]: osdmap e27: 4 total, 4 up, 4 in 2024-04-17T05:55:04.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:03 smithi047 ceph-mon[32223]: osdmap e27: 4 total, 4 up, 4 in 2024-04-17T05:55:04.820 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:04 smithi098 ceph-mon[36516]: pgmap v102: 1 pgs: 1 remapped+peering; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-17T05:55:04.820 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:04 smithi098 ceph-mon[36516]: osdmap e28: 4 total, 4 up, 4 in 2024-04-17T05:55:04.820 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:04 smithi098 ceph-mon[36516]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T05:55:04.820 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:04 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-17T05:55:04.821 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:04 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-17T05:55:04.821 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:04 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:55:04.821 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:04 smithi098 ceph-mon[36516]: osdmap e29: 4 total, 4 up, 4 in 2024-04-17T05:55:05.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:04 smithi103 ceph-mon[36243]: pgmap v102: 1 pgs: 1 remapped+peering; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-17T05:55:05.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:04 smithi103 ceph-mon[36243]: osdmap e28: 4 total, 4 up, 4 in 2024-04-17T05:55:05.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:04 smithi103 ceph-mon[36243]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T05:55:05.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:04 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-17T05:55:05.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:04 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-17T05:55:05.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:04 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:55:05.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:04 smithi103 ceph-mon[36243]: osdmap e29: 4 total, 4 up, 4 in 2024-04-17T05:55:05.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:04 smithi047 ceph-mon[32223]: pgmap v102: 1 pgs: 1 remapped+peering; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-17T05:55:05.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:04 smithi047 ceph-mon[32223]: osdmap e28: 4 total, 4 up, 4 in 2024-04-17T05:55:05.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:04 smithi047 ceph-mon[32223]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T05:55:05.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:04 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-17T05:55:05.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:04 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-17T05:55:05.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:04 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:55:05.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:04 smithi047 ceph-mon[32223]: osdmap e29: 4 total, 4 up, 4 in 2024-04-17T05:55:06.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:06 smithi098 ceph-mon[36516]: osdmap e30: 4 total, 4 up, 4 in 2024-04-17T05:55:06.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:06 smithi047 ceph-mon[32223]: osdmap e30: 4 total, 4 up, 4 in 2024-04-17T05:55:07.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:06 smithi103 ceph-mon[36243]: osdmap e30: 4 total, 4 up, 4 in 2024-04-17T05:55:07.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:07 smithi098 ceph-mon[36516]: pgmap v106: 1 pgs: 1 remapped+peering; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-17T05:55:07.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:07 smithi098 ceph-mon[36516]: from='client.? 172.21.15.98:0/88065135' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2c26217e-02df-466d-bc01-0149d4aec3c5"}]: dispatch 2024-04-17T05:55:07.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:07 smithi098 ceph-mon[36516]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2c26217e-02df-466d-bc01-0149d4aec3c5"}]: dispatch 2024-04-17T05:55:07.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:07 smithi098 ceph-mon[36516]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2c26217e-02df-466d-bc01-0149d4aec3c5"}]': finished 2024-04-17T05:55:07.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:07 smithi098 ceph-mon[36516]: osdmap e31: 5 total, 4 up, 5 in 2024-04-17T05:55:07.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:07 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-17T05:55:07.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:07 smithi098 ceph-mon[36516]: from='client.? 172.21.15.98:0/3744857573' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-17T05:55:07.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:07 smithi047 ceph-mon[32223]: pgmap v106: 1 pgs: 1 remapped+peering; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-17T05:55:07.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:07 smithi047 ceph-mon[32223]: from='client.? 172.21.15.98:0/88065135' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2c26217e-02df-466d-bc01-0149d4aec3c5"}]: dispatch 2024-04-17T05:55:07.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:07 smithi047 ceph-mon[32223]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2c26217e-02df-466d-bc01-0149d4aec3c5"}]: dispatch 2024-04-17T05:55:07.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:07 smithi047 ceph-mon[32223]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2c26217e-02df-466d-bc01-0149d4aec3c5"}]': finished 2024-04-17T05:55:07.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:07 smithi047 ceph-mon[32223]: osdmap e31: 5 total, 4 up, 5 in 2024-04-17T05:55:07.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:07 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-17T05:55:07.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:07 smithi047 ceph-mon[32223]: from='client.? 172.21.15.98:0/3744857573' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-17T05:55:08.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:07 smithi103 ceph-mon[36243]: pgmap v106: 1 pgs: 1 remapped+peering; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-17T05:55:08.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:07 smithi103 ceph-mon[36243]: from='client.? 172.21.15.98:0/88065135' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2c26217e-02df-466d-bc01-0149d4aec3c5"}]: dispatch 2024-04-17T05:55:08.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:07 smithi103 ceph-mon[36243]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2c26217e-02df-466d-bc01-0149d4aec3c5"}]: dispatch 2024-04-17T05:55:08.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:07 smithi103 ceph-mon[36243]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2c26217e-02df-466d-bc01-0149d4aec3c5"}]': finished 2024-04-17T05:55:08.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:07 smithi103 ceph-mon[36243]: osdmap e31: 5 total, 4 up, 5 in 2024-04-17T05:55:08.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:07 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-17T05:55:08.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:07 smithi103 ceph-mon[36243]: from='client.? 172.21.15.98:0/3744857573' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-17T05:55:09.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:09 smithi098 ceph-mon[36516]: pgmap v108: 1 pgs: 1 remapped+peering; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-17T05:55:09.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:09 smithi047 ceph-mon[32223]: pgmap v108: 1 pgs: 1 remapped+peering; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-17T05:55:10.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:09 smithi103 ceph-mon[36243]: pgmap v108: 1 pgs: 1 remapped+peering; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-17T05:55:11.745 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:11 smithi098 ceph-mon[36516]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-04-17T05:55:11.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:11 smithi047 ceph-mon[32223]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-04-17T05:55:12.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:11 smithi103 ceph-mon[36243]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-04-17T05:55:13.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:13 smithi098 ceph-mon[36516]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-04-17T05:55:13.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:13 smithi047 ceph-mon[32223]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-04-17T05:55:14.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:13 smithi103 ceph-mon[36243]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-04-17T05:55:14.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:14 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-17T05:55:14.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:14 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:55:14.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:14 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-17T05:55:14.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:14 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:55:15.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:14 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-17T05:55:15.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:14 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:55:15.741 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:15 smithi098 ceph-mon[36516]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-04-17T05:55:15.741 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:15 smithi098 ceph-mon[36516]: Deploying daemon osd.4 on smithi098 2024-04-17T05:55:15.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:15 smithi047 ceph-mon[32223]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-04-17T05:55:15.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:15 smithi047 ceph-mon[32223]: Deploying daemon osd.4 on smithi098 2024-04-17T05:55:16.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:15 smithi103 ceph-mon[36243]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-04-17T05:55:16.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:15 smithi103 ceph-mon[36243]: Deploying daemon osd.4 on smithi098 2024-04-17T05:55:17.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:17 smithi098 ceph-mon[36516]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-17T05:55:17.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:17 smithi047 ceph-mon[32223]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-17T05:55:18.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:17 smithi103 ceph-mon[36243]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-17T05:55:19.771 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:19 smithi098 ceph-mon[36516]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-04-17T05:55:19.771 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:19 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:55:19.771 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:19 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:55:19.771 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:19 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T05:55:19.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:19 smithi047 ceph-mon[32223]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-04-17T05:55:19.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:19 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:55:19.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:19 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:55:19.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:19 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T05:55:20.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:19 smithi103 ceph-mon[36243]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-04-17T05:55:20.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:19 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:55:20.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:19 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:55:20.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:19 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T05:55:21.838 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:21 smithi098 ceph-mon[36516]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-17T05:55:21.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:21 smithi047 ceph-mon[32223]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-17T05:55:22.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:21 smithi103 ceph-mon[36243]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-17T05:55:22.799 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:55:22.801+0000 7f3359ffb700 1 -- 172.21.15.98:0/3068234145 <== mgr.14152 v2:172.21.15.47:6800/3157902806 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f335c062e90 con 0x7f334c0a3c40 2024-04-17T05:55:22.799 INFO:teuthology.orchestra.run.smithi098.stdout:Created osd(s) 4 on host 'smithi098' 2024-04-17T05:55:22.801 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:55:22.803+0000 7f3362cb6700 1 -- 172.21.15.98:0/3068234145 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f334c0a3c40 msgr2=0x7f334c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:55:22.801 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:55:22.803+0000 7f3362cb6700 1 --2- 172.21.15.98:0/3068234145 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f334c0a3c40 0x7f334c0a6100 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f335000c010 tx=0x7f3350019040 comp rx=0 tx=0).stop 2024-04-17T05:55:22.801 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:55:22.803+0000 7f3362cb6700 1 -- 172.21.15.98:0/3068234145 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f335c104650 msgr2=0x7f335c1a0300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:55:22.801 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:55:22.803+0000 7f3362cb6700 1 --2- 172.21.15.98:0/3068234145 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f335c104650 0x7f335c1a0300 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f335400ef30 tx=0x7f335400c3a0 comp rx=0 tx=0).stop 2024-04-17T05:55:22.802 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:55:22.803+0000 7f3362cb6700 1 -- 172.21.15.98:0/3068234145 shutdown_connections 2024-04-17T05:55:22.802 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:55:22.803+0000 7f3362cb6700 1 --2- 172.21.15.98:0/3068234145 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f334c0a3c40 0x7f334c0a6100 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:55:22.802 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:55:22.803+0000 7f3362cb6700 1 --2- 172.21.15.98:0/3068234145 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f335c104650 0x7f335c1a0300 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:55:22.802 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:55:22.803+0000 7f3362cb6700 1 --2- 172.21.15.98:0/3068234145 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f335c102ab0 0x7f335c198d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:55:22.802 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:55:22.803+0000 7f3362cb6700 1 --2- 172.21.15.98:0/3068234145 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f335c103cb0 0x7f335c199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:55:22.802 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:55:22.803+0000 7f3362cb6700 1 -- 172.21.15.98:0/3068234145 >> 172.21.15.98:0/3068234145 conn(0x7f335c0fe050 msgr2=0x7f335c107030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:55:22.802 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:55:22.803+0000 7f3362cb6700 1 -- 172.21.15.98:0/3068234145 shutdown_connections 2024-04-17T05:55:22.802 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:55:22.803+0000 7f3362cb6700 1 -- 172.21.15.98:0/3068234145 wait complete. 2024-04-17T05:55:23.113 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:22 smithi098 ceph-mon[36516]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-17T05:55:23.113 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:22 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:55:23.113 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:22 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:55:23.113 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:22 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:55:23.113 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:22 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T05:55:23.113 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:22 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:55:23.113 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:22 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:55:23.113 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:22 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:55:23.113 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:22 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T05:55:23.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:22 smithi103 ceph-mon[36243]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-17T05:55:23.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:22 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:55:23.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:22 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:55:23.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:22 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:55:23.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:22 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T05:55:23.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:22 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:55:23.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:22 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:55:23.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:22 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:55:23.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:22 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T05:55:23.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:22 smithi047 ceph-mon[32223]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-17T05:55:23.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:22 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:55:23.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:22 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:55:23.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:22 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:55:23.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:22 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T05:55:23.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:22 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:55:23.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:22 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:55:23.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:22 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:55:23.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:22 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T05:55:23.431 DEBUG:teuthology.orchestra.run.smithi098:osd.4> sudo journalctl -f -n 0 -u ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@osd.4.service 2024-04-17T05:55:23.433 INFO:tasks.cephadm:Deploying osd.5 on smithi103 with /dev/vg_nvme/lv_3... 2024-04-17T05:55:23.433 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 565db2d8-fc7e-11ee-bc91-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-17T05:55:23.680 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/mon.c/config 2024-04-17T05:55:23.984 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:23 smithi098 ceph-mon[36516]: from='osd.4 [v2:172.21.15.98:6808/3060482485,v1:172.21.15.98:6809/3060482485]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-17T05:55:23.984 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:23 smithi098 ceph-mon[36516]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-17T05:55:24.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:23 smithi103 ceph-mon[36243]: from='osd.4 [v2:172.21.15.98:6808/3060482485,v1:172.21.15.98:6809/3060482485]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-17T05:55:24.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:23 smithi103 ceph-mon[36243]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-17T05:55:24.388 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:23 smithi047 ceph-mon[32223]: from='osd.4 [v2:172.21.15.98:6808/3060482485,v1:172.21.15.98:6809/3060482485]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-17T05:55:24.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:23 smithi047 ceph-mon[32223]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-17T05:55:25.001 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:24 smithi103 ceph-mon[36243]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-17T05:55:25.001 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:24 smithi103 ceph-mon[36243]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-17T05:55:25.001 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:24 smithi103 ceph-mon[36243]: from='osd.4 [v2:172.21.15.98:6808/3060482485,v1:172.21.15.98:6809/3060482485]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2024-04-17T05:55:25.001 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:24 smithi103 ceph-mon[36243]: osdmap e32: 5 total, 4 up, 5 in 2024-04-17T05:55:25.002 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:24 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-17T05:55:25.002 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:24 smithi103 ceph-mon[36243]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2024-04-17T05:55:25.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:24 smithi098 ceph-mon[36516]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-17T05:55:25.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:24 smithi098 ceph-mon[36516]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-17T05:55:25.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:24 smithi098 ceph-mon[36516]: from='osd.4 [v2:172.21.15.98:6808/3060482485,v1:172.21.15.98:6809/3060482485]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2024-04-17T05:55:25.134 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:24 smithi098 ceph-mon[36516]: osdmap e32: 5 total, 4 up, 5 in 2024-04-17T05:55:25.134 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:24 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-17T05:55:25.134 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:24 smithi098 ceph-mon[36516]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2024-04-17T05:55:25.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:24 smithi047 ceph-mon[32223]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-17T05:55:25.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:24 smithi047 ceph-mon[32223]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-17T05:55:25.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:24 smithi047 ceph-mon[32223]: from='osd.4 [v2:172.21.15.98:6808/3060482485,v1:172.21.15.98:6809/3060482485]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2024-04-17T05:55:25.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:24 smithi047 ceph-mon[32223]: osdmap e32: 5 total, 4 up, 5 in 2024-04-17T05:55:25.390 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:24 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-17T05:55:25.390 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:24 smithi047 ceph-mon[32223]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2024-04-17T05:55:25.633 INFO:journalctl@ceph.osd.4.smithi098.stdout:Apr 17 05:55:25 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-4[48120]: 2024-04-17T05:55:25.365+0000 7f8cf0474700 -1 osd.4 0 waiting for initial osdmap 2024-04-17T05:55:25.633 INFO:journalctl@ceph.osd.4.smithi098.stdout:Apr 17 05:55:25 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-4[48120]: 2024-04-17T05:55:25.371+0000 7f8ce825f700 -1 osd.4 33 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-17T05:55:25.646 INFO:teuthology.orchestra.run.smithi103.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-17T05:55:25.646 INFO:teuthology.orchestra.run.smithi103.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-17T05:55:25.646 INFO:teuthology.orchestra.run.smithi103.stderr: stderr: 10+0 records in 2024-04-17T05:55:25.647 INFO:teuthology.orchestra.run.smithi103.stderr:10+0 records out 2024-04-17T05:55:25.647 INFO:teuthology.orchestra.run.smithi103.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.013723 s, 764 MB/s 2024-04-17T05:55:25.647 INFO:teuthology.orchestra.run.smithi103.stderr: stderr: 2024-04-17T05:55:25.647 INFO:teuthology.orchestra.run.smithi103.stderr:--> Zapping successful for: 2024-04-17T05:55:26.018 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:25 smithi098 ceph-mon[36516]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]': finished 2024-04-17T05:55:26.018 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:25 smithi098 ceph-mon[36516]: osdmap e33: 5 total, 4 up, 5 in 2024-04-17T05:55:26.018 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:25 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-17T05:55:26.019 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:25 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-17T05:55:26.019 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:25 smithi098 ceph-mon[36516]: from='osd.4 ' entity='osd.4' 2024-04-17T05:55:26.246 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:25 smithi103 ceph-mon[36243]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]': finished 2024-04-17T05:55:26.246 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:25 smithi103 ceph-mon[36243]: osdmap e33: 5 total, 4 up, 5 in 2024-04-17T05:55:26.246 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:25 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-17T05:55:26.246 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:25 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-17T05:55:26.246 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:25 smithi103 ceph-mon[36243]: from='osd.4 ' entity='osd.4' 2024-04-17T05:55:26.274 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 565db2d8-fc7e-11ee-bc91-c7b262605968 -- ceph orch daemon add osd smithi103:vg_nvme/lv_3 2024-04-17T05:55:26.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:25 smithi047 ceph-mon[32223]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]': finished 2024-04-17T05:55:26.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:25 smithi047 ceph-mon[32223]: osdmap e33: 5 total, 4 up, 5 in 2024-04-17T05:55:26.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:25 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-17T05:55:26.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:25 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-17T05:55:26.390 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:25 smithi047 ceph-mon[32223]: from='osd.4 ' entity='osd.4' 2024-04-17T05:55:26.510 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/mon.c/config 2024-04-17T05:55:27.186 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:26 smithi103 ceph-mon[36243]: purged_snaps scrub starts 2024-04-17T05:55:27.186 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:26 smithi103 ceph-mon[36243]: purged_snaps scrub ok 2024-04-17T05:55:27.186 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:26 smithi103 ceph-mon[36243]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-17T05:55:27.186 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:26 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-17T05:55:27.186 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:26 smithi103 ceph-mon[36243]: osd.4 [v2:172.21.15.98:6808/3060482485,v1:172.21.15.98:6809/3060482485] boot 2024-04-17T05:55:27.186 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:26 smithi103 ceph-mon[36243]: osdmap e34: 5 total, 5 up, 5 in 2024-04-17T05:55:27.186 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:26 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-17T05:55:27.186 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:26 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:55:27.186 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:26 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:55:27.186 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:26 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-17T05:55:27.187 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:26 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-17T05:55:27.187 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:26 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:55:27.187 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:26 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:55:27.187 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:26 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T05:55:27.187 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:26 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:55:27.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:26 smithi098 ceph-mon[36516]: purged_snaps scrub starts 2024-04-17T05:55:27.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:26 smithi098 ceph-mon[36516]: purged_snaps scrub ok 2024-04-17T05:55:27.384 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:26 smithi098 ceph-mon[36516]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-17T05:55:27.384 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:26 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-17T05:55:27.384 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:26 smithi098 ceph-mon[36516]: osd.4 [v2:172.21.15.98:6808/3060482485,v1:172.21.15.98:6809/3060482485] boot 2024-04-17T05:55:27.384 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:26 smithi098 ceph-mon[36516]: osdmap e34: 5 total, 5 up, 5 in 2024-04-17T05:55:27.384 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:26 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-17T05:55:27.385 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:26 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:55:27.385 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:26 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:55:27.385 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:26 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-17T05:55:27.385 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:26 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-17T05:55:27.385 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:26 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:55:27.385 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:26 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:55:27.386 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:26 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T05:55:27.386 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:26 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:55:27.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:26 smithi047 ceph-mon[32223]: purged_snaps scrub starts 2024-04-17T05:55:27.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:26 smithi047 ceph-mon[32223]: purged_snaps scrub ok 2024-04-17T05:55:27.390 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:26 smithi047 ceph-mon[32223]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-17T05:55:27.390 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:26 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-17T05:55:27.390 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:26 smithi047 ceph-mon[32223]: osd.4 [v2:172.21.15.98:6808/3060482485,v1:172.21.15.98:6809/3060482485] boot 2024-04-17T05:55:27.390 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:26 smithi047 ceph-mon[32223]: osdmap e34: 5 total, 5 up, 5 in 2024-04-17T05:55:27.390 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:26 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-17T05:55:27.390 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:26 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:55:27.391 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:26 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:55:27.391 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:26 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-17T05:55:27.391 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:26 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-17T05:55:27.391 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:26 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:55:27.391 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:26 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:55:27.392 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:26 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T05:55:27.392 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:26 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:55:27.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:55:27.965+0000 7f77ee420700 1 -- 172.21.15.103:0/3287695423 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f77e81019c0 msgr2=0x7f77e8101de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:55:27.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:55:27.965+0000 7f77ee420700 1 --2- 172.21.15.103:0/3287695423 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f77e81019c0 0x7f77e8101de0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f77d8009990 tx=0x7f77d8009ca0 comp rx=0 tx=0).stop 2024-04-17T05:55:27.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:55:27.965+0000 7f77ee420700 1 -- 172.21.15.103:0/3287695423 shutdown_connections 2024-04-17T05:55:27.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:55:27.965+0000 7f77ee420700 1 --2- 172.21.15.103:0/3287695423 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f77e8103560 0x7f77e8109e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:55:27.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:55:27.965+0000 7f77ee420700 1 --2- 172.21.15.103:0/3287695423 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f77e81019c0 0x7f77e8101de0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:55:27.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:55:27.965+0000 7f77ee420700 1 --2- 172.21.15.103:0/3287695423 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f77e8102bc0 0x7f77e8103020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:55:27.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:55:27.965+0000 7f77ee420700 1 -- 172.21.15.103:0/3287695423 >> 172.21.15.103:0/3287695423 conn(0x7f77e80fcf80 msgr2=0x7f77e80ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:55:27.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:55:27.965+0000 7f77ee420700 1 -- 172.21.15.103:0/3287695423 shutdown_connections 2024-04-17T05:55:27.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:55:27.965+0000 7f77ee420700 1 -- 172.21.15.103:0/3287695423 wait complete. 2024-04-17T05:55:27.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:55:27.965+0000 7f77ee420700 1 Processor -- start 2024-04-17T05:55:27.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:55:27.965+0000 7f77ee420700 1 -- start start 2024-04-17T05:55:27.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:55:27.966+0000 7f77ee420700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f77e81019c0 0x7f77e8197c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:55:27.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:55:27.966+0000 7f77ee420700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f77e8102bc0 0x7f77e8198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:55:27.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:55:27.966+0000 7f77ee420700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f77e8103560 0x7f77e819f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:55:27.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:55:27.966+0000 7f77ee420700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f77e810d5a0 con 0x7f77e8103560 2024-04-17T05:55:27.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:55:27.966+0000 7f77ee420700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f77e810d420 con 0x7f77e8102bc0 2024-04-17T05:55:27.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:55:27.966+0000 7f77ee420700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f77e810d720 con 0x7f77e81019c0 2024-04-17T05:55:27.991 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:55:27.966+0000 7f77e7fff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f77e81019c0 0x7f77e8197c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:55:27.991 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:55:27.966+0000 7f77e7fff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f77e81019c0 0x7f77e8197c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42790/0 (socket says 172.21.15.103:42790) 2024-04-17T05:55:27.991 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:55:27.966+0000 7f77e7fff700 1 -- 172.21.15.103:0/232433091 learned_addr learned my addr 172.21.15.103:0/232433091 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-04-17T05:55:27.991 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:55:27.966+0000 7f77e77fe700 1 --2- 172.21.15.103:0/232433091 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f77e8102bc0 0x7f77e8198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:55:27.991 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:55:27.966+0000 7f77ec9bd700 1 --2- 172.21.15.103:0/232433091 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f77e8103560 0x7f77e819f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:55:27.991 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:55:27.966+0000 7f77e7fff700 1 -- 172.21.15.103:0/232433091 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f77e8102bc0 msgr2=0x7f77e8198190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:55:27.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:55:27.966+0000 7f77e7fff700 1 --2- 172.21.15.103:0/232433091 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f77e8102bc0 0x7f77e8198190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:55:27.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:55:27.966+0000 7f77e7fff700 1 -- 172.21.15.103:0/232433091 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f77e8103560 msgr2=0x7f77e819f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:55:27.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:55:27.966+0000 7f77e7fff700 1 --2- 172.21.15.103:0/232433091 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f77e8103560 0x7f77e819f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:55:27.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:55:27.966+0000 7f77e7fff700 1 -- 172.21.15.103:0/232433091 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77e819f8a0 con 0x7f77e81019c0 2024-04-17T05:55:27.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:55:27.966+0000 7f77e7fff700 1 --2- 172.21.15.103:0/232433091 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f77e81019c0 0x7f77e8197c50 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f77d80098e0 tx=0x7f77d8005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:55:27.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:55:27.966+0000 7f77e57fa700 1 -- 172.21.15.103:0/232433091 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d8027070 con 0x7f77e81019c0 2024-04-17T05:55:27.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:55:27.967+0000 7f77ee420700 1 -- 172.21.15.103:0/232433091 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77e819fb30 con 0x7f77e81019c0 2024-04-17T05:55:27.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:55:27.967+0000 7f77e57fa700 1 -- 172.21.15.103:0/232433091 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77d8003f40 con 0x7f77e81019c0 2024-04-17T05:55:27.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:55:27.967+0000 7f77e57fa700 1 -- 172.21.15.103:0/232433091 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d8022e00 con 0x7f77e81019c0 2024-04-17T05:55:27.993 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:55:27.967+0000 7f77ee420700 1 -- 172.21.15.103:0/232433091 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77e819ffe0 con 0x7f77e81019c0 2024-04-17T05:55:27.993 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:55:27.967+0000 7f77ee420700 1 -- 172.21.15.103:0/232433091 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77e8069740 con 0x7f77e81019c0 2024-04-17T05:55:27.993 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:55:27.968+0000 7f77e57fa700 1 -- 172.21.15.103:0/232433091 <== mon.1 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f77d802a070 con 0x7f77e81019c0 2024-04-17T05:55:27.993 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:55:27.969+0000 7f77e57fa700 1 --2- 172.21.15.103:0/232433091 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f77d00a3c40 0x7f77d00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:55:27.993 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:55:27.969+0000 7f77e57fa700 1 -- 172.21.15.103:0/232433091 <== mon.1 v2:172.21.15.103:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4101+0+0 (secure 0 0 0) 0x7f77d80d0090 con 0x7f77e81019c0 2024-04-17T05:55:27.993 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:55:27.969+0000 7f77e77fe700 1 --2- 172.21.15.103:0/232433091 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f77d00a3c40 0x7f77d00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:55:27.993 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:55:27.970+0000 7f77e77fe700 1 --2- 172.21.15.103:0/232433091 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f77d00a3c40 0x7f77d00a6100 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f77dc005950 tx=0x7f77dc00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:55:27.993 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:55:27.974+0000 7f77e57fa700 1 -- 172.21.15.103:0/232433091 <== mon.1 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f77d805eff0 con 0x7f77e81019c0 2024-04-17T05:55:28.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:27 smithi103 ceph-mon[36243]: Detected new or changed devices on smithi098 2024-04-17T05:55:28.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:27 smithi103 ceph-mon[36243]: Adjusting osd_memory_target on smithi098 to 8540M 2024-04-17T05:55:28.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:55:28.181+0000 7f77ee420700 1 -- 172.21.15.103:0/232433091 --> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f77e8062e90 con 0x7f77d00a3c40 2024-04-17T05:55:28.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:27 smithi098 ceph-mon[36516]: Detected new or changed devices on smithi098 2024-04-17T05:55:28.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:27 smithi098 ceph-mon[36516]: Adjusting osd_memory_target on smithi098 to 8540M 2024-04-17T05:55:28.388 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:27 smithi047 ceph-mon[32223]: Detected new or changed devices on smithi098 2024-04-17T05:55:28.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:27 smithi047 ceph-mon[32223]: Adjusting osd_memory_target on smithi098 to 8540M 2024-04-17T05:55:29.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:28 smithi103 ceph-mon[36243]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-17T05:55:29.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:28 smithi103 ceph-mon[36243]: osdmap e35: 5 total, 5 up, 5 in 2024-04-17T05:55:29.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:28 smithi103 ceph-mon[36243]: from='client.24229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T05:55:29.035 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:28 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-17T05:55:29.035 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:28 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-17T05:55:29.035 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:28 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:55:29.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:28 smithi098 ceph-mon[36516]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-17T05:55:29.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:28 smithi098 ceph-mon[36516]: osdmap e35: 5 total, 5 up, 5 in 2024-04-17T05:55:29.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:28 smithi098 ceph-mon[36516]: from='client.24229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T05:55:29.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:28 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-17T05:55:29.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:28 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-17T05:55:29.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:28 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:55:29.388 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:28 smithi047 ceph-mon[32223]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-17T05:55:29.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:28 smithi047 ceph-mon[32223]: osdmap e35: 5 total, 5 up, 5 in 2024-04-17T05:55:29.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:28 smithi047 ceph-mon[32223]: from='client.24229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T05:55:29.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:28 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-17T05:55:29.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:28 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-17T05:55:29.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:28 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:55:31.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:30 smithi103 ceph-mon[36243]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-17T05:55:31.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:30 smithi103 ceph-mon[36243]: from='client.? 172.21.15.103:0/885444878' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "740bf3e0-bea8-496b-8d6f-e42d0b017258"}]: dispatch 2024-04-17T05:55:31.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:30 smithi103 ceph-mon[36243]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "740bf3e0-bea8-496b-8d6f-e42d0b017258"}]: dispatch 2024-04-17T05:55:31.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:30 smithi103 ceph-mon[36243]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "740bf3e0-bea8-496b-8d6f-e42d0b017258"}]': finished 2024-04-17T05:55:31.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:30 smithi103 ceph-mon[36243]: osdmap e36: 6 total, 5 up, 6 in 2024-04-17T05:55:31.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:30 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-17T05:55:31.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:30 smithi098 ceph-mon[36516]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-17T05:55:31.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:30 smithi098 ceph-mon[36516]: from='client.? 172.21.15.103:0/885444878' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "740bf3e0-bea8-496b-8d6f-e42d0b017258"}]: dispatch 2024-04-17T05:55:31.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:30 smithi098 ceph-mon[36516]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "740bf3e0-bea8-496b-8d6f-e42d0b017258"}]: dispatch 2024-04-17T05:55:31.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:30 smithi098 ceph-mon[36516]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "740bf3e0-bea8-496b-8d6f-e42d0b017258"}]': finished 2024-04-17T05:55:31.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:30 smithi098 ceph-mon[36516]: osdmap e36: 6 total, 5 up, 6 in 2024-04-17T05:55:31.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:30 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-17T05:55:31.388 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:30 smithi047 ceph-mon[32223]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-17T05:55:31.388 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:30 smithi047 ceph-mon[32223]: from='client.? 172.21.15.103:0/885444878' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "740bf3e0-bea8-496b-8d6f-e42d0b017258"}]: dispatch 2024-04-17T05:55:31.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:30 smithi047 ceph-mon[32223]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "740bf3e0-bea8-496b-8d6f-e42d0b017258"}]: dispatch 2024-04-17T05:55:31.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:30 smithi047 ceph-mon[32223]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "740bf3e0-bea8-496b-8d6f-e42d0b017258"}]': finished 2024-04-17T05:55:31.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:30 smithi047 ceph-mon[32223]: osdmap e36: 6 total, 5 up, 6 in 2024-04-17T05:55:31.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:30 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-17T05:55:32.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:31 smithi103 ceph-mon[36243]: from='client.? 172.21.15.103:0/3499073620' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-17T05:55:32.382 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:31 smithi098 ceph-mon[36516]: from='client.? 172.21.15.103:0/3499073620' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-17T05:55:32.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:31 smithi047 ceph-mon[32223]: from='client.? 172.21.15.103:0/3499073620' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-17T05:55:33.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:32 smithi103 ceph-mon[36243]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-17T05:55:33.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:32 smithi098 ceph-mon[36516]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-17T05:55:33.388 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:32 smithi047 ceph-mon[32223]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-17T05:55:35.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:34 smithi103 ceph-mon[36243]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-17T05:55:35.382 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:34 smithi098 ceph-mon[36516]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-17T05:55:35.388 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:34 smithi047 ceph-mon[32223]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-17T05:55:37.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:36 smithi103 ceph-mon[36243]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-17T05:55:37.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:36 smithi098 ceph-mon[36516]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-17T05:55:37.388 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:36 smithi047 ceph-mon[32223]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-17T05:55:39.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:38 smithi103 ceph-mon[36243]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-17T05:55:39.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:38 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-17T05:55:39.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:38 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:55:39.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:38 smithi103 ceph-mon[36243]: Deploying daemon osd.5 on smithi103 2024-04-17T05:55:39.382 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:38 smithi098 ceph-mon[36516]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-17T05:55:39.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:38 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-17T05:55:39.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:38 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:55:39.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:38 smithi098 ceph-mon[36516]: Deploying daemon osd.5 on smithi103 2024-04-17T05:55:39.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:38 smithi047 ceph-mon[32223]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-17T05:55:39.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:38 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-17T05:55:39.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:38 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:55:39.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:38 smithi047 ceph-mon[32223]: Deploying daemon osd.5 on smithi103 2024-04-17T05:55:41.239 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:40 smithi103 ceph-mon[36243]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-17T05:55:41.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:40 smithi098 ceph-mon[36516]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-17T05:55:41.388 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:40 smithi047 ceph-mon[32223]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-17T05:55:43.265 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:42 smithi103 ceph-mon[36243]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-17T05:55:43.385 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:42 smithi098 ceph-mon[36516]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-17T05:55:43.388 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:42 smithi047 ceph-mon[32223]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-17T05:55:44.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:44 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:55:44.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:44 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:55:44.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:44 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T05:55:44.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:44 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:55:44.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:44 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:55:44.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:44 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T05:55:45.023 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:44 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:55:45.023 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:44 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:55:45.023 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:44 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T05:55:45.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:45 smithi098 ceph-mon[36516]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-17T05:55:45.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:45 smithi047 ceph-mon[32223]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-17T05:55:46.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:45 smithi103 ceph-mon[36243]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-17T05:55:47.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:55:47.500+0000 7f77e57fa700 1 -- 172.21.15.103:0/232433091 <== mgr.14152 v2:172.21.15.47:6800/3157902806 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f77e8062e90 con 0x7f77d00a3c40 2024-04-17T05:55:47.500 INFO:teuthology.orchestra.run.smithi103.stdout:Created osd(s) 5 on host 'smithi103' 2024-04-17T05:55:47.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:55:47.503+0000 7f77ee420700 1 -- 172.21.15.103:0/232433091 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f77d00a3c40 msgr2=0x7f77d00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:55:47.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:55:47.503+0000 7f77ee420700 1 --2- 172.21.15.103:0/232433091 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f77d00a3c40 0x7f77d00a6100 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f77dc005950 tx=0x7f77dc00b410 comp rx=0 tx=0).stop 2024-04-17T05:55:47.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:55:47.503+0000 7f77ee420700 1 -- 172.21.15.103:0/232433091 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f77e81019c0 msgr2=0x7f77e8197c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:55:47.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:55:47.503+0000 7f77ee420700 1 --2- 172.21.15.103:0/232433091 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f77e81019c0 0x7f77e8197c50 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f77d80098e0 tx=0x7f77d8005e80 comp rx=0 tx=0).stop 2024-04-17T05:55:47.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:55:47.503+0000 7f77ee420700 1 -- 172.21.15.103:0/232433091 shutdown_connections 2024-04-17T05:55:47.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:55:47.503+0000 7f77ee420700 1 --2- 172.21.15.103:0/232433091 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f77d00a3c40 0x7f77d00a6100 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:55:47.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:55:47.503+0000 7f77ee420700 1 --2- 172.21.15.103:0/232433091 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f77e8103560 0x7f77e819f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:55:47.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:55:47.503+0000 7f77ee420700 1 --2- 172.21.15.103:0/232433091 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f77e81019c0 0x7f77e8197c50 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:55:47.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:55:47.503+0000 7f77ee420700 1 --2- 172.21.15.103:0/232433091 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f77e8102bc0 0x7f77e8198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:55:47.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:55:47.503+0000 7f77ee420700 1 -- 172.21.15.103:0/232433091 >> 172.21.15.103:0/232433091 conn(0x7f77e80fcf80 msgr2=0x7f77e8105f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:55:47.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:55:47.503+0000 7f77ee420700 1 -- 172.21.15.103:0/232433091 shutdown_connections 2024-04-17T05:55:47.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:55:47.503+0000 7f77ee420700 1 -- 172.21.15.103:0/232433091 wait complete. 2024-04-17T05:55:47.783 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:47 smithi103 ceph-mon[36243]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-17T05:55:47.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:47 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:55:47.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:47 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:55:47.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:47 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:55:47.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:47 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T05:55:47.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:47 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:55:47.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:47 smithi103 ceph-mon[36243]: from='osd.5 [v2:172.21.15.103:6808/4181363567,v1:172.21.15.103:6809/4181363567]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-17T05:55:47.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:47 smithi103 ceph-mon[36243]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-17T05:55:47.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:47 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:55:47.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:47 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:55:47.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:47 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T05:55:47.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:47 smithi098 ceph-mon[36516]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-17T05:55:47.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:47 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:55:47.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:47 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:55:47.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:47 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:55:47.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:47 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T05:55:47.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:47 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:55:47.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:47 smithi098 ceph-mon[36516]: from='osd.5 [v2:172.21.15.103:6808/4181363567,v1:172.21.15.103:6809/4181363567]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-17T05:55:47.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:47 smithi098 ceph-mon[36516]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-17T05:55:47.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:47 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:55:47.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:47 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:55:47.884 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:47 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T05:55:47.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:47 smithi047 ceph-mon[32223]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-17T05:55:47.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:47 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:55:47.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:47 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:55:47.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:47 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:55:47.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:47 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T05:55:47.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:47 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:55:47.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:47 smithi047 ceph-mon[32223]: from='osd.5 [v2:172.21.15.103:6808/4181363567,v1:172.21.15.103:6809/4181363567]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-17T05:55:47.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:47 smithi047 ceph-mon[32223]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-17T05:55:47.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:47 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:55:47.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:47 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:55:47.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:47 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T05:55:48.082 DEBUG:teuthology.orchestra.run.smithi103:osd.5> sudo journalctl -f -n 0 -u ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@osd.5.service 2024-04-17T05:55:48.085 INFO:tasks.cephadm:Deploying osd.6 on smithi047 with /dev/vg_nvme/lv_2... 2024-04-17T05:55:48.085 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 565db2d8-fc7e-11ee-bc91-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-17T05:55:48.343 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T05:55:48.783 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:48 smithi103 ceph-mon[36243]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-17T05:55:48.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:48 smithi103 ceph-mon[36243]: osdmap e37: 6 total, 5 up, 6 in 2024-04-17T05:55:48.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:48 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-17T05:55:48.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:48 smithi103 ceph-mon[36243]: from='osd.5 [v2:172.21.15.103:6808/4181363567,v1:172.21.15.103:6809/4181363567]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-04-17T05:55:48.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:48 smithi103 ceph-mon[36243]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-04-17T05:55:48.784 INFO:journalctl@ceph.osd.5.smithi103.stdout:Apr 17 05:55:48 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-5[48375]: 2024-04-17T05:55:48.587+0000 7f6ed017d700 -1 osd.5 0 waiting for initial osdmap 2024-04-17T05:55:48.784 INFO:journalctl@ceph.osd.5.smithi103.stdout:Apr 17 05:55:48 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-5[48375]: 2024-04-17T05:55:48.593+0000 7f6ecbf70700 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-17T05:55:48.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:48 smithi047 ceph-mon[32223]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-17T05:55:48.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:48 smithi047 ceph-mon[32223]: osdmap e37: 6 total, 5 up, 6 in 2024-04-17T05:55:48.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:48 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-17T05:55:48.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:48 smithi047 ceph-mon[32223]: from='osd.5 [v2:172.21.15.103:6808/4181363567,v1:172.21.15.103:6809/4181363567]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-04-17T05:55:48.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:48 smithi047 ceph-mon[32223]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-04-17T05:55:48.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:48 smithi098 ceph-mon[36516]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-17T05:55:48.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:48 smithi098 ceph-mon[36516]: osdmap e37: 6 total, 5 up, 6 in 2024-04-17T05:55:48.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:48 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-17T05:55:48.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:48 smithi098 ceph-mon[36516]: from='osd.5 [v2:172.21.15.103:6808/4181363567,v1:172.21.15.103:6809/4181363567]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-04-17T05:55:48.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:48 smithi098 ceph-mon[36516]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-04-17T05:55:49.857 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:49 smithi047 ceph-mon[32223]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-17T05:55:49.858 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:49 smithi047 ceph-mon[32223]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-04-17T05:55:49.858 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:49 smithi047 ceph-mon[32223]: osdmap e38: 6 total, 5 up, 6 in 2024-04-17T05:55:49.858 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:49 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-17T05:55:49.858 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:49 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-17T05:55:49.858 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:49 smithi047 ceph-mon[32223]: osd.5 [v2:172.21.15.103:6808/4181363567,v1:172.21.15.103:6809/4181363567] boot 2024-04-17T05:55:49.858 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:49 smithi047 ceph-mon[32223]: osdmap e39: 6 total, 6 up, 6 in 2024-04-17T05:55:49.858 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:49 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-17T05:55:49.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:49 smithi098 ceph-mon[36516]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-17T05:55:49.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:49 smithi098 ceph-mon[36516]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-04-17T05:55:49.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:49 smithi098 ceph-mon[36516]: osdmap e38: 6 total, 5 up, 6 in 2024-04-17T05:55:49.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:49 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-17T05:55:49.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:49 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-17T05:55:49.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:49 smithi098 ceph-mon[36516]: osd.5 [v2:172.21.15.103:6808/4181363567,v1:172.21.15.103:6809/4181363567] boot 2024-04-17T05:55:49.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:49 smithi098 ceph-mon[36516]: osdmap e39: 6 total, 6 up, 6 in 2024-04-17T05:55:49.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:49 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-17T05:55:50.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:49 smithi103 ceph-mon[36243]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-17T05:55:50.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:49 smithi103 ceph-mon[36243]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-04-17T05:55:50.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:49 smithi103 ceph-mon[36243]: osdmap e38: 6 total, 5 up, 6 in 2024-04-17T05:55:50.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:49 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-17T05:55:50.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:49 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-17T05:55:50.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:49 smithi103 ceph-mon[36243]: osd.5 [v2:172.21.15.103:6808/4181363567,v1:172.21.15.103:6809/4181363567] boot 2024-04-17T05:55:50.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:49 smithi103 ceph-mon[36243]: osdmap e39: 6 total, 6 up, 6 in 2024-04-17T05:55:50.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:49 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-17T05:55:50.611 INFO:teuthology.orchestra.run.smithi047.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-17T05:55:50.611 INFO:teuthology.orchestra.run.smithi047.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-17T05:55:50.611 INFO:teuthology.orchestra.run.smithi047.stderr: stderr: 10+0 records in 2024-04-17T05:55:50.612 INFO:teuthology.orchestra.run.smithi047.stderr:10+0 records out 2024-04-17T05:55:50.612 INFO:teuthology.orchestra.run.smithi047.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00845962 s, 1.2 GB/s 2024-04-17T05:55:50.612 INFO:teuthology.orchestra.run.smithi047.stderr:--> Zapping successful for: 2024-04-17T05:55:50.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:50 smithi103 ceph-mon[36243]: purged_snaps scrub starts 2024-04-17T05:55:50.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:50 smithi103 ceph-mon[36243]: purged_snaps scrub ok 2024-04-17T05:55:50.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:50 smithi103 ceph-mon[36243]: osdmap e40: 6 total, 6 up, 6 in 2024-04-17T05:55:50.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:50 smithi098 ceph-mon[36516]: purged_snaps scrub starts 2024-04-17T05:55:50.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:50 smithi098 ceph-mon[36516]: purged_snaps scrub ok 2024-04-17T05:55:50.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:50 smithi098 ceph-mon[36516]: osdmap e40: 6 total, 6 up, 6 in 2024-04-17T05:55:50.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:50 smithi047 ceph-mon[32223]: purged_snaps scrub starts 2024-04-17T05:55:50.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:50 smithi047 ceph-mon[32223]: purged_snaps scrub ok 2024-04-17T05:55:50.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:50 smithi047 ceph-mon[32223]: osdmap e40: 6 total, 6 up, 6 in 2024-04-17T05:55:51.455 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 565db2d8-fc7e-11ee-bc91-c7b262605968 -- ceph orch daemon add osd smithi047:vg_nvme/lv_2 2024-04-17T05:55:51.702 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T05:55:51.824 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:51 smithi047 ceph-mon[32223]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail 2024-04-17T05:55:51.824 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:51 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:55:51.824 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:51 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:55:51.824 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:51 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-17T05:55:51.824 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:51 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-17T05:55:51.824 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:51 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:55:51.824 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:51 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:55:51.824 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:51 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T05:55:51.825 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:51 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:55:51.825 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:51 smithi047 ceph-mon[32223]: osdmap e41: 6 total, 6 up, 6 in 2024-04-17T05:55:51.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:51 smithi098 ceph-mon[36516]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail 2024-04-17T05:55:51.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:51 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:55:51.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:51 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:55:51.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:51 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-17T05:55:51.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:51 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-17T05:55:51.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:51 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:55:51.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:51 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:55:51.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:51 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T05:55:51.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:51 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:55:51.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:51 smithi098 ceph-mon[36516]: osdmap e41: 6 total, 6 up, 6 in 2024-04-17T05:55:52.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:51 smithi103 ceph-mon[36243]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail 2024-04-17T05:55:52.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:51 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:55:52.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:51 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:55:52.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:51 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-17T05:55:52.035 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:51 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-17T05:55:52.035 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:51 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:55:52.035 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:51 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:55:52.035 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:51 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T05:55:52.035 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:51 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:55:52.035 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:51 smithi103 ceph-mon[36243]: osdmap e41: 6 total, 6 up, 6 in 2024-04-17T05:55:52.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:52 smithi047 ceph-mon[32223]: Detected new or changed devices on smithi103 2024-04-17T05:55:52.862 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:52 smithi047 ceph-mon[32223]: Adjusting osd_memory_target on smithi103 to 8540M 2024-04-17T05:55:52.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:52 smithi098 ceph-mon[36516]: Detected new or changed devices on smithi103 2024-04-17T05:55:52.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:52 smithi098 ceph-mon[36516]: Adjusting osd_memory_target on smithi103 to 8540M 2024-04-17T05:55:53.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:52 smithi103 ceph-mon[36243]: Detected new or changed devices on smithi103 2024-04-17T05:55:53.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:52 smithi103 ceph-mon[36243]: Adjusting osd_memory_target on smithi103 to 8540M 2024-04-17T05:55:53.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:55:53.517+0000 7f9229a14700 1 -- 172.21.15.47:0/3552972589 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9224105eb0 msgr2=0x7f92241082a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:55:53.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:55:53.517+0000 7f9229a14700 1 --2- 172.21.15.47:0/3552972589 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9224105eb0 0x7f92241082a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f9218009990 tx=0x7f9218009ca0 comp rx=0 tx=0).stop 2024-04-17T05:55:53.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:55:53.518+0000 7f9229a14700 1 -- 172.21.15.47:0/3552972589 shutdown_connections 2024-04-17T05:55:53.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:55:53.518+0000 7f9229a14700 1 --2- 172.21.15.47:0/3552972589 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f92241087e0 0x7f922410ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:55:53.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:55:53.518+0000 7f9229a14700 1 --2- 172.21.15.47:0/3552972589 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9224103580 0x7f9224105970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:55:53.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:55:53.518+0000 7f9229a14700 1 --2- 172.21.15.47:0/3552972589 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9224105eb0 0x7f92241082a0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:55:53.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:55:53.518+0000 7f9229a14700 1 -- 172.21.15.47:0/3552972589 >> 172.21.15.47:0/3552972589 conn(0x7f92240fcfb0 msgr2=0x7f92240ff3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:55:53.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:55:53.518+0000 7f9229a14700 1 -- 172.21.15.47:0/3552972589 shutdown_connections 2024-04-17T05:55:53.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:55:53.518+0000 7f9229a14700 1 -- 172.21.15.47:0/3552972589 wait complete. 2024-04-17T05:55:53.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:55:53.518+0000 7f9229a14700 1 Processor -- start 2024-04-17T05:55:53.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:55:53.518+0000 7f9229a14700 1 -- start start 2024-04-17T05:55:53.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:55:53.519+0000 7f9229a14700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9224103580 0x7f9224198da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:55:53.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:55:53.519+0000 7f9229a14700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9224105eb0 0x7f92241992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:55:53.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:55:53.519+0000 7f9229a14700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f92241087e0 0x7f92241a0310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:55:53.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:55:53.519+0000 7f9229a14700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f922410e7c0 con 0x7f9224103580 2024-04-17T05:55:53.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:55:53.519+0000 7f9229a14700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f922410e640 con 0x7f9224105eb0 2024-04-17T05:55:53.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:55:53.519+0000 7f9229a14700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f922410e940 con 0x7f92241087e0 2024-04-17T05:55:53.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:55:53.519+0000 7f9222ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9224103580 0x7f9224198da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:55:53.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:55:53.519+0000 7f9222ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9224103580 0x7f9224198da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57342/0 (socket says 172.21.15.47:57342) 2024-04-17T05:55:53.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:55:53.519+0000 7f9222ffd700 1 -- 172.21.15.47:0/3424485667 learned_addr learned my addr 172.21.15.47:0/3424485667 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:55:53.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:55:53.519+0000 7f92227fc700 1 --2- 172.21.15.47:0/3424485667 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9224105eb0 0x7f92241992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:55:53.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:55:53.519+0000 7f92237fe700 1 --2- 172.21.15.47:0/3424485667 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f92241087e0 0x7f92241a0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:55:53.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:55:53.519+0000 7f9222ffd700 1 -- 172.21.15.47:0/3424485667 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f92241087e0 msgr2=0x7f92241a0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:55:53.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:55:53.519+0000 7f9222ffd700 1 --2- 172.21.15.47:0/3424485667 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f92241087e0 0x7f92241a0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:55:53.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:55:53.519+0000 7f9222ffd700 1 -- 172.21.15.47:0/3424485667 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9224105eb0 msgr2=0x7f92241992e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:55:53.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:55:53.519+0000 7f9222ffd700 1 --2- 172.21.15.47:0/3424485667 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9224105eb0 0x7f92241992e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:55:53.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:55:53.519+0000 7f9222ffd700 1 -- 172.21.15.47:0/3424485667 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9224101370 con 0x7f9224103580 2024-04-17T05:55:53.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:55:53.519+0000 7f92227fc700 1 --2- 172.21.15.47:0/3424485667 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9224105eb0 0x7f92241992e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T05:55:53.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:55:53.519+0000 7f9222ffd700 1 --2- 172.21.15.47:0/3424485667 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9224103580 0x7f9224198da0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f921400b560 tx=0x7f921400b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:55:53.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:55:53.519+0000 7f9228a12700 1 -- 172.21.15.47:0/3424485667 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9214015020 con 0x7f9224103580 2024-04-17T05:55:53.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:55:53.519+0000 7f9228a12700 1 -- 172.21.15.47:0/3424485667 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9214011d10 con 0x7f9224103580 2024-04-17T05:55:53.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:55:53.519+0000 7f9228a12700 1 -- 172.21.15.47:0/3424485667 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f921400f2e0 con 0x7f9224103580 2024-04-17T05:55:53.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:55:53.519+0000 7f9229a14700 1 -- 172.21.15.47:0/3424485667 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9224101660 con 0x7f9224103580 2024-04-17T05:55:53.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:55:53.519+0000 7f9229a14700 1 -- 172.21.15.47:0/3424485667 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9224101c70 con 0x7f9224103580 2024-04-17T05:55:53.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:55:53.520+0000 7f9229a14700 1 -- 172.21.15.47:0/3424485667 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9224069740 con 0x7f9224103580 2024-04-17T05:55:53.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:55:53.522+0000 7f9228a12700 1 -- 172.21.15.47:0/3424485667 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9214018020 con 0x7f9224103580 2024-04-17T05:55:53.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:55:53.522+0000 7f9228a12700 1 --2- 172.21.15.47:0/3424485667 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f920c0a3cc0 0x7f920c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:55:53.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:55:53.522+0000 7f92227fc700 1 --2- 172.21.15.47:0/3424485667 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f920c0a3cc0 0x7f920c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:55:53.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:55:53.522+0000 7f9228a12700 1 -- 172.21.15.47:0/3424485667 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 4533+0+0 (secure 0 0 0) 0x7f92140cf230 con 0x7f9224103580 2024-04-17T05:55:53.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:55:53.523+0000 7f92227fc700 1 --2- 172.21.15.47:0/3424485667 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f920c0a3cc0 0x7f920c0a6180 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f92180039a0 tx=0x7f921800b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:55:53.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:55:53.526+0000 7f9228a12700 1 -- 172.21.15.47:0/3424485667 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f921401f020 con 0x7f9224103580 2024-04-17T05:55:53.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:55:53.745+0000 7f9229a14700 1 -- 172.21.15.47:0/3424485667 --> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f92240fe810 con 0x7f920c0a3cc0 2024-04-17T05:55:53.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:53 smithi098 ceph-mon[36516]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail 2024-04-17T05:55:53.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:53 smithi098 ceph-mon[36516]: osdmap e42: 6 total, 6 up, 6 in 2024-04-17T05:55:53.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:53 smithi047 ceph-mon[32223]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail 2024-04-17T05:55:53.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:53 smithi047 ceph-mon[32223]: osdmap e42: 6 total, 6 up, 6 in 2024-04-17T05:55:53.935 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:53 smithi103 ceph-mon[36243]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail 2024-04-17T05:55:53.935 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:53 smithi103 ceph-mon[36243]: osdmap e42: 6 total, 6 up, 6 in 2024-04-17T05:55:54.642 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:54 smithi047 ceph-mon[32223]: osdmap e43: 6 total, 6 up, 6 in 2024-04-17T05:55:54.642 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:54 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-17T05:55:54.642 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:54 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-17T05:55:54.642 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:54 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:55:54.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:54 smithi098 ceph-mon[36516]: osdmap e43: 6 total, 6 up, 6 in 2024-04-17T05:55:54.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:54 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-17T05:55:54.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:54 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-17T05:55:54.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:54 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:55:54.948 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:54 smithi103 ceph-mon[36243]: osdmap e43: 6 total, 6 up, 6 in 2024-04-17T05:55:54.948 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:54 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-17T05:55:54.948 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:54 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-17T05:55:54.949 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:54 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:55:55.730 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:55 smithi047 ceph-mon[32223]: pgmap v143: 1 pgs: 1 peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-17T05:55:55.730 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:55 smithi047 ceph-mon[32223]: from='client.14382 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T05:55:55.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:55 smithi098 ceph-mon[36516]: pgmap v143: 1 pgs: 1 peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-17T05:55:55.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:55 smithi098 ceph-mon[36516]: from='client.14382 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T05:55:56.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:55 smithi103 ceph-mon[36243]: pgmap v143: 1 pgs: 1 peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-17T05:55:56.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:55 smithi103 ceph-mon[36243]: from='client.14382 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T05:55:57.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:57 smithi098 ceph-mon[36516]: pgmap v144: 1 pgs: 1 peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 112 KiB/s, 0 objects/s recovering 2024-04-17T05:55:57.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:57 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/1205286345' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "356a6255-2c07-4f0d-a233-26be04c92b38"}]: dispatch 2024-04-17T05:55:57.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:57 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/1205286345' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "356a6255-2c07-4f0d-a233-26be04c92b38"}]': finished 2024-04-17T05:55:57.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:57 smithi098 ceph-mon[36516]: osdmap e44: 7 total, 6 up, 7 in 2024-04-17T05:55:57.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:57 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-17T05:55:57.884 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:57 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/2898833071' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-17T05:55:57.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:57 smithi047 ceph-mon[32223]: pgmap v144: 1 pgs: 1 peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 112 KiB/s, 0 objects/s recovering 2024-04-17T05:55:57.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:57 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/1205286345' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "356a6255-2c07-4f0d-a233-26be04c92b38"}]: dispatch 2024-04-17T05:55:57.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:57 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/1205286345' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "356a6255-2c07-4f0d-a233-26be04c92b38"}]': finished 2024-04-17T05:55:57.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:57 smithi047 ceph-mon[32223]: osdmap e44: 7 total, 6 up, 7 in 2024-04-17T05:55:57.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:57 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-17T05:55:57.890 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:57 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/2898833071' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-17T05:55:58.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:57 smithi103 ceph-mon[36243]: pgmap v144: 1 pgs: 1 peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 112 KiB/s, 0 objects/s recovering 2024-04-17T05:55:58.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:57 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/1205286345' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "356a6255-2c07-4f0d-a233-26be04c92b38"}]: dispatch 2024-04-17T05:55:58.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:57 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/1205286345' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "356a6255-2c07-4f0d-a233-26be04c92b38"}]': finished 2024-04-17T05:55:58.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:57 smithi103 ceph-mon[36243]: osdmap e44: 7 total, 6 up, 7 in 2024-04-17T05:55:58.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:57 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-17T05:55:58.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:57 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/2898833071' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-17T05:55:59.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:55:59 smithi098 ceph-mon[36516]: pgmap v146: 1 pgs: 1 peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-17T05:55:59.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:55:59 smithi047 ceph-mon[32223]: pgmap v146: 1 pgs: 1 peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-17T05:56:00.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:55:59 smithi103 ceph-mon[36243]: pgmap v146: 1 pgs: 1 peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-17T05:56:01.732 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:01 smithi047 ceph-mon[32223]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-04-17T05:56:01.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:01 smithi098 ceph-mon[36516]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-04-17T05:56:02.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:01 smithi103 ceph-mon[36243]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-04-17T05:56:03.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:03 smithi098 ceph-mon[36516]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-17T05:56:03.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:03 smithi047 ceph-mon[32223]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-17T05:56:04.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:03 smithi103 ceph-mon[36243]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-17T05:56:04.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:04 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-17T05:56:04.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:04 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:56:04.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:04 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-17T05:56:04.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:04 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:56:05.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:04 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-17T05:56:05.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:04 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:56:05.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:05 smithi098 ceph-mon[36516]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-17T05:56:05.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:05 smithi098 ceph-mon[36516]: Deploying daemon osd.6 on smithi047 2024-04-17T05:56:05.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:05 smithi047 ceph-mon[32223]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-17T05:56:05.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:05 smithi047 ceph-mon[32223]: Deploying daemon osd.6 on smithi047 2024-04-17T05:56:06.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:05 smithi103 ceph-mon[36243]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-17T05:56:06.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:05 smithi103 ceph-mon[36243]: Deploying daemon osd.6 on smithi047 2024-04-17T05:56:07.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:07 smithi098 ceph-mon[36516]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-17T05:56:07.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:07 smithi047 ceph-mon[32223]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-17T05:56:08.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:07 smithi103 ceph-mon[36243]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-17T05:56:09.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:09 smithi098 ceph-mon[36516]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-17T05:56:09.883 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:09 smithi047 ceph-mon[32223]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-17T05:56:10.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:09 smithi103 ceph-mon[36243]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-17T05:56:11.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:10 smithi103 ceph-mon[36243]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-17T05:56:11.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:10 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:56:11.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:10 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:56:11.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:10 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T05:56:11.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:10 smithi098 ceph-mon[36516]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-17T05:56:11.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:10 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:56:11.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:10 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:56:11.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:10 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T05:56:11.388 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:10 smithi047 ceph-mon[32223]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-17T05:56:11.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:10 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:56:11.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:10 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:56:11.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:10 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T05:56:13.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:12 smithi047 ceph-mon[32223]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-17T05:56:13.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:12 smithi103 ceph-mon[36243]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-17T05:56:13.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:12 smithi098 ceph-mon[36516]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-17T05:56:14.052 INFO:teuthology.orchestra.run.smithi047.stdout:Created osd(s) 6 on host 'smithi047' 2024-04-17T05:56:14.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:56:14.052+0000 7f9228a12700 1 -- 172.21.15.47:0/3424485667 <== mgr.14152 v2:172.21.15.47:6800/3157902806 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f92240fe810 con 0x7f920c0a3cc0 2024-04-17T05:56:14.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:56:14.055+0000 7f9229a14700 1 -- 172.21.15.47:0/3424485667 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f920c0a3cc0 msgr2=0x7f920c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:56:14.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:56:14.055+0000 7f9229a14700 1 --2- 172.21.15.47:0/3424485667 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f920c0a3cc0 0x7f920c0a6180 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f92180039a0 tx=0x7f921800b540 comp rx=0 tx=0).stop 2024-04-17T05:56:14.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:56:14.055+0000 7f9229a14700 1 -- 172.21.15.47:0/3424485667 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9224103580 msgr2=0x7f9224198da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:56:14.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:56:14.055+0000 7f9229a14700 1 --2- 172.21.15.47:0/3424485667 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9224103580 0x7f9224198da0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f921400b560 tx=0x7f921400b920 comp rx=0 tx=0).stop 2024-04-17T05:56:14.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:56:14.055+0000 7f9229a14700 1 -- 172.21.15.47:0/3424485667 shutdown_connections 2024-04-17T05:56:14.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:56:14.055+0000 7f9229a14700 1 --2- 172.21.15.47:0/3424485667 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f920c0a3cc0 0x7f920c0a6180 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:56:14.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:56:14.055+0000 7f9229a14700 1 --2- 172.21.15.47:0/3424485667 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f92241087e0 0x7f92241a0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:56:14.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:56:14.055+0000 7f9229a14700 1 --2- 172.21.15.47:0/3424485667 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9224103580 0x7f9224198da0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:56:14.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:56:14.055+0000 7f9229a14700 1 --2- 172.21.15.47:0/3424485667 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9224105eb0 0x7f92241992e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:56:14.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:56:14.055+0000 7f9229a14700 1 -- 172.21.15.47:0/3424485667 >> 172.21.15.47:0/3424485667 conn(0x7f92240fcfb0 msgr2=0x7f9224103ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:56:14.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:56:14.055+0000 7f9229a14700 1 -- 172.21.15.47:0/3424485667 shutdown_connections 2024-04-17T05:56:14.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:56:14.055+0000 7f9229a14700 1 -- 172.21.15.47:0/3424485667 wait complete. 2024-04-17T05:56:14.374 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:14 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:56:14.374 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:14 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:56:14.374 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:14 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:56:14.374 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:14 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T05:56:14.374 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:14 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:56:14.374 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:14 smithi047 ceph-mon[32223]: from='osd.6 [v2:172.21.15.47:6818/2517529668,v1:172.21.15.47:6819/2517529668]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-17T05:56:14.374 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:14 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:56:14.374 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:14 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:56:14.374 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:14 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T05:56:14.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:14 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:56:14.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:14 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:56:14.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:14 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:56:14.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:14 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T05:56:14.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:14 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:56:14.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:14 smithi098 ceph-mon[36516]: from='osd.6 [v2:172.21.15.47:6818/2517529668,v1:172.21.15.47:6819/2517529668]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-17T05:56:14.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:14 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:56:14.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:14 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:56:14.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:14 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T05:56:14.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:14 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:56:14.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:14 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:56:14.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:14 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:56:14.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:14 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T05:56:14.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:14 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:56:14.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:14 smithi103 ceph-mon[36243]: from='osd.6 [v2:172.21.15.47:6818/2517529668,v1:172.21.15.47:6819/2517529668]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-17T05:56:14.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:14 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:56:14.535 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:14 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:56:14.535 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:14 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T05:56:14.834 DEBUG:teuthology.orchestra.run.smithi047:osd.6> sudo journalctl -f -n 0 -u ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@osd.6.service 2024-04-17T05:56:14.837 INFO:tasks.cephadm:Deploying osd.7 on smithi098 with /dev/vg_nvme/lv_2... 2024-04-17T05:56:14.837 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 565db2d8-fc7e-11ee-bc91-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-17T05:56:15.083 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/mon.b/config 2024-04-17T05:56:15.345 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:15 smithi047 ceph-mon[32223]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-17T05:56:15.345 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:15 smithi047 ceph-mon[32223]: from='osd.6 [v2:172.21.15.47:6818/2517529668,v1:172.21.15.47:6819/2517529668]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-17T05:56:15.345 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:15 smithi047 ceph-mon[32223]: osdmap e45: 7 total, 6 up, 7 in 2024-04-17T05:56:15.345 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:15 smithi047 ceph-mon[32223]: from='osd.6 [v2:172.21.15.47:6818/2517529668,v1:172.21.15.47:6819/2517529668]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-04-17T05:56:15.346 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:15 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-17T05:56:15.346 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 17 05:56:15 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-6[60771]: 2024-04-17T05:56:15.129+0000 7f3aac3ed700 -1 osd.6 0 waiting for initial osdmap 2024-04-17T05:56:15.346 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 17 05:56:15 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-6[60771]: 2024-04-17T05:56:15.138+0000 7f3aa81e0700 -1 osd.6 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-17T05:56:15.382 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:15 smithi098 ceph-mon[36516]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-17T05:56:15.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:15 smithi098 ceph-mon[36516]: from='osd.6 [v2:172.21.15.47:6818/2517529668,v1:172.21.15.47:6819/2517529668]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-17T05:56:15.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:15 smithi098 ceph-mon[36516]: osdmap e45: 7 total, 6 up, 7 in 2024-04-17T05:56:15.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:15 smithi098 ceph-mon[36516]: from='osd.6 [v2:172.21.15.47:6818/2517529668,v1:172.21.15.47:6819/2517529668]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-04-17T05:56:15.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:15 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-17T05:56:15.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:15 smithi103 ceph-mon[36243]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-17T05:56:15.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:15 smithi103 ceph-mon[36243]: from='osd.6 [v2:172.21.15.47:6818/2517529668,v1:172.21.15.47:6819/2517529668]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-17T05:56:15.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:15 smithi103 ceph-mon[36243]: osdmap e45: 7 total, 6 up, 7 in 2024-04-17T05:56:15.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:15 smithi103 ceph-mon[36243]: from='osd.6 [v2:172.21.15.47:6818/2517529668,v1:172.21.15.47:6819/2517529668]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-04-17T05:56:15.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:15 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-17T05:56:16.382 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:16 smithi098 ceph-mon[36516]: from='osd.6 [v2:172.21.15.47:6818/2517529668,v1:172.21.15.47:6819/2517529668]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]': finished 2024-04-17T05:56:16.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:16 smithi098 ceph-mon[36516]: osdmap e46: 7 total, 6 up, 7 in 2024-04-17T05:56:16.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:16 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-17T05:56:16.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:16 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-17T05:56:16.384 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:16 smithi047 ceph-mon[32223]: from='osd.6 [v2:172.21.15.47:6818/2517529668,v1:172.21.15.47:6819/2517529668]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]': finished 2024-04-17T05:56:16.384 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:16 smithi047 ceph-mon[32223]: osdmap e46: 7 total, 6 up, 7 in 2024-04-17T05:56:16.384 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:16 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-17T05:56:16.385 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:16 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-17T05:56:16.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:16 smithi103 ceph-mon[36243]: from='osd.6 [v2:172.21.15.47:6818/2517529668,v1:172.21.15.47:6819/2517529668]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]': finished 2024-04-17T05:56:16.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:16 smithi103 ceph-mon[36243]: osdmap e46: 7 total, 6 up, 7 in 2024-04-17T05:56:16.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:16 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-17T05:56:16.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:16 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-17T05:56:17.192 INFO:teuthology.orchestra.run.smithi098.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-17T05:56:17.192 INFO:teuthology.orchestra.run.smithi098.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-17T05:56:17.192 INFO:teuthology.orchestra.run.smithi098.stderr: stderr: 10+0 records in 2024-04-17T05:56:17.192 INFO:teuthology.orchestra.run.smithi098.stderr:10+0 records out 2024-04-17T05:56:17.192 INFO:teuthology.orchestra.run.smithi098.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0113918 s, 920 MB/s 2024-04-17T05:56:17.192 INFO:teuthology.orchestra.run.smithi098.stderr:--> Zapping successful for: 2024-04-17T05:56:17.382 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:17 smithi098 ceph-mon[36516]: purged_snaps scrub starts 2024-04-17T05:56:17.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:17 smithi098 ceph-mon[36516]: purged_snaps scrub ok 2024-04-17T05:56:17.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:17 smithi098 ceph-mon[36516]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-17T05:56:17.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:17 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-17T05:56:17.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:17 smithi098 ceph-mon[36516]: osd.6 [v2:172.21.15.47:6818/2517529668,v1:172.21.15.47:6819/2517529668] boot 2024-04-17T05:56:17.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:17 smithi098 ceph-mon[36516]: osdmap e47: 7 total, 7 up, 7 in 2024-04-17T05:56:17.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:17 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-17T05:56:17.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:17 smithi047 ceph-mon[32223]: purged_snaps scrub starts 2024-04-17T05:56:17.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:17 smithi047 ceph-mon[32223]: purged_snaps scrub ok 2024-04-17T05:56:17.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:17 smithi047 ceph-mon[32223]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-17T05:56:17.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:17 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-17T05:56:17.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:17 smithi047 ceph-mon[32223]: osd.6 [v2:172.21.15.47:6818/2517529668,v1:172.21.15.47:6819/2517529668] boot 2024-04-17T05:56:17.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:17 smithi047 ceph-mon[32223]: osdmap e47: 7 total, 7 up, 7 in 2024-04-17T05:56:17.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:17 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-17T05:56:17.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:17 smithi103 ceph-mon[36243]: purged_snaps scrub starts 2024-04-17T05:56:17.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:17 smithi103 ceph-mon[36243]: purged_snaps scrub ok 2024-04-17T05:56:17.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:17 smithi103 ceph-mon[36243]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-17T05:56:17.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:17 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-17T05:56:17.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:17 smithi103 ceph-mon[36243]: osd.6 [v2:172.21.15.47:6818/2517529668,v1:172.21.15.47:6819/2517529668] boot 2024-04-17T05:56:17.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:17 smithi103 ceph-mon[36243]: osdmap e47: 7 total, 7 up, 7 in 2024-04-17T05:56:17.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:17 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-17T05:56:17.772 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 565db2d8-fc7e-11ee-bc91-c7b262605968 -- ceph orch daemon add osd smithi098:vg_nvme/lv_2 2024-04-17T05:56:18.052 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/mon.b/config 2024-04-17T05:56:18.388 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:18 smithi047 ceph-mon[32223]: osdmap e48: 7 total, 7 up, 7 in 2024-04-17T05:56:18.414 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:18 smithi098 ceph-mon[36516]: osdmap e48: 7 total, 7 up, 7 in 2024-04-17T05:56:18.533 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:18 smithi103 ceph-mon[36243]: osdmap e48: 7 total, 7 up, 7 in 2024-04-17T05:56:19.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:19 smithi098 ceph-mon[36516]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-17T05:56:19.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:19 smithi098 ceph-mon[36516]: Detected new or changed devices on smithi047 2024-04-17T05:56:19.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:19 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:56:19.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:19 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:56:19.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:19 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-17T05:56:19.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:19 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-17T05:56:19.384 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:19 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-17T05:56:19.384 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:19 smithi098 ceph-mon[36516]: Adjusting osd_memory_target on smithi047 to 5697M 2024-04-17T05:56:19.384 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:19 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:56:19.384 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:19 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:56:19.384 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:19 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T05:56:19.384 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:19 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:56:19.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:19 smithi103 ceph-mon[36243]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-17T05:56:19.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:19 smithi103 ceph-mon[36243]: Detected new or changed devices on smithi047 2024-04-17T05:56:19.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:19 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:56:19.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:19 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:56:19.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:19 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-17T05:56:19.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:19 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-17T05:56:19.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:19 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-17T05:56:19.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:19 smithi103 ceph-mon[36243]: Adjusting osd_memory_target on smithi047 to 5697M 2024-04-17T05:56:19.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:19 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:56:19.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:19 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:56:19.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:19 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T05:56:19.535 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:19 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:56:19.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:56:19.602+0000 7f18132c9700 1 -- 172.21.15.98:0/222930514 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f180c108840 msgr2=0x7f180c10ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:56:19.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:56:19.602+0000 7f18132c9700 1 --2- 172.21.15.98:0/222930514 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f180c108840 0x7f180c10ac30 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f180400b0a0 tx=0x7f180400b3b0 comp rx=0 tx=0).stop 2024-04-17T05:56:19.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:56:19.602+0000 7f18132c9700 1 -- 172.21.15.98:0/222930514 shutdown_connections 2024-04-17T05:56:19.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:56:19.602+0000 7f18132c9700 1 --2- 172.21.15.98:0/222930514 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f180c108840 0x7f180c10ac30 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:56:19.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:56:19.602+0000 7f18132c9700 1 --2- 172.21.15.98:0/222930514 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f180c1035e0 0x7f180c1059d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:56:19.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:56:19.602+0000 7f18132c9700 1 --2- 172.21.15.98:0/222930514 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f180c105f10 0x7f180c108300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:56:19.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:56:19.602+0000 7f18132c9700 1 -- 172.21.15.98:0/222930514 >> 172.21.15.98:0/222930514 conn(0x7f180c0fcf50 msgr2=0x7f180c0ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:56:19.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:56:19.602+0000 7f18132c9700 1 -- 172.21.15.98:0/222930514 shutdown_connections 2024-04-17T05:56:19.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:56:19.602+0000 7f18132c9700 1 -- 172.21.15.98:0/222930514 wait complete. 2024-04-17T05:56:19.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:56:19.602+0000 7f18132c9700 1 Processor -- start 2024-04-17T05:56:19.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:56:19.602+0000 7f18132c9700 1 -- start start 2024-04-17T05:56:19.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:56:19.602+0000 7f18132c9700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f180c1035e0 0x7f180c198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:56:19.604 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:56:19.602+0000 7f18132c9700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f180c105f10 0x7f180c199190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:56:19.604 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:56:19.602+0000 7f18132c9700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f180c108840 0x7f180c1a01d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:56:19.604 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:56:19.602+0000 7f18132c9700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f180c10e6e0 con 0x7f180c108840 2024-04-17T05:56:19.604 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:56:19.603+0000 7f18132c9700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f180c10e560 con 0x7f180c105f10 2024-04-17T05:56:19.604 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:56:19.603+0000 7f18132c9700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f180c10e860 con 0x7f180c1035e0 2024-04-17T05:56:19.604 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:56:19.603+0000 7f1810864700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f180c105f10 0x7f180c199190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:56:19.604 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:56:19.603+0000 7f1810864700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f180c105f10 0x7f180c199190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:40058/0 (socket says 172.21.15.98:40058) 2024-04-17T05:56:19.604 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:56:19.603+0000 7f1810864700 1 -- 172.21.15.98:0/1331254312 learned_addr learned my addr 172.21.15.98:0/1331254312 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-04-17T05:56:19.604 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:56:19.603+0000 7f1811065700 1 --2- 172.21.15.98:0/1331254312 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f180c1035e0 0x7f180c198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:56:19.604 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:56:19.603+0000 7f1811866700 1 --2- 172.21.15.98:0/1331254312 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f180c108840 0x7f180c1a01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:56:19.604 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:56:19.603+0000 7f1810864700 1 -- 172.21.15.98:0/1331254312 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f180c1035e0 msgr2=0x7f180c198c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:56:19.604 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:56:19.603+0000 7f1810864700 1 --2- 172.21.15.98:0/1331254312 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f180c1035e0 0x7f180c198c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:56:19.605 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:56:19.603+0000 7f1810864700 1 -- 172.21.15.98:0/1331254312 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f180c108840 msgr2=0x7f180c1a01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:56:19.605 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:56:19.603+0000 7f1810864700 1 --2- 172.21.15.98:0/1331254312 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f180c108840 0x7f180c1a01d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:56:19.605 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:56:19.603+0000 7f1810864700 1 -- 172.21.15.98:0/1331254312 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f180c1a08a0 con 0x7f180c105f10 2024-04-17T05:56:19.605 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:56:19.603+0000 7f1810864700 1 --2- 172.21.15.98:0/1331254312 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f180c105f10 0x7f180c199190 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f17fc00c9b0 tx=0x7f17fc00ccc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:56:19.605 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:56:19.603+0000 7f18027fc700 1 -- 172.21.15.98:0/1331254312 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17fc007b00 con 0x7f180c105f10 2024-04-17T05:56:19.605 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:56:19.603+0000 7f18027fc700 1 -- 172.21.15.98:0/1331254312 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17fc007ca0 con 0x7f180c105f10 2024-04-17T05:56:19.605 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:56:19.603+0000 7f18132c9700 1 -- 172.21.15.98:0/1331254312 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f180c1a0b90 con 0x7f180c105f10 2024-04-17T05:56:19.605 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:56:19.603+0000 7f18027fc700 1 -- 172.21.15.98:0/1331254312 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17fc018730 con 0x7f180c105f10 2024-04-17T05:56:19.605 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:56:19.603+0000 7f18132c9700 1 -- 172.21.15.98:0/1331254312 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f180c1a1120 con 0x7f180c105f10 2024-04-17T05:56:19.606 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:56:19.604+0000 7f18132c9700 1 -- 172.21.15.98:0/1331254312 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f180c069740 con 0x7f180c105f10 2024-04-17T05:56:19.606 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:56:19.605+0000 7f18027fc700 1 -- 172.21.15.98:0/1331254312 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f17fc0188d0 con 0x7f180c105f10 2024-04-17T05:56:19.606 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:56:19.606+0000 7f18027fc700 1 --2- 172.21.15.98:0/1331254312 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f17f80a3cc0 0x7f17f80a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:56:19.606 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:56:19.606+0000 7f18027fc700 1 -- 172.21.15.98:0/1331254312 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 4965+0+0 (secure 0 0 0) 0x7f17fc0cf670 con 0x7f180c105f10 2024-04-17T05:56:19.606 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:56:19.606+0000 7f1811065700 1 --2- 172.21.15.98:0/1331254312 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f17f80a3cc0 0x7f17f80a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:56:19.606 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:56:19.607+0000 7f1811065700 1 --2- 172.21.15.98:0/1331254312 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f17f80a3cc0 0x7f17f80a6180 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f180c19a130 tx=0x7f1808009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:56:19.609 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:56:19.610+0000 7f18027fc700 1 -- 172.21.15.98:0/1331254312 <== mon.2 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f17fc014030 con 0x7f180c105f10 2024-04-17T05:56:19.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:19 smithi047 ceph-mon[32223]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-17T05:56:19.640 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:19 smithi047 ceph-mon[32223]: Detected new or changed devices on smithi047 2024-04-17T05:56:19.640 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:19 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:56:19.640 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:19 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:56:19.640 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:19 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-17T05:56:19.640 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:19 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-17T05:56:19.640 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:19 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-17T05:56:19.640 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:19 smithi047 ceph-mon[32223]: Adjusting osd_memory_target on smithi047 to 5697M 2024-04-17T05:56:19.640 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:19 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:56:19.640 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:19 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:56:19.640 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:19 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T05:56:19.640 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:19 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:56:19.814 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:56:19.815+0000 7f18132c9700 1 -- 172.21.15.98:0/1331254312 --> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f180c062e90 con 0x7f17f80a3cc0 2024-04-17T05:56:20.533 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:20 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-17T05:56:20.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:20 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-17T05:56:20.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:20 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:56:20.581 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:20 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-17T05:56:20.581 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:20 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-17T05:56:20.581 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:20 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:56:20.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:20 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-17T05:56:20.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:20 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-17T05:56:20.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:20 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:56:21.455 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:21 smithi098 ceph-mon[36516]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-17T05:56:21.455 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:21 smithi098 ceph-mon[36516]: from='client.24281 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T05:56:21.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:21 smithi103 ceph-mon[36243]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-17T05:56:21.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:21 smithi103 ceph-mon[36243]: from='client.24281 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T05:56:21.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:21 smithi047 ceph-mon[32223]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-17T05:56:21.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:21 smithi047 ceph-mon[32223]: from='client.24281 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T05:56:23.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:23 smithi103 ceph-mon[36243]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-17T05:56:23.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:23 smithi103 ceph-mon[36243]: from='client.? 172.21.15.98:0/1974890524' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "349d0120-8290-417d-a800-5a8985f554f0"}]: dispatch 2024-04-17T05:56:23.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:23 smithi103 ceph-mon[36243]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "349d0120-8290-417d-a800-5a8985f554f0"}]: dispatch 2024-04-17T05:56:23.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:23 smithi103 ceph-mon[36243]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "349d0120-8290-417d-a800-5a8985f554f0"}]': finished 2024-04-17T05:56:23.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:23 smithi103 ceph-mon[36243]: osdmap e49: 8 total, 7 up, 8 in 2024-04-17T05:56:23.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:23 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-17T05:56:23.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:23 smithi098 ceph-mon[36516]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-17T05:56:23.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:23 smithi098 ceph-mon[36516]: from='client.? 172.21.15.98:0/1974890524' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "349d0120-8290-417d-a800-5a8985f554f0"}]: dispatch 2024-04-17T05:56:23.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:23 smithi098 ceph-mon[36516]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "349d0120-8290-417d-a800-5a8985f554f0"}]: dispatch 2024-04-17T05:56:23.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:23 smithi098 ceph-mon[36516]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "349d0120-8290-417d-a800-5a8985f554f0"}]': finished 2024-04-17T05:56:23.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:23 smithi098 ceph-mon[36516]: osdmap e49: 8 total, 7 up, 8 in 2024-04-17T05:56:23.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:23 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-17T05:56:23.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:23 smithi047 ceph-mon[32223]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-17T05:56:23.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:23 smithi047 ceph-mon[32223]: from='client.? 172.21.15.98:0/1974890524' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "349d0120-8290-417d-a800-5a8985f554f0"}]: dispatch 2024-04-17T05:56:23.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:23 smithi047 ceph-mon[32223]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "349d0120-8290-417d-a800-5a8985f554f0"}]: dispatch 2024-04-17T05:56:23.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:23 smithi047 ceph-mon[32223]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "349d0120-8290-417d-a800-5a8985f554f0"}]': finished 2024-04-17T05:56:23.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:23 smithi047 ceph-mon[32223]: osdmap e49: 8 total, 7 up, 8 in 2024-04-17T05:56:23.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:23 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-17T05:56:24.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:24 smithi103 ceph-mon[36243]: from='client.? 172.21.15.98:0/3150252186' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-17T05:56:24.632 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:24 smithi098 ceph-mon[36516]: from='client.? 172.21.15.98:0/3150252186' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-17T05:56:24.638 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:24 smithi047 ceph-mon[32223]: from='client.? 172.21.15.98:0/3150252186' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-17T05:56:25.533 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:25 smithi103 ceph-mon[36243]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-17T05:56:25.534 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:25 smithi098 ceph-mon[36516]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-17T05:56:25.638 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:25 smithi047 ceph-mon[32223]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-17T05:56:27.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:27 smithi047 ceph-mon[32223]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-17T05:56:27.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:27 smithi103 ceph-mon[36243]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-17T05:56:27.620 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:27 smithi098 ceph-mon[36516]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-17T05:56:29.533 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:29 smithi103 ceph-mon[36243]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-17T05:56:29.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:29 smithi098 ceph-mon[36516]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-17T05:56:29.638 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:29 smithi047 ceph-mon[32223]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-17T05:56:30.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:30 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-17T05:56:30.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:30 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:56:30.559 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:30 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-17T05:56:30.559 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:30 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:56:30.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:30 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-17T05:56:30.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:30 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:56:31.513 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:31 smithi098 ceph-mon[36516]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-17T05:56:31.513 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:31 smithi098 ceph-mon[36516]: Deploying daemon osd.7 on smithi098 2024-04-17T05:56:31.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:31 smithi047 ceph-mon[32223]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-17T05:56:31.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:31 smithi047 ceph-mon[32223]: Deploying daemon osd.7 on smithi098 2024-04-17T05:56:31.783 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:31 smithi103 ceph-mon[36243]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-17T05:56:31.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:31 smithi103 ceph-mon[36243]: Deploying daemon osd.7 on smithi098 2024-04-17T05:56:33.545 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:33 smithi098 ceph-mon[36516]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-17T05:56:33.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:33 smithi047 ceph-mon[32223]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-17T05:56:33.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:33 smithi103 ceph-mon[36243]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-17T05:56:35.612 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:35 smithi103 ceph-mon[36243]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-17T05:56:35.632 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:35 smithi098 ceph-mon[36516]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-17T05:56:35.638 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:35 smithi047 ceph-mon[32223]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-17T05:56:36.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:36 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:56:36.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:36 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:56:36.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:36 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T05:56:36.783 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:36 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:56:36.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:36 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:56:36.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:36 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T05:56:36.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:36 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:56:36.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:36 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:56:36.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:36 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T05:56:37.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:37 smithi098 ceph-mon[36516]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-17T05:56:37.785 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:37 smithi103 ceph-mon[36243]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-17T05:56:37.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:37 smithi047 ceph-mon[32223]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-17T05:56:38.831 INFO:teuthology.orchestra.run.smithi098.stdout:Created osd(s) 7 on host 'smithi098' 2024-04-17T05:56:38.831 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:56:38.832+0000 7f18027fc700 1 -- 172.21.15.98:0/1331254312 <== mgr.14152 v2:172.21.15.47:6800/3157902806 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f180c062e90 con 0x7f17f80a3cc0 2024-04-17T05:56:38.833 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:56:38.835+0000 7f18132c9700 1 -- 172.21.15.98:0/1331254312 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f17f80a3cc0 msgr2=0x7f17f80a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:56:38.833 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:56:38.835+0000 7f18132c9700 1 --2- 172.21.15.98:0/1331254312 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f17f80a3cc0 0x7f17f80a6180 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f180c19a130 tx=0x7f1808009190 comp rx=0 tx=0).stop 2024-04-17T05:56:38.833 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:56:38.835+0000 7f18132c9700 1 -- 172.21.15.98:0/1331254312 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f180c105f10 msgr2=0x7f180c199190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:56:38.833 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:56:38.835+0000 7f18132c9700 1 --2- 172.21.15.98:0/1331254312 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f180c105f10 0x7f180c199190 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f17fc00c9b0 tx=0x7f17fc00ccc0 comp rx=0 tx=0).stop 2024-04-17T05:56:38.833 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:56:38.835+0000 7f18132c9700 1 -- 172.21.15.98:0/1331254312 shutdown_connections 2024-04-17T05:56:38.833 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:56:38.835+0000 7f18132c9700 1 --2- 172.21.15.98:0/1331254312 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f17f80a3cc0 0x7f17f80a6180 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:56:38.833 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:56:38.835+0000 7f18132c9700 1 --2- 172.21.15.98:0/1331254312 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f180c108840 0x7f180c1a01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:56:38.834 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:56:38.835+0000 7f18132c9700 1 --2- 172.21.15.98:0/1331254312 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f180c1035e0 0x7f180c198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:56:38.834 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:56:38.835+0000 7f18132c9700 1 --2- 172.21.15.98:0/1331254312 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f180c105f10 0x7f180c199190 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:56:38.834 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:56:38.835+0000 7f18132c9700 1 -- 172.21.15.98:0/1331254312 >> 172.21.15.98:0/1331254312 conn(0x7f180c0fcf50 msgr2=0x7f180c103f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:56:38.834 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:56:38.835+0000 7f18132c9700 1 -- 172.21.15.98:0/1331254312 shutdown_connections 2024-04-17T05:56:38.834 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:56:38.835+0000 7f18132c9700 1 -- 172.21.15.98:0/1331254312 wait complete. 2024-04-17T05:56:39.429 DEBUG:teuthology.orchestra.run.smithi098:osd.7> sudo journalctl -f -n 0 -u ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@osd.7.service 2024-04-17T05:56:39.432 INFO:tasks.cephadm:Deploying osd.8 on smithi103 with /dev/vg_nvme/lv_2... 2024-04-17T05:56:39.432 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 565db2d8-fc7e-11ee-bc91-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-17T05:56:39.682 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/mon.c/config 2024-04-17T05:56:39.783 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:39 smithi103 ceph-mon[36243]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-17T05:56:39.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:39 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:56:39.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:39 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:56:39.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:39 smithi103 ceph-mon[36243]: from='osd.7 [v2:172.21.15.98:6816/4225498496,v1:172.21.15.98:6817/4225498496]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-17T05:56:39.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:39 smithi103 ceph-mon[36243]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-17T05:56:39.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:39 smithi098 ceph-mon[36516]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-17T05:56:39.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:39 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:56:39.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:39 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:56:39.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:39 smithi098 ceph-mon[36516]: from='osd.7 [v2:172.21.15.98:6816/4225498496,v1:172.21.15.98:6817/4225498496]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-17T05:56:39.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:39 smithi098 ceph-mon[36516]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-17T05:56:39.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:39 smithi047 ceph-mon[32223]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-17T05:56:39.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:39 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:56:39.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:39 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:56:39.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:39 smithi047 ceph-mon[32223]: from='osd.7 [v2:172.21.15.98:6816/4225498496,v1:172.21.15.98:6817/4225498496]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-17T05:56:39.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:39 smithi047 ceph-mon[32223]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-17T05:56:40.835 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:40 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:56:40.835 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:40 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:56:40.835 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:40 smithi098 ceph-mon[36516]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-17T05:56:40.835 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:40 smithi098 ceph-mon[36516]: from='osd.7 [v2:172.21.15.98:6816/4225498496,v1:172.21.15.98:6817/4225498496]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2024-04-17T05:56:40.835 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:40 smithi098 ceph-mon[36516]: osdmap e50: 8 total, 7 up, 8 in 2024-04-17T05:56:40.835 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:40 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-17T05:56:40.835 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:40 smithi098 ceph-mon[36516]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2024-04-17T05:56:40.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:40 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:56:40.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:40 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:56:40.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:40 smithi047 ceph-mon[32223]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-17T05:56:40.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:40 smithi047 ceph-mon[32223]: from='osd.7 [v2:172.21.15.98:6816/4225498496,v1:172.21.15.98:6817/4225498496]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2024-04-17T05:56:40.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:40 smithi047 ceph-mon[32223]: osdmap e50: 8 total, 7 up, 8 in 2024-04-17T05:56:40.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:40 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-17T05:56:40.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:40 smithi047 ceph-mon[32223]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2024-04-17T05:56:40.970 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:40 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:56:40.970 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:40 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:56:40.970 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:40 smithi103 ceph-mon[36243]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-17T05:56:40.970 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:40 smithi103 ceph-mon[36243]: from='osd.7 [v2:172.21.15.98:6816/4225498496,v1:172.21.15.98:6817/4225498496]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2024-04-17T05:56:40.970 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:40 smithi103 ceph-mon[36243]: osdmap e50: 8 total, 7 up, 8 in 2024-04-17T05:56:40.970 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:40 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-17T05:56:40.970 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:40 smithi103 ceph-mon[36243]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2024-04-17T05:56:41.133 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 05:56:40 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-7[54731]: 2024-04-17T05:56:40.836+0000 7f1a0b908700 -1 osd.7 0 waiting for initial osdmap 2024-04-17T05:56:41.133 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 05:56:40 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-7[54731]: 2024-04-17T05:56:40.844+0000 7f1a076fb700 -1 osd.7 51 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-17T05:56:41.628 INFO:teuthology.orchestra.run.smithi103.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-17T05:56:41.628 INFO:teuthology.orchestra.run.smithi103.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-17T05:56:41.628 INFO:teuthology.orchestra.run.smithi103.stderr: stderr: 10+0 records in 2024-04-17T05:56:41.628 INFO:teuthology.orchestra.run.smithi103.stderr:10+0 records out 2024-04-17T05:56:41.628 INFO:teuthology.orchestra.run.smithi103.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0131345 s, 798 MB/s 2024-04-17T05:56:41.629 INFO:teuthology.orchestra.run.smithi103.stderr:--> Zapping successful for: 2024-04-17T05:56:41.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:41 smithi098 ceph-mon[36516]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-17T05:56:41.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:41 smithi098 ceph-mon[36516]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]': finished 2024-04-17T05:56:41.884 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:41 smithi098 ceph-mon[36516]: osdmap e51: 8 total, 7 up, 8 in 2024-04-17T05:56:41.884 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:41 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-17T05:56:41.884 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:41 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-17T05:56:41.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:41 smithi047 ceph-mon[32223]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-17T05:56:41.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:41 smithi047 ceph-mon[32223]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]': finished 2024-04-17T05:56:41.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:41 smithi047 ceph-mon[32223]: osdmap e51: 8 total, 7 up, 8 in 2024-04-17T05:56:41.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:41 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-17T05:56:41.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:41 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-17T05:56:41.902 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:41 smithi103 ceph-mon[36243]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-17T05:56:41.902 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:41 smithi103 ceph-mon[36243]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]': finished 2024-04-17T05:56:41.902 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:41 smithi103 ceph-mon[36243]: osdmap e51: 8 total, 7 up, 8 in 2024-04-17T05:56:41.902 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:41 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-17T05:56:41.903 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:41 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-17T05:56:42.179 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 565db2d8-fc7e-11ee-bc91-c7b262605968 -- ceph orch daemon add osd smithi103:vg_nvme/lv_2 2024-04-17T05:56:42.418 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/mon.c/config 2024-04-17T05:56:42.901 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:42 smithi098 ceph-mon[36516]: purged_snaps scrub starts 2024-04-17T05:56:42.901 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:42 smithi098 ceph-mon[36516]: purged_snaps scrub ok 2024-04-17T05:56:42.901 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:42 smithi098 ceph-mon[36516]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-17T05:56:42.901 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:42 smithi098 ceph-mon[36516]: osd.7 [v2:172.21.15.98:6816/4225498496,v1:172.21.15.98:6817/4225498496] boot 2024-04-17T05:56:42.901 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:42 smithi098 ceph-mon[36516]: osdmap e52: 8 total, 8 up, 8 in 2024-04-17T05:56:42.901 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:42 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-17T05:56:43.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:42 smithi103 ceph-mon[36243]: purged_snaps scrub starts 2024-04-17T05:56:43.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:42 smithi103 ceph-mon[36243]: purged_snaps scrub ok 2024-04-17T05:56:43.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:42 smithi103 ceph-mon[36243]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-17T05:56:43.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:42 smithi103 ceph-mon[36243]: osd.7 [v2:172.21.15.98:6816/4225498496,v1:172.21.15.98:6817/4225498496] boot 2024-04-17T05:56:43.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:42 smithi103 ceph-mon[36243]: osdmap e52: 8 total, 8 up, 8 in 2024-04-17T05:56:43.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:42 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-17T05:56:43.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:42 smithi047 ceph-mon[32223]: purged_snaps scrub starts 2024-04-17T05:56:43.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:42 smithi047 ceph-mon[32223]: purged_snaps scrub ok 2024-04-17T05:56:43.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:42 smithi047 ceph-mon[32223]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-17T05:56:43.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:42 smithi047 ceph-mon[32223]: osd.7 [v2:172.21.15.98:6816/4225498496,v1:172.21.15.98:6817/4225498496] boot 2024-04-17T05:56:43.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:42 smithi047 ceph-mon[32223]: osdmap e52: 8 total, 8 up, 8 in 2024-04-17T05:56:43.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:42 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-17T05:56:44.107 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:43 smithi103 ceph-mon[36243]: osdmap e53: 8 total, 8 up, 8 in 2024-04-17T05:56:44.107 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:43 smithi103 ceph-mon[36243]: Detected new or changed devices on smithi098 2024-04-17T05:56:44.107 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:43 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:56:44.107 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:43 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:56:44.107 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:43 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-17T05:56:44.107 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:43 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-17T05:56:44.107 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:43 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-17T05:56:44.107 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:43 smithi103 ceph-mon[36243]: Adjusting osd_memory_target on smithi098 to 5693M 2024-04-17T05:56:44.107 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:43 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:56:44.107 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:43 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:56:44.108 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:43 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T05:56:44.108 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:43 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:56:44.108 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:43 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T05:56:44.108 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:43 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:56:44.108 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:43 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T05:56:44.108 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:43 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:56:44.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:56:44.118+0000 7f8b15048700 1 -- 172.21.15.103:0/12778553 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8b10102ba0 msgr2=0x7f8b10103000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:56:44.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:56:44.118+0000 7f8b15048700 1 --2- 172.21.15.103:0/12778553 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8b10102ba0 0x7f8b10103000 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f8af8009990 tx=0x7f8af8009ca0 comp rx=0 tx=0).stop 2024-04-17T05:56:44.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:56:44.118+0000 7f8b15048700 1 -- 172.21.15.103:0/12778553 shutdown_connections 2024-04-17T05:56:44.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:56:44.118+0000 7f8b15048700 1 --2- 172.21.15.103:0/12778553 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b10103540 0x7f8b10109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:56:44.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:56:44.118+0000 7f8b15048700 1 --2- 172.21.15.103:0/12778553 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8b101019a0 0x7f8b10101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:56:44.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:56:44.118+0000 7f8b15048700 1 --2- 172.21.15.103:0/12778553 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8b10102ba0 0x7f8b10103000 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:56:44.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:56:44.118+0000 7f8b15048700 1 -- 172.21.15.103:0/12778553 >> 172.21.15.103:0/12778553 conn(0x7f8b100fcf40 msgr2=0x7f8b100ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:56:44.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:56:44.118+0000 7f8b15048700 1 -- 172.21.15.103:0/12778553 shutdown_connections 2024-04-17T05:56:44.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:56:44.118+0000 7f8b15048700 1 -- 172.21.15.103:0/12778553 wait complete. 2024-04-17T05:56:44.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:56:44.119+0000 7f8b15048700 1 Processor -- start 2024-04-17T05:56:44.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:56:44.119+0000 7f8b15048700 1 -- start start 2024-04-17T05:56:44.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:56:44.119+0000 7f8b15048700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8b101019a0 0x7f8b10198ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:56:44.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:56:44.119+0000 7f8b15048700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8b10102ba0 0x7f8b10199220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:56:44.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:56:44.119+0000 7f8b15048700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b10103540 0x7f8b101a0260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:56:44.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:56:44.119+0000 7f8b15048700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f8b1010d580 con 0x7f8b10103540 2024-04-17T05:56:44.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:56:44.119+0000 7f8b15048700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8b1010d400 con 0x7f8b10102ba0 2024-04-17T05:56:44.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:56:44.119+0000 7f8b15048700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8b1010d700 con 0x7f8b101019a0 2024-04-17T05:56:44.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:56:44.119+0000 7f8b0ed9d700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8b101019a0 0x7f8b10198ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:56:44.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:56:44.119+0000 7f8b0ed9d700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8b101019a0 0x7f8b10198ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55702/0 (socket says 172.21.15.103:55702) 2024-04-17T05:56:44.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:56:44.119+0000 7f8b0ed9d700 1 -- 172.21.15.103:0/2363731149 learned_addr learned my addr 172.21.15.103:0/2363731149 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-04-17T05:56:44.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:56:44.119+0000 7f8b07fff700 1 --2- 172.21.15.103:0/2363731149 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8b10102ba0 0x7f8b10199220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:56:44.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:56:44.119+0000 7f8b0f59e700 1 --2- 172.21.15.103:0/2363731149 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b10103540 0x7f8b101a0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:56:44.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:56:44.120+0000 7f8b0ed9d700 1 -- 172.21.15.103:0/2363731149 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8b10102ba0 msgr2=0x7f8b10199220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:56:44.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:56:44.120+0000 7f8b0ed9d700 1 --2- 172.21.15.103:0/2363731149 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8b10102ba0 0x7f8b10199220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:56:44.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:56:44.120+0000 7f8b0ed9d700 1 -- 172.21.15.103:0/2363731149 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b10103540 msgr2=0x7f8b101a0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:56:44.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:56:44.120+0000 7f8b0ed9d700 1 --2- 172.21.15.103:0/2363731149 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b10103540 0x7f8b101a0260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:56:44.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:56:44.120+0000 7f8b0ed9d700 1 -- 172.21.15.103:0/2363731149 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b101a0930 con 0x7f8b101019a0 2024-04-17T05:56:44.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:56:44.120+0000 7f8b0f59e700 1 --2- 172.21.15.103:0/2363731149 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b10103540 0x7f8b101a0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T05:56:44.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:56:44.120+0000 7f8b0ed9d700 1 --2- 172.21.15.103:0/2363731149 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8b101019a0 0x7f8b10198ce0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f8b0000e950 tx=0x7f8b0000ec60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:56:44.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:56:44.120+0000 7f8b0cd99700 1 -- 172.21.15.103:0/2363731149 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b0000eea0 con 0x7f8b101019a0 2024-04-17T05:56:44.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:56:44.120+0000 7f8b15048700 1 -- 172.21.15.103:0/2363731149 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b101a0c20 con 0x7f8b101019a0 2024-04-17T05:56:44.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:56:44.120+0000 7f8b0cd99700 1 -- 172.21.15.103:0/2363731149 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b00004510 con 0x7f8b101019a0 2024-04-17T05:56:44.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:56:44.120+0000 7f8b0cd99700 1 -- 172.21.15.103:0/2363731149 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b00010430 con 0x7f8b101019a0 2024-04-17T05:56:44.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:56:44.120+0000 7f8b15048700 1 -- 172.21.15.103:0/2363731149 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b101a1160 con 0x7f8b101019a0 2024-04-17T05:56:44.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:56:44.121+0000 7f8b15048700 1 -- 172.21.15.103:0/2363731149 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b10069740 con 0x7f8b101019a0 2024-04-17T05:56:44.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:56:44.122+0000 7f8b0cd99700 1 -- 172.21.15.103:0/2363731149 <== mon.1 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8b00003e90 con 0x7f8b101019a0 2024-04-17T05:56:44.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:56:44.123+0000 7f8b0cd99700 1 --2- 172.21.15.103:0/2363731149 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f8afc0a3c40 0x7f8afc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:56:44.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:56:44.123+0000 7f8b07fff700 1 --2- 172.21.15.103:0/2363731149 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f8afc0a3c40 0x7f8afc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:56:44.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:56:44.123+0000 7f8b0cd99700 1 -- 172.21.15.103:0/2363731149 <== mon.1 v2:172.21.15.103:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5426+0+0 (secure 0 0 0) 0x7f8b000cfd30 con 0x7f8b101019a0 2024-04-17T05:56:44.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:56:44.123+0000 7f8b07fff700 1 --2- 172.21.15.103:0/2363731149 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f8afc0a3c40 0x7f8afc0a6100 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8af800b2e0 tx=0x7f8af8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:56:44.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:56:44.127+0000 7f8b0cd99700 1 -- 172.21.15.103:0/2363731149 <== mon.1 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f8b0009d1b0 con 0x7f8b101019a0 2024-04-17T05:56:44.134 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:43 smithi098 ceph-mon[36516]: osdmap e53: 8 total, 8 up, 8 in 2024-04-17T05:56:44.134 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:43 smithi098 ceph-mon[36516]: Detected new or changed devices on smithi098 2024-04-17T05:56:44.134 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:43 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:56:44.134 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:43 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:56:44.134 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:43 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-17T05:56:44.134 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:43 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-17T05:56:44.134 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:43 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-17T05:56:44.134 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:43 smithi098 ceph-mon[36516]: Adjusting osd_memory_target on smithi098 to 5693M 2024-04-17T05:56:44.134 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:43 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:56:44.135 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:43 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:56:44.135 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:43 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T05:56:44.135 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:43 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:56:44.135 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:43 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T05:56:44.135 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:43 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:56:44.135 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:43 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T05:56:44.136 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:43 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:56:44.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:43 smithi047 ceph-mon[32223]: osdmap e53: 8 total, 8 up, 8 in 2024-04-17T05:56:44.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:43 smithi047 ceph-mon[32223]: Detected new or changed devices on smithi098 2024-04-17T05:56:44.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:43 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:56:44.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:43 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:56:44.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:43 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-17T05:56:44.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:43 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-17T05:56:44.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:43 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-17T05:56:44.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:43 smithi047 ceph-mon[32223]: Adjusting osd_memory_target on smithi098 to 5693M 2024-04-17T05:56:44.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:43 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:56:44.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:43 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:56:44.140 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:43 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T05:56:44.140 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:43 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:56:44.140 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:43 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T05:56:44.140 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:43 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:56:44.140 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:43 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T05:56:44.140 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:43 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:56:44.332 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:56:44.334+0000 7f8b15048700 1 -- 172.21.15.103:0/2363731149 --> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f8b1006b6d0 con 0x7f8afc0a3c40 2024-04-17T05:56:45.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:44 smithi098 ceph-mon[36516]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-04-17T05:56:45.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:44 smithi098 ceph-mon[36516]: osdmap e54: 8 total, 8 up, 8 in 2024-04-17T05:56:45.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:44 smithi098 ceph-mon[36516]: from='client.24283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T05:56:45.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:44 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-17T05:56:45.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:44 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-17T05:56:45.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:44 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:56:45.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:44 smithi098 ceph-mon[36516]: osdmap e55: 8 total, 8 up, 8 in 2024-04-17T05:56:45.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:44 smithi047 ceph-mon[32223]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-04-17T05:56:45.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:44 smithi047 ceph-mon[32223]: osdmap e54: 8 total, 8 up, 8 in 2024-04-17T05:56:45.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:44 smithi047 ceph-mon[32223]: from='client.24283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T05:56:45.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:44 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-17T05:56:45.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:44 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-17T05:56:45.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:44 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:56:45.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:44 smithi047 ceph-mon[32223]: osdmap e55: 8 total, 8 up, 8 in 2024-04-17T05:56:45.159 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:44 smithi103 ceph-mon[36243]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-04-17T05:56:45.160 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:44 smithi103 ceph-mon[36243]: osdmap e54: 8 total, 8 up, 8 in 2024-04-17T05:56:45.160 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:44 smithi103 ceph-mon[36243]: from='client.24283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T05:56:45.160 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:44 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-17T05:56:45.160 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:44 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-17T05:56:45.160 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:44 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:56:45.160 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:44 smithi103 ceph-mon[36243]: osdmap e55: 8 total, 8 up, 8 in 2024-04-17T05:56:46.638 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:46 smithi047 ceph-mon[32223]: osdmap e56: 8 total, 8 up, 8 in 2024-04-17T05:56:46.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:46 smithi098 ceph-mon[36516]: osdmap e56: 8 total, 8 up, 8 in 2024-04-17T05:56:47.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:46 smithi103 ceph-mon[36243]: osdmap e56: 8 total, 8 up, 8 in 2024-04-17T05:56:47.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:47 smithi103 ceph-mon[36243]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-04-17T05:56:47.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:47 smithi103 ceph-mon[36243]: from='client.? 172.21.15.103:0/2997395802' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cee4712f-321b-42c0-92ba-e15144c00f57"}]: dispatch 2024-04-17T05:56:47.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:47 smithi103 ceph-mon[36243]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cee4712f-321b-42c0-92ba-e15144c00f57"}]: dispatch 2024-04-17T05:56:47.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:47 smithi103 ceph-mon[36243]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cee4712f-321b-42c0-92ba-e15144c00f57"}]': finished 2024-04-17T05:56:47.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:47 smithi103 ceph-mon[36243]: osdmap e57: 9 total, 8 up, 9 in 2024-04-17T05:56:47.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:47 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-17T05:56:47.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:47 smithi098 ceph-mon[36516]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-04-17T05:56:47.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:47 smithi098 ceph-mon[36516]: from='client.? 172.21.15.103:0/2997395802' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cee4712f-321b-42c0-92ba-e15144c00f57"}]: dispatch 2024-04-17T05:56:47.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:47 smithi098 ceph-mon[36516]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cee4712f-321b-42c0-92ba-e15144c00f57"}]: dispatch 2024-04-17T05:56:47.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:47 smithi098 ceph-mon[36516]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cee4712f-321b-42c0-92ba-e15144c00f57"}]': finished 2024-04-17T05:56:47.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:47 smithi098 ceph-mon[36516]: osdmap e57: 9 total, 8 up, 9 in 2024-04-17T05:56:47.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:47 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-17T05:56:47.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:47 smithi047 ceph-mon[32223]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-04-17T05:56:47.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:47 smithi047 ceph-mon[32223]: from='client.? 172.21.15.103:0/2997395802' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cee4712f-321b-42c0-92ba-e15144c00f57"}]: dispatch 2024-04-17T05:56:47.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:47 smithi047 ceph-mon[32223]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cee4712f-321b-42c0-92ba-e15144c00f57"}]: dispatch 2024-04-17T05:56:47.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:47 smithi047 ceph-mon[32223]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cee4712f-321b-42c0-92ba-e15144c00f57"}]': finished 2024-04-17T05:56:47.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:47 smithi047 ceph-mon[32223]: osdmap e57: 9 total, 8 up, 9 in 2024-04-17T05:56:47.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:47 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-17T05:56:48.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:48 smithi098 ceph-mon[36516]: from='client.? 172.21.15.103:0/1699574287' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-17T05:56:48.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:48 smithi047 ceph-mon[32223]: from='client.? 172.21.15.103:0/1699574287' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-17T05:56:49.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:48 smithi103 ceph-mon[36243]: from='client.? 172.21.15.103:0/1699574287' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-17T05:56:49.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:49 smithi098 ceph-mon[36516]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-04-17T05:56:49.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:49 smithi047 ceph-mon[32223]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-04-17T05:56:50.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:49 smithi103 ceph-mon[36243]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-04-17T05:56:51.867 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:51 smithi103 ceph-mon[36243]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-04-17T05:56:51.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:51 smithi098 ceph-mon[36516]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-04-17T05:56:51.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:51 smithi047 ceph-mon[32223]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-04-17T05:56:53.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:53 smithi103 ceph-mon[36243]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-04-17T05:56:53.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:53 smithi098 ceph-mon[36516]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-04-17T05:56:53.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:53 smithi047 ceph-mon[32223]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-04-17T05:56:54.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:54 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-17T05:56:54.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:54 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:56:54.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:54 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-17T05:56:54.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:54 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:56:54.893 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:54 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-17T05:56:54.893 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:54 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:56:55.849 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:55 smithi103 ceph-mon[36243]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-04-17T05:56:55.849 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:55 smithi103 ceph-mon[36243]: Deploying daemon osd.8 on smithi103 2024-04-17T05:56:55.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:55 smithi098 ceph-mon[36516]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-04-17T05:56:55.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:55 smithi098 ceph-mon[36516]: Deploying daemon osd.8 on smithi103 2024-04-17T05:56:55.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:55 smithi047 ceph-mon[32223]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-04-17T05:56:55.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:55 smithi047 ceph-mon[32223]: Deploying daemon osd.8 on smithi103 2024-04-17T05:56:57.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:57 smithi047 ceph-mon[32223]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-04-17T05:56:57.757 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:57 smithi103 ceph-mon[36243]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-04-17T05:56:57.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:57 smithi098 ceph-mon[36516]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-04-17T05:56:59.783 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:56:59 smithi103 ceph-mon[36243]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-04-17T05:56:59.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:56:59 smithi098 ceph-mon[36516]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-04-17T05:56:59.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:56:59 smithi047 ceph-mon[32223]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-04-17T05:57:01.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:00 smithi103 ceph-mon[36243]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-04-17T05:57:01.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:00 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:57:01.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:00 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T05:57:01.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:00 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:57:01.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:00 smithi098 ceph-mon[36516]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-04-17T05:57:01.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:00 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:57:01.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:00 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T05:57:01.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:00 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:57:01.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:00 smithi047 ceph-mon[32223]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-04-17T05:57:01.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:00 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:57:01.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:00 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T05:57:01.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:00 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:57:03.500 INFO:teuthology.orchestra.run.smithi103.stdout:Created osd(s) 8 on host 'smithi103' 2024-04-17T05:57:03.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:57:03.501+0000 7f8b0cd99700 1 -- 172.21.15.103:0/2363731149 <== mgr.14152 v2:172.21.15.47:6800/3157902806 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f8b1006b6d0 con 0x7f8afc0a3c40 2024-04-17T05:57:03.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:57:03.503+0000 7f8b15048700 1 -- 172.21.15.103:0/2363731149 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f8afc0a3c40 msgr2=0x7f8afc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:57:03.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:57:03.503+0000 7f8b15048700 1 --2- 172.21.15.103:0/2363731149 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f8afc0a3c40 0x7f8afc0a6100 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8af800b2e0 tx=0x7f8af8005fd0 comp rx=0 tx=0).stop 2024-04-17T05:57:03.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:57:03.503+0000 7f8b15048700 1 -- 172.21.15.103:0/2363731149 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8b101019a0 msgr2=0x7f8b10198ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:57:03.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:57:03.503+0000 7f8b15048700 1 --2- 172.21.15.103:0/2363731149 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8b101019a0 0x7f8b10198ce0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f8b0000e950 tx=0x7f8b0000ec60 comp rx=0 tx=0).stop 2024-04-17T05:57:03.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:57:03.504+0000 7f8b15048700 1 -- 172.21.15.103:0/2363731149 shutdown_connections 2024-04-17T05:57:03.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:57:03.504+0000 7f8b15048700 1 --2- 172.21.15.103:0/2363731149 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f8afc0a3c40 0x7f8afc0a6100 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:57:03.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:57:03.504+0000 7f8b15048700 1 --2- 172.21.15.103:0/2363731149 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b10103540 0x7f8b101a0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:57:03.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:57:03.504+0000 7f8b15048700 1 --2- 172.21.15.103:0/2363731149 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8b101019a0 0x7f8b10198ce0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:57:03.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:57:03.504+0000 7f8b15048700 1 --2- 172.21.15.103:0/2363731149 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8b10102ba0 0x7f8b10199220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:57:03.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:57:03.504+0000 7f8b15048700 1 -- 172.21.15.103:0/2363731149 >> 172.21.15.103:0/2363731149 conn(0x7f8b100fcf40 msgr2=0x7f8b1006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:57:03.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:57:03.504+0000 7f8b15048700 1 -- 172.21.15.103:0/2363731149 shutdown_connections 2024-04-17T05:57:03.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:57:03.504+0000 7f8b15048700 1 -- 172.21.15.103:0/2363731149 wait complete. 2024-04-17T05:57:03.783 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:03 smithi103 ceph-mon[36243]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-04-17T05:57:03.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:03 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:57:03.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:03 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:57:03.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:03 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:57:03.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:03 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T05:57:03.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:03 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:57:03.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:03 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:57:03.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:03 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:57:03.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:03 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T05:57:03.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:03 smithi103 ceph-mon[36243]: from='osd.8 [v2:172.21.15.103:6816/3433905520,v1:172.21.15.103:6817/3433905520]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-17T05:57:03.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:03 smithi103 ceph-mon[36243]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-17T05:57:03.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:03 smithi098 ceph-mon[36516]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-04-17T05:57:03.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:03 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:57:03.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:03 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:57:03.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:03 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:57:03.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:03 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T05:57:03.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:03 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:57:03.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:03 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:57:03.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:03 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:57:03.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:03 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T05:57:03.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:03 smithi098 ceph-mon[36516]: from='osd.8 [v2:172.21.15.103:6816/3433905520,v1:172.21.15.103:6817/3433905520]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-17T05:57:03.884 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:03 smithi098 ceph-mon[36516]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-17T05:57:03.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:03 smithi047 ceph-mon[32223]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-04-17T05:57:03.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:03 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:57:03.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:03 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:57:03.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:03 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:57:03.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:03 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T05:57:03.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:03 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:57:03.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:03 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:57:03.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:03 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:57:03.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:03 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T05:57:03.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:03 smithi047 ceph-mon[32223]: from='osd.8 [v2:172.21.15.103:6816/3433905520,v1:172.21.15.103:6817/3433905520]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-17T05:57:03.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:03 smithi047 ceph-mon[32223]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-17T05:57:04.197 DEBUG:teuthology.orchestra.run.smithi103:osd.8> sudo journalctl -f -n 0 -u ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@osd.8.service 2024-04-17T05:57:04.200 INFO:tasks.cephadm:Deploying osd.9 on smithi047 with /dev/vg_nvme/lv_1... 2024-04-17T05:57:04.200 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 565db2d8-fc7e-11ee-bc91-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-17T05:57:04.458 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T05:57:05.722 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:05 smithi103 ceph-mon[36243]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-04-17T05:57:05.722 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:05 smithi103 ceph-mon[36243]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-17T05:57:05.722 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:05 smithi103 ceph-mon[36243]: osdmap e58: 9 total, 8 up, 9 in 2024-04-17T05:57:05.722 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:05 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-17T05:57:05.723 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:05 smithi103 ceph-mon[36243]: from='osd.8 [v2:172.21.15.103:6816/3433905520,v1:172.21.15.103:6817/3433905520]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-04-17T05:57:05.723 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:05 smithi103 ceph-mon[36243]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-04-17T05:57:05.723 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 05:57:05 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-8[54941]: 2024-04-17T05:57:05.509+0000 7f9c302c2700 -1 osd.8 0 waiting for initial osdmap 2024-04-17T05:57:05.723 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 05:57:05 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-8[54941]: 2024-04-17T05:57:05.518+0000 7f9c2c0b5700 -1 osd.8 59 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-17T05:57:05.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:05 smithi098 ceph-mon[36516]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-04-17T05:57:05.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:05 smithi098 ceph-mon[36516]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-17T05:57:05.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:05 smithi098 ceph-mon[36516]: osdmap e58: 9 total, 8 up, 9 in 2024-04-17T05:57:05.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:05 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-17T05:57:05.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:05 smithi098 ceph-mon[36516]: from='osd.8 [v2:172.21.15.103:6816/3433905520,v1:172.21.15.103:6817/3433905520]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-04-17T05:57:05.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:05 smithi098 ceph-mon[36516]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-04-17T05:57:05.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:05 smithi047 ceph-mon[32223]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-04-17T05:57:05.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:05 smithi047 ceph-mon[32223]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-17T05:57:05.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:05 smithi047 ceph-mon[32223]: osdmap e58: 9 total, 8 up, 9 in 2024-04-17T05:57:05.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:05 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-17T05:57:05.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:05 smithi047 ceph-mon[32223]: from='osd.8 [v2:172.21.15.103:6816/3433905520,v1:172.21.15.103:6817/3433905520]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-04-17T05:57:05.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:05 smithi047 ceph-mon[32223]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-04-17T05:57:06.657 INFO:teuthology.orchestra.run.smithi047.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-17T05:57:06.657 INFO:teuthology.orchestra.run.smithi047.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-17T05:57:06.657 INFO:teuthology.orchestra.run.smithi047.stderr: stderr: 10+0 records in 2024-04-17T05:57:06.657 INFO:teuthology.orchestra.run.smithi047.stderr:10+0 records out 2024-04-17T05:57:06.657 INFO:teuthology.orchestra.run.smithi047.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00970446 s, 1.1 GB/s 2024-04-17T05:57:06.657 INFO:teuthology.orchestra.run.smithi047.stderr:--> Zapping successful for: 2024-04-17T05:57:06.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:06 smithi103 ceph-mon[36243]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-04-17T05:57:06.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:06 smithi103 ceph-mon[36243]: osdmap e59: 9 total, 8 up, 9 in 2024-04-17T05:57:06.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:06 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-17T05:57:06.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:06 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-17T05:57:06.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:06 smithi098 ceph-mon[36516]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-04-17T05:57:06.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:06 smithi098 ceph-mon[36516]: osdmap e59: 9 total, 8 up, 9 in 2024-04-17T05:57:06.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:06 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-17T05:57:06.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:06 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-17T05:57:06.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:06 smithi047 ceph-mon[32223]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-04-17T05:57:06.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:06 smithi047 ceph-mon[32223]: osdmap e59: 9 total, 8 up, 9 in 2024-04-17T05:57:06.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:06 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-17T05:57:06.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:06 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-17T05:57:07.500 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 565db2d8-fc7e-11ee-bc91-c7b262605968 -- ceph orch daemon add osd smithi047:vg_nvme/lv_1 2024-04-17T05:57:07.747 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T05:57:07.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:07 smithi103 ceph-mon[36243]: purged_snaps scrub starts 2024-04-17T05:57:07.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:07 smithi103 ceph-mon[36243]: purged_snaps scrub ok 2024-04-17T05:57:07.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:07 smithi103 ceph-mon[36243]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-04-17T05:57:07.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:07 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-17T05:57:07.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:07 smithi103 ceph-mon[36243]: osd.8 [v2:172.21.15.103:6816/3433905520,v1:172.21.15.103:6817/3433905520] boot 2024-04-17T05:57:07.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:07 smithi103 ceph-mon[36243]: osdmap e60: 9 total, 9 up, 9 in 2024-04-17T05:57:07.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:07 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-17T05:57:07.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:07 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:57:07.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:07 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:57:07.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:07 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-17T05:57:07.785 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:07 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-17T05:57:07.785 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:07 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-17T05:57:07.785 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:07 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:57:07.785 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:07 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:57:07.785 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:07 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T05:57:07.785 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:07 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:57:07.869 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:07 smithi047 ceph-mon[32223]: purged_snaps scrub starts 2024-04-17T05:57:07.870 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:07 smithi047 ceph-mon[32223]: purged_snaps scrub ok 2024-04-17T05:57:07.870 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:07 smithi047 ceph-mon[32223]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-04-17T05:57:07.870 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:07 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-17T05:57:07.870 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:07 smithi047 ceph-mon[32223]: osd.8 [v2:172.21.15.103:6816/3433905520,v1:172.21.15.103:6817/3433905520] boot 2024-04-17T05:57:07.870 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:07 smithi047 ceph-mon[32223]: osdmap e60: 9 total, 9 up, 9 in 2024-04-17T05:57:07.870 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:07 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-17T05:57:07.870 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:07 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:57:07.870 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:07 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:57:07.870 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:07 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-17T05:57:07.870 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:07 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-17T05:57:07.870 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:07 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-17T05:57:07.870 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:07 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:57:07.870 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:07 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:57:07.871 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:07 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T05:57:07.871 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:07 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:57:07.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:07 smithi098 ceph-mon[36516]: purged_snaps scrub starts 2024-04-17T05:57:07.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:07 smithi098 ceph-mon[36516]: purged_snaps scrub ok 2024-04-17T05:57:07.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:07 smithi098 ceph-mon[36516]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-04-17T05:57:07.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:07 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-17T05:57:07.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:07 smithi098 ceph-mon[36516]: osd.8 [v2:172.21.15.103:6816/3433905520,v1:172.21.15.103:6817/3433905520] boot 2024-04-17T05:57:07.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:07 smithi098 ceph-mon[36516]: osdmap e60: 9 total, 9 up, 9 in 2024-04-17T05:57:07.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:07 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-17T05:57:07.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:07 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:57:07.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:07 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:57:07.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:07 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-17T05:57:07.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:07 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-17T05:57:07.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:07 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-17T05:57:07.884 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:07 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:57:07.884 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:07 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:57:07.884 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:07 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T05:57:07.884 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:07 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:57:08.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:08 smithi103 ceph-mon[36243]: Detected new or changed devices on smithi103 2024-04-17T05:57:08.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:08 smithi103 ceph-mon[36243]: Adjusting osd_memory_target on smithi103 to 5693M 2024-04-17T05:57:08.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:08 smithi103 ceph-mon[36243]: osdmap e61: 9 total, 9 up, 9 in 2024-04-17T05:57:08.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:08 smithi098 ceph-mon[36516]: Detected new or changed devices on smithi103 2024-04-17T05:57:08.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:08 smithi098 ceph-mon[36516]: Adjusting osd_memory_target on smithi103 to 5693M 2024-04-17T05:57:08.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:08 smithi098 ceph-mon[36516]: osdmap e61: 9 total, 9 up, 9 in 2024-04-17T05:57:08.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:08 smithi047 ceph-mon[32223]: Detected new or changed devices on smithi103 2024-04-17T05:57:08.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:08 smithi047 ceph-mon[32223]: Adjusting osd_memory_target on smithi103 to 5693M 2024-04-17T05:57:08.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:08 smithi047 ceph-mon[32223]: osdmap e61: 9 total, 9 up, 9 in 2024-04-17T05:57:09.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:57:09.440+0000 7fd5ca9a1700 1 -- 172.21.15.47:0/1598908204 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5c4102bb0 msgr2=0x7fd5c4103010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:57:09.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:57:09.440+0000 7fd5ca9a1700 1 --2- 172.21.15.47:0/1598908204 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5c4102bb0 0x7fd5c4103010 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac0099c0 tx=0x7fd5ac009cd0 comp rx=0 tx=0).stop 2024-04-17T05:57:09.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:57:09.440+0000 7fd5ca9a1700 1 -- 172.21.15.47:0/1598908204 shutdown_connections 2024-04-17T05:57:09.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:57:09.440+0000 7fd5ca9a1700 1 --2- 172.21.15.47:0/1598908204 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd5c4103550 0x7fd5c4109e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:57:09.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:57:09.440+0000 7fd5ca9a1700 1 --2- 172.21.15.47:0/1598908204 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd5c41019b0 0x7fd5c4101dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:57:09.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:57:09.440+0000 7fd5ca9a1700 1 --2- 172.21.15.47:0/1598908204 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5c4102bb0 0x7fd5c4103010 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:57:09.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:57:09.440+0000 7fd5ca9a1700 1 -- 172.21.15.47:0/1598908204 >> 172.21.15.47:0/1598908204 conn(0x7fd5c40fcf50 msgr2=0x7fd5c40ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:57:09.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:57:09.440+0000 7fd5ca9a1700 1 -- 172.21.15.47:0/1598908204 shutdown_connections 2024-04-17T05:57:09.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:57:09.440+0000 7fd5ca9a1700 1 -- 172.21.15.47:0/1598908204 wait complete. 2024-04-17T05:57:09.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:57:09.441+0000 7fd5ca9a1700 1 Processor -- start 2024-04-17T05:57:09.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:57:09.441+0000 7fd5ca9a1700 1 -- start start 2024-04-17T05:57:09.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:57:09.441+0000 7fd5ca9a1700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd5c41019b0 0x7fd5c406b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:57:09.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:57:09.441+0000 7fd5ca9a1700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd5c4102bb0 0x7fd5c419da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:57:09.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:57:09.441+0000 7fd5ca9a1700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5c4103550 0x7fd5c41a0070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:57:09.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:57:09.441+0000 7fd5ca9a1700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c410d5c0 con 0x7fd5c4103550 2024-04-17T05:57:09.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:57:09.441+0000 7fd5ca9a1700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c410d440 con 0x7fd5c4102bb0 2024-04-17T05:57:09.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:57:09.441+0000 7fd5ca9a1700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c410d740 con 0x7fd5c41019b0 2024-04-17T05:57:09.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:57:09.442+0000 7fd5c8f3e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5c4103550 0x7fd5c41a0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:57:09.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:57:09.442+0000 7fd5c8f3e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5c4103550 0x7fd5c41a0070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51602/0 (socket says 172.21.15.47:51602) 2024-04-17T05:57:09.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:57:09.442+0000 7fd5c8f3e700 1 -- 172.21.15.47:0/3199881596 learned_addr learned my addr 172.21.15.47:0/3199881596 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:57:09.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:57:09.442+0000 7fd5c3fff700 1 --2- 172.21.15.47:0/3199881596 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd5c41019b0 0x7fd5c406b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:57:09.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:57:09.442+0000 7fd5bbfff700 1 --2- 172.21.15.47:0/3199881596 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd5c4102bb0 0x7fd5c419da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:57:09.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:57:09.442+0000 7fd5c8f3e700 1 -- 172.21.15.47:0/3199881596 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd5c41019b0 msgr2=0x7fd5c406b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:57:09.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:57:09.442+0000 7fd5c8f3e700 1 --2- 172.21.15.47:0/3199881596 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd5c41019b0 0x7fd5c406b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:57:09.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:57:09.442+0000 7fd5c8f3e700 1 -- 172.21.15.47:0/3199881596 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd5c4102bb0 msgr2=0x7fd5c419da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:57:09.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:57:09.442+0000 7fd5c8f3e700 1 --2- 172.21.15.47:0/3199881596 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd5c4102bb0 0x7fd5c419da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:57:09.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:57:09.442+0000 7fd5c8f3e700 1 -- 172.21.15.47:0/3199881596 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5c41a0940 con 0x7fd5c4103550 2024-04-17T05:57:09.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:57:09.442+0000 7fd5c8f3e700 1 --2- 172.21.15.47:0/3199881596 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5c4103550 0x7fd5c41a0070 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc007d70 tx=0x7fd5bc00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:57:09.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:57:09.442+0000 7fd5c1ffb700 1 -- 172.21.15.47:0/3199881596 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5bc016020 con 0x7fd5c4103550 2024-04-17T05:57:09.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:57:09.442+0000 7fd5c1ffb700 1 -- 172.21.15.47:0/3199881596 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5bc00ac90 con 0x7fd5c4103550 2024-04-17T05:57:09.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:57:09.442+0000 7fd5ca9a1700 1 -- 172.21.15.47:0/3199881596 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5c41a0c30 con 0x7fd5c4103550 2024-04-17T05:57:09.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:57:09.442+0000 7fd5c1ffb700 1 -- 172.21.15.47:0/3199881596 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5bc0115c0 con 0x7fd5c4103550 2024-04-17T05:57:09.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:57:09.442+0000 7fd5ca9a1700 1 -- 172.21.15.47:0/3199881596 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5c41a1140 con 0x7fd5c4103550 2024-04-17T05:57:09.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:57:09.443+0000 7fd5ca9a1700 1 -- 172.21.15.47:0/3199881596 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5c4069740 con 0x7fd5c4103550 2024-04-17T05:57:09.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:57:09.444+0000 7fd5c1ffb700 1 -- 172.21.15.47:0/3199881596 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fd5bc019020 con 0x7fd5c4103550 2024-04-17T05:57:09.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:57:09.445+0000 7fd5c1ffb700 1 --2- 172.21.15.47:0/3199881596 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fd5b00a3cc0 0x7fd5b00a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:57:09.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:57:09.445+0000 7fd5c3fff700 1 --2- 172.21.15.47:0/3199881596 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fd5b00a3cc0 0x7fd5b00a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:57:09.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:57:09.445+0000 7fd5c1ffb700 1 -- 172.21.15.47:0/3199881596 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5858+0+0 (secure 0 0 0) 0x7fd5bc0cfd30 con 0x7fd5c4103550 2024-04-17T05:57:09.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:57:09.446+0000 7fd5c3fff700 1 --2- 172.21.15.47:0/3199881596 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fd5b00a3cc0 0x7fd5b00a6180 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd5c4102a10 tx=0x7fd5b4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:57:09.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:57:09.449+0000 7fd5c1ffb700 1 -- 172.21.15.47:0/3199881596 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fd5bc020020 con 0x7fd5c4103550 2024-04-17T05:57:09.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:57:09.669+0000 7fd5ca9a1700 1 -- 172.21.15.47:0/3199881596 --> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fd5c4062e90 con 0x7fd5b00a3cc0 2024-04-17T05:57:09.783 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:09 smithi103 ceph-mon[36243]: pgmap v198: 1 pgs: 1 remapped+peering; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-04-17T05:57:09.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:09 smithi103 ceph-mon[36243]: osdmap e62: 9 total, 9 up, 9 in 2024-04-17T05:57:09.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:09 smithi098 ceph-mon[36516]: pgmap v198: 1 pgs: 1 remapped+peering; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-04-17T05:57:09.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:09 smithi098 ceph-mon[36516]: osdmap e62: 9 total, 9 up, 9 in 2024-04-17T05:57:09.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:09 smithi047 ceph-mon[32223]: pgmap v198: 1 pgs: 1 remapped+peering; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-04-17T05:57:09.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:09 smithi047 ceph-mon[32223]: osdmap e62: 9 total, 9 up, 9 in 2024-04-17T05:57:10.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:10 smithi098 ceph-mon[36516]: osdmap e63: 9 total, 9 up, 9 in 2024-04-17T05:57:10.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:10 smithi098 ceph-mon[36516]: from='client.14448 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T05:57:10.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:10 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-17T05:57:10.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:10 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-17T05:57:10.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:10 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:57:10.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:10 smithi047 ceph-mon[32223]: osdmap e63: 9 total, 9 up, 9 in 2024-04-17T05:57:10.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:10 smithi047 ceph-mon[32223]: from='client.14448 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T05:57:10.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:10 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-17T05:57:10.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:10 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-17T05:57:10.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:10 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:57:11.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:10 smithi103 ceph-mon[36243]: osdmap e63: 9 total, 9 up, 9 in 2024-04-17T05:57:11.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:10 smithi103 ceph-mon[36243]: from='client.14448 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T05:57:11.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:10 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-17T05:57:11.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:10 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-17T05:57:11.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:10 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:57:11.840 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:11 smithi047 ceph-mon[32223]: pgmap v201: 1 pgs: 1 remapped+peering; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-04-17T05:57:11.840 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:11 smithi047 ceph-mon[32223]: osdmap e64: 9 total, 9 up, 9 in 2024-04-17T05:57:11.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:11 smithi098 ceph-mon[36516]: pgmap v201: 1 pgs: 1 remapped+peering; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-04-17T05:57:11.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:11 smithi098 ceph-mon[36516]: osdmap e64: 9 total, 9 up, 9 in 2024-04-17T05:57:12.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:11 smithi103 ceph-mon[36243]: pgmap v201: 1 pgs: 1 remapped+peering; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-04-17T05:57:12.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:11 smithi103 ceph-mon[36243]: osdmap e64: 9 total, 9 up, 9 in 2024-04-17T05:57:12.847 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:12 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/620174355' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "63fc73c1-5a22-487d-80b8-9894e2ddd207"}]: dispatch 2024-04-17T05:57:12.848 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:12 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/620174355' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "63fc73c1-5a22-487d-80b8-9894e2ddd207"}]': finished 2024-04-17T05:57:12.848 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:12 smithi047 ceph-mon[32223]: osdmap e65: 10 total, 9 up, 10 in 2024-04-17T05:57:12.848 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:12 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-17T05:57:12.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:12 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/620174355' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "63fc73c1-5a22-487d-80b8-9894e2ddd207"}]: dispatch 2024-04-17T05:57:12.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:12 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/620174355' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "63fc73c1-5a22-487d-80b8-9894e2ddd207"}]': finished 2024-04-17T05:57:12.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:12 smithi098 ceph-mon[36516]: osdmap e65: 10 total, 9 up, 10 in 2024-04-17T05:57:12.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:12 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-17T05:57:13.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:12 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/620174355' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "63fc73c1-5a22-487d-80b8-9894e2ddd207"}]: dispatch 2024-04-17T05:57:13.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:12 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/620174355' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "63fc73c1-5a22-487d-80b8-9894e2ddd207"}]': finished 2024-04-17T05:57:13.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:12 smithi103 ceph-mon[36243]: osdmap e65: 10 total, 9 up, 10 in 2024-04-17T05:57:13.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:12 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-17T05:57:13.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:13 smithi098 ceph-mon[36516]: pgmap v203: 1 pgs: 1 remapped+peering; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-04-17T05:57:13.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:13 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/2854544147' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-17T05:57:13.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:13 smithi047 ceph-mon[32223]: pgmap v203: 1 pgs: 1 remapped+peering; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-04-17T05:57:13.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:13 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/2854544147' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-17T05:57:14.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:13 smithi103 ceph-mon[36243]: pgmap v203: 1 pgs: 1 remapped+peering; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-04-17T05:57:14.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:13 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/2854544147' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-17T05:57:15.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:15 smithi098 ceph-mon[36516]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-04-17T05:57:15.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:15 smithi047 ceph-mon[32223]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-04-17T05:57:16.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:15 smithi103 ceph-mon[36243]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-04-17T05:57:17.858 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:17 smithi047 ceph-mon[32223]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-04-17T05:57:17.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:17 smithi098 ceph-mon[36516]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-04-17T05:57:18.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:17 smithi103 ceph-mon[36243]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-04-17T05:57:19.828 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:19 smithi047 ceph-mon[32223]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-17T05:57:19.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:19 smithi098 ceph-mon[36516]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-17T05:57:20.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:19 smithi103 ceph-mon[36243]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-17T05:57:20.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:20 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-17T05:57:20.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:20 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:57:20.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:20 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-17T05:57:20.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:20 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:57:21.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:20 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-17T05:57:21.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:20 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:57:21.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:21 smithi098 ceph-mon[36516]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-04-17T05:57:21.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:21 smithi098 ceph-mon[36516]: Deploying daemon osd.9 on smithi047 2024-04-17T05:57:21.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:21 smithi047 ceph-mon[32223]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-04-17T05:57:21.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:21 smithi047 ceph-mon[32223]: Deploying daemon osd.9 on smithi047 2024-04-17T05:57:22.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:21 smithi103 ceph-mon[36243]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-04-17T05:57:22.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:21 smithi103 ceph-mon[36243]: Deploying daemon osd.9 on smithi047 2024-04-17T05:57:23.836 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:23 smithi103 ceph-mon[36243]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-17T05:57:23.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:23 smithi098 ceph-mon[36516]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-17T05:57:23.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:23 smithi047 ceph-mon[32223]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-17T05:57:25.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:25 smithi047 ceph-mon[32223]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-04-17T05:57:25.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:25 smithi098 ceph-mon[36516]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-04-17T05:57:26.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:25 smithi103 ceph-mon[36243]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-04-17T05:57:27.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:27 smithi098 ceph-mon[36516]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-04-17T05:57:27.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:27 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:57:27.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:27 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:57:27.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:27 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T05:57:27.638 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:27 smithi047 ceph-mon[32223]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-04-17T05:57:27.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:27 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:57:27.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:27 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:57:27.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:27 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T05:57:27.783 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:27 smithi103 ceph-mon[36243]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-04-17T05:57:27.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:27 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:57:27.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:27 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:57:27.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:27 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T05:57:29.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:29 smithi098 ceph-mon[36516]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-04-17T05:57:29.638 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:29 smithi047 ceph-mon[32223]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-04-17T05:57:29.783 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:29 smithi103 ceph-mon[36243]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-04-17T05:57:30.634 INFO:teuthology.orchestra.run.smithi047.stdout:Created osd(s) 9 on host 'smithi047' 2024-04-17T05:57:30.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:57:30.636+0000 7fd5c1ffb700 1 -- 172.21.15.47:0/3199881596 <== mgr.14152 v2:172.21.15.47:6800/3157902806 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd5c4062e90 con 0x7fd5b00a3cc0 2024-04-17T05:57:30.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:57:30.638+0000 7fd5ca9a1700 1 -- 172.21.15.47:0/3199881596 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fd5b00a3cc0 msgr2=0x7fd5b00a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:57:30.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:57:30.638+0000 7fd5ca9a1700 1 --2- 172.21.15.47:0/3199881596 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fd5b00a3cc0 0x7fd5b00a6180 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd5c4102a10 tx=0x7fd5b4009260 comp rx=0 tx=0).stop 2024-04-17T05:57:30.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:57:30.638+0000 7fd5ca9a1700 1 -- 172.21.15.47:0/3199881596 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5c4103550 msgr2=0x7fd5c41a0070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:57:30.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:57:30.638+0000 7fd5ca9a1700 1 --2- 172.21.15.47:0/3199881596 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5c4103550 0x7fd5c41a0070 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc007d70 tx=0x7fd5bc00a340 comp rx=0 tx=0).stop 2024-04-17T05:57:30.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:57:30.639+0000 7fd5ca9a1700 1 -- 172.21.15.47:0/3199881596 shutdown_connections 2024-04-17T05:57:30.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:57:30.639+0000 7fd5ca9a1700 1 --2- 172.21.15.47:0/3199881596 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fd5b00a3cc0 0x7fd5b00a6180 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:57:30.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:57:30.639+0000 7fd5ca9a1700 1 --2- 172.21.15.47:0/3199881596 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5c4103550 0x7fd5c41a0070 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:57:30.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:57:30.639+0000 7fd5ca9a1700 1 --2- 172.21.15.47:0/3199881596 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd5c41019b0 0x7fd5c406b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:57:30.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:57:30.639+0000 7fd5ca9a1700 1 --2- 172.21.15.47:0/3199881596 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd5c4102bb0 0x7fd5c419da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:57:30.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:57:30.639+0000 7fd5ca9a1700 1 -- 172.21.15.47:0/3199881596 >> 172.21.15.47:0/3199881596 conn(0x7fd5c40fcf50 msgr2=0x7fd5c4105ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:57:30.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:57:30.639+0000 7fd5ca9a1700 1 -- 172.21.15.47:0/3199881596 shutdown_connections 2024-04-17T05:57:30.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:57:30.639+0000 7fd5ca9a1700 1 -- 172.21.15.47:0/3199881596 wait complete. 2024-04-17T05:57:30.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:30 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:57:30.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:30 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:57:30.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:30 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:57:30.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:30 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T05:57:30.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:30 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:57:30.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:30 smithi098 ceph-mon[36516]: from='osd.9 [v2:172.21.15.47:6826/728359554,v1:172.21.15.47:6827/728359554]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-17T05:57:30.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:30 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:57:30.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:30 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:57:30.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:30 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T05:57:30.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:30 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:57:30.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:30 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:57:30.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:30 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:57:30.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:30 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T05:57:30.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:30 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:57:30.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:30 smithi047 ceph-mon[32223]: from='osd.9 [v2:172.21.15.47:6826/728359554,v1:172.21.15.47:6827/728359554]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-17T05:57:30.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:30 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:57:30.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:30 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:57:30.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:30 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T05:57:31.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:30 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:57:31.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:30 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:57:31.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:30 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:57:31.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:30 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T05:57:31.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:30 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:57:31.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:30 smithi103 ceph-mon[36243]: from='osd.9 [v2:172.21.15.47:6826/728359554,v1:172.21.15.47:6827/728359554]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-17T05:57:31.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:30 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:57:31.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:30 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:57:31.035 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:30 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T05:57:31.335 DEBUG:teuthology.orchestra.run.smithi047:osd.9> sudo journalctl -f -n 0 -u ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@osd.9.service 2024-04-17T05:57:31.337 INFO:tasks.cephadm:Deploying osd.10 on smithi098 with /dev/vg_nvme/lv_1... 2024-04-17T05:57:31.338 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 565db2d8-fc7e-11ee-bc91-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-17T05:57:31.585 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/mon.b/config 2024-04-17T05:57:31.823 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:31 smithi047 ceph-mon[32223]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-04-17T05:57:31.823 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:31 smithi047 ceph-mon[32223]: from='osd.9 [v2:172.21.15.47:6826/728359554,v1:172.21.15.47:6827/728359554]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-17T05:57:31.823 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:31 smithi047 ceph-mon[32223]: osdmap e66: 10 total, 9 up, 10 in 2024-04-17T05:57:31.823 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:31 smithi047 ceph-mon[32223]: from='osd.9 [v2:172.21.15.47:6826/728359554,v1:172.21.15.47:6827/728359554]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-04-17T05:57:31.823 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:31 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-17T05:57:31.945 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:31 smithi098 ceph-mon[36516]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-04-17T05:57:31.945 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:31 smithi098 ceph-mon[36516]: from='osd.9 [v2:172.21.15.47:6826/728359554,v1:172.21.15.47:6827/728359554]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-17T05:57:31.945 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:31 smithi098 ceph-mon[36516]: osdmap e66: 10 total, 9 up, 10 in 2024-04-17T05:57:31.945 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:31 smithi098 ceph-mon[36516]: from='osd.9 [v2:172.21.15.47:6826/728359554,v1:172.21.15.47:6827/728359554]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-04-17T05:57:31.945 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:31 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-17T05:57:32.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:31 smithi103 ceph-mon[36243]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-04-17T05:57:32.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:31 smithi103 ceph-mon[36243]: from='osd.9 [v2:172.21.15.47:6826/728359554,v1:172.21.15.47:6827/728359554]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-17T05:57:32.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:31 smithi103 ceph-mon[36243]: osdmap e66: 10 total, 9 up, 10 in 2024-04-17T05:57:32.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:31 smithi103 ceph-mon[36243]: from='osd.9 [v2:172.21.15.47:6826/728359554,v1:172.21.15.47:6827/728359554]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-04-17T05:57:32.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:31 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-17T05:57:32.389 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 05:57:32 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T05:57:32.101+0000 7fd511587700 -1 osd.9 0 waiting for initial osdmap 2024-04-17T05:57:32.389 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 05:57:32 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T05:57:32.112+0000 7fd509372700 -1 osd.9 67 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-17T05:57:32.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:32 smithi047 ceph-mon[32223]: from='osd.9 [v2:172.21.15.47:6826/728359554,v1:172.21.15.47:6827/728359554]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]': finished 2024-04-17T05:57:32.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:32 smithi047 ceph-mon[32223]: osdmap e67: 10 total, 9 up, 10 in 2024-04-17T05:57:32.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:32 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-17T05:57:32.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:32 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-17T05:57:32.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:32 smithi047 ceph-mon[32223]: from='osd.9 [v2:172.21.15.47:6826/728359554,v1:172.21.15.47:6827/728359554]' entity='osd.9' 2024-04-17T05:57:32.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:32 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-17T05:57:32.995 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:32 smithi098 ceph-mon[36516]: from='osd.9 [v2:172.21.15.47:6826/728359554,v1:172.21.15.47:6827/728359554]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]': finished 2024-04-17T05:57:32.996 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:32 smithi098 ceph-mon[36516]: osdmap e67: 10 total, 9 up, 10 in 2024-04-17T05:57:32.996 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:32 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-17T05:57:32.996 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:32 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-17T05:57:32.996 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:32 smithi098 ceph-mon[36516]: from='osd.9 [v2:172.21.15.47:6826/728359554,v1:172.21.15.47:6827/728359554]' entity='osd.9' 2024-04-17T05:57:32.996 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:32 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-17T05:57:33.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:32 smithi103 ceph-mon[36243]: from='osd.9 [v2:172.21.15.47:6826/728359554,v1:172.21.15.47:6827/728359554]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]': finished 2024-04-17T05:57:33.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:32 smithi103 ceph-mon[36243]: osdmap e67: 10 total, 9 up, 10 in 2024-04-17T05:57:33.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:32 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-17T05:57:33.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:32 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-17T05:57:33.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:32 smithi103 ceph-mon[36243]: from='osd.9 [v2:172.21.15.47:6826/728359554,v1:172.21.15.47:6827/728359554]' entity='osd.9' 2024-04-17T05:57:33.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:32 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-17T05:57:33.550 INFO:teuthology.orchestra.run.smithi098.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-17T05:57:33.550 INFO:teuthology.orchestra.run.smithi098.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-17T05:57:33.550 INFO:teuthology.orchestra.run.smithi098.stderr: stderr: 10+0 records in 2024-04-17T05:57:33.550 INFO:teuthology.orchestra.run.smithi098.stderr:10+0 records out 2024-04-17T05:57:33.550 INFO:teuthology.orchestra.run.smithi098.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0131554 s, 797 MB/s 2024-04-17T05:57:33.550 INFO:teuthology.orchestra.run.smithi098.stderr:--> Zapping successful for: 2024-04-17T05:57:33.932 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:33 smithi098 ceph-mon[36516]: purged_snaps scrub starts 2024-04-17T05:57:33.933 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:33 smithi098 ceph-mon[36516]: purged_snaps scrub ok 2024-04-17T05:57:33.933 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:33 smithi098 ceph-mon[36516]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-04-17T05:57:33.933 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:33 smithi098 ceph-mon[36516]: osd.9 [v2:172.21.15.47:6826/728359554,v1:172.21.15.47:6827/728359554] boot 2024-04-17T05:57:33.933 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:33 smithi098 ceph-mon[36516]: osdmap e68: 10 total, 10 up, 10 in 2024-04-17T05:57:33.933 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:33 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-17T05:57:34.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:33 smithi103 ceph-mon[36243]: purged_snaps scrub starts 2024-04-17T05:57:34.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:33 smithi103 ceph-mon[36243]: purged_snaps scrub ok 2024-04-17T05:57:34.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:33 smithi103 ceph-mon[36243]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-04-17T05:57:34.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:33 smithi103 ceph-mon[36243]: osd.9 [v2:172.21.15.47:6826/728359554,v1:172.21.15.47:6827/728359554] boot 2024-04-17T05:57:34.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:33 smithi103 ceph-mon[36243]: osdmap e68: 10 total, 10 up, 10 in 2024-04-17T05:57:34.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:33 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-17T05:57:34.110 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:33 smithi047 ceph-mon[32223]: purged_snaps scrub starts 2024-04-17T05:57:34.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:33 smithi047 ceph-mon[32223]: purged_snaps scrub ok 2024-04-17T05:57:34.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:33 smithi047 ceph-mon[32223]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-04-17T05:57:34.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:33 smithi047 ceph-mon[32223]: osd.9 [v2:172.21.15.47:6826/728359554,v1:172.21.15.47:6827/728359554] boot 2024-04-17T05:57:34.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:33 smithi047 ceph-mon[32223]: osdmap e68: 10 total, 10 up, 10 in 2024-04-17T05:57:34.111 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:33 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-17T05:57:34.224 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 565db2d8-fc7e-11ee-bc91-c7b262605968 -- ceph orch daemon add osd smithi098:vg_nvme/lv_1 2024-04-17T05:57:34.466 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/mon.b/config 2024-04-17T05:57:35.264 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:35 smithi098 ceph-mon[36516]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-17T05:57:35.265 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:35 smithi098 ceph-mon[36516]: osdmap e69: 10 total, 10 up, 10 in 2024-04-17T05:57:35.265 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:35 smithi098 ceph-mon[36516]: Detected new or changed devices on smithi047 2024-04-17T05:57:35.265 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:35 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:57:35.265 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:35 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:57:35.265 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:35 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-17T05:57:35.265 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:35 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-17T05:57:35.265 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:35 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-17T05:57:35.265 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:35 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-17T05:57:35.265 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:35 smithi098 ceph-mon[36516]: Adjusting osd_memory_target on smithi047 to 4273M 2024-04-17T05:57:35.265 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:35 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:57:35.265 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:35 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:57:35.265 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:35 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T05:57:35.265 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:35 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:57:35.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:35 smithi047 ceph-mon[32223]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-17T05:57:35.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:35 smithi047 ceph-mon[32223]: osdmap e69: 10 total, 10 up, 10 in 2024-04-17T05:57:35.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:35 smithi047 ceph-mon[32223]: Detected new or changed devices on smithi047 2024-04-17T05:57:35.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:35 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:57:35.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:35 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:57:35.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:35 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-17T05:57:35.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:35 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-17T05:57:35.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:35 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-17T05:57:35.390 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:35 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-17T05:57:35.390 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:35 smithi047 ceph-mon[32223]: Adjusting osd_memory_target on smithi047 to 4273M 2024-04-17T05:57:35.390 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:35 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:57:35.390 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:35 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:57:35.390 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:35 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T05:57:35.390 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:35 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:57:35.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:35 smithi103 ceph-mon[36243]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-17T05:57:35.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:35 smithi103 ceph-mon[36243]: osdmap e69: 10 total, 10 up, 10 in 2024-04-17T05:57:35.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:35 smithi103 ceph-mon[36243]: Detected new or changed devices on smithi047 2024-04-17T05:57:35.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:35 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:57:35.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:35 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:57:35.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:35 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-17T05:57:35.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:35 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-17T05:57:35.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:35 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-17T05:57:35.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:35 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-17T05:57:35.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:35 smithi103 ceph-mon[36243]: Adjusting osd_memory_target on smithi047 to 4273M 2024-04-17T05:57:35.535 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:35 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:57:35.535 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:35 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:57:35.535 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:35 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T05:57:35.535 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:35 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:57:36.093 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:57:36.093+0000 7f1f367aa700 1 -- 172.21.15.98:0/187432803 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1f30102bb0 msgr2=0x7f1f30103010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:57:36.094 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:57:36.093+0000 7f1f367aa700 1 --2- 172.21.15.98:0/187432803 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1f30102bb0 0x7f1f30103010 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1f200099c0 tx=0x7f1f20009cd0 comp rx=0 tx=0).stop 2024-04-17T05:57:36.094 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:57:36.093+0000 7f1f367aa700 1 -- 172.21.15.98:0/187432803 shutdown_connections 2024-04-17T05:57:36.094 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:57:36.093+0000 7f1f367aa700 1 --2- 172.21.15.98:0/187432803 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f30103550 0x7f1f30109e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:57:36.094 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:57:36.093+0000 7f1f367aa700 1 --2- 172.21.15.98:0/187432803 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1f301019b0 0x7f1f30101dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:57:36.094 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:57:36.093+0000 7f1f367aa700 1 --2- 172.21.15.98:0/187432803 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1f30102bb0 0x7f1f30103010 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:57:36.094 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:57:36.093+0000 7f1f367aa700 1 -- 172.21.15.98:0/187432803 >> 172.21.15.98:0/187432803 conn(0x7f1f300fcf50 msgr2=0x7f1f300ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:57:36.094 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:57:36.093+0000 7f1f367aa700 1 -- 172.21.15.98:0/187432803 shutdown_connections 2024-04-17T05:57:36.094 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:57:36.093+0000 7f1f367aa700 1 -- 172.21.15.98:0/187432803 wait complete. 2024-04-17T05:57:36.094 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:57:36.094+0000 7f1f367aa700 1 Processor -- start 2024-04-17T05:57:36.094 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:57:36.094+0000 7f1f367aa700 1 -- start start 2024-04-17T05:57:36.094 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:57:36.094+0000 7f1f367aa700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1f301019b0 0x7f1f30198dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:57:36.094 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:57:36.094+0000 7f1f367aa700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1f30102bb0 0x7f1f30199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:57:36.094 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:57:36.094+0000 7f1f367aa700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f30103550 0x7f1f301a02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:57:36.095 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:57:36.094+0000 7f1f367aa700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f1f3010e710 con 0x7f1f30103550 2024-04-17T05:57:36.095 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:57:36.094+0000 7f1f367aa700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1f3010e590 con 0x7f1f30102bb0 2024-04-17T05:57:36.095 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:57:36.094+0000 7f1f367aa700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1f3010e890 con 0x7f1f301019b0 2024-04-17T05:57:36.095 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:57:36.094+0000 7f1f2f7fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1f30102bb0 0x7f1f30199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:57:36.095 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:57:36.094+0000 7f1f2f7fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1f30102bb0 0x7f1f30199310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:55386/0 (socket says 172.21.15.98:55386) 2024-04-17T05:57:36.095 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:57:36.094+0000 7f1f2f7fe700 1 -- 172.21.15.98:0/1980535439 learned_addr learned my addr 172.21.15.98:0/1980535439 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-04-17T05:57:36.095 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:57:36.095+0000 7f1f34d47700 1 --2- 172.21.15.98:0/1980535439 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f30103550 0x7f1f301a02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:57:36.095 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:57:36.095+0000 7f1f2ffff700 1 --2- 172.21.15.98:0/1980535439 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1f301019b0 0x7f1f30198dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:57:36.095 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:57:36.095+0000 7f1f2f7fe700 1 -- 172.21.15.98:0/1980535439 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1f301019b0 msgr2=0x7f1f30198dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:57:36.095 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:57:36.095+0000 7f1f2f7fe700 1 --2- 172.21.15.98:0/1980535439 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1f301019b0 0x7f1f30198dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:57:36.095 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:57:36.095+0000 7f1f2f7fe700 1 -- 172.21.15.98:0/1980535439 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f30103550 msgr2=0x7f1f301a02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:57:36.095 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:57:36.095+0000 7f1f2f7fe700 1 --2- 172.21.15.98:0/1980535439 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f30103550 0x7f1f301a02b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:57:36.095 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:57:36.095+0000 7f1f2f7fe700 1 -- 172.21.15.98:0/1980535439 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f301a0980 con 0x7f1f30102bb0 2024-04-17T05:57:36.096 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:57:36.095+0000 7f1f34d47700 1 --2- 172.21.15.98:0/1980535439 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f30103550 0x7f1f301a02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T05:57:36.096 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:57:36.095+0000 7f1f2f7fe700 1 --2- 172.21.15.98:0/1980535439 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1f30102bb0 0x7f1f30199310 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1f20003890 tx=0x7f1f20003730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:57:36.096 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:57:36.095+0000 7f1f2d7fa700 1 -- 172.21.15.98:0/1980535439 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f20027070 con 0x7f1f30102bb0 2024-04-17T05:57:36.096 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:57:36.095+0000 7f1f2d7fa700 1 -- 172.21.15.98:0/1980535439 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f20022470 con 0x7f1f30102bb0 2024-04-17T05:57:36.096 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:57:36.095+0000 7f1f367aa700 1 -- 172.21.15.98:0/1980535439 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f301a0c10 con 0x7f1f30102bb0 2024-04-17T05:57:36.096 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:57:36.095+0000 7f1f2d7fa700 1 -- 172.21.15.98:0/1980535439 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f2000f6c0 con 0x7f1f30102bb0 2024-04-17T05:57:36.096 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:57:36.095+0000 7f1f367aa700 1 -- 172.21.15.98:0/1980535439 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f301a1060 con 0x7f1f30102bb0 2024-04-17T05:57:36.097 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:57:36.096+0000 7f1f367aa700 1 -- 172.21.15.98:0/1980535439 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f3002df90 con 0x7f1f30102bb0 2024-04-17T05:57:36.097 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:57:36.097+0000 7f1f2d7fa700 1 -- 172.21.15.98:0/1980535439 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f1f2002a070 con 0x7f1f30102bb0 2024-04-17T05:57:36.097 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:57:36.098+0000 7f1f2d7fa700 1 --2- 172.21.15.98:0/1980535439 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f1f180a3c40 0x7f1f180a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:57:36.097 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:57:36.098+0000 7f1f2d7fa700 1 -- 172.21.15.98:0/1980535439 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6261+0+0 (secure 0 0 0) 0x7f1f200d0d00 con 0x7f1f30102bb0 2024-04-17T05:57:36.097 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:57:36.098+0000 7f1f2ffff700 1 --2- 172.21.15.98:0/1980535439 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f1f180a3c40 0x7f1f180a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:57:36.097 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:57:36.099+0000 7f1f2ffff700 1 --2- 172.21.15.98:0/1980535439 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f1f180a3c40 0x7f1f180a6100 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1f24006fd0 tx=0x7f1f24008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:57:36.100 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:57:36.102+0000 7f1f2d7fa700 1 -- 172.21.15.98:0/1980535439 <== mon.2 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f1f2005eff0 con 0x7f1f30102bb0 2024-04-17T05:57:36.313 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:57:36.314+0000 7f1f367aa700 1 -- 172.21.15.98:0/1980535439 --> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f1f30062e90 con 0x7f1f180a3c40 2024-04-17T05:57:37.251 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:37 smithi098 ceph-mon[36516]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-17T05:57:37.251 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:37 smithi098 ceph-mon[36516]: from='client.24341 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T05:57:37.251 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:37 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-17T05:57:37.251 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:37 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-17T05:57:37.251 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:37 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:57:37.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:37 smithi047 ceph-mon[32223]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-17T05:57:37.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:37 smithi047 ceph-mon[32223]: from='client.24341 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T05:57:37.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:37 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-17T05:57:37.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:37 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-17T05:57:37.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:37 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:57:37.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:37 smithi103 ceph-mon[36243]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-17T05:57:37.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:37 smithi103 ceph-mon[36243]: from='client.24341 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T05:57:37.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:37 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-17T05:57:37.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:37 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-17T05:57:37.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:37 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:57:39.382 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:39 smithi098 ceph-mon[36516]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-17T05:57:39.388 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:39 smithi047 ceph-mon[32223]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-17T05:57:39.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:39 smithi103 ceph-mon[36243]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-17T05:57:40.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:40 smithi098 ceph-mon[36516]: from='client.? 172.21.15.98:0/1146657591' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9565bcc3-62ee-48eb-a07a-03a81e0cc8e5"}]: dispatch 2024-04-17T05:57:40.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:40 smithi098 ceph-mon[36516]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9565bcc3-62ee-48eb-a07a-03a81e0cc8e5"}]: dispatch 2024-04-17T05:57:40.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:40 smithi098 ceph-mon[36516]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9565bcc3-62ee-48eb-a07a-03a81e0cc8e5"}]': finished 2024-04-17T05:57:40.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:40 smithi098 ceph-mon[36516]: osdmap e70: 11 total, 10 up, 11 in 2024-04-17T05:57:40.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:40 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-17T05:57:40.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:40 smithi098 ceph-mon[36516]: from='client.? 172.21.15.98:0/3552880858' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-17T05:57:40.388 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:40 smithi047 ceph-mon[32223]: from='client.? 172.21.15.98:0/1146657591' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9565bcc3-62ee-48eb-a07a-03a81e0cc8e5"}]: dispatch 2024-04-17T05:57:40.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:40 smithi047 ceph-mon[32223]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9565bcc3-62ee-48eb-a07a-03a81e0cc8e5"}]: dispatch 2024-04-17T05:57:40.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:40 smithi047 ceph-mon[32223]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9565bcc3-62ee-48eb-a07a-03a81e0cc8e5"}]': finished 2024-04-17T05:57:40.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:40 smithi047 ceph-mon[32223]: osdmap e70: 11 total, 10 up, 11 in 2024-04-17T05:57:40.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:40 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-17T05:57:40.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:40 smithi047 ceph-mon[32223]: from='client.? 172.21.15.98:0/3552880858' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-17T05:57:40.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:40 smithi103 ceph-mon[36243]: from='client.? 172.21.15.98:0/1146657591' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9565bcc3-62ee-48eb-a07a-03a81e0cc8e5"}]: dispatch 2024-04-17T05:57:40.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:40 smithi103 ceph-mon[36243]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9565bcc3-62ee-48eb-a07a-03a81e0cc8e5"}]: dispatch 2024-04-17T05:57:40.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:40 smithi103 ceph-mon[36243]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9565bcc3-62ee-48eb-a07a-03a81e0cc8e5"}]': finished 2024-04-17T05:57:40.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:40 smithi103 ceph-mon[36243]: osdmap e70: 11 total, 10 up, 11 in 2024-04-17T05:57:40.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:40 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-17T05:57:40.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:40 smithi103 ceph-mon[36243]: from='client.? 172.21.15.98:0/3552880858' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-17T05:57:41.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:41 smithi103 ceph-mon[36243]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-17T05:57:41.632 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:41 smithi098 ceph-mon[36516]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-17T05:57:41.638 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:41 smithi047 ceph-mon[32223]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-17T05:57:43.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:43 smithi103 ceph-mon[36243]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-17T05:57:43.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:43 smithi098 ceph-mon[36516]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-17T05:57:43.638 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:43 smithi047 ceph-mon[32223]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-17T05:57:45.441 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:45 smithi047 ceph-mon[32223]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-17T05:57:45.519 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:45 smithi098 ceph-mon[36516]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-17T05:57:45.533 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:45 smithi103 ceph-mon[36243]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-17T05:57:47.495 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:47 smithi098 ceph-mon[36516]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-17T05:57:47.495 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:47 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-17T05:57:47.495 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:47 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:57:47.495 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:47 smithi098 ceph-mon[36516]: Deploying daemon osd.10 on smithi098 2024-04-17T05:57:47.533 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:47 smithi103 ceph-mon[36243]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-17T05:57:47.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:47 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-17T05:57:47.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:47 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:57:47.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:47 smithi103 ceph-mon[36243]: Deploying daemon osd.10 on smithi098 2024-04-17T05:57:47.638 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:47 smithi047 ceph-mon[32223]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-17T05:57:47.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:47 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-17T05:57:47.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:47 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:57:47.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:47 smithi047 ceph-mon[32223]: Deploying daemon osd.10 on smithi098 2024-04-17T05:57:49.533 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:49 smithi098 ceph-mon[36516]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-17T05:57:49.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:49 smithi103 ceph-mon[36243]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-17T05:57:49.638 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:49 smithi047 ceph-mon[32223]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-17T05:57:51.292 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:51 smithi098 ceph-mon[36516]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-17T05:57:51.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:51 smithi103 ceph-mon[36243]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-17T05:57:51.638 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:51 smithi047 ceph-mon[32223]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-17T05:57:53.533 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:53 smithi103 ceph-mon[36243]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-17T05:57:53.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:53 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:57:53.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:53 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:57:53.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:53 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T05:57:53.632 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:53 smithi098 ceph-mon[36516]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-17T05:57:53.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:53 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:57:53.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:53 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:57:53.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:53 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T05:57:53.638 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:53 smithi047 ceph-mon[32223]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-17T05:57:53.638 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:53 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:57:53.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:53 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:57:53.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:53 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T05:57:55.301 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:55 smithi098 ceph-mon[36516]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-17T05:57:55.533 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:55 smithi103 ceph-mon[36243]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-17T05:57:55.632 INFO:teuthology.orchestra.run.smithi098.stdout:Created osd(s) 10 on host 'smithi098' 2024-04-17T05:57:55.633 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:57:55.634+0000 7f1f2d7fa700 1 -- 172.21.15.98:0/1980535439 <== mgr.14152 v2:172.21.15.47:6800/3157902806 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f1f30062e90 con 0x7f1f180a3c40 2024-04-17T05:57:55.635 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:57:55.637+0000 7f1f367aa700 1 -- 172.21.15.98:0/1980535439 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f1f180a3c40 msgr2=0x7f1f180a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:57:55.635 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:57:55.637+0000 7f1f367aa700 1 --2- 172.21.15.98:0/1980535439 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f1f180a3c40 0x7f1f180a6100 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1f24006fd0 tx=0x7f1f24008040 comp rx=0 tx=0).stop 2024-04-17T05:57:55.635 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:57:55.637+0000 7f1f367aa700 1 -- 172.21.15.98:0/1980535439 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1f30102bb0 msgr2=0x7f1f30199310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:57:55.635 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:57:55.637+0000 7f1f367aa700 1 --2- 172.21.15.98:0/1980535439 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1f30102bb0 0x7f1f30199310 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1f20003890 tx=0x7f1f20003730 comp rx=0 tx=0).stop 2024-04-17T05:57:55.635 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:57:55.637+0000 7f1f367aa700 1 -- 172.21.15.98:0/1980535439 shutdown_connections 2024-04-17T05:57:55.635 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:57:55.637+0000 7f1f367aa700 1 --2- 172.21.15.98:0/1980535439 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f1f180a3c40 0x7f1f180a6100 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:57:55.635 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:57:55.637+0000 7f1f367aa700 1 --2- 172.21.15.98:0/1980535439 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f30103550 0x7f1f301a02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:57:55.635 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:57:55.637+0000 7f1f367aa700 1 --2- 172.21.15.98:0/1980535439 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1f301019b0 0x7f1f30198dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:57:55.635 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:57:55.637+0000 7f1f367aa700 1 --2- 172.21.15.98:0/1980535439 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1f30102bb0 0x7f1f30199310 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:57:55.635 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:57:55.637+0000 7f1f367aa700 1 -- 172.21.15.98:0/1980535439 >> 172.21.15.98:0/1980535439 conn(0x7f1f300fcf50 msgr2=0x7f1f30105eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:57:55.635 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:57:55.637+0000 7f1f367aa700 1 -- 172.21.15.98:0/1980535439 shutdown_connections 2024-04-17T05:57:55.635 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T05:57:55.637+0000 7f1f367aa700 1 -- 172.21.15.98:0/1980535439 wait complete. 2024-04-17T05:57:55.638 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:55 smithi047 ceph-mon[32223]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-17T05:57:56.283 DEBUG:teuthology.orchestra.run.smithi098:osd.10> sudo journalctl -f -n 0 -u ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@osd.10.service 2024-04-17T05:57:56.286 INFO:tasks.cephadm:Deploying osd.11 on smithi103 with /dev/vg_nvme/lv_1... 2024-04-17T05:57:56.286 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 565db2d8-fc7e-11ee-bc91-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-17T05:57:56.525 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/mon.c/config 2024-04-17T05:57:56.873 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:56 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:57:56.873 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:56 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:57:56.873 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:56 smithi103 ceph-mon[36243]: from='osd.10 [v2:172.21.15.98:6824/468205689,v1:172.21.15.98:6825/468205689]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-17T05:57:56.873 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:56 smithi103 ceph-mon[36243]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-17T05:57:56.873 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:56 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:57:56.873 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:56 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:57:56.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:56 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:57:56.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:56 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:57:56.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:56 smithi098 ceph-mon[36516]: from='osd.10 [v2:172.21.15.98:6824/468205689,v1:172.21.15.98:6825/468205689]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-17T05:57:56.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:56 smithi098 ceph-mon[36516]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-17T05:57:56.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:56 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:57:56.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:56 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:57:56.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:56 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:57:56.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:56 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:57:56.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:56 smithi047 ceph-mon[32223]: from='osd.10 [v2:172.21.15.98:6824/468205689,v1:172.21.15.98:6825/468205689]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-17T05:57:56.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:56 smithi047 ceph-mon[32223]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-17T05:57:56.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:56 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:57:56.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:56 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:57:57.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:57 smithi098 ceph-mon[36516]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-17T05:57:57.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:57 smithi098 ceph-mon[36516]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-17T05:57:57.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:57 smithi098 ceph-mon[36516]: osdmap e71: 11 total, 10 up, 11 in 2024-04-17T05:57:57.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:57 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-17T05:57:57.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:57 smithi098 ceph-mon[36516]: from='osd.10 [v2:172.21.15.98:6824/468205689,v1:172.21.15.98:6825/468205689]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2024-04-17T05:57:57.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:57 smithi098 ceph-mon[36516]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2024-04-17T05:57:57.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:57 smithi047 ceph-mon[32223]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-17T05:57:57.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:57 smithi047 ceph-mon[32223]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-17T05:57:57.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:57 smithi047 ceph-mon[32223]: osdmap e71: 11 total, 10 up, 11 in 2024-04-17T05:57:57.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:57 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-17T05:57:57.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:57 smithi047 ceph-mon[32223]: from='osd.10 [v2:172.21.15.98:6824/468205689,v1:172.21.15.98:6825/468205689]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2024-04-17T05:57:57.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:57 smithi047 ceph-mon[32223]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2024-04-17T05:57:57.919 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:57 smithi103 ceph-mon[36243]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-17T05:57:57.919 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:57 smithi103 ceph-mon[36243]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-17T05:57:57.919 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:57 smithi103 ceph-mon[36243]: osdmap e71: 11 total, 10 up, 11 in 2024-04-17T05:57:57.919 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:57 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-17T05:57:57.919 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:57 smithi103 ceph-mon[36243]: from='osd.10 [v2:172.21.15.98:6824/468205689,v1:172.21.15.98:6825/468205689]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2024-04-17T05:57:57.919 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:57 smithi103 ceph-mon[36243]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2024-04-17T05:57:58.369 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 05:57:58 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T05:57:58.077+0000 7fdb3937b700 -1 osd.10 0 waiting for initial osdmap 2024-04-17T05:57:58.369 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 05:57:58 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T05:57:58.090+0000 7fdb31166700 -1 osd.10 72 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-17T05:57:58.654 INFO:teuthology.orchestra.run.smithi103.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-17T05:57:58.655 INFO:teuthology.orchestra.run.smithi103.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-17T05:57:58.655 INFO:teuthology.orchestra.run.smithi103.stderr: stderr: 10+0 records in 2024-04-17T05:57:58.655 INFO:teuthology.orchestra.run.smithi103.stderr:10+0 records out 2024-04-17T05:57:58.655 INFO:teuthology.orchestra.run.smithi103.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0115057 s, 911 MB/s 2024-04-17T05:57:58.655 INFO:teuthology.orchestra.run.smithi103.stderr:--> Zapping successful for: 2024-04-17T05:57:58.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:58 smithi047 ceph-mon[32223]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]': finished 2024-04-17T05:57:58.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:58 smithi047 ceph-mon[32223]: osdmap e72: 11 total, 10 up, 11 in 2024-04-17T05:57:58.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:58 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-17T05:57:58.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:58 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-17T05:57:58.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:58 smithi047 ceph-mon[32223]: from='osd.10 ' entity='osd.10' 2024-04-17T05:57:58.978 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:58 smithi103 ceph-mon[36243]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]': finished 2024-04-17T05:57:58.978 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:58 smithi103 ceph-mon[36243]: osdmap e72: 11 total, 10 up, 11 in 2024-04-17T05:57:58.978 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:58 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-17T05:57:58.978 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:58 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-17T05:57:58.978 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:58 smithi103 ceph-mon[36243]: from='osd.10 ' entity='osd.10' 2024-04-17T05:57:59.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:58 smithi098 ceph-mon[36516]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]': finished 2024-04-17T05:57:59.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:58 smithi098 ceph-mon[36516]: osdmap e72: 11 total, 10 up, 11 in 2024-04-17T05:57:59.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:58 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-17T05:57:59.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:58 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-17T05:57:59.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:58 smithi098 ceph-mon[36516]: from='osd.10 ' entity='osd.10' 2024-04-17T05:57:59.436 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 565db2d8-fc7e-11ee-bc91-c7b262605968 -- ceph orch daemon add osd smithi103:vg_nvme/lv_1 2024-04-17T05:57:59.676 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/mon.c/config 2024-04-17T05:57:59.783 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:59 smithi103 ceph-mon[36243]: purged_snaps scrub starts 2024-04-17T05:57:59.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:59 smithi103 ceph-mon[36243]: purged_snaps scrub ok 2024-04-17T05:57:59.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:59 smithi103 ceph-mon[36243]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-17T05:57:59.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:59 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-17T05:57:59.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:59 smithi103 ceph-mon[36243]: osd.10 [v2:172.21.15.98:6824/468205689,v1:172.21.15.98:6825/468205689] boot 2024-04-17T05:57:59.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:59 smithi103 ceph-mon[36243]: osdmap e73: 11 total, 11 up, 11 in 2024-04-17T05:57:59.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:59 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-17T05:57:59.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:57:59 smithi103 ceph-mon[36243]: osdmap e74: 11 total, 11 up, 11 in 2024-04-17T05:57:59.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:59 smithi047 ceph-mon[32223]: purged_snaps scrub starts 2024-04-17T05:57:59.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:59 smithi047 ceph-mon[32223]: purged_snaps scrub ok 2024-04-17T05:57:59.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:59 smithi047 ceph-mon[32223]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-17T05:57:59.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:59 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-17T05:57:59.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:59 smithi047 ceph-mon[32223]: osd.10 [v2:172.21.15.98:6824/468205689,v1:172.21.15.98:6825/468205689] boot 2024-04-17T05:57:59.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:59 smithi047 ceph-mon[32223]: osdmap e73: 11 total, 11 up, 11 in 2024-04-17T05:57:59.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:59 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-17T05:57:59.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:57:59 smithi047 ceph-mon[32223]: osdmap e74: 11 total, 11 up, 11 in 2024-04-17T05:58:00.058 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:59 smithi098 ceph-mon[36516]: purged_snaps scrub starts 2024-04-17T05:58:00.059 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:59 smithi098 ceph-mon[36516]: purged_snaps scrub ok 2024-04-17T05:58:00.059 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:59 smithi098 ceph-mon[36516]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-17T05:58:00.059 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:59 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-17T05:58:00.059 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:59 smithi098 ceph-mon[36516]: osd.10 [v2:172.21.15.98:6824/468205689,v1:172.21.15.98:6825/468205689] boot 2024-04-17T05:58:00.059 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:59 smithi098 ceph-mon[36516]: osdmap e73: 11 total, 11 up, 11 in 2024-04-17T05:58:00.059 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:59 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-17T05:58:00.059 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:57:59 smithi098 ceph-mon[36516]: osdmap e74: 11 total, 11 up, 11 in 2024-04-17T05:58:01.194 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:01 smithi103 ceph-mon[36243]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-17T05:58:01.194 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:01 smithi103 ceph-mon[36243]: Detected new or changed devices on smithi098 2024-04-17T05:58:01.194 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:01 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:01.194 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:01 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:01.194 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:01 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-17T05:58:01.194 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:01 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-17T05:58:01.194 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:01 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-17T05:58:01.194 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:01 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-17T05:58:01.195 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:01 smithi103 ceph-mon[36243]: Adjusting osd_memory_target on smithi098 to 4270M 2024-04-17T05:58:01.195 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:01 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:01.195 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:01 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:58:01.195 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:01 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T05:58:01.195 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:01 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:01.195 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:01 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T05:58:01.195 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:01 smithi103 ceph-mon[36243]: osdmap e75: 11 total, 11 up, 11 in 2024-04-17T05:58:01.195 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:01 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:58:01.195 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:01 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T05:58:01.195 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:01 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:01.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:01 smithi098 ceph-mon[36516]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-17T05:58:01.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:01 smithi098 ceph-mon[36516]: Detected new or changed devices on smithi098 2024-04-17T05:58:01.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:01 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:01.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:01 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:01.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:01 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-17T05:58:01.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:01 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-17T05:58:01.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:01 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-17T05:58:01.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:01 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-17T05:58:01.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:01 smithi098 ceph-mon[36516]: Adjusting osd_memory_target on smithi098 to 4270M 2024-04-17T05:58:01.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:01 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:01.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:01 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:58:01.384 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:01 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T05:58:01.384 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:01 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:01.384 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:01 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T05:58:01.384 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:01 smithi098 ceph-mon[36516]: osdmap e75: 11 total, 11 up, 11 in 2024-04-17T05:58:01.384 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:01 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:58:01.384 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:01 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T05:58:01.384 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:01 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:01.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:01 smithi047 ceph-mon[32223]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-17T05:58:01.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:01 smithi047 ceph-mon[32223]: Detected new or changed devices on smithi098 2024-04-17T05:58:01.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:01 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:01.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:01 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:01.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:01 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-17T05:58:01.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:01 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-17T05:58:01.390 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:01 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-17T05:58:01.390 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:01 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-17T05:58:01.390 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:01 smithi047 ceph-mon[32223]: Adjusting osd_memory_target on smithi098 to 4270M 2024-04-17T05:58:01.390 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:01 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:01.390 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:01 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:58:01.390 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:01 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T05:58:01.390 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:01 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:01.390 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:01 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T05:58:01.390 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:01 smithi047 ceph-mon[32223]: osdmap e75: 11 total, 11 up, 11 in 2024-04-17T05:58:01.390 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:01 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:58:01.390 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:01 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T05:58:01.390 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:01 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:01.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:58:01.431+0000 7fe1c472a700 1 -- 172.21.15.103:0/3117088575 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe1bc1019b0 msgr2=0x7fe1bc101dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:58:01.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:58:01.431+0000 7fe1c472a700 1 --2- 172.21.15.103:0/3117088575 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe1bc1019b0 0x7fe1bc101dd0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe1b8009990 tx=0x7fe1b8009ca0 comp rx=0 tx=0).stop 2024-04-17T05:58:01.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:58:01.431+0000 7fe1c472a700 1 -- 172.21.15.103:0/3117088575 shutdown_connections 2024-04-17T05:58:01.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:58:01.431+0000 7fe1c472a700 1 --2- 172.21.15.103:0/3117088575 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe1bc103550 0x7fe1bc109e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:01.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:58:01.431+0000 7fe1c472a700 1 --2- 172.21.15.103:0/3117088575 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe1bc1019b0 0x7fe1bc101dd0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:01.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:58:01.431+0000 7fe1c472a700 1 --2- 172.21.15.103:0/3117088575 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe1bc102bb0 0x7fe1bc103010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:01.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:58:01.431+0000 7fe1c472a700 1 -- 172.21.15.103:0/3117088575 >> 172.21.15.103:0/3117088575 conn(0x7fe1bc0fcf50 msgr2=0x7fe1bc0ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:58:01.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:58:01.431+0000 7fe1c472a700 1 -- 172.21.15.103:0/3117088575 shutdown_connections 2024-04-17T05:58:01.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:58:01.431+0000 7fe1c472a700 1 -- 172.21.15.103:0/3117088575 wait complete. 2024-04-17T05:58:01.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:58:01.431+0000 7fe1c472a700 1 Processor -- start 2024-04-17T05:58:01.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:58:01.432+0000 7fe1c472a700 1 -- start start 2024-04-17T05:58:01.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:58:01.432+0000 7fe1c472a700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe1bc1019b0 0x7fe1bc105ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:58:01.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:58:01.432+0000 7fe1c472a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe1bc102bb0 0x7fe1bc104180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:58:01.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:58:01.432+0000 7fe1c472a700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe1bc103550 0x7fe1bc1046c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:58:01.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:58:01.432+0000 7fe1c472a700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fe1bc10d590 con 0x7fe1bc102bb0 2024-04-17T05:58:01.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:58:01.432+0000 7fe1c472a700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe1bc10d410 con 0x7fe1bc1019b0 2024-04-17T05:58:01.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:58:01.432+0000 7fe1c472a700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe1bc10d710 con 0x7fe1bc103550 2024-04-17T05:58:01.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:58:01.432+0000 7fe1c2cc7700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe1bc103550 0x7fe1bc1046c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:58:01.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:58:01.432+0000 7fe1c2cc7700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe1bc103550 0x7fe1bc1046c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54560/0 (socket says 172.21.15.103:54560) 2024-04-17T05:58:01.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:58:01.432+0000 7fe1c1cc5700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe1bc102bb0 0x7fe1bc104180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:58:01.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:58:01.432+0000 7fe1c2cc7700 1 -- 172.21.15.103:0/1576456453 learned_addr learned my addr 172.21.15.103:0/1576456453 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-04-17T05:58:01.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:58:01.432+0000 7fe1c24c6700 1 --2- 172.21.15.103:0/1576456453 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe1bc1019b0 0x7fe1bc105ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:58:01.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:58:01.432+0000 7fe1c2cc7700 1 -- 172.21.15.103:0/1576456453 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe1bc1019b0 msgr2=0x7fe1bc105ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:58:01.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:58:01.432+0000 7fe1c2cc7700 1 --2- 172.21.15.103:0/1576456453 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe1bc1019b0 0x7fe1bc105ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:01.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:58:01.432+0000 7fe1c2cc7700 1 -- 172.21.15.103:0/1576456453 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe1bc102bb0 msgr2=0x7fe1bc104180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:58:01.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:58:01.432+0000 7fe1c2cc7700 1 --2- 172.21.15.103:0/1576456453 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe1bc102bb0 0x7fe1bc104180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:01.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:58:01.432+0000 7fe1c2cc7700 1 -- 172.21.15.103:0/1576456453 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1bc104f90 con 0x7fe1bc103550 2024-04-17T05:58:01.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:58:01.432+0000 7fe1c1cc5700 1 --2- 172.21.15.103:0/1576456453 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe1bc102bb0 0x7fe1bc104180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T05:58:01.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:58:01.432+0000 7fe1c2cc7700 1 --2- 172.21.15.103:0/1576456453 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe1bc103550 0x7fe1bc1046c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe1b400ada0 tx=0x7fe1b400c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:58:01.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:58:01.432+0000 7fe1b37fe700 1 -- 172.21.15.103:0/1576456453 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b4017020 con 0x7fe1bc103550 2024-04-17T05:58:01.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:58:01.433+0000 7fe1b37fe700 1 -- 172.21.15.103:0/1576456453 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe1b400a0d0 con 0x7fe1bc103550 2024-04-17T05:58:01.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:58:01.433+0000 7fe1c472a700 1 -- 172.21.15.103:0/1576456453 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1bc1a2ee0 con 0x7fe1bc103550 2024-04-17T05:58:01.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:58:01.433+0000 7fe1b37fe700 1 -- 172.21.15.103:0/1576456453 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b4005420 con 0x7fe1bc103550 2024-04-17T05:58:01.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:58:01.433+0000 7fe1c472a700 1 -- 172.21.15.103:0/1576456453 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1bc1a32e0 con 0x7fe1bc103550 2024-04-17T05:58:01.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:58:01.434+0000 7fe1c472a700 1 -- 172.21.15.103:0/1576456453 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1bc069770 con 0x7fe1bc103550 2024-04-17T05:58:01.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:58:01.435+0000 7fe1b37fe700 1 -- 172.21.15.103:0/1576456453 <== mon.1 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe1b400c980 con 0x7fe1bc103550 2024-04-17T05:58:01.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:58:01.435+0000 7fe1b37fe700 1 --2- 172.21.15.103:0/1576456453 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fe1a80a3c40 0x7fe1a80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:58:01.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:58:01.435+0000 7fe1b37fe700 1 -- 172.21.15.103:0/1576456453 <== mon.1 v2:172.21.15.103:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6723+0+0 (secure 0 0 0) 0x7fe1b40cfa30 con 0x7fe1bc103550 2024-04-17T05:58:01.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:58:01.436+0000 7fe1c24c6700 1 --2- 172.21.15.103:0/1576456453 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fe1a80a3c40 0x7fe1a80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:58:01.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:58:01.436+0000 7fe1c24c6700 1 --2- 172.21.15.103:0/1576456453 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fe1a80a3c40 0x7fe1a80a6100 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe1b800b2e0 tx=0x7fe1b801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:58:01.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:58:01.439+0000 7fe1b37fe700 1 -- 172.21.15.103:0/1576456453 <== mon.1 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fe1bc069770 con 0x7fe1bc103550 2024-04-17T05:58:01.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:58:01.663+0000 7fe1c472a700 1 -- 172.21.15.103:0/1576456453 --> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fe1bc06ae40 con 0x7fe1a80a3c40 2024-04-17T05:58:02.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:02 smithi098 ceph-mon[36516]: osdmap e76: 11 total, 11 up, 11 in 2024-04-17T05:58:02.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:02 smithi098 ceph-mon[36516]: from='client.24337 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T05:58:02.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:02 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-17T05:58:02.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:02 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-17T05:58:02.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:02 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:58:02.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:02 smithi047 ceph-mon[32223]: osdmap e76: 11 total, 11 up, 11 in 2024-04-17T05:58:02.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:02 smithi047 ceph-mon[32223]: from='client.24337 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T05:58:02.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:02 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-17T05:58:02.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:02 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-17T05:58:02.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:02 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:58:02.934 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:02 smithi103 ceph-mon[36243]: osdmap e76: 11 total, 11 up, 11 in 2024-04-17T05:58:02.934 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:02 smithi103 ceph-mon[36243]: from='client.24337 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T05:58:02.934 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:02 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-17T05:58:02.934 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:02 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-17T05:58:02.934 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:02 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:58:03.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:03 smithi098 ceph-mon[36516]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-17T05:58:03.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:03 smithi098 ceph-mon[36516]: osdmap e77: 11 total, 11 up, 11 in 2024-04-17T05:58:03.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:03 smithi047 ceph-mon[32223]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-17T05:58:03.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:03 smithi047 ceph-mon[32223]: osdmap e77: 11 total, 11 up, 11 in 2024-04-17T05:58:04.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:03 smithi103 ceph-mon[36243]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-17T05:58:04.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:03 smithi103 ceph-mon[36243]: osdmap e77: 11 total, 11 up, 11 in 2024-04-17T05:58:04.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:04 smithi098 ceph-mon[36516]: from='client.? 172.21.15.103:0/2521855924' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2e85e8d9-bf6c-4a12-b124-a4690dd61108"}]: dispatch 2024-04-17T05:58:04.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:04 smithi098 ceph-mon[36516]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2e85e8d9-bf6c-4a12-b124-a4690dd61108"}]: dispatch 2024-04-17T05:58:04.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:04 smithi098 ceph-mon[36516]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2e85e8d9-bf6c-4a12-b124-a4690dd61108"}]': finished 2024-04-17T05:58:04.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:04 smithi098 ceph-mon[36516]: osdmap e78: 12 total, 11 up, 12 in 2024-04-17T05:58:04.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:04 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-17T05:58:04.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:04 smithi047 ceph-mon[32223]: from='client.? 172.21.15.103:0/2521855924' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2e85e8d9-bf6c-4a12-b124-a4690dd61108"}]: dispatch 2024-04-17T05:58:04.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:04 smithi047 ceph-mon[32223]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2e85e8d9-bf6c-4a12-b124-a4690dd61108"}]: dispatch 2024-04-17T05:58:04.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:04 smithi047 ceph-mon[32223]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2e85e8d9-bf6c-4a12-b124-a4690dd61108"}]': finished 2024-04-17T05:58:04.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:04 smithi047 ceph-mon[32223]: osdmap e78: 12 total, 11 up, 12 in 2024-04-17T05:58:04.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:04 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-17T05:58:05.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:04 smithi103 ceph-mon[36243]: from='client.? 172.21.15.103:0/2521855924' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2e85e8d9-bf6c-4a12-b124-a4690dd61108"}]: dispatch 2024-04-17T05:58:05.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:04 smithi103 ceph-mon[36243]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2e85e8d9-bf6c-4a12-b124-a4690dd61108"}]: dispatch 2024-04-17T05:58:05.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:04 smithi103 ceph-mon[36243]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2e85e8d9-bf6c-4a12-b124-a4690dd61108"}]': finished 2024-04-17T05:58:05.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:04 smithi103 ceph-mon[36243]: osdmap e78: 12 total, 11 up, 12 in 2024-04-17T05:58:05.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:04 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-17T05:58:05.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:05 smithi098 ceph-mon[36516]: pgmap v242: 1 pgs: 1 peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 140 KiB/s, 0 objects/s recovering 2024-04-17T05:58:05.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:05 smithi098 ceph-mon[36516]: from='client.? 172.21.15.103:0/2407586164' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-17T05:58:05.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:05 smithi047 ceph-mon[32223]: pgmap v242: 1 pgs: 1 peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 140 KiB/s, 0 objects/s recovering 2024-04-17T05:58:05.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:05 smithi047 ceph-mon[32223]: from='client.? 172.21.15.103:0/2407586164' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-17T05:58:06.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:05 smithi103 ceph-mon[36243]: pgmap v242: 1 pgs: 1 peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 140 KiB/s, 0 objects/s recovering 2024-04-17T05:58:06.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:05 smithi103 ceph-mon[36243]: from='client.? 172.21.15.103:0/2407586164' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-17T05:58:07.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:07 smithi098 ceph-mon[36516]: pgmap v244: 1 pgs: 1 peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 112 KiB/s, 0 objects/s recovering 2024-04-17T05:58:07.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:07 smithi047 ceph-mon[32223]: pgmap v244: 1 pgs: 1 peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 112 KiB/s, 0 objects/s recovering 2024-04-17T05:58:08.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:07 smithi103 ceph-mon[36243]: pgmap v244: 1 pgs: 1 peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 112 KiB/s, 0 objects/s recovering 2024-04-17T05:58:09.783 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:09 smithi103 ceph-mon[36243]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-04-17T05:58:09.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:09 smithi098 ceph-mon[36516]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-04-17T05:58:09.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:09 smithi047 ceph-mon[32223]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-04-17T05:58:11.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:11 smithi098 ceph-mon[36516]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-17T05:58:11.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:11 smithi047 ceph-mon[32223]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-17T05:58:12.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:11 smithi103 ceph-mon[36243]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-17T05:58:12.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:12 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-17T05:58:12.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:12 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:58:12.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:12 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-17T05:58:12.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:12 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:58:12.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:12 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-17T05:58:12.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:12 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:58:13.783 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:13 smithi103 ceph-mon[36243]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-17T05:58:13.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:13 smithi103 ceph-mon[36243]: Deploying daemon osd.11 on smithi103 2024-04-17T05:58:13.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:13 smithi098 ceph-mon[36516]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-17T05:58:13.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:13 smithi098 ceph-mon[36516]: Deploying daemon osd.11 on smithi103 2024-04-17T05:58:13.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:13 smithi047 ceph-mon[32223]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-17T05:58:13.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:13 smithi047 ceph-mon[32223]: Deploying daemon osd.11 on smithi103 2024-04-17T05:58:15.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:15 smithi103 ceph-mon[36243]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-17T05:58:15.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:15 smithi098 ceph-mon[36516]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-17T05:58:15.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:15 smithi047 ceph-mon[32223]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-17T05:58:17.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:17 smithi098 ceph-mon[36516]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-17T05:58:17.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:17 smithi047 ceph-mon[32223]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-17T05:58:18.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:17 smithi103 ceph-mon[36243]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-17T05:58:19.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:18 smithi103 ceph-mon[36243]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-17T05:58:19.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:18 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:19.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:18 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:19.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:18 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T05:58:19.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:18 smithi098 ceph-mon[36516]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-17T05:58:19.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:18 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:19.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:18 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:19.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:18 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T05:58:19.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:18 smithi047 ceph-mon[32223]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-17T05:58:19.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:18 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:19.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:18 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:19.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:18 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T05:58:21.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:20 smithi103 ceph-mon[36243]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-17T05:58:21.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:20 smithi098 ceph-mon[36516]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-17T05:58:21.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:20 smithi047 ceph-mon[32223]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-17T05:58:21.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:58:21.881+0000 7fe1b37fe700 1 -- 172.21.15.103:0/1576456453 <== mgr.14152 v2:172.21.15.47:6800/3157902806 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fe1bc06ae40 con 0x7fe1a80a3c40 2024-04-17T05:58:21.880 INFO:teuthology.orchestra.run.smithi103.stdout:Created osd(s) 11 on host 'smithi103' 2024-04-17T05:58:21.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:58:21.883+0000 7fe1c472a700 1 -- 172.21.15.103:0/1576456453 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fe1a80a3c40 msgr2=0x7fe1a80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:58:21.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:58:21.883+0000 7fe1c472a700 1 --2- 172.21.15.103:0/1576456453 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fe1a80a3c40 0x7fe1a80a6100 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe1b800b2e0 tx=0x7fe1b801a040 comp rx=0 tx=0).stop 2024-04-17T05:58:21.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:58:21.883+0000 7fe1c472a700 1 -- 172.21.15.103:0/1576456453 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe1bc103550 msgr2=0x7fe1bc1046c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:58:21.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:58:21.883+0000 7fe1c472a700 1 --2- 172.21.15.103:0/1576456453 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe1bc103550 0x7fe1bc1046c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe1b400ada0 tx=0x7fe1b400c330 comp rx=0 tx=0).stop 2024-04-17T05:58:21.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:58:21.883+0000 7fe1c472a700 1 -- 172.21.15.103:0/1576456453 shutdown_connections 2024-04-17T05:58:21.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:58:21.883+0000 7fe1c472a700 1 --2- 172.21.15.103:0/1576456453 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fe1a80a3c40 0x7fe1a80a6100 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:21.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:58:21.883+0000 7fe1c472a700 1 --2- 172.21.15.103:0/1576456453 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe1bc103550 0x7fe1bc1046c0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:21.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:58:21.883+0000 7fe1c472a700 1 --2- 172.21.15.103:0/1576456453 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe1bc1019b0 0x7fe1bc105ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:21.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:58:21.883+0000 7fe1c472a700 1 --2- 172.21.15.103:0/1576456453 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe1bc102bb0 0x7fe1bc104180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:21.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:58:21.883+0000 7fe1c472a700 1 -- 172.21.15.103:0/1576456453 >> 172.21.15.103:0/1576456453 conn(0x7fe1bc0fcf50 msgr2=0x7fe1bc06a730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:58:21.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:58:21.883+0000 7fe1c472a700 1 -- 172.21.15.103:0/1576456453 shutdown_connections 2024-04-17T05:58:21.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:58:21.883+0000 7fe1c472a700 1 -- 172.21.15.103:0/1576456453 wait complete. 2024-04-17T05:58:22.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:21 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:22.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:21 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:22.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:21 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:58:22.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:21 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T05:58:22.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:21 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:22.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:21 smithi103 ceph-mon[36243]: from='osd.11 [v2:172.21.15.103:6824/2639831013,v1:172.21.15.103:6825/2639831013]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-17T05:58:22.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:21 smithi103 ceph-mon[36243]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-17T05:58:22.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:21 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:22.035 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:21 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:22.035 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:21 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T05:58:22.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:21 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:22.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:21 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:22.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:21 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:58:22.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:21 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T05:58:22.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:21 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:22.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:21 smithi098 ceph-mon[36516]: from='osd.11 [v2:172.21.15.103:6824/2639831013,v1:172.21.15.103:6825/2639831013]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-17T05:58:22.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:21 smithi098 ceph-mon[36516]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-17T05:58:22.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:21 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:22.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:21 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:22.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:21 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T05:58:22.388 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:21 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:22.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:21 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:22.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:21 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:58:22.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:21 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T05:58:22.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:21 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:22.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:21 smithi047 ceph-mon[32223]: from='osd.11 [v2:172.21.15.103:6824/2639831013,v1:172.21.15.103:6825/2639831013]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-17T05:58:22.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:21 smithi047 ceph-mon[32223]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-17T05:58:22.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:21 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:22.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:21 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:22.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:21 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T05:58:22.531 DEBUG:teuthology.orchestra.run.smithi103:osd.11> sudo journalctl -f -n 0 -u ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@osd.11.service 2024-04-17T05:58:22.534 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-04-17T05:58:22.534 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 565db2d8-fc7e-11ee-bc91-c7b262605968 -- ceph osd stat -f json 2024-04-17T05:58:22.792 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T05:58:23.167 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:22 smithi047 ceph-mon[32223]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-17T05:58:23.168 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:22 smithi047 ceph-mon[32223]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-17T05:58:23.168 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:22 smithi047 ceph-mon[32223]: osdmap e79: 12 total, 11 up, 12 in 2024-04-17T05:58:23.168 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:22 smithi047 ceph-mon[32223]: from='osd.11 [v2:172.21.15.103:6824/2639831013,v1:172.21.15.103:6825/2639831013]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-04-17T05:58:23.168 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:22 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-17T05:58:23.168 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:22 smithi047 ceph-mon[32223]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-04-17T05:58:23.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:22 smithi103 ceph-mon[36243]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-17T05:58:23.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:22 smithi103 ceph-mon[36243]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-17T05:58:23.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:22 smithi103 ceph-mon[36243]: osdmap e79: 12 total, 11 up, 12 in 2024-04-17T05:58:23.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:22 smithi103 ceph-mon[36243]: from='osd.11 [v2:172.21.15.103:6824/2639831013,v1:172.21.15.103:6825/2639831013]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-04-17T05:58:23.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:22 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-17T05:58:23.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:22 smithi103 ceph-mon[36243]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-04-17T05:58:23.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:22 smithi098 ceph-mon[36516]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-17T05:58:23.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:22 smithi098 ceph-mon[36516]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-17T05:58:23.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:22 smithi098 ceph-mon[36516]: osdmap e79: 12 total, 11 up, 12 in 2024-04-17T05:58:23.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:22 smithi098 ceph-mon[36516]: from='osd.11 [v2:172.21.15.103:6824/2639831013,v1:172.21.15.103:6825/2639831013]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-04-17T05:58:23.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:22 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-17T05:58:23.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:22 smithi098 ceph-mon[36516]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-04-17T05:58:23.757 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 05:58:23 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T05:58:23.376+0000 7f4e5b451700 -1 osd.11 0 waiting for initial osdmap 2024-04-17T05:58:23.758 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 05:58:23 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T05:58:23.388+0000 7f4e5323c700 -1 osd.11 80 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-17T05:58:24.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:23 smithi103 ceph-mon[36243]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-04-17T05:58:24.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:23 smithi103 ceph-mon[36243]: osdmap e80: 12 total, 11 up, 12 in 2024-04-17T05:58:24.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:23 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-17T05:58:24.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:23 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-17T05:58:24.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:23 smithi103 ceph-mon[36243]: from='osd.11 ' entity='osd.11' 2024-04-17T05:58:24.196 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:23 smithi047 ceph-mon[32223]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-04-17T05:58:24.196 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:23 smithi047 ceph-mon[32223]: osdmap e80: 12 total, 11 up, 12 in 2024-04-17T05:58:24.196 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:23 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-17T05:58:24.196 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:23 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-17T05:58:24.196 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:23 smithi047 ceph-mon[32223]: from='osd.11 ' entity='osd.11' 2024-04-17T05:58:24.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:23 smithi098 ceph-mon[36516]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-04-17T05:58:24.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:23 smithi098 ceph-mon[36516]: osdmap e80: 12 total, 11 up, 12 in 2024-04-17T05:58:24.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:23 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-17T05:58:24.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:23 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-17T05:58:24.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:23 smithi098 ceph-mon[36516]: from='osd.11 ' entity='osd.11' 2024-04-17T05:58:24.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:24.564+0000 7fb517d41700 1 -- 172.21.15.47:0/296292365 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb510104650 msgr2=0x7fb51010af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:58:24.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:24.564+0000 7fb517d41700 1 --2- 172.21.15.47:0/296292365 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb510104650 0x7fb51010af00 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb50800b0d0 tx=0x7fb50800b3e0 comp rx=0 tx=0).stop 2024-04-17T05:58:24.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:24.564+0000 7fb517d41700 1 -- 172.21.15.47:0/296292365 shutdown_connections 2024-04-17T05:58:24.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:24.564+0000 7fb517d41700 1 --2- 172.21.15.47:0/296292365 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb510104650 0x7fb51010af00 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:24.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:24.564+0000 7fb517d41700 1 --2- 172.21.15.47:0/296292365 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb510102ab0 0x7fb510102ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:24.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:24.564+0000 7fb517d41700 1 --2- 172.21.15.47:0/296292365 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb510103cb0 0x7fb510104110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:24.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:24.564+0000 7fb517d41700 1 -- 172.21.15.47:0/296292365 >> 172.21.15.47:0/296292365 conn(0x7fb5100fe050 msgr2=0x7fb510100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:58:24.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:24.565+0000 7fb517d41700 1 -- 172.21.15.47:0/296292365 shutdown_connections 2024-04-17T05:58:24.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:24.565+0000 7fb517d41700 1 -- 172.21.15.47:0/296292365 wait complete. 2024-04-17T05:58:24.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:24.565+0000 7fb517d41700 1 Processor -- start 2024-04-17T05:58:24.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:24.565+0000 7fb517d41700 1 -- start start 2024-04-17T05:58:24.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:24.565+0000 7fb517d41700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb510102ab0 0x7fb510198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:58:24.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:24.565+0000 7fb517d41700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb510103cb0 0x7fb5101991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:58:24.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:24.565+0000 7fb517d41700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb510104650 0x7fb5101a0250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:58:24.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:24.565+0000 7fb517d41700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb51010e6f0 con 0x7fb510102ab0 2024-04-17T05:58:24.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:24.565+0000 7fb517d41700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb51010e570 con 0x7fb510104650 2024-04-17T05:58:24.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:24.565+0000 7fb517d41700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb51010e870 con 0x7fb510103cb0 2024-04-17T05:58:24.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:24.565+0000 7fb515add700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb510102ab0 0x7fb510198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:58:24.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:24.565+0000 7fb515add700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb510102ab0 0x7fb510198ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54378/0 (socket says 172.21.15.47:54378) 2024-04-17T05:58:24.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:24.565+0000 7fb5152dc700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb510103cb0 0x7fb5101991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:58:24.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:24.565+0000 7fb515add700 1 -- 172.21.15.47:0/829189511 learned_addr learned my addr 172.21.15.47:0/829189511 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:58:24.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:24.566+0000 7fb5162de700 1 --2- 172.21.15.47:0/829189511 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb510104650 0x7fb5101a0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:58:24.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:24.566+0000 7fb515add700 1 -- 172.21.15.47:0/829189511 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb510103cb0 msgr2=0x7fb5101991e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:58:24.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:24.566+0000 7fb515add700 1 --2- 172.21.15.47:0/829189511 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb510103cb0 0x7fb5101991e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:24.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:24.566+0000 7fb515add700 1 -- 172.21.15.47:0/829189511 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb510104650 msgr2=0x7fb5101a0250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:58:24.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:24.566+0000 7fb515add700 1 --2- 172.21.15.47:0/829189511 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb510104650 0x7fb5101a0250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:24.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:24.566+0000 7fb515add700 1 -- 172.21.15.47:0/829189511 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5101a0920 con 0x7fb510102ab0 2024-04-17T05:58:24.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:24.566+0000 7fb5162de700 1 --2- 172.21.15.47:0/829189511 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb510104650 0x7fb5101a0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T05:58:24.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:24.566+0000 7fb515add700 1 --2- 172.21.15.47:0/829189511 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb510102ab0 0x7fb510198ca0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb50c00d980 tx=0x7fb50c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:58:24.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:24.566+0000 7fb506ffd700 1 -- 172.21.15.47:0/829189511 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb50c017030 con 0x7fb510102ab0 2024-04-17T05:58:24.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:24.566+0000 7fb506ffd700 1 -- 172.21.15.47:0/829189511 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb50c004500 con 0x7fb510102ab0 2024-04-17T05:58:24.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:24.566+0000 7fb517d41700 1 -- 172.21.15.47:0/829189511 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5101a0c10 con 0x7fb510102ab0 2024-04-17T05:58:24.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:24.566+0000 7fb506ffd700 1 -- 172.21.15.47:0/829189511 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb50c003aa0 con 0x7fb510102ab0 2024-04-17T05:58:24.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:24.566+0000 7fb517d41700 1 -- 172.21.15.47:0/829189511 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5101a11a0 con 0x7fb510102ab0 2024-04-17T05:58:24.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:24.567+0000 7fb517d41700 1 -- 172.21.15.47:0/829189511 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb510069740 con 0x7fb510102ab0 2024-04-17T05:58:24.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:24.568+0000 7fb506ffd700 1 -- 172.21.15.47:0/829189511 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb50c01a070 con 0x7fb510102ab0 2024-04-17T05:58:24.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:24.569+0000 7fb506ffd700 1 --2- 172.21.15.47:0/829189511 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fb4fc0a3cc0 0x7fb4fc0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:58:24.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:24.569+0000 7fb5152dc700 1 --2- 172.21.15.47:0/829189511 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fb4fc0a3cc0 0x7fb4fc0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:58:24.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:24.569+0000 7fb506ffd700 1 -- 172.21.15.47:0/829189511 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fb50c0cfbe0 con 0x7fb510102ab0 2024-04-17T05:58:24.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:24.570+0000 7fb5152dc700 1 --2- 172.21.15.47:0/829189511 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fb4fc0a3cc0 0x7fb4fc0a6180 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb51019a1b0 tx=0x7fb500009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:58:24.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:24.573+0000 7fb506ffd700 1 -- 172.21.15.47:0/829189511 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fb50c09cae0 con 0x7fb510102ab0 2024-04-17T05:58:24.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:24.782+0000 7fb517d41700 1 -- 172.21.15.47:0/829189511 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fb51002df90 con 0x7fb510102ab0 2024-04-17T05:58:24.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:24.783+0000 7fb506ffd700 1 -- 172.21.15.47:0/829189511 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v81) ==== 74+0+133 (secure 0 0 0) 0x7fb50c09c3e0 con 0x7fb510102ab0 2024-04-17T05:58:24.782 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-17T05:58:24.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:24.785+0000 7fb517d41700 1 -- 172.21.15.47:0/829189511 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fb4fc0a3cc0 msgr2=0x7fb4fc0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:58:24.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:24.785+0000 7fb517d41700 1 --2- 172.21.15.47:0/829189511 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fb4fc0a3cc0 0x7fb4fc0a6180 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb51019a1b0 tx=0x7fb500009190 comp rx=0 tx=0).stop 2024-04-17T05:58:24.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:24.785+0000 7fb517d41700 1 -- 172.21.15.47:0/829189511 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb510102ab0 msgr2=0x7fb510198ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:58:24.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:24.785+0000 7fb517d41700 1 --2- 172.21.15.47:0/829189511 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb510102ab0 0x7fb510198ca0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb50c00d980 tx=0x7fb50c00dc90 comp rx=0 tx=0).stop 2024-04-17T05:58:24.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:24.785+0000 7fb517d41700 1 -- 172.21.15.47:0/829189511 shutdown_connections 2024-04-17T05:58:24.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:24.785+0000 7fb517d41700 1 --2- 172.21.15.47:0/829189511 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fb4fc0a3cc0 0x7fb4fc0a6180 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:24.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:24.785+0000 7fb517d41700 1 --2- 172.21.15.47:0/829189511 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb510104650 0x7fb5101a0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:24.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:24.785+0000 7fb517d41700 1 --2- 172.21.15.47:0/829189511 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb510102ab0 0x7fb510198ca0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:24.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:24.785+0000 7fb517d41700 1 --2- 172.21.15.47:0/829189511 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb510103cb0 0x7fb5101991e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:24.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:24.786+0000 7fb517d41700 1 -- 172.21.15.47:0/829189511 >> 172.21.15.47:0/829189511 conn(0x7fb5100fe050 msgr2=0x7fb510106fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:58:24.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:24.786+0000 7fb517d41700 1 -- 172.21.15.47:0/829189511 shutdown_connections 2024-04-17T05:58:24.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:24.786+0000 7fb517d41700 1 -- 172.21.15.47:0/829189511 wait complete. 2024-04-17T05:58:25.097 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:24 smithi047 ceph-mon[32223]: purged_snaps scrub starts 2024-04-17T05:58:25.097 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:24 smithi047 ceph-mon[32223]: purged_snaps scrub ok 2024-04-17T05:58:25.097 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:24 smithi047 ceph-mon[32223]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-17T05:58:25.097 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:24 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-17T05:58:25.097 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:24 smithi047 ceph-mon[32223]: osd.11 [v2:172.21.15.103:6824/2639831013,v1:172.21.15.103:6825/2639831013] boot 2024-04-17T05:58:25.097 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:24 smithi047 ceph-mon[32223]: osdmap e81: 12 total, 12 up, 12 in 2024-04-17T05:58:25.097 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:24 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-17T05:58:25.098 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:24 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/829189511' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-17T05:58:25.186 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:24 smithi103 ceph-mon[36243]: purged_snaps scrub starts 2024-04-17T05:58:25.186 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:24 smithi103 ceph-mon[36243]: purged_snaps scrub ok 2024-04-17T05:58:25.187 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:24 smithi103 ceph-mon[36243]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-17T05:58:25.187 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:24 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-17T05:58:25.187 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:24 smithi103 ceph-mon[36243]: osd.11 [v2:172.21.15.103:6824/2639831013,v1:172.21.15.103:6825/2639831013] boot 2024-04-17T05:58:25.187 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:24 smithi103 ceph-mon[36243]: osdmap e81: 12 total, 12 up, 12 in 2024-04-17T05:58:25.187 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:24 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-17T05:58:25.187 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:24 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/829189511' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-17T05:58:25.284 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:24 smithi098 ceph-mon[36516]: purged_snaps scrub starts 2024-04-17T05:58:25.284 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:24 smithi098 ceph-mon[36516]: purged_snaps scrub ok 2024-04-17T05:58:25.284 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:24 smithi098 ceph-mon[36516]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-17T05:58:25.284 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:24 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-17T05:58:25.284 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:24 smithi098 ceph-mon[36516]: osd.11 [v2:172.21.15.103:6824/2639831013,v1:172.21.15.103:6825/2639831013] boot 2024-04-17T05:58:25.284 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:24 smithi098 ceph-mon[36516]: osdmap e81: 12 total, 12 up, 12 in 2024-04-17T05:58:25.285 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:24 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-17T05:58:25.285 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:24 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/829189511' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-17T05:58:25.567 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":81,"num_osds":12,"num_up_osds":12,"osd_up_since":1713333504,"num_in_osds":12,"osd_in_since":1713333484,"num_remapped_pgs":0} 2024-04-17T05:58:25.568 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 565db2d8-fc7e-11ee-bc91-c7b262605968 -- ceph osd dump --format=json 2024-04-17T05:58:25.823 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T05:58:26.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:26 smithi047 ceph-mon[32223]: osdmap e82: 12 total, 12 up, 12 in 2024-04-17T05:58:26.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:26 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:26.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:26 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:26.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:26 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-17T05:58:26.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:26 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-17T05:58:26.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:26 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-17T05:58:26.640 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:26 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-17T05:58:26.640 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:26 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:26.640 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:26 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:58:26.640 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:26 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T05:58:26.640 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:26 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:26.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:26 smithi103 ceph-mon[36243]: osdmap e82: 12 total, 12 up, 12 in 2024-04-17T05:58:26.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:26 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:26.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:26 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:26.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:26 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-17T05:58:26.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:26 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-17T05:58:26.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:26 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-17T05:58:26.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:26 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-17T05:58:26.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:26 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:26.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:26 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:58:26.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:26 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T05:58:26.785 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:26 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:26.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:26 smithi098 ceph-mon[36516]: osdmap e82: 12 total, 12 up, 12 in 2024-04-17T05:58:26.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:26 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:26.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:26 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:26.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:26 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-17T05:58:26.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:26 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-17T05:58:26.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:26 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-17T05:58:26.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:26 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-17T05:58:26.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:26 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:26.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:26 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:58:26.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:26 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T05:58:26.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:26 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:27.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:27.592+0000 7fc1ea39e700 1 -- 172.21.15.47:0/2420022374 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc1e4102aa0 msgr2=0x7fc1e4102f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:58:27.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:27.592+0000 7fc1ea39e700 1 --2- 172.21.15.47:0/2420022374 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc1e4102aa0 0x7fc1e4102f20 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc009990 tx=0x7fc1cc009ca0 comp rx=0 tx=0).stop 2024-04-17T05:58:27.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:27.592+0000 7fc1ea39e700 1 -- 172.21.15.47:0/2420022374 shutdown_connections 2024-04-17T05:58:27.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:27.592+0000 7fc1ea39e700 1 --2- 172.21.15.47:0/2420022374 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc1e4103460 0x7fc1e410f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:27.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:27.592+0000 7fc1ea39e700 1 --2- 172.21.15.47:0/2420022374 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc1e4108a60 0x7fc1e4108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:27.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:27.592+0000 7fc1ea39e700 1 --2- 172.21.15.47:0/2420022374 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc1e4102aa0 0x7fc1e4102f20 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:27.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:27.592+0000 7fc1ea39e700 1 -- 172.21.15.47:0/2420022374 >> 172.21.15.47:0/2420022374 conn(0x7fc1e4079bb0 msgr2=0x7fc1e4079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:58:27.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:27.592+0000 7fc1ea39e700 1 -- 172.21.15.47:0/2420022374 shutdown_connections 2024-04-17T05:58:27.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:27.592+0000 7fc1ea39e700 1 -- 172.21.15.47:0/2420022374 wait complete. 2024-04-17T05:58:27.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:27.592+0000 7fc1ea39e700 1 Processor -- start 2024-04-17T05:58:27.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:27.593+0000 7fc1ea39e700 1 -- start start 2024-04-17T05:58:27.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:27.593+0000 7fc1ea39e700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc1e4102aa0 0x7fc1e419b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:58:27.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:27.593+0000 7fc1ea39e700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc1e4103460 0x7fc1e419b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:58:27.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:27.593+0000 7fc1ea39e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc1e4108a60 0x7fc1e419fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:58:27.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:27.593+0000 7fc1ea39e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fc1e4112d40 con 0x7fc1e4108a60 2024-04-17T05:58:27.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:27.593+0000 7fc1ea39e700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc1e4112bc0 con 0x7fc1e4103460 2024-04-17T05:58:27.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:27.593+0000 7fc1ea39e700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc1e4112ec0 con 0x7fc1e4102aa0 2024-04-17T05:58:27.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:27.593+0000 7fc1e893b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc1e4108a60 0x7fc1e419fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:58:27.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:27.593+0000 7fc1e893b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc1e4108a60 0x7fc1e419fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54402/0 (socket says 172.21.15.47:54402) 2024-04-17T05:58:27.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:27.593+0000 7fc1e893b700 1 -- 172.21.15.47:0/100898860 learned_addr learned my addr 172.21.15.47:0/100898860 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:58:27.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:27.593+0000 7fc1dbfff700 1 --2- 172.21.15.47:0/100898860 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc1e4103460 0x7fc1e419b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:58:27.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:27.593+0000 7fc1e3fff700 1 --2- 172.21.15.47:0/100898860 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc1e4102aa0 0x7fc1e419b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:58:27.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:27.593+0000 7fc1e893b700 1 -- 172.21.15.47:0/100898860 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc1e4102aa0 msgr2=0x7fc1e419b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:58:27.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:27.593+0000 7fc1e893b700 1 --2- 172.21.15.47:0/100898860 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc1e4102aa0 0x7fc1e419b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:27.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:27.593+0000 7fc1e893b700 1 -- 172.21.15.47:0/100898860 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc1e4103460 msgr2=0x7fc1e419b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:58:27.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:27.593+0000 7fc1e893b700 1 --2- 172.21.15.47:0/100898860 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc1e4103460 0x7fc1e419b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:27.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:27.593+0000 7fc1e893b700 1 -- 172.21.15.47:0/100898860 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1e41a02f0 con 0x7fc1e4108a60 2024-04-17T05:58:27.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:27.593+0000 7fc1e893b700 1 --2- 172.21.15.47:0/100898860 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc1e4108a60 0x7fc1e419fbb0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc1dc007d70 tx=0x7fc1dc00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:58:27.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:27.594+0000 7fc1e1ffb700 1 -- 172.21.15.47:0/100898860 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1dc016020 con 0x7fc1e4108a60 2024-04-17T05:58:27.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:27.594+0000 7fc1ea39e700 1 -- 172.21.15.47:0/100898860 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1e41a05e0 con 0x7fc1e4108a60 2024-04-17T05:58:27.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:27.594+0000 7fc1e1ffb700 1 -- 172.21.15.47:0/100898860 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc1dc00ac90 con 0x7fc1e4108a60 2024-04-17T05:58:27.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:27.594+0000 7fc1e1ffb700 1 -- 172.21.15.47:0/100898860 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1dc0115e0 con 0x7fc1e4108a60 2024-04-17T05:58:27.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:27.594+0000 7fc1ea39e700 1 -- 172.21.15.47:0/100898860 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1e41a7e60 con 0x7fc1e4108a60 2024-04-17T05:58:27.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:27.595+0000 7fc1ea39e700 1 -- 172.21.15.47:0/100898860 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1e4069740 con 0x7fc1e4108a60 2024-04-17T05:58:27.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:27.596+0000 7fc1e1ffb700 1 -- 172.21.15.47:0/100898860 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fc1dc019020 con 0x7fc1e4108a60 2024-04-17T05:58:27.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:27.596+0000 7fc1e1ffb700 1 --2- 172.21.15.47:0/100898860 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fc1d00a3c40 0x7fc1d00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:58:27.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:27.596+0000 7fc1e3fff700 1 --2- 172.21.15.47:0/100898860 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fc1d00a3c40 0x7fc1d00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:58:27.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:27.597+0000 7fc1e1ffb700 1 -- 172.21.15.47:0/100898860 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fc1dc0cfdc0 con 0x7fc1e4108a60 2024-04-17T05:58:27.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:27.597+0000 7fc1e3fff700 1 --2- 172.21.15.47:0/100898860 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fc1d00a3c40 0x7fc1d00a6100 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc1d400abd0 tx=0x7fc1d400a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:58:27.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:27.601+0000 7fc1e1ffb700 1 -- 172.21.15.47:0/100898860 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fc1dc020020 con 0x7fc1e4108a60 2024-04-17T05:58:27.638 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:27 smithi047 ceph-mon[32223]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T05:58:27.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:27 smithi047 ceph-mon[32223]: Detected new or changed devices on smithi103 2024-04-17T05:58:27.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:27 smithi047 ceph-mon[32223]: Adjusting osd_memory_target on smithi103 to 4270M 2024-04-17T05:58:27.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:27 smithi103 ceph-mon[36243]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T05:58:27.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:27 smithi103 ceph-mon[36243]: Detected new or changed devices on smithi103 2024-04-17T05:58:27.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:27 smithi103 ceph-mon[36243]: Adjusting osd_memory_target on smithi103 to 4270M 2024-04-17T05:58:27.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:27.810+0000 7fc1ea39e700 1 -- 172.21.15.47:0/100898860 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc1e41a8090 con 0x7fc1e4108a60 2024-04-17T05:58:27.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:27.811+0000 7fc1e1ffb700 1 -- 172.21.15.47:0/100898860 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18406 (secure 0 0 0) 0x7fc1dc09cc40 con 0x7fc1e4108a60 2024-04-17T05:58:27.811 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-17T05:58:27.811 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":82,"fsid":"565db2d8-fc7e-11ee-bc91-c7b262605968","created":"2024-04-17T05:51:19.888865+0000","modified":"2024-04-17T05:58:25.403856+0000","last_up_change":"2024-04-17T05:58:24.380131+0000","last_in_change":"2024-04-17T05:58:04.418236+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T05:54:37.734306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8453f300-533d-4736-a343-55619c08e966","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":1290127544},{"type":"v1","addr":"172.21.15.47:6803","nonce":1290127544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":1290127544},{"type":"v1","addr":"172.21.15.47:6805","nonce":1290127544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":1290127544},{"type":"v1","addr":"172.21.15.47:6809","nonce":1290127544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":1290127544},{"type":"v1","addr":"172.21.15.47:6807","nonce":1290127544}]},"public_addr":"172.21.15.47:6803/1290127544","cluster_addr":"172.21.15.47:6805/1290127544","heartbeat_back_addr":"172.21.15.47:6809/1290127544","heartbeat_front_addr":"172.21.15.47:6807/1290127544","state":["exists","up"]},{"osd":1,"uuid":"80b0f03e-c1bc-4658-8028-1e6b8eedcc47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":1180430954},{"type":"v1","addr":"172.21.15.98:6801","nonce":1180430954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":1180430954},{"type":"v1","addr":"172.21.15.98:6803","nonce":1180430954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":1180430954},{"type":"v1","addr":"172.21.15.98:6807","nonce":1180430954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":1180430954},{"type":"v1","addr":"172.21.15.98:6805","nonce":1180430954}]},"public_addr":"172.21.15.98:6801/1180430954","cluster_addr":"172.21.15.98:6803/1180430954","heartbeat_back_addr":"172.21.15.98:6807/1180430954","heartbeat_front_addr":"172.21.15.98:6805/1180430954","state":["exists","up"]},{"osd":2,"uuid":"0fc9d383-05b4-498d-b530-fccf7aeddd7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":494676216},{"type":"v1","addr":"172.21.15.103:6801","nonce":494676216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":494676216},{"type":"v1","addr":"172.21.15.103:6803","nonce":494676216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":494676216},{"type":"v1","addr":"172.21.15.103:6807","nonce":494676216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":494676216},{"type":"v1","addr":"172.21.15.103:6805","nonce":494676216}]},"public_addr":"172.21.15.103:6801/494676216","cluster_addr":"172.21.15.103:6803/494676216","heartbeat_back_addr":"172.21.15.103:6807/494676216","heartbeat_front_addr":"172.21.15.103:6805/494676216","state":["exists","up"]},{"osd":3,"uuid":"f6e0b0d5-0fb3-42d0-9e84-f7cfef21a998","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":1998369447},{"type":"v1","addr":"172.21.15.47:6811","nonce":1998369447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":1998369447},{"type":"v1","addr":"172.21.15.47:6813","nonce":1998369447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":1998369447},{"type":"v1","addr":"172.21.15.47:6817","nonce":1998369447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":1998369447},{"type":"v1","addr":"172.21.15.47:6815","nonce":1998369447}]},"public_addr":"172.21.15.47:6811/1998369447","cluster_addr":"172.21.15.47:6813/1998369447","heartbeat_back_addr":"172.21.15.47:6817/1998369447","heartbeat_front_addr":"172.21.15.47:6815/1998369447","state":["exists","up"]},{"osd":4,"uuid":"2c26217e-02df-466d-bc01-0149d4aec3c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":3060482485},{"type":"v1","addr":"172.21.15.98:6809","nonce":3060482485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":3060482485},{"type":"v1","addr":"172.21.15.98:6811","nonce":3060482485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":3060482485},{"type":"v1","addr":"172.21.15.98:6815","nonce":3060482485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":3060482485},{"type":"v1","addr":"172.21.15.98:6813","nonce":3060482485}]},"public_addr":"172.21.15.98:6809/3060482485","cluster_addr":"172.21.15.98:6811/3060482485","heartbeat_back_addr":"172.21.15.98:6815/3060482485","heartbeat_front_addr":"172.21.15.98:6813/3060482485","state":["exists","up"]},{"osd":5,"uuid":"740bf3e0-bea8-496b-8d6f-e42d0b017258","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":4181363567},{"type":"v1","addr":"172.21.15.103:6809","nonce":4181363567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":4181363567},{"type":"v1","addr":"172.21.15.103:6811","nonce":4181363567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":4181363567},{"type":"v1","addr":"172.21.15.103:6815","nonce":4181363567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":4181363567},{"type":"v1","addr":"172.21.15.103:6813","nonce":4181363567}]},"public_addr":"172.21.15.103:6809/4181363567","cluster_addr":"172.21.15.103:6811/4181363567","heartbeat_back_addr":"172.21.15.103:6815/4181363567","heartbeat_front_addr":"172.21.15.103:6813/4181363567","state":["exists","up"]},{"osd":6,"uuid":"356a6255-2c07-4f0d-a233-26be04c92b38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":2517529668},{"type":"v1","addr":"172.21.15.47:6819","nonce":2517529668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2517529668},{"type":"v1","addr":"172.21.15.47:6821","nonce":2517529668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2517529668},{"type":"v1","addr":"172.21.15.47:6825","nonce":2517529668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2517529668},{"type":"v1","addr":"172.21.15.47:6823","nonce":2517529668}]},"public_addr":"172.21.15.47:6819/2517529668","cluster_addr":"172.21.15.47:6821/2517529668","heartbeat_back_addr":"172.21.15.47:6825/2517529668","heartbeat_front_addr":"172.21.15.47:6823/2517529668","state":["exists","up"]},{"osd":7,"uuid":"349d0120-8290-417d-a800-5a8985f554f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":4225498496},{"type":"v1","addr":"172.21.15.98:6817","nonce":4225498496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":4225498496},{"type":"v1","addr":"172.21.15.98:6819","nonce":4225498496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":4225498496},{"type":"v1","addr":"172.21.15.98:6823","nonce":4225498496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":4225498496},{"type":"v1","addr":"172.21.15.98:6821","nonce":4225498496}]},"public_addr":"172.21.15.98:6817/4225498496","cluster_addr":"172.21.15.98:6819/4225498496","heartbeat_back_addr":"172.21.15.98:6823/4225498496","heartbeat_front_addr":"172.21.15.98:6821/4225498496","state":["exists","up"]},{"osd":8,"uuid":"cee4712f-321b-42c0-92ba-e15144c00f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":3433905520},{"type":"v1","addr":"172.21.15.103:6817","nonce":3433905520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":3433905520},{"type":"v1","addr":"172.21.15.103:6819","nonce":3433905520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":3433905520},{"type":"v1","addr":"172.21.15.103:6823","nonce":3433905520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":3433905520},{"type":"v1","addr":"172.21.15.103:6821","nonce":3433905520}]},"public_addr":"172.21.15.103:6817/3433905520","cluster_addr":"172.21.15.103:6819/3433905520","heartbeat_back_addr":"172.21.15.103:6823/3433905520","heartbeat_front_addr":"172.21.15.103:6821/3433905520","state":["exists","up"]},{"osd":9,"uuid":"63fc73c1-5a22-487d-80b8-9894e2ddd207","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":728359554},{"type":"v1","addr":"172.21.15.47:6827","nonce":728359554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":728359554},{"type":"v1","addr":"172.21.15.47:6829","nonce":728359554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":728359554},{"type":"v1","addr":"172.21.15.47:6833","nonce":728359554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":728359554},{"type":"v1","addr":"172.21.15.47:6831","nonce":728359554}]},"public_addr":"172.21.15.47:6827/728359554","cluster_addr":"172.21.15.47:6829/728359554","heartbeat_back_addr":"172.21.15.47:6833/728359554","heartbeat_front_addr":"172.21.15.47:6831/728359554","state":["exists","up"]},{"osd":10,"uuid":"9565bcc3-62ee-48eb-a07a-03a81e0cc8e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":468205689},{"type":"v1","addr":"172.21.15.98:6825","nonce":468205689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":468205689},{"type":"v1","addr":"172.21.15.98:6827","nonce":468205689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":468205689},{"type":"v1","addr":"172.21.15.98:6831","nonce":468205689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":468205689},{"type":"v1","addr":"172.21.15.98:6829","nonce":468205689}]},"public_addr":"172.21.15.98:6825/468205689","cluster_addr":"172.21.15.98:6827/468205689","heartbeat_back_addr":"172.21.15.98:6831/468205689","heartbeat_front_addr":"172.21.15.98:6829/468205689","state":["exists","up"]},{"osd":11,"uuid":"2e85e8d9-bf6c-4a12-b124-a4690dd61108","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":2639831013},{"type":"v1","addr":"172.21.15.103:6825","nonce":2639831013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":2639831013},{"type":"v1","addr":"172.21.15.103:6827","nonce":2639831013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":2639831013},{"type":"v1","addr":"172.21.15.103:6831","nonce":2639831013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":2639831013},{"type":"v1","addr":"172.21.15.103:6829","nonce":2639831013}]},"public_addr":"172.21.15.103:6825/2639831013","cluster_addr":"172.21.15.103:6827/2639831013","heartbeat_back_addr":"172.21.15.103:6831/2639831013","heartbeat_front_addr":"172.21.15.103:6829/2639831013","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:53:47.366438+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:54:10.953559+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:54:34.320332+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:54:59.751360+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:55:23.982052+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:55:48.425017+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:56:14.793368+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:56:40.440830+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:57:04.623680+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:57:31.215262+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:57:57.040346+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:58:22.671572+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:6801/3632157175":"2024-04-18T05:52:13.645775+0000","172.21.15.47:0/4288219269":"2024-04-18T05:52:13.645775+0000","172.21.15.47:0/2584487544":"2024-04-18T05:52:13.645775+0000","172.21.15.47:6801/953064786":"2024-04-18T05:51:45.979351+0000","172.21.15.47:0/3484500927":"2024-04-18T05:51:45.979351+0000","172.21.15.47:6800/3632157175":"2024-04-18T05:52:13.645775+0000","172.21.15.47:0/112268670":"2024-04-18T05:51:45.979351+0000","172.21.15.47:0/1638423256":"2024-04-18T05:51:45.979351+0000","172.21.15.47:6800/953064786":"2024-04-18T05:51:45.979351+0000","172.21.15.47:0/849479010":"2024-04-18T05:52:13.645775+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T05:58:27.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:27.813+0000 7fc1ea39e700 1 -- 172.21.15.47:0/100898860 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fc1d00a3c40 msgr2=0x7fc1d00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:58:27.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:27.813+0000 7fc1ea39e700 1 --2- 172.21.15.47:0/100898860 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fc1d00a3c40 0x7fc1d00a6100 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc1d400abd0 tx=0x7fc1d400a380 comp rx=0 tx=0).stop 2024-04-17T05:58:27.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:27.813+0000 7fc1ea39e700 1 -- 172.21.15.47:0/100898860 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc1e4108a60 msgr2=0x7fc1e419fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:58:27.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:27.813+0000 7fc1ea39e700 1 --2- 172.21.15.47:0/100898860 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc1e4108a60 0x7fc1e419fbb0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc1dc007d70 tx=0x7fc1dc00a340 comp rx=0 tx=0).stop 2024-04-17T05:58:27.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:27.814+0000 7fc1ea39e700 1 -- 172.21.15.47:0/100898860 shutdown_connections 2024-04-17T05:58:27.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:27.814+0000 7fc1ea39e700 1 --2- 172.21.15.47:0/100898860 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fc1d00a3c40 0x7fc1d00a6100 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:27.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:27.814+0000 7fc1ea39e700 1 --2- 172.21.15.47:0/100898860 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc1e4108a60 0x7fc1e419fbb0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:27.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:27.814+0000 7fc1ea39e700 1 --2- 172.21.15.47:0/100898860 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc1e4102aa0 0x7fc1e419b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:27.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:27.814+0000 7fc1ea39e700 1 --2- 172.21.15.47:0/100898860 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc1e4103460 0x7fc1e419b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:27.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:27.814+0000 7fc1ea39e700 1 -- 172.21.15.47:0/100898860 >> 172.21.15.47:0/100898860 conn(0x7fc1e4079bb0 msgr2=0x7fc1e4074c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:58:27.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:27.814+0000 7fc1ea39e700 1 -- 172.21.15.47:0/100898860 shutdown_connections 2024-04-17T05:58:27.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:27.814+0000 7fc1ea39e700 1 -- 172.21.15.47:0/100898860 wait complete. 2024-04-17T05:58:27.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:27 smithi098 ceph-mon[36516]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T05:58:27.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:27 smithi098 ceph-mon[36516]: Detected new or changed devices on smithi103 2024-04-17T05:58:27.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:27 smithi098 ceph-mon[36516]: Adjusting osd_memory_target on smithi103 to 4270M 2024-04-17T05:58:28.463 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T05:54:37.734306+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T05:58:28.463 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 565db2d8-fc7e-11ee-bc91-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-04-17T05:58:28.710 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T05:58:28.783 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:28 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/100898860' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T05:58:28.832 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:28 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/100898860' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T05:58:28.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:28 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/100898860' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T05:58:29.533 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:29 smithi047 ceph-mon[32223]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T05:58:29.783 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:29 smithi103 ceph-mon[36243]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T05:58:29.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:29 smithi098 ceph-mon[36516]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T05:58:30.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:30.366+0000 7fe86bbef700 1 -- 172.21.15.47:0/2974948747 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe864101980 msgr2=0x7fe864101e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:58:30.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:30.366+0000 7fe86bbef700 1 --2- 172.21.15.47:0/2974948747 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe864101980 0x7fe864101e00 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe854009990 tx=0x7fe854009ca0 comp rx=0 tx=0).stop 2024-04-17T05:58:30.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:30.366+0000 7fe86bbef700 1 -- 172.21.15.47:0/2974948747 shutdown_connections 2024-04-17T05:58:30.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:30.366+0000 7fe86bbef700 1 --2- 172.21.15.47:0/2974948747 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe864102340 0x7fe86410e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:30.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:30.366+0000 7fe86bbef700 1 --2- 172.21.15.47:0/2974948747 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe864103770 0x7fe864103b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:30.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:30.366+0000 7fe86bbef700 1 --2- 172.21.15.47:0/2974948747 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe864101980 0x7fe864101e00 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:30.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:30.366+0000 7fe86bbef700 1 -- 172.21.15.47:0/2974948747 >> 172.21.15.47:0/2974948747 conn(0x7fe8640779c0 msgr2=0x7fe864077dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:58:30.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:30.366+0000 7fe86bbef700 1 -- 172.21.15.47:0/2974948747 shutdown_connections 2024-04-17T05:58:30.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:30.366+0000 7fe86bbef700 1 -- 172.21.15.47:0/2974948747 wait complete. 2024-04-17T05:58:30.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:30.366+0000 7fe86bbef700 1 Processor -- start 2024-04-17T05:58:30.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:30.366+0000 7fe86bbef700 1 -- start start 2024-04-17T05:58:30.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:30.367+0000 7fe86bbef700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe864101980 0x7fe86410f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:58:30.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:30.367+0000 7fe86bbef700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe864102340 0x7fe86410f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:58:30.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:30.367+0000 7fe86bbef700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe864103770 0x7fe86410fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:58:30.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:30.367+0000 7fe86bbef700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fe864112f20 con 0x7fe864101980 2024-04-17T05:58:30.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:30.367+0000 7fe86bbef700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe864112da0 con 0x7fe864102340 2024-04-17T05:58:30.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:30.367+0000 7fe86bbef700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe8641130a0 con 0x7fe864103770 2024-04-17T05:58:30.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:30.367+0000 7fe86998b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe864101980 0x7fe86410f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:58:30.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:30.367+0000 7fe86998b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe864101980 0x7fe86410f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54430/0 (socket says 172.21.15.47:54430) 2024-04-17T05:58:30.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:30.367+0000 7fe86998b700 1 -- 172.21.15.47:0/2545586157 learned_addr learned my addr 172.21.15.47:0/2545586157 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:58:30.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:30.367+0000 7fe86a18c700 1 --2- 172.21.15.47:0/2545586157 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe864103770 0x7fe86410fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:58:30.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:30.367+0000 7fe86918a700 1 --2- 172.21.15.47:0/2545586157 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe864102340 0x7fe86410f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:58:30.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:30.367+0000 7fe86998b700 1 -- 172.21.15.47:0/2545586157 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe864103770 msgr2=0x7fe86410fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:58:30.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:30.367+0000 7fe86998b700 1 --2- 172.21.15.47:0/2545586157 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe864103770 0x7fe86410fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:30.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:30.367+0000 7fe86998b700 1 -- 172.21.15.47:0/2545586157 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe864102340 msgr2=0x7fe86410f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:58:30.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:30.367+0000 7fe86998b700 1 --2- 172.21.15.47:0/2545586157 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe864102340 0x7fe86410f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:30.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:30.367+0000 7fe86998b700 1 -- 172.21.15.47:0/2545586157 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8641a77c0 con 0x7fe864101980 2024-04-17T05:58:30.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:30.367+0000 7fe86918a700 1 --2- 172.21.15.47:0/2545586157 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe864102340 0x7fe86410f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T05:58:30.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:30.367+0000 7fe86998b700 1 --2- 172.21.15.47:0/2545586157 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe864101980 0x7fe86410f210 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe86000d950 tx=0x7fe86000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:58:30.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:30.367+0000 7fe85affd700 1 -- 172.21.15.47:0/2545586157 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe860016070 con 0x7fe864101980 2024-04-17T05:58:30.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:30.367+0000 7fe85affd700 1 -- 172.21.15.47:0/2545586157 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe860009e20 con 0x7fe864101980 2024-04-17T05:58:30.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:30.367+0000 7fe86bbef700 1 -- 172.21.15.47:0/2545586157 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8641a7ab0 con 0x7fe864101980 2024-04-17T05:58:30.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:30.367+0000 7fe85affd700 1 -- 172.21.15.47:0/2545586157 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe86000f620 con 0x7fe864101980 2024-04-17T05:58:30.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:30.368+0000 7fe86bbef700 1 -- 172.21.15.47:0/2545586157 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8641a7ff0 con 0x7fe864101980 2024-04-17T05:58:30.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:30.368+0000 7fe86bbef700 1 -- 172.21.15.47:0/2545586157 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe864069740 con 0x7fe864101980 2024-04-17T05:58:30.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:30.370+0000 7fe85affd700 1 -- 172.21.15.47:0/2545586157 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe860019070 con 0x7fe864101980 2024-04-17T05:58:30.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:30.370+0000 7fe85affd700 1 --2- 172.21.15.47:0/2545586157 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fe8500a3c40 0x7fe8500a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:58:30.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:30.370+0000 7fe86918a700 1 --2- 172.21.15.47:0/2545586157 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fe8500a3c40 0x7fe8500a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:58:30.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:30.371+0000 7fe85affd700 1 -- 172.21.15.47:0/2545586157 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fe8600cfa70 con 0x7fe864101980 2024-04-17T05:58:30.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:30.371+0000 7fe86918a700 1 --2- 172.21.15.47:0/2545586157 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fe8500a3c40 0x7fe8500a6100 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe85400b5c0 tx=0x7fe854005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:58:30.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:30.374+0000 7fe85affd700 1 -- 172.21.15.47:0/2545586157 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fe86009c8f0 con 0x7fe864101980 2024-04-17T05:58:30.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:30.580+0000 7fe86bbef700 1 -- 172.21.15.47:0/2545586157 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe86402df90 con 0x7fe864101980 2024-04-17T05:58:30.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:30.581+0000 7fe85affd700 1 -- 172.21.15.47:0/2545586157 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7fe86009c1f0 con 0x7fe864101980 2024-04-17T05:58:30.580 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-04-17T05:58:30.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:30.583+0000 7fe86bbef700 1 -- 172.21.15.47:0/2545586157 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fe8500a3c40 msgr2=0x7fe8500a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:58:30.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:30.583+0000 7fe86bbef700 1 --2- 172.21.15.47:0/2545586157 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fe8500a3c40 0x7fe8500a6100 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe85400b5c0 tx=0x7fe854005fd0 comp rx=0 tx=0).stop 2024-04-17T05:58:30.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:30.583+0000 7fe86bbef700 1 -- 172.21.15.47:0/2545586157 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe864101980 msgr2=0x7fe86410f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:58:30.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:30.583+0000 7fe86bbef700 1 --2- 172.21.15.47:0/2545586157 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe864101980 0x7fe86410f210 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe86000d950 tx=0x7fe86000dc60 comp rx=0 tx=0).stop 2024-04-17T05:58:30.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:30.583+0000 7fe86bbef700 1 -- 172.21.15.47:0/2545586157 shutdown_connections 2024-04-17T05:58:30.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:30.583+0000 7fe86bbef700 1 --2- 172.21.15.47:0/2545586157 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fe8500a3c40 0x7fe8500a6100 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:30.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:30.583+0000 7fe86bbef700 1 --2- 172.21.15.47:0/2545586157 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe864103770 0x7fe86410fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:30.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:30.583+0000 7fe86bbef700 1 --2- 172.21.15.47:0/2545586157 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe864101980 0x7fe86410f210 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:30.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:30.583+0000 7fe86bbef700 1 --2- 172.21.15.47:0/2545586157 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe864102340 0x7fe86410f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:30.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:30.583+0000 7fe86bbef700 1 -- 172.21.15.47:0/2545586157 >> 172.21.15.47:0/2545586157 conn(0x7fe8640779c0 msgr2=0x7fe864106740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:58:30.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:30.583+0000 7fe86bbef700 1 -- 172.21.15.47:0/2545586157 shutdown_connections 2024-04-17T05:58:30.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:30.583+0000 7fe86bbef700 1 -- 172.21.15.47:0/2545586157 wait complete. 2024-04-17T05:58:31.288 INFO:tasks.cephadm:Adding mds.a on smithi047 2024-04-17T05:58:31.288 INFO:tasks.cephadm:Adding mds.d on smithi047 2024-04-17T05:58:31.288 INFO:tasks.cephadm:Adding mds.g on smithi047 2024-04-17T05:58:31.288 INFO:tasks.cephadm:Adding mds.j on smithi047 2024-04-17T05:58:31.288 INFO:tasks.cephadm:Adding mds.b on smithi098 2024-04-17T05:58:31.288 INFO:tasks.cephadm:Adding mds.e on smithi098 2024-04-17T05:58:31.288 INFO:tasks.cephadm:Adding mds.h on smithi098 2024-04-17T05:58:31.288 INFO:tasks.cephadm:Adding mds.k on smithi098 2024-04-17T05:58:31.288 INFO:tasks.cephadm:Adding mds.c on smithi103 2024-04-17T05:58:31.289 INFO:tasks.cephadm:Adding mds.f on smithi103 2024-04-17T05:58:31.289 INFO:tasks.cephadm:Adding mds.i on smithi103 2024-04-17T05:58:31.289 INFO:tasks.cephadm:Adding mds.l on smithi103 2024-04-17T05:58:31.289 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 565db2d8-fc7e-11ee-bc91-c7b262605968 -- ceph orch apply mds all '12;smithi047=a;smithi047=d;smithi047=g;smithi047=j;smithi098=b;smithi098=e;smithi098=h;smithi098=k;smithi103=c;smithi103=f;smithi103=i;smithi103=l' 2024-04-17T05:58:31.544 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/mon.c/config 2024-04-17T05:58:31.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:31 smithi047 ceph-mon[32223]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T05:58:31.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:31 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/2545586157' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-17T05:58:31.665 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:31 smithi103 ceph-mon[36243]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T05:58:31.665 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:31 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/2545586157' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-17T05:58:31.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:31 smithi098 ceph-mon[36516]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T05:58:31.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:31 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/2545586157' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-17T05:58:33.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:58:33.146+0000 7f1463024700 1 -- 172.21.15.103:0/3673739505 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f145c1019c0 msgr2=0x7f145c101de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:58:33.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:58:33.146+0000 7f1463024700 1 --2- 172.21.15.103:0/3673739505 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f145c1019c0 0x7f145c101de0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1450009990 tx=0x7f1450009ca0 comp rx=0 tx=0).stop 2024-04-17T05:58:33.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:58:33.146+0000 7f1463024700 1 -- 172.21.15.103:0/3673739505 shutdown_connections 2024-04-17T05:58:33.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:58:33.146+0000 7f1463024700 1 --2- 172.21.15.103:0/3673739505 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f145c103560 0x7f145c109e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:33.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:58:33.146+0000 7f1463024700 1 --2- 172.21.15.103:0/3673739505 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f145c1019c0 0x7f145c101de0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:33.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:58:33.146+0000 7f1463024700 1 --2- 172.21.15.103:0/3673739505 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f145c102bc0 0x7f145c103020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:33.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:58:33.146+0000 7f1463024700 1 -- 172.21.15.103:0/3673739505 >> 172.21.15.103:0/3673739505 conn(0x7f145c0fcf80 msgr2=0x7f145c0ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:58:33.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:58:33.146+0000 7f1463024700 1 -- 172.21.15.103:0/3673739505 shutdown_connections 2024-04-17T05:58:33.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:58:33.146+0000 7f1463024700 1 -- 172.21.15.103:0/3673739505 wait complete. 2024-04-17T05:58:33.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:58:33.146+0000 7f1463024700 1 Processor -- start 2024-04-17T05:58:33.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:58:33.146+0000 7f1463024700 1 -- start start 2024-04-17T05:58:33.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:58:33.147+0000 7f1463024700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f145c1019c0 0x7f145c06bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:58:33.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:58:33.147+0000 7f1463024700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f145c102bc0 0x7f145c19da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:58:33.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:58:33.147+0000 7f1463024700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f145c103560 0x7f145c1a00d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:58:33.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:58:33.147+0000 7f1463024700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f145c10d580 con 0x7f145c1019c0 2024-04-17T05:58:33.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:58:33.147+0000 7f1463024700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f145c10d400 con 0x7f145c103560 2024-04-17T05:58:33.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:58:33.147+0000 7f1463024700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f145c10d700 con 0x7f145c102bc0 2024-04-17T05:58:33.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:58:33.147+0000 7f145bfff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f145c102bc0 0x7f145c19da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:58:33.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:58:33.147+0000 7f145bfff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f145c102bc0 0x7f145c19da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34002/0 (socket says 172.21.15.103:34002) 2024-04-17T05:58:33.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:58:33.147+0000 7f145bfff700 1 -- 172.21.15.103:0/971732755 learned_addr learned my addr 172.21.15.103:0/971732755 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-04-17T05:58:33.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:58:33.147+0000 7f1460dc0700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f145c1019c0 0x7f145c06bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:58:33.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:58:33.147+0000 7f14615c1700 1 --2- 172.21.15.103:0/971732755 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f145c103560 0x7f145c1a00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:58:33.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:58:33.147+0000 7f145bfff700 1 -- 172.21.15.103:0/971732755 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f145c103560 msgr2=0x7f145c1a00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:58:33.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:58:33.147+0000 7f145bfff700 1 --2- 172.21.15.103:0/971732755 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f145c103560 0x7f145c1a00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:33.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:58:33.147+0000 7f145bfff700 1 -- 172.21.15.103:0/971732755 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f145c1019c0 msgr2=0x7f145c06bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:58:33.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:58:33.147+0000 7f145bfff700 1 --2- 172.21.15.103:0/971732755 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f145c1019c0 0x7f145c06bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:33.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:58:33.147+0000 7f145bfff700 1 -- 172.21.15.103:0/971732755 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f145c1a09a0 con 0x7f145c102bc0 2024-04-17T05:58:33.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:58:33.147+0000 7f145bfff700 1 --2- 172.21.15.103:0/971732755 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f145c102bc0 0x7f145c19da90 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f144800d950 tx=0x7f144800dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:58:33.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:58:33.147+0000 7f1459ffb700 1 -- 172.21.15.103:0/971732755 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1448016070 con 0x7f145c102bc0 2024-04-17T05:58:33.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:58:33.147+0000 7f1459ffb700 1 -- 172.21.15.103:0/971732755 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1448009e20 con 0x7f145c102bc0 2024-04-17T05:58:33.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:58:33.147+0000 7f1459ffb700 1 -- 172.21.15.103:0/971732755 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f144800f620 con 0x7f145c102bc0 2024-04-17T05:58:33.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:58:33.147+0000 7f1463024700 1 -- 172.21.15.103:0/971732755 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f145c1a0c90 con 0x7f145c102bc0 2024-04-17T05:58:33.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:58:33.148+0000 7f1463024700 1 -- 172.21.15.103:0/971732755 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f145c1a1180 con 0x7f145c102bc0 2024-04-17T05:58:33.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:58:33.149+0000 7f1463024700 1 -- 172.21.15.103:0/971732755 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f145c069740 con 0x7f145c102bc0 2024-04-17T05:58:33.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:58:33.150+0000 7f1459ffb700 1 -- 172.21.15.103:0/971732755 <== mon.1 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f1448019070 con 0x7f145c102bc0 2024-04-17T05:58:33.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:58:33.150+0000 7f1459ffb700 1 --2- 172.21.15.103:0/971732755 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f144c0a3c40 0x7f144c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:58:33.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:58:33.151+0000 7f1459ffb700 1 -- 172.21.15.103:0/971732755 <== mon.1 v2:172.21.15.103:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f14480cfb00 con 0x7f145c102bc0 2024-04-17T05:58:33.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:58:33.151+0000 7f1460dc0700 1 --2- 172.21.15.103:0/971732755 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f144c0a3c40 0x7f144c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:58:33.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:58:33.151+0000 7f1460dc0700 1 --2- 172.21.15.103:0/971732755 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f144c0a3c40 0x7f144c0a6100 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f145000b5c0 tx=0x7f1450003960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:58:33.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:58:33.155+0000 7f1459ffb700 1 -- 172.21.15.103:0/971732755 <== mon.1 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f144809c980 con 0x7f145c102bc0 2024-04-17T05:58:33.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:58:33.366+0000 7f1463024700 1 -- 172.21.15.103:0/971732755 --> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi047=a;smithi047=d;smithi047=g;smithi047=j;smithi098=b;smithi098=e;smithi098=h;smithi098=k;smithi103=c;smithi103=f;smithi103=i;smithi103=l", "target": ["mon-mgr", ""]}) -- 0x7f145c062f90 con 0x7f144c0a3c40 2024-04-17T05:58:33.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:58:33.374+0000 7f1459ffb700 1 -- 172.21.15.103:0/971732755 <== mgr.14152 v2:172.21.15.47:6800/3157902806 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f145c062f90 con 0x7f144c0a3c40 2024-04-17T05:58:33.373 INFO:teuthology.orchestra.run.smithi103.stdout:Scheduled mds.all update... 2024-04-17T05:58:33.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:58:33.377+0000 7f1463024700 1 -- 172.21.15.103:0/971732755 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f144c0a3c40 msgr2=0x7f144c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:58:33.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:58:33.377+0000 7f1463024700 1 --2- 172.21.15.103:0/971732755 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f144c0a3c40 0x7f144c0a6100 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f145000b5c0 tx=0x7f1450003960 comp rx=0 tx=0).stop 2024-04-17T05:58:33.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:58:33.377+0000 7f1463024700 1 -- 172.21.15.103:0/971732755 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f145c102bc0 msgr2=0x7f145c19da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:58:33.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:58:33.377+0000 7f1463024700 1 --2- 172.21.15.103:0/971732755 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f145c102bc0 0x7f145c19da90 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f144800d950 tx=0x7f144800dc60 comp rx=0 tx=0).stop 2024-04-17T05:58:33.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:58:33.377+0000 7f1463024700 1 -- 172.21.15.103:0/971732755 shutdown_connections 2024-04-17T05:58:33.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:58:33.377+0000 7f1463024700 1 --2- 172.21.15.103:0/971732755 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f144c0a3c40 0x7f144c0a6100 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:33.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:58:33.377+0000 7f1463024700 1 --2- 172.21.15.103:0/971732755 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f145c103560 0x7f145c1a00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:33.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:58:33.377+0000 7f1463024700 1 --2- 172.21.15.103:0/971732755 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f145c1019c0 0x7f145c06bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:33.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:58:33.377+0000 7f1463024700 1 --2- 172.21.15.103:0/971732755 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f145c102bc0 0x7f145c19da90 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:33.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:58:33.377+0000 7f1463024700 1 -- 172.21.15.103:0/971732755 >> 172.21.15.103:0/971732755 conn(0x7f145c0fcf80 msgr2=0x7f145c106030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:58:33.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:58:33.377+0000 7f1463024700 1 -- 172.21.15.103:0/971732755 shutdown_connections 2024-04-17T05:58:33.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T05:58:33.377+0000 7f1463024700 1 -- 172.21.15.103:0/971732755 wait complete. 2024-04-17T05:58:33.700 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:33 smithi103 ceph-mon[36243]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T05:58:33.701 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:33 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:33.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:33 smithi098 ceph-mon[36516]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T05:58:33.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:33 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:33.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:33 smithi047 ceph-mon[32223]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T05:58:33.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:33 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:34.005 DEBUG:teuthology.orchestra.run.smithi047:mds.a> sudo journalctl -f -n 0 -u ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@mds.a.service 2024-04-17T05:58:34.008 DEBUG:teuthology.orchestra.run.smithi047:mds.d> sudo journalctl -f -n 0 -u ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@mds.d.service 2024-04-17T05:58:34.051 DEBUG:teuthology.orchestra.run.smithi047:mds.g> sudo journalctl -f -n 0 -u ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@mds.g.service 2024-04-17T05:58:34.094 DEBUG:teuthology.orchestra.run.smithi047:mds.j> sudo journalctl -f -n 0 -u ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@mds.j.service 2024-04-17T05:58:34.137 DEBUG:teuthology.orchestra.run.smithi098:mds.b> sudo journalctl -f -n 0 -u ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@mds.b.service 2024-04-17T05:58:34.140 DEBUG:teuthology.orchestra.run.smithi098:mds.e> sudo journalctl -f -n 0 -u ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@mds.e.service 2024-04-17T05:58:34.183 DEBUG:teuthology.orchestra.run.smithi098:mds.h> sudo journalctl -f -n 0 -u ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@mds.h.service 2024-04-17T05:58:34.226 DEBUG:teuthology.orchestra.run.smithi098:mds.k> sudo journalctl -f -n 0 -u ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@mds.k.service 2024-04-17T05:58:34.269 DEBUG:teuthology.orchestra.run.smithi103:mds.c> sudo journalctl -f -n 0 -u ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@mds.c.service 2024-04-17T05:58:34.271 DEBUG:teuthology.orchestra.run.smithi103:mds.f> sudo journalctl -f -n 0 -u ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@mds.f.service 2024-04-17T05:58:34.314 DEBUG:teuthology.orchestra.run.smithi103:mds.i> sudo journalctl -f -n 0 -u ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@mds.i.service 2024-04-17T05:58:34.357 DEBUG:teuthology.orchestra.run.smithi103:mds.l> sudo journalctl -f -n 0 -u ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@mds.l.service 2024-04-17T05:58:34.400 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-04-17T05:58:34.401 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T05:58:34.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:34.530+0000 7fef4e352640 1 -- 172.21.15.47:0/2624182527 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef48101fb0 msgr2=0x7fef481023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:58:34.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:34.530+0000 7fef4e352640 1 --2- 172.21.15.47:0/2624182527 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef48101fb0 0x7fef481023d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fef38009900 tx=0x7fef3802bc30 comp rx=0 tx=0).stop 2024-04-17T05:58:34.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:34.530+0000 7fef4e352640 1 -- 172.21.15.47:0/2624182527 shutdown_connections 2024-04-17T05:58:34.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:34.530+0000 7fef4e352640 1 --2- 172.21.15.47:0/2624182527 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fef48103b60 0x7fef4810a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:34.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:34.530+0000 7fef4e352640 1 --2- 172.21.15.47:0/2624182527 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fef481031a0 0x7fef48103620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:34.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:34.530+0000 7fef4e352640 1 --2- 172.21.15.47:0/2624182527 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef48101fb0 0x7fef481023d0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:34.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:34.530+0000 7fef4e352640 1 -- 172.21.15.47:0/2624182527 >> 172.21.15.47:0/2624182527 conn(0x7fef480fd540 msgr2=0x7fef480ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:58:34.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:34.530+0000 7fef4e352640 1 -- 172.21.15.47:0/2624182527 shutdown_connections 2024-04-17T05:58:34.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:34.530+0000 7fef4e352640 1 -- 172.21.15.47:0/2624182527 wait complete. 2024-04-17T05:58:34.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:34.531+0000 7fef4e352640 1 Processor -- start 2024-04-17T05:58:34.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:34.531+0000 7fef4e352640 1 -- start start 2024-04-17T05:58:34.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:34.531+0000 7fef4e352640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fef48101fb0 0x7fef481982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:58:34.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:34.531+0000 7fef4e352640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fef481031a0 0x7fef481987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:58:34.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:34.531+0000 7fef4e352640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef48103b60 0x7fef4819f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:58:34.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:34.531+0000 7fef4e352640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fef4810dc90 con 0x7fef48103b60 2024-04-17T05:58:34.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:34.531+0000 7fef4e352640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fef4810db10 con 0x7fef48101fb0 2024-04-17T05:58:34.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:34.531+0000 7fef4e352640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fef4810de10 con 0x7fef481031a0 2024-04-17T05:58:34.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:34.531+0000 7fef4c8ef640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef48103b60 0x7fef4819f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:58:34.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:34.531+0000 7fef4c8ef640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef48103b60 0x7fef4819f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45660/0 (socket says 172.21.15.47:45660) 2024-04-17T05:58:34.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:34.531+0000 7fef4c8ef640 1 -- 172.21.15.47:0/1552042968 learned_addr learned my addr 172.21.15.47:0/1552042968 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:58:34.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:34.531+0000 7fef477fe640 1 --2- 172.21.15.47:0/1552042968 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fef481031a0 0x7fef481987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:58:34.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:34.532+0000 7fef47fff640 1 --2- 172.21.15.47:0/1552042968 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fef48101fb0 0x7fef481982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:58:34.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:34.532+0000 7fef4c8ef640 1 -- 172.21.15.47:0/1552042968 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fef481031a0 msgr2=0x7fef481987f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:58:34.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:34.532+0000 7fef4c8ef640 1 --2- 172.21.15.47:0/1552042968 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fef481031a0 0x7fef481987f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:34.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:34.532+0000 7fef4c8ef640 1 -- 172.21.15.47:0/1552042968 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fef48101fb0 msgr2=0x7fef481982b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:58:34.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:34.532+0000 7fef4c8ef640 1 --2- 172.21.15.47:0/1552042968 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fef48101fb0 0x7fef481982b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:34.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:34.532+0000 7fef4c8ef640 1 -- 172.21.15.47:0/1552042968 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef4819fed0 con 0x7fef48103b60 2024-04-17T05:58:34.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:34.532+0000 7fef4c8ef640 1 --2- 172.21.15.47:0/1552042968 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef48103b60 0x7fef4819f860 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fef4000eb10 tx=0x7fef4000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:58:34.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:34.532+0000 7fef457fa640 1 -- 172.21.15.47:0/1552042968 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef40004b20 con 0x7fef48103b60 2024-04-17T05:58:34.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:34.532+0000 7fef457fa640 1 -- 172.21.15.47:0/1552042968 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef40004cc0 con 0x7fef48103b60 2024-04-17T05:58:34.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:34.532+0000 7fef4e352640 1 -- 172.21.15.47:0/1552042968 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef481a01c0 con 0x7fef48103b60 2024-04-17T05:58:34.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:34.532+0000 7fef457fa640 1 -- 172.21.15.47:0/1552042968 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef40010760 con 0x7fef48103b60 2024-04-17T05:58:34.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:34.532+0000 7fef4e352640 1 -- 172.21.15.47:0/1552042968 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef481a0710 con 0x7fef48103b60 2024-04-17T05:58:34.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:34.533+0000 7fef4e352640 1 -- 172.21.15.47:0/1552042968 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef481092d0 con 0x7fef48103b60 2024-04-17T05:58:34.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:34.534+0000 7fef457fa640 1 -- 172.21.15.47:0/1552042968 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fef40005020 con 0x7fef48103b60 2024-04-17T05:58:34.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:34.535+0000 7fef457fa640 1 --2- 172.21.15.47:0/1552042968 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fef300a3bb0 0x7fef300a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:58:34.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:34.535+0000 7fef47fff640 1 --2- 172.21.15.47:0/1552042968 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fef300a3bb0 0x7fef300a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:58:34.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:34.535+0000 7fef457fa640 1 -- 172.21.15.47:0/1552042968 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fef400d06a0 con 0x7fef48103b60 2024-04-17T05:58:34.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:34.535+0000 7fef47fff640 1 --2- 172.21.15.47:0/1552042968 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fef300a3bb0 0x7fef300a6070 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fef380096c0 tx=0x7fef3802df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:58:34.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:34.540+0000 7fef457fa640 1 -- 172.21.15.47:0/1552042968 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fef4009d520 con 0x7fef48103b60 2024-04-17T05:58:34.679 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:34 smithi103 ceph-mon[36243]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi047=a;smithi047=d;smithi047=g;smithi047=j;smithi098=b;smithi098=e;smithi098=h;smithi098=k;smithi103=c;smithi103=f;smithi103=i;smithi103=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T05:58:34.679 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:34 smithi103 ceph-mon[36243]: Saving service mds.all spec with placement smithi047=a;smithi047=d;smithi047=g;smithi047=j;smithi098=b;smithi098=e;smithi098=h;smithi098=k;smithi103=c;smithi103=f;smithi103=i;smithi103=l;count:12 2024-04-17T05:58:34.679 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:34 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T05:58:34.679 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:34 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:58:34.679 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:34 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T05:58:34.679 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:34 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:34.679 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:34 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-17T05:58:34.679 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:34 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-17T05:58:34.679 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:34 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:58:34.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:34.728+0000 7fef4e352640 1 -- 172.21.15.47:0/1552042968 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fef48062b60 con 0x7fef48103b60 2024-04-17T05:58:34.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:34.729+0000 7fef457fa640 1 -- 172.21.15.47:0/1552042968 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18406 (secure 0 0 0) 0x7fef4009ce20 con 0x7fef48103b60 2024-04-17T05:58:34.728 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-17T05:58:34.728 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":82,"fsid":"565db2d8-fc7e-11ee-bc91-c7b262605968","created":"2024-04-17T05:51:19.888865+0000","modified":"2024-04-17T05:58:25.403856+0000","last_up_change":"2024-04-17T05:58:24.380131+0000","last_in_change":"2024-04-17T05:58:04.418236+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T05:54:37.734306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8453f300-533d-4736-a343-55619c08e966","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":1290127544},{"type":"v1","addr":"172.21.15.47:6803","nonce":1290127544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":1290127544},{"type":"v1","addr":"172.21.15.47:6805","nonce":1290127544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":1290127544},{"type":"v1","addr":"172.21.15.47:6809","nonce":1290127544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":1290127544},{"type":"v1","addr":"172.21.15.47:6807","nonce":1290127544}]},"public_addr":"172.21.15.47:6803/1290127544","cluster_addr":"172.21.15.47:6805/1290127544","heartbeat_back_addr":"172.21.15.47:6809/1290127544","heartbeat_front_addr":"172.21.15.47:6807/1290127544","state":["exists","up"]},{"osd":1,"uuid":"80b0f03e-c1bc-4658-8028-1e6b8eedcc47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":1180430954},{"type":"v1","addr":"172.21.15.98:6801","nonce":1180430954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":1180430954},{"type":"v1","addr":"172.21.15.98:6803","nonce":1180430954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":1180430954},{"type":"v1","addr":"172.21.15.98:6807","nonce":1180430954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":1180430954},{"type":"v1","addr":"172.21.15.98:6805","nonce":1180430954}]},"public_addr":"172.21.15.98:6801/1180430954","cluster_addr":"172.21.15.98:6803/1180430954","heartbeat_back_addr":"172.21.15.98:6807/1180430954","heartbeat_front_addr":"172.21.15.98:6805/1180430954","state":["exists","up"]},{"osd":2,"uuid":"0fc9d383-05b4-498d-b530-fccf7aeddd7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":494676216},{"type":"v1","addr":"172.21.15.103:6801","nonce":494676216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":494676216},{"type":"v1","addr":"172.21.15.103:6803","nonce":494676216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":494676216},{"type":"v1","addr":"172.21.15.103:6807","nonce":494676216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":494676216},{"type":"v1","addr":"172.21.15.103:6805","nonce":494676216}]},"public_addr":"172.21.15.103:6801/494676216","cluster_addr":"172.21.15.103:6803/494676216","heartbeat_back_addr":"172.21.15.103:6807/494676216","heartbeat_front_addr":"172.21.15.103:6805/494676216","state":["exists","up"]},{"osd":3,"uuid":"f6e0b0d5-0fb3-42d0-9e84-f7cfef21a998","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":1998369447},{"type":"v1","addr":"172.21.15.47:6811","nonce":1998369447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":1998369447},{"type":"v1","addr":"172.21.15.47:6813","nonce":1998369447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":1998369447},{"type":"v1","addr":"172.21.15.47:6817","nonce":1998369447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":1998369447},{"type":"v1","addr":"172.21.15.47:6815","nonce":1998369447}]},"public_addr":"172.21.15.47:6811/1998369447","cluster_addr":"172.21.15.47:6813/1998369447","heartbeat_back_addr":"172.21.15.47:6817/1998369447","heartbeat_front_addr":"172.21.15.47:6815/1998369447","state":["exists","up"]},{"osd":4,"uuid":"2c26217e-02df-466d-bc01-0149d4aec3c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":3060482485},{"type":"v1","addr":"172.21.15.98:6809","nonce":3060482485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":3060482485},{"type":"v1","addr":"172.21.15.98:6811","nonce":3060482485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":3060482485},{"type":"v1","addr":"172.21.15.98:6815","nonce":3060482485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":3060482485},{"type":"v1","addr":"172.21.15.98:6813","nonce":3060482485}]},"public_addr":"172.21.15.98:6809/3060482485","cluster_addr":"172.21.15.98:6811/3060482485","heartbeat_back_addr":"172.21.15.98:6815/3060482485","heartbeat_front_addr":"172.21.15.98:6813/3060482485","state":["exists","up"]},{"osd":5,"uuid":"740bf3e0-bea8-496b-8d6f-e42d0b017258","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":4181363567},{"type":"v1","addr":"172.21.15.103:6809","nonce":4181363567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":4181363567},{"type":"v1","addr":"172.21.15.103:6811","nonce":4181363567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":4181363567},{"type":"v1","addr":"172.21.15.103:6815","nonce":4181363567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":4181363567},{"type":"v1","addr":"172.21.15.103:6813","nonce":4181363567}]},"public_addr":"172.21.15.103:6809/4181363567","cluster_addr":"172.21.15.103:6811/4181363567","heartbeat_back_addr":"172.21.15.103:6815/4181363567","heartbeat_front_addr":"172.21.15.103:6813/4181363567","state":["exists","up"]},{"osd":6,"uuid":"356a6255-2c07-4f0d-a233-26be04c92b38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":2517529668},{"type":"v1","addr":"172.21.15.47:6819","nonce":2517529668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2517529668},{"type":"v1","addr":"172.21.15.47:6821","nonce":2517529668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2517529668},{"type":"v1","addr":"172.21.15.47:6825","nonce":2517529668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2517529668},{"type":"v1","addr":"172.21.15.47:6823","nonce":2517529668}]},"public_addr":"172.21.15.47:6819/2517529668","cluster_addr":"172.21.15.47:6821/2517529668","heartbeat_back_addr":"172.21.15.47:6825/2517529668","heartbeat_front_addr":"172.21.15.47:6823/2517529668","state":["exists","up"]},{"osd":7,"uuid":"349d0120-8290-417d-a800-5a8985f554f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":4225498496},{"type":"v1","addr":"172.21.15.98:6817","nonce":4225498496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":4225498496},{"type":"v1","addr":"172.21.15.98:6819","nonce":4225498496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":4225498496},{"type":"v1","addr":"172.21.15.98:6823","nonce":4225498496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":4225498496},{"type":"v1","addr":"172.21.15.98:6821","nonce":4225498496}]},"public_addr":"172.21.15.98:6817/4225498496","cluster_addr":"172.21.15.98:6819/4225498496","heartbeat_back_addr":"172.21.15.98:6823/4225498496","heartbeat_front_addr":"172.21.15.98:6821/4225498496","state":["exists","up"]},{"osd":8,"uuid":"cee4712f-321b-42c0-92ba-e15144c00f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":3433905520},{"type":"v1","addr":"172.21.15.103:6817","nonce":3433905520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":3433905520},{"type":"v1","addr":"172.21.15.103:6819","nonce":3433905520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":3433905520},{"type":"v1","addr":"172.21.15.103:6823","nonce":3433905520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":3433905520},{"type":"v1","addr":"172.21.15.103:6821","nonce":3433905520}]},"public_addr":"172.21.15.103:6817/3433905520","cluster_addr":"172.21.15.103:6819/3433905520","heartbeat_back_addr":"172.21.15.103:6823/3433905520","heartbeat_front_addr":"172.21.15.103:6821/3433905520","state":["exists","up"]},{"osd":9,"uuid":"63fc73c1-5a22-487d-80b8-9894e2ddd207","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":728359554},{"type":"v1","addr":"172.21.15.47:6827","nonce":728359554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":728359554},{"type":"v1","addr":"172.21.15.47:6829","nonce":728359554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":728359554},{"type":"v1","addr":"172.21.15.47:6833","nonce":728359554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":728359554},{"type":"v1","addr":"172.21.15.47:6831","nonce":728359554}]},"public_addr":"172.21.15.47:6827/728359554","cluster_addr":"172.21.15.47:6829/728359554","heartbeat_back_addr":"172.21.15.47:6833/728359554","heartbeat_front_addr":"172.21.15.47:6831/728359554","state":["exists","up"]},{"osd":10,"uuid":"9565bcc3-62ee-48eb-a07a-03a81e0cc8e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":468205689},{"type":"v1","addr":"172.21.15.98:6825","nonce":468205689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":468205689},{"type":"v1","addr":"172.21.15.98:6827","nonce":468205689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":468205689},{"type":"v1","addr":"172.21.15.98:6831","nonce":468205689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":468205689},{"type":"v1","addr":"172.21.15.98:6829","nonce":468205689}]},"public_addr":"172.21.15.98:6825/468205689","cluster_addr":"172.21.15.98:6827/468205689","heartbeat_back_addr":"172.21.15.98:6831/468205689","heartbeat_front_addr":"172.21.15.98:6829/468205689","state":["exists","up"]},{"osd":11,"uuid":"2e85e8d9-bf6c-4a12-b124-a4690dd61108","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":2639831013},{"type":"v1","addr":"172.21.15.103:6825","nonce":2639831013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":2639831013},{"type":"v1","addr":"172.21.15.103:6827","nonce":2639831013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":2639831013},{"type":"v1","addr":"172.21.15.103:6831","nonce":2639831013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":2639831013},{"type":"v1","addr":"172.21.15.103:6829","nonce":2639831013}]},"public_addr":"172.21.15.103:6825/2639831013","cluster_addr":"172.21.15.103:6827/2639831013","heartbeat_back_addr":"172.21.15.103:6831/2639831013","heartbeat_front_addr":"172.21.15.103:6829/2639831013","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:53:47.366438+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:54:10.953559+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:54:34.320332+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:54:59.751360+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:55:23.982052+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:55:48.425017+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:56:14.793368+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:56:40.440830+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:57:04.623680+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:57:31.215262+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:57:57.040346+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:58:22.671572+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:6801/3632157175":"2024-04-18T05:52:13.645775+0000","172.21.15.47:0/4288219269":"2024-04-18T05:52:13.645775+0000","172.21.15.47:0/2584487544":"2024-04-18T05:52:13.645775+0000","172.21.15.47:6801/953064786":"2024-04-18T05:51:45.979351+0000","172.21.15.47:0/3484500927":"2024-04-18T05:51:45.979351+0000","172.21.15.47:6800/3632157175":"2024-04-18T05:52:13.645775+0000","172.21.15.47:0/112268670":"2024-04-18T05:51:45.979351+0000","172.21.15.47:0/1638423256":"2024-04-18T05:51:45.979351+0000","172.21.15.47:6800/953064786":"2024-04-18T05:51:45.979351+0000","172.21.15.47:0/849479010":"2024-04-18T05:52:13.645775+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T05:58:34.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:34.731+0000 7fef4e352640 1 -- 172.21.15.47:0/1552042968 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fef300a3bb0 msgr2=0x7fef300a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:58:34.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:34.731+0000 7fef4e352640 1 --2- 172.21.15.47:0/1552042968 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fef300a3bb0 0x7fef300a6070 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fef380096c0 tx=0x7fef3802df90 comp rx=0 tx=0).stop 2024-04-17T05:58:34.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:34.731+0000 7fef4e352640 1 -- 172.21.15.47:0/1552042968 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef48103b60 msgr2=0x7fef4819f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:58:34.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:34.731+0000 7fef4e352640 1 --2- 172.21.15.47:0/1552042968 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef48103b60 0x7fef4819f860 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fef4000eb10 tx=0x7fef4000c2d0 comp rx=0 tx=0).stop 2024-04-17T05:58:34.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:34.731+0000 7fef4e352640 1 -- 172.21.15.47:0/1552042968 shutdown_connections 2024-04-17T05:58:34.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:34.731+0000 7fef4e352640 1 --2- 172.21.15.47:0/1552042968 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fef300a3bb0 0x7fef300a6070 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:34.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:34.731+0000 7fef4e352640 1 --2- 172.21.15.47:0/1552042968 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef48103b60 0x7fef4819f860 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:34.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:34.731+0000 7fef4e352640 1 --2- 172.21.15.47:0/1552042968 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fef481031a0 0x7fef481987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:34.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:34.731+0000 7fef4e352640 1 --2- 172.21.15.47:0/1552042968 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fef48101fb0 0x7fef481982b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:34.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:34.731+0000 7fef4e352640 1 -- 172.21.15.47:0/1552042968 >> 172.21.15.47:0/1552042968 conn(0x7fef480fd540 msgr2=0x7fef481064a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:58:34.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:34.731+0000 7fef4e352640 1 -- 172.21.15.47:0/1552042968 shutdown_connections 2024-04-17T05:58:34.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:34.731+0000 7fef4e352640 1 -- 172.21.15.47:0/1552042968 wait complete. 2024-04-17T05:58:34.739 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:34 smithi047 ceph-mon[32223]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi047=a;smithi047=d;smithi047=g;smithi047=j;smithi098=b;smithi098=e;smithi098=h;smithi098=k;smithi103=c;smithi103=f;smithi103=i;smithi103=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T05:58:34.740 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:34 smithi047 ceph-mon[32223]: Saving service mds.all spec with placement smithi047=a;smithi047=d;smithi047=g;smithi047=j;smithi098=b;smithi098=e;smithi098=h;smithi098=k;smithi103=c;smithi103=f;smithi103=i;smithi103=l;count:12 2024-04-17T05:58:34.740 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:34 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T05:58:34.740 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:34 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:58:34.740 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:34 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T05:58:34.740 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:34 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:34.740 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:34 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-17T05:58:34.740 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:34 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-17T05:58:34.740 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:34 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:58:34.741 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T05:54:37.734306+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T05:58:34.742 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T05:58:34.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:34.851+0000 7fb550915640 1 -- 172.21.15.47:0/2114351006 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb54c102050 msgr2=0x7fb54c1024d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:58:34.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:34.851+0000 7fb550915640 1 --2- 172.21.15.47:0/2114351006 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb54c102050 0x7fb54c1024d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb534009900 tx=0x7fb53402bc30 comp rx=0 tx=0).stop 2024-04-17T05:58:34.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:34.851+0000 7fb550915640 1 -- 172.21.15.47:0/2114351006 shutdown_connections 2024-04-17T05:58:34.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:34.851+0000 7fb550915640 1 --2- 172.21.15.47:0/2114351006 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb54c102a10 0x7fb54c109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:34.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:34.851+0000 7fb550915640 1 --2- 172.21.15.47:0/2114351006 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb54c102050 0x7fb54c1024d0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:34.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:34.851+0000 7fb550915640 1 --2- 172.21.15.47:0/2114351006 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb54c100e70 0x7fb54c101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:34.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:34.851+0000 7fb550915640 1 -- 172.21.15.47:0/2114351006 >> 172.21.15.47:0/2114351006 conn(0x7fb54c0fc430 msgr2=0x7fb54c0fe870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:58:34.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:34.851+0000 7fb550915640 1 -- 172.21.15.47:0/2114351006 shutdown_connections 2024-04-17T05:58:34.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:34.851+0000 7fb550915640 1 -- 172.21.15.47:0/2114351006 wait complete. 2024-04-17T05:58:34.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:34.852+0000 7fb550915640 1 Processor -- start 2024-04-17T05:58:34.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:34.852+0000 7fb550915640 1 -- start start 2024-04-17T05:58:34.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:34.852+0000 7fb550915640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb54c100e70 0x7fb54c06b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:58:34.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:34.852+0000 7fb550915640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb54c102050 0x7fb54c19d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:58:34.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:34.852+0000 7fb550915640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb54c102a10 0x7fb54c19f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:58:34.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:34.852+0000 7fb550915640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb54c10cb70 con 0x7fb54c100e70 2024-04-17T05:58:34.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:34.852+0000 7fb550915640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb54c10c9f0 con 0x7fb54c102a10 2024-04-17T05:58:34.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:34.852+0000 7fb550915640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb54c10ccf0 con 0x7fb54c102050 2024-04-17T05:58:34.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:34.852+0000 7fb54a59c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb54c100e70 0x7fb54c06b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:58:34.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:34.852+0000 7fb54a59c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb54c100e70 0x7fb54c06b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45666/0 (socket says 172.21.15.47:45666) 2024-04-17T05:58:34.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:34.852+0000 7fb54a59c640 1 -- 172.21.15.47:0/1311708757 learned_addr learned my addr 172.21.15.47:0/1311708757 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:58:34.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:34.852+0000 7fb54a59c640 1 -- 172.21.15.47:0/1311708757 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb54c102050 msgr2=0x7fb54c19d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:58:34.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:34.852+0000 7fb54a59c640 1 --2- 172.21.15.47:0/1311708757 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb54c102050 0x7fb54c19d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:34.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:34.852+0000 7fb54a59c640 1 -- 172.21.15.47:0/1311708757 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb54c102a10 msgr2=0x7fb54c19f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:58:34.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:34.853+0000 7fb54ad9d640 1 --2- 172.21.15.47:0/1311708757 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb54c102a10 0x7fb54c19f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:58:34.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:34.853+0000 7fb54a59c640 1 --2- 172.21.15.47:0/1311708757 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb54c102a10 0x7fb54c19f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:34.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:34.853+0000 7fb54a59c640 1 -- 172.21.15.47:0/1311708757 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb54c19ff60 con 0x7fb54c100e70 2024-04-17T05:58:34.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:34.853+0000 7fb54a59c640 1 --2- 172.21.15.47:0/1311708757 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb54c100e70 0x7fb54c06b0d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb53c00c970 tx=0x7fb53c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:58:34.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:34.853+0000 7fb5437fe640 1 -- 172.21.15.47:0/1311708757 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb53c004220 con 0x7fb54c100e70 2024-04-17T05:58:34.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:34.853+0000 7fb5437fe640 1 -- 172.21.15.47:0/1311708757 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb53c0043c0 con 0x7fb54c100e70 2024-04-17T05:58:34.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:34.853+0000 7fb550915640 1 -- 172.21.15.47:0/1311708757 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb54c1a01f0 con 0x7fb54c100e70 2024-04-17T05:58:34.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:34.853+0000 7fb5437fe640 1 -- 172.21.15.47:0/1311708757 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb53c004a00 con 0x7fb54c100e70 2024-04-17T05:58:34.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:34.853+0000 7fb550915640 1 -- 172.21.15.47:0/1311708757 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb54c1a0740 con 0x7fb54c100e70 2024-04-17T05:58:34.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:34.854+0000 7fb550915640 1 -- 172.21.15.47:0/1311708757 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb54c108260 con 0x7fb54c100e70 2024-04-17T05:58:34.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:34.855+0000 7fb5437fe640 1 -- 172.21.15.47:0/1311708757 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb53c0079b0 con 0x7fb54c100e70 2024-04-17T05:58:34.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:34.856+0000 7fb5437fe640 1 --2- 172.21.15.47:0/1311708757 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fb5380a3c30 0x7fb5380a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:58:34.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:34.856+0000 7fb543fff640 1 --2- 172.21.15.47:0/1311708757 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fb5380a3c30 0x7fb5380a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:58:34.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:34.856+0000 7fb5437fe640 1 -- 172.21.15.47:0/1311708757 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fb53c0cfda0 con 0x7fb54c100e70 2024-04-17T05:58:34.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:34.856+0000 7fb543fff640 1 --2- 172.21.15.47:0/1311708757 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fb5380a3c30 0x7fb5380a60f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb534036040 tx=0x7fb534000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:58:34.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:34.860+0000 7fb5437fe640 1 -- 172.21.15.47:0/1311708757 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fb53c014030 con 0x7fb54c100e70 2024-04-17T05:58:34.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:34 smithi098 ceph-mon[36516]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi047=a;smithi047=d;smithi047=g;smithi047=j;smithi098=b;smithi098=e;smithi098=h;smithi098=k;smithi103=c;smithi103=f;smithi103=i;smithi103=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T05:58:34.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:34 smithi098 ceph-mon[36516]: Saving service mds.all spec with placement smithi047=a;smithi047=d;smithi047=g;smithi047=j;smithi098=b;smithi098=e;smithi098=h;smithi098=k;smithi103=c;smithi103=f;smithi103=i;smithi103=l;count:12 2024-04-17T05:58:34.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:34 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T05:58:34.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:34 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:58:34.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:34 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T05:58:34.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:34 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:34.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:34 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-17T05:58:34.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:34 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-17T05:58:34.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:34 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:58:35.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:35.046+0000 7fb550915640 1 -- 172.21.15.47:0/1311708757 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb54c062b60 con 0x7fb54c100e70 2024-04-17T05:58:35.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:35.046+0000 7fb5437fe640 1 -- 172.21.15.47:0/1311708757 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7fb53c09cb80 con 0x7fb54c100e70 2024-04-17T05:58:35.046 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-04-17T05:58:35.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:35.048+0000 7fb550915640 1 -- 172.21.15.47:0/1311708757 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fb5380a3c30 msgr2=0x7fb5380a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:58:35.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:35.048+0000 7fb550915640 1 --2- 172.21.15.47:0/1311708757 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fb5380a3c30 0x7fb5380a60f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb534036040 tx=0x7fb534000bc0 comp rx=0 tx=0).stop 2024-04-17T05:58:35.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:35.048+0000 7fb550915640 1 -- 172.21.15.47:0/1311708757 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb54c100e70 msgr2=0x7fb54c06b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:58:35.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:35.048+0000 7fb550915640 1 --2- 172.21.15.47:0/1311708757 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb54c100e70 0x7fb54c06b0d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb53c00c970 tx=0x7fb53c00ce30 comp rx=0 tx=0).stop 2024-04-17T05:58:35.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:35.048+0000 7fb550915640 1 -- 172.21.15.47:0/1311708757 shutdown_connections 2024-04-17T05:58:35.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:35.048+0000 7fb550915640 1 --2- 172.21.15.47:0/1311708757 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fb5380a3c30 0x7fb5380a60f0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:35.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:35.048+0000 7fb550915640 1 --2- 172.21.15.47:0/1311708757 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb54c102a10 0x7fb54c19f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:35.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:35.048+0000 7fb550915640 1 --2- 172.21.15.47:0/1311708757 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb54c102050 0x7fb54c19d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:35.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:35.048+0000 7fb550915640 1 --2- 172.21.15.47:0/1311708757 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb54c100e70 0x7fb54c06b0d0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:35.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:35.048+0000 7fb550915640 1 -- 172.21.15.47:0/1311708757 >> 172.21.15.47:0/1311708757 conn(0x7fb54c0fc430 msgr2=0x7fb54c105430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:58:35.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:35.048+0000 7fb550915640 1 -- 172.21.15.47:0/1311708757 shutdown_connections 2024-04-17T05:58:35.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:35.048+0000 7fb550915640 1 -- 172.21.15.47:0/1311708757 wait complete. 2024-04-17T05:58:35.058 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T05:58:35.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:35.165+0000 7f0987132640 1 -- 172.21.15.47:0/4097205526 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0980102a00 msgr2=0x7f0980109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:58:35.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:35.165+0000 7f0987132640 1 --2- 172.21.15.47:0/4097205526 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0980102a00 0x7f0980109320 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0978009ca0 tx=0x7f097802c0f0 comp rx=0 tx=0).stop 2024-04-17T05:58:35.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:35.165+0000 7f0987132640 1 -- 172.21.15.47:0/4097205526 shutdown_connections 2024-04-17T05:58:35.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:35.165+0000 7f0987132640 1 --2- 172.21.15.47:0/4097205526 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0980102a00 0x7f0980109320 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:35.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:35.165+0000 7f0987132640 1 --2- 172.21.15.47:0/4097205526 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0980102040 0x7f09801024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:35.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:35.165+0000 7f0987132640 1 --2- 172.21.15.47:0/4097205526 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0980100e60 0x7f0980101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:35.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:35.165+0000 7f0987132640 1 -- 172.21.15.47:0/4097205526 >> 172.21.15.47:0/4097205526 conn(0x7f09800fc440 msgr2=0x7f09800fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:58:35.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:35.165+0000 7f0987132640 1 -- 172.21.15.47:0/4097205526 shutdown_connections 2024-04-17T05:58:35.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:35.165+0000 7f0987132640 1 -- 172.21.15.47:0/4097205526 wait complete. 2024-04-17T05:58:35.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:35.166+0000 7f0987132640 1 Processor -- start 2024-04-17T05:58:35.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:35.166+0000 7f0987132640 1 -- start start 2024-04-17T05:58:35.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:35.166+0000 7f0987132640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0980100e60 0x7f0980198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:58:35.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:35.166+0000 7f0987132640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0980102040 0x7f09801987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:58:35.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:35.166+0000 7f0987132640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0980102a00 0x7f098019f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:58:35.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:35.166+0000 7f0987132640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f098010dc80 con 0x7f0980100e60 2024-04-17T05:58:35.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:35.166+0000 7f0987132640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f098010db00 con 0x7f0980102040 2024-04-17T05:58:35.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:35.166+0000 7f0987132640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f098010de00 con 0x7f0980102a00 2024-04-17T05:58:35.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:35.166+0000 7f0984ece640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0980100e60 0x7f0980198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:58:35.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:35.166+0000 7f0984ece640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0980100e60 0x7f0980198270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45686/0 (socket says 172.21.15.47:45686) 2024-04-17T05:58:35.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:35.166+0000 7f0984ece640 1 -- 172.21.15.47:0/1036838256 learned_addr learned my addr 172.21.15.47:0/1036838256 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:58:35.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:35.166+0000 7f0984ece640 1 -- 172.21.15.47:0/1036838256 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0980102a00 msgr2=0x7f098019f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:58:35.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:35.166+0000 7f0984ece640 1 --2- 172.21.15.47:0/1036838256 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0980102a00 0x7f098019f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:35.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:35.166+0000 7f0984ece640 1 -- 172.21.15.47:0/1036838256 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0980102040 msgr2=0x7f09801987b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:58:35.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:35.166+0000 7f097ffff640 1 --2- 172.21.15.47:0/1036838256 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0980102040 0x7f09801987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:58:35.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:35.166+0000 7f0984ece640 1 --2- 172.21.15.47:0/1036838256 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0980102040 0x7f09801987b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:35.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:35.166+0000 7f0984ece640 1 -- 172.21.15.47:0/1036838256 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f098019fe90 con 0x7f0980100e60 2024-04-17T05:58:35.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:35.166+0000 7f0984ece640 1 --2- 172.21.15.47:0/1036838256 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0980100e60 0x7f0980198270 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f097400e9c0 tx=0x7f097400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:58:35.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:35.167+0000 7f097dffb640 1 -- 172.21.15.47:0/1036838256 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f097400cd80 con 0x7f0980100e60 2024-04-17T05:58:35.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:35.167+0000 7f097dffb640 1 -- 172.21.15.47:0/1036838256 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09740035f0 con 0x7f0980100e60 2024-04-17T05:58:35.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:35.167+0000 7f0987132640 1 -- 172.21.15.47:0/1036838256 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09801a0180 con 0x7f0980100e60 2024-04-17T05:58:35.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:35.167+0000 7f097dffb640 1 -- 172.21.15.47:0/1036838256 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0974010690 con 0x7f0980100e60 2024-04-17T05:58:35.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:35.167+0000 7f0987132640 1 -- 172.21.15.47:0/1036838256 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09801a06d0 con 0x7f0980100e60 2024-04-17T05:58:35.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:35.168+0000 7f0987132640 1 -- 172.21.15.47:0/1036838256 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0980108200 con 0x7f0980100e60 2024-04-17T05:58:35.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:35.169+0000 7f097dffb640 1 -- 172.21.15.47:0/1036838256 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f09740037a0 con 0x7f0980100e60 2024-04-17T05:58:35.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:35.169+0000 7f097dffb640 1 --2- 172.21.15.47:0/1036838256 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f09700a3c30 0x7f09700a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:58:35.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:35.170+0000 7f097dffb640 1 -- 172.21.15.47:0/1036838256 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f09740d0420 con 0x7f0980100e60 2024-04-17T05:58:35.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:35.170+0000 7f097ffff640 1 --2- 172.21.15.47:0/1036838256 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f09700a3c30 0x7f09700a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:58:35.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:35.170+0000 7f097ffff640 1 --2- 172.21.15.47:0/1036838256 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f09700a3c30 0x7f09700a60f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f096c005fd0 tx=0x7f096c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:58:35.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:35.174+0000 7f097dffb640 1 -- 172.21.15.47:0/1036838256 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f097409d200 con 0x7f0980100e60 2024-04-17T05:58:35.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:35.402+0000 7f0987132640 1 -- 172.21.15.47:0/1036838256 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f098002d5d0 con 0x7f0980100e60 2024-04-17T05:58:35.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:35.403+0000 7f097dffb640 1 -- 172.21.15.47:0/1036838256 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7f098002d5d0 con 0x7f0980100e60 2024-04-17T05:58:35.402 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-17T05:58:35.402 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-04-17T05:58:35.402 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 1 2024-04-17T05:58:35.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:35.405+0000 7f0987132640 1 -- 172.21.15.47:0/1036838256 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f09700a3c30 msgr2=0x7f09700a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:58:35.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:35.405+0000 7f0987132640 1 --2- 172.21.15.47:0/1036838256 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f09700a3c30 0x7f09700a60f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f096c005fd0 tx=0x7f096c005850 comp rx=0 tx=0).stop 2024-04-17T05:58:35.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:35.405+0000 7f0987132640 1 -- 172.21.15.47:0/1036838256 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0980100e60 msgr2=0x7f0980198270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:58:35.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:35.405+0000 7f0987132640 1 --2- 172.21.15.47:0/1036838256 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0980100e60 0x7f0980198270 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f097400e9c0 tx=0x7f097400ee80 comp rx=0 tx=0).stop 2024-04-17T05:58:35.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:35.405+0000 7f0987132640 1 -- 172.21.15.47:0/1036838256 shutdown_connections 2024-04-17T05:58:35.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:35.405+0000 7f0987132640 1 --2- 172.21.15.47:0/1036838256 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f09700a3c30 0x7f09700a60f0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:35.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:35.405+0000 7f0987132640 1 --2- 172.21.15.47:0/1036838256 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0980102a00 0x7f098019f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:35.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:35.405+0000 7f0987132640 1 --2- 172.21.15.47:0/1036838256 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0980102040 0x7f09801987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:35.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:35.405+0000 7f0987132640 1 --2- 172.21.15.47:0/1036838256 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0980100e60 0x7f0980198270 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:35.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:35.405+0000 7f0987132640 1 -- 172.21.15.47:0/1036838256 >> 172.21.15.47:0/1036838256 conn(0x7f09800fc440 msgr2=0x7f09801053d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:58:35.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:35.405+0000 7f0987132640 1 -- 172.21.15.47:0/1036838256 shutdown_connections 2024-04-17T05:58:35.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:35.405+0000 7f0987132640 1 -- 172.21.15.47:0/1036838256 wait complete. 2024-04-17T05:58:35.702 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:35 smithi103 ceph-mon[36243]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T05:58:35.702 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:35 smithi103 ceph-mon[36243]: Deploying daemon mds.l on smithi103 2024-04-17T05:58:35.702 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:35 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/1552042968' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T05:58:35.702 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:35 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/1311708757' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-17T05:58:35.702 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:35 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/1036838256' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T05:58:35.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:35 smithi098 ceph-mon[36516]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T05:58:35.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:35 smithi098 ceph-mon[36516]: Deploying daemon mds.l on smithi103 2024-04-17T05:58:35.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:35 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/1552042968' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T05:58:35.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:35 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/1311708757' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-17T05:58:35.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:35 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/1036838256' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T05:58:35.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:35 smithi047 ceph-mon[32223]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T05:58:35.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:35 smithi047 ceph-mon[32223]: Deploying daemon mds.l on smithi103 2024-04-17T05:58:35.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:35 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/1552042968' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T05:58:35.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:35 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/1311708757' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-17T05:58:35.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:35 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/1036838256' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T05:58:36.355 INFO:journalctl@ceph.mds.l.smithi103.stdout:Apr 17 05:58:36 smithi103 podman[65966]: 2024-04-17 05:58:36.21996979 +0000 UTC m=+0.017592000 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-17T05:58:36.355 INFO:journalctl@ceph.mds.l.smithi103.stdout:Apr 17 05:58:36 smithi103 podman[65966]: 2024-04-17 05:58:36.357761751 +0000 UTC m=+0.155383952 container create 903cc67b7c0d4393f7f453966226c0bb8aaabb45dde78318626a9b5374627c0f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-l, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416) 2024-04-17T05:58:37.034 INFO:journalctl@ceph.mds.l.smithi103.stdout:Apr 17 05:58:36 smithi103 podman[65966]: 2024-04-17 05:58:36.627551567 +0000 UTC m=+0.425173779 container init 903cc67b7c0d4393f7f453966226c0bb8aaabb45dde78318626a9b5374627c0f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-l, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-04-17T05:58:37.034 INFO:journalctl@ceph.mds.l.smithi103.stdout:Apr 17 05:58:36 smithi103 podman[65966]: 2024-04-17 05:58:36.630031654 +0000 UTC m=+0.427653857 container start 903cc67b7c0d4393f7f453966226c0bb8aaabb45dde78318626a9b5374627c0f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-l, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-04-17T05:58:37.034 INFO:journalctl@ceph.mds.l.smithi103.stdout:Apr 17 05:58:36 smithi103 ceph-mds[65985]: -- 172.21.15.103:0/2309298543 <== mon.1 v2:172.21.15.103:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fa9291fa40 con 0x55fa928f2c00 2024-04-17T05:58:37.034 INFO:journalctl@ceph.mds.l.smithi103.stdout:Apr 17 05:58:36 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-l[65981]: starting mds.l at 2024-04-17T05:58:37.034 INFO:journalctl@ceph.mds.l.smithi103.stdout:Apr 17 05:58:36 smithi103 bash[65966]: 903cc67b7c0d4393f7f453966226c0bb8aaabb45dde78318626a9b5374627c0f 2024-04-17T05:58:37.034 INFO:journalctl@ceph.mds.l.smithi103.stdout:Apr 17 05:58:36 smithi103 systemd[1]: Started Ceph mds.l for 565db2d8-fc7e-11ee-bc91-c7b262605968. 2024-04-17T05:58:37.415 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T05:58:37.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:37.536+0000 7f7dd013d640 1 -- 172.21.15.47:0/912321980 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7dc8101ff0 msgr2=0x7f7dc8102470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:58:37.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:37.536+0000 7f7dd013d640 1 --2- 172.21.15.47:0/912321980 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7dc8101ff0 0x7f7dc8102470 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7dc4009900 tx=0x7f7dc402bc30 comp rx=0 tx=0).stop 2024-04-17T05:58:37.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:37.536+0000 7f7dd013d640 1 -- 172.21.15.47:0/912321980 shutdown_connections 2024-04-17T05:58:37.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:37.536+0000 7f7dd013d640 1 --2- 172.21.15.47:0/912321980 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7dc81029b0 0x7f7dc81092d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:37.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:37.536+0000 7f7dd013d640 1 --2- 172.21.15.47:0/912321980 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7dc8101ff0 0x7f7dc8102470 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:37.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:37.536+0000 7f7dd013d640 1 --2- 172.21.15.47:0/912321980 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7dc8100ea0 0x7f7dc81012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:37.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:37.536+0000 7f7dd013d640 1 -- 172.21.15.47:0/912321980 >> 172.21.15.47:0/912321980 conn(0x7f7dc80fc410 msgr2=0x7f7dc80fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:58:37.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:37.536+0000 7f7dd013d640 1 -- 172.21.15.47:0/912321980 shutdown_connections 2024-04-17T05:58:37.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:37.536+0000 7f7dd013d640 1 -- 172.21.15.47:0/912321980 wait complete. 2024-04-17T05:58:37.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:37.536+0000 7f7dd013d640 1 Processor -- start 2024-04-17T05:58:37.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:37.537+0000 7f7dd013d640 1 -- start start 2024-04-17T05:58:37.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:37.537+0000 7f7dd013d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7dc8100ea0 0x7f7dc8069cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:58:37.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:37.537+0000 7f7dd013d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7dc8101ff0 0x7f7dc819cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:58:37.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:37.537+0000 7f7dd013d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7dc81029b0 0x7f7dc819f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:58:37.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:37.537+0000 7f7dd013d640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f7dc810cb40 con 0x7f7dc8101ff0 2024-04-17T05:58:37.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:37.537+0000 7f7dd013d640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f7dc810c9c0 con 0x7f7dc8100ea0 2024-04-17T05:58:37.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:37.537+0000 7f7dd013d640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7dc810ccc0 con 0x7f7dc81029b0 2024-04-17T05:58:37.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:37.537+0000 7f7dcd6d8640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7dc8101ff0 0x7f7dc819cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:58:37.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:37.537+0000 7f7dcd6d8640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7dc8101ff0 0x7f7dc819cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45714/0 (socket says 172.21.15.47:45714) 2024-04-17T05:58:37.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:37.537+0000 7f7dcd6d8640 1 -- 172.21.15.47:0/1810463498 learned_addr learned my addr 172.21.15.47:0/1810463498 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:58:37.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:37.537+0000 7f7dcded9640 1 --2- 172.21.15.47:0/1810463498 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7dc8100ea0 0x7f7dc8069cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:58:37.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:37.537+0000 7f7dce6da640 1 --2- 172.21.15.47:0/1810463498 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7dc81029b0 0x7f7dc819f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:58:37.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:37.537+0000 7f7dcd6d8640 1 -- 172.21.15.47:0/1810463498 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7dc81029b0 msgr2=0x7f7dc819f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:58:37.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:37.537+0000 7f7dcd6d8640 1 --2- 172.21.15.47:0/1810463498 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7dc81029b0 0x7f7dc819f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:37.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:37.537+0000 7f7dcd6d8640 1 -- 172.21.15.47:0/1810463498 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7dc8100ea0 msgr2=0x7f7dc8069cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:58:37.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:37.537+0000 7f7dcd6d8640 1 --2- 172.21.15.47:0/1810463498 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7dc8100ea0 0x7f7dc8069cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:37.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:37.537+0000 7f7dcd6d8640 1 -- 172.21.15.47:0/1810463498 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dc819fed0 con 0x7f7dc8101ff0 2024-04-17T05:58:37.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:37.537+0000 7f7dce6da640 1 --2- 172.21.15.47:0/1810463498 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7dc81029b0 0x7f7dc819f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T05:58:37.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:37.537+0000 7f7dcd6d8640 1 --2- 172.21.15.47:0/1810463498 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7dc8101ff0 0x7f7dc819cfa0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7dc4000c00 tx=0x7f7dc4005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:58:37.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:37.537+0000 7f7dbaffd640 1 -- 172.21.15.47:0/1810463498 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dc4041070 con 0x7f7dc8101ff0 2024-04-17T05:58:37.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:37.537+0000 7f7dbaffd640 1 -- 172.21.15.47:0/1810463498 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7dc40044d0 con 0x7f7dc8101ff0 2024-04-17T05:58:37.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:37.537+0000 7f7dd013d640 1 -- 172.21.15.47:0/1810463498 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7dc81a0160 con 0x7f7dc8101ff0 2024-04-17T05:58:37.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:37.538+0000 7f7dbaffd640 1 -- 172.21.15.47:0/1810463498 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dc4034440 con 0x7f7dc8101ff0 2024-04-17T05:58:37.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:37.538+0000 7f7dd013d640 1 -- 172.21.15.47:0/1810463498 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7dc81a0650 con 0x7f7dc8101ff0 2024-04-17T05:58:37.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:37.538+0000 7f7dd013d640 1 -- 172.21.15.47:0/1810463498 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7dc81081b0 con 0x7f7dc8101ff0 2024-04-17T05:58:37.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:37.540+0000 7f7dbaffd640 1 -- 172.21.15.47:0/1810463498 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f7dc4044070 con 0x7f7dc8101ff0 2024-04-17T05:58:37.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:37.540+0000 7f7dbaffd640 1 --2- 172.21.15.47:0/1810463498 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f7db40a3bb0 0x7f7db40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:58:37.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:37.540+0000 7f7dcded9640 1 --2- 172.21.15.47:0/1810463498 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f7db40a3bb0 0x7f7db40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:58:37.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:37.540+0000 7f7dbaffd640 1 -- 172.21.15.47:0/1810463498 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f7dc40f1af0 con 0x7f7dc8101ff0 2024-04-17T05:58:37.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:37.541+0000 7f7dcded9640 1 --2- 172.21.15.47:0/1810463498 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f7db40a3bb0 0x7f7db40a6070 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc009bd0 tx=0x7f7dbc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:58:37.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:37.545+0000 7f7dbaffd640 1 -- 172.21.15.47:0/1810463498 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f7dc40be970 con 0x7f7dc8101ff0 2024-04-17T05:58:37.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:37.772+0000 7f7dd013d640 1 -- 172.21.15.47:0/1810463498 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7dc802d5d0 con 0x7f7dc8101ff0 2024-04-17T05:58:37.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:37.772+0000 7f7dbaffd640 1 -- 172.21.15.47:0/1810463498 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7f7dc40be270 con 0x7f7dc8101ff0 2024-04-17T05:58:37.770 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-17T05:58:37.770 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/2398363754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":2398363754},{"type":"v1","addr":"172.21.15.103:6833","nonce":2398363754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-17T05:58:37.771 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 2 2024-04-17T05:58:37.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:37.774+0000 7f7dd013d640 1 -- 172.21.15.47:0/1810463498 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f7db40a3bb0 msgr2=0x7f7db40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:58:37.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:37.774+0000 7f7dd013d640 1 --2- 172.21.15.47:0/1810463498 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f7db40a3bb0 0x7f7db40a6070 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc009bd0 tx=0x7f7dbc005f90 comp rx=0 tx=0).stop 2024-04-17T05:58:37.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:37.774+0000 7f7dd013d640 1 -- 172.21.15.47:0/1810463498 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7dc8101ff0 msgr2=0x7f7dc819cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:58:37.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:37.774+0000 7f7dd013d640 1 --2- 172.21.15.47:0/1810463498 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7dc8101ff0 0x7f7dc819cfa0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7dc4000c00 tx=0x7f7dc4005d50 comp rx=0 tx=0).stop 2024-04-17T05:58:37.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:37.774+0000 7f7dd013d640 1 -- 172.21.15.47:0/1810463498 shutdown_connections 2024-04-17T05:58:37.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:37.774+0000 7f7dd013d640 1 --2- 172.21.15.47:0/1810463498 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f7db40a3bb0 0x7f7db40a6070 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:37.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:37.774+0000 7f7dd013d640 1 --2- 172.21.15.47:0/1810463498 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7dc81029b0 0x7f7dc819f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:37.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:37.774+0000 7f7dd013d640 1 --2- 172.21.15.47:0/1810463498 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7dc8101ff0 0x7f7dc819cfa0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:37.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:37.774+0000 7f7dd013d640 1 --2- 172.21.15.47:0/1810463498 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7dc8100ea0 0x7f7dc8069cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:37.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:37.774+0000 7f7dd013d640 1 -- 172.21.15.47:0/1810463498 >> 172.21.15.47:0/1810463498 conn(0x7f7dc80fc410 msgr2=0x7f7dc8105380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:58:37.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:37.774+0000 7f7dd013d640 1 -- 172.21.15.47:0/1810463498 shutdown_connections 2024-04-17T05:58:37.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:37.774+0000 7f7dd013d640 1 -- 172.21.15.47:0/1810463498 wait complete. 2024-04-17T05:58:37.782 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:37 smithi047 ceph-mon[32223]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T05:58:37.782 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:37 smithi047 ceph-mon[32223]: mds.? [v2:172.21.15.103:6832/2398363754,v1:172.21.15.103:6833/2398363754] up:boot 2024-04-17T05:58:37.782 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:37 smithi047 ceph-mon[32223]: fsmap 1 up:standby 2024-04-17T05:58:37.783 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:37 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-17T05:58:37.783 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:37 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:37.783 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:37 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:37.783 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:37 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:37.783 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:37 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-17T05:58:37.783 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:37 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-17T05:58:37.783 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:37 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:58:38.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:37 smithi103 ceph-mon[36243]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T05:58:38.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:37 smithi103 ceph-mon[36243]: mds.? [v2:172.21.15.103:6832/2398363754,v1:172.21.15.103:6833/2398363754] up:boot 2024-04-17T05:58:38.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:37 smithi103 ceph-mon[36243]: fsmap 1 up:standby 2024-04-17T05:58:38.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:37 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-17T05:58:38.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:37 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:38.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:37 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:38.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:37 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:38.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:37 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-17T05:58:38.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:37 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-17T05:58:38.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:37 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:58:38.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:37 smithi098 ceph-mon[36516]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T05:58:38.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:37 smithi098 ceph-mon[36516]: mds.? [v2:172.21.15.103:6832/2398363754,v1:172.21.15.103:6833/2398363754] up:boot 2024-04-17T05:58:38.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:37 smithi098 ceph-mon[36516]: fsmap 1 up:standby 2024-04-17T05:58:38.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:37 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-17T05:58:38.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:37 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:38.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:37 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:38.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:37 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:38.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:37 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-17T05:58:38.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:37 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-17T05:58:38.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:37 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:58:38.804 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:38 smithi098 ceph-mon[36516]: Deploying daemon mds.k on smithi098 2024-04-17T05:58:38.804 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:38 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/1810463498' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T05:58:39.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:38 smithi103 ceph-mon[36243]: Deploying daemon mds.k on smithi098 2024-04-17T05:58:39.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:38 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/1810463498' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T05:58:39.133 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 05:58:38 smithi098 systemd[1]: Starting Ceph mds.k for 565db2d8-fc7e-11ee-bc91-c7b262605968... 2024-04-17T05:58:39.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:38 smithi047 ceph-mon[32223]: Deploying daemon mds.k on smithi098 2024-04-17T05:58:39.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:38 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/1810463498' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T05:58:39.448 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 05:58:39 smithi098 podman[66403]: 2024-04-17 05:58:39.101988918 +0000 UTC m=+0.017777160 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-17T05:58:39.448 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 05:58:39 smithi098 podman[66403]: 2024-04-17 05:58:39.230975793 +0000 UTC m=+0.146764023 container create 3f6debd36fb7e3aa462cbb0d74d76a322b2e30e193182f61c0da29c7c7a2f6ec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, ceph=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-17T05:58:39.448 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 05:58:39 smithi098 podman[66403]: 2024-04-17 05:58:39.450491373 +0000 UTC m=+0.366279604 container init 3f6debd36fb7e3aa462cbb0d74d76a322b2e30e193182f61c0da29c7c7a2f6ec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240416) 2024-04-17T05:58:39.784 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T05:58:39.883 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 05:58:39 smithi098 podman[66403]: 2024-04-17 05:58:39.453159282 +0000 UTC m=+0.368947529 container start 3f6debd36fb7e3aa462cbb0d74d76a322b2e30e193182f61c0da29c7c7a2f6ec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-04-17T05:58:39.883 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 05:58:39 smithi098 ceph-mds[66421]: -- 172.21.15.98:0/1785374081 <== mon.2 v2:172.21.15.98:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5628a163ba40 con 0x5628a160f000 2024-04-17T05:58:39.883 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 05:58:39 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[66417]: starting mds.k at 2024-04-17T05:58:39.883 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 05:58:39 smithi098 bash[66403]: 3f6debd36fb7e3aa462cbb0d74d76a322b2e30e193182f61c0da29c7c7a2f6ec 2024-04-17T05:58:39.883 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 05:58:39 smithi098 systemd[1]: Started Ceph mds.k for 565db2d8-fc7e-11ee-bc91-c7b262605968. 2024-04-17T05:58:39.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:39 smithi098 ceph-mon[36516]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T05:58:39.884 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:39 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:39.884 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:39 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:39.884 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:39 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:39.884 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:39 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-17T05:58:39.884 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:39 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-17T05:58:39.884 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:39 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:58:39.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:39.894+0000 7fc38b1f2640 1 -- 172.21.15.47:0/1062238278 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc384101ff0 msgr2=0x7fc384102470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:58:39.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:39.894+0000 7fc38b1f2640 1 --2- 172.21.15.47:0/1062238278 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc384101ff0 0x7fc384102470 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc370009900 tx=0x7fc37002bc30 comp rx=0 tx=0).stop 2024-04-17T05:58:39.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:39.894+0000 7fc38b1f2640 1 -- 172.21.15.47:0/1062238278 shutdown_connections 2024-04-17T05:58:39.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:39.894+0000 7fc38b1f2640 1 --2- 172.21.15.47:0/1062238278 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc3841029b0 0x7fc3841092d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:39.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:39.894+0000 7fc38b1f2640 1 --2- 172.21.15.47:0/1062238278 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc384101ff0 0x7fc384102470 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:39.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:39.894+0000 7fc38b1f2640 1 --2- 172.21.15.47:0/1062238278 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc384100ea0 0x7fc3841012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:39.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:39.894+0000 7fc38b1f2640 1 -- 172.21.15.47:0/1062238278 >> 172.21.15.47:0/1062238278 conn(0x7fc3840fc410 msgr2=0x7fc3840fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:58:39.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:39.894+0000 7fc38b1f2640 1 -- 172.21.15.47:0/1062238278 shutdown_connections 2024-04-17T05:58:39.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:39.894+0000 7fc38b1f2640 1 -- 172.21.15.47:0/1062238278 wait complete. 2024-04-17T05:58:39.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:39.895+0000 7fc38b1f2640 1 Processor -- start 2024-04-17T05:58:39.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:39.895+0000 7fc38b1f2640 1 -- start start 2024-04-17T05:58:39.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:39.895+0000 7fc38b1f2640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc384100ea0 0x7fc384069c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:58:39.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:39.895+0000 7fc38b1f2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc384101ff0 0x7fc38419d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:58:39.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:39.895+0000 7fc38b1f2640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc3841029b0 0x7fc38419f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:58:39.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:39.895+0000 7fc38b1f2640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fc38410cb40 con 0x7fc384100ea0 2024-04-17T05:58:39.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:39.895+0000 7fc38b1f2640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc38410c9c0 con 0x7fc3841029b0 2024-04-17T05:58:39.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:39.895+0000 7fc38b1f2640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc38410ccc0 con 0x7fc384101ff0 2024-04-17T05:58:39.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:39.895+0000 7fc388f8e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc384100ea0 0x7fc384069c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:58:39.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:39.895+0000 7fc388f8e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc384100ea0 0x7fc384069c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45726/0 (socket says 172.21.15.47:45726) 2024-04-17T05:58:39.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:39.895+0000 7fc388f8e640 1 -- 172.21.15.47:0/3211861117 learned_addr learned my addr 172.21.15.47:0/3211861117 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:58:39.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:39.895+0000 7fc38978f640 1 --2- 172.21.15.47:0/3211861117 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc3841029b0 0x7fc38419f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:58:39.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:39.895+0000 7fc383fff640 1 --2- 172.21.15.47:0/3211861117 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc384101ff0 0x7fc38419d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:58:39.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:39.895+0000 7fc388f8e640 1 -- 172.21.15.47:0/3211861117 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc384101ff0 msgr2=0x7fc38419d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:58:39.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:39.895+0000 7fc388f8e640 1 --2- 172.21.15.47:0/3211861117 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc384101ff0 0x7fc38419d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:39.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:39.895+0000 7fc388f8e640 1 -- 172.21.15.47:0/3211861117 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc3841029b0 msgr2=0x7fc38419f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:58:39.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:39.895+0000 7fc388f8e640 1 --2- 172.21.15.47:0/3211861117 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc3841029b0 0x7fc38419f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:39.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:39.895+0000 7fc388f8e640 1 -- 172.21.15.47:0/3211861117 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc38419ff30 con 0x7fc384100ea0 2024-04-17T05:58:39.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:39.895+0000 7fc388f8e640 1 --2- 172.21.15.47:0/3211861117 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc384100ea0 0x7fc384069c60 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc37800d950 tx=0x7fc37800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:58:39.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:39.896+0000 7fc381ffb640 1 -- 172.21.15.47:0/3211861117 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc378017070 con 0x7fc384100ea0 2024-04-17T05:58:39.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:39.896+0000 7fc381ffb640 1 -- 172.21.15.47:0/3211861117 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc378009c30 con 0x7fc384100ea0 2024-04-17T05:58:39.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:39.896+0000 7fc38b1f2640 1 -- 172.21.15.47:0/3211861117 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3841a01c0 con 0x7fc384100ea0 2024-04-17T05:58:39.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:39.896+0000 7fc381ffb640 1 -- 172.21.15.47:0/3211861117 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc37800f6b0 con 0x7fc384100ea0 2024-04-17T05:58:39.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:39.896+0000 7fc38b1f2640 1 -- 172.21.15.47:0/3211861117 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3841a06e0 con 0x7fc384100ea0 2024-04-17T05:58:39.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:39.897+0000 7fc38b1f2640 1 -- 172.21.15.47:0/3211861117 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3841081b0 con 0x7fc384100ea0 2024-04-17T05:58:39.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:39.898+0000 7fc381ffb640 1 -- 172.21.15.47:0/3211861117 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fc3780109d0 con 0x7fc384100ea0 2024-04-17T05:58:39.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:39.898+0000 7fc381ffb640 1 --2- 172.21.15.47:0/3211861117 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fc3740a3bb0 0x7fc3740a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:58:39.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:39.899+0000 7fc381ffb640 1 -- 172.21.15.47:0/3211861117 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fc3780cfb90 con 0x7fc384100ea0 2024-04-17T05:58:39.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:39.899+0000 7fc383fff640 1 --2- 172.21.15.47:0/3211861117 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fc3740a3bb0 0x7fc3740a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:58:39.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:39.899+0000 7fc383fff640 1 --2- 172.21.15.47:0/3211861117 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fc3740a3bb0 0x7fc3740a6070 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc370000c00 tx=0x7fc370005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:58:39.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:39.903+0000 7fc381ffb640 1 -- 172.21.15.47:0/3211861117 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fc378009de0 con 0x7fc384100ea0 2024-04-17T05:58:40.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:39 smithi103 ceph-mon[36243]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T05:58:40.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:39 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:40.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:39 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:40.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:39 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:40.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:39 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-17T05:58:40.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:39 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-17T05:58:40.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:39 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:58:40.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:39 smithi047 ceph-mon[32223]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T05:58:40.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:39 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:40.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:39 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:40.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:39 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:40.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:39 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-17T05:58:40.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:39 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-17T05:58:40.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:39 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:58:40.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:40.144+0000 7fc38b1f2640 1 -- 172.21.15.47:0/3211861117 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc384062b90 con 0x7fc384100ea0 2024-04-17T05:58:40.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:40.144+0000 7fc381ffb640 1 -- 172.21.15.47:0/3211861117 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2218 (secure 0 0 0) 0x7fc37809c320 con 0x7fc384100ea0 2024-04-17T05:58:40.142 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-17T05:58:40.142 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/2398363754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":2398363754},{"type":"v1","addr":"172.21.15.103:6833","nonce":2398363754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/384473493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":384473493},{"type":"v1","addr":"172.21.15.98:6833","nonce":384473493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-17T05:58:40.143 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 3 2024-04-17T05:58:40.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:40.146+0000 7fc38b1f2640 1 -- 172.21.15.47:0/3211861117 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fc3740a3bb0 msgr2=0x7fc3740a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:58:40.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:40.146+0000 7fc38b1f2640 1 --2- 172.21.15.47:0/3211861117 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fc3740a3bb0 0x7fc3740a6070 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc370000c00 tx=0x7fc370005fd0 comp rx=0 tx=0).stop 2024-04-17T05:58:40.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:40.146+0000 7fc38b1f2640 1 -- 172.21.15.47:0/3211861117 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc384100ea0 msgr2=0x7fc384069c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:58:40.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:40.146+0000 7fc38b1f2640 1 --2- 172.21.15.47:0/3211861117 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc384100ea0 0x7fc384069c60 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc37800d950 tx=0x7fc37800de10 comp rx=0 tx=0).stop 2024-04-17T05:58:40.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:40.147+0000 7fc38b1f2640 1 -- 172.21.15.47:0/3211861117 shutdown_connections 2024-04-17T05:58:40.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:40.147+0000 7fc38b1f2640 1 --2- 172.21.15.47:0/3211861117 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fc3740a3bb0 0x7fc3740a6070 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:40.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:40.147+0000 7fc38b1f2640 1 --2- 172.21.15.47:0/3211861117 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc3841029b0 0x7fc38419f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:40.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:40.147+0000 7fc38b1f2640 1 --2- 172.21.15.47:0/3211861117 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc384101ff0 0x7fc38419d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:40.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:40.147+0000 7fc38b1f2640 1 --2- 172.21.15.47:0/3211861117 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc384100ea0 0x7fc384069c60 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:40.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:40.147+0000 7fc38b1f2640 1 -- 172.21.15.47:0/3211861117 >> 172.21.15.47:0/3211861117 conn(0x7fc3840fc410 msgr2=0x7fc384105380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:58:40.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:40.147+0000 7fc38b1f2640 1 -- 172.21.15.47:0/3211861117 shutdown_connections 2024-04-17T05:58:40.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:40.147+0000 7fc38b1f2640 1 -- 172.21.15.47:0/3211861117 wait complete. 2024-04-17T05:58:40.851 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:40 smithi098 ceph-mon[36516]: Deploying daemon mds.j on smithi047 2024-04-17T05:58:40.852 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:40 smithi098 ceph-mon[36516]: mds.? [v2:172.21.15.98:6832/384473493,v1:172.21.15.98:6833/384473493] up:boot 2024-04-17T05:58:40.852 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:40 smithi098 ceph-mon[36516]: fsmap 2 up:standby 2024-04-17T05:58:40.852 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:40 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-17T05:58:40.852 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:40 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/3211861117' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T05:58:40.984 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:40 smithi047 ceph-mon[32223]: Deploying daemon mds.j on smithi047 2024-04-17T05:58:40.985 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:40 smithi047 ceph-mon[32223]: mds.? [v2:172.21.15.98:6832/384473493,v1:172.21.15.98:6833/384473493] up:boot 2024-04-17T05:58:40.985 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:40 smithi047 ceph-mon[32223]: fsmap 2 up:standby 2024-04-17T05:58:40.985 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:40 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-17T05:58:40.985 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:40 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/3211861117' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T05:58:41.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:40 smithi103 ceph-mon[36243]: Deploying daemon mds.j on smithi047 2024-04-17T05:58:41.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:40 smithi103 ceph-mon[36243]: mds.? [v2:172.21.15.98:6832/384473493,v1:172.21.15.98:6833/384473493] up:boot 2024-04-17T05:58:41.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:40 smithi103 ceph-mon[36243]: fsmap 2 up:standby 2024-04-17T05:58:41.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:40 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-17T05:58:41.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:40 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/3211861117' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T05:58:42.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:41 smithi103 ceph-mon[36243]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T05:58:42.047 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:41 smithi047 ceph-mon[32223]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T05:58:42.047 INFO:journalctl@ceph.mds.j.smithi047.stdout:Apr 17 05:58:41 smithi047 systemd[1]: Starting Ceph mds.j for 565db2d8-fc7e-11ee-bc91-c7b262605968... 2024-04-17T05:58:42.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:41 smithi098 ceph-mon[36516]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T05:58:42.157 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T05:58:42.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:42.273+0000 7fb5fbbb5640 1 -- 172.21.15.47:0/3514295263 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5f41031a0 msgr2=0x7fb5f4103620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:58:42.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:42.273+0000 7fb5fbbb5640 1 --2- 172.21.15.47:0/3514295263 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5f41031a0 0x7fb5f4103620 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb5e4009900 tx=0x7fb5e402bc30 comp rx=0 tx=0).stop 2024-04-17T05:58:42.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:42.274+0000 7fb5fbbb5640 1 -- 172.21.15.47:0/3514295263 shutdown_connections 2024-04-17T05:58:42.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:42.274+0000 7fb5fbbb5640 1 --2- 172.21.15.47:0/3514295263 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb5f4103b60 0x7fb5f410a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:42.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:42.274+0000 7fb5fbbb5640 1 --2- 172.21.15.47:0/3514295263 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5f41031a0 0x7fb5f4103620 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:42.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:42.274+0000 7fb5fbbb5640 1 --2- 172.21.15.47:0/3514295263 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb5f4101fb0 0x7fb5f41023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:42.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:42.274+0000 7fb5fbbb5640 1 -- 172.21.15.47:0/3514295263 >> 172.21.15.47:0/3514295263 conn(0x7fb5f40fd560 msgr2=0x7fb5f40ff980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:58:42.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:42.274+0000 7fb5fbbb5640 1 -- 172.21.15.47:0/3514295263 shutdown_connections 2024-04-17T05:58:42.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:42.274+0000 7fb5fbbb5640 1 -- 172.21.15.47:0/3514295263 wait complete. 2024-04-17T05:58:42.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:42.274+0000 7fb5fbbb5640 1 Processor -- start 2024-04-17T05:58:42.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:42.274+0000 7fb5fbbb5640 1 -- start start 2024-04-17T05:58:42.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:42.274+0000 7fb5fbbb5640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb5f4101fb0 0x7fb5f41982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:58:42.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:42.274+0000 7fb5fbbb5640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5f41031a0 0x7fb5f4198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:58:42.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:42.274+0000 7fb5fbbb5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb5f4103b60 0x7fb5f419f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:58:42.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:42.274+0000 7fb5fbbb5640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f410dc40 con 0x7fb5f41031a0 2024-04-17T05:58:42.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:42.274+0000 7fb5fbbb5640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f410dac0 con 0x7fb5f4101fb0 2024-04-17T05:58:42.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:42.274+0000 7fb5fbbb5640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f410ddc0 con 0x7fb5f4103b60 2024-04-17T05:58:42.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:42.275+0000 7fb5f9150640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5f41031a0 0x7fb5f4198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:58:42.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:42.275+0000 7fb5f9150640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5f41031a0 0x7fb5f4198800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45742/0 (socket says 172.21.15.47:45742) 2024-04-17T05:58:42.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:42.275+0000 7fb5f9150640 1 -- 172.21.15.47:0/840761969 learned_addr learned my addr 172.21.15.47:0/840761969 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:58:42.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:42.275+0000 7fb5fa152640 1 --2- 172.21.15.47:0/840761969 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb5f4103b60 0x7fb5f419f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:58:42.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:42.275+0000 7fb5f9951640 1 --2- 172.21.15.47:0/840761969 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb5f4101fb0 0x7fb5f41982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:58:42.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:42.275+0000 7fb5f9150640 1 -- 172.21.15.47:0/840761969 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb5f4103b60 msgr2=0x7fb5f419f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:58:42.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:42.275+0000 7fb5f9150640 1 --2- 172.21.15.47:0/840761969 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb5f4103b60 0x7fb5f419f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:42.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:42.275+0000 7fb5f9150640 1 -- 172.21.15.47:0/840761969 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb5f4101fb0 msgr2=0x7fb5f41982c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:58:42.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:42.275+0000 7fb5f9150640 1 --2- 172.21.15.47:0/840761969 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb5f4101fb0 0x7fb5f41982c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:42.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:42.275+0000 7fb5f9150640 1 -- 172.21.15.47:0/840761969 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5f419fee0 con 0x7fb5f41031a0 2024-04-17T05:58:42.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:42.275+0000 7fb5f9951640 1 --2- 172.21.15.47:0/840761969 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb5f4101fb0 0x7fb5f41982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T05:58:42.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:42.275+0000 7fb5f9150640 1 --2- 172.21.15.47:0/840761969 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5f41031a0 0x7fb5f4198800 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb5e4009230 tx=0x7fb5e4005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:58:42.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:42.275+0000 7fb5eaffd640 1 -- 172.21.15.47:0/840761969 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e4041070 con 0x7fb5f41031a0 2024-04-17T05:58:42.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:42.275+0000 7fb5eaffd640 1 -- 172.21.15.47:0/840761969 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5e40054e0 con 0x7fb5f41031a0 2024-04-17T05:58:42.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:42.275+0000 7fb5fbbb5640 1 -- 172.21.15.47:0/840761969 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5f41a0170 con 0x7fb5f41031a0 2024-04-17T05:58:42.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:42.275+0000 7fb5eaffd640 1 -- 172.21.15.47:0/840761969 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e4034440 con 0x7fb5f41031a0 2024-04-17T05:58:42.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:42.275+0000 7fb5fbbb5640 1 -- 172.21.15.47:0/840761969 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5f41a0660 con 0x7fb5f41031a0 2024-04-17T05:58:42.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:42.276+0000 7fb5fbbb5640 1 -- 172.21.15.47:0/840761969 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5f41092d0 con 0x7fb5f41031a0 2024-04-17T05:58:42.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:42.277+0000 7fb5eaffd640 1 -- 172.21.15.47:0/840761969 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb5e4044070 con 0x7fb5f41031a0 2024-04-17T05:58:42.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:42.278+0000 7fb5eaffd640 1 --2- 172.21.15.47:0/840761969 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fb5e00a3bb0 0x7fb5e00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:58:42.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:42.278+0000 7fb5f9951640 1 --2- 172.21.15.47:0/840761969 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fb5e00a3bb0 0x7fb5e00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:58:42.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:42.278+0000 7fb5eaffd640 1 -- 172.21.15.47:0/840761969 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fb5e40f20f0 con 0x7fb5f41031a0 2024-04-17T05:58:42.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:42.279+0000 7fb5f9951640 1 --2- 172.21.15.47:0/840761969 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fb5e00a3bb0 0x7fb5e00a6070 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb5f00058c0 tx=0x7fb5f0005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:58:42.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:42.282+0000 7fb5eaffd640 1 -- 172.21.15.47:0/840761969 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fb5e40beed0 con 0x7fb5f41031a0 2024-04-17T05:58:42.325 INFO:journalctl@ceph.mds.j.smithi047.stdout:Apr 17 05:58:42 smithi047 podman[73302]: 2024-04-17 05:58:41.952625282 +0000 UTC m=+0.018191884 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-17T05:58:42.325 INFO:journalctl@ceph.mds.j.smithi047.stdout:Apr 17 05:58:42 smithi047 podman[73302]: 2024-04-17 05:58:42.074526537 +0000 UTC m=+0.140093119 container create 90546cf558c860dec36fdb766e61c2079eae1f0984f96ff480c6f6dfb61b8ad1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-j, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS) 2024-04-17T05:58:42.325 INFO:journalctl@ceph.mds.j.smithi047.stdout:Apr 17 05:58:42 smithi047 podman[73302]: 2024-04-17 05:58:42.327426745 +0000 UTC m=+0.392993343 container init 90546cf558c860dec36fdb766e61c2079eae1f0984f96ff480c6f6dfb61b8ad1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-j, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-17T05:58:42.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:42.523+0000 7fb5fbbb5640 1 -- 172.21.15.47:0/840761969 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb5f402d790 con 0x7fb5f41031a0 2024-04-17T05:58:42.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:42.523+0000 7fb5eaffd640 1 -- 172.21.15.47:0/840761969 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2218 (secure 0 0 0) 0x7fb5e40be7d0 con 0x7fb5f41031a0 2024-04-17T05:58:42.521 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-17T05:58:42.521 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/2398363754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":2398363754},{"type":"v1","addr":"172.21.15.103:6833","nonce":2398363754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/384473493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":384473493},{"type":"v1","addr":"172.21.15.98:6833","nonce":384473493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-17T05:58:42.521 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 3 2024-04-17T05:58:42.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:42.526+0000 7fb5fbbb5640 1 -- 172.21.15.47:0/840761969 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fb5e00a3bb0 msgr2=0x7fb5e00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:58:42.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:42.526+0000 7fb5fbbb5640 1 --2- 172.21.15.47:0/840761969 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fb5e00a3bb0 0x7fb5e00a6070 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb5f00058c0 tx=0x7fb5f0005850 comp rx=0 tx=0).stop 2024-04-17T05:58:42.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:42.526+0000 7fb5fbbb5640 1 -- 172.21.15.47:0/840761969 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5f41031a0 msgr2=0x7fb5f4198800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:58:42.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:42.526+0000 7fb5fbbb5640 1 --2- 172.21.15.47:0/840761969 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5f41031a0 0x7fb5f4198800 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb5e4009230 tx=0x7fb5e4005dd0 comp rx=0 tx=0).stop 2024-04-17T05:58:42.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:42.526+0000 7fb5fbbb5640 1 -- 172.21.15.47:0/840761969 shutdown_connections 2024-04-17T05:58:42.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:42.526+0000 7fb5fbbb5640 1 --2- 172.21.15.47:0/840761969 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fb5e00a3bb0 0x7fb5e00a6070 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:42.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:42.526+0000 7fb5fbbb5640 1 --2- 172.21.15.47:0/840761969 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb5f4103b60 0x7fb5f419f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:42.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:42.526+0000 7fb5fbbb5640 1 --2- 172.21.15.47:0/840761969 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5f41031a0 0x7fb5f4198800 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:42.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:42.526+0000 7fb5fbbb5640 1 --2- 172.21.15.47:0/840761969 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb5f4101fb0 0x7fb5f41982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:42.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:42.526+0000 7fb5fbbb5640 1 -- 172.21.15.47:0/840761969 >> 172.21.15.47:0/840761969 conn(0x7fb5f40fd560 msgr2=0x7fb5f41064a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:58:42.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:42.526+0000 7fb5fbbb5640 1 -- 172.21.15.47:0/840761969 shutdown_connections 2024-04-17T05:58:42.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:42.526+0000 7fb5fbbb5640 1 -- 172.21.15.47:0/840761969 wait complete. 2024-04-17T05:58:42.639 INFO:journalctl@ceph.mds.j.smithi047.stdout:Apr 17 05:58:42 smithi047 podman[73302]: 2024-04-17 05:58:42.330720056 +0000 UTC m=+0.396286639 container start 90546cf558c860dec36fdb766e61c2079eae1f0984f96ff480c6f6dfb61b8ad1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-j, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-04-17T05:58:42.639 INFO:journalctl@ceph.mds.j.smithi047.stdout:Apr 17 05:58:42 smithi047 ceph-mds[73359]: -- 172.21.15.47:0/2509883872 <== mon.0 v2:172.21.15.47:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558ce20f5a40 con 0x558ce20c8800 2024-04-17T05:58:42.639 INFO:journalctl@ceph.mds.j.smithi047.stdout:Apr 17 05:58:42 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-j[73332]: starting mds.j at 2024-04-17T05:58:42.639 INFO:journalctl@ceph.mds.j.smithi047.stdout:Apr 17 05:58:42 smithi047 bash[73302]: 90546cf558c860dec36fdb766e61c2079eae1f0984f96ff480c6f6dfb61b8ad1 2024-04-17T05:58:42.639 INFO:journalctl@ceph.mds.j.smithi047.stdout:Apr 17 05:58:42 smithi047 systemd[1]: Started Ceph mds.j for 565db2d8-fc7e-11ee-bc91-c7b262605968. 2024-04-17T05:58:43.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:43 smithi103 ceph-mon[36243]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T05:58:43.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:43 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:43.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:43 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:43.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:43 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:43.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:43 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-17T05:58:43.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:43 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-17T05:58:43.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:43 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:58:43.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:43 smithi103 ceph-mon[36243]: Deploying daemon mds.d on smithi047 2024-04-17T05:58:43.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:43 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/840761969' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T05:58:43.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:43 smithi103 ceph-mon[36243]: mds.? [v2:172.21.15.47:6834/3189958720,v1:172.21.15.47:6835/3189958720] up:boot 2024-04-17T05:58:43.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:43 smithi103 ceph-mon[36243]: fsmap 3 up:standby 2024-04-17T05:58:43.785 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:43 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-17T05:58:43.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:43 smithi098 ceph-mon[36516]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T05:58:43.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:43 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:43.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:43 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:43.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:43 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:43.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:43 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-17T05:58:43.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:43 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-17T05:58:43.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:43 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:58:43.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:43 smithi098 ceph-mon[36516]: Deploying daemon mds.d on smithi047 2024-04-17T05:58:43.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:43 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/840761969' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T05:58:43.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:43 smithi098 ceph-mon[36516]: mds.? [v2:172.21.15.47:6834/3189958720,v1:172.21.15.47:6835/3189958720] up:boot 2024-04-17T05:58:43.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:43 smithi098 ceph-mon[36516]: fsmap 3 up:standby 2024-04-17T05:58:43.884 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:43 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-17T05:58:43.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:43 smithi047 ceph-mon[32223]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T05:58:43.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:43 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:43.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:43 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:43.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:43 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:43.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:43 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-17T05:58:43.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:43 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-17T05:58:43.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:43 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:58:43.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:43 smithi047 ceph-mon[32223]: Deploying daemon mds.d on smithi047 2024-04-17T05:58:43.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:43 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/840761969' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T05:58:43.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:43 smithi047 ceph-mon[32223]: mds.? [v2:172.21.15.47:6834/3189958720,v1:172.21.15.47:6835/3189958720] up:boot 2024-04-17T05:58:43.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:43 smithi047 ceph-mon[32223]: fsmap 3 up:standby 2024-04-17T05:58:43.890 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:43 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-17T05:58:44.536 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T05:58:44.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:44.804+0000 7fa561b75640 1 -- 172.21.15.47:0/3758503258 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa55c100ea0 msgr2=0x7fa55c1012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:58:44.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:44.804+0000 7fa561b75640 1 --2- 172.21.15.47:0/3758503258 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa55c100ea0 0x7fa55c1012c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa54c009900 tx=0x7fa54c02bc30 comp rx=0 tx=0).stop 2024-04-17T05:58:44.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:44.804+0000 7fa561b75640 1 -- 172.21.15.47:0/3758503258 shutdown_connections 2024-04-17T05:58:44.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:44.805+0000 7fa561b75640 1 --2- 172.21.15.47:0/3758503258 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa55c1029b0 0x7fa55c1092d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:44.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:44.805+0000 7fa561b75640 1 --2- 172.21.15.47:0/3758503258 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa55c101ff0 0x7fa55c102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:44.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:44.805+0000 7fa561b75640 1 --2- 172.21.15.47:0/3758503258 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa55c100ea0 0x7fa55c1012c0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:44.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:44.805+0000 7fa561b75640 1 -- 172.21.15.47:0/3758503258 >> 172.21.15.47:0/3758503258 conn(0x7fa55c0fc410 msgr2=0x7fa55c0fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:58:44.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:44.805+0000 7fa561b75640 1 -- 172.21.15.47:0/3758503258 shutdown_connections 2024-04-17T05:58:44.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:44.805+0000 7fa561b75640 1 -- 172.21.15.47:0/3758503258 wait complete. 2024-04-17T05:58:44.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:44.805+0000 7fa561b75640 1 Processor -- start 2024-04-17T05:58:44.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:44.805+0000 7fa561b75640 1 -- start start 2024-04-17T05:58:44.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:44.805+0000 7fa561b75640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa55c100ea0 0x7fa55c104e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:58:44.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:44.805+0000 7fa561b75640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa55c101ff0 0x7fa55c103460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:58:44.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:44.806+0000 7fa561b75640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa55c1029b0 0x7fa55c1039a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:58:44.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:44.806+0000 7fa561b75640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa55c10cbc0 con 0x7fa55c1029b0 2024-04-17T05:58:44.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:44.806+0000 7fa561b75640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa55c10ca40 con 0x7fa55c101ff0 2024-04-17T05:58:44.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:44.806+0000 7fa561b75640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa55c10cd40 con 0x7fa55c100ea0 2024-04-17T05:58:44.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:44.806+0000 7fa55bfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa55c1029b0 0x7fa55c1039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:58:44.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:44.806+0000 7fa55bfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa55c1029b0 0x7fa55c1039a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53924/0 (socket says 172.21.15.47:53924) 2024-04-17T05:58:44.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:44.806+0000 7fa55b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa55c100ea0 0x7fa55c104e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:58:44.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:44.806+0000 7fa55bfff640 1 -- 172.21.15.47:0/402287086 learned_addr learned my addr 172.21.15.47:0/402287086 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:58:44.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:44.806+0000 7fa55affd640 1 --2- 172.21.15.47:0/402287086 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa55c101ff0 0x7fa55c103460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:58:44.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:44.806+0000 7fa55bfff640 1 -- 172.21.15.47:0/402287086 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa55c100ea0 msgr2=0x7fa55c104e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:58:44.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:44.806+0000 7fa55bfff640 1 --2- 172.21.15.47:0/402287086 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa55c100ea0 0x7fa55c104e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:44.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:44.806+0000 7fa55bfff640 1 -- 172.21.15.47:0/402287086 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa55c101ff0 msgr2=0x7fa55c103460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:58:44.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:44.806+0000 7fa55bfff640 1 --2- 172.21.15.47:0/402287086 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa55c101ff0 0x7fa55c103460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:44.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:44.806+0000 7fa55bfff640 1 -- 172.21.15.47:0/402287086 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa55c104230 con 0x7fa55c1029b0 2024-04-17T05:58:44.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:44.806+0000 7fa55bfff640 1 --2- 172.21.15.47:0/402287086 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa55c1029b0 0x7fa55c1039a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa55400cce0 tx=0x7fa55400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:58:44.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:44.806+0000 7fa558ff9640 1 -- 172.21.15.47:0/402287086 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa55400f450 con 0x7fa55c1029b0 2024-04-17T05:58:44.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:44.806+0000 7fa561b75640 1 -- 172.21.15.47:0/402287086 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa55c1a2470 con 0x7fa55c1029b0 2024-04-17T05:58:44.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:44.806+0000 7fa558ff9640 1 -- 172.21.15.47:0/402287086 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa55400fad0 con 0x7fa55c1029b0 2024-04-17T05:58:44.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:44.806+0000 7fa558ff9640 1 -- 172.21.15.47:0/402287086 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa554018b60 con 0x7fa55c1029b0 2024-04-17T05:58:44.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:44.806+0000 7fa561b75640 1 -- 172.21.15.47:0/402287086 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa55c1a29c0 con 0x7fa55c1029b0 2024-04-17T05:58:44.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:44.807+0000 7fa561b75640 1 -- 172.21.15.47:0/402287086 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa55c1a2ce0 con 0x7fa55c1029b0 2024-04-17T05:58:44.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:44.810+0000 7fa558ff9640 1 -- 172.21.15.47:0/402287086 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fa55400fc80 con 0x7fa55c1029b0 2024-04-17T05:58:44.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:44.810+0000 7fa558ff9640 1 --2- 172.21.15.47:0/402287086 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fa5440a3bb0 0x7fa5440a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:58:44.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:44.810+0000 7fa55b7fe640 1 --2- 172.21.15.47:0/402287086 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fa5440a3bb0 0x7fa5440a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:58:44.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:44.810+0000 7fa558ff9640 1 -- 172.21.15.47:0/402287086 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fa5540d0640 con 0x7fa55c1029b0 2024-04-17T05:58:44.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:44.811+0000 7fa55b7fe640 1 --2- 172.21.15.47:0/402287086 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fa5440a3bb0 0x7fa5440a6070 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa54c005ab0 tx=0x7fa54c004f10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:58:44.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:44.813+0000 7fa558ff9640 1 -- 172.21.15.47:0/402287086 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fa554014030 con 0x7fa55c1029b0 2024-04-17T05:58:45.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:45.050+0000 7fa561b75640 1 -- 172.21.15.47:0/402287086 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa55c1a2eb0 con 0x7fa55c1029b0 2024-04-17T05:58:45.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:45.050+0000 7fa558ff9640 1 -- 172.21.15.47:0/402287086 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3043 (secure 0 0 0) 0x7fa55409d4c0 con 0x7fa55c1029b0 2024-04-17T05:58:45.049 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-17T05:58:45.049 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3189958720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3189958720},{"type":"v1","addr":"172.21.15.47:6835","nonce":3189958720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/2398363754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":2398363754},{"type":"v1","addr":"172.21.15.103:6833","nonce":2398363754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/384473493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":384473493},{"type":"v1","addr":"172.21.15.98:6833","nonce":384473493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-17T05:58:45.049 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 4 2024-04-17T05:58:45.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:45.053+0000 7fa561b75640 1 -- 172.21.15.47:0/402287086 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fa5440a3bb0 msgr2=0x7fa5440a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:58:45.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:45.053+0000 7fa561b75640 1 --2- 172.21.15.47:0/402287086 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fa5440a3bb0 0x7fa5440a6070 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa54c005ab0 tx=0x7fa54c004f10 comp rx=0 tx=0).stop 2024-04-17T05:58:45.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:45.053+0000 7fa561b75640 1 -- 172.21.15.47:0/402287086 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa55c1029b0 msgr2=0x7fa55c1039a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:58:45.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:45.053+0000 7fa561b75640 1 --2- 172.21.15.47:0/402287086 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa55c1029b0 0x7fa55c1039a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa55400cce0 tx=0x7fa55400a620 comp rx=0 tx=0).stop 2024-04-17T05:58:45.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:45.053+0000 7fa561b75640 1 -- 172.21.15.47:0/402287086 shutdown_connections 2024-04-17T05:58:45.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:45.053+0000 7fa561b75640 1 --2- 172.21.15.47:0/402287086 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fa5440a3bb0 0x7fa5440a6070 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:45.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:45.053+0000 7fa561b75640 1 --2- 172.21.15.47:0/402287086 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa55c1029b0 0x7fa55c1039a0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:45.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:45.053+0000 7fa561b75640 1 --2- 172.21.15.47:0/402287086 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa55c101ff0 0x7fa55c103460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:45.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:45.053+0000 7fa561b75640 1 --2- 172.21.15.47:0/402287086 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa55c100ea0 0x7fa55c104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:45.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:45.053+0000 7fa561b75640 1 -- 172.21.15.47:0/402287086 >> 172.21.15.47:0/402287086 conn(0x7fa55c0fc410 msgr2=0x7fa55c0fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:58:45.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:45.053+0000 7fa561b75640 1 -- 172.21.15.47:0/402287086 shutdown_connections 2024-04-17T05:58:45.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:45.053+0000 7fa561b75640 1 -- 172.21.15.47:0/402287086 wait complete. 2024-04-17T05:58:45.179 INFO:journalctl@ceph.mds.d.smithi047.stdout:Apr 17 05:58:45 smithi047 podman[73694]: 2024-04-17 05:58:45.045706165 +0000 UTC m=+0.018028901 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-17T05:58:45.179 INFO:journalctl@ceph.mds.d.smithi047.stdout:Apr 17 05:58:45 smithi047 podman[73694]: 2024-04-17 05:58:45.181551381 +0000 UTC m=+0.153874099 container create 5dad8c0cc8741985ab623a302834d9239cb1efc7c43ab3c872fe6fd8700b6124 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-04-17T05:58:45.783 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:45 smithi103 ceph-mon[36243]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T05:58:45.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:45 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/402287086' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T05:58:45.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:45 smithi098 ceph-mon[36516]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T05:58:45.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:45 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/402287086' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T05:58:45.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:45 smithi047 ceph-mon[32223]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T05:58:45.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:45 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/402287086' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T05:58:45.889 INFO:journalctl@ceph.mds.d.smithi047.stdout:Apr 17 05:58:45 smithi047 podman[73694]: 2024-04-17 05:58:45.476698675 +0000 UTC m=+0.449021392 container init 5dad8c0cc8741985ab623a302834d9239cb1efc7c43ab3c872fe6fd8700b6124 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-d, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux ) 2024-04-17T05:58:45.889 INFO:journalctl@ceph.mds.d.smithi047.stdout:Apr 17 05:58:45 smithi047 podman[73694]: 2024-04-17 05:58:45.479513566 +0000 UTC m=+0.451836291 container start 5dad8c0cc8741985ab623a302834d9239cb1efc7c43ab3c872fe6fd8700b6124 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-d, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD) 2024-04-17T05:58:45.889 INFO:journalctl@ceph.mds.d.smithi047.stdout:Apr 17 05:58:45 smithi047 ceph-mds[73714]: -- 172.21.15.47:0/1561998132 <== mon.0 v2:172.21.15.47:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563d8f067a40 con 0x563d8f03a800 2024-04-17T05:58:45.889 INFO:journalctl@ceph.mds.d.smithi047.stdout:Apr 17 05:58:45 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-d[73710]: starting mds.d at 2024-04-17T05:58:45.889 INFO:journalctl@ceph.mds.d.smithi047.stdout:Apr 17 05:58:45 smithi047 bash[73694]: 5dad8c0cc8741985ab623a302834d9239cb1efc7c43ab3c872fe6fd8700b6124 2024-04-17T05:58:45.889 INFO:journalctl@ceph.mds.d.smithi047.stdout:Apr 17 05:58:45 smithi047 systemd[1]: Started Ceph mds.d for 565db2d8-fc7e-11ee-bc91-c7b262605968. 2024-04-17T05:58:46.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:46 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:46.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:46 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:46.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:46 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:46.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:46 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-17T05:58:46.884 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:46 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-17T05:58:46.884 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:46 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:58:46.884 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:46 smithi098 ceph-mon[36516]: Deploying daemon mds.e on smithi098 2024-04-17T05:58:46.884 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:46 smithi098 ceph-mon[36516]: mds.? [v2:172.21.15.47:6836/3143529973,v1:172.21.15.47:6837/3143529973] up:boot 2024-04-17T05:58:46.884 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:46 smithi098 ceph-mon[36516]: fsmap 4 up:standby 2024-04-17T05:58:46.884 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:46 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-17T05:58:46.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:46 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:46.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:46 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:46.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:46 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:46.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:46 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-17T05:58:46.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:46 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-17T05:58:46.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:46 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:58:46.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:46 smithi047 ceph-mon[32223]: Deploying daemon mds.e on smithi098 2024-04-17T05:58:46.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:46 smithi047 ceph-mon[32223]: mds.? [v2:172.21.15.47:6836/3143529973,v1:172.21.15.47:6837/3143529973] up:boot 2024-04-17T05:58:46.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:46 smithi047 ceph-mon[32223]: fsmap 4 up:standby 2024-04-17T05:58:46.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:46 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-17T05:58:47.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:46 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:47.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:46 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:47.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:46 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:47.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:46 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-17T05:58:47.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:46 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-17T05:58:47.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:46 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:58:47.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:46 smithi103 ceph-mon[36243]: Deploying daemon mds.e on smithi098 2024-04-17T05:58:47.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:46 smithi103 ceph-mon[36243]: mds.? [v2:172.21.15.47:6836/3143529973,v1:172.21.15.47:6837/3143529973] up:boot 2024-04-17T05:58:47.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:46 smithi103 ceph-mon[36243]: fsmap 4 up:standby 2024-04-17T05:58:47.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:46 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-17T05:58:47.063 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T05:58:47.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:47.176+0000 7f353470e640 1 -- 172.21.15.47:0/524213299 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f352c100ea0 msgr2=0x7f352c1012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:58:47.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:47.176+0000 7f353470e640 1 --2- 172.21.15.47:0/524213299 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f352c100ea0 0x7f352c1012c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f3528009900 tx=0x7f352802bc30 comp rx=0 tx=0).stop 2024-04-17T05:58:47.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:47.176+0000 7f353470e640 1 -- 172.21.15.47:0/524213299 shutdown_connections 2024-04-17T05:58:47.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:47.176+0000 7f353470e640 1 --2- 172.21.15.47:0/524213299 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f352c1029b0 0x7f352c1092d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:47.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:47.176+0000 7f353470e640 1 --2- 172.21.15.47:0/524213299 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f352c101ff0 0x7f352c102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:47.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:47.176+0000 7f353470e640 1 --2- 172.21.15.47:0/524213299 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f352c100ea0 0x7f352c1012c0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:47.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:47.176+0000 7f353470e640 1 -- 172.21.15.47:0/524213299 >> 172.21.15.47:0/524213299 conn(0x7f352c0fc410 msgr2=0x7f352c0fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:58:47.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:47.176+0000 7f353470e640 1 -- 172.21.15.47:0/524213299 shutdown_connections 2024-04-17T05:58:47.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:47.176+0000 7f353470e640 1 -- 172.21.15.47:0/524213299 wait complete. 2024-04-17T05:58:47.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:47.177+0000 7f353470e640 1 Processor -- start 2024-04-17T05:58:47.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:47.177+0000 7f353470e640 1 -- start start 2024-04-17T05:58:47.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:47.177+0000 7f353470e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f352c100ea0 0x7f352c19a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:58:47.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:47.177+0000 7f353470e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f352c101ff0 0x7f352c19aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:58:47.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:47.177+0000 7f353470e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f352c1029b0 0x7f352c1a1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:58:47.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:47.177+0000 7f353470e640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f352c10cb40 con 0x7f352c100ea0 2024-04-17T05:58:47.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:47.177+0000 7f353470e640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f352c10c9c0 con 0x7f352c1029b0 2024-04-17T05:58:47.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:47.177+0000 7f353470e640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f352c10ccc0 con 0x7f352c101ff0 2024-04-17T05:58:47.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:47.177+0000 7f35324aa640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f352c100ea0 0x7f352c19a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:58:47.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:47.177+0000 7f35324aa640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f352c100ea0 0x7f352c19a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53956/0 (socket says 172.21.15.47:53956) 2024-04-17T05:58:47.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:47.177+0000 7f35324aa640 1 -- 172.21.15.47:0/1861754737 learned_addr learned my addr 172.21.15.47:0/1861754737 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:58:47.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:47.177+0000 7f3531ca9640 1 --2- 172.21.15.47:0/1861754737 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f352c101ff0 0x7f352c19aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:58:47.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:47.177+0000 7f35324aa640 1 -- 172.21.15.47:0/1861754737 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f352c101ff0 msgr2=0x7f352c19aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:58:47.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:47.177+0000 7f35324aa640 1 --2- 172.21.15.47:0/1861754737 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f352c101ff0 0x7f352c19aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:47.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:47.177+0000 7f35324aa640 1 -- 172.21.15.47:0/1861754737 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f352c1029b0 msgr2=0x7f352c1a1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:58:47.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:47.177+0000 7f3532cab640 1 --2- 172.21.15.47:0/1861754737 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f352c1029b0 0x7f352c1a1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:58:47.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:47.177+0000 7f35324aa640 1 --2- 172.21.15.47:0/1861754737 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f352c1029b0 0x7f352c1a1ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:47.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:47.177+0000 7f35324aa640 1 -- 172.21.15.47:0/1861754737 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f352c1a2130 con 0x7f352c100ea0 2024-04-17T05:58:47.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:47.178+0000 7f35324aa640 1 --2- 172.21.15.47:0/1861754737 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f352c100ea0 0x7f352c19a510 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3528009230 tx=0x7f3528005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:58:47.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:47.178+0000 7f351f7fe640 1 -- 172.21.15.47:0/1861754737 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3528041070 con 0x7f352c100ea0 2024-04-17T05:58:47.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:47.178+0000 7f351f7fe640 1 -- 172.21.15.47:0/1861754737 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35280054e0 con 0x7f352c100ea0 2024-04-17T05:58:47.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:47.178+0000 7f351f7fe640 1 -- 172.21.15.47:0/1861754737 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3528034440 con 0x7f352c100ea0 2024-04-17T05:58:47.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:47.178+0000 7f353470e640 1 -- 172.21.15.47:0/1861754737 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f352c1a23c0 con 0x7f352c100ea0 2024-04-17T05:58:47.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:47.178+0000 7f353470e640 1 -- 172.21.15.47:0/1861754737 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f352c1a2900 con 0x7f352c100ea0 2024-04-17T05:58:47.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:47.179+0000 7f353470e640 1 -- 172.21.15.47:0/1861754737 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f352c1a2c20 con 0x7f352c100ea0 2024-04-17T05:58:47.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:47.180+0000 7f351f7fe640 1 -- 172.21.15.47:0/1861754737 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f3528044070 con 0x7f352c100ea0 2024-04-17T05:58:47.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:47.181+0000 7f351f7fe640 1 --2- 172.21.15.47:0/1861754737 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f35180a3c30 0x7f35180a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:58:47.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:47.181+0000 7f3531ca9640 1 --2- 172.21.15.47:0/1861754737 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f35180a3c30 0x7f35180a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:58:47.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:47.181+0000 7f351f7fe640 1 -- 172.21.15.47:0/1861754737 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f35280f1ca0 con 0x7f352c100ea0 2024-04-17T05:58:47.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:47.181+0000 7f3531ca9640 1 --2- 172.21.15.47:0/1861754737 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f35180a3c30 0x7f35180a60f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3520005fd0 tx=0x7f3520005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:58:47.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:47.185+0000 7f351f7fe640 1 -- 172.21.15.47:0/1861754737 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f35280beb20 con 0x7f352c100ea0 2024-04-17T05:58:47.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:47.430+0000 7f353470e640 1 -- 172.21.15.47:0/1861754737 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f352c1a2df0 con 0x7f352c100ea0 2024-04-17T05:58:47.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:47.430+0000 7f351f7fe640 1 -- 172.21.15.47:0/1861754737 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3868 (secure 0 0 0) 0x7f35280be420 con 0x7f352c100ea0 2024-04-17T05:58:47.428 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-17T05:58:47.429 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3189958720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3189958720},{"type":"v1","addr":"172.21.15.47:6835","nonce":3189958720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14595,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/3143529973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":3143529973},{"type":"v1","addr":"172.21.15.47:6837","nonce":3143529973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/2398363754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":2398363754},{"type":"v1","addr":"172.21.15.103:6833","nonce":2398363754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/384473493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":384473493},{"type":"v1","addr":"172.21.15.98:6833","nonce":384473493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-17T05:58:47.429 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 5 2024-04-17T05:58:47.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:47.432+0000 7f353470e640 1 -- 172.21.15.47:0/1861754737 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f35180a3c30 msgr2=0x7f35180a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:58:47.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:47.432+0000 7f353470e640 1 --2- 172.21.15.47:0/1861754737 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f35180a3c30 0x7f35180a60f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3520005fd0 tx=0x7f3520005850 comp rx=0 tx=0).stop 2024-04-17T05:58:47.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:47.432+0000 7f353470e640 1 -- 172.21.15.47:0/1861754737 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f352c100ea0 msgr2=0x7f352c19a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:58:47.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:47.432+0000 7f353470e640 1 --2- 172.21.15.47:0/1861754737 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f352c100ea0 0x7f352c19a510 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3528009230 tx=0x7f3528005dd0 comp rx=0 tx=0).stop 2024-04-17T05:58:47.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:47.432+0000 7f353470e640 1 -- 172.21.15.47:0/1861754737 shutdown_connections 2024-04-17T05:58:47.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:47.432+0000 7f353470e640 1 --2- 172.21.15.47:0/1861754737 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f35180a3c30 0x7f35180a60f0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:47.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:47.432+0000 7f353470e640 1 --2- 172.21.15.47:0/1861754737 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f352c1029b0 0x7f352c1a1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:47.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:47.432+0000 7f353470e640 1 --2- 172.21.15.47:0/1861754737 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f352c101ff0 0x7f352c19aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:47.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:47.432+0000 7f353470e640 1 --2- 172.21.15.47:0/1861754737 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f352c100ea0 0x7f352c19a510 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:47.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:47.432+0000 7f353470e640 1 -- 172.21.15.47:0/1861754737 >> 172.21.15.47:0/1861754737 conn(0x7f352c0fc410 msgr2=0x7f352c0690f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:58:47.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:47.432+0000 7f353470e640 1 -- 172.21.15.47:0/1861754737 shutdown_connections 2024-04-17T05:58:47.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:47.432+0000 7f353470e640 1 -- 172.21.15.47:0/1861754737 wait complete. 2024-04-17T05:58:47.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:47 smithi047 ceph-mon[32223]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T05:58:47.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:47 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/1861754737' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T05:58:47.924 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:47 smithi098 ceph-mon[36516]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T05:58:47.925 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:47 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/1861754737' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T05:58:48.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:47 smithi103 ceph-mon[36243]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T05:58:48.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:47 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/1861754737' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T05:58:48.633 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 05:58:48 smithi098 systemd[1]: Started Ceph mds.e for 565db2d8-fc7e-11ee-bc91-c7b262605968. 2024-04-17T05:58:49.442 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T05:58:49.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:49.565+0000 7fd5f8a14640 1 -- 172.21.15.47:0/2287532897 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5f4107d40 msgr2=0x7fd5f410a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:58:49.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:49.565+0000 7fd5f8a14640 1 --2- 172.21.15.47:0/2287532897 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5f4107d40 0x7fd5f410a130 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec009ca0 tx=0x7fd5ec02c0f0 comp rx=0 tx=0).stop 2024-04-17T05:58:49.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:49.565+0000 7fd5f8a14640 1 -- 172.21.15.47:0/2287532897 shutdown_connections 2024-04-17T05:58:49.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:49.565+0000 7fd5f8a14640 1 --2- 172.21.15.47:0/2287532897 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5f4107d40 0x7fd5f410a130 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:49.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:49.565+0000 7fd5f8a14640 1 --2- 172.21.15.47:0/2287532897 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd5f4105410 0x7fd5f4107800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:49.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:49.565+0000 7fd5f8a14640 1 --2- 172.21.15.47:0/2287532897 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd5f4102ae0 0x7fd5f4104ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:49.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:49.565+0000 7fd5f8a14640 1 -- 172.21.15.47:0/2287532897 >> 172.21.15.47:0/2287532897 conn(0x7fd5f40fc450 msgr2=0x7fd5f40fe870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:58:49.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:49.565+0000 7fd5f8a14640 1 -- 172.21.15.47:0/2287532897 shutdown_connections 2024-04-17T05:58:49.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:49.565+0000 7fd5f8a14640 1 -- 172.21.15.47:0/2287532897 wait complete. 2024-04-17T05:58:49.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:49.566+0000 7fd5f8a14640 1 Processor -- start 2024-04-17T05:58:49.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:49.566+0000 7fd5f8a14640 1 -- start start 2024-04-17T05:58:49.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:49.566+0000 7fd5f8a14640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5f4102ae0 0x7fd5f4198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:58:49.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:49.566+0000 7fd5f8a14640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd5f4105410 0x7fd5f41987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:58:49.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:49.566+0000 7fd5f8a14640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd5f4107d40 0x7fd5f419f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:58:49.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:49.566+0000 7fd5f8a14640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f410dc70 con 0x7fd5f4102ae0 2024-04-17T05:58:49.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:49.566+0000 7fd5f8a14640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f410daf0 con 0x7fd5f4107d40 2024-04-17T05:58:49.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:49.566+0000 7fd5f8a14640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f410ddf0 con 0x7fd5f4105410 2024-04-17T05:58:49.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:49.566+0000 7fd5f259c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5f4102ae0 0x7fd5f4198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:58:49.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:49.566+0000 7fd5f259c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5f4102ae0 0x7fd5f4198280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53986/0 (socket says 172.21.15.47:53986) 2024-04-17T05:58:49.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:49.566+0000 7fd5f259c640 1 -- 172.21.15.47:0/4206086752 learned_addr learned my addr 172.21.15.47:0/4206086752 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:58:49.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:49.566+0000 7fd5ebfff640 1 --2- 172.21.15.47:0/4206086752 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd5f4105410 0x7fd5f41987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:58:49.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:49.567+0000 7fd5f2d9d640 1 --2- 172.21.15.47:0/4206086752 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd5f4107d40 0x7fd5f419f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:58:49.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:49.567+0000 7fd5f259c640 1 -- 172.21.15.47:0/4206086752 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd5f4105410 msgr2=0x7fd5f41987c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:58:49.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:49.567+0000 7fd5f259c640 1 --2- 172.21.15.47:0/4206086752 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd5f4105410 0x7fd5f41987c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:49.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:49.567+0000 7fd5f259c640 1 -- 172.21.15.47:0/4206086752 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd5f4107d40 msgr2=0x7fd5f419f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:58:49.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:49.567+0000 7fd5f259c640 1 --2- 172.21.15.47:0/4206086752 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd5f4107d40 0x7fd5f419f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:49.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:49.567+0000 7fd5f259c640 1 -- 172.21.15.47:0/4206086752 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5f419fe50 con 0x7fd5f4102ae0 2024-04-17T05:58:49.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:49.567+0000 7fd5ebfff640 1 --2- 172.21.15.47:0/4206086752 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd5f4105410 0x7fd5f41987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T05:58:49.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:49.567+0000 7fd5f259c640 1 --2- 172.21.15.47:0/4206086752 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5f4102ae0 0x7fd5f4198280 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd5e400e9f0 tx=0x7fd5e400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:58:49.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:49.567+0000 7fd5eb7fe640 1 -- 172.21.15.47:0/4206086752 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5e400cd80 con 0x7fd5f4102ae0 2024-04-17T05:58:49.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:49.567+0000 7fd5eb7fe640 1 -- 172.21.15.47:0/4206086752 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5e4010910 con 0x7fd5f4102ae0 2024-04-17T05:58:49.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:49.567+0000 7fd5f8a14640 1 -- 172.21.15.47:0/4206086752 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5f41a0140 con 0x7fd5f4102ae0 2024-04-17T05:58:49.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:49.567+0000 7fd5eb7fe640 1 -- 172.21.15.47:0/4206086752 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5e4018990 con 0x7fd5f4102ae0 2024-04-17T05:58:49.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:49.567+0000 7fd5f8a14640 1 -- 172.21.15.47:0/4206086752 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5f41a0690 con 0x7fd5f4102ae0 2024-04-17T05:58:49.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:49.568+0000 7fd5f8a14640 1 -- 172.21.15.47:0/4206086752 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5f40fdf40 con 0x7fd5f4102ae0 2024-04-17T05:58:49.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:49.569+0000 7fd5eb7fe640 1 -- 172.21.15.47:0/4206086752 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fd5e4010ac0 con 0x7fd5f4102ae0 2024-04-17T05:58:49.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:49.570+0000 7fd5eb7fe640 1 --2- 172.21.15.47:0/4206086752 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fd5e00a3bb0 0x7fd5e00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:58:49.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:49.570+0000 7fd5ebfff640 1 --2- 172.21.15.47:0/4206086752 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fd5e00a3bb0 0x7fd5e00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:58:49.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:49.570+0000 7fd5eb7fe640 1 -- 172.21.15.47:0/4206086752 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fd5e40d0730 con 0x7fd5f4102ae0 2024-04-17T05:58:49.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:49.570+0000 7fd5ebfff640 1 --2- 172.21.15.47:0/4206086752 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fd5e00a3bb0 0x7fd5e00a6070 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd5f4199740 tx=0x7fd5dc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:58:49.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:49.574+0000 7fd5eb7fe640 1 -- 172.21.15.47:0/4206086752 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fd5e409cec0 con 0x7fd5f4102ae0 2024-04-17T05:58:49.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:49 smithi103 ceph-mon[36243]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T05:58:49.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:49 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:49.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:49 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:49.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:49 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:49.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:49 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-17T05:58:49.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:49 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-17T05:58:49.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:49 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:58:49.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:49 smithi103 ceph-mon[36243]: Deploying daemon mds.f on smithi103 2024-04-17T05:58:49.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:49 smithi103 ceph-mon[36243]: mds.? [v2:172.21.15.98:6834/1966500278,v1:172.21.15.98:6835/1966500278] up:boot 2024-04-17T05:58:49.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:49 smithi103 ceph-mon[36243]: fsmap 5 up:standby 2024-04-17T05:58:49.785 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:49 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-17T05:58:49.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:49.801+0000 7fd5f8a14640 1 -- 172.21.15.47:0/4206086752 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd5f402d5d0 con 0x7fd5f4102ae0 2024-04-17T05:58:49.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:49.801+0000 7fd5eb7fe640 1 -- 172.21.15.47:0/4206086752 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4693 (secure 0 0 0) 0x7fd5e40a11c0 con 0x7fd5f4102ae0 2024-04-17T05:58:49.799 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-17T05:58:49.799 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3189958720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3189958720},{"type":"v1","addr":"172.21.15.47:6835","nonce":3189958720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14595,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/3143529973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":3143529973},{"type":"v1","addr":"172.21.15.47:6837","nonce":3143529973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/2398363754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":2398363754},{"type":"v1","addr":"172.21.15.103:6833","nonce":2398363754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/384473493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":384473493},{"type":"v1","addr":"172.21.15.98:6833","nonce":384473493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24449,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/1966500278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":1966500278},{"type":"v1","addr":"172.21.15.98:6835","nonce":1966500278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-17T05:58:49.799 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 6 2024-04-17T05:58:49.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:49.803+0000 7fd5f8a14640 1 -- 172.21.15.47:0/4206086752 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fd5e00a3bb0 msgr2=0x7fd5e00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:58:49.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:49.803+0000 7fd5f8a14640 1 --2- 172.21.15.47:0/4206086752 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fd5e00a3bb0 0x7fd5e00a6070 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd5f4199740 tx=0x7fd5dc005f90 comp rx=0 tx=0).stop 2024-04-17T05:58:49.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:49.803+0000 7fd5f8a14640 1 -- 172.21.15.47:0/4206086752 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5f4102ae0 msgr2=0x7fd5f4198280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:58:49.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:49.803+0000 7fd5f8a14640 1 --2- 172.21.15.47:0/4206086752 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5f4102ae0 0x7fd5f4198280 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd5e400e9f0 tx=0x7fd5e400eeb0 comp rx=0 tx=0).stop 2024-04-17T05:58:49.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:49.803+0000 7fd5f8a14640 1 -- 172.21.15.47:0/4206086752 shutdown_connections 2024-04-17T05:58:49.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:49.803+0000 7fd5f8a14640 1 --2- 172.21.15.47:0/4206086752 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fd5e00a3bb0 0x7fd5e00a6070 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:49.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:49.803+0000 7fd5f8a14640 1 --2- 172.21.15.47:0/4206086752 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd5f4107d40 0x7fd5f419f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:49.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:49.803+0000 7fd5f8a14640 1 --2- 172.21.15.47:0/4206086752 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd5f4105410 0x7fd5f41987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:49.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:49.803+0000 7fd5f8a14640 1 --2- 172.21.15.47:0/4206086752 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5f4102ae0 0x7fd5f4198280 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:49.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:49.803+0000 7fd5f8a14640 1 -- 172.21.15.47:0/4206086752 >> 172.21.15.47:0/4206086752 conn(0x7fd5f40fc450 msgr2=0x7fd5f4103420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:58:49.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:49.804+0000 7fd5f8a14640 1 -- 172.21.15.47:0/4206086752 shutdown_connections 2024-04-17T05:58:49.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:49.804+0000 7fd5f8a14640 1 -- 172.21.15.47:0/4206086752 wait complete. 2024-04-17T05:58:49.811 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:49 smithi047 ceph-mon[32223]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T05:58:49.811 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:49 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:49.812 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:49 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:49.812 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:49 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:49.812 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:49 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-17T05:58:49.812 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:49 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-17T05:58:49.812 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:49 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:58:49.812 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:49 smithi047 ceph-mon[32223]: Deploying daemon mds.f on smithi103 2024-04-17T05:58:49.813 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:49 smithi047 ceph-mon[32223]: mds.? [v2:172.21.15.98:6834/1966500278,v1:172.21.15.98:6835/1966500278] up:boot 2024-04-17T05:58:49.813 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:49 smithi047 ceph-mon[32223]: fsmap 5 up:standby 2024-04-17T05:58:49.813 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:49 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-17T05:58:49.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:49 smithi098 ceph-mon[36516]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T05:58:49.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:49 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:49.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:49 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:49.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:49 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:49.884 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:49 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-17T05:58:49.884 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:49 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-17T05:58:49.884 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:49 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:58:49.884 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:49 smithi098 ceph-mon[36516]: Deploying daemon mds.f on smithi103 2024-04-17T05:58:49.884 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:49 smithi098 ceph-mon[36516]: mds.? [v2:172.21.15.98:6834/1966500278,v1:172.21.15.98:6835/1966500278] up:boot 2024-04-17T05:58:49.884 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:49 smithi098 ceph-mon[36516]: fsmap 5 up:standby 2024-04-17T05:58:49.885 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:49 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-17T05:58:50.614 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:50 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/4206086752' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T05:58:50.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:50 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/4206086752' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T05:58:50.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:50 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/4206086752' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T05:58:51.016 INFO:journalctl@ceph.mds.f.smithi103.stdout:Apr 17 05:58:50 smithi103 systemd[1]: Starting Ceph mds.f for 565db2d8-fc7e-11ee-bc91-c7b262605968... 2024-04-17T05:58:51.281 INFO:journalctl@ceph.mds.f.smithi103.stdout:Apr 17 05:58:51 smithi103 podman[66276]: 2024-04-17 05:58:50.921166357 +0000 UTC m=+0.017739305 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-17T05:58:51.281 INFO:journalctl@ceph.mds.f.smithi103.stdout:Apr 17 05:58:51 smithi103 podman[66276]: 2024-04-17 05:58:51.039273028 +0000 UTC m=+0.135845970 container create fa8c8ed243365c17696df32ed92348728d19f73ea9d4adef1fcdc2528d78e3c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-f, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-04-17T05:58:51.282 INFO:journalctl@ceph.mds.f.smithi103.stdout:Apr 17 05:58:51 smithi103 podman[66276]: 2024-04-17 05:58:51.283895117 +0000 UTC m=+0.380468062 container init fa8c8ed243365c17696df32ed92348728d19f73ea9d4adef1fcdc2528d78e3c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-f, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, ceph=True, org.label-schema.schema-version=1.0) 2024-04-17T05:58:51.533 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:51 smithi103 ceph-mon[36243]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T05:58:51.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:51 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:51.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:51 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:51.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:51 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:51.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:51 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-17T05:58:51.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:51 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-17T05:58:51.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:51 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:58:51.534 INFO:journalctl@ceph.mds.f.smithi103.stdout:Apr 17 05:58:51 smithi103 podman[66276]: 2024-04-17 05:58:51.28683511 +0000 UTC m=+0.383408057 container start fa8c8ed243365c17696df32ed92348728d19f73ea9d4adef1fcdc2528d78e3c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-f, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True) 2024-04-17T05:58:51.534 INFO:journalctl@ceph.mds.f.smithi103.stdout:Apr 17 05:58:51 smithi103 ceph-mds[66296]: -- 172.21.15.103:0/4294617969 <== mon.1 v2:172.21.15.103:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55dca9723a40 con 0x55dca96f6800 2024-04-17T05:58:51.535 INFO:journalctl@ceph.mds.f.smithi103.stdout:Apr 17 05:58:51 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-f[66292]: starting mds.f at 2024-04-17T05:58:51.535 INFO:journalctl@ceph.mds.f.smithi103.stdout:Apr 17 05:58:51 smithi103 bash[66276]: fa8c8ed243365c17696df32ed92348728d19f73ea9d4adef1fcdc2528d78e3c7 2024-04-17T05:58:51.535 INFO:journalctl@ceph.mds.f.smithi103.stdout:Apr 17 05:58:51 smithi103 systemd[1]: Started Ceph mds.f for 565db2d8-fc7e-11ee-bc91-c7b262605968. 2024-04-17T05:58:51.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:51 smithi098 ceph-mon[36516]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T05:58:51.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:51 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:51.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:51 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:51.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:51 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:51.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:51 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-17T05:58:51.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:51 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-17T05:58:51.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:51 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:58:51.815 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T05:58:51.845 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:51 smithi047 ceph-mon[32223]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T05:58:51.845 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:51 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:51.845 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:51 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:51.845 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:51 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:51.845 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:51 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-17T05:58:51.845 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:51 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-17T05:58:51.845 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:51 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:58:51.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:51.935+0000 7f293c775640 1 -- 172.21.15.47:0/2554398358 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f29341029b0 msgr2=0x7f29341092d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:58:51.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:51.935+0000 7f293c775640 1 --2- 172.21.15.47:0/2554398358 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f29341029b0 0x7f29341092d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f292c009ca0 tx=0x7f292c02c0f0 comp rx=0 tx=0).stop 2024-04-17T05:58:51.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:51.935+0000 7f293c775640 1 -- 172.21.15.47:0/2554398358 shutdown_connections 2024-04-17T05:58:51.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:51.935+0000 7f293c775640 1 --2- 172.21.15.47:0/2554398358 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f29341029b0 0x7f29341092d0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:51.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:51.935+0000 7f293c775640 1 --2- 172.21.15.47:0/2554398358 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2934101ff0 0x7f2934102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:51.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:51.935+0000 7f293c775640 1 --2- 172.21.15.47:0/2554398358 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2934100ea0 0x7f29341012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:51.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:51.935+0000 7f293c775640 1 -- 172.21.15.47:0/2554398358 >> 172.21.15.47:0/2554398358 conn(0x7f29340fc410 msgr2=0x7f29340fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:58:51.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:51.935+0000 7f293c775640 1 -- 172.21.15.47:0/2554398358 shutdown_connections 2024-04-17T05:58:51.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:51.935+0000 7f293c775640 1 -- 172.21.15.47:0/2554398358 wait complete. 2024-04-17T05:58:51.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:51.936+0000 7f293c775640 1 Processor -- start 2024-04-17T05:58:51.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:51.936+0000 7f293c775640 1 -- start start 2024-04-17T05:58:51.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:51.936+0000 7f293c775640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2934100ea0 0x7f29341982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:58:51.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:51.936+0000 7f293c775640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2934101ff0 0x7f29341987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:58:51.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:51.936+0000 7f293c775640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f29341029b0 0x7f293419f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:58:51.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:51.936+0000 7f293c775640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f293410cb40 con 0x7f2934101ff0 2024-04-17T05:58:51.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:51.936+0000 7f293c775640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f293410c9c0 con 0x7f2934100ea0 2024-04-17T05:58:51.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:51.936+0000 7f293c775640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f293410ccc0 con 0x7f29341029b0 2024-04-17T05:58:51.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:51.936+0000 7f2939d10640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2934101ff0 0x7f29341987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:58:51.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:51.936+0000 7f2939d10640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2934101ff0 0x7f29341987e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54004/0 (socket says 172.21.15.47:54004) 2024-04-17T05:58:51.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:51.936+0000 7f2939d10640 1 -- 172.21.15.47:0/2194334389 learned_addr learned my addr 172.21.15.47:0/2194334389 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:58:51.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:51.936+0000 7f293ad12640 1 --2- 172.21.15.47:0/2194334389 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f29341029b0 0x7f293419f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:58:51.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:51.936+0000 7f293a511640 1 --2- 172.21.15.47:0/2194334389 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2934100ea0 0x7f29341982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:58:51.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:51.936+0000 7f2939d10640 1 -- 172.21.15.47:0/2194334389 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f29341029b0 msgr2=0x7f293419f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:58:51.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:51.936+0000 7f2939d10640 1 --2- 172.21.15.47:0/2194334389 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f29341029b0 0x7f293419f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:51.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:51.936+0000 7f2939d10640 1 -- 172.21.15.47:0/2194334389 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2934100ea0 msgr2=0x7f29341982a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:58:51.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:51.936+0000 7f2939d10640 1 --2- 172.21.15.47:0/2194334389 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2934100ea0 0x7f29341982a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:51.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:51.936+0000 7f2939d10640 1 -- 172.21.15.47:0/2194334389 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f293419fec0 con 0x7f2934101ff0 2024-04-17T05:58:51.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:51.936+0000 7f293a511640 1 --2- 172.21.15.47:0/2194334389 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2934100ea0 0x7f29341982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T05:58:51.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:51.937+0000 7f2939d10640 1 --2- 172.21.15.47:0/2194334389 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2934101ff0 0x7f29341987e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f292400e9c0 tx=0x7f292400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:58:51.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:51.937+0000 7f292b7fe640 1 -- 172.21.15.47:0/2194334389 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f292400cd80 con 0x7f2934101ff0 2024-04-17T05:58:51.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:51.937+0000 7f292b7fe640 1 -- 172.21.15.47:0/2194334389 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29240035f0 con 0x7f2934101ff0 2024-04-17T05:58:51.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:51.937+0000 7f293c775640 1 -- 172.21.15.47:0/2194334389 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29341a01b0 con 0x7f2934101ff0 2024-04-17T05:58:51.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:51.937+0000 7f292b7fe640 1 -- 172.21.15.47:0/2194334389 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2924010690 con 0x7f2934101ff0 2024-04-17T05:58:51.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:51.937+0000 7f293c775640 1 -- 172.21.15.47:0/2194334389 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29341a0700 con 0x7f2934101ff0 2024-04-17T05:58:51.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:51.938+0000 7f293c775640 1 -- 172.21.15.47:0/2194334389 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29341a09c0 con 0x7f2934101ff0 2024-04-17T05:58:51.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:51.939+0000 7f292b7fe640 1 -- 172.21.15.47:0/2194334389 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f29240037a0 con 0x7f2934101ff0 2024-04-17T05:58:51.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:51.940+0000 7f292b7fe640 1 --2- 172.21.15.47:0/2194334389 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f29200a3c30 0x7f29200a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:58:51.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:51.940+0000 7f293a511640 1 --2- 172.21.15.47:0/2194334389 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f29200a3c30 0x7f29200a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:58:51.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:51.940+0000 7f292b7fe640 1 -- 172.21.15.47:0/2194334389 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f29240d0450 con 0x7f2934101ff0 2024-04-17T05:58:51.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:51.940+0000 7f293a511640 1 --2- 172.21.15.47:0/2194334389 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f29200a3c30 0x7f29200a60f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f29300058c0 tx=0x7f2930005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:58:51.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:51.943+0000 7f292b7fe640 1 -- 172.21.15.47:0/2194334389 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f292409d230 con 0x7f2934101ff0 2024-04-17T05:58:52.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:52.171+0000 7f293c775640 1 -- 172.21.15.47:0/2194334389 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f29341a0b90 con 0x7f2934101ff0 2024-04-17T05:58:52.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:52.172+0000 7f292b7fe640 1 -- 172.21.15.47:0/2194334389 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5518 (secure 0 0 0) 0x7f292409cb30 con 0x7f2934101ff0 2024-04-17T05:58:52.170 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-17T05:58:52.170 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3189958720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3189958720},{"type":"v1","addr":"172.21.15.47:6835","nonce":3189958720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14595,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/3143529973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":3143529973},{"type":"v1","addr":"172.21.15.47:6837","nonce":3143529973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/2398363754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":2398363754},{"type":"v1","addr":"172.21.15.103:6833","nonce":2398363754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/384473493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":384473493},{"type":"v1","addr":"172.21.15.98:6833","nonce":384473493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/270543522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":270543522},{"type":"v1","addr":"172.21.15.103:6835","nonce":270543522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24449,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/1966500278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":1966500278},{"type":"v1","addr":"172.21.15.98:6835","nonce":1966500278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-17T05:58:52.170 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 7 2024-04-17T05:58:52.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:52.174+0000 7f293c775640 1 -- 172.21.15.47:0/2194334389 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f29200a3c30 msgr2=0x7f29200a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:58:52.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:52.174+0000 7f293c775640 1 --2- 172.21.15.47:0/2194334389 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f29200a3c30 0x7f29200a60f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f29300058c0 tx=0x7f2930005850 comp rx=0 tx=0).stop 2024-04-17T05:58:52.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:52.174+0000 7f293c775640 1 -- 172.21.15.47:0/2194334389 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2934101ff0 msgr2=0x7f29341987e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:58:52.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:52.174+0000 7f293c775640 1 --2- 172.21.15.47:0/2194334389 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2934101ff0 0x7f29341987e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f292400e9c0 tx=0x7f292400ee80 comp rx=0 tx=0).stop 2024-04-17T05:58:52.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:52.174+0000 7f293c775640 1 -- 172.21.15.47:0/2194334389 shutdown_connections 2024-04-17T05:58:52.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:52.174+0000 7f293c775640 1 --2- 172.21.15.47:0/2194334389 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f29200a3c30 0x7f29200a60f0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:52.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:52.174+0000 7f293c775640 1 --2- 172.21.15.47:0/2194334389 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f29341029b0 0x7f293419f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:52.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:52.174+0000 7f293c775640 1 --2- 172.21.15.47:0/2194334389 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2934101ff0 0x7f29341987e0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:52.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:52.174+0000 7f293c775640 1 --2- 172.21.15.47:0/2194334389 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2934100ea0 0x7f29341982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:52.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:52.174+0000 7f293c775640 1 -- 172.21.15.47:0/2194334389 >> 172.21.15.47:0/2194334389 conn(0x7f29340fc410 msgr2=0x7f29340fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:58:52.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:52.174+0000 7f293c775640 1 -- 172.21.15.47:0/2194334389 shutdown_connections 2024-04-17T05:58:52.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:52.174+0000 7f293c775640 1 -- 172.21.15.47:0/2194334389 wait complete. 2024-04-17T05:58:52.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:52 smithi103 ceph-mon[36243]: Deploying daemon mds.c on smithi103 2024-04-17T05:58:52.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:52 smithi103 ceph-mon[36243]: mds.? [v2:172.21.15.103:6834/270543522,v1:172.21.15.103:6835/270543522] up:boot 2024-04-17T05:58:52.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:52 smithi103 ceph-mon[36243]: fsmap 6 up:standby 2024-04-17T05:58:52.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:52 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-17T05:58:52.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:52 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/2194334389' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T05:58:52.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:52 smithi098 ceph-mon[36516]: Deploying daemon mds.c on smithi103 2024-04-17T05:58:52.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:52 smithi098 ceph-mon[36516]: mds.? [v2:172.21.15.103:6834/270543522,v1:172.21.15.103:6835/270543522] up:boot 2024-04-17T05:58:52.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:52 smithi098 ceph-mon[36516]: fsmap 6 up:standby 2024-04-17T05:58:52.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:52 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-17T05:58:52.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:52 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/2194334389' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T05:58:52.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:52 smithi047 ceph-mon[32223]: Deploying daemon mds.c on smithi103 2024-04-17T05:58:52.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:52 smithi047 ceph-mon[32223]: mds.? [v2:172.21.15.103:6834/270543522,v1:172.21.15.103:6835/270543522] up:boot 2024-04-17T05:58:52.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:52 smithi047 ceph-mon[32223]: fsmap 6 up:standby 2024-04-17T05:58:52.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:52 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-17T05:58:52.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:52 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/2194334389' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T05:58:53.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:53 smithi103 ceph-mon[36243]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T05:58:53.784 INFO:journalctl@ceph.mds.c.smithi103.stdout:Apr 17 05:58:53 smithi103 systemd[1]: Starting Ceph mds.c for 565db2d8-fc7e-11ee-bc91-c7b262605968... 2024-04-17T05:58:53.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:53 smithi098 ceph-mon[36516]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T05:58:53.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:53 smithi047 ceph-mon[32223]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T05:58:54.066 INFO:journalctl@ceph.mds.c.smithi103.stdout:Apr 17 05:58:53 smithi103 podman[66585]: 2024-04-17 05:58:53.697298967 +0000 UTC m=+0.017957684 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-17T05:58:54.066 INFO:journalctl@ceph.mds.c.smithi103.stdout:Apr 17 05:58:53 smithi103 podman[66585]: 2024-04-17 05:58:53.808206051 +0000 UTC m=+0.128864769 container create 957484b8ec18dd512d85ee0307b416fabc87f42ce23c7a99a08c732ae4f5154f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-c, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416) 2024-04-17T05:58:54.066 INFO:journalctl@ceph.mds.c.smithi103.stdout:Apr 17 05:58:54 smithi103 podman[66585]: 2024-04-17 05:58:54.06629036 +0000 UTC m=+0.386949078 container init 957484b8ec18dd512d85ee0307b416fabc87f42ce23c7a99a08c732ae4f5154f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-c, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-17T05:58:54.183 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T05:58:54.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:54.307+0000 7f1de6816640 1 -- 172.21.15.47:0/967340769 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1de0101fb0 msgr2=0x7f1de01023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:58:54.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:54.307+0000 7f1de6816640 1 --2- 172.21.15.47:0/967340769 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1de0101fb0 0x7f1de01023d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1dd0009900 tx=0x7f1dd002bc30 comp rx=0 tx=0).stop 2024-04-17T05:58:54.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:54.307+0000 7f1de6816640 1 -- 172.21.15.47:0/967340769 shutdown_connections 2024-04-17T05:58:54.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:54.307+0000 7f1de6816640 1 --2- 172.21.15.47:0/967340769 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1de0103b60 0x7f1de010a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:54.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:54.307+0000 7f1de6816640 1 --2- 172.21.15.47:0/967340769 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1de01031a0 0x7f1de0103620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:54.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:54.307+0000 7f1de6816640 1 --2- 172.21.15.47:0/967340769 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1de0101fb0 0x7f1de01023d0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:54.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:54.307+0000 7f1de6816640 1 -- 172.21.15.47:0/967340769 >> 172.21.15.47:0/967340769 conn(0x7f1de00fd560 msgr2=0x7f1de00ff980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:58:54.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:54.307+0000 7f1de6816640 1 -- 172.21.15.47:0/967340769 shutdown_connections 2024-04-17T05:58:54.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:54.307+0000 7f1de6816640 1 -- 172.21.15.47:0/967340769 wait complete. 2024-04-17T05:58:54.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:54.308+0000 7f1de6816640 1 Processor -- start 2024-04-17T05:58:54.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:54.308+0000 7f1de6816640 1 -- start start 2024-04-17T05:58:54.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:54.308+0000 7f1de6816640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1de0101fb0 0x7f1de0198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:58:54.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:54.308+0000 7f1de6816640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1de01031a0 0x7f1de0198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:58:54.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:54.308+0000 7f1de6816640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1de0103b60 0x7f1de019f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:58:54.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:54.308+0000 7f1de6816640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f1de010dc40 con 0x7f1de01031a0 2024-04-17T05:58:54.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:54.308+0000 7f1de6816640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1de010dac0 con 0x7f1de0101fb0 2024-04-17T05:58:54.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:54.308+0000 7f1de6816640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1de010ddc0 con 0x7f1de0103b60 2024-04-17T05:58:54.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:54.308+0000 7f1ddf7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1de01031a0 0x7f1de0198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:58:54.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:54.309+0000 7f1ddf7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1de01031a0 0x7f1de0198750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54032/0 (socket says 172.21.15.47:54032) 2024-04-17T05:58:54.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:54.309+0000 7f1ddf7fe640 1 -- 172.21.15.47:0/4029931282 learned_addr learned my addr 172.21.15.47:0/4029931282 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:58:54.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:54.309+0000 7f1de4db3640 1 --2- 172.21.15.47:0/4029931282 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1de0103b60 0x7f1de019f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:58:54.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:54.309+0000 7f1ddffff640 1 --2- 172.21.15.47:0/4029931282 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1de0101fb0 0x7f1de0198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:58:54.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:54.309+0000 7f1ddf7fe640 1 -- 172.21.15.47:0/4029931282 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1de0103b60 msgr2=0x7f1de019f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:58:54.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:54.309+0000 7f1ddf7fe640 1 --2- 172.21.15.47:0/4029931282 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1de0103b60 0x7f1de019f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:54.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:54.309+0000 7f1ddf7fe640 1 -- 172.21.15.47:0/4029931282 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1de0101fb0 msgr2=0x7f1de0198210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:58:54.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:54.309+0000 7f1ddf7fe640 1 --2- 172.21.15.47:0/4029931282 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1de0101fb0 0x7f1de0198210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:54.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:54.309+0000 7f1ddf7fe640 1 -- 172.21.15.47:0/4029931282 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1de019fe30 con 0x7f1de01031a0 2024-04-17T05:58:54.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:54.309+0000 7f1ddf7fe640 1 --2- 172.21.15.47:0/4029931282 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1de01031a0 0x7f1de0198750 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1dd400c920 tx=0x7f1dd400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:58:54.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:54.309+0000 7f1ddd7fa640 1 -- 172.21.15.47:0/4029931282 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dd4007c70 con 0x7f1de01031a0 2024-04-17T05:58:54.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:54.309+0000 7f1ddd7fa640 1 -- 172.21.15.47:0/4029931282 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1dd4007e10 con 0x7f1de01031a0 2024-04-17T05:58:54.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:54.309+0000 7f1de6816640 1 -- 172.21.15.47:0/4029931282 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1de01a0120 con 0x7f1de01031a0 2024-04-17T05:58:54.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:54.309+0000 7f1ddd7fa640 1 -- 172.21.15.47:0/4029931282 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dd4018890 con 0x7f1de01031a0 2024-04-17T05:58:54.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:54.309+0000 7f1de6816640 1 -- 172.21.15.47:0/4029931282 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1de01a0670 con 0x7f1de01031a0 2024-04-17T05:58:54.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:54.310+0000 7f1de6816640 1 -- 172.21.15.47:0/4029931282 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1de01092d0 con 0x7f1de01031a0 2024-04-17T05:58:54.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:54.312+0000 7f1ddd7fa640 1 -- 172.21.15.47:0/4029931282 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f1dd4018a70 con 0x7f1de01031a0 2024-04-17T05:58:54.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:54.312+0000 7f1ddd7fa640 1 --2- 172.21.15.47:0/4029931282 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f1dc80a3bb0 0x7f1dc80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:58:54.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:54.312+0000 7f1ddd7fa640 1 -- 172.21.15.47:0/4029931282 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f1dd40cfff0 con 0x7f1de01031a0 2024-04-17T05:58:54.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:54.312+0000 7f1ddffff640 1 --2- 172.21.15.47:0/4029931282 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f1dc80a3bb0 0x7f1dc80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:58:54.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:54.313+0000 7f1ddffff640 1 --2- 172.21.15.47:0/4029931282 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f1dc80a3bb0 0x7f1dc80a6070 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1dd0000c00 tx=0x7f1dd0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:58:54.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:54.317+0000 7f1ddd7fa640 1 -- 172.21.15.47:0/4029931282 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f1dd4014030 con 0x7f1de01031a0 2024-04-17T05:58:54.534 INFO:journalctl@ceph.mds.c.smithi103.stdout:Apr 17 05:58:54 smithi103 podman[66585]: 2024-04-17 05:58:54.06898886 +0000 UTC m=+0.389647578 container start 957484b8ec18dd512d85ee0307b416fabc87f42ce23c7a99a08c732ae4f5154f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-17T05:58:54.534 INFO:journalctl@ceph.mds.c.smithi103.stdout:Apr 17 05:58:54 smithi103 ceph-mds[66604]: -- 172.21.15.103:0/2737069795 <== mon.1 v2:172.21.15.103:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d54becda40 con 0x55d54bea0800 2024-04-17T05:58:54.534 INFO:journalctl@ceph.mds.c.smithi103.stdout:Apr 17 05:58:54 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-c[66600]: starting mds.c at 2024-04-17T05:58:54.534 INFO:journalctl@ceph.mds.c.smithi103.stdout:Apr 17 05:58:54 smithi103 bash[66585]: 957484b8ec18dd512d85ee0307b416fabc87f42ce23c7a99a08c732ae4f5154f 2024-04-17T05:58:54.534 INFO:journalctl@ceph.mds.c.smithi103.stdout:Apr 17 05:58:54 smithi103 systemd[1]: Started Ceph mds.c for 565db2d8-fc7e-11ee-bc91-c7b262605968. 2024-04-17T05:58:54.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:54.545+0000 7f1de6816640 1 -- 172.21.15.47:0/4029931282 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1de002d5d0 con 0x7f1de01031a0 2024-04-17T05:58:54.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:54.546+0000 7f1ddd7fa640 1 -- 172.21.15.47:0/4029931282 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6343 (secure 0 0 0) 0x7f1dd409ce70 con 0x7f1de01031a0 2024-04-17T05:58:54.544 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-17T05:58:54.544 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3189958720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3189958720},{"type":"v1","addr":"172.21.15.47:6835","nonce":3189958720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14595,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/3143529973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":3143529973},{"type":"v1","addr":"172.21.15.47:6837","nonce":3143529973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/2398363754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":2398363754},{"type":"v1","addr":"172.21.15.103:6833","nonce":2398363754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/384473493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":384473493},{"type":"v1","addr":"172.21.15.98:6833","nonce":384473493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/270543522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":270543522},{"type":"v1","addr":"172.21.15.103:6835","nonce":270543522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24448,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/116936024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":116936024},{"type":"v1","addr":"172.21.15.103:6837","nonce":116936024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24449,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/1966500278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":1966500278},{"type":"v1","addr":"172.21.15.98:6835","nonce":1966500278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-17T05:58:54.544 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 8 2024-04-17T05:58:54.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:54.548+0000 7f1de6816640 1 -- 172.21.15.47:0/4029931282 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f1dc80a3bb0 msgr2=0x7f1dc80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:58:54.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:54.548+0000 7f1de6816640 1 --2- 172.21.15.47:0/4029931282 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f1dc80a3bb0 0x7f1dc80a6070 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1dd0000c00 tx=0x7f1dd0005fd0 comp rx=0 tx=0).stop 2024-04-17T05:58:54.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:54.548+0000 7f1de6816640 1 -- 172.21.15.47:0/4029931282 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1de01031a0 msgr2=0x7f1de0198750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:58:54.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:54.548+0000 7f1de6816640 1 --2- 172.21.15.47:0/4029931282 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1de01031a0 0x7f1de0198750 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1dd400c920 tx=0x7f1dd400cde0 comp rx=0 tx=0).stop 2024-04-17T05:58:54.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:54.548+0000 7f1de6816640 1 -- 172.21.15.47:0/4029931282 shutdown_connections 2024-04-17T05:58:54.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:54.548+0000 7f1de6816640 1 --2- 172.21.15.47:0/4029931282 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f1dc80a3bb0 0x7f1dc80a6070 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:54.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:54.548+0000 7f1de6816640 1 --2- 172.21.15.47:0/4029931282 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1de0103b60 0x7f1de019f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:54.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:54.548+0000 7f1de6816640 1 --2- 172.21.15.47:0/4029931282 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1de01031a0 0x7f1de0198750 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:54.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:54.548+0000 7f1de6816640 1 --2- 172.21.15.47:0/4029931282 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1de0101fb0 0x7f1de0198210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:54.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:54.548+0000 7f1de6816640 1 -- 172.21.15.47:0/4029931282 >> 172.21.15.47:0/4029931282 conn(0x7f1de00fd560 msgr2=0x7f1de01064a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:58:54.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:54.548+0000 7f1de6816640 1 -- 172.21.15.47:0/4029931282 shutdown_connections 2024-04-17T05:58:54.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:54.548+0000 7f1de6816640 1 -- 172.21.15.47:0/4029931282 wait complete. 2024-04-17T05:58:55.440 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:55 smithi098 ceph-mon[36516]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T05:58:55.441 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:55 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:55.441 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:55 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:55.441 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:55 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:55.441 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:55 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-17T05:58:55.441 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:55 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-17T05:58:55.441 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:55 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:58:55.441 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:55 smithi098 ceph-mon[36516]: Deploying daemon mds.h on smithi098 2024-04-17T05:58:55.441 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:55 smithi098 ceph-mon[36516]: mds.? [v2:172.21.15.103:6836/116936024,v1:172.21.15.103:6837/116936024] up:boot 2024-04-17T05:58:55.441 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:55 smithi098 ceph-mon[36516]: fsmap 7 up:standby 2024-04-17T05:58:55.441 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:55 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-17T05:58:55.441 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:55 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/4029931282' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T05:58:55.482 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:55 smithi103 ceph-mon[36243]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T05:58:55.482 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:55 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:55.482 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:55 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:55.482 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:55 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:55.482 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:55 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-17T05:58:55.482 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:55 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-17T05:58:55.482 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:55 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:58:55.482 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:55 smithi103 ceph-mon[36243]: Deploying daemon mds.h on smithi098 2024-04-17T05:58:55.482 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:55 smithi103 ceph-mon[36243]: mds.? [v2:172.21.15.103:6836/116936024,v1:172.21.15.103:6837/116936024] up:boot 2024-04-17T05:58:55.483 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:55 smithi103 ceph-mon[36243]: fsmap 7 up:standby 2024-04-17T05:58:55.483 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:55 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-17T05:58:55.483 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:55 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/4029931282' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T05:58:55.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:55 smithi047 ceph-mon[32223]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T05:58:55.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:55 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:55.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:55 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:55.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:55 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:55.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:55 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-17T05:58:55.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:55 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-17T05:58:55.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:55 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:58:55.640 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:55 smithi047 ceph-mon[32223]: Deploying daemon mds.h on smithi098 2024-04-17T05:58:55.640 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:55 smithi047 ceph-mon[32223]: mds.? [v2:172.21.15.103:6836/116936024,v1:172.21.15.103:6837/116936024] up:boot 2024-04-17T05:58:55.640 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:55 smithi047 ceph-mon[32223]: fsmap 7 up:standby 2024-04-17T05:58:55.640 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:55 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-17T05:58:55.640 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:55 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/4029931282' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T05:58:56.557 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T05:58:56.633 INFO:journalctl@ceph.mds.h.smithi098.stdout:Apr 17 05:58:56 smithi098 systemd[1]: Starting Ceph mds.h for 565db2d8-fc7e-11ee-bc91-c7b262605968... 2024-04-17T05:58:56.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:56.677+0000 7f9657c2a640 1 -- 172.21.15.47:0/2051147882 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9650107c90 msgr2=0x7f965010a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:58:56.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:56.677+0000 7f9657c2a640 1 --2- 172.21.15.47:0/2051147882 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9650107c90 0x7f965010a0e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9648009ca0 tx=0x7f964802c0f0 comp rx=0 tx=0).stop 2024-04-17T05:58:56.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:56.677+0000 7f9657c2a640 1 -- 172.21.15.47:0/2051147882 shutdown_connections 2024-04-17T05:58:56.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:56.677+0000 7f9657c2a640 1 --2- 172.21.15.47:0/2051147882 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9650107c90 0x7f965010a0e0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:56.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:56.677+0000 7f9657c2a640 1 --2- 172.21.15.47:0/2051147882 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9650105360 0x7f9650107750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:56.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:56.677+0000 7f9657c2a640 1 --2- 172.21.15.47:0/2051147882 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9650102a30 0x7f9650104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:56.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:56.677+0000 7f9657c2a640 1 -- 172.21.15.47:0/2051147882 >> 172.21.15.47:0/2051147882 conn(0x7f96500fc440 msgr2=0x7f96500fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:58:56.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:56.677+0000 7f9657c2a640 1 -- 172.21.15.47:0/2051147882 shutdown_connections 2024-04-17T05:58:56.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:56.677+0000 7f9657c2a640 1 -- 172.21.15.47:0/2051147882 wait complete. 2024-04-17T05:58:56.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:56.678+0000 7f9657c2a640 1 Processor -- start 2024-04-17T05:58:56.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:56.678+0000 7f9657c2a640 1 -- start start 2024-04-17T05:58:56.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:56.678+0000 7f9657c2a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9650102a30 0x7f96501023a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:58:56.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:56.678+0000 7f9657c2a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9650105360 0x7f96501009e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:58:56.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:56.678+0000 7f9657c2a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9650107c90 0x7f9650100f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:58:56.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:56.678+0000 7f9657c2a640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f965010dc40 con 0x7f9650102a30 2024-04-17T05:58:56.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:56.678+0000 7f9657c2a640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f965010dac0 con 0x7f9650107c90 2024-04-17T05:58:56.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:56.678+0000 7f9657c2a640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f965010ddc0 con 0x7f9650105360 2024-04-17T05:58:56.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:56.678+0000 7f96559c6640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9650102a30 0x7f96501023a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:58:56.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:56.678+0000 7f96559c6640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9650102a30 0x7f96501023a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58924/0 (socket says 172.21.15.47:58924) 2024-04-17T05:58:56.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:56.678+0000 7f96559c6640 1 -- 172.21.15.47:0/4110692081 learned_addr learned my addr 172.21.15.47:0/4110692081 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:58:56.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:56.678+0000 7f96561c7640 1 --2- 172.21.15.47:0/4110692081 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9650107c90 0x7f9650100f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:58:56.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:56.678+0000 7f96551c5640 1 --2- 172.21.15.47:0/4110692081 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9650105360 0x7f96501009e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:58:56.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:56.679+0000 7f96559c6640 1 -- 172.21.15.47:0/4110692081 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9650105360 msgr2=0x7f96501009e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:58:56.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:56.679+0000 7f96559c6640 1 --2- 172.21.15.47:0/4110692081 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9650105360 0x7f96501009e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:56.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:56.679+0000 7f96559c6640 1 -- 172.21.15.47:0/4110692081 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9650107c90 msgr2=0x7f9650100f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:58:56.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:56.679+0000 7f96559c6640 1 --2- 172.21.15.47:0/4110692081 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9650107c90 0x7f9650100f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:56.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:56.679+0000 7f96559c6640 1 -- 172.21.15.47:0/4110692081 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96501017b0 con 0x7f9650102a30 2024-04-17T05:58:56.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:56.679+0000 7f96551c5640 1 --2- 172.21.15.47:0/4110692081 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9650105360 0x7f96501009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T05:58:56.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:56.679+0000 7f96559c6640 1 --2- 172.21.15.47:0/4110692081 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9650102a30 0x7f96501023a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f964c00d770 tx=0x7f964c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:58:56.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:56.679+0000 7f9646ffd640 1 -- 172.21.15.47:0/4110692081 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f964c017070 con 0x7f9650102a30 2024-04-17T05:58:56.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:56.679+0000 7f9657c2a640 1 -- 172.21.15.47:0/4110692081 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96501a2330 con 0x7f9650102a30 2024-04-17T05:58:56.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:56.679+0000 7f9646ffd640 1 -- 172.21.15.47:0/4110692081 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f964c005500 con 0x7f9650102a30 2024-04-17T05:58:56.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:56.679+0000 7f9646ffd640 1 -- 172.21.15.47:0/4110692081 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f964c004e00 con 0x7f9650102a30 2024-04-17T05:58:56.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:56.679+0000 7f9657c2a640 1 -- 172.21.15.47:0/4110692081 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96501a2830 con 0x7f9650102a30 2024-04-17T05:58:56.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:56.680+0000 7f9657c2a640 1 -- 172.21.15.47:0/4110692081 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96500fdf30 con 0x7f9650102a30 2024-04-17T05:58:56.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:56.681+0000 7f9646ffd640 1 -- 172.21.15.47:0/4110692081 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f964c004fa0 con 0x7f9650102a30 2024-04-17T05:58:56.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:56.682+0000 7f9646ffd640 1 --2- 172.21.15.47:0/4110692081 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f963c0a3bb0 0x7f963c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:58:56.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:56.682+0000 7f96551c5640 1 --2- 172.21.15.47:0/4110692081 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f963c0a3bb0 0x7f963c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:58:56.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:56.682+0000 7f9646ffd640 1 -- 172.21.15.47:0/4110692081 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f964c0cf9b0 con 0x7f9650102a30 2024-04-17T05:58:56.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:56.682+0000 7f96551c5640 1 --2- 172.21.15.47:0/4110692081 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f963c0a3bb0 0x7f963c0a6070 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9650102110 tx=0x7f9640005d90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:58:56.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:56.687+0000 7f9646ffd640 1 -- 172.21.15.47:0/4110692081 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f964c09c830 con 0x7f9650102a30 2024-04-17T05:58:56.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:56.912+0000 7f9657c2a640 1 -- 172.21.15.47:0/4110692081 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f965002d5d0 con 0x7f9650102a30 2024-04-17T05:58:56.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:56.913+0000 7f9646ffd640 1 -- 172.21.15.47:0/4110692081 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6343 (secure 0 0 0) 0x7f964c09c130 con 0x7f9650102a30 2024-04-17T05:58:56.911 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-17T05:58:56.911 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3189958720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3189958720},{"type":"v1","addr":"172.21.15.47:6835","nonce":3189958720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14595,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/3143529973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":3143529973},{"type":"v1","addr":"172.21.15.47:6837","nonce":3143529973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/2398363754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":2398363754},{"type":"v1","addr":"172.21.15.103:6833","nonce":2398363754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/384473493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":384473493},{"type":"v1","addr":"172.21.15.98:6833","nonce":384473493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/270543522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":270543522},{"type":"v1","addr":"172.21.15.103:6835","nonce":270543522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24448,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/116936024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":116936024},{"type":"v1","addr":"172.21.15.103:6837","nonce":116936024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24449,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/1966500278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":1966500278},{"type":"v1","addr":"172.21.15.98:6835","nonce":1966500278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-17T05:58:56.911 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 8 2024-04-17T05:58:56.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:56.915+0000 7f9657c2a640 1 -- 172.21.15.47:0/4110692081 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f963c0a3bb0 msgr2=0x7f963c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:58:56.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:56.915+0000 7f9657c2a640 1 --2- 172.21.15.47:0/4110692081 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f963c0a3bb0 0x7f963c0a6070 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9650102110 tx=0x7f9640005d90 comp rx=0 tx=0).stop 2024-04-17T05:58:56.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:56.915+0000 7f9657c2a640 1 -- 172.21.15.47:0/4110692081 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9650102a30 msgr2=0x7f96501023a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:58:56.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:56.915+0000 7f9657c2a640 1 --2- 172.21.15.47:0/4110692081 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9650102a30 0x7f96501023a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f964c00d770 tx=0x7f964c00dc30 comp rx=0 tx=0).stop 2024-04-17T05:58:56.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:56.915+0000 7f9657c2a640 1 -- 172.21.15.47:0/4110692081 shutdown_connections 2024-04-17T05:58:56.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:56.915+0000 7f9657c2a640 1 --2- 172.21.15.47:0/4110692081 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f963c0a3bb0 0x7f963c0a6070 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:56.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:56.915+0000 7f9657c2a640 1 --2- 172.21.15.47:0/4110692081 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9650107c90 0x7f9650100f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:56.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:56.915+0000 7f9657c2a640 1 --2- 172.21.15.47:0/4110692081 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9650105360 0x7f96501009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:56.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:56.915+0000 7f9657c2a640 1 --2- 172.21.15.47:0/4110692081 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9650102a30 0x7f96501023a0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:56.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:56.915+0000 7f9657c2a640 1 -- 172.21.15.47:0/4110692081 >> 172.21.15.47:0/4110692081 conn(0x7f96500fc440 msgr2=0x7f9650103330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:58:56.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:56.915+0000 7f9657c2a640 1 -- 172.21.15.47:0/4110692081 shutdown_connections 2024-04-17T05:58:56.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:56.915+0000 7f9657c2a640 1 -- 172.21.15.47:0/4110692081 wait complete. 2024-04-17T05:58:56.990 INFO:journalctl@ceph.mds.h.smithi098.stdout:Apr 17 05:58:56 smithi098 podman[67044]: 2024-04-17 05:58:56.602093924 +0000 UTC m=+0.018063023 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-17T05:58:56.990 INFO:journalctl@ceph.mds.h.smithi098.stdout:Apr 17 05:58:56 smithi098 podman[67044]: 2024-04-17 05:58:56.869815539 +0000 UTC m=+0.285784682 container create 94c7c390af79f132dbf9d065fd5f3f5ee1c9fe6bfcd12dcf8b55821b1ca8b045 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-h, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-17T05:58:57.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:57 smithi098 ceph-mon[36516]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T05:58:57.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:57 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/4110692081' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T05:58:57.383 INFO:journalctl@ceph.mds.h.smithi098.stdout:Apr 17 05:58:57 smithi098 podman[67044]: 2024-04-17 05:58:57.13144798 +0000 UTC m=+0.547417086 container init 94c7c390af79f132dbf9d065fd5f3f5ee1c9fe6bfcd12dcf8b55821b1ca8b045 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-h, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux ) 2024-04-17T05:58:57.383 INFO:journalctl@ceph.mds.h.smithi098.stdout:Apr 17 05:58:57 smithi098 podman[67044]: 2024-04-17 05:58:57.133957107 +0000 UTC m=+0.549926211 container start 94c7c390af79f132dbf9d065fd5f3f5ee1c9fe6bfcd12dcf8b55821b1ca8b045 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-h, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-04-17T05:58:57.383 INFO:journalctl@ceph.mds.h.smithi098.stdout:Apr 17 05:58:57 smithi098 ceph-mds[67063]: -- 172.21.15.98:0/2387699643 <== mon.2 v2:172.21.15.98:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b6ab807a40 con 0x55b6ab7da800 2024-04-17T05:58:57.383 INFO:journalctl@ceph.mds.h.smithi098.stdout:Apr 17 05:58:57 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-h[67059]: starting mds.h at 2024-04-17T05:58:57.383 INFO:journalctl@ceph.mds.h.smithi098.stdout:Apr 17 05:58:57 smithi098 bash[67044]: 94c7c390af79f132dbf9d065fd5f3f5ee1c9fe6bfcd12dcf8b55821b1ca8b045 2024-04-17T05:58:57.383 INFO:journalctl@ceph.mds.h.smithi098.stdout:Apr 17 05:58:57 smithi098 systemd[1]: Started Ceph mds.h for 565db2d8-fc7e-11ee-bc91-c7b262605968. 2024-04-17T05:58:57.533 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:57 smithi103 ceph-mon[36243]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T05:58:57.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:57 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/4110692081' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T05:58:57.638 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:57 smithi047 ceph-mon[32223]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T05:58:57.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:57 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/4110692081' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T05:58:58.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:58 smithi103 ceph-mon[36243]: mds.? [v2:172.21.15.98:6836/1117440719,v1:172.21.15.98:6837/1117440719] up:boot 2024-04-17T05:58:58.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:58 smithi103 ceph-mon[36243]: fsmap 8 up:standby 2024-04-17T05:58:58.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:58 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-17T05:58:58.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:58 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:58.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:58 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:58.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:58 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:58.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:58 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-17T05:58:58.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:58 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-17T05:58:58.535 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:58 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:58:58.535 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:58 smithi103 ceph-mon[36243]: Deploying daemon mds.i on smithi103 2024-04-17T05:58:58.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:58 smithi098 ceph-mon[36516]: mds.? [v2:172.21.15.98:6836/1117440719,v1:172.21.15.98:6837/1117440719] up:boot 2024-04-17T05:58:58.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:58 smithi098 ceph-mon[36516]: fsmap 8 up:standby 2024-04-17T05:58:58.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:58 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-17T05:58:58.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:58 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:58.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:58 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:58.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:58 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:58.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:58 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-17T05:58:58.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:58 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-17T05:58:58.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:58 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:58:58.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:58 smithi098 ceph-mon[36516]: Deploying daemon mds.i on smithi103 2024-04-17T05:58:58.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:58 smithi047 ceph-mon[32223]: mds.? [v2:172.21.15.98:6836/1117440719,v1:172.21.15.98:6837/1117440719] up:boot 2024-04-17T05:58:58.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:58 smithi047 ceph-mon[32223]: fsmap 8 up:standby 2024-04-17T05:58:58.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:58 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-17T05:58:58.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:58 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:58.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:58 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:58.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:58 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:58:58.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:58 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-17T05:58:58.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:58 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-17T05:58:58.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:58 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:58:58.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:58 smithi047 ceph-mon[32223]: Deploying daemon mds.i on smithi103 2024-04-17T05:58:58.927 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T05:58:59.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:59.048+0000 7f8966455640 1 -- 172.21.15.47:0/3321828734 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8960107d20 msgr2=0x7f896010a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:58:59.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:59.048+0000 7f8966455640 1 --2- 172.21.15.47:0/3321828734 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8960107d20 0x7f896010a110 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f8958009ca0 tx=0x7f895802c100 comp rx=0 tx=0).stop 2024-04-17T05:58:59.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:59.049+0000 7f8966455640 1 -- 172.21.15.47:0/3321828734 shutdown_connections 2024-04-17T05:58:59.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:59.049+0000 7f8966455640 1 --2- 172.21.15.47:0/3321828734 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8960107d20 0x7f896010a110 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:59.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:59.049+0000 7f8966455640 1 --2- 172.21.15.47:0/3321828734 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f89601053f0 0x7f89601077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:59.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:59.049+0000 7f8966455640 1 --2- 172.21.15.47:0/3321828734 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8960102ac0 0x7f8960104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:59.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:59.049+0000 7f8966455640 1 -- 172.21.15.47:0/3321828734 >> 172.21.15.47:0/3321828734 conn(0x7f89600fc410 msgr2=0x7f89600fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:58:59.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:59.049+0000 7f8966455640 1 -- 172.21.15.47:0/3321828734 shutdown_connections 2024-04-17T05:58:59.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:59.049+0000 7f8966455640 1 -- 172.21.15.47:0/3321828734 wait complete. 2024-04-17T05:58:59.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:59.049+0000 7f8966455640 1 Processor -- start 2024-04-17T05:58:59.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:59.049+0000 7f8966455640 1 -- start start 2024-04-17T05:58:59.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:59.049+0000 7f8966455640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8960102ac0 0x7f8960102430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:58:59.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:59.049+0000 7f8966455640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f89601053f0 0x7f8960100a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:58:59.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:59.049+0000 7f8966455640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8960107d20 0x7f8960100fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:58:59.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:59.049+0000 7f8966455640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f896010dc50 con 0x7f89601053f0 2024-04-17T05:58:59.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:59.049+0000 7f8966455640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f896010dad0 con 0x7f8960107d20 2024-04-17T05:58:59.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:59.049+0000 7f8966455640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f896010ddd0 con 0x7f8960102ac0 2024-04-17T05:58:59.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:59.049+0000 7f8957fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f89601053f0 0x7f8960100a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:58:59.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:59.050+0000 7f8957fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f89601053f0 0x7f8960100a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58932/0 (socket says 172.21.15.47:58932) 2024-04-17T05:58:59.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:59.050+0000 7f8957fff640 1 -- 172.21.15.47:0/1066675521 learned_addr learned my addr 172.21.15.47:0/1066675521 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:58:59.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:59.050+0000 7f895ffff640 1 --2- 172.21.15.47:0/1066675521 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8960102ac0 0x7f8960102430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:58:59.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:59.050+0000 7f89649f2640 1 --2- 172.21.15.47:0/1066675521 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8960107d20 0x7f8960100fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:58:59.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:59.050+0000 7f8957fff640 1 -- 172.21.15.47:0/1066675521 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8960102ac0 msgr2=0x7f8960102430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:58:59.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:59.050+0000 7f8957fff640 1 --2- 172.21.15.47:0/1066675521 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8960102ac0 0x7f8960102430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:59.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:59.050+0000 7f8957fff640 1 -- 172.21.15.47:0/1066675521 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8960107d20 msgr2=0x7f8960100fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:58:59.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:59.050+0000 7f8957fff640 1 --2- 172.21.15.47:0/1066675521 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8960107d20 0x7f8960100fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:59.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:59.050+0000 7f8957fff640 1 -- 172.21.15.47:0/1066675521 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8960101840 con 0x7f89601053f0 2024-04-17T05:58:59.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:59.050+0000 7f89649f2640 1 --2- 172.21.15.47:0/1066675521 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8960107d20 0x7f8960100fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T05:58:59.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:59.050+0000 7f8957fff640 1 --2- 172.21.15.47:0/1066675521 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f89601053f0 0x7f8960100a70 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f894800c970 tx=0x7f894800cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:58:59.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:59.050+0000 7f895dffb640 1 -- 172.21.15.47:0/1066675521 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8948014070 con 0x7f89601053f0 2024-04-17T05:58:59.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:59.050+0000 7f895dffb640 1 -- 172.21.15.47:0/1066675521 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8948004440 con 0x7f89601053f0 2024-04-17T05:58:59.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:59.050+0000 7f8966455640 1 -- 172.21.15.47:0/1066675521 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89601a2280 con 0x7f89601053f0 2024-04-17T05:58:59.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:59.050+0000 7f895dffb640 1 -- 172.21.15.47:0/1066675521 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8948003e10 con 0x7f89601053f0 2024-04-17T05:58:59.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:59.050+0000 7f8966455640 1 -- 172.21.15.47:0/1066675521 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89601a27d0 con 0x7f89601053f0 2024-04-17T05:58:59.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:59.051+0000 7f8966455640 1 -- 172.21.15.47:0/1066675521 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89600fdf00 con 0x7f89601053f0 2024-04-17T05:58:59.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:59.052+0000 7f895dffb640 1 -- 172.21.15.47:0/1066675521 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8948017070 con 0x7f89601053f0 2024-04-17T05:58:59.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:59.053+0000 7f895dffb640 1 --2- 172.21.15.47:0/1066675521 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f894c0a3bb0 0x7f894c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:58:59.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:59.053+0000 7f895dffb640 1 -- 172.21.15.47:0/1066675521 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f89480cff20 con 0x7f89601053f0 2024-04-17T05:58:59.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:59.053+0000 7f895ffff640 1 --2- 172.21.15.47:0/1066675521 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f894c0a3bb0 0x7f894c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:58:59.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:59.054+0000 7f895ffff640 1 --2- 172.21.15.47:0/1066675521 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f894c0a3bb0 0x7f894c0a6070 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f895000abe0 tx=0x7f895000a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:58:59.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:59.058+0000 7f895dffb640 1 -- 172.21.15.47:0/1066675521 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f894801e020 con 0x7f89601053f0 2024-04-17T05:58:59.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:59.284+0000 7f8966455640 1 -- 172.21.15.47:0/1066675521 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8960062b60 con 0x7f89601053f0 2024-04-17T05:58:59.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:59.284+0000 7f895dffb640 1 -- 172.21.15.47:0/1066675521 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7168 (secure 0 0 0) 0x7f894809cd00 con 0x7f89601053f0 2024-04-17T05:58:59.282 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-17T05:58:59.282 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3189958720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3189958720},{"type":"v1","addr":"172.21.15.47:6835","nonce":3189958720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14595,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/3143529973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":3143529973},{"type":"v1","addr":"172.21.15.47:6837","nonce":3143529973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/2398363754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":2398363754},{"type":"v1","addr":"172.21.15.103:6833","nonce":2398363754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/384473493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":384473493},{"type":"v1","addr":"172.21.15.98:6833","nonce":384473493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/270543522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":270543522},{"type":"v1","addr":"172.21.15.103:6835","nonce":270543522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24448,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/116936024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":116936024},{"type":"v1","addr":"172.21.15.103:6837","nonce":116936024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24449,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/1966500278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":1966500278},{"type":"v1","addr":"172.21.15.98:6835","nonce":1966500278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24479,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/1117440719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1117440719},{"type":"v1","addr":"172.21.15.98:6837","nonce":1117440719}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-17T05:58:59.283 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 9 2024-04-17T05:58:59.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:59.286+0000 7f8966455640 1 -- 172.21.15.47:0/1066675521 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f894c0a3bb0 msgr2=0x7f894c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:58:59.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:59.286+0000 7f8966455640 1 --2- 172.21.15.47:0/1066675521 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f894c0a3bb0 0x7f894c0a6070 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f895000abe0 tx=0x7f895000a2d0 comp rx=0 tx=0).stop 2024-04-17T05:58:59.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:59.286+0000 7f8966455640 1 -- 172.21.15.47:0/1066675521 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f89601053f0 msgr2=0x7f8960100a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:58:59.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:59.286+0000 7f8966455640 1 --2- 172.21.15.47:0/1066675521 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f89601053f0 0x7f8960100a70 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f894800c970 tx=0x7f894800cd70 comp rx=0 tx=0).stop 2024-04-17T05:58:59.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:59.286+0000 7f8966455640 1 -- 172.21.15.47:0/1066675521 shutdown_connections 2024-04-17T05:58:59.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:59.286+0000 7f8966455640 1 --2- 172.21.15.47:0/1066675521 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f894c0a3bb0 0x7f894c0a6070 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:59.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:59.286+0000 7f8966455640 1 --2- 172.21.15.47:0/1066675521 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8960107d20 0x7f8960100fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:59.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:59.286+0000 7f8966455640 1 --2- 172.21.15.47:0/1066675521 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f89601053f0 0x7f8960100a70 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:59.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:59.286+0000 7f8966455640 1 --2- 172.21.15.47:0/1066675521 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8960102ac0 0x7f8960102430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:58:59.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:59.286+0000 7f8966455640 1 -- 172.21.15.47:0/1066675521 >> 172.21.15.47:0/1066675521 conn(0x7f89600fc410 msgr2=0x7f89601033c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:58:59.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:59.287+0000 7f8966455640 1 -- 172.21.15.47:0/1066675521 shutdown_connections 2024-04-17T05:58:59.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:58:59.287+0000 7f8966455640 1 -- 172.21.15.47:0/1066675521 wait complete. 2024-04-17T05:58:59.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:59 smithi103 ceph-mon[36243]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T05:58:59.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:58:59 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/1066675521' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T05:58:59.535 INFO:journalctl@ceph.mds.i.smithi103.stdout:Apr 17 05:58:59 smithi103 systemd[1]: Starting Ceph mds.i for 565db2d8-fc7e-11ee-bc91-c7b262605968... 2024-04-17T05:58:59.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:59 smithi098 ceph-mon[36516]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T05:58:59.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:58:59 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/1066675521' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T05:58:59.638 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:59 smithi047 ceph-mon[32223]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T05:58:59.638 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:58:59 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/1066675521' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T05:58:59.903 INFO:journalctl@ceph.mds.i.smithi103.stdout:Apr 17 05:58:59 smithi103 podman[66903]: 2024-04-17 05:58:59.51143132 +0000 UTC m=+0.017476564 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-17T05:58:59.903 INFO:journalctl@ceph.mds.i.smithi103.stdout:Apr 17 05:58:59 smithi103 podman[66903]: 2024-04-17 05:58:59.636148312 +0000 UTC m=+0.142193545 container create e6268839d96ab24bf4e242b908e7fb4da2286439cb6c3744a39a484d97b8e85f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-i, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.build-date=20240416) 2024-04-17T05:59:00.284 INFO:journalctl@ceph.mds.i.smithi103.stdout:Apr 17 05:58:59 smithi103 podman[66903]: 2024-04-17 05:58:59.905544939 +0000 UTC m=+0.411590173 container init e6268839d96ab24bf4e242b908e7fb4da2286439cb6c3744a39a484d97b8e85f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-i, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True) 2024-04-17T05:59:00.284 INFO:journalctl@ceph.mds.i.smithi103.stdout:Apr 17 05:58:59 smithi103 podman[66903]: 2024-04-17 05:58:59.908267199 +0000 UTC m=+0.414312434 container start e6268839d96ab24bf4e242b908e7fb4da2286439cb6c3744a39a484d97b8e85f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-i, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416) 2024-04-17T05:59:00.284 INFO:journalctl@ceph.mds.i.smithi103.stdout:Apr 17 05:58:59 smithi103 ceph-mds[66922]: -- 172.21.15.103:0/2552802363 <== mon.1 v2:172.21.15.103:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56190750ba40 con 0x5619074dec00 2024-04-17T05:59:00.284 INFO:journalctl@ceph.mds.i.smithi103.stdout:Apr 17 05:58:59 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-i[66918]: starting mds.i at 2024-04-17T05:59:00.284 INFO:journalctl@ceph.mds.i.smithi103.stdout:Apr 17 05:58:59 smithi103 bash[66903]: e6268839d96ab24bf4e242b908e7fb4da2286439cb6c3744a39a484d97b8e85f 2024-04-17T05:59:00.284 INFO:journalctl@ceph.mds.i.smithi103.stdout:Apr 17 05:58:59 smithi103 systemd[1]: Started Ceph mds.i for 565db2d8-fc7e-11ee-bc91-c7b262605968. 2024-04-17T05:59:01.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:01 smithi103 ceph-mon[36243]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T05:59:01.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:01 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:59:01.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:01 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:59:01.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:01 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:59:01.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:01 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-17T05:59:01.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:01 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-17T05:59:01.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:01 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:59:01.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:01 smithi103 ceph-mon[36243]: Deploying daemon mds.g on smithi047 2024-04-17T05:59:01.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:01 smithi103 ceph-mon[36243]: mds.? [v2:172.21.15.103:6838/1467157889,v1:172.21.15.103:6839/1467157889] up:boot 2024-04-17T05:59:01.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:01 smithi103 ceph-mon[36243]: fsmap 9 up:standby 2024-04-17T05:59:01.285 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:01 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-17T05:59:01.297 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T05:59:01.325 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:01 smithi047 ceph-mon[32223]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T05:59:01.325 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:01 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:59:01.325 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:01 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:59:01.325 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:01 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:59:01.325 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:01 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-17T05:59:01.325 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:01 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-17T05:59:01.325 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:01 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:59:01.325 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:01 smithi047 ceph-mon[32223]: Deploying daemon mds.g on smithi047 2024-04-17T05:59:01.325 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:01 smithi047 ceph-mon[32223]: mds.? [v2:172.21.15.103:6838/1467157889,v1:172.21.15.103:6839/1467157889] up:boot 2024-04-17T05:59:01.325 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:01 smithi047 ceph-mon[32223]: fsmap 9 up:standby 2024-04-17T05:59:01.325 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:01 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-17T05:59:01.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:01 smithi098 ceph-mon[36516]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T05:59:01.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:01 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:59:01.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:01 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:59:01.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:01 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:59:01.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:01 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-17T05:59:01.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:01 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-17T05:59:01.384 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:01 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:59:01.384 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:01 smithi098 ceph-mon[36516]: Deploying daemon mds.g on smithi047 2024-04-17T05:59:01.384 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:01 smithi098 ceph-mon[36516]: mds.? [v2:172.21.15.103:6838/1467157889,v1:172.21.15.103:6839/1467157889] up:boot 2024-04-17T05:59:01.384 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:01 smithi098 ceph-mon[36516]: fsmap 9 up:standby 2024-04-17T05:59:01.384 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:01 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-17T05:59:01.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:01.411+0000 7f5f10704640 1 -- 172.21.15.47:0/4123347994 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5f08101b40 msgr2=0x7f5f081093e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:01.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:01.411+0000 7f5f10704640 1 --2- 172.21.15.47:0/4123347994 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5f08101b40 0x7f5f081093e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5f00009900 tx=0x7f5f0002bc30 comp rx=0 tx=0).stop 2024-04-17T05:59:01.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:01.411+0000 7f5f10704640 1 -- 172.21.15.47:0/4123347994 shutdown_connections 2024-04-17T05:59:01.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:01.411+0000 7f5f10704640 1 --2- 172.21.15.47:0/4123347994 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5f08101b40 0x7f5f081093e0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:01.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:01.411+0000 7f5f10704640 1 --2- 172.21.15.47:0/4123347994 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5f08101180 0x7f5f08101600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:01.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:01.411+0000 7f5f10704640 1 --2- 172.21.15.47:0/4123347994 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5f08102450 0x7f5f08102870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:01.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:01.411+0000 7f5f10704640 1 -- 172.21.15.47:0/4123347994 >> 172.21.15.47:0/4123347994 conn(0x7f5f080fbbd0 msgr2=0x7f5f080fdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:59:01.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:01.411+0000 7f5f10704640 1 -- 172.21.15.47:0/4123347994 shutdown_connections 2024-04-17T05:59:01.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:01.411+0000 7f5f10704640 1 -- 172.21.15.47:0/4123347994 wait complete. 2024-04-17T05:59:01.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:01.412+0000 7f5f10704640 1 Processor -- start 2024-04-17T05:59:01.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:01.412+0000 7f5f10704640 1 -- start start 2024-04-17T05:59:01.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:01.412+0000 7f5f10704640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5f08101180 0x7f5f08198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:01.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:01.412+0000 7f5f10704640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5f08101b40 0x7f5f08198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:01.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:01.412+0000 7f5f10704640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5f08102450 0x7f5f0819f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:01.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:01.412+0000 7f5f10704640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f5f0810dcd0 con 0x7f5f08101b40 2024-04-17T05:59:01.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:01.412+0000 7f5f10704640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5f0810db50 con 0x7f5f08101180 2024-04-17T05:59:01.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:01.412+0000 7f5f10704640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5f0810de50 con 0x7f5f08102450 2024-04-17T05:59:01.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:01.412+0000 7f5f0dc9f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5f08101b40 0x7f5f08198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:01.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:01.412+0000 7f5f0dc9f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5f08101b40 0x7f5f08198850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58954/0 (socket says 172.21.15.47:58954) 2024-04-17T05:59:01.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:01.412+0000 7f5f0dc9f640 1 -- 172.21.15.47:0/514057517 learned_addr learned my addr 172.21.15.47:0/514057517 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:59:01.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:01.412+0000 7f5f0e4a0640 1 --2- 172.21.15.47:0/514057517 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5f08101180 0x7f5f08198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:01.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:01.412+0000 7f5f0dc9f640 1 -- 172.21.15.47:0/514057517 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5f08102450 msgr2=0x7f5f0819f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:01.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:01.412+0000 7f5f0eca1640 1 --2- 172.21.15.47:0/514057517 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5f08102450 0x7f5f0819f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:01.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:01.412+0000 7f5f0dc9f640 1 --2- 172.21.15.47:0/514057517 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5f08102450 0x7f5f0819f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:01.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:01.412+0000 7f5f0dc9f640 1 -- 172.21.15.47:0/514057517 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5f08101180 msgr2=0x7f5f08198310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:01.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:01.412+0000 7f5f0dc9f640 1 --2- 172.21.15.47:0/514057517 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5f08101180 0x7f5f08198310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:01.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:01.412+0000 7f5f0dc9f640 1 -- 172.21.15.47:0/514057517 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f0819ff30 con 0x7f5f08101b40 2024-04-17T05:59:01.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:01.412+0000 7f5f0eca1640 1 --2- 172.21.15.47:0/514057517 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5f08102450 0x7f5f0819f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T05:59:01.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:01.413+0000 7f5f0dc9f640 1 --2- 172.21.15.47:0/514057517 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5f08101b40 0x7f5f08198850 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f5ef800d950 tx=0x7f5ef800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:59:01.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:01.413+0000 7f5eff7fe640 1 -- 172.21.15.47:0/514057517 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef8017070 con 0x7f5f08101b40 2024-04-17T05:59:01.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:01.413+0000 7f5f10704640 1 -- 172.21.15.47:0/514057517 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f081a0220 con 0x7f5f08101b40 2024-04-17T05:59:01.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:01.413+0000 7f5eff7fe640 1 -- 172.21.15.47:0/514057517 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ef8009c30 con 0x7f5f08101b40 2024-04-17T05:59:01.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:01.413+0000 7f5eff7fe640 1 -- 172.21.15.47:0/514057517 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef800f6b0 con 0x7f5f08101b40 2024-04-17T05:59:01.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:01.413+0000 7f5f10704640 1 -- 172.21.15.47:0/514057517 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f081a0770 con 0x7f5f08101b40 2024-04-17T05:59:01.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:01.414+0000 7f5f10704640 1 -- 172.21.15.47:0/514057517 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f081082c0 con 0x7f5f08101b40 2024-04-17T05:59:01.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:01.415+0000 7f5eff7fe640 1 -- 172.21.15.47:0/514057517 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f5ef80109d0 con 0x7f5f08101b40 2024-04-17T05:59:01.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:01.415+0000 7f5eff7fe640 1 --2- 172.21.15.47:0/514057517 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f5ef40a3bb0 0x7f5ef40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:01.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:01.416+0000 7f5f0e4a0640 1 --2- 172.21.15.47:0/514057517 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f5ef40a3bb0 0x7f5ef40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:01.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:01.416+0000 7f5eff7fe640 1 -- 172.21.15.47:0/514057517 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f5ef80cfc30 con 0x7f5f08101b40 2024-04-17T05:59:01.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:01.416+0000 7f5f0e4a0640 1 --2- 172.21.15.47:0/514057517 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f5ef40a3bb0 0x7f5ef40a6070 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5f0806a340 tx=0x7f5f0400d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:59:01.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:01.420+0000 7f5eff7fe640 1 -- 172.21.15.47:0/514057517 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f5ef8009de0 con 0x7f5f08101b40 2024-04-17T05:59:01.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:01.646+0000 7f5f10704640 1 -- 172.21.15.47:0/514057517 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5f08062b60 con 0x7f5f08101b40 2024-04-17T05:59:01.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:01.647+0000 7f5eff7fe640 1 -- 172.21.15.47:0/514057517 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+7998 (secure 0 0 0) 0x7f5ef809ca10 con 0x7f5f08101b40 2024-04-17T05:59:01.645 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-17T05:59:01.645 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3189958720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3189958720},{"type":"v1","addr":"172.21.15.47:6835","nonce":3189958720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14595,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/3143529973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":3143529973},{"type":"v1","addr":"172.21.15.47:6837","nonce":3143529973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/2398363754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":2398363754},{"type":"v1","addr":"172.21.15.103:6833","nonce":2398363754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/384473493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":384473493},{"type":"v1","addr":"172.21.15.98:6833","nonce":384473493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/270543522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":270543522},{"type":"v1","addr":"172.21.15.103:6835","nonce":270543522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24448,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/116936024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":116936024},{"type":"v1","addr":"172.21.15.103:6837","nonce":116936024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24449,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/1966500278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":1966500278},{"type":"v1","addr":"172.21.15.98:6835","nonce":1966500278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24466,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6839/1467157889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":1467157889},{"type":"v1","addr":"172.21.15.103:6839","nonce":1467157889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24479,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/1117440719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1117440719},{"type":"v1","addr":"172.21.15.98:6837","nonce":1117440719}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-17T05:59:01.645 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 10 2024-04-17T05:59:01.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:01.648+0000 7f5f10704640 1 -- 172.21.15.47:0/514057517 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f5ef40a3bb0 msgr2=0x7f5ef40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:01.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:01.648+0000 7f5f10704640 1 --2- 172.21.15.47:0/514057517 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f5ef40a3bb0 0x7f5ef40a6070 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5f0806a340 tx=0x7f5f0400d040 comp rx=0 tx=0).stop 2024-04-17T05:59:01.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:01.649+0000 7f5f10704640 1 -- 172.21.15.47:0/514057517 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5f08101b40 msgr2=0x7f5f08198850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:01.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:01.649+0000 7f5f10704640 1 --2- 172.21.15.47:0/514057517 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5f08101b40 0x7f5f08198850 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f5ef800d950 tx=0x7f5ef800de10 comp rx=0 tx=0).stop 2024-04-17T05:59:01.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:01.649+0000 7f5f10704640 1 -- 172.21.15.47:0/514057517 shutdown_connections 2024-04-17T05:59:01.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:01.649+0000 7f5f10704640 1 --2- 172.21.15.47:0/514057517 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f5ef40a3bb0 0x7f5ef40a6070 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:01.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:01.649+0000 7f5f10704640 1 --2- 172.21.15.47:0/514057517 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5f08102450 0x7f5f0819f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:01.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:01.649+0000 7f5f10704640 1 --2- 172.21.15.47:0/514057517 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5f08101b40 0x7f5f08198850 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:01.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:01.649+0000 7f5f10704640 1 --2- 172.21.15.47:0/514057517 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5f08101180 0x7f5f08198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:01.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:01.649+0000 7f5f10704640 1 -- 172.21.15.47:0/514057517 >> 172.21.15.47:0/514057517 conn(0x7f5f080fbbd0 msgr2=0x7f5f08105490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:59:01.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:01.649+0000 7f5f10704640 1 -- 172.21.15.47:0/514057517 shutdown_connections 2024-04-17T05:59:01.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:01.649+0000 7f5f10704640 1 -- 172.21.15.47:0/514057517 wait complete. 2024-04-17T05:59:02.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:02 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/514057517' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T05:59:02.382 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:02 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/514057517' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T05:59:02.533 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:02 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/514057517' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T05:59:02.639 INFO:journalctl@ceph.mds.g.smithi047.stdout:Apr 17 05:59:02 smithi047 podman[74308]: 2024-04-17 05:59:02.556157952 +0000 UTC m=+0.143575064 container create 9b688587ce629a4e4a8f887e2cf637fac63876838623abb07f3e97e04ba10488 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-g, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, CEPH_POINT_RELEASE=) 2024-04-17T05:59:03.045 INFO:journalctl@ceph.mds.g.smithi047.stdout:Apr 17 05:59:02 smithi047 podman[74308]: 2024-04-17 05:59:02.800640505 +0000 UTC m=+0.388057576 container init 9b688587ce629a4e4a8f887e2cf637fac63876838623abb07f3e97e04ba10488 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-g, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-17T05:59:03.045 INFO:journalctl@ceph.mds.g.smithi047.stdout:Apr 17 05:59:02 smithi047 podman[74308]: 2024-04-17 05:59:02.803612537 +0000 UTC m=+0.391029608 container start 9b688587ce629a4e4a8f887e2cf637fac63876838623abb07f3e97e04ba10488 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-g, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2) 2024-04-17T05:59:03.045 INFO:journalctl@ceph.mds.g.smithi047.stdout:Apr 17 05:59:02 smithi047 ceph-mds[74327]: -- 172.21.15.47:0/3632657404 <== mon.0 v2:172.21.15.47:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559ff8cd5a40 con 0x559ff8ca8800 2024-04-17T05:59:03.045 INFO:journalctl@ceph.mds.g.smithi047.stdout:Apr 17 05:59:02 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-g[74323]: starting mds.g at 2024-04-17T05:59:03.045 INFO:journalctl@ceph.mds.g.smithi047.stdout:Apr 17 05:59:02 smithi047 bash[74308]: 9b688587ce629a4e4a8f887e2cf637fac63876838623abb07f3e97e04ba10488 2024-04-17T05:59:03.045 INFO:journalctl@ceph.mds.g.smithi047.stdout:Apr 17 05:59:02 smithi047 systemd[1]: Started Ceph mds.g for 565db2d8-fc7e-11ee-bc91-c7b262605968. 2024-04-17T05:59:03.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:03 smithi098 ceph-mon[36516]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T05:59:03.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:03 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:59:03.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:03 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:59:03.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:03 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:59:03.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:03 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-17T05:59:03.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:03 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-17T05:59:03.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:03 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:59:03.388 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:03 smithi047 ceph-mon[32223]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T05:59:03.388 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:03 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:59:03.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:03 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:59:03.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:03 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:59:03.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:03 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-17T05:59:03.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:03 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-17T05:59:03.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:03 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:59:03.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:03 smithi103 ceph-mon[36243]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T05:59:03.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:03 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:59:03.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:03 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:59:03.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:03 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:59:03.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:03 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-17T05:59:03.535 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:03 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-17T05:59:03.535 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:03 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:59:03.658 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T05:59:03.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:03.773+0000 7f08d1263640 1 -- 172.21.15.47:0/96638358 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08cc1031c0 msgr2=0x7f08cc103640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:03.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:03.773+0000 7f08d1263640 1 --2- 172.21.15.47:0/96638358 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08cc1031c0 0x7f08cc103640 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f08b4009900 tx=0x7f08b402bc30 comp rx=0 tx=0).stop 2024-04-17T05:59:03.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:03.773+0000 7f08d1263640 1 -- 172.21.15.47:0/96638358 shutdown_connections 2024-04-17T05:59:03.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:03.773+0000 7f08d1263640 1 --2- 172.21.15.47:0/96638358 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f08cc103b80 0x7f08cc10a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:03.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:03.773+0000 7f08d1263640 1 --2- 172.21.15.47:0/96638358 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08cc1031c0 0x7f08cc103640 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:03.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:03.773+0000 7f08d1263640 1 --2- 172.21.15.47:0/96638358 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f08cc101fd0 0x7f08cc1023f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:03.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:03.773+0000 7f08d1263640 1 -- 172.21.15.47:0/96638358 >> 172.21.15.47:0/96638358 conn(0x7f08cc0fd540 msgr2=0x7f08cc0ff980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:59:03.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:03.773+0000 7f08d1263640 1 -- 172.21.15.47:0/96638358 shutdown_connections 2024-04-17T05:59:03.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:03.773+0000 7f08d1263640 1 -- 172.21.15.47:0/96638358 wait complete. 2024-04-17T05:59:03.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:03.773+0000 7f08d1263640 1 Processor -- start 2024-04-17T05:59:03.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:03.773+0000 7f08d1263640 1 -- start start 2024-04-17T05:59:03.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:03.773+0000 7f08d1263640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08cc101fd0 0x7f08cc1982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:03.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:03.773+0000 7f08d1263640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f08cc1031c0 0x7f08cc198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:03.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:03.773+0000 7f08d1263640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f08cc103b80 0x7f08cc19f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:03.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:03.773+0000 7f08d1263640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f08cc10dc60 con 0x7f08cc101fd0 2024-04-17T05:59:03.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:03.773+0000 7f08d1263640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f08cc10dae0 con 0x7f08cc103b80 2024-04-17T05:59:03.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:03.773+0000 7f08d1263640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f08cc10dde0 con 0x7f08cc1031c0 2024-04-17T05:59:03.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:03.774+0000 7f08caffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08cc101fd0 0x7f08cc1982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:03.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:03.774+0000 7f08caffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08cc101fd0 0x7f08cc1982c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58992/0 (socket says 172.21.15.47:58992) 2024-04-17T05:59:03.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:03.774+0000 7f08caffd640 1 -- 172.21.15.47:0/793750509 learned_addr learned my addr 172.21.15.47:0/793750509 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:59:03.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:03.774+0000 7f08c3fff640 1 --2- 172.21.15.47:0/793750509 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f08cc1031c0 0x7f08cc198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:03.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:03.774+0000 7f08cb7fe640 1 --2- 172.21.15.47:0/793750509 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f08cc103b80 0x7f08cc19f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:03.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:03.774+0000 7f08caffd640 1 -- 172.21.15.47:0/793750509 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f08cc1031c0 msgr2=0x7f08cc198800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:03.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:03.774+0000 7f08caffd640 1 --2- 172.21.15.47:0/793750509 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f08cc1031c0 0x7f08cc198800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:03.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:03.774+0000 7f08caffd640 1 -- 172.21.15.47:0/793750509 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f08cc103b80 msgr2=0x7f08cc19f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:03.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:03.774+0000 7f08caffd640 1 --2- 172.21.15.47:0/793750509 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f08cc103b80 0x7f08cc19f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:03.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:03.774+0000 7f08caffd640 1 -- 172.21.15.47:0/793750509 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08cc19fee0 con 0x7f08cc101fd0 2024-04-17T05:59:03.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:03.774+0000 7f08cb7fe640 1 --2- 172.21.15.47:0/793750509 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f08cc103b80 0x7f08cc19f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T05:59:03.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:03.774+0000 7f08caffd640 1 --2- 172.21.15.47:0/793750509 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08cc101fd0 0x7f08cc1982c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f08bc00d770 tx=0x7f08bc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:59:03.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:03.774+0000 7f08c8ff9640 1 -- 172.21.15.47:0/793750509 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08bc017070 con 0x7f08cc101fd0 2024-04-17T05:59:03.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:03.774+0000 7f08c8ff9640 1 -- 172.21.15.47:0/793750509 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08bc009c30 con 0x7f08cc101fd0 2024-04-17T05:59:03.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:03.774+0000 7f08d1263640 1 -- 172.21.15.47:0/793750509 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08cc1a01d0 con 0x7f08cc101fd0 2024-04-17T05:59:03.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:03.774+0000 7f08c8ff9640 1 -- 172.21.15.47:0/793750509 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08bc003dd0 con 0x7f08cc101fd0 2024-04-17T05:59:03.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:03.774+0000 7f08d1263640 1 -- 172.21.15.47:0/793750509 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08cc1a0740 con 0x7f08cc101fd0 2024-04-17T05:59:03.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:03.775+0000 7f08d1263640 1 -- 172.21.15.47:0/793750509 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08cc1092f0 con 0x7f08cc101fd0 2024-04-17T05:59:03.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:03.777+0000 7f08c8ff9640 1 -- 172.21.15.47:0/793750509 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f08bc003f90 con 0x7f08cc101fd0 2024-04-17T05:59:03.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:03.778+0000 7f08c8ff9640 1 --2- 172.21.15.47:0/793750509 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f08b80a3c30 0x7f08b80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:03.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:03.778+0000 7f08c3fff640 1 --2- 172.21.15.47:0/793750509 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f08b80a3c30 0x7f08b80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:03.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:03.778+0000 7f08c8ff9640 1 -- 172.21.15.47:0/793750509 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f08bc0cf510 con 0x7f08cc101fd0 2024-04-17T05:59:03.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:03.778+0000 7f08c3fff640 1 --2- 172.21.15.47:0/793750509 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f08b80a3c30 0x7f08b80a60f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f08b402c7b0 tx=0x7f08b4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:59:03.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:03.782+0000 7f08c8ff9640 1 -- 172.21.15.47:0/793750509 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f08bc009de0 con 0x7f08cc101fd0 2024-04-17T05:59:04.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:04.017+0000 7f08d1263640 1 -- 172.21.15.47:0/793750509 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f08cc02d5d0 con 0x7f08cc101fd0 2024-04-17T05:59:04.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:04.018+0000 7f08c8ff9640 1 -- 172.21.15.47:0/793750509 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8824 (secure 0 0 0) 0x7f08bc09c390 con 0x7f08cc101fd0 2024-04-17T05:59:04.016 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-17T05:59:04.017 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3189958720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3189958720},{"type":"v1","addr":"172.21.15.47:6835","nonce":3189958720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14595,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/3143529973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":3143529973},{"type":"v1","addr":"172.21.15.47:6837","nonce":3143529973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14652,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/2375823701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":2375823701},{"type":"v1","addr":"172.21.15.47:6839","nonce":2375823701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/2398363754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":2398363754},{"type":"v1","addr":"172.21.15.103:6833","nonce":2398363754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/384473493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":384473493},{"type":"v1","addr":"172.21.15.98:6833","nonce":384473493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/270543522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":270543522},{"type":"v1","addr":"172.21.15.103:6835","nonce":270543522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24448,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/116936024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":116936024},{"type":"v1","addr":"172.21.15.103:6837","nonce":116936024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24449,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/1966500278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":1966500278},{"type":"v1","addr":"172.21.15.98:6835","nonce":1966500278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24466,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6839/1467157889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":1467157889},{"type":"v1","addr":"172.21.15.103:6839","nonce":1467157889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24479,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/1117440719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1117440719},{"type":"v1","addr":"172.21.15.98:6837","nonce":1117440719}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-17T05:59:04.017 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 11 2024-04-17T05:59:04.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:04.020+0000 7f08d1263640 1 -- 172.21.15.47:0/793750509 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f08b80a3c30 msgr2=0x7f08b80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:04.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:04.020+0000 7f08d1263640 1 --2- 172.21.15.47:0/793750509 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f08b80a3c30 0x7f08b80a60f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f08b402c7b0 tx=0x7f08b4005fd0 comp rx=0 tx=0).stop 2024-04-17T05:59:04.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:04.020+0000 7f08d1263640 1 -- 172.21.15.47:0/793750509 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08cc101fd0 msgr2=0x7f08cc1982c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:04.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:04.020+0000 7f08d1263640 1 --2- 172.21.15.47:0/793750509 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08cc101fd0 0x7f08cc1982c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f08bc00d770 tx=0x7f08bc00dc30 comp rx=0 tx=0).stop 2024-04-17T05:59:04.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:04.020+0000 7f08d1263640 1 -- 172.21.15.47:0/793750509 shutdown_connections 2024-04-17T05:59:04.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:04.020+0000 7f08d1263640 1 --2- 172.21.15.47:0/793750509 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f08b80a3c30 0x7f08b80a60f0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:04.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:04.020+0000 7f08d1263640 1 --2- 172.21.15.47:0/793750509 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f08cc103b80 0x7f08cc19f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:04.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:04.020+0000 7f08d1263640 1 --2- 172.21.15.47:0/793750509 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f08cc1031c0 0x7f08cc198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:04.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:04.020+0000 7f08d1263640 1 --2- 172.21.15.47:0/793750509 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08cc101fd0 0x7f08cc1982c0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:04.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:04.020+0000 7f08d1263640 1 -- 172.21.15.47:0/793750509 >> 172.21.15.47:0/793750509 conn(0x7f08cc0fd540 msgr2=0x7f08cc1064c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:59:04.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:04.020+0000 7f08d1263640 1 -- 172.21.15.47:0/793750509 shutdown_connections 2024-04-17T05:59:04.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:04.020+0000 7f08d1263640 1 -- 172.21.15.47:0/793750509 wait complete. 2024-04-17T05:59:04.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:04 smithi047 ceph-mon[32223]: Deploying daemon mds.a on smithi047 2024-04-17T05:59:04.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:04 smithi047 ceph-mon[32223]: mds.? [v2:172.21.15.47:6838/2375823701,v1:172.21.15.47:6839/2375823701] up:boot 2024-04-17T05:59:04.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:04 smithi047 ceph-mon[32223]: fsmap 10 up:standby 2024-04-17T05:59:04.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:04 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-17T05:59:04.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:04 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/793750509' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T05:59:04.382 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:04 smithi098 ceph-mon[36516]: Deploying daemon mds.a on smithi047 2024-04-17T05:59:04.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:04 smithi098 ceph-mon[36516]: mds.? [v2:172.21.15.47:6838/2375823701,v1:172.21.15.47:6839/2375823701] up:boot 2024-04-17T05:59:04.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:04 smithi098 ceph-mon[36516]: fsmap 10 up:standby 2024-04-17T05:59:04.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:04 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-17T05:59:04.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:04 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/793750509' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T05:59:04.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:04 smithi103 ceph-mon[36243]: Deploying daemon mds.a on smithi047 2024-04-17T05:59:04.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:04 smithi103 ceph-mon[36243]: mds.? [v2:172.21.15.47:6838/2375823701,v1:172.21.15.47:6839/2375823701] up:boot 2024-04-17T05:59:04.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:04 smithi103 ceph-mon[36243]: fsmap 10 up:standby 2024-04-17T05:59:04.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:04 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-17T05:59:04.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:04 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/793750509' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T05:59:05.301 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:05 smithi047 ceph-mon[32223]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T05:59:05.302 INFO:journalctl@ceph.mds.a.smithi047.stdout:Apr 17 05:59:05 smithi047 systemd[1]: Starting Ceph mds.a for 565db2d8-fc7e-11ee-bc91-c7b262605968... 2024-04-17T05:59:05.382 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:05 smithi098 ceph-mon[36516]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T05:59:05.533 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:05 smithi103 ceph-mon[36243]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T05:59:05.638 INFO:journalctl@ceph.mds.a.smithi047.stdout:Apr 17 05:59:05 smithi047 podman[74663]: 2024-04-17 05:59:05.24215163 +0000 UTC m=+0.018165257 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-17T05:59:05.638 INFO:journalctl@ceph.mds.a.smithi047.stdout:Apr 17 05:59:05 smithi047 podman[74663]: 2024-04-17 05:59:05.354964229 +0000 UTC m=+0.130977833 container create ba359aa70ff5c0e4f19bcf246156f6ac9df49cb842d01fbbda88f0f1fd38ea98 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-a, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-04-17T05:59:05.889 INFO:journalctl@ceph.mds.a.smithi047.stdout:Apr 17 05:59:05 smithi047 podman[74663]: 2024-04-17 05:59:05.641123222 +0000 UTC m=+0.417136826 container init ba359aa70ff5c0e4f19bcf246156f6ac9df49cb842d01fbbda88f0f1fd38ea98 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-a, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-17T05:59:05.889 INFO:journalctl@ceph.mds.a.smithi047.stdout:Apr 17 05:59:05 smithi047 podman[74663]: 2024-04-17 05:59:05.64405722 +0000 UTC m=+0.420070825 container start ba359aa70ff5c0e4f19bcf246156f6ac9df49cb842d01fbbda88f0f1fd38ea98 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-a, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-17T05:59:05.889 INFO:journalctl@ceph.mds.a.smithi047.stdout:Apr 17 05:59:05 smithi047 ceph-mds[74683]: -- 172.21.15.47:0/4272516241 <== mon.0 v2:172.21.15.47:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5620aa723a40 con 0x5620aa6f7000 2024-04-17T05:59:05.889 INFO:journalctl@ceph.mds.a.smithi047.stdout:Apr 17 05:59:05 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-a[74679]: starting mds.a at 2024-04-17T05:59:05.889 INFO:journalctl@ceph.mds.a.smithi047.stdout:Apr 17 05:59:05 smithi047 bash[74663]: ba359aa70ff5c0e4f19bcf246156f6ac9df49cb842d01fbbda88f0f1fd38ea98 2024-04-17T05:59:05.889 INFO:journalctl@ceph.mds.a.smithi047.stdout:Apr 17 05:59:05 smithi047 systemd[1]: Started Ceph mds.a for 565db2d8-fc7e-11ee-bc91-c7b262605968. 2024-04-17T05:59:06.029 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T05:59:06.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:06.143+0000 7f4204b04640 1 -- 172.21.15.47:0/2324859211 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4200102070 msgr2=0x7f42001024f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:06.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:06.143+0000 7f4204b04640 1 --2- 172.21.15.47:0/2324859211 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4200102070 0x7f42001024f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f41e8009900 tx=0x7f41e802bc30 comp rx=0 tx=0).stop 2024-04-17T05:59:06.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:06.143+0000 7f4204b04640 1 -- 172.21.15.47:0/2324859211 shutdown_connections 2024-04-17T05:59:06.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:06.143+0000 7f4204b04640 1 --2- 172.21.15.47:0/2324859211 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4200102a30 0x7f4200109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:06.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:06.143+0000 7f4204b04640 1 --2- 172.21.15.47:0/2324859211 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4200102070 0x7f42001024f0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:06.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:06.143+0000 7f4204b04640 1 --2- 172.21.15.47:0/2324859211 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4200100f20 0x7f4200101340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:06.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:06.143+0000 7f4204b04640 1 -- 172.21.15.47:0/2324859211 >> 172.21.15.47:0/2324859211 conn(0x7f42000fc4b0 msgr2=0x7f42000fe8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:59:06.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:06.143+0000 7f4204b04640 1 -- 172.21.15.47:0/2324859211 shutdown_connections 2024-04-17T05:59:06.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:06.143+0000 7f4204b04640 1 -- 172.21.15.47:0/2324859211 wait complete. 2024-04-17T05:59:06.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:06.144+0000 7f4204b04640 1 Processor -- start 2024-04-17T05:59:06.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:06.144+0000 7f4204b04640 1 -- start start 2024-04-17T05:59:06.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:06.144+0000 7f4204b04640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4200100f20 0x7f420010b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:06.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:06.144+0000 7f4204b04640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4200102070 0x7f4200109f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:06.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:06.144+0000 7f4204b04640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4200102a30 0x7f420010a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:06.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:06.144+0000 7f4204b04640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f420010dcb0 con 0x7f4200102a30 2024-04-17T05:59:06.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:06.144+0000 7f4204b04640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f420010db30 con 0x7f4200100f20 2024-04-17T05:59:06.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:06.144+0000 7f4204b04640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f420010de30 con 0x7f4200102070 2024-04-17T05:59:06.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:06.144+0000 7f41fed9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4200102a30 0x7f420010a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:06.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:06.144+0000 7f41fed9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4200102a30 0x7f420010a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41600/0 (socket says 172.21.15.47:41600) 2024-04-17T05:59:06.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:06.144+0000 7f41fed9d640 1 -- 172.21.15.47:0/3499738040 learned_addr learned my addr 172.21.15.47:0/3499738040 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:59:06.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:06.144+0000 7f41fe59c640 1 --2- 172.21.15.47:0/3499738040 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4200100f20 0x7f420010b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:06.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:06.144+0000 7f41f7fff640 1 --2- 172.21.15.47:0/3499738040 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4200102070 0x7f4200109f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:06.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:06.144+0000 7f41fed9d640 1 -- 172.21.15.47:0/3499738040 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4200102070 msgr2=0x7f4200109f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:06.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:06.144+0000 7f41fed9d640 1 --2- 172.21.15.47:0/3499738040 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4200102070 0x7f4200109f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:06.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:06.144+0000 7f41fed9d640 1 -- 172.21.15.47:0/3499738040 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4200100f20 msgr2=0x7f420010b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:06.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:06.144+0000 7f41fed9d640 1 --2- 172.21.15.47:0/3499738040 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4200100f20 0x7f420010b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:06.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:06.144+0000 7f41fed9d640 1 -- 172.21.15.47:0/3499738040 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f420010acd0 con 0x7f4200102a30 2024-04-17T05:59:06.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:06.145+0000 7f41fed9d640 1 --2- 172.21.15.47:0/3499738040 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4200102a30 0x7f420010a440 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f41f800b970 tx=0x7f41f800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:59:06.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:06.145+0000 7f41f77fe640 1 -- 172.21.15.47:0/3499738040 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f8019070 con 0x7f4200102a30 2024-04-17T05:59:06.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:06.145+0000 7f4204b04640 1 -- 172.21.15.47:0/3499738040 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42001a2440 con 0x7f4200102a30 2024-04-17T05:59:06.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:06.145+0000 7f41f77fe640 1 -- 172.21.15.47:0/3499738040 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41f800cb40 con 0x7f4200102a30 2024-04-17T05:59:06.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:06.145+0000 7f41f77fe640 1 -- 172.21.15.47:0/3499738040 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f8007650 con 0x7f4200102a30 2024-04-17T05:59:06.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:06.145+0000 7f4204b04640 1 -- 172.21.15.47:0/3499738040 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42001a2940 con 0x7f4200102a30 2024-04-17T05:59:06.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:06.146+0000 7f4204b04640 1 -- 172.21.15.47:0/3499738040 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4200108280 con 0x7f4200102a30 2024-04-17T05:59:06.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:06.147+0000 7f41f77fe640 1 -- 172.21.15.47:0/3499738040 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f41f801c070 con 0x7f4200102a30 2024-04-17T05:59:06.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:06.148+0000 7f41f77fe640 1 --2- 172.21.15.47:0/3499738040 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f41ec0a3bb0 0x7f41ec0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:06.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:06.148+0000 7f41fe59c640 1 --2- 172.21.15.47:0/3499738040 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f41ec0a3bb0 0x7f41ec0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:06.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:06.148+0000 7f41f77fe640 1 -- 172.21.15.47:0/3499738040 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f41f80cfa90 con 0x7f4200102a30 2024-04-17T05:59:06.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:06.148+0000 7f41fe59c640 1 --2- 172.21.15.47:0/3499738040 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f41ec0a3bb0 0x7f41ec0a6070 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f41f0005d00 tx=0x7f41f000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:59:06.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:06.152+0000 7f41f77fe640 1 -- 172.21.15.47:0/3499738040 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f41f809c2c0 con 0x7f4200102a30 2024-04-17T05:59:06.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:06.380+0000 7f4204b04640 1 -- 172.21.15.47:0/3499738040 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f420002d5d0 con 0x7f4200102a30 2024-04-17T05:59:06.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:06.380+0000 7f41f77fe640 1 -- 172.21.15.47:0/3499738040 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9650 (secure 0 0 0) 0x7f41f80a05c0 con 0x7f4200102a30 2024-04-17T05:59:06.378 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-17T05:59:06.378 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3189958720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3189958720},{"type":"v1","addr":"172.21.15.47:6835","nonce":3189958720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14595,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/3143529973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":3143529973},{"type":"v1","addr":"172.21.15.47:6837","nonce":3143529973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14652,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/2375823701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":2375823701},{"type":"v1","addr":"172.21.15.47:6839","nonce":2375823701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14664,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6841/3334309157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":3334309157},{"type":"v1","addr":"172.21.15.47:6841","nonce":3334309157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/2398363754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":2398363754},{"type":"v1","addr":"172.21.15.103:6833","nonce":2398363754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/384473493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":384473493},{"type":"v1","addr":"172.21.15.98:6833","nonce":384473493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/270543522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":270543522},{"type":"v1","addr":"172.21.15.103:6835","nonce":270543522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24448,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/116936024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":116936024},{"type":"v1","addr":"172.21.15.103:6837","nonce":116936024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24449,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/1966500278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":1966500278},{"type":"v1","addr":"172.21.15.98:6835","nonce":1966500278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24466,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6839/1467157889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":1467157889},{"type":"v1","addr":"172.21.15.103:6839","nonce":1467157889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24479,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/1117440719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1117440719},{"type":"v1","addr":"172.21.15.98:6837","nonce":1117440719}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-17T05:59:06.379 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 12 2024-04-17T05:59:06.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:06.382+0000 7f4204b04640 1 -- 172.21.15.47:0/3499738040 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f41ec0a3bb0 msgr2=0x7f41ec0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:06.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:06.382+0000 7f4204b04640 1 --2- 172.21.15.47:0/3499738040 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f41ec0a3bb0 0x7f41ec0a6070 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f41f0005d00 tx=0x7f41f000b410 comp rx=0 tx=0).stop 2024-04-17T05:59:06.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:06.382+0000 7f4204b04640 1 -- 172.21.15.47:0/3499738040 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4200102a30 msgr2=0x7f420010a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:06.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:06.382+0000 7f4204b04640 1 --2- 172.21.15.47:0/3499738040 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4200102a30 0x7f420010a440 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f41f800b970 tx=0x7f41f800be30 comp rx=0 tx=0).stop 2024-04-17T05:59:06.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:06.382+0000 7f4204b04640 1 -- 172.21.15.47:0/3499738040 shutdown_connections 2024-04-17T05:59:06.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:06.382+0000 7f4204b04640 1 --2- 172.21.15.47:0/3499738040 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f41ec0a3bb0 0x7f41ec0a6070 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:06.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:06.382+0000 7f4204b04640 1 --2- 172.21.15.47:0/3499738040 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4200102a30 0x7f420010a440 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:06.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:06.382+0000 7f4204b04640 1 --2- 172.21.15.47:0/3499738040 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4200102070 0x7f4200109f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:06.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:06.382+0000 7f4204b04640 1 --2- 172.21.15.47:0/3499738040 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4200100f20 0x7f420010b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:06.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:06.382+0000 7f4204b04640 1 -- 172.21.15.47:0/3499738040 >> 172.21.15.47:0/3499738040 conn(0x7f42000fc4b0 msgr2=0x7f4200105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:59:06.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:06.382+0000 7f4204b04640 1 -- 172.21.15.47:0/3499738040 shutdown_connections 2024-04-17T05:59:06.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:06.382+0000 7f4204b04640 1 -- 172.21.15.47:0/3499738040 wait complete. 2024-04-17T05:59:07.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:06 smithi098 ceph-mon[36516]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T05:59:07.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:06 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:59:07.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:06 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:59:07.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:06 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:59:07.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:06 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-17T05:59:07.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:06 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-17T05:59:07.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:06 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:59:07.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:06 smithi098 ceph-mon[36516]: Deploying daemon mds.b on smithi098 2024-04-17T05:59:07.134 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:06 smithi098 ceph-mon[36516]: mds.? [v2:172.21.15.47:6840/3334309157,v1:172.21.15.47:6841/3334309157] up:boot 2024-04-17T05:59:07.134 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:06 smithi098 ceph-mon[36516]: fsmap 11 up:standby 2024-04-17T05:59:07.134 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:06 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-17T05:59:07.134 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:06 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/3499738040' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T05:59:07.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:06 smithi047 ceph-mon[32223]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T05:59:07.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:06 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:59:07.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:06 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:59:07.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:06 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:59:07.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:06 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-17T05:59:07.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:06 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-17T05:59:07.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:06 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:59:07.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:06 smithi047 ceph-mon[32223]: Deploying daemon mds.b on smithi098 2024-04-17T05:59:07.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:06 smithi047 ceph-mon[32223]: mds.? [v2:172.21.15.47:6840/3334309157,v1:172.21.15.47:6841/3334309157] up:boot 2024-04-17T05:59:07.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:06 smithi047 ceph-mon[32223]: fsmap 11 up:standby 2024-04-17T05:59:07.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:06 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-17T05:59:07.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:06 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/3499738040' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T05:59:07.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:06 smithi103 ceph-mon[36243]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T05:59:07.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:06 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:59:07.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:06 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:59:07.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:06 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:59:07.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:06 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-17T05:59:07.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:06 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-17T05:59:07.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:06 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:59:07.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:06 smithi103 ceph-mon[36243]: Deploying daemon mds.b on smithi098 2024-04-17T05:59:07.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:06 smithi103 ceph-mon[36243]: mds.? [v2:172.21.15.47:6840/3334309157,v1:172.21.15.47:6841/3334309157] up:boot 2024-04-17T05:59:07.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:06 smithi103 ceph-mon[36243]: fsmap 11 up:standby 2024-04-17T05:59:07.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:06 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-17T05:59:07.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:06 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/3499738040' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T05:59:08.133 INFO:journalctl@ceph.mds.b.smithi098.stdout:Apr 17 05:59:08 smithi098 systemd[1]: Starting Ceph mds.b for 565db2d8-fc7e-11ee-bc91-c7b262605968... 2024-04-17T05:59:08.392 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T05:59:08.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:08.512+0000 7f12248f1640 1 -- 172.21.15.47:0/3619044021 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1220102050 msgr2=0x7f12201024d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:08.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:08.512+0000 7f12248f1640 1 --2- 172.21.15.47:0/3619044021 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1220102050 0x7f12201024d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f1208009900 tx=0x7f120802bc30 comp rx=0 tx=0).stop 2024-04-17T05:59:08.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:08.512+0000 7f12248f1640 1 -- 172.21.15.47:0/3619044021 shutdown_connections 2024-04-17T05:59:08.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:08.512+0000 7f12248f1640 1 --2- 172.21.15.47:0/3619044021 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1220102a10 0x7f1220109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:08.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:08.512+0000 7f12248f1640 1 --2- 172.21.15.47:0/3619044021 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1220102050 0x7f12201024d0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:08.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:08.512+0000 7f12248f1640 1 --2- 172.21.15.47:0/3619044021 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1220100e70 0x7f1220101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:08.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:08.512+0000 7f12248f1640 1 -- 172.21.15.47:0/3619044021 >> 172.21.15.47:0/3619044021 conn(0x7f12200fc450 msgr2=0x7f12200fe870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:59:08.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:08.512+0000 7f12248f1640 1 -- 172.21.15.47:0/3619044021 shutdown_connections 2024-04-17T05:59:08.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:08.512+0000 7f12248f1640 1 -- 172.21.15.47:0/3619044021 wait complete. 2024-04-17T05:59:08.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:08.513+0000 7f12248f1640 1 Processor -- start 2024-04-17T05:59:08.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:08.513+0000 7f12248f1640 1 -- start start 2024-04-17T05:59:08.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:08.513+0000 7f12248f1640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1220100e70 0x7f1220198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:08.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:08.513+0000 7f12248f1640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1220102050 0x7f1220198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:08.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:08.513+0000 7f12248f1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1220102a10 0x7f122019f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:08.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:08.513+0000 7f12248f1640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f122010dcd0 con 0x7f1220102050 2024-04-17T05:59:08.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:08.513+0000 7f12248f1640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f122010db50 con 0x7f1220100e70 2024-04-17T05:59:08.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:08.513+0000 7f12248f1640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f122010de50 con 0x7f1220102a10 2024-04-17T05:59:08.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:08.513+0000 7f121dd9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1220102050 0x7f1220198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:08.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:08.513+0000 7f121dd9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1220102050 0x7f1220198880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41616/0 (socket says 172.21.15.47:41616) 2024-04-17T05:59:08.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:08.513+0000 7f121dd9b640 1 -- 172.21.15.47:0/936363950 learned_addr learned my addr 172.21.15.47:0/936363950 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:59:08.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:08.513+0000 7f121e59c640 1 --2- 172.21.15.47:0/936363950 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1220100e70 0x7f1220198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:08.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:08.513+0000 7f121dd9b640 1 -- 172.21.15.47:0/936363950 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1220102a10 msgr2=0x7f122019f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:08.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:08.513+0000 7f121ed9d640 1 --2- 172.21.15.47:0/936363950 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1220102a10 0x7f122019f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:08.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:08.513+0000 7f121dd9b640 1 --2- 172.21.15.47:0/936363950 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1220102a10 0x7f122019f8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:08.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:08.513+0000 7f121dd9b640 1 -- 172.21.15.47:0/936363950 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1220100e70 msgr2=0x7f1220198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:08.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:08.513+0000 7f121dd9b640 1 --2- 172.21.15.47:0/936363950 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1220100e70 0x7f1220198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:08.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:08.513+0000 7f121dd9b640 1 -- 172.21.15.47:0/936363950 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f122019ff60 con 0x7f1220102050 2024-04-17T05:59:08.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:08.513+0000 7f121e59c640 1 --2- 172.21.15.47:0/936363950 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1220100e70 0x7f1220198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T05:59:08.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:08.514+0000 7f121dd9b640 1 --2- 172.21.15.47:0/936363950 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1220102050 0x7f1220198880 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1208009230 tx=0x7f1208005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:59:08.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:08.514+0000 7f12177fe640 1 -- 172.21.15.47:0/936363950 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1208041070 con 0x7f1220102050 2024-04-17T05:59:08.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:08.514+0000 7f12177fe640 1 -- 172.21.15.47:0/936363950 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12080044d0 con 0x7f1220102050 2024-04-17T05:59:08.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:08.514+0000 7f12248f1640 1 -- 172.21.15.47:0/936363950 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12201a01f0 con 0x7f1220102050 2024-04-17T05:59:08.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:08.514+0000 7f12177fe640 1 -- 172.21.15.47:0/936363950 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1208034440 con 0x7f1220102050 2024-04-17T05:59:08.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:08.514+0000 7f12248f1640 1 -- 172.21.15.47:0/936363950 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12201a06e0 con 0x7f1220102050 2024-04-17T05:59:08.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:08.515+0000 7f12248f1640 1 -- 172.21.15.47:0/936363950 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1220108210 con 0x7f1220102050 2024-04-17T05:59:08.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:08.516+0000 7f12177fe640 1 -- 172.21.15.47:0/936363950 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f1208044070 con 0x7f1220102050 2024-04-17T05:59:08.517 INFO:journalctl@ceph.mds.b.smithi098.stdout:Apr 17 05:59:08 smithi098 podman[67348]: 2024-04-17 05:59:08.167405736 +0000 UTC m=+0.017777009 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-17T05:59:08.517 INFO:journalctl@ceph.mds.b.smithi098.stdout:Apr 17 05:59:08 smithi098 podman[67348]: 2024-04-17 05:59:08.268115974 +0000 UTC m=+0.118487272 container create b45b802cb363de6617f0822ce68347498cc102bf1f82c39d693cf8712f74717d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-b, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-17T05:59:08.517 INFO:journalctl@ceph.mds.b.smithi098.stdout:Apr 17 05:59:08 smithi098 podman[67348]: 2024-04-17 05:59:08.48769934 +0000 UTC m=+0.338070602 container init b45b802cb363de6617f0822ce68347498cc102bf1f82c39d693cf8712f74717d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-b, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-17T05:59:08.517 INFO:journalctl@ceph.mds.b.smithi098.stdout:Apr 17 05:59:08 smithi098 podman[67348]: 2024-04-17 05:59:08.490493704 +0000 UTC m=+0.340864964 container start b45b802cb363de6617f0822ce68347498cc102bf1f82c39d693cf8712f74717d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-b, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-17T05:59:08.517 INFO:journalctl@ceph.mds.b.smithi098.stdout:Apr 17 05:59:08 smithi098 ceph-mds[67367]: -- 172.21.15.98:0/2259795181 <== mon.2 v2:172.21.15.98:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d2ab285a40 con 0x55d2ab258c00 2024-04-17T05:59:08.518 INFO:journalctl@ceph.mds.b.smithi098.stdout:Apr 17 05:59:08 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-b[67363]: starting mds.b at 2024-04-17T05:59:08.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:08.516+0000 7f12177fe640 1 --2- 172.21.15.47:0/936363950 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f120c0a3bb0 0x7f120c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:08.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:08.517+0000 7f121e59c640 1 --2- 172.21.15.47:0/936363950 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f120c0a3bb0 0x7f120c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:08.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:08.517+0000 7f12177fe640 1 -- 172.21.15.47:0/936363950 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f12080f1c70 con 0x7f1220102050 2024-04-17T05:59:08.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:08.517+0000 7f121e59c640 1 --2- 172.21.15.47:0/936363950 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f120c0a3bb0 0x7f120c0a6070 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1220101bc0 tx=0x7f1210009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:59:08.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:08.521+0000 7f12177fe640 1 -- 172.21.15.47:0/936363950 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f12080beaf0 con 0x7f1220102050 2024-04-17T05:59:08.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:08.750+0000 7f12248f1640 1 -- 172.21.15.47:0/936363950 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f122002d780 con 0x7f1220102050 2024-04-17T05:59:08.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:08.750+0000 7f12177fe640 1 -- 172.21.15.47:0/936363950 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9650 (secure 0 0 0) 0x7f12080be3f0 con 0x7f1220102050 2024-04-17T05:59:08.749 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-17T05:59:08.750 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3189958720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3189958720},{"type":"v1","addr":"172.21.15.47:6835","nonce":3189958720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14595,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/3143529973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":3143529973},{"type":"v1","addr":"172.21.15.47:6837","nonce":3143529973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14652,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/2375823701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":2375823701},{"type":"v1","addr":"172.21.15.47:6839","nonce":2375823701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14664,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6841/3334309157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":3334309157},{"type":"v1","addr":"172.21.15.47:6841","nonce":3334309157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/2398363754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":2398363754},{"type":"v1","addr":"172.21.15.103:6833","nonce":2398363754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/384473493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":384473493},{"type":"v1","addr":"172.21.15.98:6833","nonce":384473493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/270543522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":270543522},{"type":"v1","addr":"172.21.15.103:6835","nonce":270543522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24448,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/116936024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":116936024},{"type":"v1","addr":"172.21.15.103:6837","nonce":116936024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24449,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/1966500278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":1966500278},{"type":"v1","addr":"172.21.15.98:6835","nonce":1966500278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24466,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6839/1467157889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":1467157889},{"type":"v1","addr":"172.21.15.103:6839","nonce":1467157889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24479,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/1117440719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1117440719},{"type":"v1","addr":"172.21.15.98:6837","nonce":1117440719}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-17T05:59:08.750 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 12 2024-04-17T05:59:08.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:08.752+0000 7f12248f1640 1 -- 172.21.15.47:0/936363950 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f120c0a3bb0 msgr2=0x7f120c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:08.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:08.752+0000 7f12248f1640 1 --2- 172.21.15.47:0/936363950 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f120c0a3bb0 0x7f120c0a6070 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1220101bc0 tx=0x7f1210009500 comp rx=0 tx=0).stop 2024-04-17T05:59:08.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:08.752+0000 7f12248f1640 1 -- 172.21.15.47:0/936363950 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1220102050 msgr2=0x7f1220198880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:08.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:08.752+0000 7f12248f1640 1 --2- 172.21.15.47:0/936363950 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1220102050 0x7f1220198880 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1208009230 tx=0x7f1208005d50 comp rx=0 tx=0).stop 2024-04-17T05:59:08.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:08.752+0000 7f12248f1640 1 -- 172.21.15.47:0/936363950 shutdown_connections 2024-04-17T05:59:08.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:08.752+0000 7f12248f1640 1 --2- 172.21.15.47:0/936363950 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f120c0a3bb0 0x7f120c0a6070 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:08.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:08.752+0000 7f12248f1640 1 --2- 172.21.15.47:0/936363950 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1220102a10 0x7f122019f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:08.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:08.752+0000 7f12248f1640 1 --2- 172.21.15.47:0/936363950 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1220102050 0x7f1220198880 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:08.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:08.752+0000 7f12248f1640 1 --2- 172.21.15.47:0/936363950 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1220100e70 0x7f1220198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:08.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:08.752+0000 7f12248f1640 1 -- 172.21.15.47:0/936363950 >> 172.21.15.47:0/936363950 conn(0x7f12200fc450 msgr2=0x7f12201053e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:59:08.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:08.752+0000 7f12248f1640 1 -- 172.21.15.47:0/936363950 shutdown_connections 2024-04-17T05:59:08.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:08.752+0000 7f12248f1640 1 -- 172.21.15.47:0/936363950 wait complete. 2024-04-17T05:59:08.882 INFO:journalctl@ceph.mds.b.smithi098.stdout:Apr 17 05:59:08 smithi098 bash[67348]: b45b802cb363de6617f0822ce68347498cc102bf1f82c39d693cf8712f74717d 2024-04-17T05:59:08.882 INFO:journalctl@ceph.mds.b.smithi098.stdout:Apr 17 05:59:08 smithi098 systemd[1]: Started Ceph mds.b for 565db2d8-fc7e-11ee-bc91-c7b262605968. 2024-04-17T05:59:09.879 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:09 smithi103 ceph-mon[36243]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T05:59:09.880 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:09 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:59:09.880 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:09 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:59:09.880 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:09 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:59:09.880 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:09 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:59:09.880 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:09 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:59:09.880 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:09 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:59:09.880 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:09 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/936363950' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T05:59:09.880 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:09 smithi103 ceph-mon[36243]: mds.? [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] up:boot 2024-04-17T05:59:09.880 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:09 smithi103 ceph-mon[36243]: fsmap 12 up:standby 2024-04-17T05:59:09.881 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:09 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-17T05:59:09.881 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:09 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T05:59:09.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:09 smithi098 ceph-mon[36516]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T05:59:09.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:09 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:59:09.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:09 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:59:09.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:09 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:59:09.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:09 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:59:09.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:09 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:59:09.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:09 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:59:09.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:09 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/936363950' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T05:59:09.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:09 smithi098 ceph-mon[36516]: mds.? [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] up:boot 2024-04-17T05:59:09.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:09 smithi098 ceph-mon[36516]: fsmap 12 up:standby 2024-04-17T05:59:09.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:09 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-17T05:59:09.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:09 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T05:59:09.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:09 smithi047 ceph-mon[32223]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T05:59:09.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:09 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:59:09.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:09 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:59:09.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:09 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:59:09.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:09 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:59:09.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:09 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:59:09.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:09 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:59:09.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:09 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/936363950' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T05:59:09.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:09 smithi047 ceph-mon[32223]: mds.? [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] up:boot 2024-04-17T05:59:09.890 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:09 smithi047 ceph-mon[32223]: fsmap 12 up:standby 2024-04-17T05:59:09.890 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:09 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-17T05:59:09.890 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:09 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T05:59:10.764 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T05:59:10.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:10.874+0000 7fcd05635640 1 -- 172.21.15.47:0/3044233133 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcd00101ff0 msgr2=0x7fcd00102470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:10.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:10.874+0000 7fcd05635640 1 --2- 172.21.15.47:0/3044233133 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcd00101ff0 0x7fcd00102470 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fccf0009900 tx=0x7fccf002bc30 comp rx=0 tx=0).stop 2024-04-17T05:59:10.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:10.874+0000 7fcd05635640 1 -- 172.21.15.47:0/3044233133 shutdown_connections 2024-04-17T05:59:10.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:10.874+0000 7fcd05635640 1 --2- 172.21.15.47:0/3044233133 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcd001029b0 0x7fcd001092d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:10.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:10.874+0000 7fcd05635640 1 --2- 172.21.15.47:0/3044233133 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcd00101ff0 0x7fcd00102470 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:10.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:10.874+0000 7fcd05635640 1 --2- 172.21.15.47:0/3044233133 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcd00100ea0 0x7fcd001012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:10.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:10.874+0000 7fcd05635640 1 -- 172.21.15.47:0/3044233133 >> 172.21.15.47:0/3044233133 conn(0x7fcd000fc410 msgr2=0x7fcd000fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:59:10.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:10.874+0000 7fcd05635640 1 -- 172.21.15.47:0/3044233133 shutdown_connections 2024-04-17T05:59:10.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:10.874+0000 7fcd05635640 1 -- 172.21.15.47:0/3044233133 wait complete. 2024-04-17T05:59:10.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:10.874+0000 7fcd05635640 1 Processor -- start 2024-04-17T05:59:10.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:10.874+0000 7fcd05635640 1 -- start start 2024-04-17T05:59:10.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:10.875+0000 7fcd05635640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcd00100ea0 0x7fcd00069cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:10.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:10.875+0000 7fcd05635640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcd00101ff0 0x7fcd0019cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:10.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:10.875+0000 7fcd05635640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcd001029b0 0x7fcd0019f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:10.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:10.875+0000 7fcd05635640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fcd0010cb90 con 0x7fcd001029b0 2024-04-17T05:59:10.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:10.875+0000 7fcd05635640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fcd0010ca10 con 0x7fcd00100ea0 2024-04-17T05:59:10.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:10.875+0000 7fcd05635640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcd0010cd10 con 0x7fcd00101ff0 2024-04-17T05:59:10.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:10.875+0000 7fccff7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcd001029b0 0x7fcd0019f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:10.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:10.875+0000 7fccfeffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcd00100ea0 0x7fcd00069cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:10.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:10.875+0000 7fccff7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcd001029b0 0x7fcd0019f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41634/0 (socket says 172.21.15.47:41634) 2024-04-17T05:59:10.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:10.875+0000 7fccff7fe640 1 -- 172.21.15.47:0/2641397535 learned_addr learned my addr 172.21.15.47:0/2641397535 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:59:10.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:10.875+0000 7fccfe7fc640 1 --2- 172.21.15.47:0/2641397535 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcd00101ff0 0x7fcd0019cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:10.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:10.875+0000 7fccff7fe640 1 -- 172.21.15.47:0/2641397535 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcd00101ff0 msgr2=0x7fcd0019cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:10.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:10.875+0000 7fccff7fe640 1 --2- 172.21.15.47:0/2641397535 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcd00101ff0 0x7fcd0019cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:10.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:10.875+0000 7fccff7fe640 1 -- 172.21.15.47:0/2641397535 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcd00100ea0 msgr2=0x7fcd00069cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:10.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:10.875+0000 7fccff7fe640 1 --2- 172.21.15.47:0/2641397535 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcd00100ea0 0x7fcd00069cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:10.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:10.875+0000 7fccff7fe640 1 -- 172.21.15.47:0/2641397535 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd0019ff20 con 0x7fcd001029b0 2024-04-17T05:59:10.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:10.875+0000 7fccff7fe640 1 --2- 172.21.15.47:0/2641397535 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcd001029b0 0x7fcd0019f660 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fccf800eb10 tx=0x7fccf800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:59:10.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:10.875+0000 7fccf7fff640 1 -- 172.21.15.47:0/2641397535 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccf8004b20 con 0x7fcd001029b0 2024-04-17T05:59:10.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:10.876+0000 7fccf7fff640 1 -- 172.21.15.47:0/2641397535 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fccf8004cc0 con 0x7fcd001029b0 2024-04-17T05:59:10.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:10.876+0000 7fcd05635640 1 -- 172.21.15.47:0/2641397535 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd001a0210 con 0x7fcd001029b0 2024-04-17T05:59:10.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:10.876+0000 7fccf7fff640 1 -- 172.21.15.47:0/2641397535 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccf8010760 con 0x7fcd001029b0 2024-04-17T05:59:10.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:10.876+0000 7fcd05635640 1 -- 172.21.15.47:0/2641397535 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd001a0730 con 0x7fcd001029b0 2024-04-17T05:59:10.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:10.877+0000 7fcd05635640 1 -- 172.21.15.47:0/2641397535 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd001081b0 con 0x7fcd001029b0 2024-04-17T05:59:10.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:10.878+0000 7fccf7fff640 1 -- 172.21.15.47:0/2641397535 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fccf8005020 con 0x7fcd001029b0 2024-04-17T05:59:10.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:10.879+0000 7fccf7fff640 1 --2- 172.21.15.47:0/2641397535 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fccec0a3bb0 0x7fccec0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:10.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:10.880+0000 7fccfeffd640 1 --2- 172.21.15.47:0/2641397535 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fccec0a3bb0 0x7fccec0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:10.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:10.880+0000 7fccf7fff640 1 -- 172.21.15.47:0/2641397535 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fccf80d0580 con 0x7fcd001029b0 2024-04-17T05:59:10.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:10.880+0000 7fccfeffd640 1 --2- 172.21.15.47:0/2641397535 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fccec0a3bb0 0x7fccec0a6070 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fcce8005d00 tx=0x7fcce800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:59:10.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:10.884+0000 7fccf7fff640 1 -- 172.21.15.47:0/2641397535 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fccf809d400 con 0x7fcd001029b0 2024-04-17T05:59:11.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.112+0000 7fcd05635640 1 -- 172.21.15.47:0/2641397535 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcd0002d7f0 con 0x7fcd001029b0 2024-04-17T05:59:11.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.113+0000 7fccf7fff640 1 -- 172.21.15.47:0/2641397535 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10476 (secure 0 0 0) 0x7fccf809cd00 con 0x7fcd001029b0 2024-04-17T05:59:11.112 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-17T05:59:11.113 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3189958720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3189958720},{"type":"v1","addr":"172.21.15.47:6835","nonce":3189958720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14595,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/3143529973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":3143529973},{"type":"v1","addr":"172.21.15.47:6837","nonce":3143529973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14652,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/2375823701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":2375823701},{"type":"v1","addr":"172.21.15.47:6839","nonce":2375823701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14664,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6841/3334309157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":3334309157},{"type":"v1","addr":"172.21.15.47:6841","nonce":3334309157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/2398363754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":2398363754},{"type":"v1","addr":"172.21.15.103:6833","nonce":2398363754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/384473493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":384473493},{"type":"v1","addr":"172.21.15.98:6833","nonce":384473493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/270543522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":270543522},{"type":"v1","addr":"172.21.15.103:6835","nonce":270543522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24448,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/116936024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":116936024},{"type":"v1","addr":"172.21.15.103:6837","nonce":116936024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24449,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/1966500278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":1966500278},{"type":"v1","addr":"172.21.15.98:6835","nonce":1966500278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24466,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6839/1467157889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":1467157889},{"type":"v1","addr":"172.21.15.103:6839","nonce":1467157889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24479,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/1117440719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1117440719},{"type":"v1","addr":"172.21.15.98:6837","nonce":1117440719}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24518,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6839/2841287917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":2841287917},{"type":"v1","addr":"172.21.15.98:6839","nonce":2841287917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-04-17T05:59:11.113 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 13 2024-04-17T05:59:11.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.115+0000 7fcd05635640 1 -- 172.21.15.47:0/2641397535 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fccec0a3bb0 msgr2=0x7fccec0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:11.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.116+0000 7fcd05635640 1 --2- 172.21.15.47:0/2641397535 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fccec0a3bb0 0x7fccec0a6070 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fcce8005d00 tx=0x7fcce800b410 comp rx=0 tx=0).stop 2024-04-17T05:59:11.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.116+0000 7fcd05635640 1 -- 172.21.15.47:0/2641397535 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcd001029b0 msgr2=0x7fcd0019f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:11.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.116+0000 7fcd05635640 1 --2- 172.21.15.47:0/2641397535 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcd001029b0 0x7fcd0019f660 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fccf800eb10 tx=0x7fccf800c2d0 comp rx=0 tx=0).stop 2024-04-17T05:59:11.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.116+0000 7fcd05635640 1 -- 172.21.15.47:0/2641397535 shutdown_connections 2024-04-17T05:59:11.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.116+0000 7fcd05635640 1 --2- 172.21.15.47:0/2641397535 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fccec0a3bb0 0x7fccec0a6070 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:11.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.116+0000 7fcd05635640 1 --2- 172.21.15.47:0/2641397535 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcd001029b0 0x7fcd0019f660 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:11.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.116+0000 7fcd05635640 1 --2- 172.21.15.47:0/2641397535 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcd00101ff0 0x7fcd0019cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:11.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.116+0000 7fcd05635640 1 --2- 172.21.15.47:0/2641397535 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcd00100ea0 0x7fcd00069cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:11.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.116+0000 7fcd05635640 1 -- 172.21.15.47:0/2641397535 >> 172.21.15.47:0/2641397535 conn(0x7fcd000fc410 msgr2=0x7fcd00105380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:59:11.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.116+0000 7fcd05635640 1 -- 172.21.15.47:0/2641397535 shutdown_connections 2024-04-17T05:59:11.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.116+0000 7fcd05635640 1 -- 172.21.15.47:0/2641397535 wait complete. 2024-04-17T05:59:11.126 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T05:59:11.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.250+0000 7f0917e4a640 1 -- 172.21.15.47:0/48261180 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0910101ff0 msgr2=0x7f0910102470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:11.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.250+0000 7f0917e4a640 1 --2- 172.21.15.47:0/48261180 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0910101ff0 0x7f0910102470 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0904009900 tx=0x7f090402bc30 comp rx=0 tx=0).stop 2024-04-17T05:59:11.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.250+0000 7f0917e4a640 1 -- 172.21.15.47:0/48261180 shutdown_connections 2024-04-17T05:59:11.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.250+0000 7f0917e4a640 1 --2- 172.21.15.47:0/48261180 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f09101029b0 0x7f09101092d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:11.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.250+0000 7f0917e4a640 1 --2- 172.21.15.47:0/48261180 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0910101ff0 0x7f0910102470 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:11.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.250+0000 7f0917e4a640 1 --2- 172.21.15.47:0/48261180 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0910100ea0 0x7f09101012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:11.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.250+0000 7f0917e4a640 1 -- 172.21.15.47:0/48261180 >> 172.21.15.47:0/48261180 conn(0x7f09100fc410 msgr2=0x7f09100fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:59:11.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.250+0000 7f0917e4a640 1 -- 172.21.15.47:0/48261180 shutdown_connections 2024-04-17T05:59:11.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.250+0000 7f0917e4a640 1 -- 172.21.15.47:0/48261180 wait complete. 2024-04-17T05:59:11.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.250+0000 7f0917e4a640 1 Processor -- start 2024-04-17T05:59:11.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.250+0000 7f0917e4a640 1 -- start start 2024-04-17T05:59:11.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.251+0000 7f0917e4a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0910100ea0 0x7f09101982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:11.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.251+0000 7f0917e4a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0910101ff0 0x7f0910198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:11.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.251+0000 7f0917e4a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f09101029b0 0x7f091019f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:11.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.251+0000 7f0917e4a640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f091010dc90 con 0x7f0910100ea0 2024-04-17T05:59:11.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.251+0000 7f0917e4a640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f091010db10 con 0x7f0910101ff0 2024-04-17T05:59:11.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.251+0000 7f0917e4a640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f091010de10 con 0x7f09101029b0 2024-04-17T05:59:11.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.251+0000 7f0915be6640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0910100ea0 0x7f09101982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:11.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.251+0000 7f0915be6640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0910100ea0 0x7f09101982d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41662/0 (socket says 172.21.15.47:41662) 2024-04-17T05:59:11.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.251+0000 7f0915be6640 1 -- 172.21.15.47:0/1900389858 learned_addr learned my addr 172.21.15.47:0/1900389858 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:59:11.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.251+0000 7f09163e7640 1 --2- 172.21.15.47:0/1900389858 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f09101029b0 0x7f091019f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:11.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.251+0000 7f09153e5640 1 --2- 172.21.15.47:0/1900389858 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0910101ff0 0x7f0910198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:11.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.251+0000 7f0915be6640 1 -- 172.21.15.47:0/1900389858 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f09101029b0 msgr2=0x7f091019f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:11.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.251+0000 7f0915be6640 1 --2- 172.21.15.47:0/1900389858 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f09101029b0 0x7f091019f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:11.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.251+0000 7f0915be6640 1 -- 172.21.15.47:0/1900389858 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0910101ff0 msgr2=0x7f0910198810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:11.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.251+0000 7f0915be6640 1 --2- 172.21.15.47:0/1900389858 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0910101ff0 0x7f0910198810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:11.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.251+0000 7f0915be6640 1 -- 172.21.15.47:0/1900389858 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f091019fef0 con 0x7f0910100ea0 2024-04-17T05:59:11.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.251+0000 7f09153e5640 1 --2- 172.21.15.47:0/1900389858 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0910101ff0 0x7f0910198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T05:59:11.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.251+0000 7f0915be6640 1 --2- 172.21.15.47:0/1900389858 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0910100ea0 0x7f09101982d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f090c00d950 tx=0x7f090c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:59:11.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.251+0000 7f0902ffd640 1 -- 172.21.15.47:0/1900389858 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f090c017070 con 0x7f0910100ea0 2024-04-17T05:59:11.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.251+0000 7f0902ffd640 1 -- 172.21.15.47:0/1900389858 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f090c009c30 con 0x7f0910100ea0 2024-04-17T05:59:11.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.251+0000 7f0917e4a640 1 -- 172.21.15.47:0/1900389858 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09101a01e0 con 0x7f0910100ea0 2024-04-17T05:59:11.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.251+0000 7f0902ffd640 1 -- 172.21.15.47:0/1900389858 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f090c00f6b0 con 0x7f0910100ea0 2024-04-17T05:59:11.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.251+0000 7f0917e4a640 1 -- 172.21.15.47:0/1900389858 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09101a0780 con 0x7f0910100ea0 2024-04-17T05:59:11.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.252+0000 7f0917e4a640 1 -- 172.21.15.47:0/1900389858 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0910108230 con 0x7f0910100ea0 2024-04-17T05:59:11.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.253+0000 7f0902ffd640 1 -- 172.21.15.47:0/1900389858 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f090c0109d0 con 0x7f0910100ea0 2024-04-17T05:59:11.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.254+0000 7f0902ffd640 1 --2- 172.21.15.47:0/1900389858 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f08fc0a3c30 0x7f08fc0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:11.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.254+0000 7f09153e5640 1 --2- 172.21.15.47:0/1900389858 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f08fc0a3c30 0x7f08fc0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:11.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.254+0000 7f0902ffd640 1 -- 172.21.15.47:0/1900389858 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f090c0cfbb0 con 0x7f0910100ea0 2024-04-17T05:59:11.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.255+0000 7f09153e5640 1 --2- 172.21.15.47:0/1900389858 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f08fc0a3c30 0x7f08fc0a60f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0904006010 tx=0x7f0904000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:59:11.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.259+0000 7f0902ffd640 1 -- 172.21.15.47:0/1900389858 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f090c009de0 con 0x7f0910100ea0 2024-04-17T05:59:11.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.456+0000 7f0917e4a640 1 -- 172.21.15.47:0/1900389858 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0910062b90 con 0x7f0910100ea0 2024-04-17T05:59:11.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.457+0000 7f0902ffd640 1 -- 172.21.15.47:0/1900389858 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18406 (secure 0 0 0) 0x7f090c09ca30 con 0x7f0910100ea0 2024-04-17T05:59:11.456 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-17T05:59:11.456 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":82,"fsid":"565db2d8-fc7e-11ee-bc91-c7b262605968","created":"2024-04-17T05:51:19.888865+0000","modified":"2024-04-17T05:58:25.403856+0000","last_up_change":"2024-04-17T05:58:24.380131+0000","last_in_change":"2024-04-17T05:58:04.418236+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T05:54:37.734306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8453f300-533d-4736-a343-55619c08e966","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":1290127544},{"type":"v1","addr":"172.21.15.47:6803","nonce":1290127544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":1290127544},{"type":"v1","addr":"172.21.15.47:6805","nonce":1290127544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":1290127544},{"type":"v1","addr":"172.21.15.47:6809","nonce":1290127544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":1290127544},{"type":"v1","addr":"172.21.15.47:6807","nonce":1290127544}]},"public_addr":"172.21.15.47:6803/1290127544","cluster_addr":"172.21.15.47:6805/1290127544","heartbeat_back_addr":"172.21.15.47:6809/1290127544","heartbeat_front_addr":"172.21.15.47:6807/1290127544","state":["exists","up"]},{"osd":1,"uuid":"80b0f03e-c1bc-4658-8028-1e6b8eedcc47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":1180430954},{"type":"v1","addr":"172.21.15.98:6801","nonce":1180430954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":1180430954},{"type":"v1","addr":"172.21.15.98:6803","nonce":1180430954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":1180430954},{"type":"v1","addr":"172.21.15.98:6807","nonce":1180430954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":1180430954},{"type":"v1","addr":"172.21.15.98:6805","nonce":1180430954}]},"public_addr":"172.21.15.98:6801/1180430954","cluster_addr":"172.21.15.98:6803/1180430954","heartbeat_back_addr":"172.21.15.98:6807/1180430954","heartbeat_front_addr":"172.21.15.98:6805/1180430954","state":["exists","up"]},{"osd":2,"uuid":"0fc9d383-05b4-498d-b530-fccf7aeddd7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":494676216},{"type":"v1","addr":"172.21.15.103:6801","nonce":494676216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":494676216},{"type":"v1","addr":"172.21.15.103:6803","nonce":494676216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":494676216},{"type":"v1","addr":"172.21.15.103:6807","nonce":494676216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":494676216},{"type":"v1","addr":"172.21.15.103:6805","nonce":494676216}]},"public_addr":"172.21.15.103:6801/494676216","cluster_addr":"172.21.15.103:6803/494676216","heartbeat_back_addr":"172.21.15.103:6807/494676216","heartbeat_front_addr":"172.21.15.103:6805/494676216","state":["exists","up"]},{"osd":3,"uuid":"f6e0b0d5-0fb3-42d0-9e84-f7cfef21a998","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":1998369447},{"type":"v1","addr":"172.21.15.47:6811","nonce":1998369447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":1998369447},{"type":"v1","addr":"172.21.15.47:6813","nonce":1998369447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":1998369447},{"type":"v1","addr":"172.21.15.47:6817","nonce":1998369447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":1998369447},{"type":"v1","addr":"172.21.15.47:6815","nonce":1998369447}]},"public_addr":"172.21.15.47:6811/1998369447","cluster_addr":"172.21.15.47:6813/1998369447","heartbeat_back_addr":"172.21.15.47:6817/1998369447","heartbeat_front_addr":"172.21.15.47:6815/1998369447","state":["exists","up"]},{"osd":4,"uuid":"2c26217e-02df-466d-bc01-0149d4aec3c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":3060482485},{"type":"v1","addr":"172.21.15.98:6809","nonce":3060482485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":3060482485},{"type":"v1","addr":"172.21.15.98:6811","nonce":3060482485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":3060482485},{"type":"v1","addr":"172.21.15.98:6815","nonce":3060482485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":3060482485},{"type":"v1","addr":"172.21.15.98:6813","nonce":3060482485}]},"public_addr":"172.21.15.98:6809/3060482485","cluster_addr":"172.21.15.98:6811/3060482485","heartbeat_back_addr":"172.21.15.98:6815/3060482485","heartbeat_front_addr":"172.21.15.98:6813/3060482485","state":["exists","up"]},{"osd":5,"uuid":"740bf3e0-bea8-496b-8d6f-e42d0b017258","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":4181363567},{"type":"v1","addr":"172.21.15.103:6809","nonce":4181363567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":4181363567},{"type":"v1","addr":"172.21.15.103:6811","nonce":4181363567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":4181363567},{"type":"v1","addr":"172.21.15.103:6815","nonce":4181363567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":4181363567},{"type":"v1","addr":"172.21.15.103:6813","nonce":4181363567}]},"public_addr":"172.21.15.103:6809/4181363567","cluster_addr":"172.21.15.103:6811/4181363567","heartbeat_back_addr":"172.21.15.103:6815/4181363567","heartbeat_front_addr":"172.21.15.103:6813/4181363567","state":["exists","up"]},{"osd":6,"uuid":"356a6255-2c07-4f0d-a233-26be04c92b38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":2517529668},{"type":"v1","addr":"172.21.15.47:6819","nonce":2517529668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2517529668},{"type":"v1","addr":"172.21.15.47:6821","nonce":2517529668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2517529668},{"type":"v1","addr":"172.21.15.47:6825","nonce":2517529668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2517529668},{"type":"v1","addr":"172.21.15.47:6823","nonce":2517529668}]},"public_addr":"172.21.15.47:6819/2517529668","cluster_addr":"172.21.15.47:6821/2517529668","heartbeat_back_addr":"172.21.15.47:6825/2517529668","heartbeat_front_addr":"172.21.15.47:6823/2517529668","state":["exists","up"]},{"osd":7,"uuid":"349d0120-8290-417d-a800-5a8985f554f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":4225498496},{"type":"v1","addr":"172.21.15.98:6817","nonce":4225498496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":4225498496},{"type":"v1","addr":"172.21.15.98:6819","nonce":4225498496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":4225498496},{"type":"v1","addr":"172.21.15.98:6823","nonce":4225498496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":4225498496},{"type":"v1","addr":"172.21.15.98:6821","nonce":4225498496}]},"public_addr":"172.21.15.98:6817/4225498496","cluster_addr":"172.21.15.98:6819/4225498496","heartbeat_back_addr":"172.21.15.98:6823/4225498496","heartbeat_front_addr":"172.21.15.98:6821/4225498496","state":["exists","up"]},{"osd":8,"uuid":"cee4712f-321b-42c0-92ba-e15144c00f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":3433905520},{"type":"v1","addr":"172.21.15.103:6817","nonce":3433905520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":3433905520},{"type":"v1","addr":"172.21.15.103:6819","nonce":3433905520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":3433905520},{"type":"v1","addr":"172.21.15.103:6823","nonce":3433905520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":3433905520},{"type":"v1","addr":"172.21.15.103:6821","nonce":3433905520}]},"public_addr":"172.21.15.103:6817/3433905520","cluster_addr":"172.21.15.103:6819/3433905520","heartbeat_back_addr":"172.21.15.103:6823/3433905520","heartbeat_front_addr":"172.21.15.103:6821/3433905520","state":["exists","up"]},{"osd":9,"uuid":"63fc73c1-5a22-487d-80b8-9894e2ddd207","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":728359554},{"type":"v1","addr":"172.21.15.47:6827","nonce":728359554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":728359554},{"type":"v1","addr":"172.21.15.47:6829","nonce":728359554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":728359554},{"type":"v1","addr":"172.21.15.47:6833","nonce":728359554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":728359554},{"type":"v1","addr":"172.21.15.47:6831","nonce":728359554}]},"public_addr":"172.21.15.47:6827/728359554","cluster_addr":"172.21.15.47:6829/728359554","heartbeat_back_addr":"172.21.15.47:6833/728359554","heartbeat_front_addr":"172.21.15.47:6831/728359554","state":["exists","up"]},{"osd":10,"uuid":"9565bcc3-62ee-48eb-a07a-03a81e0cc8e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":468205689},{"type":"v1","addr":"172.21.15.98:6825","nonce":468205689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":468205689},{"type":"v1","addr":"172.21.15.98:6827","nonce":468205689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":468205689},{"type":"v1","addr":"172.21.15.98:6831","nonce":468205689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":468205689},{"type":"v1","addr":"172.21.15.98:6829","nonce":468205689}]},"public_addr":"172.21.15.98:6825/468205689","cluster_addr":"172.21.15.98:6827/468205689","heartbeat_back_addr":"172.21.15.98:6831/468205689","heartbeat_front_addr":"172.21.15.98:6829/468205689","state":["exists","up"]},{"osd":11,"uuid":"2e85e8d9-bf6c-4a12-b124-a4690dd61108","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":2639831013},{"type":"v1","addr":"172.21.15.103:6825","nonce":2639831013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":2639831013},{"type":"v1","addr":"172.21.15.103:6827","nonce":2639831013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":2639831013},{"type":"v1","addr":"172.21.15.103:6831","nonce":2639831013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":2639831013},{"type":"v1","addr":"172.21.15.103:6829","nonce":2639831013}]},"public_addr":"172.21.15.103:6825/2639831013","cluster_addr":"172.21.15.103:6827/2639831013","heartbeat_back_addr":"172.21.15.103:6831/2639831013","heartbeat_front_addr":"172.21.15.103:6829/2639831013","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:53:47.366438+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:54:10.953559+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:54:34.320332+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:54:59.751360+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:55:23.982052+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:55:48.425017+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:56:14.793368+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:56:40.440830+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:57:04.623680+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:57:31.215262+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:57:57.040346+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:58:22.671572+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:6801/3632157175":"2024-04-18T05:52:13.645775+0000","172.21.15.47:0/4288219269":"2024-04-18T05:52:13.645775+0000","172.21.15.47:0/2584487544":"2024-04-18T05:52:13.645775+0000","172.21.15.47:6801/953064786":"2024-04-18T05:51:45.979351+0000","172.21.15.47:0/3484500927":"2024-04-18T05:51:45.979351+0000","172.21.15.47:6800/3632157175":"2024-04-18T05:52:13.645775+0000","172.21.15.47:0/112268670":"2024-04-18T05:51:45.979351+0000","172.21.15.47:0/1638423256":"2024-04-18T05:51:45.979351+0000","172.21.15.47:6800/953064786":"2024-04-18T05:51:45.979351+0000","172.21.15.47:0/849479010":"2024-04-18T05:52:13.645775+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T05:59:11.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.459+0000 7f0917e4a640 1 -- 172.21.15.47:0/1900389858 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f08fc0a3c30 msgr2=0x7f08fc0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:11.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.459+0000 7f0917e4a640 1 --2- 172.21.15.47:0/1900389858 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f08fc0a3c30 0x7f08fc0a60f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0904006010 tx=0x7f0904000bc0 comp rx=0 tx=0).stop 2024-04-17T05:59:11.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.459+0000 7f0917e4a640 1 -- 172.21.15.47:0/1900389858 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0910100ea0 msgr2=0x7f09101982d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:11.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.459+0000 7f0917e4a640 1 --2- 172.21.15.47:0/1900389858 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0910100ea0 0x7f09101982d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f090c00d950 tx=0x7f090c00de10 comp rx=0 tx=0).stop 2024-04-17T05:59:11.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.459+0000 7f0917e4a640 1 -- 172.21.15.47:0/1900389858 shutdown_connections 2024-04-17T05:59:11.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.460+0000 7f0917e4a640 1 --2- 172.21.15.47:0/1900389858 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f08fc0a3c30 0x7f08fc0a60f0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:11.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.460+0000 7f0917e4a640 1 --2- 172.21.15.47:0/1900389858 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f09101029b0 0x7f091019f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:11.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.460+0000 7f0917e4a640 1 --2- 172.21.15.47:0/1900389858 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0910101ff0 0x7f0910198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:11.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.460+0000 7f0917e4a640 1 --2- 172.21.15.47:0/1900389858 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0910100ea0 0x7f09101982d0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:11.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.460+0000 7f0917e4a640 1 -- 172.21.15.47:0/1900389858 >> 172.21.15.47:0/1900389858 conn(0x7f09100fc410 msgr2=0x7f0910105400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:59:11.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.460+0000 7f0917e4a640 1 -- 172.21.15.47:0/1900389858 shutdown_connections 2024-04-17T05:59:11.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.460+0000 7f0917e4a640 1 -- 172.21.15.47:0/1900389858 wait complete. 2024-04-17T05:59:11.471 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T05:54:37.734306+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T05:59:11.471 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T05:59:11.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.583+0000 7fc778e7b640 1 -- 172.21.15.47:0/2442878881 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc774107c80 msgr2=0x7fc77410a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:11.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.583+0000 7fc778e7b640 1 --2- 172.21.15.47:0/2442878881 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc774107c80 0x7fc77410a0d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc764009ca0 tx=0x7fc76402c0f0 comp rx=0 tx=0).stop 2024-04-17T05:59:11.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.583+0000 7fc778e7b640 1 -- 172.21.15.47:0/2442878881 shutdown_connections 2024-04-17T05:59:11.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.583+0000 7fc778e7b640 1 --2- 172.21.15.47:0/2442878881 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc774107c80 0x7fc77410a0d0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:11.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.583+0000 7fc778e7b640 1 --2- 172.21.15.47:0/2442878881 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc774105350 0x7fc774107740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:11.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.583+0000 7fc778e7b640 1 --2- 172.21.15.47:0/2442878881 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc774102a20 0x7fc774104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:11.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.583+0000 7fc778e7b640 1 -- 172.21.15.47:0/2442878881 >> 172.21.15.47:0/2442878881 conn(0x7fc7740fc410 msgr2=0x7fc7740fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:59:11.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.583+0000 7fc778e7b640 1 -- 172.21.15.47:0/2442878881 shutdown_connections 2024-04-17T05:59:11.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.583+0000 7fc778e7b640 1 -- 172.21.15.47:0/2442878881 wait complete. 2024-04-17T05:59:11.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.584+0000 7fc778e7b640 1 Processor -- start 2024-04-17T05:59:11.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.584+0000 7fc778e7b640 1 -- start start 2024-04-17T05:59:11.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.584+0000 7fc778e7b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc774102a20 0x7fc7741982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:11.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.584+0000 7fc778e7b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc774105350 0x7fc774198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:11.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.584+0000 7fc778e7b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc774107c80 0x7fc77419f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:11.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.584+0000 7fc778e7b640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fc77410dc30 con 0x7fc774107c80 2024-04-17T05:59:11.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.584+0000 7fc778e7b640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc77410dab0 con 0x7fc774102a20 2024-04-17T05:59:11.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.584+0000 7fc778e7b640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc77410ddb0 con 0x7fc774105350 2024-04-17T05:59:11.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.584+0000 7fc772d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc774107c80 0x7fc77419f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:11.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.584+0000 7fc772d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc774107c80 0x7fc77419f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41676/0 (socket says 172.21.15.47:41676) 2024-04-17T05:59:11.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.584+0000 7fc772d9d640 1 -- 172.21.15.47:0/793912751 learned_addr learned my addr 172.21.15.47:0/793912751 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:59:11.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.584+0000 7fc771d9b640 1 --2- 172.21.15.47:0/793912751 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc774105350 0x7fc774198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:11.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.584+0000 7fc77259c640 1 --2- 172.21.15.47:0/793912751 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc774102a20 0x7fc7741982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:11.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.584+0000 7fc772d9d640 1 -- 172.21.15.47:0/793912751 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc774105350 msgr2=0x7fc774198810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:11.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.585+0000 7fc772d9d640 1 --2- 172.21.15.47:0/793912751 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc774105350 0x7fc774198810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:11.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.585+0000 7fc772d9d640 1 -- 172.21.15.47:0/793912751 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc774102a20 msgr2=0x7fc7741982d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:11.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.585+0000 7fc772d9d640 1 --2- 172.21.15.47:0/793912751 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc774102a20 0x7fc7741982d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:11.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.585+0000 7fc772d9d640 1 -- 172.21.15.47:0/793912751 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc77419fef0 con 0x7fc774107c80 2024-04-17T05:59:11.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.585+0000 7fc772d9d640 1 --2- 172.21.15.47:0/793912751 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc774107c80 0x7fc77419f880 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc764037040 tx=0x7fc764032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:59:11.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.585+0000 7fc76b7fe640 1 -- 172.21.15.47:0/793912751 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc764041020 con 0x7fc774107c80 2024-04-17T05:59:11.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.585+0000 7fc76b7fe640 1 -- 172.21.15.47:0/793912751 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc764034d90 con 0x7fc774107c80 2024-04-17T05:59:11.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.585+0000 7fc778e7b640 1 -- 172.21.15.47:0/793912751 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7741a0180 con 0x7fc774107c80 2024-04-17T05:59:11.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.585+0000 7fc76b7fe640 1 -- 172.21.15.47:0/793912751 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc764039040 con 0x7fc774107c80 2024-04-17T05:59:11.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.585+0000 7fc778e7b640 1 -- 172.21.15.47:0/793912751 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7741a0670 con 0x7fc774107c80 2024-04-17T05:59:11.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.586+0000 7fc778e7b640 1 -- 172.21.15.47:0/793912751 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7740fdf00 con 0x7fc774107c80 2024-04-17T05:59:11.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.587+0000 7fc76b7fe640 1 -- 172.21.15.47:0/793912751 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fc764044020 con 0x7fc774107c80 2024-04-17T05:59:11.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.588+0000 7fc76b7fe640 1 --2- 172.21.15.47:0/793912751 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fc7600a3bb0 0x7fc7600a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:11.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.588+0000 7fc77259c640 1 --2- 172.21.15.47:0/793912751 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fc7600a3bb0 0x7fc7600a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:11.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.588+0000 7fc76b7fe640 1 -- 172.21.15.47:0/793912751 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fc7640f2260 con 0x7fc774107c80 2024-04-17T05:59:11.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.588+0000 7fc77259c640 1 --2- 172.21.15.47:0/793912751 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fc7600a3bb0 0x7fc7600a6070 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc76c005d00 tx=0x7fc76c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:59:11.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.596+0000 7fc76b7fe640 1 -- 172.21.15.47:0/793912751 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fc76404b020 con 0x7fc774107c80 2024-04-17T05:59:11.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.782+0000 7fc778e7b640 1 -- 172.21.15.47:0/793912751 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc77402d5d0 con 0x7fc774107c80 2024-04-17T05:59:11.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.782+0000 7fc76b7fe640 1 -- 172.21.15.47:0/793912751 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7fc7640bf0e0 con 0x7fc774107c80 2024-04-17T05:59:11.781 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-04-17T05:59:11.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.784+0000 7fc778e7b640 1 -- 172.21.15.47:0/793912751 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fc7600a3bb0 msgr2=0x7fc7600a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:11.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.784+0000 7fc778e7b640 1 --2- 172.21.15.47:0/793912751 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fc7600a3bb0 0x7fc7600a6070 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc76c005d00 tx=0x7fc76c00b410 comp rx=0 tx=0).stop 2024-04-17T05:59:11.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.784+0000 7fc778e7b640 1 -- 172.21.15.47:0/793912751 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc774107c80 msgr2=0x7fc77419f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:11.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.784+0000 7fc778e7b640 1 --2- 172.21.15.47:0/793912751 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc774107c80 0x7fc77419f880 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc764037040 tx=0x7fc764032f20 comp rx=0 tx=0).stop 2024-04-17T05:59:11.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.784+0000 7fc778e7b640 1 -- 172.21.15.47:0/793912751 shutdown_connections 2024-04-17T05:59:11.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.784+0000 7fc778e7b640 1 --2- 172.21.15.47:0/793912751 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fc7600a3bb0 0x7fc7600a6070 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:11.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.784+0000 7fc778e7b640 1 --2- 172.21.15.47:0/793912751 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc774107c80 0x7fc77419f880 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:11.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.784+0000 7fc778e7b640 1 --2- 172.21.15.47:0/793912751 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc774105350 0x7fc774198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:11.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.784+0000 7fc778e7b640 1 --2- 172.21.15.47:0/793912751 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc774102a20 0x7fc7741982d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:11.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.784+0000 7fc778e7b640 1 -- 172.21.15.47:0/793912751 >> 172.21.15.47:0/793912751 conn(0x7fc7740fc410 msgr2=0x7fc774103480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:59:11.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.784+0000 7fc778e7b640 1 -- 172.21.15.47:0/793912751 shutdown_connections 2024-04-17T05:59:11.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.784+0000 7fc778e7b640 1 -- 172.21.15.47:0/793912751 wait complete. 2024-04-17T05:59:11.793 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:11 smithi047 ceph-mon[32223]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T05:59:11.793 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:11 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:59:11.793 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:11 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:59:11.794 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:11 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:59:11.794 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:11 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:59:11.794 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:11 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:59:11.794 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:11 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:59:11.794 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:11 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/2641397535' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T05:59:11.794 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:11 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:59:11.794 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:11 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T05:59:11.794 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:11 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:59:11.794 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:11 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T05:59:11.794 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:11 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/1900389858' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T05:59:11.795 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-17T05:59:11.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:11 smithi098 ceph-mon[36516]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T05:59:11.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:11 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:59:11.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:11 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:59:11.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:11 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:59:11.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:11 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:59:11.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:11 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:59:11.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:11 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:59:11.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:11 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/2641397535' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T05:59:11.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:11 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:59:11.884 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:11 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T05:59:11.884 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:11 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:59:11.884 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:11 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T05:59:11.884 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:11 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/1900389858' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T05:59:11.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.909+0000 7f182c7bd640 1 -- 172.21.15.47:0/2202072359 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1824101ff0 msgr2=0x7f1824102470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:11.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.909+0000 7f182c7bd640 1 --2- 172.21.15.47:0/2202072359 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1824101ff0 0x7f1824102470 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1814009900 tx=0x7f181402bc30 comp rx=0 tx=0).stop 2024-04-17T05:59:11.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.909+0000 7f182c7bd640 1 -- 172.21.15.47:0/2202072359 shutdown_connections 2024-04-17T05:59:11.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.909+0000 7f182c7bd640 1 --2- 172.21.15.47:0/2202072359 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f18241029b0 0x7f18241092d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:11.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.909+0000 7f182c7bd640 1 --2- 172.21.15.47:0/2202072359 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1824101ff0 0x7f1824102470 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:11.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.909+0000 7f182c7bd640 1 --2- 172.21.15.47:0/2202072359 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1824100ea0 0x7f18241012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:11.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.909+0000 7f182c7bd640 1 -- 172.21.15.47:0/2202072359 >> 172.21.15.47:0/2202072359 conn(0x7f18240fc410 msgr2=0x7f18240fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:59:11.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.909+0000 7f182c7bd640 1 -- 172.21.15.47:0/2202072359 shutdown_connections 2024-04-17T05:59:11.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.909+0000 7f182c7bd640 1 -- 172.21.15.47:0/2202072359 wait complete. 2024-04-17T05:59:11.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.910+0000 7f182c7bd640 1 Processor -- start 2024-04-17T05:59:11.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.910+0000 7f182c7bd640 1 -- start start 2024-04-17T05:59:11.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.910+0000 7f182c7bd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1824100ea0 0x7f1824104e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:11.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.910+0000 7f182c7bd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1824101ff0 0x7f1824103460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:11.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.910+0000 7f182c7bd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f18241029b0 0x7f18241039a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:11.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.910+0000 7f182c7bd640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f182410cb50 con 0x7f18241029b0 2024-04-17T05:59:11.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.910+0000 7f182c7bd640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f182410c9d0 con 0x7f1824100ea0 2024-04-17T05:59:11.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.910+0000 7f182c7bd640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f182410ccd0 con 0x7f1824101ff0 2024-04-17T05:59:11.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.910+0000 7f182ad5a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f18241029b0 0x7f18241039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:11.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.910+0000 7f182ad5a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f18241029b0 0x7f18241039a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41700/0 (socket says 172.21.15.47:41700) 2024-04-17T05:59:11.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.910+0000 7f182ad5a640 1 -- 172.21.15.47:0/2115039564 learned_addr learned my addr 172.21.15.47:0/2115039564 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:59:11.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.910+0000 7f182a559640 1 --2- 172.21.15.47:0/2115039564 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1824100ea0 0x7f1824104e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:11.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.910+0000 7f1829d58640 1 --2- 172.21.15.47:0/2115039564 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1824101ff0 0x7f1824103460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:11.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.911+0000 7f182ad5a640 1 -- 172.21.15.47:0/2115039564 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1824101ff0 msgr2=0x7f1824103460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:11.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.911+0000 7f182ad5a640 1 --2- 172.21.15.47:0/2115039564 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1824101ff0 0x7f1824103460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:11.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.911+0000 7f182ad5a640 1 -- 172.21.15.47:0/2115039564 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1824100ea0 msgr2=0x7f1824104e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:11.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.911+0000 7f182ad5a640 1 --2- 172.21.15.47:0/2115039564 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1824100ea0 0x7f1824104e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:11.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.911+0000 7f182ad5a640 1 -- 172.21.15.47:0/2115039564 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1824104230 con 0x7f18241029b0 2024-04-17T05:59:11.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.911+0000 7f1829d58640 1 --2- 172.21.15.47:0/2115039564 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1824101ff0 0x7f1824103460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T05:59:11.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.911+0000 7f182ad5a640 1 --2- 172.21.15.47:0/2115039564 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f18241029b0 0x7f18241039a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f181c00eb10 tx=0x7f181c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:59:11.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.911+0000 7f181b7fe640 1 -- 172.21.15.47:0/2115039564 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f181c004b20 con 0x7f18241029b0 2024-04-17T05:59:11.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.911+0000 7f181b7fe640 1 -- 172.21.15.47:0/2115039564 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f181c004cc0 con 0x7f18241029b0 2024-04-17T05:59:11.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.911+0000 7f182c7bd640 1 -- 172.21.15.47:0/2115039564 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18241a2430 con 0x7f18241029b0 2024-04-17T05:59:11.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.911+0000 7f181b7fe640 1 -- 172.21.15.47:0/2115039564 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f181c010760 con 0x7f18241029b0 2024-04-17T05:59:11.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.911+0000 7f182c7bd640 1 -- 172.21.15.47:0/2115039564 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18241a2900 con 0x7f18241029b0 2024-04-17T05:59:11.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.912+0000 7f182c7bd640 1 -- 172.21.15.47:0/2115039564 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1824068d80 con 0x7f18241029b0 2024-04-17T05:59:11.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.913+0000 7f181b7fe640 1 -- 172.21.15.47:0/2115039564 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f181c005020 con 0x7f18241029b0 2024-04-17T05:59:11.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.914+0000 7f181b7fe640 1 --2- 172.21.15.47:0/2115039564 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f18100a3bb0 0x7f18100a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:11.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.914+0000 7f181b7fe640 1 -- 172.21.15.47:0/2115039564 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f181c0d0530 con 0x7f18241029b0 2024-04-17T05:59:11.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.914+0000 7f182a559640 1 --2- 172.21.15.47:0/2115039564 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f18100a3bb0 0x7f18100a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:11.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.914+0000 7f182a559640 1 --2- 172.21.15.47:0/2115039564 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f18100a3bb0 0x7f18100a6070 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1820009bd0 tx=0x7f1820005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:59:11.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:11.919+0000 7f181b7fe640 1 -- 172.21.15.47:0/2115039564 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f181c09d3b0 con 0x7f18241029b0 2024-04-17T05:59:12.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:11 smithi103 ceph-mon[36243]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T05:59:12.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:11 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:59:12.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:11 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:59:12.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:11 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:59:12.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:11 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:59:12.035 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:11 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:59:12.035 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:11 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:59:12.035 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:11 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/2641397535' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T05:59:12.035 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:11 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:59:12.035 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:11 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T05:59:12.035 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:11 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:59:12.035 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:11 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T05:59:12.035 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:11 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/1900389858' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T05:59:12.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:12.126+0000 7f182c7bd640 1 -- 172.21.15.47:0/2115039564 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f18241a2de0 con 0x7f18241029b0 2024-04-17T05:59:12.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:12.127+0000 7f181b7fe640 1 -- 172.21.15.47:0/2115039564 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v82) ==== 84+0+65 (secure 0 0 0) 0x7f18241a2de0 con 0x7f18241029b0 2024-04-17T05:59:12.126 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-17T05:59:12.126 INFO:teuthology.orchestra.run.smithi047.stdout:[ 2024-04-17T05:59:12.126 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T05:59:12.126 INFO:teuthology.orchestra.run.smithi047.stdout: "poolnum": 1, 2024-04-17T05:59:12.126 INFO:teuthology.orchestra.run.smithi047.stdout: "poolname": ".mgr" 2024-04-17T05:59:12.126 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T05:59:12.127 INFO:teuthology.orchestra.run.smithi047.stdout:] 2024-04-17T05:59:12.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:12.129+0000 7f182c7bd640 1 -- 172.21.15.47:0/2115039564 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f18100a3bb0 msgr2=0x7f18100a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:12.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:12.129+0000 7f182c7bd640 1 --2- 172.21.15.47:0/2115039564 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f18100a3bb0 0x7f18100a6070 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1820009bd0 tx=0x7f1820005f90 comp rx=0 tx=0).stop 2024-04-17T05:59:12.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:12.129+0000 7f182c7bd640 1 -- 172.21.15.47:0/2115039564 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f18241029b0 msgr2=0x7f18241039a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:12.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:12.129+0000 7f182c7bd640 1 --2- 172.21.15.47:0/2115039564 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f18241029b0 0x7f18241039a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f181c00eb10 tx=0x7f181c00c2d0 comp rx=0 tx=0).stop 2024-04-17T05:59:12.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:12.129+0000 7f182c7bd640 1 -- 172.21.15.47:0/2115039564 shutdown_connections 2024-04-17T05:59:12.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:12.129+0000 7f182c7bd640 1 --2- 172.21.15.47:0/2115039564 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f18100a3bb0 0x7f18100a6070 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:12.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:12.129+0000 7f182c7bd640 1 --2- 172.21.15.47:0/2115039564 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f18241029b0 0x7f18241039a0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:12.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:12.129+0000 7f182c7bd640 1 --2- 172.21.15.47:0/2115039564 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1824101ff0 0x7f1824103460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:12.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:12.129+0000 7f182c7bd640 1 --2- 172.21.15.47:0/2115039564 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1824100ea0 0x7f1824104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:12.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:12.129+0000 7f182c7bd640 1 -- 172.21.15.47:0/2115039564 >> 172.21.15.47:0/2115039564 conn(0x7f18240fc410 msgr2=0x7f18240fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:59:12.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:12.129+0000 7f182c7bd640 1 -- 172.21.15.47:0/2115039564 shutdown_connections 2024-04-17T05:59:12.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:12.129+0000 7f182c7bd640 1 -- 172.21.15.47:0/2115039564 wait complete. 2024-04-17T05:59:12.139 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-17T05:59:12.139 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-17T05:59:12.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:12.252+0000 7fd74ecd3640 1 -- 172.21.15.47:0/3010555520 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd748103190 msgr2=0x7fd748103610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:12.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:12.252+0000 7fd74ecd3640 1 --2- 172.21.15.47:0/3010555520 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd748103190 0x7fd748103610 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd734009900 tx=0x7fd73402bc30 comp rx=0 tx=0).stop 2024-04-17T05:59:12.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:12.252+0000 7fd74ecd3640 1 -- 172.21.15.47:0/3010555520 shutdown_connections 2024-04-17T05:59:12.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:12.252+0000 7fd74ecd3640 1 --2- 172.21.15.47:0/3010555520 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd748103b50 0x7fd74810a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:12.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:12.252+0000 7fd74ecd3640 1 --2- 172.21.15.47:0/3010555520 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd748103190 0x7fd748103610 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:12.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:12.252+0000 7fd74ecd3640 1 --2- 172.21.15.47:0/3010555520 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd748101fa0 0x7fd7481023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:12.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:12.252+0000 7fd74ecd3640 1 -- 172.21.15.47:0/3010555520 >> 172.21.15.47:0/3010555520 conn(0x7fd7480fd510 msgr2=0x7fd7480ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:59:12.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:12.252+0000 7fd74ecd3640 1 -- 172.21.15.47:0/3010555520 shutdown_connections 2024-04-17T05:59:12.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:12.252+0000 7fd74ecd3640 1 -- 172.21.15.47:0/3010555520 wait complete. 2024-04-17T05:59:12.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:12.253+0000 7fd74ecd3640 1 Processor -- start 2024-04-17T05:59:12.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:12.253+0000 7fd74ecd3640 1 -- start start 2024-04-17T05:59:12.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:12.253+0000 7fd74ecd3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd748101fa0 0x7fd748198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:12.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:12.253+0000 7fd74ecd3640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd748103190 0x7fd7481987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:12.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:12.253+0000 7fd74ecd3640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd748103b50 0x7fd74819f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:12.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:12.253+0000 7fd74ecd3640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fd74810dc30 con 0x7fd748103190 2024-04-17T05:59:12.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:12.253+0000 7fd74ecd3640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd74810dab0 con 0x7fd748103b50 2024-04-17T05:59:12.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:12.253+0000 7fd74ecd3640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd74810ddb0 con 0x7fd748101fa0 2024-04-17T05:59:12.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:12.253+0000 7fd747fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd748103190 0x7fd7481987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:12.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:12.253+0000 7fd747fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd748103190 0x7fd7481987d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41728/0 (socket says 172.21.15.47:41728) 2024-04-17T05:59:12.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:12.253+0000 7fd747fff640 1 -- 172.21.15.47:0/3025417558 learned_addr learned my addr 172.21.15.47:0/3025417558 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:59:12.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:12.253+0000 7fd74d270640 1 --2- 172.21.15.47:0/3025417558 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd748103b50 0x7fd74819f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:12.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:12.253+0000 7fd74ca6f640 1 --2- 172.21.15.47:0/3025417558 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd748101fa0 0x7fd748198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:12.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:12.254+0000 7fd747fff640 1 -- 172.21.15.47:0/3025417558 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd748101fa0 msgr2=0x7fd748198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:12.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:12.254+0000 7fd747fff640 1 --2- 172.21.15.47:0/3025417558 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd748101fa0 0x7fd748198290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:12.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:12.254+0000 7fd747fff640 1 -- 172.21.15.47:0/3025417558 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd748103b50 msgr2=0x7fd74819f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:12.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:12.254+0000 7fd747fff640 1 --2- 172.21.15.47:0/3025417558 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd748103b50 0x7fd74819f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:12.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:12.254+0000 7fd747fff640 1 -- 172.21.15.47:0/3025417558 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd74819feb0 con 0x7fd748103190 2024-04-17T05:59:12.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:12.254+0000 7fd74ca6f640 1 --2- 172.21.15.47:0/3025417558 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd748101fa0 0x7fd748198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T05:59:12.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:12.254+0000 7fd747fff640 1 --2- 172.21.15.47:0/3025417558 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd748103190 0x7fd7481987d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd734009230 tx=0x7fd73402c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:59:12.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:12.254+0000 7fd745ffb640 1 -- 172.21.15.47:0/3025417558 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd734041070 con 0x7fd748103190 2024-04-17T05:59:12.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:12.254+0000 7fd745ffb640 1 -- 172.21.15.47:0/3025417558 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd734004450 con 0x7fd748103190 2024-04-17T05:59:12.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:12.254+0000 7fd745ffb640 1 -- 172.21.15.47:0/3025417558 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd734003e20 con 0x7fd748103190 2024-04-17T05:59:12.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:12.254+0000 7fd74ecd3640 1 -- 172.21.15.47:0/3025417558 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7481a0140 con 0x7fd748103190 2024-04-17T05:59:12.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:12.254+0000 7fd74ecd3640 1 -- 172.21.15.47:0/3025417558 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7481a0630 con 0x7fd748103190 2024-04-17T05:59:12.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:12.255+0000 7fd74ecd3640 1 -- 172.21.15.47:0/3025417558 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd748109390 con 0x7fd748103190 2024-04-17T05:59:12.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:12.257+0000 7fd745ffb640 1 -- 172.21.15.47:0/3025417558 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fd734044070 con 0x7fd748103190 2024-04-17T05:59:12.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:12.257+0000 7fd745ffb640 1 --2- 172.21.15.47:0/3025417558 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fd7380a3bb0 0x7fd7380a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:12.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:12.257+0000 7fd74ca6f640 1 --2- 172.21.15.47:0/3025417558 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fd7380a3bb0 0x7fd7380a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:12.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:12.257+0000 7fd745ffb640 1 -- 172.21.15.47:0/3025417558 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fd7340f1ab0 con 0x7fd748103190 2024-04-17T05:59:12.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:12.258+0000 7fd74ca6f640 1 --2- 172.21.15.47:0/3025417558 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fd7380a3bb0 0x7fd7380a6070 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd748102ff0 tx=0x7fd73c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:59:12.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:12.262+0000 7fd745ffb640 1 -- 172.21.15.47:0/3025417558 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fd7340be2e0 con 0x7fd748103190 2024-04-17T05:59:12.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:12.452+0000 7fd74ecd3640 1 -- 172.21.15.47:0/3025417558 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fd74802d5d0 con 0x7fd748103190 2024-04-17T05:59:12.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:12.540+0000 7fd745ffb640 1 -- 172.21.15.47:0/3025417558 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v83) ==== 138+0+0 (secure 0 0 0) 0x7fd7340c25e0 con 0x7fd748103190 2024-04-17T05:59:12.540 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_metadata' created 2024-04-17T05:59:12.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:12.544+0000 7fd74ecd3640 1 -- 172.21.15.47:0/3025417558 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fd7380a3bb0 msgr2=0x7fd7380a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:12.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:12.544+0000 7fd74ecd3640 1 --2- 172.21.15.47:0/3025417558 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fd7380a3bb0 0x7fd7380a6070 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd748102ff0 tx=0x7fd73c005f90 comp rx=0 tx=0).stop 2024-04-17T05:59:12.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:12.544+0000 7fd74ecd3640 1 -- 172.21.15.47:0/3025417558 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd748103190 msgr2=0x7fd7481987d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:12.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:12.544+0000 7fd74ecd3640 1 --2- 172.21.15.47:0/3025417558 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd748103190 0x7fd7481987d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd734009230 tx=0x7fd73402c650 comp rx=0 tx=0).stop 2024-04-17T05:59:12.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:12.544+0000 7fd74ecd3640 1 -- 172.21.15.47:0/3025417558 shutdown_connections 2024-04-17T05:59:12.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:12.544+0000 7fd74ecd3640 1 --2- 172.21.15.47:0/3025417558 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fd7380a3bb0 0x7fd7380a6070 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:12.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:12.544+0000 7fd74ecd3640 1 --2- 172.21.15.47:0/3025417558 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd748103b50 0x7fd74819f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:12.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:12.544+0000 7fd74ecd3640 1 --2- 172.21.15.47:0/3025417558 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd748103190 0x7fd7481987d0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:12.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:12.544+0000 7fd74ecd3640 1 --2- 172.21.15.47:0/3025417558 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd748101fa0 0x7fd748198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:12.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:12.544+0000 7fd74ecd3640 1 -- 172.21.15.47:0/3025417558 >> 172.21.15.47:0/3025417558 conn(0x7fd7480fd510 msgr2=0x7fd748106560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:59:12.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:12.544+0000 7fd74ecd3640 1 -- 172.21.15.47:0/3025417558 shutdown_connections 2024-04-17T05:59:12.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:12.544+0000 7fd74ecd3640 1 -- 172.21.15.47:0/3025417558 wait complete. 2024-04-17T05:59:12.557 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-17T05:59:12.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:12.676+0000 7fcbb5b15640 1 -- 172.21.15.47:0/339080049 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcbb0103b60 msgr2=0x7fcbb010a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:12.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:12.676+0000 7fcbb5b15640 1 --2- 172.21.15.47:0/339080049 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcbb0103b60 0x7fcbb010a3f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fcba8009ca0 tx=0x7fcba802c0f0 comp rx=0 tx=0).stop 2024-04-17T05:59:12.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:12.677+0000 7fcbb5b15640 1 -- 172.21.15.47:0/339080049 shutdown_connections 2024-04-17T05:59:12.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:12.677+0000 7fcbb5b15640 1 --2- 172.21.15.47:0/339080049 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcbb0103b60 0x7fcbb010a3f0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:12.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:12.677+0000 7fcbb5b15640 1 --2- 172.21.15.47:0/339080049 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcbb01031a0 0x7fcbb0103620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:12.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:12.677+0000 7fcbb5b15640 1 --2- 172.21.15.47:0/339080049 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcbb0101fb0 0x7fcbb01023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:12.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:12.677+0000 7fcbb5b15640 1 -- 172.21.15.47:0/339080049 >> 172.21.15.47:0/339080049 conn(0x7fcbb00fd540 msgr2=0x7fcbb00ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:59:12.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:12.677+0000 7fcbb5b15640 1 -- 172.21.15.47:0/339080049 shutdown_connections 2024-04-17T05:59:12.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:12.677+0000 7fcbb5b15640 1 -- 172.21.15.47:0/339080049 wait complete. 2024-04-17T05:59:12.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:12.677+0000 7fcbb5b15640 1 Processor -- start 2024-04-17T05:59:12.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:12.678+0000 7fcbb5b15640 1 -- start start 2024-04-17T05:59:12.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:12.678+0000 7fcbb5b15640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcbb0101fb0 0x7fcbb0198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:12.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:12.678+0000 7fcbb5b15640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcbb01031a0 0x7fcbb01987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:12.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:12.678+0000 7fcbb5b15640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcbb0103b60 0x7fcbb019f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:12.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:12.678+0000 7fcbb5b15640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb010dc40 con 0x7fcbb0103b60 2024-04-17T05:59:12.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:12.678+0000 7fcbb5b15640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb010dac0 con 0x7fcbb01031a0 2024-04-17T05:59:12.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:12.678+0000 7fcbb5b15640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb010ddc0 con 0x7fcbb0101fb0 2024-04-17T05:59:12.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:12.678+0000 7fcbaffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcbb0103b60 0x7fcbb019f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:12.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:12.678+0000 7fcbaffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcbb0103b60 0x7fcbb019f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41744/0 (socket says 172.21.15.47:41744) 2024-04-17T05:59:12.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:12.678+0000 7fcbaffff640 1 -- 172.21.15.47:0/302624954 learned_addr learned my addr 172.21.15.47:0/302624954 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:59:12.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:12.678+0000 7fcbaf7fe640 1 --2- 172.21.15.47:0/302624954 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcbb0101fb0 0x7fcbb0198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:12.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:12.678+0000 7fcbaeffd640 1 --2- 172.21.15.47:0/302624954 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcbb01031a0 0x7fcbb01987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:12.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:12.678+0000 7fcbaffff640 1 -- 172.21.15.47:0/302624954 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcbb0101fb0 msgr2=0x7fcbb0198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:12.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:12.678+0000 7fcbaffff640 1 --2- 172.21.15.47:0/302624954 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcbb0101fb0 0x7fcbb0198290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:12.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:12.678+0000 7fcbaffff640 1 -- 172.21.15.47:0/302624954 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcbb01031a0 msgr2=0x7fcbb01987d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:12.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:12.678+0000 7fcbaffff640 1 --2- 172.21.15.47:0/302624954 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcbb01031a0 0x7fcbb01987d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:12.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:12.678+0000 7fcbaffff640 1 -- 172.21.15.47:0/302624954 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbb019feb0 con 0x7fcbb0103b60 2024-04-17T05:59:12.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:12.678+0000 7fcbaeffd640 1 --2- 172.21.15.47:0/302624954 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcbb01031a0 0x7fcbb01987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T05:59:12.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:12.678+0000 7fcbaf7fe640 1 --2- 172.21.15.47:0/302624954 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcbb0101fb0 0x7fcbb0198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T05:59:12.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:12.678+0000 7fcbaffff640 1 --2- 172.21.15.47:0/302624954 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcbb0103b60 0x7fcbb019f840 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fcba8037040 tx=0x7fcba8032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:59:12.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:12.679+0000 7fcbacff9640 1 -- 172.21.15.47:0/302624954 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba8041020 con 0x7fcbb0103b60 2024-04-17T05:59:12.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:12.679+0000 7fcbacff9640 1 -- 172.21.15.47:0/302624954 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcba8034cd0 con 0x7fcbb0103b60 2024-04-17T05:59:12.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:12.679+0000 7fcbb5b15640 1 -- 172.21.15.47:0/302624954 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbb01a0140 con 0x7fcbb0103b60 2024-04-17T05:59:12.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:12.679+0000 7fcbacff9640 1 -- 172.21.15.47:0/302624954 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba8039040 con 0x7fcbb0103b60 2024-04-17T05:59:12.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:12.679+0000 7fcbb5b15640 1 -- 172.21.15.47:0/302624954 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbb01a0630 con 0x7fcbb0103b60 2024-04-17T05:59:12.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:12.680+0000 7fcbb5b15640 1 -- 172.21.15.47:0/302624954 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbb01092d0 con 0x7fcbb0103b60 2024-04-17T05:59:12.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:12.681+0000 7fcbacff9640 1 -- 172.21.15.47:0/302624954 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fcba8044020 con 0x7fcbb0103b60 2024-04-17T05:59:12.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:12.682+0000 7fcbacff9640 1 --2- 172.21.15.47:0/302624954 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fcb980a3bb0 0x7fcb980a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:12.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:12.682+0000 7fcbaf7fe640 1 --2- 172.21.15.47:0/302624954 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fcb980a3bb0 0x7fcb980a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:12.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:12.682+0000 7fcbacff9640 1 -- 172.21.15.47:0/302624954 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7497+0+0 (secure 0 0 0) 0x7fcba80f2320 con 0x7fcbb0103b60 2024-04-17T05:59:12.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:12.682+0000 7fcbaf7fe640 1 --2- 172.21.15.47:0/302624954 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fcb980a3bb0 0x7fcb980a6070 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fcbb0103000 tx=0x7fcba0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:59:12.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:12.687+0000 7fcbacff9640 1 -- 172.21.15.47:0/302624954 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fcba804b020 con 0x7fcbb0103b60 2024-04-17T05:59:12.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:12 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/793912751' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-17T05:59:12.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:12 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/2115039564' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-17T05:59:12.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:12 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/3025417558' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-17T05:59:12.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:12 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/793912751' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-17T05:59:12.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:12 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/2115039564' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-17T05:59:12.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:12 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/3025417558' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-17T05:59:12.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:12.895+0000 7fcbb5b15640 1 -- 172.21.15.47:0/302624954 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fcbb002d5d0 con 0x7fcbb0103b60 2024-04-17T05:59:13.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:12 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/793912751' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-17T05:59:13.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:12 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/2115039564' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-17T05:59:13.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:12 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/3025417558' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-17T05:59:13.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:13.543+0000 7fcbacff9640 1 -- 172.21.15.47:0/302624954 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v84) ==== 170+0+0 (secure 0 0 0) 0x7fcba80bf0c0 con 0x7fcbb0103b60 2024-04-17T05:59:13.542 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_data' created 2024-04-17T05:59:13.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:13.546+0000 7fcbb5b15640 1 -- 172.21.15.47:0/302624954 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fcb980a3bb0 msgr2=0x7fcb980a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:13.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:13.546+0000 7fcbb5b15640 1 --2- 172.21.15.47:0/302624954 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fcb980a3bb0 0x7fcb980a6070 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fcbb0103000 tx=0x7fcba0005f90 comp rx=0 tx=0).stop 2024-04-17T05:59:13.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:13.546+0000 7fcbb5b15640 1 -- 172.21.15.47:0/302624954 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcbb0103b60 msgr2=0x7fcbb019f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:13.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:13.546+0000 7fcbb5b15640 1 --2- 172.21.15.47:0/302624954 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcbb0103b60 0x7fcbb019f840 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fcba8037040 tx=0x7fcba8032f40 comp rx=0 tx=0).stop 2024-04-17T05:59:13.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:13.546+0000 7fcbb5b15640 1 -- 172.21.15.47:0/302624954 shutdown_connections 2024-04-17T05:59:13.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:13.546+0000 7fcbb5b15640 1 --2- 172.21.15.47:0/302624954 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fcb980a3bb0 0x7fcb980a6070 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:13.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:13.546+0000 7fcbb5b15640 1 --2- 172.21.15.47:0/302624954 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcbb0103b60 0x7fcbb019f840 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:13.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:13.546+0000 7fcbb5b15640 1 --2- 172.21.15.47:0/302624954 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcbb01031a0 0x7fcbb01987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:13.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:13.546+0000 7fcbb5b15640 1 --2- 172.21.15.47:0/302624954 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcbb0101fb0 0x7fcbb0198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:13.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:13.546+0000 7fcbb5b15640 1 -- 172.21.15.47:0/302624954 >> 172.21.15.47:0/302624954 conn(0x7fcbb00fd540 msgr2=0x7fcbb01064a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:59:13.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:13.548+0000 7fcbb5b15640 1 -- 172.21.15.47:0/302624954 shutdown_connections 2024-04-17T05:59:13.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:13.549+0000 7fcbb5b15640 1 -- 172.21.15.47:0/302624954 wait complete. 2024-04-17T05:59:13.568 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-17T05:59:13.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:13.682+0000 7f8b04e14640 1 -- 172.21.15.47:0/2281601337 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b00102a10 msgr2=0x7f8b00109330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:13.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:13.683+0000 7f8b04e14640 1 --2- 172.21.15.47:0/2281601337 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b00102a10 0x7f8b00109330 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8af8009ca0 tx=0x7f8af802c0f0 comp rx=0 tx=0).stop 2024-04-17T05:59:13.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:13.683+0000 7f8b04e14640 1 -- 172.21.15.47:0/2281601337 shutdown_connections 2024-04-17T05:59:13.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:13.683+0000 7f8b04e14640 1 --2- 172.21.15.47:0/2281601337 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b00102a10 0x7f8b00109330 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:13.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:13.683+0000 7f8b04e14640 1 --2- 172.21.15.47:0/2281601337 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8b00102050 0x7f8b001024d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:13.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:13.683+0000 7f8b04e14640 1 --2- 172.21.15.47:0/2281601337 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8b00100e70 0x7f8b00101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:13.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:13.683+0000 7f8b04e14640 1 -- 172.21.15.47:0/2281601337 >> 172.21.15.47:0/2281601337 conn(0x7f8b000fc450 msgr2=0x7f8b000fe870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:59:13.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:13.683+0000 7f8b04e14640 1 -- 172.21.15.47:0/2281601337 shutdown_connections 2024-04-17T05:59:13.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:13.683+0000 7f8b04e14640 1 -- 172.21.15.47:0/2281601337 wait complete. 2024-04-17T05:59:13.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:13.683+0000 7f8b04e14640 1 Processor -- start 2024-04-17T05:59:13.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:13.683+0000 7f8b04e14640 1 -- start start 2024-04-17T05:59:13.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:13.684+0000 7f8b04e14640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b00100e70 0x7f8b001982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:13.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:13.684+0000 7f8b04e14640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8b00102050 0x7f8b00198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:13.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:13.684+0000 7f8b04e14640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8b00102a10 0x7f8b0019f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:13.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:13.684+0000 7f8b04e14640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f8b0010dcd0 con 0x7f8b00100e70 2024-04-17T05:59:13.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:13.684+0000 7f8b04e14640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8b0010db50 con 0x7f8b00102a10 2024-04-17T05:59:13.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:13.684+0000 7f8b04e14640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8b0010de50 con 0x7f8b00102050 2024-04-17T05:59:13.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:13.684+0000 7f8afe59c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b00100e70 0x7f8b001982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:13.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:13.684+0000 7f8afe59c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b00100e70 0x7f8b001982e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41756/0 (socket says 172.21.15.47:41756) 2024-04-17T05:59:13.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:13.684+0000 7f8afe59c640 1 -- 172.21.15.47:0/3883617586 learned_addr learned my addr 172.21.15.47:0/3883617586 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:59:13.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:13.684+0000 7f8afed9d640 1 --2- 172.21.15.47:0/3883617586 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8b00102a10 0x7f8b0019f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:13.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:13.684+0000 7f8af7fff640 1 --2- 172.21.15.47:0/3883617586 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8b00102050 0x7f8b00198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:13.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:13.684+0000 7f8afe59c640 1 -- 172.21.15.47:0/3883617586 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8b00102050 msgr2=0x7f8b00198820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:13.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:13.684+0000 7f8afe59c640 1 --2- 172.21.15.47:0/3883617586 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8b00102050 0x7f8b00198820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:13.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:13.684+0000 7f8afe59c640 1 -- 172.21.15.47:0/3883617586 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8b00102a10 msgr2=0x7f8b0019f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:13.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:13.684+0000 7f8afe59c640 1 --2- 172.21.15.47:0/3883617586 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8b00102a10 0x7f8b0019f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:13.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:13.684+0000 7f8afe59c640 1 -- 172.21.15.47:0/3883617586 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b0019ff00 con 0x7f8b00100e70 2024-04-17T05:59:13.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:13.684+0000 7f8afed9d640 1 --2- 172.21.15.47:0/3883617586 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8b00102a10 0x7f8b0019f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T05:59:13.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:13.684+0000 7f8afe59c640 1 --2- 172.21.15.47:0/3883617586 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b00100e70 0x7f8b001982e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8af000e9c0 tx=0x7f8af000edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:59:13.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:13.684+0000 7f8af77fe640 1 -- 172.21.15.47:0/3883617586 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8af000ccc0 con 0x7f8b00100e70 2024-04-17T05:59:13.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:13.684+0000 7f8b04e14640 1 -- 172.21.15.47:0/3883617586 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b001a01f0 con 0x7f8b00100e70 2024-04-17T05:59:13.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:13.684+0000 7f8af77fe640 1 -- 172.21.15.47:0/3883617586 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8af00035f0 con 0x7f8b00100e70 2024-04-17T05:59:13.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:13.684+0000 7f8af77fe640 1 -- 172.21.15.47:0/3883617586 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8af000ce60 con 0x7f8b00100e70 2024-04-17T05:59:13.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:13.684+0000 7f8b04e14640 1 -- 172.21.15.47:0/3883617586 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b001a0740 con 0x7f8b00100e70 2024-04-17T05:59:13.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:13.686+0000 7f8b04e14640 1 -- 172.21.15.47:0/3883617586 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b001081b0 con 0x7f8b00100e70 2024-04-17T05:59:13.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:13.687+0000 7f8af77fe640 1 -- 172.21.15.47:0/3883617586 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8af00037a0 con 0x7f8b00100e70 2024-04-17T05:59:13.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:13.687+0000 7f8af77fe640 1 --2- 172.21.15.47:0/3883617586 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f8aec0a3bb0 0x7f8aec0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:13.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:13.687+0000 7f8af7fff640 1 --2- 172.21.15.47:0/3883617586 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f8aec0a3bb0 0x7f8aec0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:13.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:13.687+0000 7f8af77fe640 1 -- 172.21.15.47:0/3883617586 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7879+0+0 (secure 0 0 0) 0x7f8af00d0610 con 0x7f8b00100e70 2024-04-17T05:59:13.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:13.688+0000 7f8af7fff640 1 --2- 172.21.15.47:0/3883617586 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f8aec0a3bb0 0x7f8aec0a6070 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8b001997f0 tx=0x7f8ae8009110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:59:13.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:13.692+0000 7f8af77fe640 1 -- 172.21.15.47:0/3883617586 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f8af009d1a0 con 0x7f8b00100e70 2024-04-17T05:59:13.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:13 smithi098 ceph-mon[36516]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T05:59:13.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:13 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/3025417558' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-17T05:59:13.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:13 smithi098 ceph-mon[36516]: osdmap e83: 12 total, 12 up, 12 in 2024-04-17T05:59:13.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:13 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/302624954' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-17T05:59:13.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:13 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:59:13.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:13 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:59:13.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:13 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:59:13.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:13 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:59:13.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:13 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:59:13.884 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:13 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:59:13.884 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:13 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:59:13.884 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:13 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T05:59:13.884 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:13 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:59:13.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:13 smithi047 ceph-mon[32223]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T05:59:13.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:13 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/3025417558' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-17T05:59:13.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:13 smithi047 ceph-mon[32223]: osdmap e83: 12 total, 12 up, 12 in 2024-04-17T05:59:13.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:13 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/302624954' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-17T05:59:13.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:13 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:59:13.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:13 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:59:13.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:13 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:59:13.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:13 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:59:13.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:13 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:59:13.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:13 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:59:13.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:13 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:59:13.890 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:13 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T05:59:13.890 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:13 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:59:13.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:13.920+0000 7f8b04e14640 1 -- 172.21.15.47:0/3883617586 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f8b0002d5d0 con 0x7f8b00100e70 2024-04-17T05:59:14.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:13 smithi103 ceph-mon[36243]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T05:59:14.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:13 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/3025417558' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-17T05:59:14.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:13 smithi103 ceph-mon[36243]: osdmap e83: 12 total, 12 up, 12 in 2024-04-17T05:59:14.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:13 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/302624954' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-17T05:59:14.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:13 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:59:14.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:13 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:59:14.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:13 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:59:14.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:13 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:59:14.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:13 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:59:14.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:13 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:59:14.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:13 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T05:59:14.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:13 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T05:59:14.035 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:13 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T05:59:14.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:14.562+0000 7f8af77fe640 1 -- 172.21.15.47:0/3883617586 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f8af009caa0 con 0x7f8b00100e70 2024-04-17T05:59:14.561 INFO:teuthology.orchestra.run.smithi047.stderr:new fs with metadata pool 2 and data pool 3 2024-04-17T05:59:14.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:14.566+0000 7f8b04e14640 1 -- 172.21.15.47:0/3883617586 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f8aec0a3bb0 msgr2=0x7f8aec0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:14.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:14.566+0000 7f8b04e14640 1 --2- 172.21.15.47:0/3883617586 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f8aec0a3bb0 0x7f8aec0a6070 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8b001997f0 tx=0x7f8ae8009110 comp rx=0 tx=0).stop 2024-04-17T05:59:14.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:14.566+0000 7f8b04e14640 1 -- 172.21.15.47:0/3883617586 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b00100e70 msgr2=0x7f8b001982e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:14.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:14.566+0000 7f8b04e14640 1 --2- 172.21.15.47:0/3883617586 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b00100e70 0x7f8b001982e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8af000e9c0 tx=0x7f8af000edc0 comp rx=0 tx=0).stop 2024-04-17T05:59:14.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:14.566+0000 7f8b04e14640 1 -- 172.21.15.47:0/3883617586 shutdown_connections 2024-04-17T05:59:14.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:14.566+0000 7f8b04e14640 1 --2- 172.21.15.47:0/3883617586 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f8aec0a3bb0 0x7f8aec0a6070 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:14.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:14.566+0000 7f8b04e14640 1 --2- 172.21.15.47:0/3883617586 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8b00102a10 0x7f8b0019f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:14.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:14.566+0000 7f8b04e14640 1 --2- 172.21.15.47:0/3883617586 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8b00102050 0x7f8b00198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:14.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:14.566+0000 7f8b04e14640 1 --2- 172.21.15.47:0/3883617586 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b00100e70 0x7f8b001982e0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:14.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:14.566+0000 7f8b04e14640 1 -- 172.21.15.47:0/3883617586 >> 172.21.15.47:0/3883617586 conn(0x7f8b000fc450 msgr2=0x7f8b00105380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:59:14.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:14.566+0000 7f8b04e14640 1 -- 172.21.15.47:0/3883617586 shutdown_connections 2024-04-17T05:59:14.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:14.566+0000 7f8b04e14640 1 -- 172.21.15.47:0/3883617586 wait complete. 2024-04-17T05:59:14.589 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T05:59:14.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:14.702+0000 7fdac884a640 1 -- 172.21.15.47:0/1273036273 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdac41031a0 msgr2=0x7fdac4103620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:14.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:14.702+0000 7fdac884a640 1 --2- 172.21.15.47:0/1273036273 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdac41031a0 0x7fdac4103620 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fdaac009900 tx=0x7fdaac02bc30 comp rx=0 tx=0).stop 2024-04-17T05:59:14.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:14.702+0000 7fdac884a640 1 -- 172.21.15.47:0/1273036273 shutdown_connections 2024-04-17T05:59:14.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:14.702+0000 7fdac884a640 1 --2- 172.21.15.47:0/1273036273 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdac4103b60 0x7fdac410a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:14.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:14.702+0000 7fdac884a640 1 --2- 172.21.15.47:0/1273036273 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdac41031a0 0x7fdac4103620 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:14.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:14.702+0000 7fdac884a640 1 --2- 172.21.15.47:0/1273036273 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdac4101fb0 0x7fdac41023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:14.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:14.702+0000 7fdac884a640 1 -- 172.21.15.47:0/1273036273 >> 172.21.15.47:0/1273036273 conn(0x7fdac40fd540 msgr2=0x7fdac40ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:59:14.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:14.702+0000 7fdac884a640 1 -- 172.21.15.47:0/1273036273 shutdown_connections 2024-04-17T05:59:14.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:14.702+0000 7fdac884a640 1 -- 172.21.15.47:0/1273036273 wait complete. 2024-04-17T05:59:14.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:14.703+0000 7fdac884a640 1 Processor -- start 2024-04-17T05:59:14.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:14.703+0000 7fdac884a640 1 -- start start 2024-04-17T05:59:14.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:14.703+0000 7fdac884a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdac4101fb0 0x7fdac4198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:14.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:14.703+0000 7fdac884a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdac41031a0 0x7fdac41987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:14.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:14.703+0000 7fdac884a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdac4103b60 0x7fdac419f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:14.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:14.703+0000 7fdac884a640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fdac410dc40 con 0x7fdac4101fb0 2024-04-17T05:59:14.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:14.703+0000 7fdac884a640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fdac410dac0 con 0x7fdac4103b60 2024-04-17T05:59:14.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:14.703+0000 7fdac884a640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdac410ddc0 con 0x7fdac41031a0 2024-04-17T05:59:14.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:14.703+0000 7fdac259c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdac4101fb0 0x7fdac4198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:14.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:14.703+0000 7fdac259c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdac4101fb0 0x7fdac4198290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57816/0 (socket says 172.21.15.47:57816) 2024-04-17T05:59:14.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:14.703+0000 7fdac259c640 1 -- 172.21.15.47:0/865994458 learned_addr learned my addr 172.21.15.47:0/865994458 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:59:14.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:14.703+0000 7fdac2d9d640 1 --2- 172.21.15.47:0/865994458 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdac4103b60 0x7fdac419f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:14.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:14.703+0000 7fdabbfff640 1 --2- 172.21.15.47:0/865994458 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdac41031a0 0x7fdac41987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:14.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:14.704+0000 7fdac259c640 1 -- 172.21.15.47:0/865994458 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdac41031a0 msgr2=0x7fdac41987d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:14.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:14.704+0000 7fdac259c640 1 --2- 172.21.15.47:0/865994458 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdac41031a0 0x7fdac41987d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:14.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:14.704+0000 7fdac259c640 1 -- 172.21.15.47:0/865994458 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdac4103b60 msgr2=0x7fdac419f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:14.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:14.704+0000 7fdac259c640 1 --2- 172.21.15.47:0/865994458 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdac4103b60 0x7fdac419f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:14.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:14.704+0000 7fdac259c640 1 -- 172.21.15.47:0/865994458 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdac419feb0 con 0x7fdac4101fb0 2024-04-17T05:59:14.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:14.704+0000 7fdac259c640 1 --2- 172.21.15.47:0/865994458 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdac4101fb0 0x7fdac4198290 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fdab400d950 tx=0x7fdab400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:59:14.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:14.704+0000 7fdabb7fe640 1 -- 172.21.15.47:0/865994458 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdab4017020 con 0x7fdac4101fb0 2024-04-17T05:59:14.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:14.704+0000 7fdabb7fe640 1 -- 172.21.15.47:0/865994458 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdab4009c30 con 0x7fdac4101fb0 2024-04-17T05:59:14.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:14.704+0000 7fdac884a640 1 -- 172.21.15.47:0/865994458 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdac41a01a0 con 0x7fdac4101fb0 2024-04-17T05:59:14.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:14.704+0000 7fdabb7fe640 1 -- 172.21.15.47:0/865994458 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdab400f740 con 0x7fdac4101fb0 2024-04-17T05:59:14.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:14.704+0000 7fdac884a640 1 -- 172.21.15.47:0/865994458 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdac41a06c0 con 0x7fdac4101fb0 2024-04-17T05:59:14.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:14.705+0000 7fdac884a640 1 -- 172.21.15.47:0/865994458 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdac41092d0 con 0x7fdac4101fb0 2024-04-17T05:59:14.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:14.706+0000 7fdabb7fe640 1 -- 172.21.15.47:0/865994458 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fdab401a020 con 0x7fdac4101fb0 2024-04-17T05:59:14.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:14.707+0000 7fdabb7fe640 1 --2- 172.21.15.47:0/865994458 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fdab00a3bb0 0x7fdab00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:14.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:14.707+0000 7fdabbfff640 1 --2- 172.21.15.47:0/865994458 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fdab00a3bb0 0x7fdab00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:14.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:14.707+0000 7fdabb7fe640 1 -- 172.21.15.47:0/865994458 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fdab40d0020 con 0x7fdac4101fb0 2024-04-17T05:59:14.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:14.707+0000 7fdabbfff640 1 --2- 172.21.15.47:0/865994458 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fdab00a3bb0 0x7fdab00a6070 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fdaac006010 tx=0x7fdaac000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:59:14.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:14.710+0000 7fdabb7fe640 1 -- 172.21.15.47:0/865994458 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fdab4021070 con 0x7fdac4101fb0 2024-04-17T05:59:14.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:14 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/302624954' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-17T05:59:14.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:14 smithi098 ceph-mon[36516]: osdmap e84: 12 total, 12 up, 12 in 2024-04-17T05:59:14.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:14 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/3883617586' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-17T05:59:14.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:14 smithi098 ceph-mon[36516]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-17T05:59:14.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:14 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/302624954' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-17T05:59:14.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:14 smithi047 ceph-mon[32223]: osdmap e84: 12 total, 12 up, 12 in 2024-04-17T05:59:14.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:14 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/3883617586' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-17T05:59:14.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:14 smithi047 ceph-mon[32223]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-17T05:59:14.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:14.901+0000 7fdac884a640 1 -- 172.21.15.47:0/865994458 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdac4062b90 con 0x7fdac4101fb0 2024-04-17T05:59:14.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:14.903+0000 7fdabb7fe640 1 -- 172.21.15.47:0/865994458 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+22182 (secure 0 0 0) 0x7fdab4009de0 con 0x7fdac4101fb0 2024-04-17T05:59:14.902 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-17T05:59:14.902 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":85,"fsid":"565db2d8-fc7e-11ee-bc91-c7b262605968","created":"2024-04-17T05:51:19.888865+0000","modified":"2024-04-17T05:59:13.922225+0000","last_up_change":"2024-04-17T05:58:24.380131+0000","last_in_change":"2024-04-17T05:58:04.418236+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T05:54:37.734306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T05:59:12.458010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T05:59:12.901339+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8453f300-533d-4736-a343-55619c08e966","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":1290127544},{"type":"v1","addr":"172.21.15.47:6803","nonce":1290127544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":1290127544},{"type":"v1","addr":"172.21.15.47:6805","nonce":1290127544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":1290127544},{"type":"v1","addr":"172.21.15.47:6809","nonce":1290127544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":1290127544},{"type":"v1","addr":"172.21.15.47:6807","nonce":1290127544}]},"public_addr":"172.21.15.47:6803/1290127544","cluster_addr":"172.21.15.47:6805/1290127544","heartbeat_back_addr":"172.21.15.47:6809/1290127544","heartbeat_front_addr":"172.21.15.47:6807/1290127544","state":["exists","up"]},{"osd":1,"uuid":"80b0f03e-c1bc-4658-8028-1e6b8eedcc47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":1180430954},{"type":"v1","addr":"172.21.15.98:6801","nonce":1180430954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":1180430954},{"type":"v1","addr":"172.21.15.98:6803","nonce":1180430954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":1180430954},{"type":"v1","addr":"172.21.15.98:6807","nonce":1180430954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":1180430954},{"type":"v1","addr":"172.21.15.98:6805","nonce":1180430954}]},"public_addr":"172.21.15.98:6801/1180430954","cluster_addr":"172.21.15.98:6803/1180430954","heartbeat_back_addr":"172.21.15.98:6807/1180430954","heartbeat_front_addr":"172.21.15.98:6805/1180430954","state":["exists","up"]},{"osd":2,"uuid":"0fc9d383-05b4-498d-b530-fccf7aeddd7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":494676216},{"type":"v1","addr":"172.21.15.103:6801","nonce":494676216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":494676216},{"type":"v1","addr":"172.21.15.103:6803","nonce":494676216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":494676216},{"type":"v1","addr":"172.21.15.103:6807","nonce":494676216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":494676216},{"type":"v1","addr":"172.21.15.103:6805","nonce":494676216}]},"public_addr":"172.21.15.103:6801/494676216","cluster_addr":"172.21.15.103:6803/494676216","heartbeat_back_addr":"172.21.15.103:6807/494676216","heartbeat_front_addr":"172.21.15.103:6805/494676216","state":["exists","up"]},{"osd":3,"uuid":"f6e0b0d5-0fb3-42d0-9e84-f7cfef21a998","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":1998369447},{"type":"v1","addr":"172.21.15.47:6811","nonce":1998369447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":1998369447},{"type":"v1","addr":"172.21.15.47:6813","nonce":1998369447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":1998369447},{"type":"v1","addr":"172.21.15.47:6817","nonce":1998369447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":1998369447},{"type":"v1","addr":"172.21.15.47:6815","nonce":1998369447}]},"public_addr":"172.21.15.47:6811/1998369447","cluster_addr":"172.21.15.47:6813/1998369447","heartbeat_back_addr":"172.21.15.47:6817/1998369447","heartbeat_front_addr":"172.21.15.47:6815/1998369447","state":["exists","up"]},{"osd":4,"uuid":"2c26217e-02df-466d-bc01-0149d4aec3c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":3060482485},{"type":"v1","addr":"172.21.15.98:6809","nonce":3060482485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":3060482485},{"type":"v1","addr":"172.21.15.98:6811","nonce":3060482485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":3060482485},{"type":"v1","addr":"172.21.15.98:6815","nonce":3060482485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":3060482485},{"type":"v1","addr":"172.21.15.98:6813","nonce":3060482485}]},"public_addr":"172.21.15.98:6809/3060482485","cluster_addr":"172.21.15.98:6811/3060482485","heartbeat_back_addr":"172.21.15.98:6815/3060482485","heartbeat_front_addr":"172.21.15.98:6813/3060482485","state":["exists","up"]},{"osd":5,"uuid":"740bf3e0-bea8-496b-8d6f-e42d0b017258","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":4181363567},{"type":"v1","addr":"172.21.15.103:6809","nonce":4181363567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":4181363567},{"type":"v1","addr":"172.21.15.103:6811","nonce":4181363567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":4181363567},{"type":"v1","addr":"172.21.15.103:6815","nonce":4181363567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":4181363567},{"type":"v1","addr":"172.21.15.103:6813","nonce":4181363567}]},"public_addr":"172.21.15.103:6809/4181363567","cluster_addr":"172.21.15.103:6811/4181363567","heartbeat_back_addr":"172.21.15.103:6815/4181363567","heartbeat_front_addr":"172.21.15.103:6813/4181363567","state":["exists","up"]},{"osd":6,"uuid":"356a6255-2c07-4f0d-a233-26be04c92b38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":2517529668},{"type":"v1","addr":"172.21.15.47:6819","nonce":2517529668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2517529668},{"type":"v1","addr":"172.21.15.47:6821","nonce":2517529668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2517529668},{"type":"v1","addr":"172.21.15.47:6825","nonce":2517529668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2517529668},{"type":"v1","addr":"172.21.15.47:6823","nonce":2517529668}]},"public_addr":"172.21.15.47:6819/2517529668","cluster_addr":"172.21.15.47:6821/2517529668","heartbeat_back_addr":"172.21.15.47:6825/2517529668","heartbeat_front_addr":"172.21.15.47:6823/2517529668","state":["exists","up"]},{"osd":7,"uuid":"349d0120-8290-417d-a800-5a8985f554f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":4225498496},{"type":"v1","addr":"172.21.15.98:6817","nonce":4225498496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":4225498496},{"type":"v1","addr":"172.21.15.98:6819","nonce":4225498496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":4225498496},{"type":"v1","addr":"172.21.15.98:6823","nonce":4225498496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":4225498496},{"type":"v1","addr":"172.21.15.98:6821","nonce":4225498496}]},"public_addr":"172.21.15.98:6817/4225498496","cluster_addr":"172.21.15.98:6819/4225498496","heartbeat_back_addr":"172.21.15.98:6823/4225498496","heartbeat_front_addr":"172.21.15.98:6821/4225498496","state":["exists","up"]},{"osd":8,"uuid":"cee4712f-321b-42c0-92ba-e15144c00f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":3433905520},{"type":"v1","addr":"172.21.15.103:6817","nonce":3433905520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":3433905520},{"type":"v1","addr":"172.21.15.103:6819","nonce":3433905520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":3433905520},{"type":"v1","addr":"172.21.15.103:6823","nonce":3433905520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":3433905520},{"type":"v1","addr":"172.21.15.103:6821","nonce":3433905520}]},"public_addr":"172.21.15.103:6817/3433905520","cluster_addr":"172.21.15.103:6819/3433905520","heartbeat_back_addr":"172.21.15.103:6823/3433905520","heartbeat_front_addr":"172.21.15.103:6821/3433905520","state":["exists","up"]},{"osd":9,"uuid":"63fc73c1-5a22-487d-80b8-9894e2ddd207","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":728359554},{"type":"v1","addr":"172.21.15.47:6827","nonce":728359554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":728359554},{"type":"v1","addr":"172.21.15.47:6829","nonce":728359554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":728359554},{"type":"v1","addr":"172.21.15.47:6833","nonce":728359554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":728359554},{"type":"v1","addr":"172.21.15.47:6831","nonce":728359554}]},"public_addr":"172.21.15.47:6827/728359554","cluster_addr":"172.21.15.47:6829/728359554","heartbeat_back_addr":"172.21.15.47:6833/728359554","heartbeat_front_addr":"172.21.15.47:6831/728359554","state":["exists","up"]},{"osd":10,"uuid":"9565bcc3-62ee-48eb-a07a-03a81e0cc8e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":468205689},{"type":"v1","addr":"172.21.15.98:6825","nonce":468205689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":468205689},{"type":"v1","addr":"172.21.15.98:6827","nonce":468205689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":468205689},{"type":"v1","addr":"172.21.15.98:6831","nonce":468205689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":468205689},{"type":"v1","addr":"172.21.15.98:6829","nonce":468205689}]},"public_addr":"172.21.15.98:6825/468205689","cluster_addr":"172.21.15.98:6827/468205689","heartbeat_back_addr":"172.21.15.98:6831/468205689","heartbeat_front_addr":"172.21.15.98:6829/468205689","state":["exists","up"]},{"osd":11,"uuid":"2e85e8d9-bf6c-4a12-b124-a4690dd61108","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":2639831013},{"type":"v1","addr":"172.21.15.103:6825","nonce":2639831013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":2639831013},{"type":"v1","addr":"172.21.15.103:6827","nonce":2639831013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":2639831013},{"type":"v1","addr":"172.21.15.103:6831","nonce":2639831013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":2639831013},{"type":"v1","addr":"172.21.15.103:6829","nonce":2639831013}]},"public_addr":"172.21.15.103:6825/2639831013","cluster_addr":"172.21.15.103:6827/2639831013","heartbeat_back_addr":"172.21.15.103:6831/2639831013","heartbeat_front_addr":"172.21.15.103:6829/2639831013","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:53:47.366438+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:54:10.953559+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:54:34.320332+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:54:59.751360+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:55:23.982052+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:55:48.425017+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:56:14.793368+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:56:40.440830+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:57:04.623680+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:57:31.215262+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:57:57.040346+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:58:22.671572+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:6801/3632157175":"2024-04-18T05:52:13.645775+0000","172.21.15.47:0/4288219269":"2024-04-18T05:52:13.645775+0000","172.21.15.47:0/2584487544":"2024-04-18T05:52:13.645775+0000","172.21.15.47:6801/953064786":"2024-04-18T05:51:45.979351+0000","172.21.15.47:0/3484500927":"2024-04-18T05:51:45.979351+0000","172.21.15.47:6800/3632157175":"2024-04-18T05:52:13.645775+0000","172.21.15.47:0/112268670":"2024-04-18T05:51:45.979351+0000","172.21.15.47:0/1638423256":"2024-04-18T05:51:45.979351+0000","172.21.15.47:6800/953064786":"2024-04-18T05:51:45.979351+0000","172.21.15.47:0/849479010":"2024-04-18T05:52:13.645775+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T05:59:14.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:14.905+0000 7fdac884a640 1 -- 172.21.15.47:0/865994458 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fdab00a3bb0 msgr2=0x7fdab00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:14.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:14.905+0000 7fdac884a640 1 --2- 172.21.15.47:0/865994458 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fdab00a3bb0 0x7fdab00a6070 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fdaac006010 tx=0x7fdaac000bc0 comp rx=0 tx=0).stop 2024-04-17T05:59:14.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:14.905+0000 7fdac884a640 1 -- 172.21.15.47:0/865994458 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdac4101fb0 msgr2=0x7fdac4198290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:14.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:14.905+0000 7fdac884a640 1 --2- 172.21.15.47:0/865994458 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdac4101fb0 0x7fdac4198290 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fdab400d950 tx=0x7fdab400de10 comp rx=0 tx=0).stop 2024-04-17T05:59:14.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:14.905+0000 7fdac884a640 1 -- 172.21.15.47:0/865994458 shutdown_connections 2024-04-17T05:59:14.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:14.905+0000 7fdac884a640 1 --2- 172.21.15.47:0/865994458 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fdab00a3bb0 0x7fdab00a6070 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:14.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:14.905+0000 7fdac884a640 1 --2- 172.21.15.47:0/865994458 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdac4103b60 0x7fdac419f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:14.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:14.905+0000 7fdac884a640 1 --2- 172.21.15.47:0/865994458 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdac41031a0 0x7fdac41987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:14.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:14.905+0000 7fdac884a640 1 --2- 172.21.15.47:0/865994458 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdac4101fb0 0x7fdac4198290 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:14.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:14.905+0000 7fdac884a640 1 -- 172.21.15.47:0/865994458 >> 172.21.15.47:0/865994458 conn(0x7fdac40fd540 msgr2=0x7fdac41064a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:59:14.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:14.905+0000 7fdac884a640 1 -- 172.21.15.47:0/865994458 shutdown_connections 2024-04-17T05:59:14.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:14.905+0000 7fdac884a640 1 -- 172.21.15.47:0/865994458 wait complete. 2024-04-17T05:59:14.915 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T05:59:15.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:15.022+0000 7f81100b1640 1 -- 172.21.15.47:0/1797487772 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8108100a60 msgr2=0x7f810810a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:15.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:15.022+0000 7f81100b1640 1 --2- 172.21.15.47:0/1797487772 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8108100a60 0x7f810810a610 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8100009900 tx=0x7f810002bc30 comp rx=0 tx=0).stop 2024-04-17T05:59:15.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:15.022+0000 7f81100b1640 1 -- 172.21.15.47:0/1797487772 shutdown_connections 2024-04-17T05:59:15.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:15.022+0000 7f81100b1640 1 --2- 172.21.15.47:0/1797487772 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8108100a60 0x7f810810a610 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:15.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:15.022+0000 7f81100b1640 1 --2- 172.21.15.47:0/1797487772 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f81081000a0 0x7f8108100520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:15.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:15.022+0000 7f81100b1640 1 --2- 172.21.15.47:0/1797487772 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8108103630 0x7f8108103a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:15.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:15.022+0000 7f81100b1640 1 -- 172.21.15.47:0/1797487772 >> 172.21.15.47:0/1797487772 conn(0x7f81080fbc40 msgr2=0x7f81080fe080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:59:15.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:15.022+0000 7f81100b1640 1 -- 172.21.15.47:0/1797487772 shutdown_connections 2024-04-17T05:59:15.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:15.022+0000 7f81100b1640 1 -- 172.21.15.47:0/1797487772 wait complete. 2024-04-17T05:59:15.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:15.023+0000 7f81100b1640 1 Processor -- start 2024-04-17T05:59:15.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:15.023+0000 7f81100b1640 1 -- start start 2024-04-17T05:59:15.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:15.023+0000 7f81100b1640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f81081000a0 0x7f8108102d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:15.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:15.023+0000 7f81100b1640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8108100a60 0x7f8108101370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:15.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:15.023+0000 7f81100b1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8108103630 0x7f81081018b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:15.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:15.023+0000 7f81100b1640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f810810dd10 con 0x7f8108100a60 2024-04-17T05:59:15.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:15.023+0000 7f81100b1640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f810810db90 con 0x7f81081000a0 2024-04-17T05:59:15.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:15.023+0000 7f81100b1640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f810810de90 con 0x7f8108103630 2024-04-17T05:59:15.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:15.023+0000 7f810d64c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8108100a60 0x7f8108101370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:15.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:15.023+0000 7f810d64c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8108100a60 0x7f8108101370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57842/0 (socket says 172.21.15.47:57842) 2024-04-17T05:59:15.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:15.023+0000 7f810d64c640 1 -- 172.21.15.47:0/3800214110 learned_addr learned my addr 172.21.15.47:0/3800214110 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:59:15.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:15.023+0000 7f810de4d640 1 --2- 172.21.15.47:0/3800214110 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f81081000a0 0x7f8108102d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:15.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:15.023+0000 7f810d64c640 1 -- 172.21.15.47:0/3800214110 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8108103630 msgr2=0x7f81081018b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:15.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:15.023+0000 7f810d64c640 1 --2- 172.21.15.47:0/3800214110 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8108103630 0x7f81081018b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:15.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:15.023+0000 7f810d64c640 1 -- 172.21.15.47:0/3800214110 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f81081000a0 msgr2=0x7f8108102d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:15.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:15.023+0000 7f810d64c640 1 --2- 172.21.15.47:0/3800214110 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f81081000a0 0x7f8108102d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:15.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:15.023+0000 7f810d64c640 1 -- 172.21.15.47:0/3800214110 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8108102140 con 0x7f8108100a60 2024-04-17T05:59:15.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:15.023+0000 7f810de4d640 1 --2- 172.21.15.47:0/3800214110 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f81081000a0 0x7f8108102d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T05:59:15.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:15.023+0000 7f810d64c640 1 --2- 172.21.15.47:0/3800214110 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8108100a60 0x7f8108101370 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f80fc00c920 tx=0x7f80fc00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:59:15.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:15.023+0000 7f8106ffd640 1 -- 172.21.15.47:0/3800214110 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80fc007c70 con 0x7f8108100a60 2024-04-17T05:59:15.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:15.024+0000 7f8106ffd640 1 -- 172.21.15.47:0/3800214110 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80fc007e10 con 0x7f8108100a60 2024-04-17T05:59:15.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:15.024+0000 7f81100b1640 1 -- 172.21.15.47:0/3800214110 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81081a22c0 con 0x7f8108100a60 2024-04-17T05:59:15.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:15.024+0000 7f8106ffd640 1 -- 172.21.15.47:0/3800214110 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80fc018890 con 0x7f8108100a60 2024-04-17T05:59:15.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:15.024+0000 7f81100b1640 1 -- 172.21.15.47:0/3800214110 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81081a2810 con 0x7f8108100a60 2024-04-17T05:59:15.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:15.025+0000 7f81100b1640 1 -- 172.21.15.47:0/3800214110 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8108109420 con 0x7f8108100a60 2024-04-17T05:59:15.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:15.026+0000 7f8106ffd640 1 -- 172.21.15.47:0/3800214110 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f80fc018a70 con 0x7f8108100a60 2024-04-17T05:59:15.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:15.026+0000 7f8106ffd640 1 --2- 172.21.15.47:0/3800214110 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f80f00a3bb0 0x7f80f00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:15.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:15.027+0000 7f810de4d640 1 --2- 172.21.15.47:0/3800214110 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f80f00a3bb0 0x7f80f00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:15.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:15.027+0000 7f8106ffd640 1 -- 172.21.15.47:0/3800214110 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f80fc0d0510 con 0x7f8108100a60 2024-04-17T05:59:15.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:15.027+0000 7f810de4d640 1 --2- 172.21.15.47:0/3800214110 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f80f00a3bb0 0x7f80f00a6070 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f810806a360 tx=0x7f80f8007870 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:59:15.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:15.031+0000 7f8106ffd640 1 -- 172.21.15.47:0/3800214110 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f80fc014030 con 0x7f8108100a60 2024-04-17T05:59:15.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:14 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/302624954' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-17T05:59:15.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:14 smithi103 ceph-mon[36243]: osdmap e84: 12 total, 12 up, 12 in 2024-04-17T05:59:15.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:14 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/3883617586' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-17T05:59:15.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:14 smithi103 ceph-mon[36243]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-17T05:59:15.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:15.218+0000 7f81100b1640 1 -- 172.21.15.47:0/3800214110 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8108062b60 con 0x7f8108100a60 2024-04-17T05:59:15.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:15.219+0000 7f8106ffd640 1 -- 172.21.15.47:0/3800214110 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+22182 (secure 0 0 0) 0x7f80fc01f300 con 0x7f8108100a60 2024-04-17T05:59:15.219 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-17T05:59:15.219 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":85,"fsid":"565db2d8-fc7e-11ee-bc91-c7b262605968","created":"2024-04-17T05:51:19.888865+0000","modified":"2024-04-17T05:59:13.922225+0000","last_up_change":"2024-04-17T05:58:24.380131+0000","last_in_change":"2024-04-17T05:58:04.418236+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T05:54:37.734306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T05:59:12.458010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T05:59:12.901339+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8453f300-533d-4736-a343-55619c08e966","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":1290127544},{"type":"v1","addr":"172.21.15.47:6803","nonce":1290127544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":1290127544},{"type":"v1","addr":"172.21.15.47:6805","nonce":1290127544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":1290127544},{"type":"v1","addr":"172.21.15.47:6809","nonce":1290127544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":1290127544},{"type":"v1","addr":"172.21.15.47:6807","nonce":1290127544}]},"public_addr":"172.21.15.47:6803/1290127544","cluster_addr":"172.21.15.47:6805/1290127544","heartbeat_back_addr":"172.21.15.47:6809/1290127544","heartbeat_front_addr":"172.21.15.47:6807/1290127544","state":["exists","up"]},{"osd":1,"uuid":"80b0f03e-c1bc-4658-8028-1e6b8eedcc47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":1180430954},{"type":"v1","addr":"172.21.15.98:6801","nonce":1180430954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":1180430954},{"type":"v1","addr":"172.21.15.98:6803","nonce":1180430954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":1180430954},{"type":"v1","addr":"172.21.15.98:6807","nonce":1180430954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":1180430954},{"type":"v1","addr":"172.21.15.98:6805","nonce":1180430954}]},"public_addr":"172.21.15.98:6801/1180430954","cluster_addr":"172.21.15.98:6803/1180430954","heartbeat_back_addr":"172.21.15.98:6807/1180430954","heartbeat_front_addr":"172.21.15.98:6805/1180430954","state":["exists","up"]},{"osd":2,"uuid":"0fc9d383-05b4-498d-b530-fccf7aeddd7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":494676216},{"type":"v1","addr":"172.21.15.103:6801","nonce":494676216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":494676216},{"type":"v1","addr":"172.21.15.103:6803","nonce":494676216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":494676216},{"type":"v1","addr":"172.21.15.103:6807","nonce":494676216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":494676216},{"type":"v1","addr":"172.21.15.103:6805","nonce":494676216}]},"public_addr":"172.21.15.103:6801/494676216","cluster_addr":"172.21.15.103:6803/494676216","heartbeat_back_addr":"172.21.15.103:6807/494676216","heartbeat_front_addr":"172.21.15.103:6805/494676216","state":["exists","up"]},{"osd":3,"uuid":"f6e0b0d5-0fb3-42d0-9e84-f7cfef21a998","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":1998369447},{"type":"v1","addr":"172.21.15.47:6811","nonce":1998369447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":1998369447},{"type":"v1","addr":"172.21.15.47:6813","nonce":1998369447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":1998369447},{"type":"v1","addr":"172.21.15.47:6817","nonce":1998369447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":1998369447},{"type":"v1","addr":"172.21.15.47:6815","nonce":1998369447}]},"public_addr":"172.21.15.47:6811/1998369447","cluster_addr":"172.21.15.47:6813/1998369447","heartbeat_back_addr":"172.21.15.47:6817/1998369447","heartbeat_front_addr":"172.21.15.47:6815/1998369447","state":["exists","up"]},{"osd":4,"uuid":"2c26217e-02df-466d-bc01-0149d4aec3c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":3060482485},{"type":"v1","addr":"172.21.15.98:6809","nonce":3060482485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":3060482485},{"type":"v1","addr":"172.21.15.98:6811","nonce":3060482485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":3060482485},{"type":"v1","addr":"172.21.15.98:6815","nonce":3060482485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":3060482485},{"type":"v1","addr":"172.21.15.98:6813","nonce":3060482485}]},"public_addr":"172.21.15.98:6809/3060482485","cluster_addr":"172.21.15.98:6811/3060482485","heartbeat_back_addr":"172.21.15.98:6815/3060482485","heartbeat_front_addr":"172.21.15.98:6813/3060482485","state":["exists","up"]},{"osd":5,"uuid":"740bf3e0-bea8-496b-8d6f-e42d0b017258","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":4181363567},{"type":"v1","addr":"172.21.15.103:6809","nonce":4181363567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":4181363567},{"type":"v1","addr":"172.21.15.103:6811","nonce":4181363567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":4181363567},{"type":"v1","addr":"172.21.15.103:6815","nonce":4181363567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":4181363567},{"type":"v1","addr":"172.21.15.103:6813","nonce":4181363567}]},"public_addr":"172.21.15.103:6809/4181363567","cluster_addr":"172.21.15.103:6811/4181363567","heartbeat_back_addr":"172.21.15.103:6815/4181363567","heartbeat_front_addr":"172.21.15.103:6813/4181363567","state":["exists","up"]},{"osd":6,"uuid":"356a6255-2c07-4f0d-a233-26be04c92b38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":2517529668},{"type":"v1","addr":"172.21.15.47:6819","nonce":2517529668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2517529668},{"type":"v1","addr":"172.21.15.47:6821","nonce":2517529668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2517529668},{"type":"v1","addr":"172.21.15.47:6825","nonce":2517529668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2517529668},{"type":"v1","addr":"172.21.15.47:6823","nonce":2517529668}]},"public_addr":"172.21.15.47:6819/2517529668","cluster_addr":"172.21.15.47:6821/2517529668","heartbeat_back_addr":"172.21.15.47:6825/2517529668","heartbeat_front_addr":"172.21.15.47:6823/2517529668","state":["exists","up"]},{"osd":7,"uuid":"349d0120-8290-417d-a800-5a8985f554f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":4225498496},{"type":"v1","addr":"172.21.15.98:6817","nonce":4225498496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":4225498496},{"type":"v1","addr":"172.21.15.98:6819","nonce":4225498496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":4225498496},{"type":"v1","addr":"172.21.15.98:6823","nonce":4225498496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":4225498496},{"type":"v1","addr":"172.21.15.98:6821","nonce":4225498496}]},"public_addr":"172.21.15.98:6817/4225498496","cluster_addr":"172.21.15.98:6819/4225498496","heartbeat_back_addr":"172.21.15.98:6823/4225498496","heartbeat_front_addr":"172.21.15.98:6821/4225498496","state":["exists","up"]},{"osd":8,"uuid":"cee4712f-321b-42c0-92ba-e15144c00f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":3433905520},{"type":"v1","addr":"172.21.15.103:6817","nonce":3433905520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":3433905520},{"type":"v1","addr":"172.21.15.103:6819","nonce":3433905520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":3433905520},{"type":"v1","addr":"172.21.15.103:6823","nonce":3433905520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":3433905520},{"type":"v1","addr":"172.21.15.103:6821","nonce":3433905520}]},"public_addr":"172.21.15.103:6817/3433905520","cluster_addr":"172.21.15.103:6819/3433905520","heartbeat_back_addr":"172.21.15.103:6823/3433905520","heartbeat_front_addr":"172.21.15.103:6821/3433905520","state":["exists","up"]},{"osd":9,"uuid":"63fc73c1-5a22-487d-80b8-9894e2ddd207","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":728359554},{"type":"v1","addr":"172.21.15.47:6827","nonce":728359554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":728359554},{"type":"v1","addr":"172.21.15.47:6829","nonce":728359554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":728359554},{"type":"v1","addr":"172.21.15.47:6833","nonce":728359554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":728359554},{"type":"v1","addr":"172.21.15.47:6831","nonce":728359554}]},"public_addr":"172.21.15.47:6827/728359554","cluster_addr":"172.21.15.47:6829/728359554","heartbeat_back_addr":"172.21.15.47:6833/728359554","heartbeat_front_addr":"172.21.15.47:6831/728359554","state":["exists","up"]},{"osd":10,"uuid":"9565bcc3-62ee-48eb-a07a-03a81e0cc8e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":468205689},{"type":"v1","addr":"172.21.15.98:6825","nonce":468205689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":468205689},{"type":"v1","addr":"172.21.15.98:6827","nonce":468205689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":468205689},{"type":"v1","addr":"172.21.15.98:6831","nonce":468205689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":468205689},{"type":"v1","addr":"172.21.15.98:6829","nonce":468205689}]},"public_addr":"172.21.15.98:6825/468205689","cluster_addr":"172.21.15.98:6827/468205689","heartbeat_back_addr":"172.21.15.98:6831/468205689","heartbeat_front_addr":"172.21.15.98:6829/468205689","state":["exists","up"]},{"osd":11,"uuid":"2e85e8d9-bf6c-4a12-b124-a4690dd61108","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":2639831013},{"type":"v1","addr":"172.21.15.103:6825","nonce":2639831013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":2639831013},{"type":"v1","addr":"172.21.15.103:6827","nonce":2639831013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":2639831013},{"type":"v1","addr":"172.21.15.103:6831","nonce":2639831013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":2639831013},{"type":"v1","addr":"172.21.15.103:6829","nonce":2639831013}]},"public_addr":"172.21.15.103:6825/2639831013","cluster_addr":"172.21.15.103:6827/2639831013","heartbeat_back_addr":"172.21.15.103:6831/2639831013","heartbeat_front_addr":"172.21.15.103:6829/2639831013","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:53:47.366438+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:54:10.953559+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:54:34.320332+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:54:59.751360+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:55:23.982052+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:55:48.425017+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:56:14.793368+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:56:40.440830+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:57:04.623680+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:57:31.215262+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:57:57.040346+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:58:22.671572+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:6801/3632157175":"2024-04-18T05:52:13.645775+0000","172.21.15.47:0/4288219269":"2024-04-18T05:52:13.645775+0000","172.21.15.47:0/2584487544":"2024-04-18T05:52:13.645775+0000","172.21.15.47:6801/953064786":"2024-04-18T05:51:45.979351+0000","172.21.15.47:0/3484500927":"2024-04-18T05:51:45.979351+0000","172.21.15.47:6800/3632157175":"2024-04-18T05:52:13.645775+0000","172.21.15.47:0/112268670":"2024-04-18T05:51:45.979351+0000","172.21.15.47:0/1638423256":"2024-04-18T05:51:45.979351+0000","172.21.15.47:6800/953064786":"2024-04-18T05:51:45.979351+0000","172.21.15.47:0/849479010":"2024-04-18T05:52:13.645775+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T05:59:15.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:15.221+0000 7f81100b1640 1 -- 172.21.15.47:0/3800214110 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f80f00a3bb0 msgr2=0x7f80f00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:15.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:15.221+0000 7f81100b1640 1 --2- 172.21.15.47:0/3800214110 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f80f00a3bb0 0x7f80f00a6070 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f810806a360 tx=0x7f80f8007870 comp rx=0 tx=0).stop 2024-04-17T05:59:15.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:15.221+0000 7f81100b1640 1 -- 172.21.15.47:0/3800214110 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8108100a60 msgr2=0x7f8108101370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:15.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:15.221+0000 7f81100b1640 1 --2- 172.21.15.47:0/3800214110 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8108100a60 0x7f8108101370 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f80fc00c920 tx=0x7f80fc00cde0 comp rx=0 tx=0).stop 2024-04-17T05:59:15.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:15.221+0000 7f81100b1640 1 -- 172.21.15.47:0/3800214110 shutdown_connections 2024-04-17T05:59:15.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:15.221+0000 7f81100b1640 1 --2- 172.21.15.47:0/3800214110 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f80f00a3bb0 0x7f80f00a6070 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:15.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:15.221+0000 7f81100b1640 1 --2- 172.21.15.47:0/3800214110 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8108103630 0x7f81081018b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:15.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:15.221+0000 7f81100b1640 1 --2- 172.21.15.47:0/3800214110 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8108100a60 0x7f8108101370 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:15.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:15.221+0000 7f81100b1640 1 --2- 172.21.15.47:0/3800214110 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f81081000a0 0x7f8108102d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:15.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:15.221+0000 7f81100b1640 1 -- 172.21.15.47:0/3800214110 >> 172.21.15.47:0/3800214110 conn(0x7f81080fbc40 msgr2=0x7f81081065f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:59:15.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:15.221+0000 7f81100b1640 1 -- 172.21.15.47:0/3800214110 shutdown_connections 2024-04-17T05:59:15.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:15.222+0000 7f81100b1640 1 -- 172.21.15.47:0/3800214110 wait complete. 2024-04-17T05:59:15.232 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-17T05:59:15.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:15.344+0000 7f91ab8b1640 1 -- 172.21.15.47:0/1442613896 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f91a41029b0 msgr2=0x7f91a41092d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:15.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:15.344+0000 7f91ab8b1640 1 --2- 172.21.15.47:0/1442613896 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f91a41029b0 0x7f91a41092d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f919c009ca0 tx=0x7f919c02c0f0 comp rx=0 tx=0).stop 2024-04-17T05:59:15.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:15.344+0000 7f91ab8b1640 1 -- 172.21.15.47:0/1442613896 shutdown_connections 2024-04-17T05:59:15.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:15.344+0000 7f91ab8b1640 1 --2- 172.21.15.47:0/1442613896 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f91a41029b0 0x7f91a41092d0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:15.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:15.344+0000 7f91ab8b1640 1 --2- 172.21.15.47:0/1442613896 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f91a4101ff0 0x7f91a4102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:15.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:15.344+0000 7f91ab8b1640 1 --2- 172.21.15.47:0/1442613896 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f91a4100ea0 0x7f91a41012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:15.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:15.344+0000 7f91ab8b1640 1 -- 172.21.15.47:0/1442613896 >> 172.21.15.47:0/1442613896 conn(0x7f91a40fc410 msgr2=0x7f91a40fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:59:15.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:15.344+0000 7f91ab8b1640 1 -- 172.21.15.47:0/1442613896 shutdown_connections 2024-04-17T05:59:15.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:15.344+0000 7f91ab8b1640 1 -- 172.21.15.47:0/1442613896 wait complete. 2024-04-17T05:59:15.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:15.345+0000 7f91ab8b1640 1 Processor -- start 2024-04-17T05:59:15.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:15.345+0000 7f91ab8b1640 1 -- start start 2024-04-17T05:59:15.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:15.345+0000 7f91ab8b1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f91a4100ea0 0x7f91a4069c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:15.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:15.345+0000 7f91ab8b1640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f91a4101ff0 0x7f91a419cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:15.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:15.345+0000 7f91ab8b1640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f91a41029b0 0x7f91a419f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:15.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:15.345+0000 7f91ab8b1640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f91a410cb30 con 0x7f91a4101ff0 2024-04-17T05:59:15.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:15.345+0000 7f91ab8b1640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f91a410c9b0 con 0x7f91a41029b0 2024-04-17T05:59:15.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:15.345+0000 7f91ab8b1640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f91a410ccb0 con 0x7f91a4100ea0 2024-04-17T05:59:15.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:15.345+0000 7f91a8e4c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f91a4101ff0 0x7f91a419cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:15.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:15.345+0000 7f91a8e4c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f91a4101ff0 0x7f91a419cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57848/0 (socket says 172.21.15.47:57848) 2024-04-17T05:59:15.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:15.345+0000 7f91a8e4c640 1 -- 172.21.15.47:0/860872304 learned_addr learned my addr 172.21.15.47:0/860872304 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:59:15.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:15.345+0000 7f91a9e4e640 1 --2- 172.21.15.47:0/860872304 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f91a41029b0 0x7f91a419f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:15.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:15.345+0000 7f91a964d640 1 --2- 172.21.15.47:0/860872304 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f91a4100ea0 0x7f91a4069c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:15.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:15.345+0000 7f91a8e4c640 1 -- 172.21.15.47:0/860872304 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f91a4100ea0 msgr2=0x7f91a4069c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:15.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:15.345+0000 7f91a8e4c640 1 --2- 172.21.15.47:0/860872304 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f91a4100ea0 0x7f91a4069c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:15.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:15.345+0000 7f91a8e4c640 1 -- 172.21.15.47:0/860872304 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f91a41029b0 msgr2=0x7f91a419f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:15.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:15.345+0000 7f91a8e4c640 1 --2- 172.21.15.47:0/860872304 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f91a41029b0 0x7f91a419f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:15.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:15.345+0000 7f91a8e4c640 1 -- 172.21.15.47:0/860872304 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91a419fef0 con 0x7f91a4101ff0 2024-04-17T05:59:15.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:15.345+0000 7f91a9e4e640 1 --2- 172.21.15.47:0/860872304 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f91a41029b0 0x7f91a419f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T05:59:15.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:15.345+0000 7f91a964d640 1 --2- 172.21.15.47:0/860872304 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f91a4100ea0 0x7f91a4069c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T05:59:15.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:15.346+0000 7f91a8e4c640 1 --2- 172.21.15.47:0/860872304 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f91a4101ff0 0x7f91a419cfc0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f919400d950 tx=0x7f919400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:59:15.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:15.346+0000 7f919a7fc640 1 -- 172.21.15.47:0/860872304 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9194017020 con 0x7f91a4101ff0 2024-04-17T05:59:15.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:15.346+0000 7f919a7fc640 1 -- 172.21.15.47:0/860872304 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9194009c30 con 0x7f91a4101ff0 2024-04-17T05:59:15.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:15.346+0000 7f91ab8b1640 1 -- 172.21.15.47:0/860872304 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91a41a0180 con 0x7f91a4101ff0 2024-04-17T05:59:15.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:15.346+0000 7f919a7fc640 1 -- 172.21.15.47:0/860872304 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f919400f720 con 0x7f91a4101ff0 2024-04-17T05:59:15.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:15.346+0000 7f91ab8b1640 1 -- 172.21.15.47:0/860872304 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91a41a06d0 con 0x7f91a4101ff0 2024-04-17T05:59:15.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:15.347+0000 7f91ab8b1640 1 -- 172.21.15.47:0/860872304 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91a41081b0 con 0x7f91a4101ff0 2024-04-17T05:59:15.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:15.348+0000 7f919a7fc640 1 -- 172.21.15.47:0/860872304 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f919401a020 con 0x7f91a4101ff0 2024-04-17T05:59:15.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:15.348+0000 7f919a7fc640 1 --2- 172.21.15.47:0/860872304 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f91900a3bb0 0x7f91900a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:15.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:15.349+0000 7f91a964d640 1 --2- 172.21.15.47:0/860872304 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f91900a3bb0 0x7f91900a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:15.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:15.349+0000 7f919a7fc640 1 -- 172.21.15.47:0/860872304 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f91940d02d0 con 0x7f91a4101ff0 2024-04-17T05:59:15.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:15.349+0000 7f91a964d640 1 --2- 172.21.15.47:0/860872304 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f91900a3bb0 0x7f91900a6070 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f91a0009b50 tx=0x7f91a0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:59:15.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:15.353+0000 7f919a7fc640 1 -- 172.21.15.47:0/860872304 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f9194021020 con 0x7f91a4101ff0 2024-04-17T05:59:15.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:15.599+0000 7f91ab8b1640 1 -- 172.21.15.47:0/860872304 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f91a4062b90 con 0x7f91a4101ff0 2024-04-17T05:59:15.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:15 smithi098 ceph-mon[36516]: pgmap v284: 97 pgs: 5 creating+peering, 4 active+clean, 88 unknown; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T05:59:15.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:15 smithi098 ceph-mon[36516]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-17T05:59:15.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:15 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/3883617586' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-17T05:59:15.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:15 smithi098 ceph-mon[36516]: osdmap e85: 12 total, 12 up, 12 in 2024-04-17T05:59:15.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:15 smithi098 ceph-mon[36516]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-17T05:59:15.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:15 smithi098 ceph-mon[36516]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-17T05:59:15.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:15 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/865994458' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T05:59:15.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:15 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/3800214110' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T05:59:15.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:15 smithi047 ceph-mon[32223]: pgmap v284: 97 pgs: 5 creating+peering, 4 active+clean, 88 unknown; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T05:59:15.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:15 smithi047 ceph-mon[32223]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-17T05:59:15.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:15 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/3883617586' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-17T05:59:15.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:15 smithi047 ceph-mon[32223]: osdmap e85: 12 total, 12 up, 12 in 2024-04-17T05:59:15.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:15 smithi047 ceph-mon[32223]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-17T05:59:15.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:15 smithi047 ceph-mon[32223]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-17T05:59:15.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:15 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/865994458' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T05:59:15.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:15 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/3800214110' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T05:59:16.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:15 smithi103 ceph-mon[36243]: pgmap v284: 97 pgs: 5 creating+peering, 4 active+clean, 88 unknown; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T05:59:16.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:15 smithi103 ceph-mon[36243]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-17T05:59:16.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:15 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/3883617586' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-17T05:59:16.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:15 smithi103 ceph-mon[36243]: osdmap e85: 12 total, 12 up, 12 in 2024-04-17T05:59:16.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:15 smithi103 ceph-mon[36243]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-17T05:59:16.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:15 smithi103 ceph-mon[36243]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-17T05:59:16.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:15 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/865994458' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T05:59:16.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:15 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/3800214110' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T05:59:16.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:16.607+0000 7f919a7fc640 1 -- 172.21.15.47:0/860872304 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) ==== 118+0+0 (secure 0 0 0) 0x7f9194009de0 con 0x7f91a4101ff0 2024-04-17T05:59:16.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:16.610+0000 7f91ab8b1640 1 -- 172.21.15.47:0/860872304 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f91900a3bb0 msgr2=0x7f91900a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:16.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:16.610+0000 7f91ab8b1640 1 --2- 172.21.15.47:0/860872304 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f91900a3bb0 0x7f91900a6070 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f91a0009b50 tx=0x7f91a0005f90 comp rx=0 tx=0).stop 2024-04-17T05:59:16.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:16.610+0000 7f91ab8b1640 1 -- 172.21.15.47:0/860872304 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f91a4101ff0 msgr2=0x7f91a419cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:16.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:16.610+0000 7f91ab8b1640 1 --2- 172.21.15.47:0/860872304 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f91a4101ff0 0x7f91a419cfc0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f919400d950 tx=0x7f919400de10 comp rx=0 tx=0).stop 2024-04-17T05:59:16.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:16.610+0000 7f91ab8b1640 1 -- 172.21.15.47:0/860872304 shutdown_connections 2024-04-17T05:59:16.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:16.610+0000 7f91ab8b1640 1 --2- 172.21.15.47:0/860872304 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f91900a3bb0 0x7f91900a6070 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:16.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:16.610+0000 7f91ab8b1640 1 --2- 172.21.15.47:0/860872304 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f91a41029b0 0x7f91a419f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:16.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:16.610+0000 7f91ab8b1640 1 --2- 172.21.15.47:0/860872304 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f91a4101ff0 0x7f91a419cfc0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:16.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:16.610+0000 7f91ab8b1640 1 --2- 172.21.15.47:0/860872304 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f91a4100ea0 0x7f91a4069c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:16.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:16.610+0000 7f91ab8b1640 1 -- 172.21.15.47:0/860872304 >> 172.21.15.47:0/860872304 conn(0x7f91a40fc410 msgr2=0x7f91a4105380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:59:16.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:16.610+0000 7f91ab8b1640 1 -- 172.21.15.47:0/860872304 shutdown_connections 2024-04-17T05:59:16.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:16.610+0000 7f91ab8b1640 1 -- 172.21.15.47:0/860872304 wait complete. 2024-04-17T05:59:16.622 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-04-17T05:59:16.623 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2024-04-17T05:59:16.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:16.739+0000 7fd91de28640 1 -- 172.21.15.47:0/3874863249 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd9181029b0 msgr2=0x7fd9181092d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:16.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:16.739+0000 7fd91de28640 1 --2- 172.21.15.47:0/3874863249 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd9181029b0 0x7fd9181092d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd910009ca0 tx=0x7fd91002c0f0 comp rx=0 tx=0).stop 2024-04-17T05:59:16.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:16.739+0000 7fd91de28640 1 -- 172.21.15.47:0/3874863249 shutdown_connections 2024-04-17T05:59:16.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:16.739+0000 7fd91de28640 1 --2- 172.21.15.47:0/3874863249 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd9181029b0 0x7fd9181092d0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:16.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:16.739+0000 7fd91de28640 1 --2- 172.21.15.47:0/3874863249 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd918101ff0 0x7fd918102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:16.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:16.739+0000 7fd91de28640 1 --2- 172.21.15.47:0/3874863249 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd918100ea0 0x7fd9181012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:16.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:16.739+0000 7fd91de28640 1 -- 172.21.15.47:0/3874863249 >> 172.21.15.47:0/3874863249 conn(0x7fd9180fc410 msgr2=0x7fd9180fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:59:16.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:16.739+0000 7fd91de28640 1 -- 172.21.15.47:0/3874863249 shutdown_connections 2024-04-17T05:59:16.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:16.739+0000 7fd91de28640 1 -- 172.21.15.47:0/3874863249 wait complete. 2024-04-17T05:59:16.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:16.739+0000 7fd91de28640 1 Processor -- start 2024-04-17T05:59:16.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:16.739+0000 7fd91de28640 1 -- start start 2024-04-17T05:59:16.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:16.740+0000 7fd91de28640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd918100ea0 0x7fd918069c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:16.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:16.740+0000 7fd91de28640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd918101ff0 0x7fd91819cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:16.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:16.740+0000 7fd91de28640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd9181029b0 0x7fd91819f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:16.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:16.740+0000 7fd91de28640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fd91810cb40 con 0x7fd9181029b0 2024-04-17T05:59:16.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:16.740+0000 7fd91de28640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd91810c9c0 con 0x7fd918101ff0 2024-04-17T05:59:16.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:16.740+0000 7fd91de28640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd91810ccc0 con 0x7fd918100ea0 2024-04-17T05:59:16.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:16.740+0000 7fd917fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd9181029b0 0x7fd91819f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:16.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:16.740+0000 7fd917fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd9181029b0 0x7fd91819f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57862/0 (socket says 172.21.15.47:57862) 2024-04-17T05:59:16.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:16.740+0000 7fd917fff640 1 -- 172.21.15.47:0/2773878743 learned_addr learned my addr 172.21.15.47:0/2773878743 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:59:16.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:16.740+0000 7fd90effd640 1 --2- 172.21.15.47:0/2773878743 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd918101ff0 0x7fd91819cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:16.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:16.740+0000 7fd9177fe640 1 --2- 172.21.15.47:0/2773878743 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd918100ea0 0x7fd918069c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:16.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:16.740+0000 7fd917fff640 1 -- 172.21.15.47:0/2773878743 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd918100ea0 msgr2=0x7fd918069c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:16.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:16.740+0000 7fd917fff640 1 --2- 172.21.15.47:0/2773878743 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd918100ea0 0x7fd918069c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:16.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:16.740+0000 7fd917fff640 1 -- 172.21.15.47:0/2773878743 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd918101ff0 msgr2=0x7fd91819cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:16.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:16.740+0000 7fd917fff640 1 --2- 172.21.15.47:0/2773878743 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd918101ff0 0x7fd91819cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:16.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:16.740+0000 7fd917fff640 1 -- 172.21.15.47:0/2773878743 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd91819fec0 con 0x7fd9181029b0 2024-04-17T05:59:16.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:16.740+0000 7fd917fff640 1 --2- 172.21.15.47:0/2773878743 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd9181029b0 0x7fd91819f600 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd910037040 tx=0x7fd910032f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:59:16.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:16.740+0000 7fd9157fa640 1 -- 172.21.15.47:0/2773878743 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd910039040 con 0x7fd9181029b0 2024-04-17T05:59:16.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:16.741+0000 7fd9157fa640 1 -- 172.21.15.47:0/2773878743 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd910044070 con 0x7fd9181029b0 2024-04-17T05:59:16.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:16.741+0000 7fd91de28640 1 -- 172.21.15.47:0/2773878743 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9181a0150 con 0x7fd9181029b0 2024-04-17T05:59:16.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:16.741+0000 7fd9157fa640 1 -- 172.21.15.47:0/2773878743 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd910004330 con 0x7fd9181029b0 2024-04-17T05:59:16.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:16.741+0000 7fd91de28640 1 -- 172.21.15.47:0/2773878743 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9181a0640 con 0x7fd9181029b0 2024-04-17T05:59:16.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:16.742+0000 7fd91de28640 1 -- 172.21.15.47:0/2773878743 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9181081b0 con 0x7fd9181029b0 2024-04-17T05:59:16.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:16.743+0000 7fd9157fa640 1 -- 172.21.15.47:0/2773878743 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fd910003a00 con 0x7fd9181029b0 2024-04-17T05:59:16.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:16.743+0000 7fd9157fa640 1 --2- 172.21.15.47:0/2773878743 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fd9040a3bb0 0x7fd9040a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:16.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:16.744+0000 7fd9177fe640 1 --2- 172.21.15.47:0/2773878743 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fd9040a3bb0 0x7fd9040a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:16.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:16.744+0000 7fd9157fa640 1 -- 172.21.15.47:0/2773878743 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fd9100f30d0 con 0x7fd9181029b0 2024-04-17T05:59:16.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:16.744+0000 7fd9177fe640 1 --2- 172.21.15.47:0/2773878743 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fd9040a3bb0 0x7fd9040a6070 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd9080058c0 tx=0x7fd90800a4c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:59:16.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:16.747+0000 7fd9157fa640 1 -- 172.21.15.47:0/2773878743 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fd9100bfc00 con 0x7fd9181029b0 2024-04-17T05:59:16.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:16 smithi098 ceph-mon[36516]: osdmap e86: 12 total, 12 up, 12 in 2024-04-17T05:59:16.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:16 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/860872304' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-17T05:59:16.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:16 smithi098 ceph-mon[36516]: mds.? [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] up:active 2024-04-17T05:59:16.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:16 smithi098 ceph-mon[36516]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-17T05:59:16.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:16 smithi047 ceph-mon[32223]: osdmap e86: 12 total, 12 up, 12 in 2024-04-17T05:59:16.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:16 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/860872304' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-17T05:59:16.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:16 smithi047 ceph-mon[32223]: mds.? [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] up:active 2024-04-17T05:59:16.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:16 smithi047 ceph-mon[32223]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-17T05:59:16.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:16.973+0000 7fd91de28640 1 -- 172.21.15.47:0/2773878743 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7fd91802d820 con 0x7fd9181029b0 2024-04-17T05:59:17.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:16 smithi103 ceph-mon[36243]: osdmap e86: 12 total, 12 up, 12 in 2024-04-17T05:59:17.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:16 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/860872304' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-17T05:59:17.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:16 smithi103 ceph-mon[36243]: mds.? [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] up:active 2024-04-17T05:59:17.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:16 smithi103 ceph-mon[36243]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-17T05:59:17.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:17.608+0000 7fd9157fa640 1 -- 172.21.15.47:0/2773878743 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v17) ==== 105+0+0 (secure 0 0 0) 0x7fd9100bf500 con 0x7fd9181029b0 2024-04-17T05:59:17.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:17.611+0000 7fd91de28640 1 -- 172.21.15.47:0/2773878743 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fd9040a3bb0 msgr2=0x7fd9040a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:17.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:17.611+0000 7fd91de28640 1 --2- 172.21.15.47:0/2773878743 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fd9040a3bb0 0x7fd9040a6070 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd9080058c0 tx=0x7fd90800a4c0 comp rx=0 tx=0).stop 2024-04-17T05:59:17.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:17.611+0000 7fd91de28640 1 -- 172.21.15.47:0/2773878743 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd9181029b0 msgr2=0x7fd91819f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:17.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:17.611+0000 7fd91de28640 1 --2- 172.21.15.47:0/2773878743 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd9181029b0 0x7fd91819f600 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd910037040 tx=0x7fd910032f90 comp rx=0 tx=0).stop 2024-04-17T05:59:17.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:17.611+0000 7fd91de28640 1 -- 172.21.15.47:0/2773878743 shutdown_connections 2024-04-17T05:59:17.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:17.611+0000 7fd91de28640 1 --2- 172.21.15.47:0/2773878743 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fd9040a3bb0 0x7fd9040a6070 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:17.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:17.611+0000 7fd91de28640 1 --2- 172.21.15.47:0/2773878743 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd9181029b0 0x7fd91819f600 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:17.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:17.611+0000 7fd91de28640 1 --2- 172.21.15.47:0/2773878743 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd918101ff0 0x7fd91819cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:17.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:17.611+0000 7fd91de28640 1 --2- 172.21.15.47:0/2773878743 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd918100ea0 0x7fd918069c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:17.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:17.611+0000 7fd91de28640 1 -- 172.21.15.47:0/2773878743 >> 172.21.15.47:0/2773878743 conn(0x7fd9180fc410 msgr2=0x7fd918105380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:59:17.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:17.612+0000 7fd91de28640 1 -- 172.21.15.47:0/2773878743 shutdown_connections 2024-04-17T05:59:17.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:17.612+0000 7fd91de28640 1 -- 172.21.15.47:0/2773878743 wait complete. 2024-04-17T05:59:17.626 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-04-17T05:59:17.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:17.742+0000 7f0f1e77c640 1 -- 172.21.15.47:0/3999133863 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f18107ca0 msgr2=0x7f0f1810a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:17.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:17.742+0000 7f0f1e77c640 1 --2- 172.21.15.47:0/3999133863 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f18107ca0 0x7f0f1810a0f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f0f10009ca0 tx=0x7f0f1002c0f0 comp rx=0 tx=0).stop 2024-04-17T05:59:17.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:17.743+0000 7f0f1e77c640 1 -- 172.21.15.47:0/3999133863 shutdown_connections 2024-04-17T05:59:17.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:17.743+0000 7f0f1e77c640 1 --2- 172.21.15.47:0/3999133863 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f18107ca0 0x7f0f1810a0f0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:17.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:17.743+0000 7f0f1e77c640 1 --2- 172.21.15.47:0/3999133863 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0f18105370 0x7f0f18107760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:17.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:17.743+0000 7f0f1e77c640 1 --2- 172.21.15.47:0/3999133863 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0f18102a40 0x7f0f18104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:17.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:17.743+0000 7f0f1e77c640 1 -- 172.21.15.47:0/3999133863 >> 172.21.15.47:0/3999133863 conn(0x7f0f180fc450 msgr2=0x7f0f180fe870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:59:17.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:17.743+0000 7f0f1e77c640 1 -- 172.21.15.47:0/3999133863 shutdown_connections 2024-04-17T05:59:17.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:17.743+0000 7f0f1e77c640 1 -- 172.21.15.47:0/3999133863 wait complete. 2024-04-17T05:59:17.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:17.743+0000 7f0f1e77c640 1 Processor -- start 2024-04-17T05:59:17.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:17.743+0000 7f0f1e77c640 1 -- start start 2024-04-17T05:59:17.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:17.743+0000 7f0f1e77c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f18102a40 0x7f0f1819a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:17.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:17.744+0000 7f0f1e77c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0f18105370 0x7f0f1819a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:17.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:17.744+0000 7f0f1e77c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0f18107ca0 0x7f0f181a19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:17.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:17.744+0000 7f0f1e77c640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f0f1810dc50 con 0x7f0f18102a40 2024-04-17T05:59:17.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:17.744+0000 7f0f1e77c640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0f1810dad0 con 0x7f0f18105370 2024-04-17T05:59:17.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:17.744+0000 7f0f1e77c640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0f1810ddd0 con 0x7f0f18107ca0 2024-04-17T05:59:17.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:17.744+0000 7f0f17fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f18102a40 0x7f0f1819a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:17.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:17.744+0000 7f0f17fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f18102a40 0x7f0f1819a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57868/0 (socket says 172.21.15.47:57868) 2024-04-17T05:59:17.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:17.744+0000 7f0f17fff640 1 -- 172.21.15.47:0/2945331400 learned_addr learned my addr 172.21.15.47:0/2945331400 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:59:17.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:17.744+0000 7f0f1cd19640 1 --2- 172.21.15.47:0/2945331400 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0f18107ca0 0x7f0f181a19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:17.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:17.744+0000 7f0f177fe640 1 --2- 172.21.15.47:0/2945331400 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0f18105370 0x7f0f1819a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:17.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:17.744+0000 7f0f17fff640 1 -- 172.21.15.47:0/2945331400 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0f18107ca0 msgr2=0x7f0f181a19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:17.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:17.744+0000 7f0f17fff640 1 --2- 172.21.15.47:0/2945331400 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0f18107ca0 0x7f0f181a19b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:17.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:17.744+0000 7f0f17fff640 1 -- 172.21.15.47:0/2945331400 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0f18105370 msgr2=0x7f0f1819a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:17.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:17.744+0000 7f0f17fff640 1 --2- 172.21.15.47:0/2945331400 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0f18105370 0x7f0f1819a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:17.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:17.744+0000 7f0f17fff640 1 -- 172.21.15.47:0/2945331400 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f181a2020 con 0x7f0f18102a40 2024-04-17T05:59:17.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:17.744+0000 7f0f1cd19640 1 --2- 172.21.15.47:0/2945331400 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0f18107ca0 0x7f0f181a19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T05:59:17.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:17.744+0000 7f0f177fe640 1 --2- 172.21.15.47:0/2945331400 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0f18105370 0x7f0f1819a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T05:59:17.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:17.744+0000 7f0f17fff640 1 --2- 172.21.15.47:0/2945331400 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f18102a40 0x7f0f1819a400 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f0f0800c970 tx=0x7f0f0800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:59:17.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:17.744+0000 7f0f157fa640 1 -- 172.21.15.47:0/2945331400 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f08007bb0 con 0x7f0f18102a40 2024-04-17T05:59:17.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:17.744+0000 7f0f157fa640 1 -- 172.21.15.47:0/2945331400 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f08007d50 con 0x7f0f18102a40 2024-04-17T05:59:17.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:17.744+0000 7f0f1e77c640 1 -- 172.21.15.47:0/2945331400 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f181a2310 con 0x7f0f18102a40 2024-04-17T05:59:17.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:17.744+0000 7f0f157fa640 1 -- 172.21.15.47:0/2945331400 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f0800f450 con 0x7f0f18102a40 2024-04-17T05:59:17.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:17.744+0000 7f0f1e77c640 1 -- 172.21.15.47:0/2945331400 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f181a2810 con 0x7f0f18102a40 2024-04-17T05:59:17.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:17.745+0000 7f0f1e77c640 1 -- 172.21.15.47:0/2945331400 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f18109790 con 0x7f0f18102a40 2024-04-17T05:59:17.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:17.747+0000 7f0f157fa640 1 -- 172.21.15.47:0/2945331400 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f0f08005020 con 0x7f0f18102a40 2024-04-17T05:59:17.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:17.747+0000 7f0f157fa640 1 --2- 172.21.15.47:0/2945331400 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f0f000a3bb0 0x7f0f000a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:17.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:17.747+0000 7f0f177fe640 1 --2- 172.21.15.47:0/2945331400 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f0f000a3bb0 0x7f0f000a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:17.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:17.747+0000 7f0f157fa640 1 -- 172.21.15.47:0/2945331400 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f0f080d0160 con 0x7f0f18102a40 2024-04-17T05:59:17.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:17.748+0000 7f0f177fe640 1 --2- 172.21.15.47:0/2945331400 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f0f000a3bb0 0x7f0f000a6070 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f0f1819b910 tx=0x7f0f0c005d90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:59:17.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:17.752+0000 7f0f157fa640 1 -- 172.21.15.47:0/2945331400 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f0f08014030 con 0x7f0f18102a40 2024-04-17T05:59:17.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:17 smithi098 ceph-mon[36516]: pgmap v287: 97 pgs: 5 creating+peering, 36 active+clean, 56 unknown; 578 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 2 op/s 2024-04-17T05:59:17.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:17 smithi098 ceph-mon[36516]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-17T05:59:17.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:17 smithi098 ceph-mon[36516]: Cluster is now healthy 2024-04-17T05:59:17.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:17 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/860872304' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-17T05:59:17.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:17 smithi098 ceph-mon[36516]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-17T05:59:17.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:17 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/2773878743' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-17T05:59:17.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:17 smithi047 ceph-mon[32223]: pgmap v287: 97 pgs: 5 creating+peering, 36 active+clean, 56 unknown; 578 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 2 op/s 2024-04-17T05:59:17.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:17 smithi047 ceph-mon[32223]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-17T05:59:17.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:17 smithi047 ceph-mon[32223]: Cluster is now healthy 2024-04-17T05:59:17.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:17 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/860872304' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-17T05:59:17.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:17 smithi047 ceph-mon[32223]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-17T05:59:17.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:17 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/2773878743' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-17T05:59:17.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:17.978+0000 7f0f1e77c640 1 -- 172.21.15.47:0/2945331400 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f0f1802d5d0 con 0x7f0f18102a40 2024-04-17T05:59:18.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:17 smithi103 ceph-mon[36243]: pgmap v287: 97 pgs: 5 creating+peering, 36 active+clean, 56 unknown; 578 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 2 op/s 2024-04-17T05:59:18.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:17 smithi103 ceph-mon[36243]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-17T05:59:18.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:17 smithi103 ceph-mon[36243]: Cluster is now healthy 2024-04-17T05:59:18.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:17 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/860872304' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-17T05:59:18.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:17 smithi103 ceph-mon[36243]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-17T05:59:18.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:17 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/2773878743' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-17T05:59:18.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:18.615+0000 7f0f157fa640 1 -- 172.21.15.47:0/2945331400 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v19) ==== 121+0+0 (secure 0 0 0) 0x7f0f0809cc90 con 0x7f0f18102a40 2024-04-17T05:59:18.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:18.618+0000 7f0f1e77c640 1 -- 172.21.15.47:0/2945331400 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f0f000a3bb0 msgr2=0x7f0f000a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:18.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:18.618+0000 7f0f1e77c640 1 --2- 172.21.15.47:0/2945331400 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f0f000a3bb0 0x7f0f000a6070 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f0f1819b910 tx=0x7f0f0c005d90 comp rx=0 tx=0).stop 2024-04-17T05:59:18.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:18.618+0000 7f0f1e77c640 1 -- 172.21.15.47:0/2945331400 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f18102a40 msgr2=0x7f0f1819a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:18.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:18.618+0000 7f0f1e77c640 1 --2- 172.21.15.47:0/2945331400 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f18102a40 0x7f0f1819a400 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f0f0800c970 tx=0x7f0f0800ce30 comp rx=0 tx=0).stop 2024-04-17T05:59:18.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:18.618+0000 7f0f1e77c640 1 -- 172.21.15.47:0/2945331400 shutdown_connections 2024-04-17T05:59:18.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:18.618+0000 7f0f1e77c640 1 --2- 172.21.15.47:0/2945331400 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f0f000a3bb0 0x7f0f000a6070 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:18.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:18.618+0000 7f0f1e77c640 1 --2- 172.21.15.47:0/2945331400 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0f18107ca0 0x7f0f181a19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:18.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:18.618+0000 7f0f1e77c640 1 --2- 172.21.15.47:0/2945331400 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0f18105370 0x7f0f1819a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:18.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:18.618+0000 7f0f1e77c640 1 --2- 172.21.15.47:0/2945331400 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f18102a40 0x7f0f1819a400 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:18.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:18.618+0000 7f0f1e77c640 1 -- 172.21.15.47:0/2945331400 >> 172.21.15.47:0/2945331400 conn(0x7f0f180fc450 msgr2=0x7f0f18102080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:59:18.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:18.619+0000 7f0f1e77c640 1 -- 172.21.15.47:0/2945331400 shutdown_connections 2024-04-17T05:59:18.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:18.619+0000 7f0f1e77c640 1 -- 172.21.15.47:0/2945331400 wait complete. 2024-04-17T05:59:18.635 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-04-17T05:59:18.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:18.750+0000 7fa66c959640 1 -- 172.21.15.47:0/3382633637 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa668105370 msgr2=0x7fa668107760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:18.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:18.750+0000 7fa66c959640 1 --2- 172.21.15.47:0/3382633637 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa668105370 0x7fa668107760 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa650009900 tx=0x7fa65002bc30 comp rx=0 tx=0).stop 2024-04-17T05:59:18.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:18.750+0000 7fa66c959640 1 -- 172.21.15.47:0/3382633637 shutdown_connections 2024-04-17T05:59:18.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:18.750+0000 7fa66c959640 1 --2- 172.21.15.47:0/3382633637 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa668107ca0 0x7fa66810a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:18.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:18.750+0000 7fa66c959640 1 --2- 172.21.15.47:0/3382633637 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa668105370 0x7fa668107760 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:18.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:18.750+0000 7fa66c959640 1 --2- 172.21.15.47:0/3382633637 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa668102a40 0x7fa668104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:18.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:18.750+0000 7fa66c959640 1 -- 172.21.15.47:0/3382633637 >> 172.21.15.47:0/3382633637 conn(0x7fa6680fc450 msgr2=0x7fa6680fe870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:59:18.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:18.750+0000 7fa66c959640 1 -- 172.21.15.47:0/3382633637 shutdown_connections 2024-04-17T05:59:18.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:18.750+0000 7fa66c959640 1 -- 172.21.15.47:0/3382633637 wait complete. 2024-04-17T05:59:18.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:18.751+0000 7fa66c959640 1 Processor -- start 2024-04-17T05:59:18.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:18.751+0000 7fa66c959640 1 -- start start 2024-04-17T05:59:18.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:18.751+0000 7fa66c959640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa668102a40 0x7fa6681023b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:18.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:18.751+0000 7fa66c959640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa668105370 0x7fa6681009f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:18.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:18.751+0000 7fa66c959640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa668107ca0 0x7fa668100f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:18.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:18.751+0000 7fa66c959640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa66810dc50 con 0x7fa668107ca0 2024-04-17T05:59:18.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:18.751+0000 7fa66c959640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa66810dad0 con 0x7fa668102a40 2024-04-17T05:59:18.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:18.751+0000 7fa66c959640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa66810ddd0 con 0x7fa668105370 2024-04-17T05:59:18.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:18.751+0000 7fa666d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa668107ca0 0x7fa668100f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:18.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:18.751+0000 7fa666d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa668107ca0 0x7fa668100f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57886/0 (socket says 172.21.15.47:57886) 2024-04-17T05:59:18.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:18.751+0000 7fa666d9d640 1 -- 172.21.15.47:0/2331905684 learned_addr learned my addr 172.21.15.47:0/2331905684 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:59:18.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:18.751+0000 7fa66659c640 1 --2- 172.21.15.47:0/2331905684 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa668102a40 0x7fa6681023b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:18.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:18.752+0000 7fa666d9d640 1 -- 172.21.15.47:0/2331905684 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa668105370 msgr2=0x7fa6681009f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:18.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:18.752+0000 7fa65ffff640 1 --2- 172.21.15.47:0/2331905684 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa668105370 0x7fa6681009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:18.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:18.752+0000 7fa666d9d640 1 --2- 172.21.15.47:0/2331905684 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa668105370 0x7fa6681009f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:18.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:18.752+0000 7fa666d9d640 1 -- 172.21.15.47:0/2331905684 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa668102a40 msgr2=0x7fa6681023b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:18.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:18.752+0000 7fa666d9d640 1 --2- 172.21.15.47:0/2331905684 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa668102a40 0x7fa6681023b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:18.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:18.752+0000 7fa666d9d640 1 -- 172.21.15.47:0/2331905684 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6681017c0 con 0x7fa668107ca0 2024-04-17T05:59:18.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:18.752+0000 7fa666d9d640 1 --2- 172.21.15.47:0/2331905684 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa668107ca0 0x7fa668100f30 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa66000cad0 tx=0x7fa66000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:59:18.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:18.752+0000 7fa65f7fe640 1 -- 172.21.15.47:0/2331905684 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa66000ae20 con 0x7fa668107ca0 2024-04-17T05:59:18.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:18.752+0000 7fa65f7fe640 1 -- 172.21.15.47:0/2331905684 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa660005020 con 0x7fa668107ca0 2024-04-17T05:59:18.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:18.752+0000 7fa66c959640 1 -- 172.21.15.47:0/2331905684 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6681a2480 con 0x7fa668107ca0 2024-04-17T05:59:18.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:18.752+0000 7fa65f7fe640 1 -- 172.21.15.47:0/2331905684 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa66000f6f0 con 0x7fa668107ca0 2024-04-17T05:59:18.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:18.752+0000 7fa66c959640 1 -- 172.21.15.47:0/2331905684 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6681a2a20 con 0x7fa668107ca0 2024-04-17T05:59:18.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:18.753+0000 7fa66c959640 1 -- 172.21.15.47:0/2331905684 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6680fdf40 con 0x7fa668107ca0 2024-04-17T05:59:18.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:18.754+0000 7fa65f7fe640 1 -- 172.21.15.47:0/2331905684 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fa6600051d0 con 0x7fa668107ca0 2024-04-17T05:59:18.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:18.755+0000 7fa65f7fe640 1 --2- 172.21.15.47:0/2331905684 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fa6540a3c30 0x7fa6540a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:18.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:18.755+0000 7fa66659c640 1 --2- 172.21.15.47:0/2331905684 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fa6540a3c30 0x7fa6540a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:18.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:18.755+0000 7fa65f7fe640 1 -- 172.21.15.47:0/2331905684 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fa6600d06a0 con 0x7fa668107ca0 2024-04-17T05:59:18.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:18.755+0000 7fa66659c640 1 --2- 172.21.15.47:0/2331905684 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fa6540a3c30 0x7fa6540a60f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa658009bd0 tx=0x7fa658005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:59:18.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:18.759+0000 7fa65f7fe640 1 -- 172.21.15.47:0/2331905684 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fa660014030 con 0x7fa668107ca0 2024-04-17T05:59:18.882 INFO:journalctl@ceph.mds.b.smithi098.stdout:Apr 17 05:59:18 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-b[67363]: 2024-04-17T05:59:18.808+0000 7f4b2370a700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-17T05:59:18.883 INFO:journalctl@ceph.mds.b.smithi098.stdout:Apr 17 05:59:18 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-b[67363]: 2024-04-17T05:59:18.808+0000 7f4b2370a700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-04-17T05:59:18.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:18 smithi098 ceph-mon[36516]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-17T05:59:18.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:18 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/2773878743' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-17T05:59:18.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:18 smithi098 ceph-mon[36516]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-17T05:59:18.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:18 smithi098 ceph-mon[36516]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-17T05:59:18.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:18 smithi098 ceph-mon[36516]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-04-17T05:59:18.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:18 smithi098 ceph-mon[36516]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-04-17T05:59:18.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:18 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/2945331400' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-17T05:59:18.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:18 smithi047 ceph-mon[32223]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-17T05:59:18.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:18 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/2773878743' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-17T05:59:18.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:18 smithi047 ceph-mon[32223]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-17T05:59:18.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:18 smithi047 ceph-mon[32223]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-17T05:59:18.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:18 smithi047 ceph-mon[32223]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-04-17T05:59:18.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:18 smithi047 ceph-mon[32223]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-04-17T05:59:18.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:18 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/2945331400' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-17T05:59:18.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:18.986+0000 7fa66c959640 1 -- 172.21.15.47:0/2331905684 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7fa668062b60 con 0x7fa668107ca0 2024-04-17T05:59:19.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:18 smithi103 ceph-mon[36243]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-17T05:59:19.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:18 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/2773878743' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-17T05:59:19.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:18 smithi103 ceph-mon[36243]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-17T05:59:19.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:18 smithi103 ceph-mon[36243]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-17T05:59:19.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:18 smithi103 ceph-mon[36243]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-04-17T05:59:19.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:18 smithi103 ceph-mon[36243]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-04-17T05:59:19.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:18 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/2945331400' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-17T05:59:19.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:19.598+0000 7fa65f7fe640 1 -- 172.21.15.47:0/2331905684 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v21) ==== 115+0+0 (secure 0 0 0) 0x7fa66009d130 con 0x7fa668107ca0 2024-04-17T05:59:19.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:19.601+0000 7fa66c959640 1 -- 172.21.15.47:0/2331905684 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fa6540a3c30 msgr2=0x7fa6540a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:19.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:19.601+0000 7fa66c959640 1 --2- 172.21.15.47:0/2331905684 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fa6540a3c30 0x7fa6540a60f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa658009bd0 tx=0x7fa658005f90 comp rx=0 tx=0).stop 2024-04-17T05:59:19.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:19.601+0000 7fa66c959640 1 -- 172.21.15.47:0/2331905684 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa668107ca0 msgr2=0x7fa668100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:19.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:19.601+0000 7fa66c959640 1 --2- 172.21.15.47:0/2331905684 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa668107ca0 0x7fa668100f30 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa66000cad0 tx=0x7fa66000cf90 comp rx=0 tx=0).stop 2024-04-17T05:59:19.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:19.601+0000 7fa66c959640 1 -- 172.21.15.47:0/2331905684 shutdown_connections 2024-04-17T05:59:19.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:19.601+0000 7fa66c959640 1 --2- 172.21.15.47:0/2331905684 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fa6540a3c30 0x7fa6540a60f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:19.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:19.601+0000 7fa66c959640 1 --2- 172.21.15.47:0/2331905684 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa668107ca0 0x7fa668100f30 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:19.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:19.601+0000 7fa66c959640 1 --2- 172.21.15.47:0/2331905684 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa668105370 0x7fa6681009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:19.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:19.601+0000 7fa66c959640 1 --2- 172.21.15.47:0/2331905684 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa668102a40 0x7fa6681023b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:19.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:19.601+0000 7fa66c959640 1 -- 172.21.15.47:0/2331905684 >> 172.21.15.47:0/2331905684 conn(0x7fa6680fc450 msgr2=0x7fa668103380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:59:19.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:19.601+0000 7fa66c959640 1 -- 172.21.15.47:0/2331905684 shutdown_connections 2024-04-17T05:59:19.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:19.601+0000 7fa66c959640 1 -- 172.21.15.47:0/2331905684 wait complete. 2024-04-17T05:59:19.614 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated'} subvols for filesystem 'cephfs' 2024-04-17T05:59:19.614 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-04-17T05:59:19.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:19.731+0000 7fade660d640 1 -- 172.21.15.47:0/4273888621 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fade0102aa0 msgr2=0x7fade0104e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:19.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:19.731+0000 7fade660d640 1 --2- 172.21.15.47:0/4273888621 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fade0102aa0 0x7fade0104e90 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fadd0009900 tx=0x7fadd002bc30 comp rx=0 tx=0).stop 2024-04-17T05:59:19.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:19.731+0000 7fade660d640 1 -- 172.21.15.47:0/4273888621 shutdown_connections 2024-04-17T05:59:19.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:19.731+0000 7fade660d640 1 --2- 172.21.15.47:0/4273888621 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fade0107d00 0x7fade010a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:19.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:19.731+0000 7fade660d640 1 --2- 172.21.15.47:0/4273888621 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fade01053d0 0x7fade01077c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:19.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:19.731+0000 7fade660d640 1 --2- 172.21.15.47:0/4273888621 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fade0102aa0 0x7fade0104e90 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:19.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:19.731+0000 7fade660d640 1 -- 172.21.15.47:0/4273888621 >> 172.21.15.47:0/4273888621 conn(0x7fade00fc410 msgr2=0x7fade00fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:59:19.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:19.731+0000 7fade660d640 1 -- 172.21.15.47:0/4273888621 shutdown_connections 2024-04-17T05:59:19.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:19.731+0000 7fade660d640 1 -- 172.21.15.47:0/4273888621 wait complete. 2024-04-17T05:59:19.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:19.731+0000 7fade660d640 1 Processor -- start 2024-04-17T05:59:19.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:19.731+0000 7fade660d640 1 -- start start 2024-04-17T05:59:19.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:19.731+0000 7fade660d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fade0102aa0 0x7fade0198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:19.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:19.731+0000 7fade660d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fade01053d0 0x7fade0198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:19.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:19.731+0000 7fade660d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fade0107d00 0x7fade019f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:19.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:19.731+0000 7fade660d640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fade010dc30 con 0x7fade0102aa0 2024-04-17T05:59:19.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:19.731+0000 7fade660d640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fade010dab0 con 0x7fade01053d0 2024-04-17T05:59:19.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:19.731+0000 7fade660d640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fade010ddb0 con 0x7fade0107d00 2024-04-17T05:59:19.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:19.732+0000 7faddffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fade0102aa0 0x7fade0198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:19.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:19.732+0000 7faddffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fade0102aa0 0x7fade0198310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57902/0 (socket says 172.21.15.47:57902) 2024-04-17T05:59:19.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:19.732+0000 7faddffff640 1 -- 172.21.15.47:0/2592343478 learned_addr learned my addr 172.21.15.47:0/2592343478 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:59:19.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:19.732+0000 7fadd7fff640 1 --2- 172.21.15.47:0/2592343478 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fade01053d0 0x7fade0198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:19.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:19.732+0000 7faddffff640 1 -- 172.21.15.47:0/2592343478 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fade0107d00 msgr2=0x7fade019f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:19.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:19.732+0000 7fade4baa640 1 --2- 172.21.15.47:0/2592343478 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fade0107d00 0x7fade019f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:19.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:19.732+0000 7faddffff640 1 --2- 172.21.15.47:0/2592343478 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fade0107d00 0x7fade019f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:19.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:19.732+0000 7faddffff640 1 -- 172.21.15.47:0/2592343478 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fade01053d0 msgr2=0x7fade0198850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:19.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:19.732+0000 7faddffff640 1 --2- 172.21.15.47:0/2592343478 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fade01053d0 0x7fade0198850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:19.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:19.732+0000 7faddffff640 1 -- 172.21.15.47:0/2592343478 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fade019ff30 con 0x7fade0102aa0 2024-04-17T05:59:19.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:19.732+0000 7fadd7fff640 1 --2- 172.21.15.47:0/2592343478 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fade01053d0 0x7fade0198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T05:59:19.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:19.732+0000 7fade4baa640 1 --2- 172.21.15.47:0/2592343478 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fade0107d00 0x7fade019f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T05:59:19.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:19.732+0000 7faddffff640 1 --2- 172.21.15.47:0/2592343478 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fade0102aa0 0x7fade0198310 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fadd0009230 tx=0x7fadd0005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:59:19.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:19.732+0000 7fadddffb640 1 -- 172.21.15.47:0/2592343478 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd0041070 con 0x7fade0102aa0 2024-04-17T05:59:19.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:19.732+0000 7fadddffb640 1 -- 172.21.15.47:0/2592343478 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fadd00054e0 con 0x7fade0102aa0 2024-04-17T05:59:19.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:19.732+0000 7fade660d640 1 -- 172.21.15.47:0/2592343478 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fade01a01c0 con 0x7fade0102aa0 2024-04-17T05:59:19.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:19.732+0000 7fadddffb640 1 -- 172.21.15.47:0/2592343478 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd0034440 con 0x7fade0102aa0 2024-04-17T05:59:19.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:19.732+0000 7fade660d640 1 -- 172.21.15.47:0/2592343478 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fade01a06b0 con 0x7fade0102aa0 2024-04-17T05:59:19.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:19.733+0000 7fade660d640 1 -- 172.21.15.47:0/2592343478 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fade00fdf00 con 0x7fade0102aa0 2024-04-17T05:59:19.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:19.735+0000 7fadddffb640 1 -- 172.21.15.47:0/2592343478 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fadd0044070 con 0x7fade0102aa0 2024-04-17T05:59:19.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:19.736+0000 7fadddffb640 1 --2- 172.21.15.47:0/2592343478 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fadcc0a3bb0 0x7fadcc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:19.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:19.736+0000 7fadd7fff640 1 --2- 172.21.15.47:0/2592343478 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fadcc0a3bb0 0x7fadcc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:19.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:19.736+0000 7fadddffb640 1 -- 172.21.15.47:0/2592343478 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fadd00f2430 con 0x7fade0102aa0 2024-04-17T05:59:19.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:19.736+0000 7fadd7fff640 1 --2- 172.21.15.47:0/2592343478 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fadcc0a3bb0 0x7fadcc0a6070 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fadc80058c0 tx=0x7fadc8005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:59:19.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:19.740+0000 7fadddffb640 1 -- 172.21.15.47:0/2592343478 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fadd00bef60 con 0x7fade0102aa0 2024-04-17T05:59:19.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:19 smithi098 ceph-mon[36516]: pgmap v288: 97 pgs: 5 creating+peering, 78 active+clean, 14 unknown; 578 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 7 op/s 2024-04-17T05:59:19.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:19 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/2945331400' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-17T05:59:19.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:19 smithi098 ceph-mon[36516]: mds.? [v2:172.21.15.98:6836/1117440719,v1:172.21.15.98:6837/1117440719] up:active 2024-04-17T05:59:19.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:19 smithi098 ceph-mon[36516]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-17T05:59:19.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:19 smithi098 ceph-mon[36516]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-17T05:59:19.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:19 smithi098 ceph-mon[36516]: Cluster is now healthy 2024-04-17T05:59:19.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:19 smithi098 ceph-mon[36516]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-04-17T05:59:19.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:19 smithi098 ceph-mon[36516]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-04-17T05:59:19.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:19 smithi098 ceph-mon[36516]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-04-17T05:59:19.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:19 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/2331905684' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-17T05:59:19.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:19 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/2331905684' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-17T05:59:19.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:19 smithi098 ceph-mon[36516]: mds.? [v2:172.21.15.103:6838/1467157889,v1:172.21.15.103:6839/1467157889] up:active 2024-04-17T05:59:19.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:19 smithi098 ceph-mon[36516]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-17T05:59:19.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:19 smithi047 ceph-mon[32223]: pgmap v288: 97 pgs: 5 creating+peering, 78 active+clean, 14 unknown; 578 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 7 op/s 2024-04-17T05:59:19.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:19 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/2945331400' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-17T05:59:19.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:19 smithi047 ceph-mon[32223]: mds.? [v2:172.21.15.98:6836/1117440719,v1:172.21.15.98:6837/1117440719] up:active 2024-04-17T05:59:19.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:19 smithi047 ceph-mon[32223]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-17T05:59:19.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:19 smithi047 ceph-mon[32223]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-17T05:59:19.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:19 smithi047 ceph-mon[32223]: Cluster is now healthy 2024-04-17T05:59:19.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:19 smithi047 ceph-mon[32223]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-04-17T05:59:19.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:19 smithi047 ceph-mon[32223]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-04-17T05:59:19.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:19 smithi047 ceph-mon[32223]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-04-17T05:59:19.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:19 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/2331905684' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-17T05:59:19.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:19 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/2331905684' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-17T05:59:19.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:19 smithi047 ceph-mon[32223]: mds.? [v2:172.21.15.103:6838/1467157889,v1:172.21.15.103:6839/1467157889] up:active 2024-04-17T05:59:19.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:19 smithi047 ceph-mon[32223]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-17T05:59:19.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:19.954+0000 7fade660d640 1 -- 172.21.15.47:0/2592343478 --> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fade00623a0 con 0x7fadcc0a3bb0 2024-04-17T05:59:19.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:19.982+0000 7fadddffb640 1 -- 172.21.15.47:0/2592343478 <== mgr.14152 v2:172.21.15.47:6800/3157902806 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fade00623a0 con 0x7fadcc0a3bb0 2024-04-17T05:59:19.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:19.984+0000 7fade660d640 1 -- 172.21.15.47:0/2592343478 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fadcc0a3bb0 msgr2=0x7fadcc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:19.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:19.984+0000 7fade660d640 1 --2- 172.21.15.47:0/2592343478 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fadcc0a3bb0 0x7fadcc0a6070 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fadc80058c0 tx=0x7fadc8005850 comp rx=0 tx=0).stop 2024-04-17T05:59:19.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:19.984+0000 7fade660d640 1 -- 172.21.15.47:0/2592343478 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fade0102aa0 msgr2=0x7fade0198310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:19.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:19.984+0000 7fade660d640 1 --2- 172.21.15.47:0/2592343478 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fade0102aa0 0x7fade0198310 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fadd0009230 tx=0x7fadd0005dd0 comp rx=0 tx=0).stop 2024-04-17T05:59:19.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:19.984+0000 7fade660d640 1 -- 172.21.15.47:0/2592343478 shutdown_connections 2024-04-17T05:59:19.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:19.984+0000 7fade660d640 1 --2- 172.21.15.47:0/2592343478 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fadcc0a3bb0 0x7fadcc0a6070 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:19.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:19.984+0000 7fade660d640 1 --2- 172.21.15.47:0/2592343478 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fade0107d00 0x7fade019f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:19.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:19.984+0000 7fade660d640 1 --2- 172.21.15.47:0/2592343478 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fade01053d0 0x7fade0198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:19.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:19.984+0000 7fade660d640 1 --2- 172.21.15.47:0/2592343478 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fade0102aa0 0x7fade0198310 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:19.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:19.984+0000 7fade660d640 1 -- 172.21.15.47:0/2592343478 >> 172.21.15.47:0/2592343478 conn(0x7fade00fc410 msgr2=0x7fade0103520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:59:19.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:19.984+0000 7fade660d640 1 -- 172.21.15.47:0/2592343478 shutdown_connections 2024-04-17T05:59:19.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:19.985+0000 7fade660d640 1 -- 172.21.15.47:0/2592343478 wait complete. 2024-04-17T05:59:19.994 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-04-17T05:59:20.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:19 smithi103 ceph-mon[36243]: pgmap v288: 97 pgs: 5 creating+peering, 78 active+clean, 14 unknown; 578 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 7 op/s 2024-04-17T05:59:20.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:19 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/2945331400' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-17T05:59:20.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:19 smithi103 ceph-mon[36243]: mds.? [v2:172.21.15.98:6836/1117440719,v1:172.21.15.98:6837/1117440719] up:active 2024-04-17T05:59:20.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:19 smithi103 ceph-mon[36243]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-17T05:59:20.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:19 smithi103 ceph-mon[36243]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-17T05:59:20.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:19 smithi103 ceph-mon[36243]: Cluster is now healthy 2024-04-17T05:59:20.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:19 smithi103 ceph-mon[36243]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-04-17T05:59:20.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:19 smithi103 ceph-mon[36243]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-04-17T05:59:20.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:19 smithi103 ceph-mon[36243]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-04-17T05:59:20.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:19 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/2331905684' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-17T05:59:20.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:19 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/2331905684' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-17T05:59:20.035 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:19 smithi103 ceph-mon[36243]: mds.? [v2:172.21.15.103:6838/1467157889,v1:172.21.15.103:6839/1467157889] up:active 2024-04-17T05:59:20.035 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:19 smithi103 ceph-mon[36243]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-17T05:59:20.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.107+0000 7f247f869640 1 -- 172.21.15.47:0/4240524915 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2478101ff0 msgr2=0x7f2478102470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:20.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.107+0000 7f247f869640 1 --2- 172.21.15.47:0/4240524915 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2478101ff0 0x7f2478102470 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2468009900 tx=0x7f246802bc30 comp rx=0 tx=0).stop 2024-04-17T05:59:20.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.107+0000 7f247f869640 1 -- 172.21.15.47:0/4240524915 shutdown_connections 2024-04-17T05:59:20.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.107+0000 7f247f869640 1 --2- 172.21.15.47:0/4240524915 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f24781029b0 0x7f24781092d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:20.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.107+0000 7f247f869640 1 --2- 172.21.15.47:0/4240524915 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2478101ff0 0x7f2478102470 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:20.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.107+0000 7f247f869640 1 --2- 172.21.15.47:0/4240524915 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2478100ea0 0x7f24781012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:20.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.107+0000 7f247f869640 1 -- 172.21.15.47:0/4240524915 >> 172.21.15.47:0/4240524915 conn(0x7f24780fc410 msgr2=0x7f24780fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:59:20.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.107+0000 7f247f869640 1 -- 172.21.15.47:0/4240524915 shutdown_connections 2024-04-17T05:59:20.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.107+0000 7f247f869640 1 -- 172.21.15.47:0/4240524915 wait complete. 2024-04-17T05:59:20.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.108+0000 7f247f869640 1 Processor -- start 2024-04-17T05:59:20.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.108+0000 7f247f869640 1 -- start start 2024-04-17T05:59:20.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.108+0000 7f247f869640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2478100ea0 0x7f2478198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:20.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.108+0000 7f247f869640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2478101ff0 0x7f2478198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:20.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.108+0000 7f247f869640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f24781029b0 0x7f247819f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:20.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.108+0000 7f247f869640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f247810cb50 con 0x7f24781029b0 2024-04-17T05:59:20.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.108+0000 7f247f869640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f247810c9d0 con 0x7f2478101ff0 2024-04-17T05:59:20.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.108+0000 7f247f869640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f247810ccd0 con 0x7f2478100ea0 2024-04-17T05:59:20.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.108+0000 7f247de06640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f24781029b0 0x7f247819f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:20.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.108+0000 7f247de06640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f24781029b0 0x7f247819f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57926/0 (socket says 172.21.15.47:57926) 2024-04-17T05:59:20.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.108+0000 7f247de06640 1 -- 172.21.15.47:0/2926982312 learned_addr learned my addr 172.21.15.47:0/2926982312 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:59:20.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.108+0000 7f247ce04640 1 --2- 172.21.15.47:0/2926982312 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2478101ff0 0x7f2478198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:20.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.108+0000 7f247de06640 1 -- 172.21.15.47:0/2926982312 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2478100ea0 msgr2=0x7f2478198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:20.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.108+0000 7f247de06640 1 --2- 172.21.15.47:0/2926982312 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2478100ea0 0x7f2478198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:20.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.108+0000 7f247de06640 1 -- 172.21.15.47:0/2926982312 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2478101ff0 msgr2=0x7f2478198860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:20.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.108+0000 7f247de06640 1 --2- 172.21.15.47:0/2926982312 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2478101ff0 0x7f2478198860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:20.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.108+0000 7f247de06640 1 -- 172.21.15.47:0/2926982312 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f247819ff40 con 0x7f24781029b0 2024-04-17T05:59:20.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.108+0000 7f247ce04640 1 --2- 172.21.15.47:0/2926982312 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2478101ff0 0x7f2478198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T05:59:20.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.108+0000 7f247de06640 1 --2- 172.21.15.47:0/2926982312 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f24781029b0 0x7f247819f8d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f247000eb10 tx=0x7f247000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:59:20.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.109+0000 7f246e7fc640 1 -- 172.21.15.47:0/2926982312 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2470004b20 con 0x7f24781029b0 2024-04-17T05:59:20.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.109+0000 7f246e7fc640 1 -- 172.21.15.47:0/2926982312 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2470004cc0 con 0x7f24781029b0 2024-04-17T05:59:20.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.109+0000 7f247f869640 1 -- 172.21.15.47:0/2926982312 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24781a0230 con 0x7f24781029b0 2024-04-17T05:59:20.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.109+0000 7f246e7fc640 1 -- 172.21.15.47:0/2926982312 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2470010760 con 0x7f24781029b0 2024-04-17T05:59:20.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.109+0000 7f247f869640 1 -- 172.21.15.47:0/2926982312 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24781a0700 con 0x7f24781029b0 2024-04-17T05:59:20.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.110+0000 7f247f869640 1 -- 172.21.15.47:0/2926982312 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24781a09c0 con 0x7f24781029b0 2024-04-17T05:59:20.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.111+0000 7f246e7fc640 1 -- 172.21.15.47:0/2926982312 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f2470005000 con 0x7f24781029b0 2024-04-17T05:59:20.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.111+0000 7f246e7fc640 1 --2- 172.21.15.47:0/2926982312 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f24640a3bb0 0x7f24640a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:20.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.111+0000 7f247d605640 1 --2- 172.21.15.47:0/2926982312 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f24640a3bb0 0x7f24640a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:20.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.112+0000 7f246e7fc640 1 -- 172.21.15.47:0/2926982312 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f24700d0be0 con 0x7f24781029b0 2024-04-17T05:59:20.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.112+0000 7f247d605640 1 --2- 172.21.15.47:0/2926982312 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f24640a3bb0 0x7f24640a6070 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f24740058c0 tx=0x7f2474005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:59:20.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.115+0000 7f246e7fc640 1 -- 172.21.15.47:0/2926982312 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f2470053ff0 con 0x7f24781029b0 2024-04-17T05:59:20.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.332+0000 7f247f869640 1 -- 172.21.15.47:0/2926982312 --> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f24780fe640 con 0x7f24640a3bb0 2024-04-17T05:59:20.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.361+0000 7f246e7fc640 1 -- 172.21.15.47:0/2926982312 <== mgr.14152 v2:172.21.15.47:6800/3157902806 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f24780fe640 con 0x7f24640a3bb0 2024-04-17T05:59:20.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.363+0000 7f247f869640 1 -- 172.21.15.47:0/2926982312 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f24640a3bb0 msgr2=0x7f24640a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:20.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.363+0000 7f247f869640 1 --2- 172.21.15.47:0/2926982312 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f24640a3bb0 0x7f24640a6070 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f24740058c0 tx=0x7f2474005850 comp rx=0 tx=0).stop 2024-04-17T05:59:20.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.363+0000 7f247f869640 1 -- 172.21.15.47:0/2926982312 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f24781029b0 msgr2=0x7f247819f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:20.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.363+0000 7f247f869640 1 --2- 172.21.15.47:0/2926982312 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f24781029b0 0x7f247819f8d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f247000eb10 tx=0x7f247000c2d0 comp rx=0 tx=0).stop 2024-04-17T05:59:20.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.363+0000 7f247f869640 1 -- 172.21.15.47:0/2926982312 shutdown_connections 2024-04-17T05:59:20.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.363+0000 7f247f869640 1 --2- 172.21.15.47:0/2926982312 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f24640a3bb0 0x7f24640a6070 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:20.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.363+0000 7f247f869640 1 --2- 172.21.15.47:0/2926982312 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f24781029b0 0x7f247819f8d0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:20.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.363+0000 7f247f869640 1 --2- 172.21.15.47:0/2926982312 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2478101ff0 0x7f2478198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:20.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.363+0000 7f247f869640 1 --2- 172.21.15.47:0/2926982312 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2478100ea0 0x7f2478198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:20.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.363+0000 7f247f869640 1 -- 172.21.15.47:0/2926982312 >> 172.21.15.47:0/2926982312 conn(0x7f24780fc410 msgr2=0x7f24780fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:59:20.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.363+0000 7f247f869640 1 -- 172.21.15.47:0/2926982312 shutdown_connections 2024-04-17T05:59:20.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.363+0000 7f247f869640 1 -- 172.21.15.47:0/2926982312 wait complete. 2024-04-17T05:59:20.372 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 05:59:19 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T05:59:19.959+0000 7f298211c700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-17T05:59:20.372 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 05:59:19 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T05:59:19.959+0000 7f298211c700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-17T05:59:20.372 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 05:59:19 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T05:59:19.959+0000 7f298211c700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-17T05:59:20.372 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 05:59:19 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T05:59:19.959+0000 7f298211c700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-17T05:59:20.372 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 05:59:19 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T05:59:19.959+0000 7f298211c700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-17T05:59:20.373 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-04-17T05:59:20.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.484+0000 7fb5e7a74640 1 -- 172.21.15.47:0/3596351727 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5e0101ff0 msgr2=0x7fb5e0102470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:20.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.484+0000 7fb5e7a74640 1 --2- 172.21.15.47:0/3596351727 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5e0101ff0 0x7fb5e0102470 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb5d0009900 tx=0x7fb5d002bc30 comp rx=0 tx=0).stop 2024-04-17T05:59:20.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.485+0000 7fb5e7a74640 1 -- 172.21.15.47:0/3596351727 shutdown_connections 2024-04-17T05:59:20.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.485+0000 7fb5e7a74640 1 --2- 172.21.15.47:0/3596351727 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb5e01029b0 0x7fb5e01092d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:20.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.485+0000 7fb5e7a74640 1 --2- 172.21.15.47:0/3596351727 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5e0101ff0 0x7fb5e0102470 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:20.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.485+0000 7fb5e7a74640 1 --2- 172.21.15.47:0/3596351727 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb5e0100ea0 0x7fb5e01012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:20.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.485+0000 7fb5e7a74640 1 -- 172.21.15.47:0/3596351727 >> 172.21.15.47:0/3596351727 conn(0x7fb5e00fc410 msgr2=0x7fb5e00fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:59:20.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.485+0000 7fb5e7a74640 1 -- 172.21.15.47:0/3596351727 shutdown_connections 2024-04-17T05:59:20.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.485+0000 7fb5e7a74640 1 -- 172.21.15.47:0/3596351727 wait complete. 2024-04-17T05:59:20.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.485+0000 7fb5e7a74640 1 Processor -- start 2024-04-17T05:59:20.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.485+0000 7fb5e7a74640 1 -- start start 2024-04-17T05:59:20.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.485+0000 7fb5e7a74640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5e0100ea0 0x7fb5e0069c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:20.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.485+0000 7fb5e7a74640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb5e0101ff0 0x7fb5e019f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:20.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.485+0000 7fb5e7a74640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb5e01029b0 0x7fb5e01a19f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:20.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.485+0000 7fb5e7a74640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e010cb50 con 0x7fb5e0100ea0 2024-04-17T05:59:20.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.485+0000 7fb5e7a74640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e010c9d0 con 0x7fb5e01029b0 2024-04-17T05:59:20.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.485+0000 7fb5e7a74640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e010ccd0 con 0x7fb5e0101ff0 2024-04-17T05:59:20.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.485+0000 7fb5e5810640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5e0100ea0 0x7fb5e0069c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:20.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.485+0000 7fb5e5810640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5e0100ea0 0x7fb5e0069c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57938/0 (socket says 172.21.15.47:57938) 2024-04-17T05:59:20.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.485+0000 7fb5e5810640 1 -- 172.21.15.47:0/155519372 learned_addr learned my addr 172.21.15.47:0/155519372 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:59:20.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.486+0000 7fb5e6011640 1 --2- 172.21.15.47:0/155519372 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb5e01029b0 0x7fb5e01a19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:20.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.486+0000 7fb5e5810640 1 -- 172.21.15.47:0/155519372 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb5e0101ff0 msgr2=0x7fb5e019f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:20.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.486+0000 7fb5e5810640 1 --2- 172.21.15.47:0/155519372 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb5e0101ff0 0x7fb5e019f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:20.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.486+0000 7fb5e5810640 1 -- 172.21.15.47:0/155519372 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb5e01029b0 msgr2=0x7fb5e01a19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:20.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.486+0000 7fb5e5810640 1 --2- 172.21.15.47:0/155519372 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb5e01029b0 0x7fb5e01a19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:20.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.486+0000 7fb5e5810640 1 -- 172.21.15.47:0/155519372 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5e01a21e0 con 0x7fb5e0100ea0 2024-04-17T05:59:20.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.486+0000 7fb5e5810640 1 --2- 172.21.15.47:0/155519372 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5e0100ea0 0x7fb5e0069c60 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc00c970 tx=0x7fb5dc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:59:20.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.486+0000 7fb5d6ffd640 1 -- 172.21.15.47:0/155519372 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5dc007bb0 con 0x7fb5e0100ea0 2024-04-17T05:59:20.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.486+0000 7fb5d6ffd640 1 -- 172.21.15.47:0/155519372 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5dc007d50 con 0x7fb5e0100ea0 2024-04-17T05:59:20.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.486+0000 7fb5d6ffd640 1 -- 172.21.15.47:0/155519372 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5dc00f450 con 0x7fb5e0100ea0 2024-04-17T05:59:20.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.486+0000 7fb5e7a74640 1 -- 172.21.15.47:0/155519372 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5e01a2470 con 0x7fb5e0100ea0 2024-04-17T05:59:20.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.486+0000 7fb5e7a74640 1 -- 172.21.15.47:0/155519372 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5e01a29c0 con 0x7fb5e0100ea0 2024-04-17T05:59:20.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.487+0000 7fb5e7a74640 1 -- 172.21.15.47:0/155519372 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5e0108280 con 0x7fb5e0100ea0 2024-04-17T05:59:20.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.488+0000 7fb5d6ffd640 1 -- 172.21.15.47:0/155519372 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb5dc004f80 con 0x7fb5e0100ea0 2024-04-17T05:59:20.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.489+0000 7fb5d6ffd640 1 --2- 172.21.15.47:0/155519372 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fb5cc0a3c60 0x7fb5cc0a6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:20.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.489+0000 7fb5e500f640 1 --2- 172.21.15.47:0/155519372 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fb5cc0a3c60 0x7fb5cc0a6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:20.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.489+0000 7fb5d6ffd640 1 -- 172.21.15.47:0/155519372 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fb5dc0d0350 con 0x7fb5e0100ea0 2024-04-17T05:59:20.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.489+0000 7fb5e500f640 1 --2- 172.21.15.47:0/155519372 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fb5cc0a3c60 0x7fb5cc0a6120 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb5d002c170 tx=0x7fb5d0000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:59:20.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.492+0000 7fb5d6ffd640 1 -- 172.21.15.47:0/155519372 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fb5dc014030 con 0x7fb5e0100ea0 2024-04-17T05:59:20.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.710+0000 7fb5e7a74640 1 -- 172.21.15.47:0/155519372 --> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7fb5e0062430 con 0x7fb5cc0a3c60 2024-04-17T05:59:20.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.715+0000 7fb5d6ffd640 1 -- 172.21.15.47:0/155519372 <== mgr.14152 v2:172.21.15.47:6800/3157902806 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fb5e0062430 con 0x7fb5cc0a3c60 2024-04-17T05:59:20.715 INFO:teuthology.orchestra.run.smithi047.stdout:/volumes/qa/sv_0/30610cbe-faff-4ec9-a39e-b3b143390ce6 2024-04-17T05:59:20.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.717+0000 7fb5e7a74640 1 -- 172.21.15.47:0/155519372 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fb5cc0a3c60 msgr2=0x7fb5cc0a6120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:20.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.717+0000 7fb5e7a74640 1 --2- 172.21.15.47:0/155519372 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fb5cc0a3c60 0x7fb5cc0a6120 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb5d002c170 tx=0x7fb5d0000bc0 comp rx=0 tx=0).stop 2024-04-17T05:59:20.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.717+0000 7fb5e7a74640 1 -- 172.21.15.47:0/155519372 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5e0100ea0 msgr2=0x7fb5e0069c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:20.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.717+0000 7fb5e7a74640 1 --2- 172.21.15.47:0/155519372 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5e0100ea0 0x7fb5e0069c60 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc00c970 tx=0x7fb5dc00ce30 comp rx=0 tx=0).stop 2024-04-17T05:59:20.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.717+0000 7fb5e7a74640 1 -- 172.21.15.47:0/155519372 shutdown_connections 2024-04-17T05:59:20.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.717+0000 7fb5e7a74640 1 --2- 172.21.15.47:0/155519372 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fb5cc0a3c60 0x7fb5cc0a6120 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:20.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.717+0000 7fb5e7a74640 1 --2- 172.21.15.47:0/155519372 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb5e01029b0 0x7fb5e01a19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:20.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.717+0000 7fb5e7a74640 1 --2- 172.21.15.47:0/155519372 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb5e0101ff0 0x7fb5e019f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:20.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.717+0000 7fb5e7a74640 1 --2- 172.21.15.47:0/155519372 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5e0100ea0 0x7fb5e0069c60 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:20.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.717+0000 7fb5e7a74640 1 -- 172.21.15.47:0/155519372 >> 172.21.15.47:0/155519372 conn(0x7fb5e00fc410 msgr2=0x7fb5e0105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:59:20.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.717+0000 7fb5e7a74640 1 -- 172.21.15.47:0/155519372 shutdown_connections 2024-04-17T05:59:20.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.717+0000 7fb5e7a74640 1 -- 172.21.15.47:0/155519372 wait complete. 2024-04-17T05:59:20.727 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-04-17T05:59:20.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.839+0000 7fd3a67e9640 1 -- 172.21.15.47:0/3338815727 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3a0100ea0 msgr2=0x7fd3a01012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:20.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.839+0000 7fd3a67e9640 1 --2- 172.21.15.47:0/3338815727 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3a0100ea0 0x7fd3a01012c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd390009900 tx=0x7fd39002bc30 comp rx=0 tx=0).stop 2024-04-17T05:59:20.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.839+0000 7fd3a67e9640 1 -- 172.21.15.47:0/3338815727 shutdown_connections 2024-04-17T05:59:20.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.839+0000 7fd3a67e9640 1 --2- 172.21.15.47:0/3338815727 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd3a01029b0 0x7fd3a01092d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:20.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.839+0000 7fd3a67e9640 1 --2- 172.21.15.47:0/3338815727 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd3a0101ff0 0x7fd3a0102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:20.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.839+0000 7fd3a67e9640 1 --2- 172.21.15.47:0/3338815727 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3a0100ea0 0x7fd3a01012c0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:20.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.839+0000 7fd3a67e9640 1 -- 172.21.15.47:0/3338815727 >> 172.21.15.47:0/3338815727 conn(0x7fd3a00fc410 msgr2=0x7fd3a00fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:59:20.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.839+0000 7fd3a67e9640 1 -- 172.21.15.47:0/3338815727 shutdown_connections 2024-04-17T05:59:20.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.839+0000 7fd3a67e9640 1 -- 172.21.15.47:0/3338815727 wait complete. 2024-04-17T05:59:20.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.839+0000 7fd3a67e9640 1 Processor -- start 2024-04-17T05:59:20.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.839+0000 7fd3a67e9640 1 -- start start 2024-04-17T05:59:20.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.840+0000 7fd3a67e9640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd3a0100ea0 0x7fd3a0198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:20.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.840+0000 7fd3a67e9640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3a0101ff0 0x7fd3a0198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:20.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.840+0000 7fd3a67e9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd3a01029b0 0x7fd3a019f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:20.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.840+0000 7fd3a67e9640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fd3a010cb90 con 0x7fd3a0101ff0 2024-04-17T05:59:20.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.840+0000 7fd3a67e9640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd3a010ca10 con 0x7fd3a0100ea0 2024-04-17T05:59:20.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.840+0000 7fd3a67e9640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd3a010cd10 con 0x7fd3a01029b0 2024-04-17T05:59:20.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.840+0000 7fd39f7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3a0101ff0 0x7fd3a0198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:20.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.840+0000 7fd39f7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3a0101ff0 0x7fd3a0198870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57960/0 (socket says 172.21.15.47:57960) 2024-04-17T05:59:20.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.840+0000 7fd39f7fe640 1 -- 172.21.15.47:0/3995268445 learned_addr learned my addr 172.21.15.47:0/3995268445 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:59:20.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.840+0000 7fd39ffff640 1 --2- 172.21.15.47:0/3995268445 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd3a0100ea0 0x7fd3a0198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:20.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.840+0000 7fd3a4d86640 1 --2- 172.21.15.47:0/3995268445 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd3a01029b0 0x7fd3a019f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:20.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.840+0000 7fd39f7fe640 1 -- 172.21.15.47:0/3995268445 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd3a01029b0 msgr2=0x7fd3a019f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:20.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.840+0000 7fd39f7fe640 1 --2- 172.21.15.47:0/3995268445 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd3a01029b0 0x7fd3a019f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:20.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.840+0000 7fd39f7fe640 1 -- 172.21.15.47:0/3995268445 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd3a0100ea0 msgr2=0x7fd3a0198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:20.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.840+0000 7fd39f7fe640 1 --2- 172.21.15.47:0/3995268445 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd3a0100ea0 0x7fd3a0198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:20.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.840+0000 7fd39f7fe640 1 -- 172.21.15.47:0/3995268445 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3a019ff50 con 0x7fd3a0101ff0 2024-04-17T05:59:20.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.840+0000 7fd39ffff640 1 --2- 172.21.15.47:0/3995268445 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd3a0100ea0 0x7fd3a0198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T05:59:20.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.840+0000 7fd3a4d86640 1 --2- 172.21.15.47:0/3995268445 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd3a01029b0 0x7fd3a019f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T05:59:20.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.840+0000 7fd39f7fe640 1 --2- 172.21.15.47:0/3995268445 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3a0101ff0 0x7fd3a0198870 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd39400d950 tx=0x7fd39400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:59:20.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.840+0000 7fd39d7fa640 1 -- 172.21.15.47:0/3995268445 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd394017070 con 0x7fd3a0101ff0 2024-04-17T05:59:20.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.840+0000 7fd39d7fa640 1 -- 172.21.15.47:0/3995268445 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd394009c30 con 0x7fd3a0101ff0 2024-04-17T05:59:20.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.840+0000 7fd3a67e9640 1 -- 172.21.15.47:0/3995268445 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3a01a0240 con 0x7fd3a0101ff0 2024-04-17T05:59:20.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.840+0000 7fd39d7fa640 1 -- 172.21.15.47:0/3995268445 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd39400f600 con 0x7fd3a0101ff0 2024-04-17T05:59:20.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.840+0000 7fd3a67e9640 1 -- 172.21.15.47:0/3995268445 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3a01a0790 con 0x7fd3a0101ff0 2024-04-17T05:59:20.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.841+0000 7fd3a67e9640 1 -- 172.21.15.47:0/3995268445 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3a01a09f0 con 0x7fd3a0101ff0 2024-04-17T05:59:20.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.842+0000 7fd39d7fa640 1 -- 172.21.15.47:0/3995268445 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fd39401a070 con 0x7fd3a0101ff0 2024-04-17T05:59:20.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.843+0000 7fd39d7fa640 1 --2- 172.21.15.47:0/3995268445 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fd3880a3c30 0x7fd3880a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:20.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.843+0000 7fd39ffff640 1 --2- 172.21.15.47:0/3995268445 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fd3880a3c30 0x7fd3880a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:20.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.843+0000 7fd39d7fa640 1 -- 172.21.15.47:0/3995268445 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fd3940cfd80 con 0x7fd3a0101ff0 2024-04-17T05:59:20.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.844+0000 7fd39ffff640 1 --2- 172.21.15.47:0/3995268445 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fd3880a3c30 0x7fd3880a60f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd3900096c0 tx=0x7fd39002df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:59:20.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:20.848+0000 7fd39d7fa640 1 -- 172.21.15.47:0/3995268445 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fd394009de0 con 0x7fd3a0101ff0 2024-04-17T05:59:21.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.062+0000 7fd3a67e9640 1 -- 172.21.15.47:0/3995268445 --> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fd3a0069800 con 0x7fd3880a3c30 2024-04-17T05:59:21.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.092+0000 7fd39d7fa640 1 -- 172.21.15.47:0/3995268445 <== mgr.14152 v2:172.21.15.47:6800/3157902806 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd3a0069800 con 0x7fd3880a3c30 2024-04-17T05:59:21.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.094+0000 7fd3a67e9640 1 -- 172.21.15.47:0/3995268445 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fd3880a3c30 msgr2=0x7fd3880a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:21.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.094+0000 7fd3a67e9640 1 --2- 172.21.15.47:0/3995268445 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fd3880a3c30 0x7fd3880a60f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd3900096c0 tx=0x7fd39002df90 comp rx=0 tx=0).stop 2024-04-17T05:59:21.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.094+0000 7fd3a67e9640 1 -- 172.21.15.47:0/3995268445 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3a0101ff0 msgr2=0x7fd3a0198870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:21.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.094+0000 7fd3a67e9640 1 --2- 172.21.15.47:0/3995268445 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3a0101ff0 0x7fd3a0198870 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd39400d950 tx=0x7fd39400de10 comp rx=0 tx=0).stop 2024-04-17T05:59:21.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.094+0000 7fd3a67e9640 1 -- 172.21.15.47:0/3995268445 shutdown_connections 2024-04-17T05:59:21.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.094+0000 7fd3a67e9640 1 --2- 172.21.15.47:0/3995268445 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fd3880a3c30 0x7fd3880a60f0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:21.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.094+0000 7fd3a67e9640 1 --2- 172.21.15.47:0/3995268445 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd3a01029b0 0x7fd3a019f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:21.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.094+0000 7fd3a67e9640 1 --2- 172.21.15.47:0/3995268445 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3a0101ff0 0x7fd3a0198870 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:21.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.094+0000 7fd3a67e9640 1 --2- 172.21.15.47:0/3995268445 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd3a0100ea0 0x7fd3a0198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:21.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.094+0000 7fd3a67e9640 1 -- 172.21.15.47:0/3995268445 >> 172.21.15.47:0/3995268445 conn(0x7fd3a00fc410 msgr2=0x7fd3a00690f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:59:21.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.094+0000 7fd3a67e9640 1 -- 172.21.15.47:0/3995268445 shutdown_connections 2024-04-17T05:59:21.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.094+0000 7fd3a67e9640 1 -- 172.21.15.47:0/3995268445 wait complete. 2024-04-17T05:59:21.104 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-04-17T05:59:21.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.215+0000 7f5dc2b1c640 1 -- 172.21.15.47:0/2163811658 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5dbc100030 msgr2=0x7f5dbc1004b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:21.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.215+0000 7f5dc2b1c640 1 --2- 172.21.15.47:0/2163811658 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5dbc100030 0x7f5dbc1004b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f5da8009900 tx=0x7f5da802bc30 comp rx=0 tx=0).stop 2024-04-17T05:59:21.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.215+0000 7f5dc2b1c640 1 -- 172.21.15.47:0/2163811658 shutdown_connections 2024-04-17T05:59:21.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.215+0000 7f5dc2b1c640 1 --2- 172.21.15.47:0/2163811658 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5dbc1009f0 0x7f5dbc10a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:21.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.215+0000 7f5dc2b1c640 1 --2- 172.21.15.47:0/2163811658 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5dbc100030 0x7f5dbc1004b0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:21.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.215+0000 7f5dc2b1c640 1 --2- 172.21.15.47:0/2163811658 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5dbc1035c0 0x7f5dbc1039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:21.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.215+0000 7f5dc2b1c640 1 -- 172.21.15.47:0/2163811658 >> 172.21.15.47:0/2163811658 conn(0x7f5dbc0fbb90 msgr2=0x7f5dbc0fdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:59:21.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.215+0000 7f5dc2b1c640 1 -- 172.21.15.47:0/2163811658 shutdown_connections 2024-04-17T05:59:21.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.215+0000 7f5dc2b1c640 1 -- 172.21.15.47:0/2163811658 wait complete. 2024-04-17T05:59:21.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.216+0000 7f5dc2b1c640 1 Processor -- start 2024-04-17T05:59:21.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.216+0000 7f5dc2b1c640 1 -- start start 2024-04-17T05:59:21.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.216+0000 7f5dc2b1c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5dbc100030 0x7f5dbc102cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:21.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.216+0000 7f5dc2b1c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5dbc1009f0 0x7f5dbc101300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:21.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.216+0000 7f5dc2b1c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5dbc1035c0 0x7f5dbc101840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:21.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.216+0000 7f5dc2b1c640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f5dbc10dcf0 con 0x7f5dbc100030 2024-04-17T05:59:21.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.216+0000 7f5dc2b1c640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5dbc10db70 con 0x7f5dbc1035c0 2024-04-17T05:59:21.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.216+0000 7f5dc2b1c640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5dbc10de70 con 0x7f5dbc1009f0 2024-04-17T05:59:21.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.216+0000 7f5dc08b8640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5dbc100030 0x7f5dbc102cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:21.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.216+0000 7f5dc08b8640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5dbc100030 0x7f5dbc102cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57966/0 (socket says 172.21.15.47:57966) 2024-04-17T05:59:21.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.216+0000 7f5dc08b8640 1 -- 172.21.15.47:0/931974707 learned_addr learned my addr 172.21.15.47:0/931974707 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:59:21.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.216+0000 7f5dbbfff640 1 --2- 172.21.15.47:0/931974707 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5dbc1009f0 0x7f5dbc101300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:21.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.216+0000 7f5dc10b9640 1 --2- 172.21.15.47:0/931974707 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5dbc1035c0 0x7f5dbc101840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:21.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.216+0000 7f5dc08b8640 1 -- 172.21.15.47:0/931974707 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5dbc1009f0 msgr2=0x7f5dbc101300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:21.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.216+0000 7f5dc08b8640 1 --2- 172.21.15.47:0/931974707 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5dbc1009f0 0x7f5dbc101300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:21.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.216+0000 7f5dc08b8640 1 -- 172.21.15.47:0/931974707 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5dbc1035c0 msgr2=0x7f5dbc101840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:21.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.216+0000 7f5dc08b8640 1 --2- 172.21.15.47:0/931974707 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5dbc1035c0 0x7f5dbc101840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:21.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.216+0000 7f5dc08b8640 1 -- 172.21.15.47:0/931974707 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dbc1020d0 con 0x7f5dbc100030 2024-04-17T05:59:21.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.216+0000 7f5dbbfff640 1 --2- 172.21.15.47:0/931974707 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5dbc1009f0 0x7f5dbc101300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T05:59:21.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.216+0000 7f5dc08b8640 1 --2- 172.21.15.47:0/931974707 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5dbc100030 0x7f5dbc102cc0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f5db000d950 tx=0x7f5db000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:59:21.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.217+0000 7f5db9ffb640 1 -- 172.21.15.47:0/931974707 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db0017020 con 0x7f5dbc100030 2024-04-17T05:59:21.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.217+0000 7f5db9ffb640 1 -- 172.21.15.47:0/931974707 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5db0009c30 con 0x7f5dbc100030 2024-04-17T05:59:21.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.217+0000 7f5db9ffb640 1 -- 172.21.15.47:0/931974707 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db000f740 con 0x7f5dbc100030 2024-04-17T05:59:21.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.217+0000 7f5dc2b1c640 1 -- 172.21.15.47:0/931974707 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dbc1a23f0 con 0x7f5dbc100030 2024-04-17T05:59:21.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.217+0000 7f5dc2b1c640 1 -- 172.21.15.47:0/931974707 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dbc1a28c0 con 0x7f5dbc100030 2024-04-17T05:59:21.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.218+0000 7f5dc2b1c640 1 -- 172.21.15.47:0/931974707 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5dbc0691d0 con 0x7f5dbc100030 2024-04-17T05:59:21.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.219+0000 7f5db9ffb640 1 -- 172.21.15.47:0/931974707 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f5db001a020 con 0x7f5dbc100030 2024-04-17T05:59:21.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.220+0000 7f5db9ffb640 1 --2- 172.21.15.47:0/931974707 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f5dac0a3c30 0x7f5dac0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:21.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.220+0000 7f5dbbfff640 1 --2- 172.21.15.47:0/931974707 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f5dac0a3c30 0x7f5dac0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:21.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.220+0000 7f5db9ffb640 1 -- 172.21.15.47:0/931974707 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f5db00cfc20 con 0x7f5dbc100030 2024-04-17T05:59:21.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.220+0000 7f5dbbfff640 1 --2- 172.21.15.47:0/931974707 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f5dac0a3c30 0x7f5dac0a60f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5da802c240 tx=0x7f5da802df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:59:21.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.224+0000 7f5db9ffb640 1 -- 172.21.15.47:0/931974707 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f5db0021070 con 0x7f5dbc100030 2024-04-17T05:59:21.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.440+0000 7f5dc2b1c640 1 -- 172.21.15.47:0/931974707 --> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f5dbc0624f0 con 0x7f5dac0a3c30 2024-04-17T05:59:21.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.445+0000 7f5db9ffb640 1 -- 172.21.15.47:0/931974707 <== mgr.14152 v2:172.21.15.47:6800/3157902806 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f5dbc0624f0 con 0x7f5dac0a3c30 2024-04-17T05:59:21.444 INFO:teuthology.orchestra.run.smithi047.stdout:/volumes/qa/sv_1/8477d852-ccf4-4ec5-8586-1c752352b391 2024-04-17T05:59:21.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.447+0000 7f5dc2b1c640 1 -- 172.21.15.47:0/931974707 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f5dac0a3c30 msgr2=0x7f5dac0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:21.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.447+0000 7f5dc2b1c640 1 --2- 172.21.15.47:0/931974707 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f5dac0a3c30 0x7f5dac0a60f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5da802c240 tx=0x7f5da802df90 comp rx=0 tx=0).stop 2024-04-17T05:59:21.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.447+0000 7f5dc2b1c640 1 -- 172.21.15.47:0/931974707 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5dbc100030 msgr2=0x7f5dbc102cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:21.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.447+0000 7f5dc2b1c640 1 --2- 172.21.15.47:0/931974707 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5dbc100030 0x7f5dbc102cc0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f5db000d950 tx=0x7f5db000de10 comp rx=0 tx=0).stop 2024-04-17T05:59:21.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.447+0000 7f5dc2b1c640 1 -- 172.21.15.47:0/931974707 shutdown_connections 2024-04-17T05:59:21.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.447+0000 7f5dc2b1c640 1 --2- 172.21.15.47:0/931974707 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f5dac0a3c30 0x7f5dac0a60f0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:21.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.447+0000 7f5dc2b1c640 1 --2- 172.21.15.47:0/931974707 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5dbc1035c0 0x7f5dbc101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:21.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.447+0000 7f5dc2b1c640 1 --2- 172.21.15.47:0/931974707 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5dbc1009f0 0x7f5dbc101300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:21.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.447+0000 7f5dc2b1c640 1 --2- 172.21.15.47:0/931974707 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5dbc100030 0x7f5dbc102cc0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:21.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.447+0000 7f5dc2b1c640 1 -- 172.21.15.47:0/931974707 >> 172.21.15.47:0/931974707 conn(0x7f5dbc0fbb90 msgr2=0x7f5dbc106650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:59:21.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.447+0000 7f5dc2b1c640 1 -- 172.21.15.47:0/931974707 shutdown_connections 2024-04-17T05:59:21.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.447+0000 7f5dc2b1c640 1 -- 172.21.15.47:0/931974707 wait complete. 2024-04-17T05:59:21.457 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T05:59:21.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.564+0000 7fafc7470640 1 -- 172.21.15.47:0/4186594320 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fafc0100ea0 msgr2=0x7fafc01012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:21.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.564+0000 7fafc7470640 1 --2- 172.21.15.47:0/4186594320 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fafc0100ea0 0x7fafc01012c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fafbc009900 tx=0x7fafbc02bc30 comp rx=0 tx=0).stop 2024-04-17T05:59:21.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.564+0000 7fafc7470640 1 -- 172.21.15.47:0/4186594320 shutdown_connections 2024-04-17T05:59:21.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.564+0000 7fafc7470640 1 --2- 172.21.15.47:0/4186594320 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fafc01029b0 0x7fafc01092d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:21.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.564+0000 7fafc7470640 1 --2- 172.21.15.47:0/4186594320 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fafc0101ff0 0x7fafc0102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:21.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.564+0000 7fafc7470640 1 --2- 172.21.15.47:0/4186594320 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fafc0100ea0 0x7fafc01012c0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:21.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.564+0000 7fafc7470640 1 -- 172.21.15.47:0/4186594320 >> 172.21.15.47:0/4186594320 conn(0x7fafc00fc410 msgr2=0x7fafc00fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:59:21.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.564+0000 7fafc7470640 1 -- 172.21.15.47:0/4186594320 shutdown_connections 2024-04-17T05:59:21.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.564+0000 7fafc7470640 1 -- 172.21.15.47:0/4186594320 wait complete. 2024-04-17T05:59:21.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.565+0000 7fafc7470640 1 Processor -- start 2024-04-17T05:59:21.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.565+0000 7fafc7470640 1 -- start start 2024-04-17T05:59:21.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.565+0000 7fafc7470640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fafc0100ea0 0x7fafc0069c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:21.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.565+0000 7fafc7470640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fafc0101ff0 0x7fafc019cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:21.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.565+0000 7fafc7470640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fafc01029b0 0x7fafc019f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:21.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.565+0000 7fafc7470640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fafc010cb20 con 0x7fafc01029b0 2024-04-17T05:59:21.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.565+0000 7fafc7470640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fafc010c9a0 con 0x7fafc0101ff0 2024-04-17T05:59:21.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.565+0000 7fafc7470640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fafc010cca0 con 0x7fafc0100ea0 2024-04-17T05:59:21.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.565+0000 7fafc5a0d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fafc01029b0 0x7fafc019f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:21.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.565+0000 7fafc520c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fafc0100ea0 0x7fafc0069c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:21.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.565+0000 7fafc5a0d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fafc01029b0 0x7fafc019f5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57994/0 (socket says 172.21.15.47:57994) 2024-04-17T05:59:21.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.565+0000 7fafc5a0d640 1 -- 172.21.15.47:0/2187541661 learned_addr learned my addr 172.21.15.47:0/2187541661 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:59:21.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.565+0000 7fafc4a0b640 1 --2- 172.21.15.47:0/2187541661 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fafc0101ff0 0x7fafc019cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:21.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.565+0000 7fafc5a0d640 1 -- 172.21.15.47:0/2187541661 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fafc0100ea0 msgr2=0x7fafc0069c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:21.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.565+0000 7fafc5a0d640 1 --2- 172.21.15.47:0/2187541661 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fafc0100ea0 0x7fafc0069c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:21.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.565+0000 7fafc5a0d640 1 -- 172.21.15.47:0/2187541661 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fafc0101ff0 msgr2=0x7fafc019cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:21.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.565+0000 7fafc5a0d640 1 --2- 172.21.15.47:0/2187541661 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fafc0101ff0 0x7fafc019cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:21.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.565+0000 7fafc5a0d640 1 -- 172.21.15.47:0/2187541661 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafc019fe70 con 0x7fafc01029b0 2024-04-17T05:59:21.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.565+0000 7fafc4a0b640 1 --2- 172.21.15.47:0/2187541661 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fafc0101ff0 0x7fafc019cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T05:59:21.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.565+0000 7fafc5a0d640 1 --2- 172.21.15.47:0/2187541661 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fafc01029b0 0x7fafc019f5b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fafb800eb10 tx=0x7fafb800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:59:21.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.566+0000 7fafb67fc640 1 -- 172.21.15.47:0/2187541661 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb8004b20 con 0x7fafc01029b0 2024-04-17T05:59:21.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.566+0000 7fafb67fc640 1 -- 172.21.15.47:0/2187541661 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafb8004cc0 con 0x7fafc01029b0 2024-04-17T05:59:21.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.566+0000 7fafc7470640 1 -- 172.21.15.47:0/2187541661 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafc01a0160 con 0x7fafc01029b0 2024-04-17T05:59:21.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.566+0000 7fafb67fc640 1 -- 172.21.15.47:0/2187541661 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb8010760 con 0x7fafc01029b0 2024-04-17T05:59:21.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.566+0000 7fafc7470640 1 -- 172.21.15.47:0/2187541661 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafc01a0660 con 0x7fafc01029b0 2024-04-17T05:59:21.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.567+0000 7fafc7470640 1 -- 172.21.15.47:0/2187541661 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafc0108150 con 0x7fafc01029b0 2024-04-17T05:59:21.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.568+0000 7fafb67fc640 1 -- 172.21.15.47:0/2187541661 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fafb8005020 con 0x7fafc01029b0 2024-04-17T05:59:21.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.568+0000 7fafb67fc640 1 --2- 172.21.15.47:0/2187541661 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fafac0a3bb0 0x7fafac0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:21.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.569+0000 7fafc520c640 1 --2- 172.21.15.47:0/2187541661 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fafac0a3bb0 0x7fafac0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:21.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.569+0000 7fafb67fc640 1 -- 172.21.15.47:0/2187541661 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fafb80d08d0 con 0x7fafc01029b0 2024-04-17T05:59:21.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.569+0000 7fafc520c640 1 --2- 172.21.15.47:0/2187541661 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fafac0a3bb0 0x7fafac0a6070 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fafbc02c7b0 tx=0x7fafbc005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:59:21.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.573+0000 7fafb67fc640 1 -- 172.21.15.47:0/2187541661 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fafb809d400 con 0x7fafc01029b0 2024-04-17T05:59:21.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.804+0000 7fafc7470640 1 -- 172.21.15.47:0/2187541661 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fafc002d5d0 con 0x7fafc01029b0 2024-04-17T05:59:21.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.805+0000 7fafb67fc640 1 -- 172.21.15.47:0/2187541661 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11881 (secure 0 0 0) 0x7fafb809cd00 con 0x7fafc01029b0 2024-04-17T05:59:21.804 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-17T05:59:21.804 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3189958720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3189958720},{"type":"v1","addr":"172.21.15.47:6835","nonce":3189958720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14595,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/3143529973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":3143529973},{"type":"v1","addr":"172.21.15.47:6837","nonce":3143529973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14652,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/2375823701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":2375823701},{"type":"v1","addr":"172.21.15.47:6839","nonce":2375823701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14664,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6841/3334309157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":3334309157},{"type":"v1","addr":"172.21.15.47:6841","nonce":3334309157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/2398363754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":2398363754},{"type":"v1","addr":"172.21.15.103:6833","nonce":2398363754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/384473493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":384473493},{"type":"v1","addr":"172.21.15.98:6833","nonce":384473493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T05:59:13.922176+0000","modified":"2024-04-17T05:59:19.598261+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24518,"mds_1":24479,"mds_2":24466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6835/270543522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":270543522},{"type":"v1","addr":"172.21.15.103:6835","nonce":270543522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6837/116936024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":116936024},{"type":"v1","addr":"172.21.15.103:6837","nonce":116936024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6835/1966500278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":1966500278},{"type":"v1","addr":"172.21.15.98:6835","nonce":1966500278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.103:6839/1467157889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":1467157889},{"type":"v1","addr":"172.21.15.103:6839","nonce":1467157889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.98:6837/1117440719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1117440719},{"type":"v1","addr":"172.21.15.98:6837","nonce":1117440719}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6839/2841287917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":2841287917},{"type":"v1","addr":"172.21.15.98:6839","nonce":2841287917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24518,"qdb_cluster":[24479,24518,24466]},"id":1}]} 2024-04-17T05:59:21.804 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 21 2024-04-17T05:59:21.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.808+0000 7fafc7470640 1 -- 172.21.15.47:0/2187541661 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fafac0a3bb0 msgr2=0x7fafac0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:21.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.808+0000 7fafc7470640 1 --2- 172.21.15.47:0/2187541661 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fafac0a3bb0 0x7fafac0a6070 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fafbc02c7b0 tx=0x7fafbc005fd0 comp rx=0 tx=0).stop 2024-04-17T05:59:21.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.808+0000 7fafc7470640 1 -- 172.21.15.47:0/2187541661 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fafc01029b0 msgr2=0x7fafc019f5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:21.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.808+0000 7fafc7470640 1 --2- 172.21.15.47:0/2187541661 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fafc01029b0 0x7fafc019f5b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fafb800eb10 tx=0x7fafb800c2d0 comp rx=0 tx=0).stop 2024-04-17T05:59:21.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.808+0000 7fafc7470640 1 -- 172.21.15.47:0/2187541661 shutdown_connections 2024-04-17T05:59:21.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.808+0000 7fafc7470640 1 --2- 172.21.15.47:0/2187541661 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fafac0a3bb0 0x7fafac0a6070 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:21.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.808+0000 7fafc7470640 1 --2- 172.21.15.47:0/2187541661 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fafc01029b0 0x7fafc019f5b0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:21.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.808+0000 7fafc7470640 1 --2- 172.21.15.47:0/2187541661 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fafc0101ff0 0x7fafc019cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:21.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.808+0000 7fafc7470640 1 --2- 172.21.15.47:0/2187541661 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fafc0100ea0 0x7fafc0069c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:21.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.808+0000 7fafc7470640 1 -- 172.21.15.47:0/2187541661 >> 172.21.15.47:0/2187541661 conn(0x7fafc00fc410 msgr2=0x7fafc0105320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:59:21.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.808+0000 7fafc7470640 1 -- 172.21.15.47:0/2187541661 shutdown_connections 2024-04-17T05:59:21.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.808+0000 7fafc7470640 1 -- 172.21.15.47:0/2187541661 wait complete. 2024-04-17T05:59:21.816 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:21 smithi047 ceph-mon[32223]: pgmap v289: 97 pgs: 97 active+clean; 580 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 994 B/s rd, 3.1 KiB/s wr, 13 op/s 2024-04-17T05:59:21.816 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:21 smithi047 ceph-mon[32223]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T05:59:21.816 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:21 smithi047 ceph-mon[32223]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T05:59:21.818 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T05:59:21.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:21 smithi098 ceph-mon[36516]: pgmap v289: 97 pgs: 97 active+clean; 580 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 994 B/s rd, 3.1 KiB/s wr, 13 op/s 2024-04-17T05:59:21.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:21 smithi098 ceph-mon[36516]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T05:59:21.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:21 smithi098 ceph-mon[36516]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T05:59:21.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.924+0000 7fb498cd9640 1 -- 172.21.15.47:0/1360941597 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb494100ea0 msgr2=0x7fb4941012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:21.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.924+0000 7fb498cd9640 1 --2- 172.21.15.47:0/1360941597 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb494100ea0 0x7fb4941012c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb484009900 tx=0x7fb48402bc30 comp rx=0 tx=0).stop 2024-04-17T05:59:21.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.924+0000 7fb498cd9640 1 -- 172.21.15.47:0/1360941597 shutdown_connections 2024-04-17T05:59:21.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.924+0000 7fb498cd9640 1 --2- 172.21.15.47:0/1360941597 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb4941029b0 0x7fb4941092d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:21.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.924+0000 7fb498cd9640 1 --2- 172.21.15.47:0/1360941597 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb494101ff0 0x7fb494102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:21.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.924+0000 7fb498cd9640 1 --2- 172.21.15.47:0/1360941597 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb494100ea0 0x7fb4941012c0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:21.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.924+0000 7fb498cd9640 1 -- 172.21.15.47:0/1360941597 >> 172.21.15.47:0/1360941597 conn(0x7fb4940fc410 msgr2=0x7fb4940fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:59:21.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.924+0000 7fb498cd9640 1 -- 172.21.15.47:0/1360941597 shutdown_connections 2024-04-17T05:59:21.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.924+0000 7fb498cd9640 1 -- 172.21.15.47:0/1360941597 wait complete. 2024-04-17T05:59:21.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.925+0000 7fb498cd9640 1 Processor -- start 2024-04-17T05:59:21.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.925+0000 7fb498cd9640 1 -- start start 2024-04-17T05:59:21.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.925+0000 7fb498cd9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb494100ea0 0x7fb4941983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:21.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.925+0000 7fb498cd9640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb494101ff0 0x7fb494198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:21.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.925+0000 7fb498cd9640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb4941029b0 0x7fb49419f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:21.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.925+0000 7fb498cd9640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb49410cbc0 con 0x7fb494101ff0 2024-04-17T05:59:21.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.925+0000 7fb498cd9640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb49410ca40 con 0x7fb4941029b0 2024-04-17T05:59:21.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.925+0000 7fb498cd9640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb49410cd40 con 0x7fb494100ea0 2024-04-17T05:59:21.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.925+0000 7fb48bfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb494101ff0 0x7fb494198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:21.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.925+0000 7fb48bfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb494101ff0 0x7fb494198900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58010/0 (socket says 172.21.15.47:58010) 2024-04-17T05:59:21.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.925+0000 7fb48bfff640 1 -- 172.21.15.47:0/2119612838 learned_addr learned my addr 172.21.15.47:0/2119612838 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:59:21.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.925+0000 7fb49259c640 1 --2- 172.21.15.47:0/2119612838 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb494100ea0 0x7fb4941983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:21.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.925+0000 7fb48bfff640 1 -- 172.21.15.47:0/2119612838 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb494100ea0 msgr2=0x7fb4941983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:21.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.925+0000 7fb48bfff640 1 --2- 172.21.15.47:0/2119612838 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb494100ea0 0x7fb4941983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:21.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.925+0000 7fb48bfff640 1 -- 172.21.15.47:0/2119612838 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb4941029b0 msgr2=0x7fb49419f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:21.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.926+0000 7fb48bfff640 1 --2- 172.21.15.47:0/2119612838 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb4941029b0 0x7fb49419f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:21.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.926+0000 7fb48bfff640 1 -- 172.21.15.47:0/2119612838 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb49419ffe0 con 0x7fb494101ff0 2024-04-17T05:59:21.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.926+0000 7fb49259c640 1 --2- 172.21.15.47:0/2119612838 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb494100ea0 0x7fb4941983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T05:59:21.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.926+0000 7fb48bfff640 1 --2- 172.21.15.47:0/2119612838 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb494101ff0 0x7fb494198900 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fb47c00b450 tx=0x7fb47c00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:59:21.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.926+0000 7fb48b7fe640 1 -- 172.21.15.47:0/2119612838 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb47c015020 con 0x7fb494101ff0 2024-04-17T05:59:21.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.926+0000 7fb48b7fe640 1 -- 172.21.15.47:0/2119612838 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb47c011d70 con 0x7fb494101ff0 2024-04-17T05:59:21.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.926+0000 7fb498cd9640 1 -- 172.21.15.47:0/2119612838 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4941a02d0 con 0x7fb494101ff0 2024-04-17T05:59:21.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.926+0000 7fb48b7fe640 1 -- 172.21.15.47:0/2119612838 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb47c00f340 con 0x7fb494101ff0 2024-04-17T05:59:21.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.926+0000 7fb498cd9640 1 -- 172.21.15.47:0/2119612838 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4941a0820 con 0x7fb494101ff0 2024-04-17T05:59:21.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.927+0000 7fb498cd9640 1 -- 172.21.15.47:0/2119612838 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4941a0ae0 con 0x7fb494101ff0 2024-04-17T05:59:21.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.928+0000 7fb48b7fe640 1 -- 172.21.15.47:0/2119612838 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb47c018020 con 0x7fb494101ff0 2024-04-17T05:59:21.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.929+0000 7fb48b7fe640 1 --2- 172.21.15.47:0/2119612838 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fb4800a3bb0 0x7fb4800a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:21.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.929+0000 7fb49259c640 1 --2- 172.21.15.47:0/2119612838 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fb4800a3bb0 0x7fb4800a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:21.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.929+0000 7fb48b7fe640 1 -- 172.21.15.47:0/2119612838 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fb47c0cfe80 con 0x7fb494101ff0 2024-04-17T05:59:21.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.930+0000 7fb49259c640 1 --2- 172.21.15.47:0/2119612838 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fb4800a3bb0 0x7fb4800a6070 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb484006010 tx=0x7fb484000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:59:21.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:21.934+0000 7fb48b7fe640 1 -- 172.21.15.47:0/2119612838 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fb47c01f020 con 0x7fb494101ff0 2024-04-17T05:59:22.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:21 smithi103 ceph-mon[36243]: pgmap v289: 97 pgs: 97 active+clean; 580 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 994 B/s rd, 3.1 KiB/s wr, 13 op/s 2024-04-17T05:59:22.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:21 smithi103 ceph-mon[36243]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T05:59:22.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:21 smithi103 ceph-mon[36243]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T05:59:22.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.122+0000 7fb498cd9640 1 -- 172.21.15.47:0/2119612838 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb4941a0cb0 con 0x7fb494101ff0 2024-04-17T05:59:22.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.123+0000 7fb48b7fe640 1 -- 172.21.15.47:0/2119612838 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22169 (secure 0 0 0) 0x7fb47c00d040 con 0x7fb494101ff0 2024-04-17T05:59:22.122 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-17T05:59:22.123 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":86,"fsid":"565db2d8-fc7e-11ee-bc91-c7b262605968","created":"2024-04-17T05:51:19.888865+0000","modified":"2024-04-17T05:59:15.578280+0000","last_up_change":"2024-04-17T05:58:24.380131+0000","last_in_change":"2024-04-17T05:58:04.418236+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T05:54:37.734306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T05:59:12.458010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T05:59:12.901339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8453f300-533d-4736-a343-55619c08e966","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":1290127544},{"type":"v1","addr":"172.21.15.47:6803","nonce":1290127544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":1290127544},{"type":"v1","addr":"172.21.15.47:6805","nonce":1290127544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":1290127544},{"type":"v1","addr":"172.21.15.47:6809","nonce":1290127544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":1290127544},{"type":"v1","addr":"172.21.15.47:6807","nonce":1290127544}]},"public_addr":"172.21.15.47:6803/1290127544","cluster_addr":"172.21.15.47:6805/1290127544","heartbeat_back_addr":"172.21.15.47:6809/1290127544","heartbeat_front_addr":"172.21.15.47:6807/1290127544","state":["exists","up"]},{"osd":1,"uuid":"80b0f03e-c1bc-4658-8028-1e6b8eedcc47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":1180430954},{"type":"v1","addr":"172.21.15.98:6801","nonce":1180430954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":1180430954},{"type":"v1","addr":"172.21.15.98:6803","nonce":1180430954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":1180430954},{"type":"v1","addr":"172.21.15.98:6807","nonce":1180430954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":1180430954},{"type":"v1","addr":"172.21.15.98:6805","nonce":1180430954}]},"public_addr":"172.21.15.98:6801/1180430954","cluster_addr":"172.21.15.98:6803/1180430954","heartbeat_back_addr":"172.21.15.98:6807/1180430954","heartbeat_front_addr":"172.21.15.98:6805/1180430954","state":["exists","up"]},{"osd":2,"uuid":"0fc9d383-05b4-498d-b530-fccf7aeddd7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":494676216},{"type":"v1","addr":"172.21.15.103:6801","nonce":494676216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":494676216},{"type":"v1","addr":"172.21.15.103:6803","nonce":494676216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":494676216},{"type":"v1","addr":"172.21.15.103:6807","nonce":494676216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":494676216},{"type":"v1","addr":"172.21.15.103:6805","nonce":494676216}]},"public_addr":"172.21.15.103:6801/494676216","cluster_addr":"172.21.15.103:6803/494676216","heartbeat_back_addr":"172.21.15.103:6807/494676216","heartbeat_front_addr":"172.21.15.103:6805/494676216","state":["exists","up"]},{"osd":3,"uuid":"f6e0b0d5-0fb3-42d0-9e84-f7cfef21a998","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":1998369447},{"type":"v1","addr":"172.21.15.47:6811","nonce":1998369447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":1998369447},{"type":"v1","addr":"172.21.15.47:6813","nonce":1998369447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":1998369447},{"type":"v1","addr":"172.21.15.47:6817","nonce":1998369447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":1998369447},{"type":"v1","addr":"172.21.15.47:6815","nonce":1998369447}]},"public_addr":"172.21.15.47:6811/1998369447","cluster_addr":"172.21.15.47:6813/1998369447","heartbeat_back_addr":"172.21.15.47:6817/1998369447","heartbeat_front_addr":"172.21.15.47:6815/1998369447","state":["exists","up"]},{"osd":4,"uuid":"2c26217e-02df-466d-bc01-0149d4aec3c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":3060482485},{"type":"v1","addr":"172.21.15.98:6809","nonce":3060482485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":3060482485},{"type":"v1","addr":"172.21.15.98:6811","nonce":3060482485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":3060482485},{"type":"v1","addr":"172.21.15.98:6815","nonce":3060482485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":3060482485},{"type":"v1","addr":"172.21.15.98:6813","nonce":3060482485}]},"public_addr":"172.21.15.98:6809/3060482485","cluster_addr":"172.21.15.98:6811/3060482485","heartbeat_back_addr":"172.21.15.98:6815/3060482485","heartbeat_front_addr":"172.21.15.98:6813/3060482485","state":["exists","up"]},{"osd":5,"uuid":"740bf3e0-bea8-496b-8d6f-e42d0b017258","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":4181363567},{"type":"v1","addr":"172.21.15.103:6809","nonce":4181363567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":4181363567},{"type":"v1","addr":"172.21.15.103:6811","nonce":4181363567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":4181363567},{"type":"v1","addr":"172.21.15.103:6815","nonce":4181363567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":4181363567},{"type":"v1","addr":"172.21.15.103:6813","nonce":4181363567}]},"public_addr":"172.21.15.103:6809/4181363567","cluster_addr":"172.21.15.103:6811/4181363567","heartbeat_back_addr":"172.21.15.103:6815/4181363567","heartbeat_front_addr":"172.21.15.103:6813/4181363567","state":["exists","up"]},{"osd":6,"uuid":"356a6255-2c07-4f0d-a233-26be04c92b38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":2517529668},{"type":"v1","addr":"172.21.15.47:6819","nonce":2517529668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2517529668},{"type":"v1","addr":"172.21.15.47:6821","nonce":2517529668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2517529668},{"type":"v1","addr":"172.21.15.47:6825","nonce":2517529668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2517529668},{"type":"v1","addr":"172.21.15.47:6823","nonce":2517529668}]},"public_addr":"172.21.15.47:6819/2517529668","cluster_addr":"172.21.15.47:6821/2517529668","heartbeat_back_addr":"172.21.15.47:6825/2517529668","heartbeat_front_addr":"172.21.15.47:6823/2517529668","state":["exists","up"]},{"osd":7,"uuid":"349d0120-8290-417d-a800-5a8985f554f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":4225498496},{"type":"v1","addr":"172.21.15.98:6817","nonce":4225498496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":4225498496},{"type":"v1","addr":"172.21.15.98:6819","nonce":4225498496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":4225498496},{"type":"v1","addr":"172.21.15.98:6823","nonce":4225498496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":4225498496},{"type":"v1","addr":"172.21.15.98:6821","nonce":4225498496}]},"public_addr":"172.21.15.98:6817/4225498496","cluster_addr":"172.21.15.98:6819/4225498496","heartbeat_back_addr":"172.21.15.98:6823/4225498496","heartbeat_front_addr":"172.21.15.98:6821/4225498496","state":["exists","up"]},{"osd":8,"uuid":"cee4712f-321b-42c0-92ba-e15144c00f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":3433905520},{"type":"v1","addr":"172.21.15.103:6817","nonce":3433905520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":3433905520},{"type":"v1","addr":"172.21.15.103:6819","nonce":3433905520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":3433905520},{"type":"v1","addr":"172.21.15.103:6823","nonce":3433905520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":3433905520},{"type":"v1","addr":"172.21.15.103:6821","nonce":3433905520}]},"public_addr":"172.21.15.103:6817/3433905520","cluster_addr":"172.21.15.103:6819/3433905520","heartbeat_back_addr":"172.21.15.103:6823/3433905520","heartbeat_front_addr":"172.21.15.103:6821/3433905520","state":["exists","up"]},{"osd":9,"uuid":"63fc73c1-5a22-487d-80b8-9894e2ddd207","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":728359554},{"type":"v1","addr":"172.21.15.47:6827","nonce":728359554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":728359554},{"type":"v1","addr":"172.21.15.47:6829","nonce":728359554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":728359554},{"type":"v1","addr":"172.21.15.47:6833","nonce":728359554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":728359554},{"type":"v1","addr":"172.21.15.47:6831","nonce":728359554}]},"public_addr":"172.21.15.47:6827/728359554","cluster_addr":"172.21.15.47:6829/728359554","heartbeat_back_addr":"172.21.15.47:6833/728359554","heartbeat_front_addr":"172.21.15.47:6831/728359554","state":["exists","up"]},{"osd":10,"uuid":"9565bcc3-62ee-48eb-a07a-03a81e0cc8e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":468205689},{"type":"v1","addr":"172.21.15.98:6825","nonce":468205689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":468205689},{"type":"v1","addr":"172.21.15.98:6827","nonce":468205689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":468205689},{"type":"v1","addr":"172.21.15.98:6831","nonce":468205689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":468205689},{"type":"v1","addr":"172.21.15.98:6829","nonce":468205689}]},"public_addr":"172.21.15.98:6825/468205689","cluster_addr":"172.21.15.98:6827/468205689","heartbeat_back_addr":"172.21.15.98:6831/468205689","heartbeat_front_addr":"172.21.15.98:6829/468205689","state":["exists","up"]},{"osd":11,"uuid":"2e85e8d9-bf6c-4a12-b124-a4690dd61108","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":2639831013},{"type":"v1","addr":"172.21.15.103:6825","nonce":2639831013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":2639831013},{"type":"v1","addr":"172.21.15.103:6827","nonce":2639831013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":2639831013},{"type":"v1","addr":"172.21.15.103:6831","nonce":2639831013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":2639831013},{"type":"v1","addr":"172.21.15.103:6829","nonce":2639831013}]},"public_addr":"172.21.15.103:6825/2639831013","cluster_addr":"172.21.15.103:6827/2639831013","heartbeat_back_addr":"172.21.15.103:6831/2639831013","heartbeat_front_addr":"172.21.15.103:6829/2639831013","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:53:47.366438+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:54:10.953559+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:54:34.320332+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:54:59.751360+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:55:23.982052+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:55:48.425017+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:56:14.793368+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:56:40.440830+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:57:04.623680+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:57:31.215262+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:57:57.040346+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:58:22.671572+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:6801/3632157175":"2024-04-18T05:52:13.645775+0000","172.21.15.47:0/4288219269":"2024-04-18T05:52:13.645775+0000","172.21.15.47:0/2584487544":"2024-04-18T05:52:13.645775+0000","172.21.15.47:6801/953064786":"2024-04-18T05:51:45.979351+0000","172.21.15.47:0/3484500927":"2024-04-18T05:51:45.979351+0000","172.21.15.47:6800/3632157175":"2024-04-18T05:52:13.645775+0000","172.21.15.47:0/112268670":"2024-04-18T05:51:45.979351+0000","172.21.15.47:0/1638423256":"2024-04-18T05:51:45.979351+0000","172.21.15.47:6800/953064786":"2024-04-18T05:51:45.979351+0000","172.21.15.47:0/849479010":"2024-04-18T05:52:13.645775+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T05:59:22.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.125+0000 7fb498cd9640 1 -- 172.21.15.47:0/2119612838 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fb4800a3bb0 msgr2=0x7fb4800a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:22.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.125+0000 7fb498cd9640 1 --2- 172.21.15.47:0/2119612838 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fb4800a3bb0 0x7fb4800a6070 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb484006010 tx=0x7fb484000bc0 comp rx=0 tx=0).stop 2024-04-17T05:59:22.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.125+0000 7fb498cd9640 1 -- 172.21.15.47:0/2119612838 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb494101ff0 msgr2=0x7fb494198900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:22.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.125+0000 7fb498cd9640 1 --2- 172.21.15.47:0/2119612838 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb494101ff0 0x7fb494198900 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fb47c00b450 tx=0x7fb47c00b910 comp rx=0 tx=0).stop 2024-04-17T05:59:22.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.125+0000 7fb498cd9640 1 -- 172.21.15.47:0/2119612838 shutdown_connections 2024-04-17T05:59:22.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.125+0000 7fb498cd9640 1 --2- 172.21.15.47:0/2119612838 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fb4800a3bb0 0x7fb4800a6070 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:22.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.125+0000 7fb498cd9640 1 --2- 172.21.15.47:0/2119612838 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb4941029b0 0x7fb49419f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:22.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.125+0000 7fb498cd9640 1 --2- 172.21.15.47:0/2119612838 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb494101ff0 0x7fb494198900 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:22.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.125+0000 7fb498cd9640 1 --2- 172.21.15.47:0/2119612838 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb494100ea0 0x7fb4941983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:22.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.125+0000 7fb498cd9640 1 -- 172.21.15.47:0/2119612838 >> 172.21.15.47:0/2119612838 conn(0x7fb4940fc410 msgr2=0x7fb4940691e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:59:22.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.126+0000 7fb498cd9640 1 -- 172.21.15.47:0/2119612838 shutdown_connections 2024-04-17T05:59:22.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.126+0000 7fb498cd9640 1 -- 172.21.15.47:0/2119612838 wait complete. 2024-04-17T05:59:22.136 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-17T05:59:22.136 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T05:59:22.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.243+0000 7f5ed2304640 1 -- 172.21.15.47:0/1654457538 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5ecc103b70 msgr2=0x7f5ecc10a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:22.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.243+0000 7f5ed2304640 1 --2- 172.21.15.47:0/1654457538 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5ecc103b70 0x7f5ecc10a400 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5ec4009ca0 tx=0x7f5ec402c0f0 comp rx=0 tx=0).stop 2024-04-17T05:59:22.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.243+0000 7f5ed2304640 1 -- 172.21.15.47:0/1654457538 shutdown_connections 2024-04-17T05:59:22.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.243+0000 7f5ed2304640 1 --2- 172.21.15.47:0/1654457538 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5ecc103b70 0x7f5ecc10a400 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:22.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.243+0000 7f5ed2304640 1 --2- 172.21.15.47:0/1654457538 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5ecc1031b0 0x7f5ecc103630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:22.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.243+0000 7f5ed2304640 1 --2- 172.21.15.47:0/1654457538 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5ecc101fc0 0x7f5ecc1023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:22.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.243+0000 7f5ed2304640 1 -- 172.21.15.47:0/1654457538 >> 172.21.15.47:0/1654457538 conn(0x7f5ecc0fd510 msgr2=0x7f5ecc0ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:59:22.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.243+0000 7f5ed2304640 1 -- 172.21.15.47:0/1654457538 shutdown_connections 2024-04-17T05:59:22.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.243+0000 7f5ed2304640 1 -- 172.21.15.47:0/1654457538 wait complete. 2024-04-17T05:59:22.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.243+0000 7f5ed2304640 1 Processor -- start 2024-04-17T05:59:22.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.243+0000 7f5ed2304640 1 -- start start 2024-04-17T05:59:22.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.243+0000 7f5ed2304640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5ecc101fc0 0x7f5ecc075d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:22.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.243+0000 7f5ed2304640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5ecc1031b0 0x7f5ecc074350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:22.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.243+0000 7f5ed2304640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5ecc103b70 0x7f5ecc074890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:22.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.244+0000 7f5ed2304640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f5ecc10dc50 con 0x7f5ecc103b70 2024-04-17T05:59:22.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.244+0000 7f5ed2304640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5ecc10dad0 con 0x7f5ecc1031b0 2024-04-17T05:59:22.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.244+0000 7f5ed2304640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5ecc10ddd0 con 0x7f5ecc101fc0 2024-04-17T05:59:22.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.244+0000 7f5ed08a1640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5ecc103b70 0x7f5ecc074890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:22.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.244+0000 7f5ed08a1640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5ecc103b70 0x7f5ecc074890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58032/0 (socket says 172.21.15.47:58032) 2024-04-17T05:59:22.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.244+0000 7f5ed08a1640 1 -- 172.21.15.47:0/2653478563 learned_addr learned my addr 172.21.15.47:0/2653478563 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:59:22.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.244+0000 7f5ecbfff640 1 --2- 172.21.15.47:0/2653478563 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5ecc101fc0 0x7f5ecc075d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:22.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.244+0000 7f5ec3fff640 1 --2- 172.21.15.47:0/2653478563 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5ecc1031b0 0x7f5ecc074350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:22.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.244+0000 7f5ed08a1640 1 -- 172.21.15.47:0/2653478563 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5ecc101fc0 msgr2=0x7f5ecc075d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:22.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.244+0000 7f5ed08a1640 1 --2- 172.21.15.47:0/2653478563 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5ecc101fc0 0x7f5ecc075d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:22.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.244+0000 7f5ed08a1640 1 -- 172.21.15.47:0/2653478563 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5ecc1031b0 msgr2=0x7f5ecc074350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:22.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.244+0000 7f5ed08a1640 1 --2- 172.21.15.47:0/2653478563 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5ecc1031b0 0x7f5ecc074350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:22.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.244+0000 7f5ed08a1640 1 -- 172.21.15.47:0/2653478563 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ecc075120 con 0x7f5ecc103b70 2024-04-17T05:59:22.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.244+0000 7f5ec3fff640 1 --2- 172.21.15.47:0/2653478563 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5ecc1031b0 0x7f5ecc074350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T05:59:22.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.244+0000 7f5ed08a1640 1 --2- 172.21.15.47:0/2653478563 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5ecc103b70 0x7f5ecc074890 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5ec402c700 tx=0x7f5ec4032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:59:22.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.244+0000 7f5ec9ffb640 1 -- 172.21.15.47:0/2653478563 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec4039040 con 0x7f5ecc103b70 2024-04-17T05:59:22.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.244+0000 7f5ed2304640 1 -- 172.21.15.47:0/2653478563 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ecc1a2300 con 0x7f5ecc103b70 2024-04-17T05:59:22.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.244+0000 7f5ec9ffb640 1 -- 172.21.15.47:0/2653478563 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ec4044070 con 0x7f5ecc103b70 2024-04-17T05:59:22.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.244+0000 7f5ec9ffb640 1 -- 172.21.15.47:0/2653478563 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec40042a0 con 0x7f5ecc103b70 2024-04-17T05:59:22.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.244+0000 7f5ed2304640 1 -- 172.21.15.47:0/2653478563 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ecc1a28a0 con 0x7f5ecc103b70 2024-04-17T05:59:22.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.245+0000 7f5ed2304640 1 -- 172.21.15.47:0/2653478563 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ecc109280 con 0x7f5ecc103b70 2024-04-17T05:59:22.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.246+0000 7f5ec9ffb640 1 -- 172.21.15.47:0/2653478563 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f5ec4003970 con 0x7f5ecc103b70 2024-04-17T05:59:22.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.247+0000 7f5ec9ffb640 1 --2- 172.21.15.47:0/2653478563 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f5eb80a3bb0 0x7f5eb80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:22.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.247+0000 7f5ecbfff640 1 --2- 172.21.15.47:0/2653478563 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f5eb80a3bb0 0x7f5eb80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:22.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.247+0000 7f5ec9ffb640 1 -- 172.21.15.47:0/2653478563 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f5ec40f2e00 con 0x7f5ecc103b70 2024-04-17T05:59:22.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.248+0000 7f5ecbfff640 1 --2- 172.21.15.47:0/2653478563 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f5eb80a3bb0 0x7f5eb80a6070 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc005fd0 tx=0x7f5ebc00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:59:22.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.252+0000 7f5ec9ffb640 1 -- 172.21.15.47:0/2653478563 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f5ec40bf9c0 con 0x7f5ecc103b70 2024-04-17T05:59:22.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.435+0000 7f5ed2304640 1 -- 172.21.15.47:0/2653478563 --> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5ecc0623a0 con 0x7f5eb80a3bb0 2024-04-17T05:59:22.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.443+0000 7f5ec9ffb640 1 -- 172.21.15.47:0/2653478563 <== mgr.14152 v2:172.21.15.47:6800/3157902806 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258864 (secure 0 0 0) 0x7f5ecc0623a0 con 0x7f5eb80a3bb0 2024-04-17T05:59:22.445 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-17T05:59:22.447 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-04-17T05:59:22.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.447+0000 7f5ed2304640 1 -- 172.21.15.47:0/2653478563 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f5eb80a3bb0 msgr2=0x7f5eb80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:22.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.447+0000 7f5ed2304640 1 --2- 172.21.15.47:0/2653478563 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f5eb80a3bb0 0x7f5eb80a6070 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc005fd0 tx=0x7f5ebc00b390 comp rx=0 tx=0).stop 2024-04-17T05:59:22.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.447+0000 7f5ed2304640 1 -- 172.21.15.47:0/2653478563 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5ecc103b70 msgr2=0x7f5ecc074890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:22.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.447+0000 7f5ed2304640 1 --2- 172.21.15.47:0/2653478563 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5ecc103b70 0x7f5ecc074890 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5ec402c700 tx=0x7f5ec4032f20 comp rx=0 tx=0).stop 2024-04-17T05:59:22.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.447+0000 7f5ed2304640 1 -- 172.21.15.47:0/2653478563 shutdown_connections 2024-04-17T05:59:22.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.447+0000 7f5ed2304640 1 --2- 172.21.15.47:0/2653478563 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f5eb80a3bb0 0x7f5eb80a6070 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:22.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.447+0000 7f5ed2304640 1 --2- 172.21.15.47:0/2653478563 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5ecc103b70 0x7f5ecc074890 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:22.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.447+0000 7f5ed2304640 1 --2- 172.21.15.47:0/2653478563 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5ecc1031b0 0x7f5ecc074350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:22.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.447+0000 7f5ed2304640 1 --2- 172.21.15.47:0/2653478563 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5ecc101fc0 0x7f5ecc075d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:22.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.447+0000 7f5ed2304640 1 -- 172.21.15.47:0/2653478563 >> 172.21.15.47:0/2653478563 conn(0x7f5ecc0fd510 msgr2=0x7f5ecc106450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:59:22.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.447+0000 7f5ed2304640 1 -- 172.21.15.47:0/2653478563 shutdown_connections 2024-04-17T05:59:22.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.447+0000 7f5ed2304640 1 -- 172.21.15.47:0/2653478563 wait complete. 2024-04-17T05:59:22.456 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":290,"stamp":"2024-04-17T05:59:21.724593+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594162,"num_objects":38,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":0,"num_read":143,"num_read_kb":125,"num_write":258,"num_write_kb":2957,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":204,"ondisk_log_size":204,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":261,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":364488,"kb_used_data":11116,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124577336,"statfs":{"total":1151940427776,"available":1151567192064,"internally_reserved":0,"allocated":11382784,"data_stored":7082048,"data_compressed":108183,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19140,"internal_metadata":361674044},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3794,"num_objects":36,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":7,"num_read_kb":6,"num_write":77,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001520"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.758360+0000","last_change":"2024-04-17T05:59:14.592927+0000","last_active":"2024-04-17T05:59:15.758360+0000","last_peered":"2024-04-17T05:59:15.758360+0000","last_clean":"2024-04-17T05:59:15.758360+0000","last_became_active":"2024-04-17T05:59:14.592036+0000","last_became_peered":"2024-04-17T05:59:14.592036+0000","last_unstale":"2024-04-17T05:59:15.758360+0000","last_undegraded":"2024-04-17T05:59:15.758360+0000","last_fullsized":"2024-04-17T05:59:15.758360+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:10:51.146397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.758295+0000","last_change":"2024-04-17T05:59:14.587862+0000","last_active":"2024-04-17T05:59:15.758295+0000","last_peered":"2024-04-17T05:59:15.758295+0000","last_clean":"2024-04-17T05:59:15.758295+0000","last_became_active":"2024-04-17T05:59:14.587553+0000","last_became_peered":"2024-04-17T05:59:14.587553+0000","last_unstale":"2024-04-17T05:59:15.758295+0000","last_undegraded":"2024-04-17T05:59:15.758295+0000","last_fullsized":"2024-04-17T05:59:15.758295+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:16:12.224247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.758168+0000","last_change":"2024-04-17T05:59:14.592223+0000","last_active":"2024-04-17T05:59:15.758168+0000","last_peered":"2024-04-17T05:59:15.758168+0000","last_clean":"2024-04-17T05:59:15.758168+0000","last_became_active":"2024-04-17T05:59:14.591892+0000","last_became_peered":"2024-04-17T05:59:14.591892+0000","last_unstale":"2024-04-17T05:59:15.758168+0000","last_undegraded":"2024-04-17T05:59:15.758168+0000","last_fullsized":"2024-04-17T05:59:15.758168+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:23:26.943368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.758499+0000","last_change":"2024-04-17T05:59:14.585607+0000","last_active":"2024-04-17T05:59:15.758499+0000","last_peered":"2024-04-17T05:59:15.758499+0000","last_clean":"2024-04-17T05:59:15.758499+0000","last_became_active":"2024-04-17T05:59:14.585255+0000","last_became_peered":"2024-04-17T05:59:14.585255+0000","last_unstale":"2024-04-17T05:59:15.758499+0000","last_undegraded":"2024-04-17T05:59:15.758499+0000","last_fullsized":"2024-04-17T05:59:15.758499+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:45:57.355035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.757860+0000","last_change":"2024-04-17T05:59:14.599703+0000","last_active":"2024-04-17T05:59:15.757860+0000","last_peered":"2024-04-17T05:59:15.757860+0000","last_clean":"2024-04-17T05:59:15.757860+0000","last_became_active":"2024-04-17T05:59:14.599305+0000","last_became_peered":"2024-04-17T05:59:14.599305+0000","last_unstale":"2024-04-17T05:59:15.757860+0000","last_undegraded":"2024-04-17T05:59:15.757860+0000","last_fullsized":"2024-04-17T05:59:15.757860+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:58:36.056590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.758977+0000","last_change":"2024-04-17T05:59:14.586176+0000","last_active":"2024-04-17T05:59:15.758977+0000","last_peered":"2024-04-17T05:59:15.758977+0000","last_clean":"2024-04-17T05:59:15.758977+0000","last_became_active":"2024-04-17T05:59:14.585850+0000","last_became_peered":"2024-04-17T05:59:14.585850+0000","last_unstale":"2024-04-17T05:59:15.758977+0000","last_undegraded":"2024-04-17T05:59:15.758977+0000","last_fullsized":"2024-04-17T05:59:15.758977+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:01:14.628048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.757673+0000","last_change":"2024-04-17T05:59:14.587750+0000","last_active":"2024-04-17T05:59:15.757673+0000","last_peered":"2024-04-17T05:59:15.757673+0000","last_clean":"2024-04-17T05:59:15.757673+0000","last_became_active":"2024-04-17T05:59:14.587394+0000","last_became_peered":"2024-04-17T05:59:14.587394+0000","last_unstale":"2024-04-17T05:59:15.757673+0000","last_undegraded":"2024-04-17T05:59:15.757673+0000","last_fullsized":"2024-04-17T05:59:15.757673+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:22:52.733877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.581516+0000","last_change":"2024-04-17T05:59:14.591458+0000","last_active":"2024-04-17T05:59:15.581516+0000","last_peered":"2024-04-17T05:59:15.581516+0000","last_clean":"2024-04-17T05:59:15.581516+0000","last_became_active":"2024-04-17T05:59:14.590898+0000","last_became_peered":"2024-04-17T05:59:14.590898+0000","last_unstale":"2024-04-17T05:59:15.581516+0000","last_undegraded":"2024-04-17T05:59:15.581516+0000","last_fullsized":"2024-04-17T05:59:15.581516+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:58:40.817928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.585312+0000","last_change":"2024-04-17T05:59:14.600199+0000","last_active":"2024-04-17T05:59:15.585312+0000","last_peered":"2024-04-17T05:59:15.585312+0000","last_clean":"2024-04-17T05:59:15.585312+0000","last_became_active":"2024-04-17T05:59:14.599286+0000","last_became_peered":"2024-04-17T05:59:14.599286+0000","last_unstale":"2024-04-17T05:59:15.585312+0000","last_undegraded":"2024-04-17T05:59:15.585312+0000","last_fullsized":"2024-04-17T05:59:15.585312+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:41:43.314103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.759270+0000","last_change":"2024-04-17T05:59:14.587999+0000","last_active":"2024-04-17T05:59:15.759270+0000","last_peered":"2024-04-17T05:59:15.759270+0000","last_clean":"2024-04-17T05:59:15.759270+0000","last_became_active":"2024-04-17T05:59:14.587655+0000","last_became_peered":"2024-04-17T05:59:14.587655+0000","last_unstale":"2024-04-17T05:59:15.759270+0000","last_undegraded":"2024-04-17T05:59:15.759270+0000","last_fullsized":"2024-04-17T05:59:15.759270+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:41:37.751776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.758882+0000","last_change":"2024-04-17T05:59:14.595003+0000","last_active":"2024-04-17T05:59:15.758882+0000","last_peered":"2024-04-17T05:59:15.758882+0000","last_clean":"2024-04-17T05:59:15.758882+0000","last_became_active":"2024-04-17T05:59:14.594424+0000","last_became_peered":"2024-04-17T05:59:14.594424+0000","last_unstale":"2024-04-17T05:59:15.758882+0000","last_undegraded":"2024-04-17T05:59:15.758882+0000","last_fullsized":"2024-04-17T05:59:15.758882+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:35:39.491074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.758537+0000","last_change":"2024-04-17T05:59:14.584957+0000","last_active":"2024-04-17T05:59:15.758537+0000","last_peered":"2024-04-17T05:59:15.758537+0000","last_clean":"2024-04-17T05:59:15.758537+0000","last_became_active":"2024-04-17T05:59:14.584625+0000","last_became_peered":"2024-04-17T05:59:14.584625+0000","last_unstale":"2024-04-17T05:59:15.758537+0000","last_undegraded":"2024-04-17T05:59:15.758537+0000","last_fullsized":"2024-04-17T05:59:15.758537+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:10:08.819831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.757871+0000","last_change":"2024-04-17T05:59:14.577633+0000","last_active":"2024-04-17T05:59:15.757871+0000","last_peered":"2024-04-17T05:59:15.757871+0000","last_clean":"2024-04-17T05:59:15.757871+0000","last_became_active":"2024-04-17T05:59:14.576608+0000","last_became_peered":"2024-04-17T05:59:14.576608+0000","last_unstale":"2024-04-17T05:59:15.757871+0000","last_undegraded":"2024-04-17T05:59:15.757871+0000","last_fullsized":"2024-04-17T05:59:15.757871+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:04:49.921958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.758060+0000","last_change":"2024-04-17T05:59:14.580912+0000","last_active":"2024-04-17T05:59:15.758060+0000","last_peered":"2024-04-17T05:59:15.758060+0000","last_clean":"2024-04-17T05:59:15.758060+0000","last_became_active":"2024-04-17T05:59:14.580500+0000","last_became_peered":"2024-04-17T05:59:14.580500+0000","last_unstale":"2024-04-17T05:59:15.758060+0000","last_undegraded":"2024-04-17T05:59:15.758060+0000","last_fullsized":"2024-04-17T05:59:15.758060+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:17:49.893499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.757641+0000","last_change":"2024-04-17T05:59:14.586109+0000","last_active":"2024-04-17T05:59:15.757641+0000","last_peered":"2024-04-17T05:59:15.757641+0000","last_clean":"2024-04-17T05:59:15.757641+0000","last_became_active":"2024-04-17T05:59:14.585667+0000","last_became_peered":"2024-04-17T05:59:14.585667+0000","last_unstale":"2024-04-17T05:59:15.757641+0000","last_undegraded":"2024-04-17T05:59:15.757641+0000","last_fullsized":"2024-04-17T05:59:15.757641+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:20:22.309678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.584844+0000","last_change":"2024-04-17T05:59:14.602921+0000","last_active":"2024-04-17T05:59:15.584844+0000","last_peered":"2024-04-17T05:59:15.584844+0000","last_clean":"2024-04-17T05:59:15.584844+0000","last_became_active":"2024-04-17T05:59:14.602589+0000","last_became_peered":"2024-04-17T05:59:14.602589+0000","last_unstale":"2024-04-17T05:59:15.584844+0000","last_undegraded":"2024-04-17T05:59:15.584844+0000","last_fullsized":"2024-04-17T05:59:15.584844+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:49:17.988067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.758438+0000","last_change":"2024-04-17T05:59:14.594201+0000","last_active":"2024-04-17T05:59:15.758438+0000","last_peered":"2024-04-17T05:59:15.758438+0000","last_clean":"2024-04-17T05:59:15.758438+0000","last_became_active":"2024-04-17T05:59:14.593850+0000","last_became_peered":"2024-04-17T05:59:14.593850+0000","last_unstale":"2024-04-17T05:59:15.758438+0000","last_undegraded":"2024-04-17T05:59:15.758438+0000","last_fullsized":"2024-04-17T05:59:15.758438+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:53:09.044324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.758499+0000","last_change":"2024-04-17T05:59:14.590439+0000","last_active":"2024-04-17T05:59:15.758499+0000","last_peered":"2024-04-17T05:59:15.758499+0000","last_clean":"2024-04-17T05:59:15.758499+0000","last_became_active":"2024-04-17T05:59:14.589844+0000","last_became_peered":"2024-04-17T05:59:14.589844+0000","last_unstale":"2024-04-17T05:59:15.758499+0000","last_undegraded":"2024-04-17T05:59:15.758499+0000","last_fullsized":"2024-04-17T05:59:15.758499+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:02:18.366048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.759032+0000","last_change":"2024-04-17T05:59:14.592297+0000","last_active":"2024-04-17T05:59:15.759032+0000","last_peered":"2024-04-17T05:59:15.759032+0000","last_clean":"2024-04-17T05:59:15.759032+0000","last_became_active":"2024-04-17T05:59:14.591976+0000","last_became_peered":"2024-04-17T05:59:14.591976+0000","last_unstale":"2024-04-17T05:59:15.759032+0000","last_undegraded":"2024-04-17T05:59:15.759032+0000","last_fullsized":"2024-04-17T05:59:15.759032+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:11:47.488231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:18.632464+0000","last_change":"2024-04-17T05:59:13.561616+0000","last_active":"2024-04-17T05:59:18.632464+0000","last_peered":"2024-04-17T05:59:18.632464+0000","last_clean":"2024-04-17T05:59:18.632464+0000","last_became_active":"2024-04-17T05:59:13.561298+0000","last_became_peered":"2024-04-17T05:59:13.561298+0000","last_unstale":"2024-04-17T05:59:18.632464+0000","last_undegraded":"2024-04-17T05:59:18.632464+0000","last_fullsized":"2024-04-17T05:59:18.632464+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:59:02.598774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.582030+0000","last_change":"2024-04-17T05:59:14.592023+0000","last_active":"2024-04-17T05:59:15.582030+0000","last_peered":"2024-04-17T05:59:15.582030+0000","last_clean":"2024-04-17T05:59:15.582030+0000","last_became_active":"2024-04-17T05:59:14.591707+0000","last_became_peered":"2024-04-17T05:59:14.591707+0000","last_unstale":"2024-04-17T05:59:15.582030+0000","last_undegraded":"2024-04-17T05:59:15.582030+0000","last_fullsized":"2024-04-17T05:59:15.582030+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:50:06.828184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.582103+0000","last_change":"2024-04-17T05:59:13.568073+0000","last_active":"2024-04-17T05:59:15.582103+0000","last_peered":"2024-04-17T05:59:15.582103+0000","last_clean":"2024-04-17T05:59:15.582103+0000","last_became_active":"2024-04-17T05:59:13.567751+0000","last_became_peered":"2024-04-17T05:59:13.567751+0000","last_unstale":"2024-04-17T05:59:15.582103+0000","last_undegraded":"2024-04-17T05:59:15.582103+0000","last_fullsized":"2024-04-17T05:59:15.582103+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:40:26.884308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.758505+0000","last_change":"2024-04-17T05:59:14.593344+0000","last_active":"2024-04-17T05:59:15.758505+0000","last_peered":"2024-04-17T05:59:15.758505+0000","last_clean":"2024-04-17T05:59:15.758505+0000","last_became_active":"2024-04-17T05:59:14.592946+0000","last_became_peered":"2024-04-17T05:59:14.592946+0000","last_unstale":"2024-04-17T05:59:15.758505+0000","last_undegraded":"2024-04-17T05:59:15.758505+0000","last_fullsized":"2024-04-17T05:59:15.758505+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:50:23.979412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.758345+0000","last_change":"2024-04-17T05:59:13.566639+0000","last_active":"2024-04-17T05:59:15.758345+0000","last_peered":"2024-04-17T05:59:15.758345+0000","last_clean":"2024-04-17T05:59:15.758345+0000","last_became_active":"2024-04-17T05:59:13.562766+0000","last_became_peered":"2024-04-17T05:59:13.562766+0000","last_unstale":"2024-04-17T05:59:15.758345+0000","last_undegraded":"2024-04-17T05:59:15.758345+0000","last_fullsized":"2024-04-17T05:59:15.758345+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:36:47.605106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.585132+0000","last_change":"2024-04-17T05:59:14.596055+0000","last_active":"2024-04-17T05:59:15.585132+0000","last_peered":"2024-04-17T05:59:15.585132+0000","last_clean":"2024-04-17T05:59:15.585132+0000","last_became_active":"2024-04-17T05:59:14.595672+0000","last_became_peered":"2024-04-17T05:59:14.595672+0000","last_unstale":"2024-04-17T05:59:15.585132+0000","last_undegraded":"2024-04-17T05:59:15.585132+0000","last_fullsized":"2024-04-17T05:59:15.585132+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:31:59.964502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.758069+0000","last_change":"2024-04-17T05:59:13.558715+0000","last_active":"2024-04-17T05:59:15.758069+0000","last_peered":"2024-04-17T05:59:15.758069+0000","last_clean":"2024-04-17T05:59:15.758069+0000","last_became_active":"2024-04-17T05:59:13.558377+0000","last_became_peered":"2024-04-17T05:59:13.558377+0000","last_unstale":"2024-04-17T05:59:15.758069+0000","last_undegraded":"2024-04-17T05:59:15.758069+0000","last_fullsized":"2024-04-17T05:59:15.758069+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:12:12.392655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.582710+0000","last_change":"2024-04-17T05:59:14.594824+0000","last_active":"2024-04-17T05:59:15.582710+0000","last_peered":"2024-04-17T05:59:15.582710+0000","last_clean":"2024-04-17T05:59:15.582710+0000","last_became_active":"2024-04-17T05:59:14.594510+0000","last_became_peered":"2024-04-17T05:59:14.594510+0000","last_unstale":"2024-04-17T05:59:15.582710+0000","last_undegraded":"2024-04-17T05:59:15.582710+0000","last_fullsized":"2024-04-17T05:59:15.582710+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:19:37.042522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'5","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:17.629557+0000","last_change":"2024-04-17T05:59:13.567654+0000","last_active":"2024-04-17T05:59:17.629557+0000","last_peered":"2024-04-17T05:59:17.629557+0000","last_clean":"2024-04-17T05:59:17.629557+0000","last_became_active":"2024-04-17T05:59:13.566474+0000","last_became_peered":"2024-04-17T05:59:13.566474+0000","last_unstale":"2024-04-17T05:59:17.629557+0000","last_undegraded":"2024-04-17T05:59:17.629557+0000","last_fullsized":"2024-04-17T05:59:17.629557+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:40:25.205832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.757985+0000","last_change":"2024-04-17T05:59:14.597145+0000","last_active":"2024-04-17T05:59:15.757985+0000","last_peered":"2024-04-17T05:59:15.757985+0000","last_clean":"2024-04-17T05:59:15.757985+0000","last_became_active":"2024-04-17T05:59:14.596802+0000","last_became_peered":"2024-04-17T05:59:14.596802+0000","last_unstale":"2024-04-17T05:59:15.757985+0000","last_undegraded":"2024-04-17T05:59:15.757985+0000","last_fullsized":"2024-04-17T05:59:15.757985+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:41:04.946052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:18.633424+0000","last_change":"2024-04-17T05:59:13.560100+0000","last_active":"2024-04-17T05:59:18.633424+0000","last_peered":"2024-04-17T05:59:18.633424+0000","last_clean":"2024-04-17T05:59:18.633424+0000","last_became_active":"2024-04-17T05:59:13.559769+0000","last_became_peered":"2024-04-17T05:59:13.559769+0000","last_unstale":"2024-04-17T05:59:18.633424+0000","last_undegraded":"2024-04-17T05:59:18.633424+0000","last_fullsized":"2024-04-17T05:59:18.633424+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:44:30.299028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.582495+0000","last_change":"2024-04-17T05:59:14.600733+0000","last_active":"2024-04-17T05:59:15.582495+0000","last_peered":"2024-04-17T05:59:15.582495+0000","last_clean":"2024-04-17T05:59:15.582495+0000","last_became_active":"2024-04-17T05:59:14.599273+0000","last_became_peered":"2024-04-17T05:59:14.599273+0000","last_unstale":"2024-04-17T05:59:15.582495+0000","last_undegraded":"2024-04-17T05:59:15.582495+0000","last_fullsized":"2024-04-17T05:59:15.582495+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:27:02.721452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.757410+0000","last_change":"2024-04-17T05:59:13.553000+0000","last_active":"2024-04-17T05:59:15.757410+0000","last_peered":"2024-04-17T05:59:15.757410+0000","last_clean":"2024-04-17T05:59:15.757410+0000","last_became_active":"2024-04-17T05:59:13.552469+0000","last_became_peered":"2024-04-17T05:59:13.552469+0000","last_unstale":"2024-04-17T05:59:15.757410+0000","last_undegraded":"2024-04-17T05:59:15.757410+0000","last_fullsized":"2024-04-17T05:59:15.757410+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:14:34.714179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.581455+0000","last_change":"2024-04-17T05:59:14.594792+0000","last_active":"2024-04-17T05:59:15.581455+0000","last_peered":"2024-04-17T05:59:15.581455+0000","last_clean":"2024-04-17T05:59:15.581455+0000","last_became_active":"2024-04-17T05:59:14.594436+0000","last_became_peered":"2024-04-17T05:59:14.594436+0000","last_unstale":"2024-04-17T05:59:15.581455+0000","last_undegraded":"2024-04-17T05:59:15.581455+0000","last_fullsized":"2024-04-17T05:59:15.581455+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:49:57.104050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.582065+0000","last_change":"2024-04-17T05:59:13.565352+0000","last_active":"2024-04-17T05:59:15.582065+0000","last_peered":"2024-04-17T05:59:15.582065+0000","last_clean":"2024-04-17T05:59:15.582065+0000","last_became_active":"2024-04-17T05:59:13.565046+0000","last_became_peered":"2024-04-17T05:59:13.565046+0000","last_unstale":"2024-04-17T05:59:15.582065+0000","last_undegraded":"2024-04-17T05:59:15.582065+0000","last_fullsized":"2024-04-17T05:59:15.582065+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:39:25.276782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.757298+0000","last_change":"2024-04-17T05:59:14.593925+0000","last_active":"2024-04-17T05:59:15.757298+0000","last_peered":"2024-04-17T05:59:15.757298+0000","last_clean":"2024-04-17T05:59:15.757298+0000","last_became_active":"2024-04-17T05:59:14.593535+0000","last_became_peered":"2024-04-17T05:59:14.593535+0000","last_unstale":"2024-04-17T05:59:15.757298+0000","last_undegraded":"2024-04-17T05:59:15.757298+0000","last_fullsized":"2024-04-17T05:59:15.757298+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:58:20.811999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'3","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:17.627171+0000","last_change":"2024-04-17T05:59:13.562281+0000","last_active":"2024-04-17T05:59:17.627171+0000","last_peered":"2024-04-17T05:59:17.627171+0000","last_clean":"2024-04-17T05:59:17.627171+0000","last_became_active":"2024-04-17T05:59:13.561931+0000","last_became_peered":"2024-04-17T05:59:13.561931+0000","last_unstale":"2024-04-17T05:59:17.627171+0000","last_undegraded":"2024-04-17T05:59:17.627171+0000","last_fullsized":"2024-04-17T05:59:17.627171+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:55:52.119923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.585188+0000","last_change":"2024-04-17T05:59:14.594708+0000","last_active":"2024-04-17T05:59:15.585188+0000","last_peered":"2024-04-17T05:59:15.585188+0000","last_clean":"2024-04-17T05:59:15.585188+0000","last_became_active":"2024-04-17T05:59:14.594206+0000","last_became_peered":"2024-04-17T05:59:14.594206+0000","last_unstale":"2024-04-17T05:59:15.585188+0000","last_undegraded":"2024-04-17T05:59:15.585188+0000","last_fullsized":"2024-04-17T05:59:15.585188+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:43:16.402659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.757728+0000","last_change":"2024-04-17T05:59:13.561056+0000","last_active":"2024-04-17T05:59:15.757728+0000","last_peered":"2024-04-17T05:59:15.757728+0000","last_clean":"2024-04-17T05:59:15.757728+0000","last_became_active":"2024-04-17T05:59:13.560670+0000","last_became_peered":"2024-04-17T05:59:13.560670+0000","last_unstale":"2024-04-17T05:59:15.757728+0000","last_undegraded":"2024-04-17T05:59:15.757728+0000","last_fullsized":"2024-04-17T05:59:15.757728+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:31:01.106013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.758679+0000","last_change":"2024-04-17T05:59:14.590243+0000","last_active":"2024-04-17T05:59:15.758679+0000","last_peered":"2024-04-17T05:59:15.758679+0000","last_clean":"2024-04-17T05:59:15.758679+0000","last_became_active":"2024-04-17T05:59:14.589911+0000","last_became_peered":"2024-04-17T05:59:14.589911+0000","last_unstale":"2024-04-17T05:59:15.758679+0000","last_undegraded":"2024-04-17T05:59:15.758679+0000","last_fullsized":"2024-04-17T05:59:15.758679+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:44:41.855928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.582791+0000","last_change":"2024-04-17T05:59:13.569983+0000","last_active":"2024-04-17T05:59:15.582791+0000","last_peered":"2024-04-17T05:59:15.582791+0000","last_clean":"2024-04-17T05:59:15.582791+0000","last_became_active":"2024-04-17T05:59:13.569667+0000","last_became_peered":"2024-04-17T05:59:13.569667+0000","last_unstale":"2024-04-17T05:59:15.582791+0000","last_undegraded":"2024-04-17T05:59:15.582791+0000","last_fullsized":"2024-04-17T05:59:15.582791+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:36:15.362530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.585072+0000","last_change":"2024-04-17T05:59:14.603404+0000","last_active":"2024-04-17T05:59:15.585072+0000","last_peered":"2024-04-17T05:59:15.585072+0000","last_clean":"2024-04-17T05:59:15.585072+0000","last_became_active":"2024-04-17T05:59:14.601593+0000","last_became_peered":"2024-04-17T05:59:14.601593+0000","last_unstale":"2024-04-17T05:59:15.585072+0000","last_undegraded":"2024-04-17T05:59:15.585072+0000","last_fullsized":"2024-04-17T05:59:15.585072+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:44:44.490496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:19.606855+0000","last_change":"2024-04-17T05:59:13.561237+0000","last_active":"2024-04-17T05:59:19.606855+0000","last_peered":"2024-04-17T05:59:19.606855+0000","last_clean":"2024-04-17T05:59:19.606855+0000","last_became_active":"2024-04-17T05:59:13.560890+0000","last_became_peered":"2024-04-17T05:59:13.560890+0000","last_unstale":"2024-04-17T05:59:19.606855+0000","last_undegraded":"2024-04-17T05:59:19.606855+0000","last_fullsized":"2024-04-17T05:59:19.606855+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:27:13.247831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.758195+0000","last_change":"2024-04-17T05:59:14.579304+0000","last_active":"2024-04-17T05:59:15.758195+0000","last_peered":"2024-04-17T05:59:15.758195+0000","last_clean":"2024-04-17T05:59:15.758195+0000","last_became_active":"2024-04-17T05:59:14.578948+0000","last_became_peered":"2024-04-17T05:59:14.578948+0000","last_unstale":"2024-04-17T05:59:15.758195+0000","last_undegraded":"2024-04-17T05:59:15.758195+0000","last_fullsized":"2024-04-17T05:59:15.758195+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:39:43.341725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.758692+0000","last_change":"2024-04-17T05:59:13.569534+0000","last_active":"2024-04-17T05:59:15.758692+0000","last_peered":"2024-04-17T05:59:15.758692+0000","last_clean":"2024-04-17T05:59:15.758692+0000","last_became_active":"2024-04-17T05:59:13.569041+0000","last_became_peered":"2024-04-17T05:59:13.569041+0000","last_unstale":"2024-04-17T05:59:15.758692+0000","last_undegraded":"2024-04-17T05:59:15.758692+0000","last_fullsized":"2024-04-17T05:59:15.758692+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:27:37.018696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.757438+0000","last_change":"2024-04-17T05:59:14.593820+0000","last_active":"2024-04-17T05:59:15.757438+0000","last_peered":"2024-04-17T05:59:15.757438+0000","last_clean":"2024-04-17T05:59:15.757438+0000","last_became_active":"2024-04-17T05:59:14.593473+0000","last_became_peered":"2024-04-17T05:59:14.593473+0000","last_unstale":"2024-04-17T05:59:15.757438+0000","last_undegraded":"2024-04-17T05:59:15.757438+0000","last_fullsized":"2024-04-17T05:59:15.757438+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:44:38.636980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:17.627799+0000","last_change":"2024-04-17T05:59:13.570124+0000","last_active":"2024-04-17T05:59:17.627799+0000","last_peered":"2024-04-17T05:59:17.627799+0000","last_clean":"2024-04-17T05:59:17.627799+0000","last_became_active":"2024-04-17T05:59:13.569778+0000","last_became_peered":"2024-04-17T05:59:13.569778+0000","last_unstale":"2024-04-17T05:59:17.627799+0000","last_undegraded":"2024-04-17T05:59:17.627799+0000","last_fullsized":"2024-04-17T05:59:17.627799+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:46:46.431207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.758225+0000","last_change":"2024-04-17T05:59:14.584863+0000","last_active":"2024-04-17T05:59:15.758225+0000","last_peered":"2024-04-17T05:59:15.758225+0000","last_clean":"2024-04-17T05:59:15.758225+0000","last_became_active":"2024-04-17T05:59:14.584554+0000","last_became_peered":"2024-04-17T05:59:14.584554+0000","last_unstale":"2024-04-17T05:59:15.758225+0000","last_undegraded":"2024-04-17T05:59:15.758225+0000","last_fullsized":"2024-04-17T05:59:15.758225+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:26:22.064337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.757742+0000","last_change":"2024-04-17T05:59:13.557625+0000","last_active":"2024-04-17T05:59:15.757742+0000","last_peered":"2024-04-17T05:59:15.757742+0000","last_clean":"2024-04-17T05:59:15.757742+0000","last_became_active":"2024-04-17T05:59:13.557307+0000","last_became_peered":"2024-04-17T05:59:13.557307+0000","last_unstale":"2024-04-17T05:59:15.757742+0000","last_undegraded":"2024-04-17T05:59:15.757742+0000","last_fullsized":"2024-04-17T05:59:15.757742+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:43:17.023049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.757952+0000","last_change":"2024-04-17T05:59:14.579295+0000","last_active":"2024-04-17T05:59:15.757952+0000","last_peered":"2024-04-17T05:59:15.757952+0000","last_clean":"2024-04-17T05:59:15.757952+0000","last_became_active":"2024-04-17T05:59:14.578902+0000","last_became_peered":"2024-04-17T05:59:14.578902+0000","last_unstale":"2024-04-17T05:59:15.757952+0000","last_undegraded":"2024-04-17T05:59:15.757952+0000","last_fullsized":"2024-04-17T05:59:15.757952+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:46:23.094552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'8","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.757548+0000","last_change":"2024-04-17T05:59:13.561834+0000","last_active":"2024-04-17T05:59:15.757548+0000","last_peered":"2024-04-17T05:59:15.757548+0000","last_clean":"2024-04-17T05:59:15.757548+0000","last_became_active":"2024-04-17T05:59:13.561536+0000","last_became_peered":"2024-04-17T05:59:13.561536+0000","last_unstale":"2024-04-17T05:59:15.757548+0000","last_undegraded":"2024-04-17T05:59:15.757548+0000","last_fullsized":"2024-04-17T05:59:15.757548+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:33:03.869380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.582129+0000","last_change":"2024-04-17T05:59:14.594082+0000","last_active":"2024-04-17T05:59:15.582129+0000","last_peered":"2024-04-17T05:59:15.582129+0000","last_clean":"2024-04-17T05:59:15.582129+0000","last_became_active":"2024-04-17T05:59:14.593754+0000","last_became_peered":"2024-04-17T05:59:14.593754+0000","last_unstale":"2024-04-17T05:59:15.582129+0000","last_undegraded":"2024-04-17T05:59:15.582129+0000","last_fullsized":"2024-04-17T05:59:15.582129+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:36:28.752766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.758402+0000","last_change":"2024-04-17T05:59:13.563470+0000","last_active":"2024-04-17T05:59:15.758402+0000","last_peered":"2024-04-17T05:59:15.758402+0000","last_clean":"2024-04-17T05:59:15.758402+0000","last_became_active":"2024-04-17T05:59:13.563067+0000","last_became_peered":"2024-04-17T05:59:13.563067+0000","last_unstale":"2024-04-17T05:59:15.758402+0000","last_undegraded":"2024-04-17T05:59:15.758402+0000","last_fullsized":"2024-04-17T05:59:15.758402+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:13:52.018604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.758193+0000","last_change":"2024-04-17T05:59:14.587339+0000","last_active":"2024-04-17T05:59:15.758193+0000","last_peered":"2024-04-17T05:59:15.758193+0000","last_clean":"2024-04-17T05:59:15.758193+0000","last_became_active":"2024-04-17T05:59:14.586956+0000","last_became_peered":"2024-04-17T05:59:14.586956+0000","last_unstale":"2024-04-17T05:59:15.758193+0000","last_undegraded":"2024-04-17T05:59:15.758193+0000","last_fullsized":"2024-04-17T05:59:15.758193+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:00:21.494832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.758131+0000","last_change":"2024-04-17T05:59:13.563915+0000","last_active":"2024-04-17T05:59:15.758131+0000","last_peered":"2024-04-17T05:59:15.758131+0000","last_clean":"2024-04-17T05:59:15.758131+0000","last_became_active":"2024-04-17T05:59:13.563218+0000","last_became_peered":"2024-04-17T05:59:13.563218+0000","last_unstale":"2024-04-17T05:59:15.758131+0000","last_undegraded":"2024-04-17T05:59:15.758131+0000","last_fullsized":"2024-04-17T05:59:15.758131+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:36:54.484278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.759089+0000","last_change":"2024-04-17T05:59:14.593168+0000","last_active":"2024-04-17T05:59:15.759089+0000","last_peered":"2024-04-17T05:59:15.759089+0000","last_clean":"2024-04-17T05:59:15.759089+0000","last_became_active":"2024-04-17T05:59:14.592744+0000","last_became_peered":"2024-04-17T05:59:14.592744+0000","last_unstale":"2024-04-17T05:59:15.759089+0000","last_undegraded":"2024-04-17T05:59:15.759089+0000","last_fullsized":"2024-04-17T05:59:15.759089+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:00:03.217864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.758028+0000","last_change":"2024-04-17T05:59:13.564703+0000","last_active":"2024-04-17T05:59:15.758028+0000","last_peered":"2024-04-17T05:59:15.758028+0000","last_clean":"2024-04-17T05:59:15.758028+0000","last_became_active":"2024-04-17T05:59:13.564263+0000","last_became_peered":"2024-04-17T05:59:13.564263+0000","last_unstale":"2024-04-17T05:59:15.758028+0000","last_undegraded":"2024-04-17T05:59:15.758028+0000","last_fullsized":"2024-04-17T05:59:15.758028+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:51:06.847989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.759260+0000","last_change":"2024-04-17T05:59:14.588074+0000","last_active":"2024-04-17T05:59:15.759260+0000","last_peered":"2024-04-17T05:59:15.759260+0000","last_clean":"2024-04-17T05:59:15.759260+0000","last_became_active":"2024-04-17T05:59:14.587691+0000","last_became_peered":"2024-04-17T05:59:14.587691+0000","last_unstale":"2024-04-17T05:59:15.759260+0000","last_undegraded":"2024-04-17T05:59:15.759260+0000","last_fullsized":"2024-04-17T05:59:15.759260+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:11:15.775356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.582169+0000","last_change":"2024-04-17T05:59:14.589178+0000","last_active":"2024-04-17T05:59:15.582169+0000","last_peered":"2024-04-17T05:59:15.582169+0000","last_clean":"2024-04-17T05:59:15.582169+0000","last_became_active":"2024-04-17T05:59:14.587351+0000","last_became_peered":"2024-04-17T05:59:14.587351+0000","last_unstale":"2024-04-17T05:59:15.582169+0000","last_undegraded":"2024-04-17T05:59:15.582169+0000","last_fullsized":"2024-04-17T05:59:15.582169+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:22:43.202731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.757297+0000","last_change":"2024-04-17T05:59:13.553754+0000","last_active":"2024-04-17T05:59:15.757297+0000","last_peered":"2024-04-17T05:59:15.757297+0000","last_clean":"2024-04-17T05:59:15.757297+0000","last_became_active":"2024-04-17T05:59:13.553437+0000","last_became_peered":"2024-04-17T05:59:13.553437+0000","last_unstale":"2024-04-17T05:59:15.757297+0000","last_undegraded":"2024-04-17T05:59:15.757297+0000","last_fullsized":"2024-04-17T05:59:15.757297+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:10:14.010264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.582022+0000","last_change":"2024-04-17T05:59:14.592074+0000","last_active":"2024-04-17T05:59:15.582022+0000","last_peered":"2024-04-17T05:59:15.582022+0000","last_clean":"2024-04-17T05:59:15.582022+0000","last_became_active":"2024-04-17T05:59:14.591695+0000","last_became_peered":"2024-04-17T05:59:14.591695+0000","last_unstale":"2024-04-17T05:59:15.582022+0000","last_undegraded":"2024-04-17T05:59:15.582022+0000","last_fullsized":"2024-04-17T05:59:15.582022+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:27:04.344257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.758641+0000","last_change":"2024-04-17T05:59:13.553916+0000","last_active":"2024-04-17T05:59:15.758641+0000","last_peered":"2024-04-17T05:59:15.758641+0000","last_clean":"2024-04-17T05:59:15.758641+0000","last_became_active":"2024-04-17T05:59:13.553597+0000","last_became_peered":"2024-04-17T05:59:13.553597+0000","last_unstale":"2024-04-17T05:59:15.758641+0000","last_undegraded":"2024-04-17T05:59:15.758641+0000","last_fullsized":"2024-04-17T05:59:15.758641+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:28:23.486188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":348,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.757544+0000","last_change":"2024-04-17T05:58:02.597504+0000","last_active":"2024-04-17T05:59:15.757544+0000","last_peered":"2024-04-17T05:59:15.757544+0000","last_clean":"2024-04-17T05:59:15.757544+0000","last_became_active":"2024-04-17T05:58:02.597086+0000","last_became_peered":"2024-04-17T05:58:02.597086+0000","last_unstale":"2024-04-17T05:59:15.757544+0000","last_undegraded":"2024-04-17T05:59:15.757544+0000","last_fullsized":"2024-04-17T05:59:15.757544+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:54:37.919204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:54:37.919204+0000","last_clean_scrub_stamp":"2024-04-17T05:54:37.919204+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:02:09.566418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.757359+0000","last_change":"2024-04-17T05:59:14.584240+0000","last_active":"2024-04-17T05:59:15.757359+0000","last_peered":"2024-04-17T05:59:15.757359+0000","last_clean":"2024-04-17T05:59:15.757359+0000","last_became_active":"2024-04-17T05:59:14.583934+0000","last_became_peered":"2024-04-17T05:59:14.583934+0000","last_unstale":"2024-04-17T05:59:15.757359+0000","last_undegraded":"2024-04-17T05:59:15.757359+0000","last_fullsized":"2024-04-17T05:59:15.757359+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:50:41.891162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.757896+0000","last_change":"2024-04-17T05:59:13.562426+0000","last_active":"2024-04-17T05:59:15.757896+0000","last_peered":"2024-04-17T05:59:15.757896+0000","last_clean":"2024-04-17T05:59:15.757896+0000","last_became_active":"2024-04-17T05:59:13.562133+0000","last_became_peered":"2024-04-17T05:59:13.562133+0000","last_unstale":"2024-04-17T05:59:15.757896+0000","last_undegraded":"2024-04-17T05:59:15.757896+0000","last_fullsized":"2024-04-17T05:59:15.757896+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:50:58.351924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.758176+0000","last_change":"2024-04-17T05:59:13.557783+0000","last_active":"2024-04-17T05:59:15.758176+0000","last_peered":"2024-04-17T05:59:15.758176+0000","last_clean":"2024-04-17T05:59:15.758176+0000","last_became_active":"2024-04-17T05:59:13.557485+0000","last_became_peered":"2024-04-17T05:59:13.557485+0000","last_unstale":"2024-04-17T05:59:15.758176+0000","last_undegraded":"2024-04-17T05:59:15.758176+0000","last_fullsized":"2024-04-17T05:59:15.758176+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:41:37.359553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.758035+0000","last_change":"2024-04-17T05:59:14.575993+0000","last_active":"2024-04-17T05:59:15.758035+0000","last_peered":"2024-04-17T05:59:15.758035+0000","last_clean":"2024-04-17T05:59:15.758035+0000","last_became_active":"2024-04-17T05:59:14.575654+0000","last_became_peered":"2024-04-17T05:59:14.575654+0000","last_unstale":"2024-04-17T05:59:15.758035+0000","last_undegraded":"2024-04-17T05:59:15.758035+0000","last_fullsized":"2024-04-17T05:59:15.758035+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:33:12.193747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:18.631820+0000","last_change":"2024-04-17T05:59:13.560256+0000","last_active":"2024-04-17T05:59:18.631820+0000","last_peered":"2024-04-17T05:59:18.631820+0000","last_clean":"2024-04-17T05:59:18.631820+0000","last_became_active":"2024-04-17T05:59:13.559938+0000","last_became_peered":"2024-04-17T05:59:13.559938+0000","last_unstale":"2024-04-17T05:59:18.631820+0000","last_undegraded":"2024-04-17T05:59:18.631820+0000","last_fullsized":"2024-04-17T05:59:18.631820+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:14:54.625298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.582029+0000","last_change":"2024-04-17T05:59:14.593955+0000","last_active":"2024-04-17T05:59:15.582029+0000","last_peered":"2024-04-17T05:59:15.582029+0000","last_clean":"2024-04-17T05:59:15.582029+0000","last_became_active":"2024-04-17T05:59:14.590626+0000","last_became_peered":"2024-04-17T05:59:14.590626+0000","last_unstale":"2024-04-17T05:59:15.582029+0000","last_undegraded":"2024-04-17T05:59:15.582029+0000","last_fullsized":"2024-04-17T05:59:15.582029+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:19:04.438634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.758774+0000","last_change":"2024-04-17T05:59:13.555734+0000","last_active":"2024-04-17T05:59:15.758774+0000","last_peered":"2024-04-17T05:59:15.758774+0000","last_clean":"2024-04-17T05:59:15.758774+0000","last_became_active":"2024-04-17T05:59:13.555284+0000","last_became_peered":"2024-04-17T05:59:13.555284+0000","last_unstale":"2024-04-17T05:59:15.758774+0000","last_undegraded":"2024-04-17T05:59:15.758774+0000","last_fullsized":"2024-04-17T05:59:15.758774+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:47:06.834332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.758444+0000","last_change":"2024-04-17T05:59:14.599078+0000","last_active":"2024-04-17T05:59:15.758444+0000","last_peered":"2024-04-17T05:59:15.758444+0000","last_clean":"2024-04-17T05:59:15.758444+0000","last_became_active":"2024-04-17T05:59:14.598670+0000","last_became_peered":"2024-04-17T05:59:14.598670+0000","last_unstale":"2024-04-17T05:59:15.758444+0000","last_undegraded":"2024-04-17T05:59:15.758444+0000","last_fullsized":"2024-04-17T05:59:15.758444+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:58:17.414704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:17.626021+0000","last_change":"2024-04-17T05:59:13.560154+0000","last_active":"2024-04-17T05:59:17.626021+0000","last_peered":"2024-04-17T05:59:17.626021+0000","last_clean":"2024-04-17T05:59:17.626021+0000","last_became_active":"2024-04-17T05:59:13.559846+0000","last_became_peered":"2024-04-17T05:59:13.559846+0000","last_unstale":"2024-04-17T05:59:17.626021+0000","last_undegraded":"2024-04-17T05:59:17.626021+0000","last_fullsized":"2024-04-17T05:59:17.626021+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:06:03.200186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.757575+0000","last_change":"2024-04-17T05:59:14.593450+0000","last_active":"2024-04-17T05:59:15.757575+0000","last_peered":"2024-04-17T05:59:15.757575+0000","last_clean":"2024-04-17T05:59:15.757575+0000","last_became_active":"2024-04-17T05:59:14.593104+0000","last_became_peered":"2024-04-17T05:59:14.593104+0000","last_unstale":"2024-04-17T05:59:15.757575+0000","last_undegraded":"2024-04-17T05:59:15.757575+0000","last_fullsized":"2024-04-17T05:59:15.757575+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:16:20.488110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:17.627101+0000","last_change":"2024-04-17T05:59:13.557330+0000","last_active":"2024-04-17T05:59:17.627101+0000","last_peered":"2024-04-17T05:59:17.627101+0000","last_clean":"2024-04-17T05:59:17.627101+0000","last_became_active":"2024-04-17T05:59:13.556970+0000","last_became_peered":"2024-04-17T05:59:13.556970+0000","last_unstale":"2024-04-17T05:59:17.627101+0000","last_undegraded":"2024-04-17T05:59:17.627101+0000","last_fullsized":"2024-04-17T05:59:17.627101+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:41:46.643845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.757892+0000","last_change":"2024-04-17T05:59:14.582281+0000","last_active":"2024-04-17T05:59:15.757892+0000","last_peered":"2024-04-17T05:59:15.757892+0000","last_clean":"2024-04-17T05:59:15.757892+0000","last_became_active":"2024-04-17T05:59:14.581922+0000","last_became_peered":"2024-04-17T05:59:14.581922+0000","last_unstale":"2024-04-17T05:59:15.757892+0000","last_undegraded":"2024-04-17T05:59:15.757892+0000","last_fullsized":"2024-04-17T05:59:15.757892+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:53:10.415786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.758188+0000","last_change":"2024-04-17T05:59:13.566514+0000","last_active":"2024-04-17T05:59:15.758188+0000","last_peered":"2024-04-17T05:59:15.758188+0000","last_clean":"2024-04-17T05:59:15.758188+0000","last_became_active":"2024-04-17T05:59:13.566004+0000","last_became_peered":"2024-04-17T05:59:13.566004+0000","last_unstale":"2024-04-17T05:59:15.758188+0000","last_undegraded":"2024-04-17T05:59:15.758188+0000","last_fullsized":"2024-04-17T05:59:15.758188+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:20:13.891422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.581731+0000","last_change":"2024-04-17T05:59:14.599101+0000","last_active":"2024-04-17T05:59:15.581731+0000","last_peered":"2024-04-17T05:59:15.581731+0000","last_clean":"2024-04-17T05:59:15.581731+0000","last_became_active":"2024-04-17T05:59:14.598310+0000","last_became_peered":"2024-04-17T05:59:14.598310+0000","last_unstale":"2024-04-17T05:59:15.581731+0000","last_undegraded":"2024-04-17T05:59:15.581731+0000","last_fullsized":"2024-04-17T05:59:15.581731+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:46:39.275195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.757856+0000","last_change":"2024-04-17T05:59:13.562531+0000","last_active":"2024-04-17T05:59:15.757856+0000","last_peered":"2024-04-17T05:59:15.757856+0000","last_clean":"2024-04-17T05:59:15.757856+0000","last_became_active":"2024-04-17T05:59:13.562148+0000","last_became_peered":"2024-04-17T05:59:13.562148+0000","last_unstale":"2024-04-17T05:59:15.757856+0000","last_undegraded":"2024-04-17T05:59:15.757856+0000","last_fullsized":"2024-04-17T05:59:15.757856+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:10:14.407871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.758266+0000","last_change":"2024-04-17T05:59:14.596371+0000","last_active":"2024-04-17T05:59:15.758266+0000","last_peered":"2024-04-17T05:59:15.758266+0000","last_clean":"2024-04-17T05:59:15.758266+0000","last_became_active":"2024-04-17T05:59:14.596024+0000","last_became_peered":"2024-04-17T05:59:14.596024+0000","last_unstale":"2024-04-17T05:59:15.758266+0000","last_undegraded":"2024-04-17T05:59:15.758266+0000","last_fullsized":"2024-04-17T05:59:15.758266+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:06:14.916212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.758890+0000","last_change":"2024-04-17T05:59:13.569364+0000","last_active":"2024-04-17T05:59:15.758890+0000","last_peered":"2024-04-17T05:59:15.758890+0000","last_clean":"2024-04-17T05:59:15.758890+0000","last_became_active":"2024-04-17T05:59:13.569029+0000","last_became_peered":"2024-04-17T05:59:13.569029+0000","last_unstale":"2024-04-17T05:59:15.758890+0000","last_undegraded":"2024-04-17T05:59:15.758890+0000","last_fullsized":"2024-04-17T05:59:15.758890+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:58:24.629274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.758437+0000","last_change":"2024-04-17T05:59:14.593088+0000","last_active":"2024-04-17T05:59:15.758437+0000","last_peered":"2024-04-17T05:59:15.758437+0000","last_clean":"2024-04-17T05:59:15.758437+0000","last_became_active":"2024-04-17T05:59:14.592755+0000","last_became_peered":"2024-04-17T05:59:14.592755+0000","last_unstale":"2024-04-17T05:59:15.758437+0000","last_undegraded":"2024-04-17T05:59:15.758437+0000","last_fullsized":"2024-04-17T05:59:15.758437+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:27:14.503323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.582616+0000","last_change":"2024-04-17T05:59:13.569773+0000","last_active":"2024-04-17T05:59:15.582616+0000","last_peered":"2024-04-17T05:59:15.582616+0000","last_clean":"2024-04-17T05:59:15.582616+0000","last_became_active":"2024-04-17T05:59:13.569461+0000","last_became_peered":"2024-04-17T05:59:13.569461+0000","last_unstale":"2024-04-17T05:59:15.582616+0000","last_undegraded":"2024-04-17T05:59:15.582616+0000","last_fullsized":"2024-04-17T05:59:15.582616+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:21:13.441453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.757996+0000","last_change":"2024-04-17T05:59:14.588939+0000","last_active":"2024-04-17T05:59:15.757996+0000","last_peered":"2024-04-17T05:59:15.757996+0000","last_clean":"2024-04-17T05:59:15.757996+0000","last_became_active":"2024-04-17T05:59:14.588600+0000","last_became_peered":"2024-04-17T05:59:14.588600+0000","last_unstale":"2024-04-17T05:59:15.757996+0000","last_undegraded":"2024-04-17T05:59:15.757996+0000","last_fullsized":"2024-04-17T05:59:15.757996+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:07:47.865808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.757878+0000","last_change":"2024-04-17T05:59:13.560566+0000","last_active":"2024-04-17T05:59:15.757878+0000","last_peered":"2024-04-17T05:59:15.757878+0000","last_clean":"2024-04-17T05:59:15.757878+0000","last_became_active":"2024-04-17T05:59:13.560228+0000","last_became_peered":"2024-04-17T05:59:13.560228+0000","last_unstale":"2024-04-17T05:59:15.757878+0000","last_undegraded":"2024-04-17T05:59:15.757878+0000","last_fullsized":"2024-04-17T05:59:15.757878+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:13:59.436598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.757853+0000","last_change":"2024-04-17T05:59:14.583621+0000","last_active":"2024-04-17T05:59:15.757853+0000","last_peered":"2024-04-17T05:59:15.757853+0000","last_clean":"2024-04-17T05:59:15.757853+0000","last_became_active":"2024-04-17T05:59:14.583199+0000","last_became_peered":"2024-04-17T05:59:14.583199+0000","last_unstale":"2024-04-17T05:59:15.757853+0000","last_undegraded":"2024-04-17T05:59:15.757853+0000","last_fullsized":"2024-04-17T05:59:15.757853+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:53:12.516661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.758071+0000","last_change":"2024-04-17T05:59:14.587989+0000","last_active":"2024-04-17T05:59:15.758071+0000","last_peered":"2024-04-17T05:59:15.758071+0000","last_clean":"2024-04-17T05:59:15.758071+0000","last_became_active":"2024-04-17T05:59:14.586686+0000","last_became_peered":"2024-04-17T05:59:14.586686+0000","last_unstale":"2024-04-17T05:59:15.758071+0000","last_undegraded":"2024-04-17T05:59:15.758071+0000","last_fullsized":"2024-04-17T05:59:15.758071+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:21:23.227328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.759265+0000","last_change":"2024-04-17T05:59:14.588567+0000","last_active":"2024-04-17T05:59:15.759265+0000","last_peered":"2024-04-17T05:59:15.759265+0000","last_clean":"2024-04-17T05:59:15.759265+0000","last_became_active":"2024-04-17T05:59:14.588207+0000","last_became_peered":"2024-04-17T05:59:14.588207+0000","last_unstale":"2024-04-17T05:59:15.759265+0000","last_undegraded":"2024-04-17T05:59:15.759265+0000","last_fullsized":"2024-04-17T05:59:15.759265+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:44:26.094334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.758071+0000","last_change":"2024-04-17T05:59:14.593885+0000","last_active":"2024-04-17T05:59:15.758071+0000","last_peered":"2024-04-17T05:59:15.758071+0000","last_clean":"2024-04-17T05:59:15.758071+0000","last_became_active":"2024-04-17T05:59:14.593483+0000","last_became_peered":"2024-04-17T05:59:14.593483+0000","last_unstale":"2024-04-17T05:59:15.758071+0000","last_undegraded":"2024-04-17T05:59:15.758071+0000","last_fullsized":"2024-04-17T05:59:15.758071+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:22:41.837363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.758704+0000","last_change":"2024-04-17T05:59:14.585666+0000","last_active":"2024-04-17T05:59:15.758704+0000","last_peered":"2024-04-17T05:59:15.758704+0000","last_clean":"2024-04-17T05:59:15.758704+0000","last_became_active":"2024-04-17T05:59:14.585349+0000","last_became_peered":"2024-04-17T05:59:14.585349+0000","last_unstale":"2024-04-17T05:59:15.758704+0000","last_undegraded":"2024-04-17T05:59:15.758704+0000","last_fullsized":"2024-04-17T05:59:15.758704+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:32:24.776931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.758054+0000","last_change":"2024-04-17T05:59:14.583266+0000","last_active":"2024-04-17T05:59:15.758054+0000","last_peered":"2024-04-17T05:59:15.758054+0000","last_clean":"2024-04-17T05:59:15.758054+0000","last_became_active":"2024-04-17T05:59:14.582949+0000","last_became_peered":"2024-04-17T05:59:14.582949+0000","last_unstale":"2024-04-17T05:59:15.758054+0000","last_undegraded":"2024-04-17T05:59:15.758054+0000","last_fullsized":"2024-04-17T05:59:15.758054+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:49:14.740168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.758111+0000","last_change":"2024-04-17T05:59:14.587046+0000","last_active":"2024-04-17T05:59:15.758111+0000","last_peered":"2024-04-17T05:59:15.758111+0000","last_clean":"2024-04-17T05:59:15.758111+0000","last_became_active":"2024-04-17T05:59:14.586653+0000","last_became_peered":"2024-04-17T05:59:14.586653+0000","last_unstale":"2024-04-17T05:59:15.758111+0000","last_undegraded":"2024-04-17T05:59:15.758111+0000","last_fullsized":"2024-04-17T05:59:15.758111+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:11:57.171410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.759198+0000","last_change":"2024-04-17T05:59:14.589670+0000","last_active":"2024-04-17T05:59:15.759198+0000","last_peered":"2024-04-17T05:59:15.759198+0000","last_clean":"2024-04-17T05:59:15.759198+0000","last_became_active":"2024-04-17T05:59:14.589003+0000","last_became_peered":"2024-04-17T05:59:14.589003+0000","last_unstale":"2024-04-17T05:59:15.759198+0000","last_undegraded":"2024-04-17T05:59:15.759198+0000","last_fullsized":"2024-04-17T05:59:15.759198+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:21:19.574178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.581833+0000","last_change":"2024-04-17T05:59:14.594107+0000","last_active":"2024-04-17T05:59:15.581833+0000","last_peered":"2024-04-17T05:59:15.581833+0000","last_clean":"2024-04-17T05:59:15.581833+0000","last_became_active":"2024-04-17T05:59:14.593799+0000","last_became_peered":"2024-04-17T05:59:14.593799+0000","last_unstale":"2024-04-17T05:59:15.581833+0000","last_undegraded":"2024-04-17T05:59:15.581833+0000","last_fullsized":"2024-04-17T05:59:15.581833+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:23:45.222667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.582004+0000","last_change":"2024-04-17T05:59:14.587671+0000","last_active":"2024-04-17T05:59:15.582004+0000","last_peered":"2024-04-17T05:59:15.582004+0000","last_clean":"2024-04-17T05:59:15.582004+0000","last_became_active":"2024-04-17T05:59:14.586473+0000","last_became_peered":"2024-04-17T05:59:14.586473+0000","last_unstale":"2024-04-17T05:59:15.582004+0000","last_undegraded":"2024-04-17T05:59:15.582004+0000","last_fullsized":"2024-04-17T05:59:15.582004+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:31:42.156354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.581565+0000","last_change":"2024-04-17T05:59:14.586961+0000","last_active":"2024-04-17T05:59:15.581565+0000","last_peered":"2024-04-17T05:59:15.581565+0000","last_clean":"2024-04-17T05:59:15.581565+0000","last_became_active":"2024-04-17T05:59:14.586601+0000","last_became_peered":"2024-04-17T05:59:14.586601+0000","last_unstale":"2024-04-17T05:59:15.581565+0000","last_undegraded":"2024-04-17T05:59:15.581565+0000","last_fullsized":"2024-04-17T05:59:15.581565+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:21:58.851523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.585223+0000","last_change":"2024-04-17T05:59:14.601239+0000","last_active":"2024-04-17T05:59:15.585223+0000","last_peered":"2024-04-17T05:59:15.585223+0000","last_clean":"2024-04-17T05:59:15.585223+0000","last_became_active":"2024-04-17T05:59:14.600852+0000","last_became_peered":"2024-04-17T05:59:14.600852+0000","last_unstale":"2024-04-17T05:59:15.585223+0000","last_undegraded":"2024-04-17T05:59:15.585223+0000","last_fullsized":"2024-04-17T05:59:15.585223+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:14:02.112456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.757920+0000","last_change":"2024-04-17T05:59:14.579952+0000","last_active":"2024-04-17T05:59:15.757920+0000","last_peered":"2024-04-17T05:59:15.757920+0000","last_clean":"2024-04-17T05:59:15.757920+0000","last_became_active":"2024-04-17T05:59:14.579625+0000","last_became_peered":"2024-04-17T05:59:14.579625+0000","last_unstale":"2024-04-17T05:59:15.757920+0000","last_undegraded":"2024-04-17T05:59:15.757920+0000","last_fullsized":"2024-04-17T05:59:15.757920+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:22:32.709762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.757838+0000","last_change":"2024-04-17T05:59:14.585187+0000","last_active":"2024-04-17T05:59:15.757838+0000","last_peered":"2024-04-17T05:59:15.757838+0000","last_clean":"2024-04-17T05:59:15.757838+0000","last_became_active":"2024-04-17T05:59:14.584820+0000","last_became_peered":"2024-04-17T05:59:14.584820+0000","last_unstale":"2024-04-17T05:59:15.757838+0000","last_undegraded":"2024-04-17T05:59:15.757838+0000","last_fullsized":"2024-04-17T05:59:15.757838+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:53:58.945885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":249153,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":3794,"num_objects":36,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":7,"num_read_kb":6,"num_write":77,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":184320,"data_stored":12204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":67,"ondisk_log_size":67,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350990,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":925696,"data_stored":442154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612627,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1015808,"data_stored":1035130,"data_compressed":36061,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776152,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36252,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708900,"statfs":{"total":95995035648,"available":95957913600,"internally_reserved":0,"allocated":937984,"data_stored":442356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":36174273},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037789,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":929792,"data_stored":442176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299424,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":974848,"data_stored":1032522,"data_compressed":36061,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462951,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":921600,"data_stored":441572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574912,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":933888,"data_stored":443184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738437,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36264,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708888,"statfs":{"total":95995035648,"available":95957901312,"internally_reserved":0,"allocated":950272,"data_stored":443050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411387,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":937984,"data_stored":443218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149750,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":929792,"data_stored":441676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888113,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":925696,"data_stored":441594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724588,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":1033416,"data_compressed":36061,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34424,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34424,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36061,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36061,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36061,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36061,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36061,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T05:59:22.460 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T05:59:22.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.606+0000 7f2444332640 1 -- 172.21.15.47:0/3588209938 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f243c1031a0 msgr2=0x7f243c103620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:22.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.606+0000 7f2444332640 1 --2- 172.21.15.47:0/3588209938 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f243c1031a0 0x7f243c103620 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f242c009900 tx=0x7f242c02bc30 comp rx=0 tx=0).stop 2024-04-17T05:59:22.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.606+0000 7f2444332640 1 -- 172.21.15.47:0/3588209938 shutdown_connections 2024-04-17T05:59:22.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.606+0000 7f2444332640 1 --2- 172.21.15.47:0/3588209938 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f243c103b60 0x7f243c10a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:22.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.606+0000 7f2444332640 1 --2- 172.21.15.47:0/3588209938 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f243c1031a0 0x7f243c103620 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:22.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.606+0000 7f2444332640 1 --2- 172.21.15.47:0/3588209938 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f243c101fb0 0x7f243c1023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:22.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.606+0000 7f2444332640 1 -- 172.21.15.47:0/3588209938 >> 172.21.15.47:0/3588209938 conn(0x7f243c0fd540 msgr2=0x7f243c0ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:59:22.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.606+0000 7f2444332640 1 -- 172.21.15.47:0/3588209938 shutdown_connections 2024-04-17T05:59:22.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.606+0000 7f2444332640 1 -- 172.21.15.47:0/3588209938 wait complete. 2024-04-17T05:59:22.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.607+0000 7f2444332640 1 Processor -- start 2024-04-17T05:59:22.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.607+0000 7f2444332640 1 -- start start 2024-04-17T05:59:22.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.607+0000 7f2444332640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f243c101fb0 0x7f243c198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:22.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.607+0000 7f2444332640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f243c1031a0 0x7f243c1987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:22.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.607+0000 7f2444332640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f243c103b60 0x7f243c19f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:22.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.607+0000 7f2444332640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f243c10dc40 con 0x7f243c1031a0 2024-04-17T05:59:22.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.607+0000 7f2444332640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f243c10dac0 con 0x7f243c101fb0 2024-04-17T05:59:22.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.607+0000 7f2444332640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f243c10ddc0 con 0x7f243c103b60 2024-04-17T05:59:22.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.607+0000 7f24418cd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f243c1031a0 0x7f243c1987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:22.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.607+0000 7f24418cd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f243c1031a0 0x7f243c1987c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58058/0 (socket says 172.21.15.47:58058) 2024-04-17T05:59:22.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.607+0000 7f24418cd640 1 -- 172.21.15.47:0/3805938598 learned_addr learned my addr 172.21.15.47:0/3805938598 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:59:22.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.607+0000 7f24428cf640 1 --2- 172.21.15.47:0/3805938598 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f243c103b60 0x7f243c19f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:22.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.607+0000 7f24420ce640 1 --2- 172.21.15.47:0/3805938598 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f243c101fb0 0x7f243c198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:22.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.607+0000 7f24418cd640 1 -- 172.21.15.47:0/3805938598 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f243c103b60 msgr2=0x7f243c19f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:22.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.608+0000 7f24418cd640 1 --2- 172.21.15.47:0/3805938598 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f243c103b60 0x7f243c19f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:22.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.608+0000 7f24418cd640 1 -- 172.21.15.47:0/3805938598 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f243c101fb0 msgr2=0x7f243c198280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:22.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.608+0000 7f24418cd640 1 --2- 172.21.15.47:0/3805938598 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f243c101fb0 0x7f243c198280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:22.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.608+0000 7f24418cd640 1 -- 172.21.15.47:0/3805938598 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f243c19fea0 con 0x7f243c1031a0 2024-04-17T05:59:22.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.608+0000 7f24420ce640 1 --2- 172.21.15.47:0/3805938598 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f243c101fb0 0x7f243c198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T05:59:22.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.608+0000 7f24418cd640 1 --2- 172.21.15.47:0/3805938598 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f243c1031a0 0x7f243c1987c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f242c009230 tx=0x7f242c005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:59:22.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.608+0000 7f24337fe640 1 -- 172.21.15.47:0/3805938598 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f242c041070 con 0x7f243c1031a0 2024-04-17T05:59:22.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.608+0000 7f24337fe640 1 -- 172.21.15.47:0/3805938598 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f242c0044d0 con 0x7f243c1031a0 2024-04-17T05:59:22.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.608+0000 7f24337fe640 1 -- 172.21.15.47:0/3805938598 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f242c034440 con 0x7f243c1031a0 2024-04-17T05:59:22.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.608+0000 7f2444332640 1 -- 172.21.15.47:0/3805938598 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f243c1a0130 con 0x7f243c1031a0 2024-04-17T05:59:22.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.608+0000 7f2444332640 1 -- 172.21.15.47:0/3805938598 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f243c1a0620 con 0x7f243c1031a0 2024-04-17T05:59:22.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.609+0000 7f2444332640 1 -- 172.21.15.47:0/3805938598 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f243c109350 con 0x7f243c1031a0 2024-04-17T05:59:22.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.610+0000 7f24337fe640 1 -- 172.21.15.47:0/3805938598 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f242c044070 con 0x7f243c1031a0 2024-04-17T05:59:22.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.611+0000 7f24337fe640 1 --2- 172.21.15.47:0/3805938598 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f24280a3bb0 0x7f24280a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:22.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.611+0000 7f24420ce640 1 --2- 172.21.15.47:0/3805938598 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f24280a3bb0 0x7f24280a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:22.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.611+0000 7f24337fe640 1 -- 172.21.15.47:0/3805938598 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f242c0f1fa0 con 0x7f243c1031a0 2024-04-17T05:59:22.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.611+0000 7f24420ce640 1 --2- 172.21.15.47:0/3805938598 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f24280a3bb0 0x7f24280a6070 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f243c103000 tx=0x7f2438005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:59:22.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.616+0000 7f24337fe640 1 -- 172.21.15.47:0/3805938598 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f242c0bea30 con 0x7f243c1031a0 2024-04-17T05:59:22.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.634+0000 7f24337fe640 1 -- 172.21.15.47:0/3805938598 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f242c0ba9a0 con 0x7f243c1031a0 2024-04-17T05:59:22.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.805+0000 7f2444332640 1 -- 172.21.15.47:0/3805938598 --> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f243c0623a0 con 0x7f24280a3bb0 2024-04-17T05:59:22.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.812+0000 7f24337fe640 1 -- 172.21.15.47:0/3805938598 <== mgr.14152 v2:172.21.15.47:6800/3157902806 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258864 (secure 0 0 0) 0x7f243c0623a0 con 0x7f24280a3bb0 2024-04-17T05:59:22.814 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-17T05:59:22.815 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-04-17T05:59:22.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.815+0000 7f2444332640 1 -- 172.21.15.47:0/3805938598 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f24280a3bb0 msgr2=0x7f24280a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:22.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.815+0000 7f2444332640 1 --2- 172.21.15.47:0/3805938598 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f24280a3bb0 0x7f24280a6070 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f243c103000 tx=0x7f2438005f90 comp rx=0 tx=0).stop 2024-04-17T05:59:22.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.815+0000 7f2444332640 1 -- 172.21.15.47:0/3805938598 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f243c1031a0 msgr2=0x7f243c1987c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:22.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.815+0000 7f2444332640 1 --2- 172.21.15.47:0/3805938598 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f243c1031a0 0x7f243c1987c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f242c009230 tx=0x7f242c005d50 comp rx=0 tx=0).stop 2024-04-17T05:59:22.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.815+0000 7f2444332640 1 -- 172.21.15.47:0/3805938598 shutdown_connections 2024-04-17T05:59:22.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.815+0000 7f2444332640 1 --2- 172.21.15.47:0/3805938598 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f24280a3bb0 0x7f24280a6070 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:22.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.815+0000 7f2444332640 1 --2- 172.21.15.47:0/3805938598 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f243c103b60 0x7f243c19f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:22.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.815+0000 7f2444332640 1 --2- 172.21.15.47:0/3805938598 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f243c1031a0 0x7f243c1987c0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:22.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.815+0000 7f2444332640 1 --2- 172.21.15.47:0/3805938598 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f243c101fb0 0x7f243c198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:22.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.815+0000 7f2444332640 1 -- 172.21.15.47:0/3805938598 >> 172.21.15.47:0/3805938598 conn(0x7f243c0fd540 msgr2=0x7f243c106520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:59:22.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.816+0000 7f2444332640 1 -- 172.21.15.47:0/3805938598 shutdown_connections 2024-04-17T05:59:22.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:22.816+0000 7f2444332640 1 -- 172.21.15.47:0/3805938598 wait complete. 2024-04-17T05:59:22.824 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:22 smithi047 ceph-mon[32223]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T05:59:22.824 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:22 smithi047 ceph-mon[32223]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T05:59:22.824 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:22 smithi047 ceph-mon[32223]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T05:59:22.824 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:22 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/2187541661' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T05:59:22.825 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:22 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/2119612838' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T05:59:22.825 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":290,"stamp":"2024-04-17T05:59:21.724593+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594162,"num_objects":38,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":0,"num_read":143,"num_read_kb":125,"num_write":258,"num_write_kb":2957,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":204,"ondisk_log_size":204,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":261,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":364488,"kb_used_data":11116,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124577336,"statfs":{"total":1151940427776,"available":1151567192064,"internally_reserved":0,"allocated":11382784,"data_stored":7082048,"data_compressed":108183,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19140,"internal_metadata":361674044},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3794,"num_objects":36,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":7,"num_read_kb":6,"num_write":77,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001520"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.758360+0000","last_change":"2024-04-17T05:59:14.592927+0000","last_active":"2024-04-17T05:59:15.758360+0000","last_peered":"2024-04-17T05:59:15.758360+0000","last_clean":"2024-04-17T05:59:15.758360+0000","last_became_active":"2024-04-17T05:59:14.592036+0000","last_became_peered":"2024-04-17T05:59:14.592036+0000","last_unstale":"2024-04-17T05:59:15.758360+0000","last_undegraded":"2024-04-17T05:59:15.758360+0000","last_fullsized":"2024-04-17T05:59:15.758360+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:10:51.146397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.758295+0000","last_change":"2024-04-17T05:59:14.587862+0000","last_active":"2024-04-17T05:59:15.758295+0000","last_peered":"2024-04-17T05:59:15.758295+0000","last_clean":"2024-04-17T05:59:15.758295+0000","last_became_active":"2024-04-17T05:59:14.587553+0000","last_became_peered":"2024-04-17T05:59:14.587553+0000","last_unstale":"2024-04-17T05:59:15.758295+0000","last_undegraded":"2024-04-17T05:59:15.758295+0000","last_fullsized":"2024-04-17T05:59:15.758295+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:16:12.224247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.758168+0000","last_change":"2024-04-17T05:59:14.592223+0000","last_active":"2024-04-17T05:59:15.758168+0000","last_peered":"2024-04-17T05:59:15.758168+0000","last_clean":"2024-04-17T05:59:15.758168+0000","last_became_active":"2024-04-17T05:59:14.591892+0000","last_became_peered":"2024-04-17T05:59:14.591892+0000","last_unstale":"2024-04-17T05:59:15.758168+0000","last_undegraded":"2024-04-17T05:59:15.758168+0000","last_fullsized":"2024-04-17T05:59:15.758168+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:23:26.943368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.758499+0000","last_change":"2024-04-17T05:59:14.585607+0000","last_active":"2024-04-17T05:59:15.758499+0000","last_peered":"2024-04-17T05:59:15.758499+0000","last_clean":"2024-04-17T05:59:15.758499+0000","last_became_active":"2024-04-17T05:59:14.585255+0000","last_became_peered":"2024-04-17T05:59:14.585255+0000","last_unstale":"2024-04-17T05:59:15.758499+0000","last_undegraded":"2024-04-17T05:59:15.758499+0000","last_fullsized":"2024-04-17T05:59:15.758499+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:45:57.355035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.757860+0000","last_change":"2024-04-17T05:59:14.599703+0000","last_active":"2024-04-17T05:59:15.757860+0000","last_peered":"2024-04-17T05:59:15.757860+0000","last_clean":"2024-04-17T05:59:15.757860+0000","last_became_active":"2024-04-17T05:59:14.599305+0000","last_became_peered":"2024-04-17T05:59:14.599305+0000","last_unstale":"2024-04-17T05:59:15.757860+0000","last_undegraded":"2024-04-17T05:59:15.757860+0000","last_fullsized":"2024-04-17T05:59:15.757860+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:58:36.056590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.758977+0000","last_change":"2024-04-17T05:59:14.586176+0000","last_active":"2024-04-17T05:59:15.758977+0000","last_peered":"2024-04-17T05:59:15.758977+0000","last_clean":"2024-04-17T05:59:15.758977+0000","last_became_active":"2024-04-17T05:59:14.585850+0000","last_became_peered":"2024-04-17T05:59:14.585850+0000","last_unstale":"2024-04-17T05:59:15.758977+0000","last_undegraded":"2024-04-17T05:59:15.758977+0000","last_fullsized":"2024-04-17T05:59:15.758977+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:01:14.628048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.757673+0000","last_change":"2024-04-17T05:59:14.587750+0000","last_active":"2024-04-17T05:59:15.757673+0000","last_peered":"2024-04-17T05:59:15.757673+0000","last_clean":"2024-04-17T05:59:15.757673+0000","last_became_active":"2024-04-17T05:59:14.587394+0000","last_became_peered":"2024-04-17T05:59:14.587394+0000","last_unstale":"2024-04-17T05:59:15.757673+0000","last_undegraded":"2024-04-17T05:59:15.757673+0000","last_fullsized":"2024-04-17T05:59:15.757673+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:22:52.733877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.581516+0000","last_change":"2024-04-17T05:59:14.591458+0000","last_active":"2024-04-17T05:59:15.581516+0000","last_peered":"2024-04-17T05:59:15.581516+0000","last_clean":"2024-04-17T05:59:15.581516+0000","last_became_active":"2024-04-17T05:59:14.590898+0000","last_became_peered":"2024-04-17T05:59:14.590898+0000","last_unstale":"2024-04-17T05:59:15.581516+0000","last_undegraded":"2024-04-17T05:59:15.581516+0000","last_fullsized":"2024-04-17T05:59:15.581516+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:58:40.817928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.585312+0000","last_change":"2024-04-17T05:59:14.600199+0000","last_active":"2024-04-17T05:59:15.585312+0000","last_peered":"2024-04-17T05:59:15.585312+0000","last_clean":"2024-04-17T05:59:15.585312+0000","last_became_active":"2024-04-17T05:59:14.599286+0000","last_became_peered":"2024-04-17T05:59:14.599286+0000","last_unstale":"2024-04-17T05:59:15.585312+0000","last_undegraded":"2024-04-17T05:59:15.585312+0000","last_fullsized":"2024-04-17T05:59:15.585312+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:41:43.314103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.759270+0000","last_change":"2024-04-17T05:59:14.587999+0000","last_active":"2024-04-17T05:59:15.759270+0000","last_peered":"2024-04-17T05:59:15.759270+0000","last_clean":"2024-04-17T05:59:15.759270+0000","last_became_active":"2024-04-17T05:59:14.587655+0000","last_became_peered":"2024-04-17T05:59:14.587655+0000","last_unstale":"2024-04-17T05:59:15.759270+0000","last_undegraded":"2024-04-17T05:59:15.759270+0000","last_fullsized":"2024-04-17T05:59:15.759270+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:41:37.751776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.758882+0000","last_change":"2024-04-17T05:59:14.595003+0000","last_active":"2024-04-17T05:59:15.758882+0000","last_peered":"2024-04-17T05:59:15.758882+0000","last_clean":"2024-04-17T05:59:15.758882+0000","last_became_active":"2024-04-17T05:59:14.594424+0000","last_became_peered":"2024-04-17T05:59:14.594424+0000","last_unstale":"2024-04-17T05:59:15.758882+0000","last_undegraded":"2024-04-17T05:59:15.758882+0000","last_fullsized":"2024-04-17T05:59:15.758882+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:35:39.491074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.758537+0000","last_change":"2024-04-17T05:59:14.584957+0000","last_active":"2024-04-17T05:59:15.758537+0000","last_peered":"2024-04-17T05:59:15.758537+0000","last_clean":"2024-04-17T05:59:15.758537+0000","last_became_active":"2024-04-17T05:59:14.584625+0000","last_became_peered":"2024-04-17T05:59:14.584625+0000","last_unstale":"2024-04-17T05:59:15.758537+0000","last_undegraded":"2024-04-17T05:59:15.758537+0000","last_fullsized":"2024-04-17T05:59:15.758537+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:10:08.819831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.757871+0000","last_change":"2024-04-17T05:59:14.577633+0000","last_active":"2024-04-17T05:59:15.757871+0000","last_peered":"2024-04-17T05:59:15.757871+0000","last_clean":"2024-04-17T05:59:15.757871+0000","last_became_active":"2024-04-17T05:59:14.576608+0000","last_became_peered":"2024-04-17T05:59:14.576608+0000","last_unstale":"2024-04-17T05:59:15.757871+0000","last_undegraded":"2024-04-17T05:59:15.757871+0000","last_fullsized":"2024-04-17T05:59:15.757871+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:04:49.921958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.758060+0000","last_change":"2024-04-17T05:59:14.580912+0000","last_active":"2024-04-17T05:59:15.758060+0000","last_peered":"2024-04-17T05:59:15.758060+0000","last_clean":"2024-04-17T05:59:15.758060+0000","last_became_active":"2024-04-17T05:59:14.580500+0000","last_became_peered":"2024-04-17T05:59:14.580500+0000","last_unstale":"2024-04-17T05:59:15.758060+0000","last_undegraded":"2024-04-17T05:59:15.758060+0000","last_fullsized":"2024-04-17T05:59:15.758060+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:17:49.893499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.757641+0000","last_change":"2024-04-17T05:59:14.586109+0000","last_active":"2024-04-17T05:59:15.757641+0000","last_peered":"2024-04-17T05:59:15.757641+0000","last_clean":"2024-04-17T05:59:15.757641+0000","last_became_active":"2024-04-17T05:59:14.585667+0000","last_became_peered":"2024-04-17T05:59:14.585667+0000","last_unstale":"2024-04-17T05:59:15.757641+0000","last_undegraded":"2024-04-17T05:59:15.757641+0000","last_fullsized":"2024-04-17T05:59:15.757641+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:20:22.309678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.584844+0000","last_change":"2024-04-17T05:59:14.602921+0000","last_active":"2024-04-17T05:59:15.584844+0000","last_peered":"2024-04-17T05:59:15.584844+0000","last_clean":"2024-04-17T05:59:15.584844+0000","last_became_active":"2024-04-17T05:59:14.602589+0000","last_became_peered":"2024-04-17T05:59:14.602589+0000","last_unstale":"2024-04-17T05:59:15.584844+0000","last_undegraded":"2024-04-17T05:59:15.584844+0000","last_fullsized":"2024-04-17T05:59:15.584844+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:49:17.988067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.758438+0000","last_change":"2024-04-17T05:59:14.594201+0000","last_active":"2024-04-17T05:59:15.758438+0000","last_peered":"2024-04-17T05:59:15.758438+0000","last_clean":"2024-04-17T05:59:15.758438+0000","last_became_active":"2024-04-17T05:59:14.593850+0000","last_became_peered":"2024-04-17T05:59:14.593850+0000","last_unstale":"2024-04-17T05:59:15.758438+0000","last_undegraded":"2024-04-17T05:59:15.758438+0000","last_fullsized":"2024-04-17T05:59:15.758438+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:53:09.044324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.758499+0000","last_change":"2024-04-17T05:59:14.590439+0000","last_active":"2024-04-17T05:59:15.758499+0000","last_peered":"2024-04-17T05:59:15.758499+0000","last_clean":"2024-04-17T05:59:15.758499+0000","last_became_active":"2024-04-17T05:59:14.589844+0000","last_became_peered":"2024-04-17T05:59:14.589844+0000","last_unstale":"2024-04-17T05:59:15.758499+0000","last_undegraded":"2024-04-17T05:59:15.758499+0000","last_fullsized":"2024-04-17T05:59:15.758499+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:02:18.366048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.759032+0000","last_change":"2024-04-17T05:59:14.592297+0000","last_active":"2024-04-17T05:59:15.759032+0000","last_peered":"2024-04-17T05:59:15.759032+0000","last_clean":"2024-04-17T05:59:15.759032+0000","last_became_active":"2024-04-17T05:59:14.591976+0000","last_became_peered":"2024-04-17T05:59:14.591976+0000","last_unstale":"2024-04-17T05:59:15.759032+0000","last_undegraded":"2024-04-17T05:59:15.759032+0000","last_fullsized":"2024-04-17T05:59:15.759032+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:11:47.488231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:18.632464+0000","last_change":"2024-04-17T05:59:13.561616+0000","last_active":"2024-04-17T05:59:18.632464+0000","last_peered":"2024-04-17T05:59:18.632464+0000","last_clean":"2024-04-17T05:59:18.632464+0000","last_became_active":"2024-04-17T05:59:13.561298+0000","last_became_peered":"2024-04-17T05:59:13.561298+0000","last_unstale":"2024-04-17T05:59:18.632464+0000","last_undegraded":"2024-04-17T05:59:18.632464+0000","last_fullsized":"2024-04-17T05:59:18.632464+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:59:02.598774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.582030+0000","last_change":"2024-04-17T05:59:14.592023+0000","last_active":"2024-04-17T05:59:15.582030+0000","last_peered":"2024-04-17T05:59:15.582030+0000","last_clean":"2024-04-17T05:59:15.582030+0000","last_became_active":"2024-04-17T05:59:14.591707+0000","last_became_peered":"2024-04-17T05:59:14.591707+0000","last_unstale":"2024-04-17T05:59:15.582030+0000","last_undegraded":"2024-04-17T05:59:15.582030+0000","last_fullsized":"2024-04-17T05:59:15.582030+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:50:06.828184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.582103+0000","last_change":"2024-04-17T05:59:13.568073+0000","last_active":"2024-04-17T05:59:15.582103+0000","last_peered":"2024-04-17T05:59:15.582103+0000","last_clean":"2024-04-17T05:59:15.582103+0000","last_became_active":"2024-04-17T05:59:13.567751+0000","last_became_peered":"2024-04-17T05:59:13.567751+0000","last_unstale":"2024-04-17T05:59:15.582103+0000","last_undegraded":"2024-04-17T05:59:15.582103+0000","last_fullsized":"2024-04-17T05:59:15.582103+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:40:26.884308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.758505+0000","last_change":"2024-04-17T05:59:14.593344+0000","last_active":"2024-04-17T05:59:15.758505+0000","last_peered":"2024-04-17T05:59:15.758505+0000","last_clean":"2024-04-17T05:59:15.758505+0000","last_became_active":"2024-04-17T05:59:14.592946+0000","last_became_peered":"2024-04-17T05:59:14.592946+0000","last_unstale":"2024-04-17T05:59:15.758505+0000","last_undegraded":"2024-04-17T05:59:15.758505+0000","last_fullsized":"2024-04-17T05:59:15.758505+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:50:23.979412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.758345+0000","last_change":"2024-04-17T05:59:13.566639+0000","last_active":"2024-04-17T05:59:15.758345+0000","last_peered":"2024-04-17T05:59:15.758345+0000","last_clean":"2024-04-17T05:59:15.758345+0000","last_became_active":"2024-04-17T05:59:13.562766+0000","last_became_peered":"2024-04-17T05:59:13.562766+0000","last_unstale":"2024-04-17T05:59:15.758345+0000","last_undegraded":"2024-04-17T05:59:15.758345+0000","last_fullsized":"2024-04-17T05:59:15.758345+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:36:47.605106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.585132+0000","last_change":"2024-04-17T05:59:14.596055+0000","last_active":"2024-04-17T05:59:15.585132+0000","last_peered":"2024-04-17T05:59:15.585132+0000","last_clean":"2024-04-17T05:59:15.585132+0000","last_became_active":"2024-04-17T05:59:14.595672+0000","last_became_peered":"2024-04-17T05:59:14.595672+0000","last_unstale":"2024-04-17T05:59:15.585132+0000","last_undegraded":"2024-04-17T05:59:15.585132+0000","last_fullsized":"2024-04-17T05:59:15.585132+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:31:59.964502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.758069+0000","last_change":"2024-04-17T05:59:13.558715+0000","last_active":"2024-04-17T05:59:15.758069+0000","last_peered":"2024-04-17T05:59:15.758069+0000","last_clean":"2024-04-17T05:59:15.758069+0000","last_became_active":"2024-04-17T05:59:13.558377+0000","last_became_peered":"2024-04-17T05:59:13.558377+0000","last_unstale":"2024-04-17T05:59:15.758069+0000","last_undegraded":"2024-04-17T05:59:15.758069+0000","last_fullsized":"2024-04-17T05:59:15.758069+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:12:12.392655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.582710+0000","last_change":"2024-04-17T05:59:14.594824+0000","last_active":"2024-04-17T05:59:15.582710+0000","last_peered":"2024-04-17T05:59:15.582710+0000","last_clean":"2024-04-17T05:59:15.582710+0000","last_became_active":"2024-04-17T05:59:14.594510+0000","last_became_peered":"2024-04-17T05:59:14.594510+0000","last_unstale":"2024-04-17T05:59:15.582710+0000","last_undegraded":"2024-04-17T05:59:15.582710+0000","last_fullsized":"2024-04-17T05:59:15.582710+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:19:37.042522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'5","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:17.629557+0000","last_change":"2024-04-17T05:59:13.567654+0000","last_active":"2024-04-17T05:59:17.629557+0000","last_peered":"2024-04-17T05:59:17.629557+0000","last_clean":"2024-04-17T05:59:17.629557+0000","last_became_active":"2024-04-17T05:59:13.566474+0000","last_became_peered":"2024-04-17T05:59:13.566474+0000","last_unstale":"2024-04-17T05:59:17.629557+0000","last_undegraded":"2024-04-17T05:59:17.629557+0000","last_fullsized":"2024-04-17T05:59:17.629557+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:40:25.205832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.757985+0000","last_change":"2024-04-17T05:59:14.597145+0000","last_active":"2024-04-17T05:59:15.757985+0000","last_peered":"2024-04-17T05:59:15.757985+0000","last_clean":"2024-04-17T05:59:15.757985+0000","last_became_active":"2024-04-17T05:59:14.596802+0000","last_became_peered":"2024-04-17T05:59:14.596802+0000","last_unstale":"2024-04-17T05:59:15.757985+0000","last_undegraded":"2024-04-17T05:59:15.757985+0000","last_fullsized":"2024-04-17T05:59:15.757985+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:41:04.946052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:18.633424+0000","last_change":"2024-04-17T05:59:13.560100+0000","last_active":"2024-04-17T05:59:18.633424+0000","last_peered":"2024-04-17T05:59:18.633424+0000","last_clean":"2024-04-17T05:59:18.633424+0000","last_became_active":"2024-04-17T05:59:13.559769+0000","last_became_peered":"2024-04-17T05:59:13.559769+0000","last_unstale":"2024-04-17T05:59:18.633424+0000","last_undegraded":"2024-04-17T05:59:18.633424+0000","last_fullsized":"2024-04-17T05:59:18.633424+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:44:30.299028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.582495+0000","last_change":"2024-04-17T05:59:14.600733+0000","last_active":"2024-04-17T05:59:15.582495+0000","last_peered":"2024-04-17T05:59:15.582495+0000","last_clean":"2024-04-17T05:59:15.582495+0000","last_became_active":"2024-04-17T05:59:14.599273+0000","last_became_peered":"2024-04-17T05:59:14.599273+0000","last_unstale":"2024-04-17T05:59:15.582495+0000","last_undegraded":"2024-04-17T05:59:15.582495+0000","last_fullsized":"2024-04-17T05:59:15.582495+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:27:02.721452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.757410+0000","last_change":"2024-04-17T05:59:13.553000+0000","last_active":"2024-04-17T05:59:15.757410+0000","last_peered":"2024-04-17T05:59:15.757410+0000","last_clean":"2024-04-17T05:59:15.757410+0000","last_became_active":"2024-04-17T05:59:13.552469+0000","last_became_peered":"2024-04-17T05:59:13.552469+0000","last_unstale":"2024-04-17T05:59:15.757410+0000","last_undegraded":"2024-04-17T05:59:15.757410+0000","last_fullsized":"2024-04-17T05:59:15.757410+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:14:34.714179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.581455+0000","last_change":"2024-04-17T05:59:14.594792+0000","last_active":"2024-04-17T05:59:15.581455+0000","last_peered":"2024-04-17T05:59:15.581455+0000","last_clean":"2024-04-17T05:59:15.581455+0000","last_became_active":"2024-04-17T05:59:14.594436+0000","last_became_peered":"2024-04-17T05:59:14.594436+0000","last_unstale":"2024-04-17T05:59:15.581455+0000","last_undegraded":"2024-04-17T05:59:15.581455+0000","last_fullsized":"2024-04-17T05:59:15.581455+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:49:57.104050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.582065+0000","last_change":"2024-04-17T05:59:13.565352+0000","last_active":"2024-04-17T05:59:15.582065+0000","last_peered":"2024-04-17T05:59:15.582065+0000","last_clean":"2024-04-17T05:59:15.582065+0000","last_became_active":"2024-04-17T05:59:13.565046+0000","last_became_peered":"2024-04-17T05:59:13.565046+0000","last_unstale":"2024-04-17T05:59:15.582065+0000","last_undegraded":"2024-04-17T05:59:15.582065+0000","last_fullsized":"2024-04-17T05:59:15.582065+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:39:25.276782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.757298+0000","last_change":"2024-04-17T05:59:14.593925+0000","last_active":"2024-04-17T05:59:15.757298+0000","last_peered":"2024-04-17T05:59:15.757298+0000","last_clean":"2024-04-17T05:59:15.757298+0000","last_became_active":"2024-04-17T05:59:14.593535+0000","last_became_peered":"2024-04-17T05:59:14.593535+0000","last_unstale":"2024-04-17T05:59:15.757298+0000","last_undegraded":"2024-04-17T05:59:15.757298+0000","last_fullsized":"2024-04-17T05:59:15.757298+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:58:20.811999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'3","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:17.627171+0000","last_change":"2024-04-17T05:59:13.562281+0000","last_active":"2024-04-17T05:59:17.627171+0000","last_peered":"2024-04-17T05:59:17.627171+0000","last_clean":"2024-04-17T05:59:17.627171+0000","last_became_active":"2024-04-17T05:59:13.561931+0000","last_became_peered":"2024-04-17T05:59:13.561931+0000","last_unstale":"2024-04-17T05:59:17.627171+0000","last_undegraded":"2024-04-17T05:59:17.627171+0000","last_fullsized":"2024-04-17T05:59:17.627171+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:55:52.119923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.585188+0000","last_change":"2024-04-17T05:59:14.594708+0000","last_active":"2024-04-17T05:59:15.585188+0000","last_peered":"2024-04-17T05:59:15.585188+0000","last_clean":"2024-04-17T05:59:15.585188+0000","last_became_active":"2024-04-17T05:59:14.594206+0000","last_became_peered":"2024-04-17T05:59:14.594206+0000","last_unstale":"2024-04-17T05:59:15.585188+0000","last_undegraded":"2024-04-17T05:59:15.585188+0000","last_fullsized":"2024-04-17T05:59:15.585188+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:43:16.402659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.757728+0000","last_change":"2024-04-17T05:59:13.561056+0000","last_active":"2024-04-17T05:59:15.757728+0000","last_peered":"2024-04-17T05:59:15.757728+0000","last_clean":"2024-04-17T05:59:15.757728+0000","last_became_active":"2024-04-17T05:59:13.560670+0000","last_became_peered":"2024-04-17T05:59:13.560670+0000","last_unstale":"2024-04-17T05:59:15.757728+0000","last_undegraded":"2024-04-17T05:59:15.757728+0000","last_fullsized":"2024-04-17T05:59:15.757728+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:31:01.106013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.758679+0000","last_change":"2024-04-17T05:59:14.590243+0000","last_active":"2024-04-17T05:59:15.758679+0000","last_peered":"2024-04-17T05:59:15.758679+0000","last_clean":"2024-04-17T05:59:15.758679+0000","last_became_active":"2024-04-17T05:59:14.589911+0000","last_became_peered":"2024-04-17T05:59:14.589911+0000","last_unstale":"2024-04-17T05:59:15.758679+0000","last_undegraded":"2024-04-17T05:59:15.758679+0000","last_fullsized":"2024-04-17T05:59:15.758679+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:44:41.855928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.582791+0000","last_change":"2024-04-17T05:59:13.569983+0000","last_active":"2024-04-17T05:59:15.582791+0000","last_peered":"2024-04-17T05:59:15.582791+0000","last_clean":"2024-04-17T05:59:15.582791+0000","last_became_active":"2024-04-17T05:59:13.569667+0000","last_became_peered":"2024-04-17T05:59:13.569667+0000","last_unstale":"2024-04-17T05:59:15.582791+0000","last_undegraded":"2024-04-17T05:59:15.582791+0000","last_fullsized":"2024-04-17T05:59:15.582791+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:36:15.362530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.585072+0000","last_change":"2024-04-17T05:59:14.603404+0000","last_active":"2024-04-17T05:59:15.585072+0000","last_peered":"2024-04-17T05:59:15.585072+0000","last_clean":"2024-04-17T05:59:15.585072+0000","last_became_active":"2024-04-17T05:59:14.601593+0000","last_became_peered":"2024-04-17T05:59:14.601593+0000","last_unstale":"2024-04-17T05:59:15.585072+0000","last_undegraded":"2024-04-17T05:59:15.585072+0000","last_fullsized":"2024-04-17T05:59:15.585072+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:44:44.490496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:19.606855+0000","last_change":"2024-04-17T05:59:13.561237+0000","last_active":"2024-04-17T05:59:19.606855+0000","last_peered":"2024-04-17T05:59:19.606855+0000","last_clean":"2024-04-17T05:59:19.606855+0000","last_became_active":"2024-04-17T05:59:13.560890+0000","last_became_peered":"2024-04-17T05:59:13.560890+0000","last_unstale":"2024-04-17T05:59:19.606855+0000","last_undegraded":"2024-04-17T05:59:19.606855+0000","last_fullsized":"2024-04-17T05:59:19.606855+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:27:13.247831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.758195+0000","last_change":"2024-04-17T05:59:14.579304+0000","last_active":"2024-04-17T05:59:15.758195+0000","last_peered":"2024-04-17T05:59:15.758195+0000","last_clean":"2024-04-17T05:59:15.758195+0000","last_became_active":"2024-04-17T05:59:14.578948+0000","last_became_peered":"2024-04-17T05:59:14.578948+0000","last_unstale":"2024-04-17T05:59:15.758195+0000","last_undegraded":"2024-04-17T05:59:15.758195+0000","last_fullsized":"2024-04-17T05:59:15.758195+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:39:43.341725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.758692+0000","last_change":"2024-04-17T05:59:13.569534+0000","last_active":"2024-04-17T05:59:15.758692+0000","last_peered":"2024-04-17T05:59:15.758692+0000","last_clean":"2024-04-17T05:59:15.758692+0000","last_became_active":"2024-04-17T05:59:13.569041+0000","last_became_peered":"2024-04-17T05:59:13.569041+0000","last_unstale":"2024-04-17T05:59:15.758692+0000","last_undegraded":"2024-04-17T05:59:15.758692+0000","last_fullsized":"2024-04-17T05:59:15.758692+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:27:37.018696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.757438+0000","last_change":"2024-04-17T05:59:14.593820+0000","last_active":"2024-04-17T05:59:15.757438+0000","last_peered":"2024-04-17T05:59:15.757438+0000","last_clean":"2024-04-17T05:59:15.757438+0000","last_became_active":"2024-04-17T05:59:14.593473+0000","last_became_peered":"2024-04-17T05:59:14.593473+0000","last_unstale":"2024-04-17T05:59:15.757438+0000","last_undegraded":"2024-04-17T05:59:15.757438+0000","last_fullsized":"2024-04-17T05:59:15.757438+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:44:38.636980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:17.627799+0000","last_change":"2024-04-17T05:59:13.570124+0000","last_active":"2024-04-17T05:59:17.627799+0000","last_peered":"2024-04-17T05:59:17.627799+0000","last_clean":"2024-04-17T05:59:17.627799+0000","last_became_active":"2024-04-17T05:59:13.569778+0000","last_became_peered":"2024-04-17T05:59:13.569778+0000","last_unstale":"2024-04-17T05:59:17.627799+0000","last_undegraded":"2024-04-17T05:59:17.627799+0000","last_fullsized":"2024-04-17T05:59:17.627799+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:46:46.431207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.758225+0000","last_change":"2024-04-17T05:59:14.584863+0000","last_active":"2024-04-17T05:59:15.758225+0000","last_peered":"2024-04-17T05:59:15.758225+0000","last_clean":"2024-04-17T05:59:15.758225+0000","last_became_active":"2024-04-17T05:59:14.584554+0000","last_became_peered":"2024-04-17T05:59:14.584554+0000","last_unstale":"2024-04-17T05:59:15.758225+0000","last_undegraded":"2024-04-17T05:59:15.758225+0000","last_fullsized":"2024-04-17T05:59:15.758225+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:26:22.064337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.757742+0000","last_change":"2024-04-17T05:59:13.557625+0000","last_active":"2024-04-17T05:59:15.757742+0000","last_peered":"2024-04-17T05:59:15.757742+0000","last_clean":"2024-04-17T05:59:15.757742+0000","last_became_active":"2024-04-17T05:59:13.557307+0000","last_became_peered":"2024-04-17T05:59:13.557307+0000","last_unstale":"2024-04-17T05:59:15.757742+0000","last_undegraded":"2024-04-17T05:59:15.757742+0000","last_fullsized":"2024-04-17T05:59:15.757742+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:43:17.023049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.757952+0000","last_change":"2024-04-17T05:59:14.579295+0000","last_active":"2024-04-17T05:59:15.757952+0000","last_peered":"2024-04-17T05:59:15.757952+0000","last_clean":"2024-04-17T05:59:15.757952+0000","last_became_active":"2024-04-17T05:59:14.578902+0000","last_became_peered":"2024-04-17T05:59:14.578902+0000","last_unstale":"2024-04-17T05:59:15.757952+0000","last_undegraded":"2024-04-17T05:59:15.757952+0000","last_fullsized":"2024-04-17T05:59:15.757952+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:46:23.094552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'8","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.757548+0000","last_change":"2024-04-17T05:59:13.561834+0000","last_active":"2024-04-17T05:59:15.757548+0000","last_peered":"2024-04-17T05:59:15.757548+0000","last_clean":"2024-04-17T05:59:15.757548+0000","last_became_active":"2024-04-17T05:59:13.561536+0000","last_became_peered":"2024-04-17T05:59:13.561536+0000","last_unstale":"2024-04-17T05:59:15.757548+0000","last_undegraded":"2024-04-17T05:59:15.757548+0000","last_fullsized":"2024-04-17T05:59:15.757548+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:33:03.869380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.582129+0000","last_change":"2024-04-17T05:59:14.594082+0000","last_active":"2024-04-17T05:59:15.582129+0000","last_peered":"2024-04-17T05:59:15.582129+0000","last_clean":"2024-04-17T05:59:15.582129+0000","last_became_active":"2024-04-17T05:59:14.593754+0000","last_became_peered":"2024-04-17T05:59:14.593754+0000","last_unstale":"2024-04-17T05:59:15.582129+0000","last_undegraded":"2024-04-17T05:59:15.582129+0000","last_fullsized":"2024-04-17T05:59:15.582129+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:36:28.752766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.758402+0000","last_change":"2024-04-17T05:59:13.563470+0000","last_active":"2024-04-17T05:59:15.758402+0000","last_peered":"2024-04-17T05:59:15.758402+0000","last_clean":"2024-04-17T05:59:15.758402+0000","last_became_active":"2024-04-17T05:59:13.563067+0000","last_became_peered":"2024-04-17T05:59:13.563067+0000","last_unstale":"2024-04-17T05:59:15.758402+0000","last_undegraded":"2024-04-17T05:59:15.758402+0000","last_fullsized":"2024-04-17T05:59:15.758402+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:13:52.018604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.758193+0000","last_change":"2024-04-17T05:59:14.587339+0000","last_active":"2024-04-17T05:59:15.758193+0000","last_peered":"2024-04-17T05:59:15.758193+0000","last_clean":"2024-04-17T05:59:15.758193+0000","last_became_active":"2024-04-17T05:59:14.586956+0000","last_became_peered":"2024-04-17T05:59:14.586956+0000","last_unstale":"2024-04-17T05:59:15.758193+0000","last_undegraded":"2024-04-17T05:59:15.758193+0000","last_fullsized":"2024-04-17T05:59:15.758193+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:00:21.494832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.758131+0000","last_change":"2024-04-17T05:59:13.563915+0000","last_active":"2024-04-17T05:59:15.758131+0000","last_peered":"2024-04-17T05:59:15.758131+0000","last_clean":"2024-04-17T05:59:15.758131+0000","last_became_active":"2024-04-17T05:59:13.563218+0000","last_became_peered":"2024-04-17T05:59:13.563218+0000","last_unstale":"2024-04-17T05:59:15.758131+0000","last_undegraded":"2024-04-17T05:59:15.758131+0000","last_fullsized":"2024-04-17T05:59:15.758131+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:36:54.484278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.759089+0000","last_change":"2024-04-17T05:59:14.593168+0000","last_active":"2024-04-17T05:59:15.759089+0000","last_peered":"2024-04-17T05:59:15.759089+0000","last_clean":"2024-04-17T05:59:15.759089+0000","last_became_active":"2024-04-17T05:59:14.592744+0000","last_became_peered":"2024-04-17T05:59:14.592744+0000","last_unstale":"2024-04-17T05:59:15.759089+0000","last_undegraded":"2024-04-17T05:59:15.759089+0000","last_fullsized":"2024-04-17T05:59:15.759089+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:00:03.217864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.758028+0000","last_change":"2024-04-17T05:59:13.564703+0000","last_active":"2024-04-17T05:59:15.758028+0000","last_peered":"2024-04-17T05:59:15.758028+0000","last_clean":"2024-04-17T05:59:15.758028+0000","last_became_active":"2024-04-17T05:59:13.564263+0000","last_became_peered":"2024-04-17T05:59:13.564263+0000","last_unstale":"2024-04-17T05:59:15.758028+0000","last_undegraded":"2024-04-17T05:59:15.758028+0000","last_fullsized":"2024-04-17T05:59:15.758028+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:51:06.847989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.759260+0000","last_change":"2024-04-17T05:59:14.588074+0000","last_active":"2024-04-17T05:59:15.759260+0000","last_peered":"2024-04-17T05:59:15.759260+0000","last_clean":"2024-04-17T05:59:15.759260+0000","last_became_active":"2024-04-17T05:59:14.587691+0000","last_became_peered":"2024-04-17T05:59:14.587691+0000","last_unstale":"2024-04-17T05:59:15.759260+0000","last_undegraded":"2024-04-17T05:59:15.759260+0000","last_fullsized":"2024-04-17T05:59:15.759260+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:11:15.775356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.582169+0000","last_change":"2024-04-17T05:59:14.589178+0000","last_active":"2024-04-17T05:59:15.582169+0000","last_peered":"2024-04-17T05:59:15.582169+0000","last_clean":"2024-04-17T05:59:15.582169+0000","last_became_active":"2024-04-17T05:59:14.587351+0000","last_became_peered":"2024-04-17T05:59:14.587351+0000","last_unstale":"2024-04-17T05:59:15.582169+0000","last_undegraded":"2024-04-17T05:59:15.582169+0000","last_fullsized":"2024-04-17T05:59:15.582169+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:22:43.202731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.757297+0000","last_change":"2024-04-17T05:59:13.553754+0000","last_active":"2024-04-17T05:59:15.757297+0000","last_peered":"2024-04-17T05:59:15.757297+0000","last_clean":"2024-04-17T05:59:15.757297+0000","last_became_active":"2024-04-17T05:59:13.553437+0000","last_became_peered":"2024-04-17T05:59:13.553437+0000","last_unstale":"2024-04-17T05:59:15.757297+0000","last_undegraded":"2024-04-17T05:59:15.757297+0000","last_fullsized":"2024-04-17T05:59:15.757297+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:10:14.010264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.582022+0000","last_change":"2024-04-17T05:59:14.592074+0000","last_active":"2024-04-17T05:59:15.582022+0000","last_peered":"2024-04-17T05:59:15.582022+0000","last_clean":"2024-04-17T05:59:15.582022+0000","last_became_active":"2024-04-17T05:59:14.591695+0000","last_became_peered":"2024-04-17T05:59:14.591695+0000","last_unstale":"2024-04-17T05:59:15.582022+0000","last_undegraded":"2024-04-17T05:59:15.582022+0000","last_fullsized":"2024-04-17T05:59:15.582022+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:27:04.344257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.758641+0000","last_change":"2024-04-17T05:59:13.553916+0000","last_active":"2024-04-17T05:59:15.758641+0000","last_peered":"2024-04-17T05:59:15.758641+0000","last_clean":"2024-04-17T05:59:15.758641+0000","last_became_active":"2024-04-17T05:59:13.553597+0000","last_became_peered":"2024-04-17T05:59:13.553597+0000","last_unstale":"2024-04-17T05:59:15.758641+0000","last_undegraded":"2024-04-17T05:59:15.758641+0000","last_fullsized":"2024-04-17T05:59:15.758641+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:28:23.486188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":348,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.757544+0000","last_change":"2024-04-17T05:58:02.597504+0000","last_active":"2024-04-17T05:59:15.757544+0000","last_peered":"2024-04-17T05:59:15.757544+0000","last_clean":"2024-04-17T05:59:15.757544+0000","last_became_active":"2024-04-17T05:58:02.597086+0000","last_became_peered":"2024-04-17T05:58:02.597086+0000","last_unstale":"2024-04-17T05:59:15.757544+0000","last_undegraded":"2024-04-17T05:59:15.757544+0000","last_fullsized":"2024-04-17T05:59:15.757544+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:54:37.919204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:54:37.919204+0000","last_clean_scrub_stamp":"2024-04-17T05:54:37.919204+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:02:09.566418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.757359+0000","last_change":"2024-04-17T05:59:14.584240+0000","last_active":"2024-04-17T05:59:15.757359+0000","last_peered":"2024-04-17T05:59:15.757359+0000","last_clean":"2024-04-17T05:59:15.757359+0000","last_became_active":"2024-04-17T05:59:14.583934+0000","last_became_peered":"2024-04-17T05:59:14.583934+0000","last_unstale":"2024-04-17T05:59:15.757359+0000","last_undegraded":"2024-04-17T05:59:15.757359+0000","last_fullsized":"2024-04-17T05:59:15.757359+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:50:41.891162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.757896+0000","last_change":"2024-04-17T05:59:13.562426+0000","last_active":"2024-04-17T05:59:15.757896+0000","last_peered":"2024-04-17T05:59:15.757896+0000","last_clean":"2024-04-17T05:59:15.757896+0000","last_became_active":"2024-04-17T05:59:13.562133+0000","last_became_peered":"2024-04-17T05:59:13.562133+0000","last_unstale":"2024-04-17T05:59:15.757896+0000","last_undegraded":"2024-04-17T05:59:15.757896+0000","last_fullsized":"2024-04-17T05:59:15.757896+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:50:58.351924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.758176+0000","last_change":"2024-04-17T05:59:13.557783+0000","last_active":"2024-04-17T05:59:15.758176+0000","last_peered":"2024-04-17T05:59:15.758176+0000","last_clean":"2024-04-17T05:59:15.758176+0000","last_became_active":"2024-04-17T05:59:13.557485+0000","last_became_peered":"2024-04-17T05:59:13.557485+0000","last_unstale":"2024-04-17T05:59:15.758176+0000","last_undegraded":"2024-04-17T05:59:15.758176+0000","last_fullsized":"2024-04-17T05:59:15.758176+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:41:37.359553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.758035+0000","last_change":"2024-04-17T05:59:14.575993+0000","last_active":"2024-04-17T05:59:15.758035+0000","last_peered":"2024-04-17T05:59:15.758035+0000","last_clean":"2024-04-17T05:59:15.758035+0000","last_became_active":"2024-04-17T05:59:14.575654+0000","last_became_peered":"2024-04-17T05:59:14.575654+0000","last_unstale":"2024-04-17T05:59:15.758035+0000","last_undegraded":"2024-04-17T05:59:15.758035+0000","last_fullsized":"2024-04-17T05:59:15.758035+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:33:12.193747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:18.631820+0000","last_change":"2024-04-17T05:59:13.560256+0000","last_active":"2024-04-17T05:59:18.631820+0000","last_peered":"2024-04-17T05:59:18.631820+0000","last_clean":"2024-04-17T05:59:18.631820+0000","last_became_active":"2024-04-17T05:59:13.559938+0000","last_became_peered":"2024-04-17T05:59:13.559938+0000","last_unstale":"2024-04-17T05:59:18.631820+0000","last_undegraded":"2024-04-17T05:59:18.631820+0000","last_fullsized":"2024-04-17T05:59:18.631820+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:14:54.625298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.582029+0000","last_change":"2024-04-17T05:59:14.593955+0000","last_active":"2024-04-17T05:59:15.582029+0000","last_peered":"2024-04-17T05:59:15.582029+0000","last_clean":"2024-04-17T05:59:15.582029+0000","last_became_active":"2024-04-17T05:59:14.590626+0000","last_became_peered":"2024-04-17T05:59:14.590626+0000","last_unstale":"2024-04-17T05:59:15.582029+0000","last_undegraded":"2024-04-17T05:59:15.582029+0000","last_fullsized":"2024-04-17T05:59:15.582029+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:19:04.438634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.758774+0000","last_change":"2024-04-17T05:59:13.555734+0000","last_active":"2024-04-17T05:59:15.758774+0000","last_peered":"2024-04-17T05:59:15.758774+0000","last_clean":"2024-04-17T05:59:15.758774+0000","last_became_active":"2024-04-17T05:59:13.555284+0000","last_became_peered":"2024-04-17T05:59:13.555284+0000","last_unstale":"2024-04-17T05:59:15.758774+0000","last_undegraded":"2024-04-17T05:59:15.758774+0000","last_fullsized":"2024-04-17T05:59:15.758774+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:47:06.834332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.758444+0000","last_change":"2024-04-17T05:59:14.599078+0000","last_active":"2024-04-17T05:59:15.758444+0000","last_peered":"2024-04-17T05:59:15.758444+0000","last_clean":"2024-04-17T05:59:15.758444+0000","last_became_active":"2024-04-17T05:59:14.598670+0000","last_became_peered":"2024-04-17T05:59:14.598670+0000","last_unstale":"2024-04-17T05:59:15.758444+0000","last_undegraded":"2024-04-17T05:59:15.758444+0000","last_fullsized":"2024-04-17T05:59:15.758444+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:58:17.414704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:17.626021+0000","last_change":"2024-04-17T05:59:13.560154+0000","last_active":"2024-04-17T05:59:17.626021+0000","last_peered":"2024-04-17T05:59:17.626021+0000","last_clean":"2024-04-17T05:59:17.626021+0000","last_became_active":"2024-04-17T05:59:13.559846+0000","last_became_peered":"2024-04-17T05:59:13.559846+0000","last_unstale":"2024-04-17T05:59:17.626021+0000","last_undegraded":"2024-04-17T05:59:17.626021+0000","last_fullsized":"2024-04-17T05:59:17.626021+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:06:03.200186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.757575+0000","last_change":"2024-04-17T05:59:14.593450+0000","last_active":"2024-04-17T05:59:15.757575+0000","last_peered":"2024-04-17T05:59:15.757575+0000","last_clean":"2024-04-17T05:59:15.757575+0000","last_became_active":"2024-04-17T05:59:14.593104+0000","last_became_peered":"2024-04-17T05:59:14.593104+0000","last_unstale":"2024-04-17T05:59:15.757575+0000","last_undegraded":"2024-04-17T05:59:15.757575+0000","last_fullsized":"2024-04-17T05:59:15.757575+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:16:20.488110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:17.627101+0000","last_change":"2024-04-17T05:59:13.557330+0000","last_active":"2024-04-17T05:59:17.627101+0000","last_peered":"2024-04-17T05:59:17.627101+0000","last_clean":"2024-04-17T05:59:17.627101+0000","last_became_active":"2024-04-17T05:59:13.556970+0000","last_became_peered":"2024-04-17T05:59:13.556970+0000","last_unstale":"2024-04-17T05:59:17.627101+0000","last_undegraded":"2024-04-17T05:59:17.627101+0000","last_fullsized":"2024-04-17T05:59:17.627101+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:41:46.643845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.757892+0000","last_change":"2024-04-17T05:59:14.582281+0000","last_active":"2024-04-17T05:59:15.757892+0000","last_peered":"2024-04-17T05:59:15.757892+0000","last_clean":"2024-04-17T05:59:15.757892+0000","last_became_active":"2024-04-17T05:59:14.581922+0000","last_became_peered":"2024-04-17T05:59:14.581922+0000","last_unstale":"2024-04-17T05:59:15.757892+0000","last_undegraded":"2024-04-17T05:59:15.757892+0000","last_fullsized":"2024-04-17T05:59:15.757892+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:53:10.415786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.758188+0000","last_change":"2024-04-17T05:59:13.566514+0000","last_active":"2024-04-17T05:59:15.758188+0000","last_peered":"2024-04-17T05:59:15.758188+0000","last_clean":"2024-04-17T05:59:15.758188+0000","last_became_active":"2024-04-17T05:59:13.566004+0000","last_became_peered":"2024-04-17T05:59:13.566004+0000","last_unstale":"2024-04-17T05:59:15.758188+0000","last_undegraded":"2024-04-17T05:59:15.758188+0000","last_fullsized":"2024-04-17T05:59:15.758188+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:20:13.891422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.581731+0000","last_change":"2024-04-17T05:59:14.599101+0000","last_active":"2024-04-17T05:59:15.581731+0000","last_peered":"2024-04-17T05:59:15.581731+0000","last_clean":"2024-04-17T05:59:15.581731+0000","last_became_active":"2024-04-17T05:59:14.598310+0000","last_became_peered":"2024-04-17T05:59:14.598310+0000","last_unstale":"2024-04-17T05:59:15.581731+0000","last_undegraded":"2024-04-17T05:59:15.581731+0000","last_fullsized":"2024-04-17T05:59:15.581731+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:46:39.275195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.757856+0000","last_change":"2024-04-17T05:59:13.562531+0000","last_active":"2024-04-17T05:59:15.757856+0000","last_peered":"2024-04-17T05:59:15.757856+0000","last_clean":"2024-04-17T05:59:15.757856+0000","last_became_active":"2024-04-17T05:59:13.562148+0000","last_became_peered":"2024-04-17T05:59:13.562148+0000","last_unstale":"2024-04-17T05:59:15.757856+0000","last_undegraded":"2024-04-17T05:59:15.757856+0000","last_fullsized":"2024-04-17T05:59:15.757856+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:10:14.407871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.758266+0000","last_change":"2024-04-17T05:59:14.596371+0000","last_active":"2024-04-17T05:59:15.758266+0000","last_peered":"2024-04-17T05:59:15.758266+0000","last_clean":"2024-04-17T05:59:15.758266+0000","last_became_active":"2024-04-17T05:59:14.596024+0000","last_became_peered":"2024-04-17T05:59:14.596024+0000","last_unstale":"2024-04-17T05:59:15.758266+0000","last_undegraded":"2024-04-17T05:59:15.758266+0000","last_fullsized":"2024-04-17T05:59:15.758266+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:06:14.916212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.758890+0000","last_change":"2024-04-17T05:59:13.569364+0000","last_active":"2024-04-17T05:59:15.758890+0000","last_peered":"2024-04-17T05:59:15.758890+0000","last_clean":"2024-04-17T05:59:15.758890+0000","last_became_active":"2024-04-17T05:59:13.569029+0000","last_became_peered":"2024-04-17T05:59:13.569029+0000","last_unstale":"2024-04-17T05:59:15.758890+0000","last_undegraded":"2024-04-17T05:59:15.758890+0000","last_fullsized":"2024-04-17T05:59:15.758890+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:58:24.629274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.758437+0000","last_change":"2024-04-17T05:59:14.593088+0000","last_active":"2024-04-17T05:59:15.758437+0000","last_peered":"2024-04-17T05:59:15.758437+0000","last_clean":"2024-04-17T05:59:15.758437+0000","last_became_active":"2024-04-17T05:59:14.592755+0000","last_became_peered":"2024-04-17T05:59:14.592755+0000","last_unstale":"2024-04-17T05:59:15.758437+0000","last_undegraded":"2024-04-17T05:59:15.758437+0000","last_fullsized":"2024-04-17T05:59:15.758437+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:27:14.503323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.582616+0000","last_change":"2024-04-17T05:59:13.569773+0000","last_active":"2024-04-17T05:59:15.582616+0000","last_peered":"2024-04-17T05:59:15.582616+0000","last_clean":"2024-04-17T05:59:15.582616+0000","last_became_active":"2024-04-17T05:59:13.569461+0000","last_became_peered":"2024-04-17T05:59:13.569461+0000","last_unstale":"2024-04-17T05:59:15.582616+0000","last_undegraded":"2024-04-17T05:59:15.582616+0000","last_fullsized":"2024-04-17T05:59:15.582616+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:21:13.441453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.757996+0000","last_change":"2024-04-17T05:59:14.588939+0000","last_active":"2024-04-17T05:59:15.757996+0000","last_peered":"2024-04-17T05:59:15.757996+0000","last_clean":"2024-04-17T05:59:15.757996+0000","last_became_active":"2024-04-17T05:59:14.588600+0000","last_became_peered":"2024-04-17T05:59:14.588600+0000","last_unstale":"2024-04-17T05:59:15.757996+0000","last_undegraded":"2024-04-17T05:59:15.757996+0000","last_fullsized":"2024-04-17T05:59:15.757996+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:07:47.865808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.757878+0000","last_change":"2024-04-17T05:59:13.560566+0000","last_active":"2024-04-17T05:59:15.757878+0000","last_peered":"2024-04-17T05:59:15.757878+0000","last_clean":"2024-04-17T05:59:15.757878+0000","last_became_active":"2024-04-17T05:59:13.560228+0000","last_became_peered":"2024-04-17T05:59:13.560228+0000","last_unstale":"2024-04-17T05:59:15.757878+0000","last_undegraded":"2024-04-17T05:59:15.757878+0000","last_fullsized":"2024-04-17T05:59:15.757878+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:13:59.436598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.757853+0000","last_change":"2024-04-17T05:59:14.583621+0000","last_active":"2024-04-17T05:59:15.757853+0000","last_peered":"2024-04-17T05:59:15.757853+0000","last_clean":"2024-04-17T05:59:15.757853+0000","last_became_active":"2024-04-17T05:59:14.583199+0000","last_became_peered":"2024-04-17T05:59:14.583199+0000","last_unstale":"2024-04-17T05:59:15.757853+0000","last_undegraded":"2024-04-17T05:59:15.757853+0000","last_fullsized":"2024-04-17T05:59:15.757853+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:53:12.516661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.758071+0000","last_change":"2024-04-17T05:59:14.587989+0000","last_active":"2024-04-17T05:59:15.758071+0000","last_peered":"2024-04-17T05:59:15.758071+0000","last_clean":"2024-04-17T05:59:15.758071+0000","last_became_active":"2024-04-17T05:59:14.586686+0000","last_became_peered":"2024-04-17T05:59:14.586686+0000","last_unstale":"2024-04-17T05:59:15.758071+0000","last_undegraded":"2024-04-17T05:59:15.758071+0000","last_fullsized":"2024-04-17T05:59:15.758071+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:21:23.227328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.759265+0000","last_change":"2024-04-17T05:59:14.588567+0000","last_active":"2024-04-17T05:59:15.759265+0000","last_peered":"2024-04-17T05:59:15.759265+0000","last_clean":"2024-04-17T05:59:15.759265+0000","last_became_active":"2024-04-17T05:59:14.588207+0000","last_became_peered":"2024-04-17T05:59:14.588207+0000","last_unstale":"2024-04-17T05:59:15.759265+0000","last_undegraded":"2024-04-17T05:59:15.759265+0000","last_fullsized":"2024-04-17T05:59:15.759265+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:44:26.094334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.758071+0000","last_change":"2024-04-17T05:59:14.593885+0000","last_active":"2024-04-17T05:59:15.758071+0000","last_peered":"2024-04-17T05:59:15.758071+0000","last_clean":"2024-04-17T05:59:15.758071+0000","last_became_active":"2024-04-17T05:59:14.593483+0000","last_became_peered":"2024-04-17T05:59:14.593483+0000","last_unstale":"2024-04-17T05:59:15.758071+0000","last_undegraded":"2024-04-17T05:59:15.758071+0000","last_fullsized":"2024-04-17T05:59:15.758071+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:22:41.837363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.758704+0000","last_change":"2024-04-17T05:59:14.585666+0000","last_active":"2024-04-17T05:59:15.758704+0000","last_peered":"2024-04-17T05:59:15.758704+0000","last_clean":"2024-04-17T05:59:15.758704+0000","last_became_active":"2024-04-17T05:59:14.585349+0000","last_became_peered":"2024-04-17T05:59:14.585349+0000","last_unstale":"2024-04-17T05:59:15.758704+0000","last_undegraded":"2024-04-17T05:59:15.758704+0000","last_fullsized":"2024-04-17T05:59:15.758704+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:32:24.776931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.758054+0000","last_change":"2024-04-17T05:59:14.583266+0000","last_active":"2024-04-17T05:59:15.758054+0000","last_peered":"2024-04-17T05:59:15.758054+0000","last_clean":"2024-04-17T05:59:15.758054+0000","last_became_active":"2024-04-17T05:59:14.582949+0000","last_became_peered":"2024-04-17T05:59:14.582949+0000","last_unstale":"2024-04-17T05:59:15.758054+0000","last_undegraded":"2024-04-17T05:59:15.758054+0000","last_fullsized":"2024-04-17T05:59:15.758054+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:49:14.740168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.758111+0000","last_change":"2024-04-17T05:59:14.587046+0000","last_active":"2024-04-17T05:59:15.758111+0000","last_peered":"2024-04-17T05:59:15.758111+0000","last_clean":"2024-04-17T05:59:15.758111+0000","last_became_active":"2024-04-17T05:59:14.586653+0000","last_became_peered":"2024-04-17T05:59:14.586653+0000","last_unstale":"2024-04-17T05:59:15.758111+0000","last_undegraded":"2024-04-17T05:59:15.758111+0000","last_fullsized":"2024-04-17T05:59:15.758111+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:11:57.171410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.759198+0000","last_change":"2024-04-17T05:59:14.589670+0000","last_active":"2024-04-17T05:59:15.759198+0000","last_peered":"2024-04-17T05:59:15.759198+0000","last_clean":"2024-04-17T05:59:15.759198+0000","last_became_active":"2024-04-17T05:59:14.589003+0000","last_became_peered":"2024-04-17T05:59:14.589003+0000","last_unstale":"2024-04-17T05:59:15.759198+0000","last_undegraded":"2024-04-17T05:59:15.759198+0000","last_fullsized":"2024-04-17T05:59:15.759198+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:21:19.574178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.581833+0000","last_change":"2024-04-17T05:59:14.594107+0000","last_active":"2024-04-17T05:59:15.581833+0000","last_peered":"2024-04-17T05:59:15.581833+0000","last_clean":"2024-04-17T05:59:15.581833+0000","last_became_active":"2024-04-17T05:59:14.593799+0000","last_became_peered":"2024-04-17T05:59:14.593799+0000","last_unstale":"2024-04-17T05:59:15.581833+0000","last_undegraded":"2024-04-17T05:59:15.581833+0000","last_fullsized":"2024-04-17T05:59:15.581833+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:23:45.222667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.582004+0000","last_change":"2024-04-17T05:59:14.587671+0000","last_active":"2024-04-17T05:59:15.582004+0000","last_peered":"2024-04-17T05:59:15.582004+0000","last_clean":"2024-04-17T05:59:15.582004+0000","last_became_active":"2024-04-17T05:59:14.586473+0000","last_became_peered":"2024-04-17T05:59:14.586473+0000","last_unstale":"2024-04-17T05:59:15.582004+0000","last_undegraded":"2024-04-17T05:59:15.582004+0000","last_fullsized":"2024-04-17T05:59:15.582004+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:31:42.156354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.581565+0000","last_change":"2024-04-17T05:59:14.586961+0000","last_active":"2024-04-17T05:59:15.581565+0000","last_peered":"2024-04-17T05:59:15.581565+0000","last_clean":"2024-04-17T05:59:15.581565+0000","last_became_active":"2024-04-17T05:59:14.586601+0000","last_became_peered":"2024-04-17T05:59:14.586601+0000","last_unstale":"2024-04-17T05:59:15.581565+0000","last_undegraded":"2024-04-17T05:59:15.581565+0000","last_fullsized":"2024-04-17T05:59:15.581565+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:21:58.851523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.585223+0000","last_change":"2024-04-17T05:59:14.601239+0000","last_active":"2024-04-17T05:59:15.585223+0000","last_peered":"2024-04-17T05:59:15.585223+0000","last_clean":"2024-04-17T05:59:15.585223+0000","last_became_active":"2024-04-17T05:59:14.600852+0000","last_became_peered":"2024-04-17T05:59:14.600852+0000","last_unstale":"2024-04-17T05:59:15.585223+0000","last_undegraded":"2024-04-17T05:59:15.585223+0000","last_fullsized":"2024-04-17T05:59:15.585223+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:14:02.112456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.757920+0000","last_change":"2024-04-17T05:59:14.579952+0000","last_active":"2024-04-17T05:59:15.757920+0000","last_peered":"2024-04-17T05:59:15.757920+0000","last_clean":"2024-04-17T05:59:15.757920+0000","last_became_active":"2024-04-17T05:59:14.579625+0000","last_became_peered":"2024-04-17T05:59:14.579625+0000","last_unstale":"2024-04-17T05:59:15.757920+0000","last_undegraded":"2024-04-17T05:59:15.757920+0000","last_fullsized":"2024-04-17T05:59:15.757920+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:22:32.709762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-17T05:59:15.757838+0000","last_change":"2024-04-17T05:59:14.585187+0000","last_active":"2024-04-17T05:59:15.757838+0000","last_peered":"2024-04-17T05:59:15.757838+0000","last_clean":"2024-04-17T05:59:15.757838+0000","last_became_active":"2024-04-17T05:59:14.584820+0000","last_became_peered":"2024-04-17T05:59:14.584820+0000","last_unstale":"2024-04-17T05:59:15.757838+0000","last_undegraded":"2024-04-17T05:59:15.757838+0000","last_fullsized":"2024-04-17T05:59:15.757838+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:53:58.945885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":249153,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":3794,"num_objects":36,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":7,"num_read_kb":6,"num_write":77,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":184320,"data_stored":12204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":67,"ondisk_log_size":67,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350990,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":925696,"data_stored":442154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612627,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1015808,"data_stored":1035130,"data_compressed":36061,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776152,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36252,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708900,"statfs":{"total":95995035648,"available":95957913600,"internally_reserved":0,"allocated":937984,"data_stored":442356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":36174273},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037789,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":929792,"data_stored":442176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299424,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":974848,"data_stored":1032522,"data_compressed":36061,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462951,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":921600,"data_stored":441572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574912,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":933888,"data_stored":443184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738437,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36264,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708888,"statfs":{"total":95995035648,"available":95957901312,"internally_reserved":0,"allocated":950272,"data_stored":443050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411387,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":937984,"data_stored":443218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149750,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":929792,"data_stored":441676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888113,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":925696,"data_stored":441594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724588,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":1033416,"data_compressed":36061,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34424,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34424,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36061,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36061,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36061,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36061,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36061,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T05:59:22.829 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-17T05:59:22.830 INFO:tasks.cephadm:Setting up client nodes... 2024-04-17T05:59:22.830 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 565db2d8-fc7e-11ee-bc91-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-17T05:59:22.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:22 smithi098 ceph-mon[36516]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T05:59:22.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:22 smithi098 ceph-mon[36516]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T05:59:22.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:22 smithi098 ceph-mon[36516]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T05:59:22.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:22 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/2187541661' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T05:59:22.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:22 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/2119612838' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T05:59:23.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:22 smithi103 ceph-mon[36243]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T05:59:23.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:22 smithi103 ceph-mon[36243]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T05:59:23.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:22 smithi103 ceph-mon[36243]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T05:59:23.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:22 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/2187541661' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T05:59:23.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:22 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/2119612838' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T05:59:23.110 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T05:59:23.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:23 smithi098 ceph-mon[36516]: pgmap v290: 97 pgs: 97 active+clean; 580 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 2.4 KiB/s wr, 10 op/s 2024-04-17T05:59:23.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:23 smithi098 ceph-mon[36516]: from='client.14814 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-17T05:59:23.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:23 smithi098 ceph-mon[36516]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-17T05:59:23.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:23 smithi047 ceph-mon[32223]: pgmap v290: 97 pgs: 97 active+clean; 580 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 2.4 KiB/s wr, 10 op/s 2024-04-17T05:59:23.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:23 smithi047 ceph-mon[32223]: from='client.14814 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-17T05:59:23.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:23 smithi047 ceph-mon[32223]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-17T05:59:23.935 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:23 smithi103 ceph-mon[36243]: pgmap v290: 97 pgs: 97 active+clean; 580 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 2.4 KiB/s wr, 10 op/s 2024-04-17T05:59:23.935 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:23 smithi103 ceph-mon[36243]: from='client.14814 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-17T05:59:23.935 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:23 smithi103 ceph-mon[36243]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-17T05:59:24.837 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:24 smithi047 ceph-mon[32223]: from='client.14820 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-17T05:59:24.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:24.850+0000 7fe88e309700 1 -- 172.21.15.47:0/3361215581 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe8881019d0 msgr2=0x7fe888101df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:24.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:24.850+0000 7fe88e309700 1 --2- 172.21.15.47:0/3361215581 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe8881019d0 0x7fe888101df0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe8780099c0 tx=0x7fe878009cd0 comp rx=0 tx=0).stop 2024-04-17T05:59:24.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:24.850+0000 7fe88e309700 1 -- 172.21.15.47:0/3361215581 shutdown_connections 2024-04-17T05:59:24.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:24.850+0000 7fe88e309700 1 --2- 172.21.15.47:0/3361215581 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe888103570 0x7fe888109e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:24.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:24.850+0000 7fe88e309700 1 --2- 172.21.15.47:0/3361215581 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe8881019d0 0x7fe888101df0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:24.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:24.850+0000 7fe88e309700 1 --2- 172.21.15.47:0/3361215581 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe888102bd0 0x7fe888103030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:24.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:24.850+0000 7fe88e309700 1 -- 172.21.15.47:0/3361215581 >> 172.21.15.47:0/3361215581 conn(0x7fe8880fcf70 msgr2=0x7fe8880ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:59:24.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:24.850+0000 7fe88e309700 1 -- 172.21.15.47:0/3361215581 shutdown_connections 2024-04-17T05:59:24.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:24.850+0000 7fe88e309700 1 -- 172.21.15.47:0/3361215581 wait complete. 2024-04-17T05:59:24.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:24.851+0000 7fe88e309700 1 Processor -- start 2024-04-17T05:59:24.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:24.851+0000 7fe88e309700 1 -- start start 2024-04-17T05:59:24.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:24.851+0000 7fe88e309700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe8881019d0 0x7fe88806bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:24.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:24.851+0000 7fe88e309700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe888102bd0 0x7fe88819d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:24.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:24.851+0000 7fe88e309700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe888103570 0x7fe8881a0020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:24.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:24.851+0000 7fe88e309700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fe88810d590 con 0x7fe8881019d0 2024-04-17T05:59:24.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:24.851+0000 7fe88e309700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe88810d410 con 0x7fe888103570 2024-04-17T05:59:24.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:24.851+0000 7fe88e309700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe88810d710 con 0x7fe888102bd0 2024-04-17T05:59:24.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:24.851+0000 7fe887fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe8881019d0 0x7fe88806bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:24.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:24.851+0000 7fe887fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe8881019d0 0x7fe88806bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46038/0 (socket says 172.21.15.47:46038) 2024-04-17T05:59:24.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:24.851+0000 7fe887fff700 1 -- 172.21.15.47:0/1630441636 learned_addr learned my addr 172.21.15.47:0/1630441636 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:59:24.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:24.852+0000 7fe8877fe700 1 --2- 172.21.15.47:0/1630441636 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe888102bd0 0x7fe88819d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:24.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:24.852+0000 7fe887fff700 1 -- 172.21.15.47:0/1630441636 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe888102bd0 msgr2=0x7fe88819d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:24.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:24.852+0000 7fe887fff700 1 --2- 172.21.15.47:0/1630441636 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe888102bd0 0x7fe88819d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:24.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:24.852+0000 7fe887fff700 1 -- 172.21.15.47:0/1630441636 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe888103570 msgr2=0x7fe8881a0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:24.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:24.852+0000 7fe887fff700 1 --2- 172.21.15.47:0/1630441636 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe888103570 0x7fe8881a0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:24.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:24.852+0000 7fe887fff700 1 -- 172.21.15.47:0/1630441636 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8881a08f0 con 0x7fe8881019d0 2024-04-17T05:59:24.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:24.852+0000 7fe8877fe700 1 --2- 172.21.15.47:0/1630441636 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe888102bd0 0x7fe88819d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T05:59:24.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:24.852+0000 7fe887fff700 1 --2- 172.21.15.47:0/1630441636 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe8881019d0 0x7fe88806bb20 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe87800bff0 tx=0x7fe878005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:59:24.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:24.852+0000 7fe8857fa700 1 -- 172.21.15.47:0/1630441636 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe878027070 con 0x7fe8881019d0 2024-04-17T05:59:24.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:24.852+0000 7fe8857fa700 1 -- 172.21.15.47:0/1630441636 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe878004150 con 0x7fe8881019d0 2024-04-17T05:59:24.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:24.852+0000 7fe88e309700 1 -- 172.21.15.47:0/1630441636 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8881a0b80 con 0x7fe8881019d0 2024-04-17T05:59:24.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:24.852+0000 7fe8857fa700 1 -- 172.21.15.47:0/1630441636 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe878022dc0 con 0x7fe8881019d0 2024-04-17T05:59:24.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:24.852+0000 7fe88e309700 1 -- 172.21.15.47:0/1630441636 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8881a1060 con 0x7fe8881019d0 2024-04-17T05:59:24.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:24.853+0000 7fe88e309700 1 -- 172.21.15.47:0/1630441636 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe888050f90 con 0x7fe8881019d0 2024-04-17T05:59:24.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:24.854+0000 7fe8857fa700 1 -- 172.21.15.47:0/1630441636 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe87802a070 con 0x7fe8881019d0 2024-04-17T05:59:24.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:24.855+0000 7fe8857fa700 1 --2- 172.21.15.47:0/1630441636 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fe8700a3d80 0x7fe8700a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:24.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:24.855+0000 7fe8857fa700 1 -- 172.21.15.47:0/1630441636 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fe8780d1290 con 0x7fe8881019d0 2024-04-17T05:59:24.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:24.855+0000 7fe8877fe700 1 --2- 172.21.15.47:0/1630441636 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fe8700a3d80 0x7fe8700a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:24.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:24.855+0000 7fe8877fe700 1 --2- 172.21.15.47:0/1630441636 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fe8700a3d80 0x7fe8700a6240 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe87c005fd0 tx=0x7fe87c005dc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:59:24.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:24.860+0000 7fe8857fa700 1 -- 172.21.15.47:0/1630441636 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fe87805eff0 con 0x7fe8881019d0 2024-04-17T05:59:24.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:24 smithi098 ceph-mon[36516]: from='client.14820 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-17T05:59:24.949 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:24 smithi103 ceph-mon[36243]: from='client.14820 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-17T05:59:25.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:25.150+0000 7fe88e309700 1 -- 172.21.15.47:0/1630441636 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7fe88802e080 con 0x7fe8881019d0 2024-04-17T05:59:25.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:25.153+0000 7fe8857fa700 1 -- 172.21.15.47:0/1630441636 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7fe878013070 con 0x7fe8881019d0 2024-04-17T05:59:25.151 INFO:teuthology.orchestra.run.smithi047.stdout:[client.0] 2024-04-17T05:59:25.151 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQA9ZR9mS7oOCRAAXEH7gE68qk/LPUI0jnBRWA== 2024-04-17T05:59:25.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:25.155+0000 7fe88e309700 1 -- 172.21.15.47:0/1630441636 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fe8700a3d80 msgr2=0x7fe8700a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:25.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:25.155+0000 7fe88e309700 1 --2- 172.21.15.47:0/1630441636 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fe8700a3d80 0x7fe8700a6240 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe87c005fd0 tx=0x7fe87c005dc0 comp rx=0 tx=0).stop 2024-04-17T05:59:25.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:25.155+0000 7fe88e309700 1 -- 172.21.15.47:0/1630441636 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe8881019d0 msgr2=0x7fe88806bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:25.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:25.155+0000 7fe88e309700 1 --2- 172.21.15.47:0/1630441636 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe8881019d0 0x7fe88806bb20 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe87800bff0 tx=0x7fe878005b00 comp rx=0 tx=0).stop 2024-04-17T05:59:25.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:25.155+0000 7fe88e309700 1 -- 172.21.15.47:0/1630441636 shutdown_connections 2024-04-17T05:59:25.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:25.155+0000 7fe88e309700 1 --2- 172.21.15.47:0/1630441636 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fe8700a3d80 0x7fe8700a6240 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:25.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:25.155+0000 7fe88e309700 1 --2- 172.21.15.47:0/1630441636 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe888103570 0x7fe8881a0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:25.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:25.155+0000 7fe88e309700 1 --2- 172.21.15.47:0/1630441636 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe8881019d0 0x7fe88806bb20 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:25.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:25.155+0000 7fe88e309700 1 --2- 172.21.15.47:0/1630441636 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe888102bd0 0x7fe88819d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:25.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:25.156+0000 7fe88e309700 1 -- 172.21.15.47:0/1630441636 >> 172.21.15.47:0/1630441636 conn(0x7fe8880fcf70 msgr2=0x7fe888105e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:59:25.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:25.156+0000 7fe88e309700 1 -- 172.21.15.47:0/1630441636 shutdown_connections 2024-04-17T05:59:25.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:25.156+0000 7fe88e309700 1 -- 172.21.15.47:0/1630441636 wait complete. 2024-04-17T05:59:25.835 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:25 smithi047 ceph-mon[32223]: pgmap v291: 97 pgs: 97 active+clean; 582 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 4.7 KiB/s wr, 17 op/s 2024-04-17T05:59:25.835 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:25 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/1630441636' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-17T05:59:25.835 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:25 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/1630441636' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-17T05:59:25.863 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-17T05:59:25.864 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-04-17T05:59:25.864 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-17T05:59:25.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:25 smithi098 ceph-mon[36516]: pgmap v291: 97 pgs: 97 active+clean; 582 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 4.7 KiB/s wr, 17 op/s 2024-04-17T05:59:25.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:25 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/1630441636' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-17T05:59:25.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:25 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/1630441636' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-17T05:59:25.909 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-17T05:59:25.910 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-04-17T05:59:25.910 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 565db2d8-fc7e-11ee-bc91-c7b262605968 -- ceph mgr dump --format=json 2024-04-17T05:59:26.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:25 smithi103 ceph-mon[36243]: pgmap v291: 97 pgs: 97 active+clean; 582 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 4.7 KiB/s wr, 17 op/s 2024-04-17T05:59:26.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:25 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/1630441636' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-17T05:59:26.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:25 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/1630441636' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-17T05:59:26.202 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T05:59:27.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:27 smithi047 ceph-mon[32223]: pgmap v292: 97 pgs: 97 active+clean; 623 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 8.3 KiB/s wr, 18 op/s 2024-04-17T05:59:27.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:27.939+0000 7f17def1e700 1 -- 172.21.15.47:0/3921053130 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f17d8101950 msgr2=0x7f17d8101dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:27.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:27.939+0000 7f17def1e700 1 --2- 172.21.15.47:0/3921053130 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f17d8101950 0x7f17d8101dd0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f17c4009990 tx=0x7f17c4009ca0 comp rx=0 tx=0).stop 2024-04-17T05:59:27.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:27.939+0000 7f17def1e700 1 -- 172.21.15.47:0/3921053130 shutdown_connections 2024-04-17T05:59:27.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:27.939+0000 7f17def1e700 1 --2- 172.21.15.47:0/3921053130 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f17d8102310 0x7f17d810e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:27.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:27.939+0000 7f17def1e700 1 --2- 172.21.15.47:0/3921053130 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f17d81079a0 0x7f17d8107d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:27.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:27.939+0000 7f17def1e700 1 --2- 172.21.15.47:0/3921053130 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f17d8101950 0x7f17d8101dd0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:27.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:27.939+0000 7f17def1e700 1 -- 172.21.15.47:0/3921053130 >> 172.21.15.47:0/3921053130 conn(0x7f17d8078900 msgr2=0x7f17d8078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:59:27.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:27.939+0000 7f17def1e700 1 -- 172.21.15.47:0/3921053130 shutdown_connections 2024-04-17T05:59:27.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:27.939+0000 7f17def1e700 1 -- 172.21.15.47:0/3921053130 wait complete. 2024-04-17T05:59:27.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:27.940+0000 7f17def1e700 1 Processor -- start 2024-04-17T05:59:27.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:27.940+0000 7f17def1e700 1 -- start start 2024-04-17T05:59:27.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:27.940+0000 7f17def1e700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f17d8101950 0x7f17d81980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:27.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:27.940+0000 7f17def1e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f17d8102310 0x7f17d8198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:27.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:27.940+0000 7f17def1e700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f17d81079a0 0x7f17d819c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:27.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:27.940+0000 7f17def1e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f17d8111c80 con 0x7f17d8102310 2024-04-17T05:59:27.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:27.940+0000 7f17def1e700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f17d8111b00 con 0x7f17d8101950 2024-04-17T05:59:27.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:27.940+0000 7f17def1e700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f17d8111e00 con 0x7f17d81079a0 2024-04-17T05:59:27.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:27.940+0000 7f17d7fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f17d8102310 0x7f17d8198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:27.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:27.940+0000 7f17d7fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f17d8102310 0x7f17d8198630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46058/0 (socket says 172.21.15.47:46058) 2024-04-17T05:59:27.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:27.940+0000 7f17d7fff700 1 -- 172.21.15.47:0/1224558841 learned_addr learned my addr 172.21.15.47:0/1224558841 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:59:27.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:27.940+0000 7f17dccba700 1 --2- 172.21.15.47:0/1224558841 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f17d8101950 0x7f17d81980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:27.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:27.940+0000 7f17dd4bb700 1 --2- 172.21.15.47:0/1224558841 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f17d81079a0 0x7f17d819c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:27.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:27.940+0000 7f17d7fff700 1 -- 172.21.15.47:0/1224558841 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f17d81079a0 msgr2=0x7f17d819c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:27.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:27.940+0000 7f17d7fff700 1 --2- 172.21.15.47:0/1224558841 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f17d81079a0 0x7f17d819c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:27.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:27.940+0000 7f17d7fff700 1 -- 172.21.15.47:0/1224558841 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f17d8101950 msgr2=0x7f17d81980f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:27.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:27.940+0000 7f17d7fff700 1 --2- 172.21.15.47:0/1224558841 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f17d8101950 0x7f17d81980f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:27.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:27.940+0000 7f17d7fff700 1 -- 172.21.15.47:0/1224558841 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17d819d100 con 0x7f17d8102310 2024-04-17T05:59:27.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:27.940+0000 7f17d7fff700 1 --2- 172.21.15.47:0/1224558841 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f17d8102310 0x7f17d8198630 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f17c4005910 tx=0x7f17c40047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:59:27.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:27.941+0000 7f17d5ffb700 1 -- 172.21.15.47:0/1224558841 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c4027070 con 0x7f17d8102310 2024-04-17T05:59:27.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:27.941+0000 7f17d5ffb700 1 -- 172.21.15.47:0/1224558841 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17c4003780 con 0x7f17d8102310 2024-04-17T05:59:27.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:27.941+0000 7f17def1e700 1 -- 172.21.15.47:0/1224558841 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17d819d390 con 0x7f17d8102310 2024-04-17T05:59:27.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:27.941+0000 7f17d5ffb700 1 -- 172.21.15.47:0/1224558841 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c4022da0 con 0x7f17d8102310 2024-04-17T05:59:27.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:27.941+0000 7f17def1e700 1 -- 172.21.15.47:0/1224558841 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17d81a4c70 con 0x7f17d8102310 2024-04-17T05:59:27.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:27.942+0000 7f17def1e700 1 -- 172.21.15.47:0/1224558841 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17d8103070 con 0x7f17d8102310 2024-04-17T05:59:27.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:27.943+0000 7f17d5ffb700 1 -- 172.21.15.47:0/1224558841 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f17c402a070 con 0x7f17d8102310 2024-04-17T05:59:27.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:27.944+0000 7f17d5ffb700 1 --2- 172.21.15.47:0/1224558841 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f17c80a3cd0 0x7f17c80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:27.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:27.944+0000 7f17dccba700 1 --2- 172.21.15.47:0/1224558841 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f17c80a3cd0 0x7f17c80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:27.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:27.944+0000 7f17d5ffb700 1 -- 172.21.15.47:0/1224558841 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f17c40d1190 con 0x7f17d8102310 2024-04-17T05:59:27.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:27.944+0000 7f17dccba700 1 --2- 172.21.15.47:0/1224558841 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f17c80a3cd0 0x7f17c80a6190 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f17cc005950 tx=0x7f17cc00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:59:27.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:27.948+0000 7f17d5ffb700 1 -- 172.21.15.47:0/1224558841 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f17c405eff0 con 0x7f17d8102310 2024-04-17T05:59:27.979 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:27 smithi098 ceph-mon[36516]: pgmap v292: 97 pgs: 97 active+clean; 623 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 8.3 KiB/s wr, 18 op/s 2024-04-17T05:59:28.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:27 smithi103 ceph-mon[36243]: pgmap v292: 97 pgs: 97 active+clean; 623 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 8.3 KiB/s wr, 18 op/s 2024-04-17T05:59:28.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:28.215+0000 7f17def1e700 1 -- 172.21.15.47:0/1224558841 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f17d802df90 con 0x7f17d8102310 2024-04-17T05:59:28.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:28.221+0000 7f17d5ffb700 1 -- 172.21.15.47:0/1224558841 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+264998 (secure 0 0 0) 0x7f17c4013070 con 0x7f17d8102310 2024-04-17T05:59:28.223 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-17T05:59:28.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:28.225+0000 7f17def1e700 1 -- 172.21.15.47:0/1224558841 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f17c80a3cd0 msgr2=0x7f17c80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:28.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:28.225+0000 7f17def1e700 1 --2- 172.21.15.47:0/1224558841 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f17c80a3cd0 0x7f17c80a6190 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f17cc005950 tx=0x7f17cc00b410 comp rx=0 tx=0).stop 2024-04-17T05:59:28.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:28.225+0000 7f17def1e700 1 -- 172.21.15.47:0/1224558841 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f17d8102310 msgr2=0x7f17d8198630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:28.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:28.225+0000 7f17def1e700 1 --2- 172.21.15.47:0/1224558841 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f17d8102310 0x7f17d8198630 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f17c4005910 tx=0x7f17c40047b0 comp rx=0 tx=0).stop 2024-04-17T05:59:28.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:28.226+0000 7f17def1e700 1 -- 172.21.15.47:0/1224558841 shutdown_connections 2024-04-17T05:59:28.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:28.226+0000 7f17def1e700 1 --2- 172.21.15.47:0/1224558841 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f17c80a3cd0 0x7f17c80a6190 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:28.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:28.226+0000 7f17def1e700 1 --2- 172.21.15.47:0/1224558841 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f17d81079a0 0x7f17d819c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:28.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:28.226+0000 7f17def1e700 1 --2- 172.21.15.47:0/1224558841 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f17d8101950 0x7f17d81980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:28.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:28.226+0000 7f17def1e700 1 --2- 172.21.15.47:0/1224558841 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f17d8102310 0x7f17d8198630 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:28.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:28.226+0000 7f17def1e700 1 -- 172.21.15.47:0/1224558841 >> 172.21.15.47:0/1224558841 conn(0x7f17d8078900 msgr2=0x7f17d80ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:59:28.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:28.226+0000 7f17def1e700 1 -- 172.21.15.47:0/1224558841 shutdown_connections 2024-04-17T05:59:28.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:28.226+0000 7f17def1e700 1 -- 172.21.15.47:0/1224558841 wait complete. 2024-04-17T05:59:28.828 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:28 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/1224558841' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-17T05:59:28.857 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":3157902806},{"type":"v1","addr":"172.21.15.47:6801","nonce":3157902806}]},"active_addr":"172.21.15.47:6801/3157902806","active_change":"2024-04-17T05:52:13.645927+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24113,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.47:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.47:0","nonce":2970926607}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.47:0","nonce":3009926768}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.47:0","nonce":3275066738}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.47:0","nonce":3681048987}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.47:0","nonce":3970759820}]}]} 2024-04-17T05:59:28.864 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-04-17T05:59:28.864 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-04-17T05:59:28.864 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 565db2d8-fc7e-11ee-bc91-c7b262605968 -- ceph osd dump --format=json 2024-04-17T05:59:29.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:28 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/1224558841' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-17T05:59:29.114 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T05:59:29.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:28 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/1224558841' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-17T05:59:29.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:29 smithi047 ceph-mon[32223]: pgmap v293: 97 pgs: 97 active+clean; 623 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 7.0 KiB/s wr, 16 op/s 2024-04-17T05:59:30.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:29 smithi103 ceph-mon[36243]: pgmap v293: 97 pgs: 97 active+clean; 623 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 7.0 KiB/s wr, 16 op/s 2024-04-17T05:59:30.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:29 smithi098 ceph-mon[36516]: pgmap v293: 97 pgs: 97 active+clean; 623 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 7.0 KiB/s wr, 16 op/s 2024-04-17T05:59:30.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:30.773+0000 7fa360afc700 1 -- 172.21.15.47:0/3918146149 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa35c1020f0 msgr2=0x7fa35c102570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:30.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:30.773+0000 7fa360afc700 1 --2- 172.21.15.47:0/3918146149 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa35c1020f0 0x7fa35c102570 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa344009990 tx=0x7fa344009ca0 comp rx=0 tx=0).stop 2024-04-17T05:59:30.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:30.773+0000 7fa360afc700 1 -- 172.21.15.47:0/3918146149 shutdown_connections 2024-04-17T05:59:30.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:30.773+0000 7fa360afc700 1 --2- 172.21.15.47:0/3918146149 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa35c102ab0 0x7fa35c10e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:30.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:30.773+0000 7fa360afc700 1 --2- 172.21.15.47:0/3918146149 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa35c106940 0x7fa35c106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:30.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:30.773+0000 7fa360afc700 1 --2- 172.21.15.47:0/3918146149 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa35c1020f0 0x7fa35c102570 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:30.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:30.773+0000 7fa360afc700 1 -- 172.21.15.47:0/3918146149 >> 172.21.15.47:0/3918146149 conn(0x7fa35c077f50 msgr2=0x7fa35c0ff2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:59:30.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:30.773+0000 7fa360afc700 1 -- 172.21.15.47:0/3918146149 shutdown_connections 2024-04-17T05:59:30.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:30.773+0000 7fa360afc700 1 -- 172.21.15.47:0/3918146149 wait complete. 2024-04-17T05:59:30.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:30.774+0000 7fa360afc700 1 Processor -- start 2024-04-17T05:59:30.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:30.774+0000 7fa360afc700 1 -- start start 2024-04-17T05:59:30.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:30.775+0000 7fa360afc700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa35c1020f0 0x7fa35c0fd390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:30.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:30.775+0000 7fa360afc700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa35c102ab0 0x7fa35c0fd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:30.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:30.775+0000 7fa360afc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa35c106940 0x7fa35c0fde10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:30.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:30.775+0000 7fa360afc700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa35c111d60 con 0x7fa35c106940 2024-04-17T05:59:30.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:30.775+0000 7fa360afc700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa35c111be0 con 0x7fa35c1020f0 2024-04-17T05:59:30.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:30.775+0000 7fa360afc700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa35c111ee0 con 0x7fa35c102ab0 2024-04-17T05:59:30.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:30.775+0000 7fa35ad9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa35c106940 0x7fa35c0fde10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:30.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:30.775+0000 7fa35ad9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa35c106940 0x7fa35c0fde10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46076/0 (socket says 172.21.15.47:46076) 2024-04-17T05:59:30.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:30.775+0000 7fa35ad9d700 1 -- 172.21.15.47:0/4269828503 learned_addr learned my addr 172.21.15.47:0/4269828503 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:59:30.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:30.775+0000 7fa351d9b700 1 --2- 172.21.15.47:0/4269828503 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa35c102ab0 0x7fa35c0fd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:30.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:30.775+0000 7fa35a59c700 1 --2- 172.21.15.47:0/4269828503 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa35c1020f0 0x7fa35c0fd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:30.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:30.775+0000 7fa35ad9d700 1 -- 172.21.15.47:0/4269828503 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa35c102ab0 msgr2=0x7fa35c0fd8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:30.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:30.775+0000 7fa35ad9d700 1 --2- 172.21.15.47:0/4269828503 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa35c102ab0 0x7fa35c0fd8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:30.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:30.775+0000 7fa35ad9d700 1 -- 172.21.15.47:0/4269828503 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa35c1020f0 msgr2=0x7fa35c0fd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:30.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:30.775+0000 7fa35ad9d700 1 --2- 172.21.15.47:0/4269828503 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa35c1020f0 0x7fa35c0fd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:30.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:30.775+0000 7fa35ad9d700 1 -- 172.21.15.47:0/4269828503 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa35c1a77e0 con 0x7fa35c106940 2024-04-17T05:59:30.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:30.775+0000 7fa35a59c700 1 --2- 172.21.15.47:0/4269828503 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa35c1020f0 0x7fa35c0fd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T05:59:30.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:30.775+0000 7fa35ad9d700 1 --2- 172.21.15.47:0/4269828503 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa35c106940 0x7fa35c0fde10 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa354007ee0 tx=0x7fa35400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:59:30.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:30.775+0000 7fa353fff700 1 -- 172.21.15.47:0/4269828503 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa35400dbd0 con 0x7fa35c106940 2024-04-17T05:59:30.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:30.775+0000 7fa360afc700 1 -- 172.21.15.47:0/4269828503 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa35c1a7ad0 con 0x7fa35c106940 2024-04-17T05:59:30.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:30.775+0000 7fa353fff700 1 -- 172.21.15.47:0/4269828503 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa35400dd70 con 0x7fa35c106940 2024-04-17T05:59:30.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:30.775+0000 7fa353fff700 1 -- 172.21.15.47:0/4269828503 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa354005a60 con 0x7fa35c106940 2024-04-17T05:59:30.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:30.775+0000 7fa360afc700 1 -- 172.21.15.47:0/4269828503 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa35c1a7fe0 con 0x7fa35c106940 2024-04-17T05:59:30.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:30.776+0000 7fa360afc700 1 -- 172.21.15.47:0/4269828503 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa35c103100 con 0x7fa35c106940 2024-04-17T05:59:30.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:30.778+0000 7fa353fff700 1 -- 172.21.15.47:0/4269828503 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fa35401a020 con 0x7fa35c106940 2024-04-17T05:59:30.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:30.778+0000 7fa353fff700 1 --2- 172.21.15.47:0/4269828503 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fa3480a3cd0 0x7fa3480a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:30.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:30.778+0000 7fa35a59c700 1 --2- 172.21.15.47:0/4269828503 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fa3480a3cd0 0x7fa3480a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:30.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:30.778+0000 7fa353fff700 1 -- 172.21.15.47:0/4269828503 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fa3540d01e0 con 0x7fa35c106940 2024-04-17T05:59:30.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:30.779+0000 7fa35a59c700 1 --2- 172.21.15.47:0/4269828503 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fa3480a3cd0 0x7fa3480a6190 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa34c005e10 tx=0x7fa34c005d60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:59:30.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:30.782+0000 7fa353fff700 1 -- 172.21.15.47:0/4269828503 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fa354010070 con 0x7fa35c106940 2024-04-17T05:59:30.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:30.990+0000 7fa360afc700 1 -- 172.21.15.47:0/4269828503 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa35c0fe8c0 con 0x7fa35c106940 2024-04-17T05:59:30.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:30.992+0000 7fa353fff700 1 -- 172.21.15.47:0/4269828503 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22169 (secure 0 0 0) 0x7fa35409cc10 con 0x7fa35c106940 2024-04-17T05:59:30.990 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-17T05:59:30.990 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":86,"fsid":"565db2d8-fc7e-11ee-bc91-c7b262605968","created":"2024-04-17T05:51:19.888865+0000","modified":"2024-04-17T05:59:15.578280+0000","last_up_change":"2024-04-17T05:58:24.380131+0000","last_in_change":"2024-04-17T05:58:04.418236+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T05:54:37.734306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T05:59:12.458010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T05:59:12.901339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8453f300-533d-4736-a343-55619c08e966","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":1290127544},{"type":"v1","addr":"172.21.15.47:6803","nonce":1290127544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":1290127544},{"type":"v1","addr":"172.21.15.47:6805","nonce":1290127544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":1290127544},{"type":"v1","addr":"172.21.15.47:6809","nonce":1290127544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":1290127544},{"type":"v1","addr":"172.21.15.47:6807","nonce":1290127544}]},"public_addr":"172.21.15.47:6803/1290127544","cluster_addr":"172.21.15.47:6805/1290127544","heartbeat_back_addr":"172.21.15.47:6809/1290127544","heartbeat_front_addr":"172.21.15.47:6807/1290127544","state":["exists","up"]},{"osd":1,"uuid":"80b0f03e-c1bc-4658-8028-1e6b8eedcc47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":1180430954},{"type":"v1","addr":"172.21.15.98:6801","nonce":1180430954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":1180430954},{"type":"v1","addr":"172.21.15.98:6803","nonce":1180430954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":1180430954},{"type":"v1","addr":"172.21.15.98:6807","nonce":1180430954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":1180430954},{"type":"v1","addr":"172.21.15.98:6805","nonce":1180430954}]},"public_addr":"172.21.15.98:6801/1180430954","cluster_addr":"172.21.15.98:6803/1180430954","heartbeat_back_addr":"172.21.15.98:6807/1180430954","heartbeat_front_addr":"172.21.15.98:6805/1180430954","state":["exists","up"]},{"osd":2,"uuid":"0fc9d383-05b4-498d-b530-fccf7aeddd7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":494676216},{"type":"v1","addr":"172.21.15.103:6801","nonce":494676216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":494676216},{"type":"v1","addr":"172.21.15.103:6803","nonce":494676216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":494676216},{"type":"v1","addr":"172.21.15.103:6807","nonce":494676216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":494676216},{"type":"v1","addr":"172.21.15.103:6805","nonce":494676216}]},"public_addr":"172.21.15.103:6801/494676216","cluster_addr":"172.21.15.103:6803/494676216","heartbeat_back_addr":"172.21.15.103:6807/494676216","heartbeat_front_addr":"172.21.15.103:6805/494676216","state":["exists","up"]},{"osd":3,"uuid":"f6e0b0d5-0fb3-42d0-9e84-f7cfef21a998","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":1998369447},{"type":"v1","addr":"172.21.15.47:6811","nonce":1998369447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":1998369447},{"type":"v1","addr":"172.21.15.47:6813","nonce":1998369447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":1998369447},{"type":"v1","addr":"172.21.15.47:6817","nonce":1998369447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":1998369447},{"type":"v1","addr":"172.21.15.47:6815","nonce":1998369447}]},"public_addr":"172.21.15.47:6811/1998369447","cluster_addr":"172.21.15.47:6813/1998369447","heartbeat_back_addr":"172.21.15.47:6817/1998369447","heartbeat_front_addr":"172.21.15.47:6815/1998369447","state":["exists","up"]},{"osd":4,"uuid":"2c26217e-02df-466d-bc01-0149d4aec3c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":3060482485},{"type":"v1","addr":"172.21.15.98:6809","nonce":3060482485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":3060482485},{"type":"v1","addr":"172.21.15.98:6811","nonce":3060482485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":3060482485},{"type":"v1","addr":"172.21.15.98:6815","nonce":3060482485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":3060482485},{"type":"v1","addr":"172.21.15.98:6813","nonce":3060482485}]},"public_addr":"172.21.15.98:6809/3060482485","cluster_addr":"172.21.15.98:6811/3060482485","heartbeat_back_addr":"172.21.15.98:6815/3060482485","heartbeat_front_addr":"172.21.15.98:6813/3060482485","state":["exists","up"]},{"osd":5,"uuid":"740bf3e0-bea8-496b-8d6f-e42d0b017258","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":4181363567},{"type":"v1","addr":"172.21.15.103:6809","nonce":4181363567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":4181363567},{"type":"v1","addr":"172.21.15.103:6811","nonce":4181363567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":4181363567},{"type":"v1","addr":"172.21.15.103:6815","nonce":4181363567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":4181363567},{"type":"v1","addr":"172.21.15.103:6813","nonce":4181363567}]},"public_addr":"172.21.15.103:6809/4181363567","cluster_addr":"172.21.15.103:6811/4181363567","heartbeat_back_addr":"172.21.15.103:6815/4181363567","heartbeat_front_addr":"172.21.15.103:6813/4181363567","state":["exists","up"]},{"osd":6,"uuid":"356a6255-2c07-4f0d-a233-26be04c92b38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":2517529668},{"type":"v1","addr":"172.21.15.47:6819","nonce":2517529668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2517529668},{"type":"v1","addr":"172.21.15.47:6821","nonce":2517529668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2517529668},{"type":"v1","addr":"172.21.15.47:6825","nonce":2517529668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2517529668},{"type":"v1","addr":"172.21.15.47:6823","nonce":2517529668}]},"public_addr":"172.21.15.47:6819/2517529668","cluster_addr":"172.21.15.47:6821/2517529668","heartbeat_back_addr":"172.21.15.47:6825/2517529668","heartbeat_front_addr":"172.21.15.47:6823/2517529668","state":["exists","up"]},{"osd":7,"uuid":"349d0120-8290-417d-a800-5a8985f554f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":4225498496},{"type":"v1","addr":"172.21.15.98:6817","nonce":4225498496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":4225498496},{"type":"v1","addr":"172.21.15.98:6819","nonce":4225498496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":4225498496},{"type":"v1","addr":"172.21.15.98:6823","nonce":4225498496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":4225498496},{"type":"v1","addr":"172.21.15.98:6821","nonce":4225498496}]},"public_addr":"172.21.15.98:6817/4225498496","cluster_addr":"172.21.15.98:6819/4225498496","heartbeat_back_addr":"172.21.15.98:6823/4225498496","heartbeat_front_addr":"172.21.15.98:6821/4225498496","state":["exists","up"]},{"osd":8,"uuid":"cee4712f-321b-42c0-92ba-e15144c00f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":3433905520},{"type":"v1","addr":"172.21.15.103:6817","nonce":3433905520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":3433905520},{"type":"v1","addr":"172.21.15.103:6819","nonce":3433905520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":3433905520},{"type":"v1","addr":"172.21.15.103:6823","nonce":3433905520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":3433905520},{"type":"v1","addr":"172.21.15.103:6821","nonce":3433905520}]},"public_addr":"172.21.15.103:6817/3433905520","cluster_addr":"172.21.15.103:6819/3433905520","heartbeat_back_addr":"172.21.15.103:6823/3433905520","heartbeat_front_addr":"172.21.15.103:6821/3433905520","state":["exists","up"]},{"osd":9,"uuid":"63fc73c1-5a22-487d-80b8-9894e2ddd207","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":728359554},{"type":"v1","addr":"172.21.15.47:6827","nonce":728359554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":728359554},{"type":"v1","addr":"172.21.15.47:6829","nonce":728359554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":728359554},{"type":"v1","addr":"172.21.15.47:6833","nonce":728359554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":728359554},{"type":"v1","addr":"172.21.15.47:6831","nonce":728359554}]},"public_addr":"172.21.15.47:6827/728359554","cluster_addr":"172.21.15.47:6829/728359554","heartbeat_back_addr":"172.21.15.47:6833/728359554","heartbeat_front_addr":"172.21.15.47:6831/728359554","state":["exists","up"]},{"osd":10,"uuid":"9565bcc3-62ee-48eb-a07a-03a81e0cc8e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":468205689},{"type":"v1","addr":"172.21.15.98:6825","nonce":468205689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":468205689},{"type":"v1","addr":"172.21.15.98:6827","nonce":468205689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":468205689},{"type":"v1","addr":"172.21.15.98:6831","nonce":468205689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":468205689},{"type":"v1","addr":"172.21.15.98:6829","nonce":468205689}]},"public_addr":"172.21.15.98:6825/468205689","cluster_addr":"172.21.15.98:6827/468205689","heartbeat_back_addr":"172.21.15.98:6831/468205689","heartbeat_front_addr":"172.21.15.98:6829/468205689","state":["exists","up"]},{"osd":11,"uuid":"2e85e8d9-bf6c-4a12-b124-a4690dd61108","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":2639831013},{"type":"v1","addr":"172.21.15.103:6825","nonce":2639831013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":2639831013},{"type":"v1","addr":"172.21.15.103:6827","nonce":2639831013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":2639831013},{"type":"v1","addr":"172.21.15.103:6831","nonce":2639831013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":2639831013},{"type":"v1","addr":"172.21.15.103:6829","nonce":2639831013}]},"public_addr":"172.21.15.103:6825/2639831013","cluster_addr":"172.21.15.103:6827/2639831013","heartbeat_back_addr":"172.21.15.103:6831/2639831013","heartbeat_front_addr":"172.21.15.103:6829/2639831013","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:53:47.366438+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:54:10.953559+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:54:34.320332+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:54:59.751360+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:55:23.982052+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:55:48.425017+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:56:14.793368+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:56:40.440830+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:57:04.623680+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:57:31.215262+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:57:57.040346+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:58:22.671572+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:6801/3632157175":"2024-04-18T05:52:13.645775+0000","172.21.15.47:0/4288219269":"2024-04-18T05:52:13.645775+0000","172.21.15.47:0/2584487544":"2024-04-18T05:52:13.645775+0000","172.21.15.47:6801/953064786":"2024-04-18T05:51:45.979351+0000","172.21.15.47:0/3484500927":"2024-04-18T05:51:45.979351+0000","172.21.15.47:6800/3632157175":"2024-04-18T05:52:13.645775+0000","172.21.15.47:0/112268670":"2024-04-18T05:51:45.979351+0000","172.21.15.47:0/1638423256":"2024-04-18T05:51:45.979351+0000","172.21.15.47:6800/953064786":"2024-04-18T05:51:45.979351+0000","172.21.15.47:0/849479010":"2024-04-18T05:52:13.645775+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T05:59:30.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:30.994+0000 7fa360afc700 1 -- 172.21.15.47:0/4269828503 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fa3480a3cd0 msgr2=0x7fa3480a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:30.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:30.994+0000 7fa360afc700 1 --2- 172.21.15.47:0/4269828503 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fa3480a3cd0 0x7fa3480a6190 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa34c005e10 tx=0x7fa34c005d60 comp rx=0 tx=0).stop 2024-04-17T05:59:30.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:30.994+0000 7fa360afc700 1 -- 172.21.15.47:0/4269828503 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa35c106940 msgr2=0x7fa35c0fde10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:30.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:30.994+0000 7fa360afc700 1 --2- 172.21.15.47:0/4269828503 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa35c106940 0x7fa35c0fde10 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa354007ee0 tx=0x7fa35400d3b0 comp rx=0 tx=0).stop 2024-04-17T05:59:30.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:30.994+0000 7fa360afc700 1 -- 172.21.15.47:0/4269828503 shutdown_connections 2024-04-17T05:59:30.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:30.994+0000 7fa360afc700 1 --2- 172.21.15.47:0/4269828503 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fa3480a3cd0 0x7fa3480a6190 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:30.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:30.994+0000 7fa360afc700 1 --2- 172.21.15.47:0/4269828503 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa35c106940 0x7fa35c0fde10 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:30.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:30.994+0000 7fa360afc700 1 --2- 172.21.15.47:0/4269828503 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa35c1020f0 0x7fa35c0fd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:30.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:30.994+0000 7fa360afc700 1 --2- 172.21.15.47:0/4269828503 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa35c102ab0 0x7fa35c0fd8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:30.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:30.994+0000 7fa360afc700 1 -- 172.21.15.47:0/4269828503 >> 172.21.15.47:0/4269828503 conn(0x7fa35c077f50 msgr2=0x7fa35c101280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:59:30.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:30.994+0000 7fa360afc700 1 -- 172.21.15.47:0/4269828503 shutdown_connections 2024-04-17T05:59:30.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:30.994+0000 7fa360afc700 1 -- 172.21.15.47:0/4269828503 wait complete. 2024-04-17T05:59:31.705 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-04-17T05:59:31.705 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 565db2d8-fc7e-11ee-bc91-c7b262605968 -- ceph osd dump --format=json 2024-04-17T05:59:31.957 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T05:59:32.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:31 smithi103 ceph-mon[36243]: pgmap v294: 97 pgs: 97 active+clean; 638 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 7.7 KiB/s wr, 16 op/s 2024-04-17T05:59:32.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:31 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/4269828503' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T05:59:32.078 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:31 smithi047 ceph-mon[32223]: pgmap v294: 97 pgs: 97 active+clean; 638 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 7.7 KiB/s wr, 16 op/s 2024-04-17T05:59:32.079 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:31 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/4269828503' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T05:59:32.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:31 smithi098 ceph-mon[36516]: pgmap v294: 97 pgs: 97 active+clean; 638 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 7.7 KiB/s wr, 16 op/s 2024-04-17T05:59:32.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:31 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/4269828503' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T05:59:33.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:33.664+0000 7f823554d700 1 -- 172.21.15.47:0/1004075989 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8230102500 msgr2=0x7f8230102980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:33.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:33.664+0000 7f823554d700 1 --2- 172.21.15.47:0/1004075989 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8230102500 0x7f8230102980 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f82240099c0 tx=0x7f8224009cd0 comp rx=0 tx=0).stop 2024-04-17T05:59:33.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:33.664+0000 7f823554d700 1 -- 172.21.15.47:0/1004075989 shutdown_connections 2024-04-17T05:59:33.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:33.664+0000 7f823554d700 1 --2- 172.21.15.47:0/1004075989 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8230102ec0 0x7f823010e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:33.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:33.664+0000 7f823554d700 1 --2- 172.21.15.47:0/1004075989 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8230107c20 0x7f8230108000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:33.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:33.664+0000 7f823554d700 1 --2- 172.21.15.47:0/1004075989 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8230102500 0x7f8230102980 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:33.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:33.664+0000 7f823554d700 1 -- 172.21.15.47:0/1004075989 >> 172.21.15.47:0/1004075989 conn(0x7f8230076c70 msgr2=0x7f8230077080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:59:33.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:33.665+0000 7f823554d700 1 -- 172.21.15.47:0/1004075989 shutdown_connections 2024-04-17T05:59:33.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:33.665+0000 7f823554d700 1 -- 172.21.15.47:0/1004075989 wait complete. 2024-04-17T05:59:33.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:33.665+0000 7f823554d700 1 Processor -- start 2024-04-17T05:59:33.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:33.665+0000 7f823554d700 1 -- start start 2024-04-17T05:59:33.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:33.666+0000 7f823554d700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8230102500 0x7f82301992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:33.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:33.666+0000 7f823554d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8230102ec0 0x7f82301997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:33.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:33.666+0000 7f823554d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8230107c20 0x7f823019db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:33.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:33.666+0000 7f823554d700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f8230111df0 con 0x7f8230107c20 2024-04-17T05:59:33.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:33.666+0000 7f823554d700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8230111c70 con 0x7f8230102ec0 2024-04-17T05:59:33.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:33.666+0000 7f823554d700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8230111f70 con 0x7f8230102500 2024-04-17T05:59:33.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:33.666+0000 7f822f7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8230107c20 0x7f823019db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:33.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:33.666+0000 7f822e7fc700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8230102ec0 0x7f82301997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:33.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:33.666+0000 7f822f7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8230107c20 0x7f823019db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46094/0 (socket says 172.21.15.47:46094) 2024-04-17T05:59:33.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:33.666+0000 7f822f7fe700 1 -- 172.21.15.47:0/856019262 learned_addr learned my addr 172.21.15.47:0/856019262 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:59:33.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:33.666+0000 7f822effd700 1 --2- 172.21.15.47:0/856019262 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8230102500 0x7f82301992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:33.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:33.666+0000 7f822f7fe700 1 -- 172.21.15.47:0/856019262 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8230102500 msgr2=0x7f82301992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:33.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:33.666+0000 7f822f7fe700 1 --2- 172.21.15.47:0/856019262 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8230102500 0x7f82301992b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:33.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:33.666+0000 7f822f7fe700 1 -- 172.21.15.47:0/856019262 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8230102ec0 msgr2=0x7f82301997f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:33.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:33.666+0000 7f822f7fe700 1 --2- 172.21.15.47:0/856019262 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8230102ec0 0x7f82301997f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:33.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:33.666+0000 7f822f7fe700 1 -- 172.21.15.47:0/856019262 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f823019e2c0 con 0x7f8230107c20 2024-04-17T05:59:33.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:33.666+0000 7f822f7fe700 1 --2- 172.21.15.47:0/856019262 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8230107c20 0x7f823019db80 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f822800ada0 tx=0x7f822800c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:59:33.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:33.666+0000 7f8217fff700 1 -- 172.21.15.47:0/856019262 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8228018070 con 0x7f8230107c20 2024-04-17T05:59:33.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:33.667+0000 7f8217fff700 1 -- 172.21.15.47:0/856019262 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f822800a0d0 con 0x7f8230107c20 2024-04-17T05:59:33.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:33.667+0000 7f823554d700 1 -- 172.21.15.47:0/856019262 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f823019e5b0 con 0x7f8230107c20 2024-04-17T05:59:33.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:33.667+0000 7f8217fff700 1 -- 172.21.15.47:0/856019262 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8228004290 con 0x7f8230107c20 2024-04-17T05:59:33.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:33.667+0000 7f823554d700 1 -- 172.21.15.47:0/856019262 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82301a5e30 con 0x7f8230107c20 2024-04-17T05:59:33.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:33.668+0000 7f823554d700 1 -- 172.21.15.47:0/856019262 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8230069740 con 0x7f8230107c20 2024-04-17T05:59:33.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:33.669+0000 7f8217fff700 1 -- 172.21.15.47:0/856019262 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f822801b070 con 0x7f8230107c20 2024-04-17T05:59:33.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:33.669+0000 7f8217fff700 1 --2- 172.21.15.47:0/856019262 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f82180a3d50 0x7f82180a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:33.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:33.670+0000 7f822effd700 1 --2- 172.21.15.47:0/856019262 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f82180a3d50 0x7f82180a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:33.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:33.670+0000 7f8217fff700 1 -- 172.21.15.47:0/856019262 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f82280cf310 con 0x7f8230107c20 2024-04-17T05:59:33.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:33.670+0000 7f822effd700 1 --2- 172.21.15.47:0/856019262 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f82180a3d50 0x7f82180a6210 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8220009a10 tx=0x7f8220009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:59:33.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:33.674+0000 7f8217fff700 1 -- 172.21.15.47:0/856019262 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f822804fff0 con 0x7f8230107c20 2024-04-17T05:59:33.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:33.882+0000 7f823554d700 1 -- 172.21.15.47:0/856019262 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f823002df90 con 0x7f8230107c20 2024-04-17T05:59:33.880 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:33 smithi047 ceph-mon[32223]: pgmap v295: 97 pgs: 97 active+clean; 638 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 7.2 KiB/s wr, 13 op/s 2024-04-17T05:59:33.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:33.885+0000 7f8217fff700 1 -- 172.21.15.47:0/856019262 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22169 (secure 0 0 0) 0x7f8228097070 con 0x7f8230107c20 2024-04-17T05:59:33.883 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-17T05:59:33.883 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":86,"fsid":"565db2d8-fc7e-11ee-bc91-c7b262605968","created":"2024-04-17T05:51:19.888865+0000","modified":"2024-04-17T05:59:15.578280+0000","last_up_change":"2024-04-17T05:58:24.380131+0000","last_in_change":"2024-04-17T05:58:04.418236+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T05:54:37.734306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T05:59:12.458010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T05:59:12.901339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8453f300-533d-4736-a343-55619c08e966","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":1290127544},{"type":"v1","addr":"172.21.15.47:6803","nonce":1290127544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":1290127544},{"type":"v1","addr":"172.21.15.47:6805","nonce":1290127544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":1290127544},{"type":"v1","addr":"172.21.15.47:6809","nonce":1290127544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":1290127544},{"type":"v1","addr":"172.21.15.47:6807","nonce":1290127544}]},"public_addr":"172.21.15.47:6803/1290127544","cluster_addr":"172.21.15.47:6805/1290127544","heartbeat_back_addr":"172.21.15.47:6809/1290127544","heartbeat_front_addr":"172.21.15.47:6807/1290127544","state":["exists","up"]},{"osd":1,"uuid":"80b0f03e-c1bc-4658-8028-1e6b8eedcc47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":1180430954},{"type":"v1","addr":"172.21.15.98:6801","nonce":1180430954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":1180430954},{"type":"v1","addr":"172.21.15.98:6803","nonce":1180430954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":1180430954},{"type":"v1","addr":"172.21.15.98:6807","nonce":1180430954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":1180430954},{"type":"v1","addr":"172.21.15.98:6805","nonce":1180430954}]},"public_addr":"172.21.15.98:6801/1180430954","cluster_addr":"172.21.15.98:6803/1180430954","heartbeat_back_addr":"172.21.15.98:6807/1180430954","heartbeat_front_addr":"172.21.15.98:6805/1180430954","state":["exists","up"]},{"osd":2,"uuid":"0fc9d383-05b4-498d-b530-fccf7aeddd7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":494676216},{"type":"v1","addr":"172.21.15.103:6801","nonce":494676216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":494676216},{"type":"v1","addr":"172.21.15.103:6803","nonce":494676216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":494676216},{"type":"v1","addr":"172.21.15.103:6807","nonce":494676216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":494676216},{"type":"v1","addr":"172.21.15.103:6805","nonce":494676216}]},"public_addr":"172.21.15.103:6801/494676216","cluster_addr":"172.21.15.103:6803/494676216","heartbeat_back_addr":"172.21.15.103:6807/494676216","heartbeat_front_addr":"172.21.15.103:6805/494676216","state":["exists","up"]},{"osd":3,"uuid":"f6e0b0d5-0fb3-42d0-9e84-f7cfef21a998","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":1998369447},{"type":"v1","addr":"172.21.15.47:6811","nonce":1998369447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":1998369447},{"type":"v1","addr":"172.21.15.47:6813","nonce":1998369447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":1998369447},{"type":"v1","addr":"172.21.15.47:6817","nonce":1998369447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":1998369447},{"type":"v1","addr":"172.21.15.47:6815","nonce":1998369447}]},"public_addr":"172.21.15.47:6811/1998369447","cluster_addr":"172.21.15.47:6813/1998369447","heartbeat_back_addr":"172.21.15.47:6817/1998369447","heartbeat_front_addr":"172.21.15.47:6815/1998369447","state":["exists","up"]},{"osd":4,"uuid":"2c26217e-02df-466d-bc01-0149d4aec3c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":3060482485},{"type":"v1","addr":"172.21.15.98:6809","nonce":3060482485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":3060482485},{"type":"v1","addr":"172.21.15.98:6811","nonce":3060482485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":3060482485},{"type":"v1","addr":"172.21.15.98:6815","nonce":3060482485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":3060482485},{"type":"v1","addr":"172.21.15.98:6813","nonce":3060482485}]},"public_addr":"172.21.15.98:6809/3060482485","cluster_addr":"172.21.15.98:6811/3060482485","heartbeat_back_addr":"172.21.15.98:6815/3060482485","heartbeat_front_addr":"172.21.15.98:6813/3060482485","state":["exists","up"]},{"osd":5,"uuid":"740bf3e0-bea8-496b-8d6f-e42d0b017258","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":4181363567},{"type":"v1","addr":"172.21.15.103:6809","nonce":4181363567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":4181363567},{"type":"v1","addr":"172.21.15.103:6811","nonce":4181363567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":4181363567},{"type":"v1","addr":"172.21.15.103:6815","nonce":4181363567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":4181363567},{"type":"v1","addr":"172.21.15.103:6813","nonce":4181363567}]},"public_addr":"172.21.15.103:6809/4181363567","cluster_addr":"172.21.15.103:6811/4181363567","heartbeat_back_addr":"172.21.15.103:6815/4181363567","heartbeat_front_addr":"172.21.15.103:6813/4181363567","state":["exists","up"]},{"osd":6,"uuid":"356a6255-2c07-4f0d-a233-26be04c92b38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":2517529668},{"type":"v1","addr":"172.21.15.47:6819","nonce":2517529668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2517529668},{"type":"v1","addr":"172.21.15.47:6821","nonce":2517529668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2517529668},{"type":"v1","addr":"172.21.15.47:6825","nonce":2517529668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2517529668},{"type":"v1","addr":"172.21.15.47:6823","nonce":2517529668}]},"public_addr":"172.21.15.47:6819/2517529668","cluster_addr":"172.21.15.47:6821/2517529668","heartbeat_back_addr":"172.21.15.47:6825/2517529668","heartbeat_front_addr":"172.21.15.47:6823/2517529668","state":["exists","up"]},{"osd":7,"uuid":"349d0120-8290-417d-a800-5a8985f554f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":4225498496},{"type":"v1","addr":"172.21.15.98:6817","nonce":4225498496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":4225498496},{"type":"v1","addr":"172.21.15.98:6819","nonce":4225498496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":4225498496},{"type":"v1","addr":"172.21.15.98:6823","nonce":4225498496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":4225498496},{"type":"v1","addr":"172.21.15.98:6821","nonce":4225498496}]},"public_addr":"172.21.15.98:6817/4225498496","cluster_addr":"172.21.15.98:6819/4225498496","heartbeat_back_addr":"172.21.15.98:6823/4225498496","heartbeat_front_addr":"172.21.15.98:6821/4225498496","state":["exists","up"]},{"osd":8,"uuid":"cee4712f-321b-42c0-92ba-e15144c00f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":3433905520},{"type":"v1","addr":"172.21.15.103:6817","nonce":3433905520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":3433905520},{"type":"v1","addr":"172.21.15.103:6819","nonce":3433905520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":3433905520},{"type":"v1","addr":"172.21.15.103:6823","nonce":3433905520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":3433905520},{"type":"v1","addr":"172.21.15.103:6821","nonce":3433905520}]},"public_addr":"172.21.15.103:6817/3433905520","cluster_addr":"172.21.15.103:6819/3433905520","heartbeat_back_addr":"172.21.15.103:6823/3433905520","heartbeat_front_addr":"172.21.15.103:6821/3433905520","state":["exists","up"]},{"osd":9,"uuid":"63fc73c1-5a22-487d-80b8-9894e2ddd207","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":728359554},{"type":"v1","addr":"172.21.15.47:6827","nonce":728359554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":728359554},{"type":"v1","addr":"172.21.15.47:6829","nonce":728359554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":728359554},{"type":"v1","addr":"172.21.15.47:6833","nonce":728359554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":728359554},{"type":"v1","addr":"172.21.15.47:6831","nonce":728359554}]},"public_addr":"172.21.15.47:6827/728359554","cluster_addr":"172.21.15.47:6829/728359554","heartbeat_back_addr":"172.21.15.47:6833/728359554","heartbeat_front_addr":"172.21.15.47:6831/728359554","state":["exists","up"]},{"osd":10,"uuid":"9565bcc3-62ee-48eb-a07a-03a81e0cc8e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":468205689},{"type":"v1","addr":"172.21.15.98:6825","nonce":468205689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":468205689},{"type":"v1","addr":"172.21.15.98:6827","nonce":468205689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":468205689},{"type":"v1","addr":"172.21.15.98:6831","nonce":468205689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":468205689},{"type":"v1","addr":"172.21.15.98:6829","nonce":468205689}]},"public_addr":"172.21.15.98:6825/468205689","cluster_addr":"172.21.15.98:6827/468205689","heartbeat_back_addr":"172.21.15.98:6831/468205689","heartbeat_front_addr":"172.21.15.98:6829/468205689","state":["exists","up"]},{"osd":11,"uuid":"2e85e8d9-bf6c-4a12-b124-a4690dd61108","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":2639831013},{"type":"v1","addr":"172.21.15.103:6825","nonce":2639831013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":2639831013},{"type":"v1","addr":"172.21.15.103:6827","nonce":2639831013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":2639831013},{"type":"v1","addr":"172.21.15.103:6831","nonce":2639831013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":2639831013},{"type":"v1","addr":"172.21.15.103:6829","nonce":2639831013}]},"public_addr":"172.21.15.103:6825/2639831013","cluster_addr":"172.21.15.103:6827/2639831013","heartbeat_back_addr":"172.21.15.103:6831/2639831013","heartbeat_front_addr":"172.21.15.103:6829/2639831013","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:53:47.366438+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:54:10.953559+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:54:34.320332+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:54:59.751360+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:55:23.982052+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:55:48.425017+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:56:14.793368+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:56:40.440830+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:57:04.623680+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:57:31.215262+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:57:57.040346+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:58:22.671572+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:6801/3632157175":"2024-04-18T05:52:13.645775+0000","172.21.15.47:0/4288219269":"2024-04-18T05:52:13.645775+0000","172.21.15.47:0/2584487544":"2024-04-18T05:52:13.645775+0000","172.21.15.47:6801/953064786":"2024-04-18T05:51:45.979351+0000","172.21.15.47:0/3484500927":"2024-04-18T05:51:45.979351+0000","172.21.15.47:6800/3632157175":"2024-04-18T05:52:13.645775+0000","172.21.15.47:0/112268670":"2024-04-18T05:51:45.979351+0000","172.21.15.47:0/1638423256":"2024-04-18T05:51:45.979351+0000","172.21.15.47:6800/953064786":"2024-04-18T05:51:45.979351+0000","172.21.15.47:0/849479010":"2024-04-18T05:52:13.645775+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T05:59:33.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:33.888+0000 7f823554d700 1 -- 172.21.15.47:0/856019262 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f82180a3d50 msgr2=0x7f82180a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:33.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:33.888+0000 7f823554d700 1 --2- 172.21.15.47:0/856019262 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f82180a3d50 0x7f82180a6210 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8220009a10 tx=0x7f8220009190 comp rx=0 tx=0).stop 2024-04-17T05:59:33.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:33.888+0000 7f823554d700 1 -- 172.21.15.47:0/856019262 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8230107c20 msgr2=0x7f823019db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:33.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:33.888+0000 7f823554d700 1 --2- 172.21.15.47:0/856019262 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8230107c20 0x7f823019db80 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f822800ada0 tx=0x7f822800c330 comp rx=0 tx=0).stop 2024-04-17T05:59:33.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:33.888+0000 7f823554d700 1 -- 172.21.15.47:0/856019262 shutdown_connections 2024-04-17T05:59:33.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:33.888+0000 7f823554d700 1 --2- 172.21.15.47:0/856019262 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f82180a3d50 0x7f82180a6210 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:33.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:33.888+0000 7f823554d700 1 --2- 172.21.15.47:0/856019262 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8230107c20 0x7f823019db80 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:33.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:33.888+0000 7f823554d700 1 --2- 172.21.15.47:0/856019262 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8230102500 0x7f82301992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:33.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:33.888+0000 7f823554d700 1 --2- 172.21.15.47:0/856019262 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8230102ec0 0x7f82301997f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:33.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:33.888+0000 7f823554d700 1 -- 172.21.15.47:0/856019262 >> 172.21.15.47:0/856019262 conn(0x7f8230076c70 msgr2=0x7f82301014b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:59:33.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:33.888+0000 7f823554d700 1 -- 172.21.15.47:0/856019262 shutdown_connections 2024-04-17T05:59:33.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:33.888+0000 7f823554d700 1 -- 172.21.15.47:0/856019262 wait complete. 2024-04-17T05:59:34.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:33 smithi103 ceph-mon[36243]: pgmap v295: 97 pgs: 97 active+clean; 638 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 7.2 KiB/s wr, 13 op/s 2024-04-17T05:59:34.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:33 smithi098 ceph-mon[36516]: pgmap v295: 97 pgs: 97 active+clean; 638 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 7.2 KiB/s wr, 13 op/s 2024-04-17T05:59:34.687 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 565db2d8-fc7e-11ee-bc91-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-04-17T05:59:34.687 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 565db2d8-fc7e-11ee-bc91-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-04-17T05:59:34.688 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 565db2d8-fc7e-11ee-bc91-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-04-17T05:59:34.689 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 565db2d8-fc7e-11ee-bc91-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-04-17T05:59:34.689 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 565db2d8-fc7e-11ee-bc91-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-04-17T05:59:34.690 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 565db2d8-fc7e-11ee-bc91-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-04-17T05:59:34.691 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 565db2d8-fc7e-11ee-bc91-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-04-17T05:59:34.691 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 565db2d8-fc7e-11ee-bc91-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-04-17T05:59:34.692 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 565db2d8-fc7e-11ee-bc91-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-04-17T05:59:34.692 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 565db2d8-fc7e-11ee-bc91-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-04-17T05:59:34.693 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 565db2d8-fc7e-11ee-bc91-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-04-17T05:59:34.694 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 565db2d8-fc7e-11ee-bc91-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-04-17T05:59:35.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:34 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/856019262' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T05:59:35.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:34 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/856019262' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T05:59:35.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:34 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/856019262' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T05:59:35.458 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T05:59:35.459 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T05:59:35.697 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:35 smithi047 ceph-mon[32223]: pgmap v296: 97 pgs: 97 active+clean; 638 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 7.2 KiB/s wr, 13 op/s 2024-04-17T05:59:35.825 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T05:59:36.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:35 smithi103 ceph-mon[36243]: pgmap v296: 97 pgs: 97 active+clean; 638 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 7.2 KiB/s wr, 13 op/s 2024-04-17T05:59:36.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:35 smithi098 ceph-mon[36516]: pgmap v296: 97 pgs: 97 active+clean; 638 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 7.2 KiB/s wr, 13 op/s 2024-04-17T05:59:37.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:37 smithi098 ceph-mon[36516]: pgmap v297: 97 pgs: 97 active+clean; 638 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 5.2 KiB/s wr, 9 op/s 2024-04-17T05:59:38.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:37 smithi103 ceph-mon[36243]: pgmap v297: 97 pgs: 97 active+clean; 638 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 5.2 KiB/s wr, 9 op/s 2024-04-17T05:59:38.086 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:37 smithi047 ceph-mon[32223]: pgmap v297: 97 pgs: 97 active+clean; 638 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 5.2 KiB/s wr, 9 op/s 2024-04-17T05:59:38.258 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T05:59:38.367 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T05:59:38.369 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T05:59:38.372 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T05:59:38.373 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T05:59:38.373 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T05:59:38.377 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T05:59:38.385 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T05:59:38.385 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T05:59:39.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:39.236+0000 7f1878c3d700 1 -- 172.21.15.47:0/3802479432 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1874102340 msgr2=0x7f187410e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:39.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:39.236+0000 7f1878c3d700 1 --2- 172.21.15.47:0/3802479432 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1874102340 0x7f187410e800 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f186c00b0a0 tx=0x7f186c00b3b0 comp rx=0 tx=0).stop 2024-04-17T05:59:39.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:39.236+0000 7f1878c3d700 1 -- 172.21.15.47:0/3802479432 shutdown_connections 2024-04-17T05:59:39.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:39.236+0000 7f1878c3d700 1 --2- 172.21.15.47:0/3802479432 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1874102340 0x7f187410e800 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:39.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:39.236+0000 7f1878c3d700 1 --2- 172.21.15.47:0/3802479432 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1874107930 0x7f1874107d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:39.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:39.236+0000 7f1878c3d700 1 --2- 172.21.15.47:0/3802479432 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1874101980 0x7f1874101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:39.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:39.236+0000 7f1878c3d700 1 -- 172.21.15.47:0/3802479432 >> 172.21.15.47:0/3802479432 conn(0x7f1874078900 msgr2=0x7f1874078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:59:39.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:39.236+0000 7f1878c3d700 1 -- 172.21.15.47:0/3802479432 shutdown_connections 2024-04-17T05:59:39.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:39.236+0000 7f1878c3d700 1 -- 172.21.15.47:0/3802479432 wait complete. 2024-04-17T05:59:39.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:39.237+0000 7f1878c3d700 1 Processor -- start 2024-04-17T05:59:39.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:39.237+0000 7f1878c3d700 1 -- start start 2024-04-17T05:59:39.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:39.237+0000 7f1878c3d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1874101980 0x7f1874103500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:39.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:39.237+0000 7f1878c3d700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1874102340 0x7f1874103a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:39.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:39.237+0000 7f1878c3d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1874107930 0x7f1874103f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:39.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:39.237+0000 7f1878c3d700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f1874111c90 con 0x7f1874101980 2024-04-17T05:59:39.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:39.237+0000 7f1878c3d700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1874111b10 con 0x7f1874107930 2024-04-17T05:59:39.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:39.237+0000 7f1878c3d700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1874111e10 con 0x7f1874102340 2024-04-17T05:59:39.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:39.237+0000 7f187259c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1874101980 0x7f1874103500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:39.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:39.237+0000 7f187259c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1874101980 0x7f1874103500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47998/0 (socket says 172.21.15.47:47998) 2024-04-17T05:59:39.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:39.237+0000 7f187259c700 1 -- 172.21.15.47:0/95967611 learned_addr learned my addr 172.21.15.47:0/95967611 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:59:39.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:39.237+0000 7f1872d9d700 1 --2- 172.21.15.47:0/95967611 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1874107930 0x7f1874103f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:39.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:39.237+0000 7f1871d9b700 1 --2- 172.21.15.47:0/95967611 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1874102340 0x7f1874103a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:39.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:39.237+0000 7f187259c700 1 -- 172.21.15.47:0/95967611 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1874102340 msgr2=0x7f1874103a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:39.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:39.237+0000 7f187259c700 1 --2- 172.21.15.47:0/95967611 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1874102340 0x7f1874103a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:39.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:39.237+0000 7f187259c700 1 -- 172.21.15.47:0/95967611 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1874107930 msgr2=0x7f1874103f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:39.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:39.237+0000 7f187259c700 1 --2- 172.21.15.47:0/95967611 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1874107930 0x7f1874103f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:39.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:39.237+0000 7f187259c700 1 -- 172.21.15.47:0/95967611 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18741a7700 con 0x7f1874101980 2024-04-17T05:59:39.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:39.238+0000 7f187259c700 1 --2- 172.21.15.47:0/95967611 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1874101980 0x7f1874103500 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f186400d980 tx=0x7f186400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:59:39.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:39.238+0000 7f18637fe700 1 -- 172.21.15.47:0/95967611 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1864016070 con 0x7f1874101980 2024-04-17T05:59:39.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:39.238+0000 7f18637fe700 1 -- 172.21.15.47:0/95967611 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1864004d10 con 0x7f1874101980 2024-04-17T05:59:39.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:39.238+0000 7f1878c3d700 1 -- 172.21.15.47:0/95967611 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18741a79f0 con 0x7f1874101980 2024-04-17T05:59:39.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:39.238+0000 7f18637fe700 1 -- 172.21.15.47:0/95967611 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18640055f0 con 0x7f1874101980 2024-04-17T05:59:39.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:39.238+0000 7f1878c3d700 1 -- 172.21.15.47:0/95967611 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18741a7f30 con 0x7f1874101980 2024-04-17T05:59:39.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:39.239+0000 7f1878c3d700 1 -- 172.21.15.47:0/95967611 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f18740630e0 con 0x7f1874101980 2024-04-17T05:59:39.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:39.240+0000 7f18637fe700 1 -- 172.21.15.47:0/95967611 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f1864004750 con 0x7f1874101980 2024-04-17T05:59:39.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:39.241+0000 7f18637fe700 1 --2- 172.21.15.47:0/95967611 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f185c0a3cd0 0x7f185c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:39.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:39.241+0000 7f1871d9b700 1 --2- 172.21.15.47:0/95967611 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f185c0a3cd0 0x7f185c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:39.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:39.241+0000 7f18637fe700 1 -- 172.21.15.47:0/95967611 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f18640d0970 con 0x7f1874101980 2024-04-17T05:59:39.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:39.241+0000 7f18637fe700 1 --2- 172.21.15.47:0/95967611 >> [v2:172.21.15.103:6816/3433905520,v1:172.21.15.103:6817/3433905520] conn(0x7f185c0ad4b0 0x7f185c0af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:39.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:39.241+0000 7f18637fe700 1 -- 172.21.15.47:0/95967611 --> [v2:172.21.15.103:6816/3433905520,v1:172.21.15.103:6817/3433905520] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1864015ec0 con 0x7f185c0ad4b0 2024-04-17T05:59:39.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:39.241+0000 7f18637fe700 1 -- 172.21.15.47:0/95967611 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f18640d0d20 con 0x7f1874101980 2024-04-17T05:59:39.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:39.241+0000 7f1871d9b700 1 --2- 172.21.15.47:0/95967611 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f185c0a3cd0 0x7f185c0a6190 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1868005950 tx=0x7f18680058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:59:39.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:39.241+0000 7f1872d9d700 1 --2- 172.21.15.47:0/95967611 >> [v2:172.21.15.103:6816/3433905520,v1:172.21.15.103:6817/3433905520] conn(0x7f185c0ad4b0 0x7f185c0af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:39.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:39.242+0000 7f1872d9d700 1 --2- 172.21.15.47:0/95967611 >> [v2:172.21.15.103:6816/3433905520,v1:172.21.15.103:6817/3433905520] conn(0x7f185c0ad4b0 0x7f185c0af8d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:59:39.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:39.243+0000 7f18637fe700 1 -- 172.21.15.47:0/95967611 <== osd.8 v2:172.21.15.103:6816/3433905520 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1864015ec0 con 0x7f185c0ad4b0 2024-04-17T05:59:39.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:39.259+0000 7f1878c3d700 1 -- 172.21.15.47:0/95967611 --> [v2:172.21.15.103:6816/3433905520,v1:172.21.15.103:6817/3433905520] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1874063560 con 0x7f185c0ad4b0 2024-04-17T05:59:39.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:39.260+0000 7f18637fe700 1 -- 172.21.15.47:0/95967611 <== osd.8 v2:172.21.15.103:6816/3433905520 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f1874063560 con 0x7f185c0ad4b0 2024-04-17T05:59:39.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:39.260+0000 7f1878c3d700 1 -- 172.21.15.47:0/95967611 >> [v2:172.21.15.103:6816/3433905520,v1:172.21.15.103:6817/3433905520] conn(0x7f185c0ad4b0 msgr2=0x7f185c0af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:39.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:39.260+0000 7f1878c3d700 1 --2- 172.21.15.47:0/95967611 >> [v2:172.21.15.103:6816/3433905520,v1:172.21.15.103:6817/3433905520] conn(0x7f185c0ad4b0 0x7f185c0af8d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:39.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:39.260+0000 7f1878c3d700 1 -- 172.21.15.47:0/95967611 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f185c0a3cd0 msgr2=0x7f185c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:39.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:39.260+0000 7f1878c3d700 1 --2- 172.21.15.47:0/95967611 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f185c0a3cd0 0x7f185c0a6190 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1868005950 tx=0x7f18680058e0 comp rx=0 tx=0).stop 2024-04-17T05:59:39.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:39.260+0000 7f1878c3d700 1 -- 172.21.15.47:0/95967611 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1874101980 msgr2=0x7f1874103500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:39.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:39.260+0000 7f1878c3d700 1 --2- 172.21.15.47:0/95967611 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1874101980 0x7f1874103500 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f186400d980 tx=0x7f186400dc90 comp rx=0 tx=0).stop 2024-04-17T05:59:39.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:39.260+0000 7f1872d9d700 1 -- 172.21.15.47:0/95967611 reap_dead start 2024-04-17T05:59:39.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:39.260+0000 7f1878c3d700 1 -- 172.21.15.47:0/95967611 shutdown_connections 2024-04-17T05:59:39.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:39.261+0000 7f1878c3d700 1 -- 172.21.15.47:0/95967611 >> 172.21.15.47:0/95967611 conn(0x7f1874078900 msgr2=0x7f187406b400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:59:39.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:39.261+0000 7f1878c3d700 1 -- 172.21.15.47:0/95967611 shutdown_connections 2024-04-17T05:59:39.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:39.261+0000 7f1878c3d700 1 -- 172.21.15.47:0/95967611 wait complete. 2024-04-17T05:59:39.987 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:39 smithi047 ceph-mon[32223]: pgmap v298: 97 pgs: 97 active+clean; 638 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-04-17T05:59:40.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:39 smithi103 ceph-mon[36243]: pgmap v298: 97 pgs: 97 active+clean; 638 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-04-17T05:59:40.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:39 smithi098 ceph-mon[36516]: pgmap v298: 97 pgs: 97 active+clean; 638 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-04-17T05:59:41.033 INFO:teuthology.orchestra.run.smithi047.stdout:257698037794 2024-04-17T05:59:41.033 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 565db2d8-fc7e-11ee-bc91-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-04-17T05:59:42.002 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:41 smithi047 ceph-mon[32223]: pgmap v299: 97 pgs: 97 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-04-17T05:59:42.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:41 smithi103 ceph-mon[36243]: pgmap v299: 97 pgs: 97 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-04-17T05:59:42.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:41 smithi098 ceph-mon[36516]: pgmap v299: 97 pgs: 97 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-04-17T05:59:43.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:43 smithi047 ceph-mon[32223]: pgmap v300: 97 pgs: 97 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-17T05:59:44.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:43 smithi103 ceph-mon[36243]: pgmap v300: 97 pgs: 97 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-17T05:59:44.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:43 smithi098 ceph-mon[36516]: pgmap v300: 97 pgs: 97 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-17T05:59:46.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:45 smithi103 ceph-mon[36243]: pgmap v301: 97 pgs: 97 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-17T05:59:46.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:45 smithi098 ceph-mon[36516]: pgmap v301: 97 pgs: 97 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-17T05:59:46.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:45 smithi047 ceph-mon[32223]: pgmap v301: 97 pgs: 97 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-17T05:59:47.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:47.404+0000 7f9b8985f700 1 -- 172.21.15.47:0/2207111376 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b84102aa0 msgr2=0x7f9b84102f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:47.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:47.404+0000 7f9b8985f700 1 --2- 172.21.15.47:0/2207111376 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b84102aa0 0x7f9b84102f20 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f9b6c009990 tx=0x7f9b6c009ca0 comp rx=0 tx=0).stop 2024-04-17T05:59:47.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:47.404+0000 7f9b8985f700 1 -- 172.21.15.47:0/2207111376 shutdown_connections 2024-04-17T05:59:47.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:47.404+0000 7f9b8985f700 1 --2- 172.21.15.47:0/2207111376 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9b84103460 0x7f9b8410f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:47.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:47.404+0000 7f9b8985f700 1 --2- 172.21.15.47:0/2207111376 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9b84108af0 0x7f9b84108ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:47.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:47.404+0000 7f9b8985f700 1 --2- 172.21.15.47:0/2207111376 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b84102aa0 0x7f9b84102f20 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:47.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:47.404+0000 7f9b8985f700 1 -- 172.21.15.47:0/2207111376 >> 172.21.15.47:0/2207111376 conn(0x7f9b84079c00 msgr2=0x7f9b8407a010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:59:47.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:47.404+0000 7f9b8985f700 1 -- 172.21.15.47:0/2207111376 shutdown_connections 2024-04-17T05:59:47.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:47.404+0000 7f9b8985f700 1 -- 172.21.15.47:0/2207111376 wait complete. 2024-04-17T05:59:47.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:47.405+0000 7f9b8985f700 1 Processor -- start 2024-04-17T05:59:47.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:47.405+0000 7f9b8985f700 1 -- start start 2024-04-17T05:59:47.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:47.405+0000 7f9b8985f700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9b84102aa0 0x7f9b841991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:47.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:47.405+0000 7f9b8985f700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9b84103460 0x7f9b84199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:47.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:47.405+0000 7f9b8985f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b84108af0 0x7f9b8419da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:47.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:47.405+0000 7f9b8985f700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f9b84112d60 con 0x7f9b84108af0 2024-04-17T05:59:47.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:47.405+0000 7f9b8985f700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9b84112be0 con 0x7f9b84103460 2024-04-17T05:59:47.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:47.405+0000 7f9b8985f700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9b84112ee0 con 0x7f9b84102aa0 2024-04-17T05:59:47.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:47.405+0000 7f9b837fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b84108af0 0x7f9b8419da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:47.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:47.405+0000 7f9b837fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b84108af0 0x7f9b8419da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47242/0 (socket says 172.21.15.47:47242) 2024-04-17T05:59:47.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:47.405+0000 7f9b7a7fc700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9b84103460 0x7f9b84199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:47.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:47.405+0000 7f9b837fe700 1 -- 172.21.15.47:0/4019214260 learned_addr learned my addr 172.21.15.47:0/4019214260 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:59:47.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:47.406+0000 7f9b82ffd700 1 --2- 172.21.15.47:0/4019214260 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9b84102aa0 0x7f9b841991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:47.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:47.406+0000 7f9b837fe700 1 -- 172.21.15.47:0/4019214260 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9b84102aa0 msgr2=0x7f9b841991d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:47.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:47.406+0000 7f9b837fe700 1 --2- 172.21.15.47:0/4019214260 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9b84102aa0 0x7f9b841991d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:47.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:47.406+0000 7f9b837fe700 1 -- 172.21.15.47:0/4019214260 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9b84103460 msgr2=0x7f9b84199710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:47.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:47.406+0000 7f9b837fe700 1 --2- 172.21.15.47:0/4019214260 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9b84103460 0x7f9b84199710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:47.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:47.406+0000 7f9b837fe700 1 -- 172.21.15.47:0/4019214260 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b8419e190 con 0x7f9b84108af0 2024-04-17T05:59:47.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:47.406+0000 7f9b82ffd700 1 --2- 172.21.15.47:0/4019214260 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9b84102aa0 0x7f9b841991d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T05:59:47.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:47.406+0000 7f9b837fe700 1 --2- 172.21.15.47:0/4019214260 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b84108af0 0x7f9b8419da50 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c00ab50 tx=0x7f9b7c00ae60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:59:47.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:47.406+0000 7f9b8885d700 1 -- 172.21.15.47:0/4019214260 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b7c017020 con 0x7f9b84108af0 2024-04-17T05:59:47.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:47.406+0000 7f9b8885d700 1 -- 172.21.15.47:0/4019214260 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b7c004bb0 con 0x7f9b84108af0 2024-04-17T05:59:47.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:47.406+0000 7f9b8985f700 1 -- 172.21.15.47:0/4019214260 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b8419e480 con 0x7f9b84108af0 2024-04-17T05:59:47.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:47.406+0000 7f9b8885d700 1 -- 172.21.15.47:0/4019214260 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b7c005310 con 0x7f9b84108af0 2024-04-17T05:59:47.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:47.406+0000 7f9b8985f700 1 -- 172.21.15.47:0/4019214260 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b841a5d00 con 0x7f9b84108af0 2024-04-17T05:59:47.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:47.407+0000 7f9b8985f700 1 -- 172.21.15.47:0/4019214260 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9b840631e0 con 0x7f9b84108af0 2024-04-17T05:59:47.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:47.408+0000 7f9b8885d700 1 -- 172.21.15.47:0/4019214260 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f9b7c00c620 con 0x7f9b84108af0 2024-04-17T05:59:47.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:47.409+0000 7f9b8885d700 1 --2- 172.21.15.47:0/4019214260 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f9b700a3d50 0x7f9b700a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:47.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:47.409+0000 7f9b82ffd700 1 --2- 172.21.15.47:0/4019214260 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f9b700a3d50 0x7f9b700a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:47.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:47.409+0000 7f9b8885d700 1 -- 172.21.15.47:0/4019214260 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f9b7c0d0410 con 0x7f9b84108af0 2024-04-17T05:59:47.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:47.409+0000 7f9b8885d700 1 --2- 172.21.15.47:0/4019214260 >> [v2:172.21.15.98:6816/4225498496,v1:172.21.15.98:6817/4225498496] conn(0x7f9b700ad530 0x7f9b700af950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:47.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:47.409+0000 7f9b8885d700 1 -- 172.21.15.47:0/4019214260 --> [v2:172.21.15.98:6816/4225498496,v1:172.21.15.98:6817/4225498496] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9b7c016e70 con 0x7f9b700ad530 2024-04-17T05:59:47.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:47.409+0000 7f9b8885d700 1 -- 172.21.15.47:0/4019214260 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f9b7c0d07c0 con 0x7f9b84108af0 2024-04-17T05:59:47.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:47.409+0000 7f9b82ffd700 1 --2- 172.21.15.47:0/4019214260 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f9b700a3d50 0x7f9b700a6210 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9b74005fd0 tx=0x7f9b7400a5e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:59:47.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:47.410+0000 7f9b7a7fc700 1 --2- 172.21.15.47:0/4019214260 >> [v2:172.21.15.98:6816/4225498496,v1:172.21.15.98:6817/4225498496] conn(0x7f9b700ad530 0x7f9b700af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:47.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:47.410+0000 7f9b7a7fc700 1 --2- 172.21.15.47:0/4019214260 >> [v2:172.21.15.98:6816/4225498496,v1:172.21.15.98:6817/4225498496] conn(0x7f9b700ad530 0x7f9b700af950 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:59:47.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:47.411+0000 7f9b8885d700 1 -- 172.21.15.47:0/4019214260 <== osd.7 v2:172.21.15.98:6816/4225498496 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9b7c016e70 con 0x7f9b700ad530 2024-04-17T05:59:47.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:47.427+0000 7f9b8985f700 1 -- 172.21.15.47:0/4019214260 --> [v2:172.21.15.98:6816/4225498496,v1:172.21.15.98:6817/4225498496] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9b84063610 con 0x7f9b700ad530 2024-04-17T05:59:47.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:47.428+0000 7f9b8885d700 1 -- 172.21.15.47:0/4019214260 <== osd.7 v2:172.21.15.98:6816/4225498496 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f9b84063610 con 0x7f9b700ad530 2024-04-17T05:59:47.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:47.429+0000 7f9b8985f700 1 -- 172.21.15.47:0/4019214260 >> [v2:172.21.15.98:6816/4225498496,v1:172.21.15.98:6817/4225498496] conn(0x7f9b700ad530 msgr2=0x7f9b700af950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:47.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:47.429+0000 7f9b8985f700 1 --2- 172.21.15.47:0/4019214260 >> [v2:172.21.15.98:6816/4225498496,v1:172.21.15.98:6817/4225498496] conn(0x7f9b700ad530 0x7f9b700af950 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:47.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:47.429+0000 7f9b8985f700 1 -- 172.21.15.47:0/4019214260 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f9b700a3d50 msgr2=0x7f9b700a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:47.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:47.429+0000 7f9b8985f700 1 --2- 172.21.15.47:0/4019214260 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f9b700a3d50 0x7f9b700a6210 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9b74005fd0 tx=0x7f9b7400a5e0 comp rx=0 tx=0).stop 2024-04-17T05:59:47.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:47.429+0000 7f9b8985f700 1 -- 172.21.15.47:0/4019214260 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b84108af0 msgr2=0x7f9b8419da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:47.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:47.429+0000 7f9b8985f700 1 --2- 172.21.15.47:0/4019214260 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b84108af0 0x7f9b8419da50 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c00ab50 tx=0x7f9b7c00ae60 comp rx=0 tx=0).stop 2024-04-17T05:59:47.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:47.429+0000 7f9b837fe700 1 -- 172.21.15.47:0/4019214260 reap_dead start 2024-04-17T05:59:47.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:47.429+0000 7f9b8985f700 1 -- 172.21.15.47:0/4019214260 shutdown_connections 2024-04-17T05:59:47.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:47.429+0000 7f9b8985f700 1 -- 172.21.15.47:0/4019214260 >> 172.21.15.47:0/4019214260 conn(0x7f9b84079c00 msgr2=0x7f9b84100a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:59:47.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:47.429+0000 7f9b8985f700 1 -- 172.21.15.47:0/4019214260 shutdown_connections 2024-04-17T05:59:47.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:47.429+0000 7f9b8985f700 1 -- 172.21.15.47:0/4019214260 wait complete. 2024-04-17T05:59:48.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:47 smithi103 ceph-mon[36243]: pgmap v302: 97 pgs: 97 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-17T05:59:48.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:47 smithi098 ceph-mon[36516]: pgmap v302: 97 pgs: 97 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-17T05:59:48.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:47 smithi047 ceph-mon[32223]: pgmap v302: 97 pgs: 97 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-17T05:59:48.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:48.264+0000 7fa947875700 1 -- 172.21.15.47:0/1295342624 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa9401079a0 msgr2=0x7fa940107d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:48.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:48.264+0000 7fa947875700 1 --2- 172.21.15.47:0/1295342624 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa9401079a0 0x7fa940107d80 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa93c009990 tx=0x7fa93c009ca0 comp rx=0 tx=0).stop 2024-04-17T05:59:48.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:48.264+0000 7fa947875700 1 -- 172.21.15.47:0/1295342624 shutdown_connections 2024-04-17T05:59:48.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:48.264+0000 7fa947875700 1 --2- 172.21.15.47:0/1295342624 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa940102310 0x7fa94010e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:48.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:48.264+0000 7fa947875700 1 --2- 172.21.15.47:0/1295342624 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa9401079a0 0x7fa940107d80 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:48.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:48.264+0000 7fa947875700 1 --2- 172.21.15.47:0/1295342624 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa940101950 0x7fa940101dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:48.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:48.264+0000 7fa947875700 1 -- 172.21.15.47:0/1295342624 >> 172.21.15.47:0/1295342624 conn(0x7fa940078900 msgr2=0x7fa940078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:59:48.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:48.265+0000 7fa947875700 1 -- 172.21.15.47:0/1295342624 shutdown_connections 2024-04-17T05:59:48.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:48.265+0000 7fa947875700 1 -- 172.21.15.47:0/1295342624 wait complete. 2024-04-17T05:59:48.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:48.265+0000 7fa947875700 1 Processor -- start 2024-04-17T05:59:48.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:48.265+0000 7fa947875700 1 -- start start 2024-04-17T05:59:48.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:48.265+0000 7fa947875700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa940101950 0x7fa9401991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:48.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:48.265+0000 7fa947875700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa940102310 0x7fa940199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:48.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:48.265+0000 7fa947875700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa9401079a0 0x7fa94019dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:48.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:48.265+0000 7fa947875700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa940112d60 con 0x7fa940102310 2024-04-17T05:59:48.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:48.265+0000 7fa947875700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa940112be0 con 0x7fa940101950 2024-04-17T05:59:48.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:48.265+0000 7fa947875700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa940112ee0 con 0x7fa9401079a0 2024-04-17T05:59:48.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:48.265+0000 7fa944e10700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa940102310 0x7fa940199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:48.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:48.266+0000 7fa944e10700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa940102310 0x7fa940199730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47254/0 (socket says 172.21.15.47:47254) 2024-04-17T05:59:48.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:48.266+0000 7fa944e10700 1 -- 172.21.15.47:0/3063170565 learned_addr learned my addr 172.21.15.47:0/3063170565 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:59:48.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:48.266+0000 7fa945611700 1 --2- 172.21.15.47:0/3063170565 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa940101950 0x7fa9401991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:48.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:48.266+0000 7fa945e12700 1 --2- 172.21.15.47:0/3063170565 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa9401079a0 0x7fa94019dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:48.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:48.266+0000 7fa944e10700 1 -- 172.21.15.47:0/3063170565 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa9401079a0 msgr2=0x7fa94019dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:48.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:48.266+0000 7fa944e10700 1 --2- 172.21.15.47:0/3063170565 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa9401079a0 0x7fa94019dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:48.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:48.266+0000 7fa944e10700 1 -- 172.21.15.47:0/3063170565 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa940101950 msgr2=0x7fa9401991f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:48.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:48.266+0000 7fa944e10700 1 --2- 172.21.15.47:0/3063170565 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa940101950 0x7fa9401991f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:48.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:48.266+0000 7fa944e10700 1 -- 172.21.15.47:0/3063170565 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa94019e200 con 0x7fa940102310 2024-04-17T05:59:48.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:48.266+0000 7fa944e10700 1 --2- 172.21.15.47:0/3063170565 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa940102310 0x7fa940199730 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa93000d950 tx=0x7fa93000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:59:48.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:48.266+0000 7fa9367fc700 1 -- 172.21.15.47:0/3063170565 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa930016070 con 0x7fa940102310 2024-04-17T05:59:48.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:48.266+0000 7fa9367fc700 1 -- 172.21.15.47:0/3063170565 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa93000fd70 con 0x7fa940102310 2024-04-17T05:59:48.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:48.266+0000 7fa9367fc700 1 -- 172.21.15.47:0/3063170565 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa93000e610 con 0x7fa940102310 2024-04-17T05:59:48.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:48.266+0000 7fa947875700 1 -- 172.21.15.47:0/3063170565 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa94019e4f0 con 0x7fa940102310 2024-04-17T05:59:48.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:48.266+0000 7fa947875700 1 -- 172.21.15.47:0/3063170565 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9401a5d70 con 0x7fa940102310 2024-04-17T05:59:48.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:48.267+0000 7fa947875700 1 -- 172.21.15.47:0/3063170565 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa940063120 con 0x7fa940102310 2024-04-17T05:59:48.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:48.268+0000 7fa9367fc700 1 -- 172.21.15.47:0/3063170565 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fa930019070 con 0x7fa940102310 2024-04-17T05:59:48.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:48.269+0000 7fa9367fc700 1 --2- 172.21.15.47:0/3063170565 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fa92c0a3cd0 0x7fa92c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:48.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:48.269+0000 7fa945611700 1 --2- 172.21.15.47:0/3063170565 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fa92c0a3cd0 0x7fa92c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:48.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:48.269+0000 7fa9367fc700 1 -- 172.21.15.47:0/3063170565 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fa9300d0380 con 0x7fa940102310 2024-04-17T05:59:48.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:48.269+0000 7fa9367fc700 1 --2- 172.21.15.47:0/3063170565 >> [v2:172.21.15.103:6824/2639831013,v1:172.21.15.103:6825/2639831013] conn(0x7fa92c0ad4b0 0x7fa92c0af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:48.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:48.269+0000 7fa9367fc700 1 -- 172.21.15.47:0/3063170565 --> [v2:172.21.15.103:6824/2639831013,v1:172.21.15.103:6825/2639831013] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa930013ec0 con 0x7fa92c0ad4b0 2024-04-17T05:59:48.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:48.269+0000 7fa9367fc700 1 -- 172.21.15.47:0/3063170565 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7fa9300d0860 con 0x7fa940102310 2024-04-17T05:59:48.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:48.269+0000 7fa945611700 1 --2- 172.21.15.47:0/3063170565 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fa92c0a3cd0 0x7fa92c0a6190 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa93c0098e0 tx=0x7fa93c005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:59:48.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:48.270+0000 7fa945e12700 1 --2- 172.21.15.47:0/3063170565 >> [v2:172.21.15.103:6824/2639831013,v1:172.21.15.103:6825/2639831013] conn(0x7fa92c0ad4b0 0x7fa92c0af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:48.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:48.270+0000 7fa945e12700 1 --2- 172.21.15.47:0/3063170565 >> [v2:172.21.15.103:6824/2639831013,v1:172.21.15.103:6825/2639831013] conn(0x7fa92c0ad4b0 0x7fa92c0af8d0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:59:48.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:48.271+0000 7fa9367fc700 1 -- 172.21.15.47:0/3063170565 <== osd.11 v2:172.21.15.103:6824/2639831013 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa930013ec0 con 0x7fa92c0ad4b0 2024-04-17T05:59:48.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:48.289+0000 7fa947875700 1 -- 172.21.15.47:0/3063170565 --> [v2:172.21.15.103:6824/2639831013,v1:172.21.15.103:6825/2639831013] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa9400635a0 con 0x7fa92c0ad4b0 2024-04-17T05:59:48.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:48.290+0000 7fa9367fc700 1 -- 172.21.15.47:0/3063170565 <== osd.11 v2:172.21.15.103:6824/2639831013 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fa9400635a0 con 0x7fa92c0ad4b0 2024-04-17T05:59:48.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:48.290+0000 7fa947875700 1 -- 172.21.15.47:0/3063170565 >> [v2:172.21.15.103:6824/2639831013,v1:172.21.15.103:6825/2639831013] conn(0x7fa92c0ad4b0 msgr2=0x7fa92c0af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:48.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:48.290+0000 7fa947875700 1 --2- 172.21.15.47:0/3063170565 >> [v2:172.21.15.103:6824/2639831013,v1:172.21.15.103:6825/2639831013] conn(0x7fa92c0ad4b0 0x7fa92c0af8d0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:48.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:48.290+0000 7fa947875700 1 -- 172.21.15.47:0/3063170565 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fa92c0a3cd0 msgr2=0x7fa92c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:48.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:48.290+0000 7fa947875700 1 --2- 172.21.15.47:0/3063170565 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fa92c0a3cd0 0x7fa92c0a6190 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa93c0098e0 tx=0x7fa93c005960 comp rx=0 tx=0).stop 2024-04-17T05:59:48.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:48.290+0000 7fa947875700 1 -- 172.21.15.47:0/3063170565 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa940102310 msgr2=0x7fa940199730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:48.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:48.290+0000 7fa947875700 1 --2- 172.21.15.47:0/3063170565 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa940102310 0x7fa940199730 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa93000d950 tx=0x7fa93000dc60 comp rx=0 tx=0).stop 2024-04-17T05:59:48.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:48.290+0000 7fa945e12700 1 -- 172.21.15.47:0/3063170565 reap_dead start 2024-04-17T05:59:48.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:48.290+0000 7fa947875700 1 -- 172.21.15.47:0/3063170565 shutdown_connections 2024-04-17T05:59:48.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:48.290+0000 7fa947875700 1 -- 172.21.15.47:0/3063170565 >> 172.21.15.47:0/3063170565 conn(0x7fa940078900 msgr2=0x7fa9401006b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:59:48.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:48.290+0000 7fa947875700 1 -- 172.21.15.47:0/3063170565 shutdown_connections 2024-04-17T05:59:48.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:48.290+0000 7fa947875700 1 -- 172.21.15.47:0/3063170565 wait complete. 2024-04-17T05:59:50.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:49 smithi103 ceph-mon[36243]: pgmap v303: 97 pgs: 97 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-17T05:59:50.094 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:49 smithi047 ceph-mon[32223]: pgmap v303: 97 pgs: 97 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-17T05:59:50.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:49 smithi098 ceph-mon[36516]: pgmap v303: 97 pgs: 97 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-17T05:59:50.673 INFO:teuthology.orchestra.run.smithi047.stdout:347892350996 2024-04-17T05:59:50.674 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 565db2d8-fc7e-11ee-bc91-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-04-17T05:59:51.876 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:51 smithi047 ceph-mon[32223]: pgmap v304: 97 pgs: 97 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-17T05:59:52.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:51 smithi103 ceph-mon[36243]: pgmap v304: 97 pgs: 97 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-17T05:59:52.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:51 smithi098 ceph-mon[36516]: pgmap v304: 97 pgs: 97 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-17T05:59:52.717 INFO:teuthology.orchestra.run.smithi047.stdout:223338299431 2024-04-17T05:59:52.717 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 565db2d8-fc7e-11ee-bc91-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-04-17T05:59:53.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:53.028+0000 7efe2c149700 1 -- 172.21.15.47:0/351489199 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efe24101d30 msgr2=0x7efe241021b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:53.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:53.028+0000 7efe2c149700 1 --2- 172.21.15.47:0/351489199 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efe24101d30 0x7efe241021b0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7efe14009990 tx=0x7efe14009ca0 comp rx=0 tx=0).stop 2024-04-17T05:59:53.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:53.028+0000 7efe2c149700 1 -- 172.21.15.47:0/351489199 shutdown_connections 2024-04-17T05:59:53.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:53.028+0000 7efe2c149700 1 --2- 172.21.15.47:0/351489199 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efe241026f0 0x7efe2410e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:53.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:53.028+0000 7efe2c149700 1 --2- 172.21.15.47:0/351489199 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efe241069a0 0x7efe24106d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:53.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:53.028+0000 7efe2c149700 1 --2- 172.21.15.47:0/351489199 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efe24101d30 0x7efe241021b0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:53.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:53.028+0000 7efe2c149700 1 -- 172.21.15.47:0/351489199 >> 172.21.15.47:0/351489199 conn(0x7efe24078900 msgr2=0x7efe24078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:59:53.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:53.029+0000 7efe2c149700 1 -- 172.21.15.47:0/351489199 shutdown_connections 2024-04-17T05:59:53.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:53.029+0000 7efe2c149700 1 -- 172.21.15.47:0/351489199 wait complete. 2024-04-17T05:59:53.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:53.029+0000 7efe2c149700 1 Processor -- start 2024-04-17T05:59:53.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:53.029+0000 7efe2c149700 1 -- start start 2024-04-17T05:59:53.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:53.029+0000 7efe2c149700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efe24101d30 0x7efe24199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:53.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:53.029+0000 7efe2c149700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efe241026f0 0x7efe24199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:53.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:53.029+0000 7efe2c149700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efe241069a0 0x7efe2419da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:53.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:53.029+0000 7efe2c149700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7efe24111c90 con 0x7efe241026f0 2024-04-17T05:59:53.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:53.029+0000 7efe2c149700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7efe24111b10 con 0x7efe241069a0 2024-04-17T05:59:53.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:53.029+0000 7efe2c149700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7efe24111e10 con 0x7efe24101d30 2024-04-17T05:59:53.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:53.030+0000 7efe296e4700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efe241026f0 0x7efe24199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:53.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:53.030+0000 7efe296e4700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efe241026f0 0x7efe24199670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47306/0 (socket says 172.21.15.47:47306) 2024-04-17T05:59:53.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:53.030+0000 7efe296e4700 1 -- 172.21.15.47:0/1364940534 learned_addr learned my addr 172.21.15.47:0/1364940534 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:59:53.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:53.030+0000 7efe29ee5700 1 --2- 172.21.15.47:0/1364940534 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efe24101d30 0x7efe24199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:53.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:53.030+0000 7efe2a6e6700 1 --2- 172.21.15.47:0/1364940534 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efe241069a0 0x7efe2419da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:53.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:53.030+0000 7efe296e4700 1 -- 172.21.15.47:0/1364940534 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efe24101d30 msgr2=0x7efe24199130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:53.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:53.030+0000 7efe296e4700 1 --2- 172.21.15.47:0/1364940534 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efe24101d30 0x7efe24199130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:53.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:53.030+0000 7efe296e4700 1 -- 172.21.15.47:0/1364940534 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efe241069a0 msgr2=0x7efe2419da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:53.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:53.030+0000 7efe296e4700 1 --2- 172.21.15.47:0/1364940534 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efe241069a0 0x7efe2419da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:53.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:53.030+0000 7efe296e4700 1 -- 172.21.15.47:0/1364940534 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe2419e140 con 0x7efe241026f0 2024-04-17T05:59:53.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:53.030+0000 7efe29ee5700 1 --2- 172.21.15.47:0/1364940534 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efe24101d30 0x7efe24199130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T05:59:53.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:53.030+0000 7efe2a6e6700 1 --2- 172.21.15.47:0/1364940534 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efe241069a0 0x7efe2419da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T05:59:53.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:53.030+0000 7efe296e4700 1 --2- 172.21.15.47:0/1364940534 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efe241026f0 0x7efe24199670 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7efe14005910 tx=0x7efe14009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:59:53.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:53.030+0000 7efe1affd700 1 -- 172.21.15.47:0/1364940534 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe14027070 con 0x7efe241026f0 2024-04-17T05:59:53.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:53.030+0000 7efe2c149700 1 -- 172.21.15.47:0/1364940534 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe2419e3d0 con 0x7efe241026f0 2024-04-17T05:59:53.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:53.030+0000 7efe1affd700 1 -- 172.21.15.47:0/1364940534 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe14004050 con 0x7efe241026f0 2024-04-17T05:59:53.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:53.030+0000 7efe1affd700 1 -- 172.21.15.47:0/1364940534 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe14022e30 con 0x7efe241026f0 2024-04-17T05:59:53.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:53.030+0000 7efe2c149700 1 -- 172.21.15.47:0/1364940534 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe241a5cb0 con 0x7efe241026f0 2024-04-17T05:59:53.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:53.031+0000 7efe2c149700 1 -- 172.21.15.47:0/1364940534 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7efe240631a0 con 0x7efe241026f0 2024-04-17T05:59:53.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:53.032+0000 7efe1affd700 1 -- 172.21.15.47:0/1364940534 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7efe1402a070 con 0x7efe241026f0 2024-04-17T05:59:53.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:53.033+0000 7efe1affd700 1 --2- 172.21.15.47:0/1364940534 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7efe100a3cd0 0x7efe100a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:53.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:53.033+0000 7efe29ee5700 1 --2- 172.21.15.47:0/1364940534 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7efe100a3cd0 0x7efe100a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:53.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:53.033+0000 7efe1affd700 1 -- 172.21.15.47:0/1364940534 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7efe140d1660 con 0x7efe241026f0 2024-04-17T05:59:53.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:53.033+0000 7efe1affd700 1 --2- 172.21.15.47:0/1364940534 >> [v2:172.21.15.47:6818/2517529668,v1:172.21.15.47:6819/2517529668] conn(0x7efe100ad4b0 0x7efe100af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:53.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:53.033+0000 7efe1affd700 1 -- 172.21.15.47:0/1364940534 --> [v2:172.21.15.47:6818/2517529668,v1:172.21.15.47:6819/2517529668] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7efe100aff80 con 0x7efe100ad4b0 2024-04-17T05:59:53.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:53.033+0000 7efe1affd700 1 -- 172.21.15.47:0/1364940534 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7efe140d1a10 con 0x7efe241026f0 2024-04-17T05:59:53.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:53.034+0000 7efe29ee5700 1 --2- 172.21.15.47:0/1364940534 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7efe100a3cd0 0x7efe100a6190 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7efe20005fd0 tx=0x7efe20005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:59:53.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:53.034+0000 7efe2a6e6700 1 --2- 172.21.15.47:0/1364940534 >> [v2:172.21.15.47:6818/2517529668,v1:172.21.15.47:6819/2517529668] conn(0x7efe100ad4b0 0x7efe100af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:53.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:53.034+0000 7efe2a6e6700 1 --2- 172.21.15.47:0/1364940534 >> [v2:172.21.15.47:6818/2517529668,v1:172.21.15.47:6819/2517529668] conn(0x7efe100ad4b0 0x7efe100af8d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:59:53.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:53.035+0000 7efe1affd700 1 -- 172.21.15.47:0/1364940534 <== osd.6 v2:172.21.15.47:6818/2517529668 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7efe100aff80 con 0x7efe100ad4b0 2024-04-17T05:59:53.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:53.051+0000 7efe2c149700 1 -- 172.21.15.47:0/1364940534 --> [v2:172.21.15.47:6818/2517529668,v1:172.21.15.47:6819/2517529668] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7efe240635d0 con 0x7efe100ad4b0 2024-04-17T05:59:53.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:53.051+0000 7efe1affd700 1 -- 172.21.15.47:0/1364940534 <== osd.6 v2:172.21.15.47:6818/2517529668 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7efe240635d0 con 0x7efe100ad4b0 2024-04-17T05:59:53.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:53.051+0000 7efe2c149700 1 -- 172.21.15.47:0/1364940534 >> [v2:172.21.15.47:6818/2517529668,v1:172.21.15.47:6819/2517529668] conn(0x7efe100ad4b0 msgr2=0x7efe100af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:53.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:53.051+0000 7efe2c149700 1 --2- 172.21.15.47:0/1364940534 >> [v2:172.21.15.47:6818/2517529668,v1:172.21.15.47:6819/2517529668] conn(0x7efe100ad4b0 0x7efe100af8d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:53.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:53.052+0000 7efe2c149700 1 -- 172.21.15.47:0/1364940534 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7efe100a3cd0 msgr2=0x7efe100a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:53.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:53.052+0000 7efe2c149700 1 --2- 172.21.15.47:0/1364940534 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7efe100a3cd0 0x7efe100a6190 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7efe20005fd0 tx=0x7efe20005e30 comp rx=0 tx=0).stop 2024-04-17T05:59:53.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:53.052+0000 7efe2c149700 1 -- 172.21.15.47:0/1364940534 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efe241026f0 msgr2=0x7efe24199670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:53.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:53.052+0000 7efe2c149700 1 --2- 172.21.15.47:0/1364940534 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efe241026f0 0x7efe24199670 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7efe14005910 tx=0x7efe14009f90 comp rx=0 tx=0).stop 2024-04-17T05:59:53.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:53.052+0000 7efe2a6e6700 1 -- 172.21.15.47:0/1364940534 reap_dead start 2024-04-17T05:59:53.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:53.052+0000 7efe2c149700 1 -- 172.21.15.47:0/1364940534 shutdown_connections 2024-04-17T05:59:53.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:53.052+0000 7efe2c149700 1 -- 172.21.15.47:0/1364940534 >> 172.21.15.47:0/1364940534 conn(0x7efe24078900 msgr2=0x7efe2406b360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:59:53.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:53.052+0000 7efe2c149700 1 -- 172.21.15.47:0/1364940534 shutdown_connections 2024-04-17T05:59:53.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:53.052+0000 7efe2c149700 1 -- 172.21.15.47:0/1364940534 wait complete. 2024-04-17T05:59:53.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:52.170+0000 7f3b5a834700 1 -- 172.21.15.47:0/1844215322 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b54101db0 msgr2=0x7f3b5410f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:53.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:52.170+0000 7f3b5a834700 1 --2- 172.21.15.47:0/1844215322 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b54101db0 0x7f3b5410f820 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c00b0a0 tx=0x7f3b4c00b3b0 comp rx=0 tx=0).stop 2024-04-17T05:59:53.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:52.170+0000 7f3b5a834700 1 -- 172.21.15.47:0/1844215322 shutdown_connections 2024-04-17T05:59:53.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:52.170+0000 7f3b5a834700 1 --2- 172.21.15.47:0/1844215322 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b54101db0 0x7f3b5410f820 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:53.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:52.170+0000 7f3b5a834700 1 --2- 172.21.15.47:0/1844215322 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3b54107b10 0x7f3b54107ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:53.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:52.170+0000 7f3b5a834700 1 --2- 172.21.15.47:0/1844215322 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3b541013f0 0x7f3b54101870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:53.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:52.170+0000 7f3b5a834700 1 -- 172.21.15.47:0/1844215322 >> 172.21.15.47:0/1844215322 conn(0x7f3b54076c70 msgr2=0x7f3b54077080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:59:53.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:52.170+0000 7f3b5a834700 1 -- 172.21.15.47:0/1844215322 shutdown_connections 2024-04-17T05:59:53.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:52.170+0000 7f3b5a834700 1 -- 172.21.15.47:0/1844215322 wait complete. 2024-04-17T05:59:53.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:52.170+0000 7f3b5a834700 1 Processor -- start 2024-04-17T05:59:53.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:52.170+0000 7f3b5a834700 1 -- start start 2024-04-17T05:59:53.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:52.171+0000 7f3b5a834700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3b541013f0 0x7f3b541991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:53.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:52.171+0000 7f3b5a834700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3b54101db0 0x7f3b541996f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:53.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:52.171+0000 7f3b5a834700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b54107b10 0x7f3b5419da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:53.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:52.171+0000 7f3b5a834700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f3b54112e40 con 0x7f3b54107b10 2024-04-17T05:59:53.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:52.171+0000 7f3b5a834700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3b54112cc0 con 0x7f3b54101db0 2024-04-17T05:59:53.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:52.171+0000 7f3b5a834700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3b54112fc0 con 0x7f3b541013f0 2024-04-17T05:59:53.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:52.171+0000 7f3b58dd1700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b54107b10 0x7f3b5419da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:53.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:52.171+0000 7f3b58dd1700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b54107b10 0x7f3b5419da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47278/0 (socket says 172.21.15.47:47278) 2024-04-17T05:59:53.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:52.171+0000 7f3b58dd1700 1 -- 172.21.15.47:0/2192094898 learned_addr learned my addr 172.21.15.47:0/2192094898 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:59:53.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:52.171+0000 7f3b53fff700 1 --2- 172.21.15.47:0/2192094898 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3b541013f0 0x7f3b541991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:53.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:52.171+0000 7f3b537fe700 1 --2- 172.21.15.47:0/2192094898 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3b54101db0 0x7f3b541996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:53.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:52.171+0000 7f3b58dd1700 1 -- 172.21.15.47:0/2192094898 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3b541013f0 msgr2=0x7f3b541991b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:53.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:52.171+0000 7f3b58dd1700 1 --2- 172.21.15.47:0/2192094898 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3b541013f0 0x7f3b541991b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:53.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:52.171+0000 7f3b58dd1700 1 -- 172.21.15.47:0/2192094898 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3b54101db0 msgr2=0x7f3b541996f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:53.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:52.171+0000 7f3b58dd1700 1 --2- 172.21.15.47:0/2192094898 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3b54101db0 0x7f3b541996f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:53.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:52.171+0000 7f3b58dd1700 1 -- 172.21.15.47:0/2192094898 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b5419e1c0 con 0x7f3b54107b10 2024-04-17T05:59:53.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:52.171+0000 7f3b58dd1700 1 --2- 172.21.15.47:0/2192094898 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b54107b10 0x7f3b5419da80 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c00d010 tx=0x7f3b4c004d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:59:53.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:52.172+0000 7f3b517fa700 1 -- 172.21.15.47:0/2192094898 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b4c021070 con 0x7f3b54107b10 2024-04-17T05:59:53.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:52.172+0000 7f3b517fa700 1 -- 172.21.15.47:0/2192094898 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b4c007c00 con 0x7f3b54107b10 2024-04-17T05:59:53.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:52.172+0000 7f3b517fa700 1 -- 172.21.15.47:0/2192094898 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b4c01d450 con 0x7f3b54107b10 2024-04-17T05:59:53.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:52.172+0000 7f3b5a834700 1 -- 172.21.15.47:0/2192094898 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b5419e450 con 0x7f3b54107b10 2024-04-17T05:59:53.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:52.172+0000 7f3b5a834700 1 -- 172.21.15.47:0/2192094898 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b541a5d30 con 0x7f3b54107b10 2024-04-17T05:59:53.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:52.172+0000 7f3b5a834700 1 -- 172.21.15.47:0/2192094898 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3b541007c0 con 0x7f3b54107b10 2024-04-17T05:59:53.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:52.174+0000 7f3b517fa700 1 -- 172.21.15.47:0/2192094898 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f3b4c004000 con 0x7f3b54107b10 2024-04-17T05:59:53.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:52.174+0000 7f3b517fa700 1 --2- 172.21.15.47:0/2192094898 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f3b3c0a3cd0 0x7f3b3c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:53.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:52.175+0000 7f3b53fff700 1 --2- 172.21.15.47:0/2192094898 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f3b3c0a3cd0 0x7f3b3c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:53.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:52.175+0000 7f3b517fa700 1 -- 172.21.15.47:0/2192094898 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f3b4c00f020 con 0x7f3b54107b10 2024-04-17T05:59:53.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:52.175+0000 7f3b517fa700 1 --2- 172.21.15.47:0/2192094898 >> [v2:172.21.15.47:6802/1290127544,v1:172.21.15.47:6803/1290127544] conn(0x7f3b3c0ad4b0 0x7f3b3c0af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:53.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:52.175+0000 7f3b517fa700 1 -- 172.21.15.47:0/2192094898 --> [v2:172.21.15.47:6802/1290127544,v1:172.21.15.47:6803/1290127544] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3b3c0aff80 con 0x7f3b3c0ad4b0 2024-04-17T05:59:53.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:52.175+0000 7f3b517fa700 1 -- 172.21.15.47:0/2192094898 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f3b4c0d1fc0 con 0x7f3b54107b10 2024-04-17T05:59:53.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:52.175+0000 7f3b537fe700 1 --2- 172.21.15.47:0/2192094898 >> [v2:172.21.15.47:6802/1290127544,v1:172.21.15.47:6803/1290127544] conn(0x7f3b3c0ad4b0 0x7f3b3c0af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:53.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:52.175+0000 7f3b53fff700 1 --2- 172.21.15.47:0/2192094898 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f3b3c0a3cd0 0x7f3b3c0a6190 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3b540781f0 tx=0x7f3b4400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:59:53.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:52.175+0000 7f3b537fe700 1 --2- 172.21.15.47:0/2192094898 >> [v2:172.21.15.47:6802/1290127544,v1:172.21.15.47:6803/1290127544] conn(0x7f3b3c0ad4b0 0x7f3b3c0af8d0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:59:53.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:52.176+0000 7f3b517fa700 1 -- 172.21.15.47:0/2192094898 <== osd.0 v2:172.21.15.47:6802/1290127544 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3b3c0aff80 con 0x7f3b3c0ad4b0 2024-04-17T05:59:53.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:52.192+0000 7f3b5a834700 1 -- 172.21.15.47:0/2192094898 --> [v2:172.21.15.47:6802/1290127544,v1:172.21.15.47:6803/1290127544] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3b5406aca0 con 0x7f3b3c0ad4b0 2024-04-17T05:59:53.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:52.193+0000 7f3b517fa700 1 -- 172.21.15.47:0/2192094898 <== osd.0 v2:172.21.15.47:6802/1290127544 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f3b5406aca0 con 0x7f3b3c0ad4b0 2024-04-17T05:59:53.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:52.193+0000 7f3b5a834700 1 -- 172.21.15.47:0/2192094898 >> [v2:172.21.15.47:6802/1290127544,v1:172.21.15.47:6803/1290127544] conn(0x7f3b3c0ad4b0 msgr2=0x7f3b3c0af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:53.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:52.193+0000 7f3b5a834700 1 --2- 172.21.15.47:0/2192094898 >> [v2:172.21.15.47:6802/1290127544,v1:172.21.15.47:6803/1290127544] conn(0x7f3b3c0ad4b0 0x7f3b3c0af8d0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:53.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:52.193+0000 7f3b5a834700 1 -- 172.21.15.47:0/2192094898 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f3b3c0a3cd0 msgr2=0x7f3b3c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:53.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:52.193+0000 7f3b5a834700 1 --2- 172.21.15.47:0/2192094898 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f3b3c0a3cd0 0x7f3b3c0a6190 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3b540781f0 tx=0x7f3b4400b410 comp rx=0 tx=0).stop 2024-04-17T05:59:53.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:52.193+0000 7f3b5a834700 1 -- 172.21.15.47:0/2192094898 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b54107b10 msgr2=0x7f3b5419da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:53.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:52.193+0000 7f3b5a834700 1 --2- 172.21.15.47:0/2192094898 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b54107b10 0x7f3b5419da80 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c00d010 tx=0x7f3b4c004d50 comp rx=0 tx=0).stop 2024-04-17T05:59:53.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:52.193+0000 7f3b58dd1700 1 -- 172.21.15.47:0/2192094898 reap_dead start 2024-04-17T05:59:53.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:52.193+0000 7f3b5a834700 1 -- 172.21.15.47:0/2192094898 shutdown_connections 2024-04-17T05:59:53.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:52.193+0000 7f3b5a834700 1 -- 172.21.15.47:0/2192094898 >> 172.21.15.47:0/2192094898 conn(0x7f3b54076c70 msgr2=0x7f3b540ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:59:53.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:52.193+0000 7f3b5a834700 1 -- 172.21.15.47:0/2192094898 shutdown_connections 2024-04-17T05:59:53.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:52.193+0000 7f3b5a834700 1 -- 172.21.15.47:0/2192094898 wait complete. 2024-04-17T05:59:54.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:53 smithi103 ceph-mon[36243]: pgmap v305: 97 pgs: 97 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-04-17T05:59:54.085 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:53 smithi047 ceph-mon[32223]: pgmap v305: 97 pgs: 97 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-04-17T05:59:54.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:53 smithi098 ceph-mon[36516]: pgmap v305: 97 pgs: 97 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-04-17T05:59:54.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:54.195+0000 7fd89b041700 1 -- 172.21.15.47:0/475757212 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd894101940 msgr2=0x7fd894101dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:54.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:54.195+0000 7fd89b041700 1 --2- 172.21.15.47:0/475757212 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd894101940 0x7fd894101dc0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd880009990 tx=0x7fd880009ca0 comp rx=0 tx=0).stop 2024-04-17T05:59:54.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:54.195+0000 7fd89b041700 1 -- 172.21.15.47:0/475757212 shutdown_connections 2024-04-17T05:59:54.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:54.195+0000 7fd89b041700 1 --2- 172.21.15.47:0/475757212 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd894102300 0x7fd89410e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:54.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:54.195+0000 7fd89b041700 1 --2- 172.21.15.47:0/475757212 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd894107990 0x7fd894107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:54.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:54.195+0000 7fd89b041700 1 --2- 172.21.15.47:0/475757212 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd894101940 0x7fd894101dc0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:54.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:54.195+0000 7fd89b041700 1 -- 172.21.15.47:0/475757212 >> 172.21.15.47:0/475757212 conn(0x7fd8940788c0 msgr2=0x7fd894078cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:59:54.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:54.195+0000 7fd89b041700 1 -- 172.21.15.47:0/475757212 shutdown_connections 2024-04-17T05:59:54.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:54.195+0000 7fd89b041700 1 -- 172.21.15.47:0/475757212 wait complete. 2024-04-17T05:59:54.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:54.196+0000 7fd89b041700 1 Processor -- start 2024-04-17T05:59:54.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:54.196+0000 7fd89b041700 1 -- start start 2024-04-17T05:59:54.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:54.196+0000 7fd89b041700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd894101940 0x7fd8941991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:54.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:54.196+0000 7fd89b041700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd894102300 0x7fd894199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:54.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:54.196+0000 7fd89b041700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd894107990 0x7fd89419da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:54.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:54.196+0000 7fd89b041700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fd894111c40 con 0x7fd894101940 2024-04-17T05:59:54.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:54.196+0000 7fd89b041700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd894111ac0 con 0x7fd894107990 2024-04-17T05:59:54.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:54.196+0000 7fd89b041700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd894111dc0 con 0x7fd894102300 2024-04-17T05:59:54.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:54.196+0000 7fd898ddd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd894101940 0x7fd8941991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:54.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:54.196+0000 7fd898ddd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd894101940 0x7fd8941991c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47332/0 (socket says 172.21.15.47:47332) 2024-04-17T05:59:54.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:54.196+0000 7fd898ddd700 1 -- 172.21.15.47:0/1464249095 learned_addr learned my addr 172.21.15.47:0/1464249095 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:59:54.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:54.196+0000 7fd893fff700 1 --2- 172.21.15.47:0/1464249095 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd894102300 0x7fd894199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:54.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:54.197+0000 7fd8995de700 1 --2- 172.21.15.47:0/1464249095 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd894107990 0x7fd89419da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:54.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:54.197+0000 7fd898ddd700 1 -- 172.21.15.47:0/1464249095 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd894102300 msgr2=0x7fd894199700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:54.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:54.197+0000 7fd898ddd700 1 --2- 172.21.15.47:0/1464249095 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd894102300 0x7fd894199700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:54.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:54.197+0000 7fd898ddd700 1 -- 172.21.15.47:0/1464249095 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd894107990 msgr2=0x7fd89419da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:54.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:54.197+0000 7fd898ddd700 1 --2- 172.21.15.47:0/1464249095 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd894107990 0x7fd89419da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:54.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:54.197+0000 7fd898ddd700 1 -- 172.21.15.47:0/1464249095 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd89419e1d0 con 0x7fd894101940 2024-04-17T05:59:54.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:54.197+0000 7fd898ddd700 1 --2- 172.21.15.47:0/1464249095 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd894101940 0x7fd8941991c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd88800c9b0 tx=0x7fd88800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:59:54.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:54.197+0000 7fd891ffb700 1 -- 172.21.15.47:0/1464249095 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd888007b00 con 0x7fd894101940 2024-04-17T05:59:54.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:54.197+0000 7fd89b041700 1 -- 172.21.15.47:0/1464249095 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd89419e4c0 con 0x7fd894101940 2024-04-17T05:59:54.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:54.197+0000 7fd891ffb700 1 -- 172.21.15.47:0/1464249095 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd888007ca0 con 0x7fd894101940 2024-04-17T05:59:54.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:54.197+0000 7fd891ffb700 1 -- 172.21.15.47:0/1464249095 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd888018730 con 0x7fd894101940 2024-04-17T05:59:54.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:54.197+0000 7fd89b041700 1 -- 172.21.15.47:0/1464249095 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8941a5d40 con 0x7fd894101940 2024-04-17T05:59:54.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:54.198+0000 7fd89b041700 1 -- 172.21.15.47:0/1464249095 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd894063160 con 0x7fd894101940 2024-04-17T05:59:54.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:54.199+0000 7fd891ffb700 1 -- 172.21.15.47:0/1464249095 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fd8880188d0 con 0x7fd894101940 2024-04-17T05:59:54.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:54.200+0000 7fd891ffb700 1 --2- 172.21.15.47:0/1464249095 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fd8840a3d50 0x7fd8840a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:54.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:54.200+0000 7fd893fff700 1 --2- 172.21.15.47:0/1464249095 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fd8840a3d50 0x7fd8840a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:54.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:54.200+0000 7fd891ffb700 1 -- 172.21.15.47:0/1464249095 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fd8880d0720 con 0x7fd894101940 2024-04-17T05:59:54.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:54.200+0000 7fd891ffb700 1 --2- 172.21.15.47:0/1464249095 >> [v2:172.21.15.47:6810/1998369447,v1:172.21.15.47:6811/1998369447] conn(0x7fd8840ad530 0x7fd8840af950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:54.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:54.200+0000 7fd891ffb700 1 -- 172.21.15.47:0/1464249095 --> [v2:172.21.15.47:6810/1998369447,v1:172.21.15.47:6811/1998369447] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd888014ec0 con 0x7fd8840ad530 2024-04-17T05:59:54.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:54.200+0000 7fd891ffb700 1 -- 172.21.15.47:0/1464249095 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7fd8880d0c00 con 0x7fd894101940 2024-04-17T05:59:54.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:54.200+0000 7fd8995de700 1 --2- 172.21.15.47:0/1464249095 >> [v2:172.21.15.47:6810/1998369447,v1:172.21.15.47:6811/1998369447] conn(0x7fd8840ad530 0x7fd8840af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:54.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:54.200+0000 7fd893fff700 1 --2- 172.21.15.47:0/1464249095 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fd8840a3d50 0x7fd8840a6210 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd8800098e0 tx=0x7fd880005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:59:54.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:54.201+0000 7fd8995de700 1 --2- 172.21.15.47:0/1464249095 >> [v2:172.21.15.47:6810/1998369447,v1:172.21.15.47:6811/1998369447] conn(0x7fd8840ad530 0x7fd8840af950 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:59:54.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:54.201+0000 7fd891ffb700 1 -- 172.21.15.47:0/1464249095 <== osd.3 v2:172.21.15.47:6810/1998369447 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd888014ec0 con 0x7fd8840ad530 2024-04-17T05:59:54.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:54.218+0000 7fd89b041700 1 -- 172.21.15.47:0/1464249095 --> [v2:172.21.15.47:6810/1998369447,v1:172.21.15.47:6811/1998369447] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd894063560 con 0x7fd8840ad530 2024-04-17T05:59:54.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:54.218+0000 7fd891ffb700 1 -- 172.21.15.47:0/1464249095 <== osd.3 v2:172.21.15.47:6810/1998369447 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fd894063560 con 0x7fd8840ad530 2024-04-17T05:59:54.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:54.218+0000 7fd89b041700 1 -- 172.21.15.47:0/1464249095 >> [v2:172.21.15.47:6810/1998369447,v1:172.21.15.47:6811/1998369447] conn(0x7fd8840ad530 msgr2=0x7fd8840af950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:54.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:54.218+0000 7fd89b041700 1 --2- 172.21.15.47:0/1464249095 >> [v2:172.21.15.47:6810/1998369447,v1:172.21.15.47:6811/1998369447] conn(0x7fd8840ad530 0x7fd8840af950 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:54.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:54.219+0000 7fd89b041700 1 -- 172.21.15.47:0/1464249095 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fd8840a3d50 msgr2=0x7fd8840a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:54.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:54.219+0000 7fd89b041700 1 --2- 172.21.15.47:0/1464249095 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fd8840a3d50 0x7fd8840a6210 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd8800098e0 tx=0x7fd880005960 comp rx=0 tx=0).stop 2024-04-17T05:59:54.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:54.219+0000 7fd89b041700 1 -- 172.21.15.47:0/1464249095 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd894101940 msgr2=0x7fd8941991c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:54.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:54.219+0000 7fd89b041700 1 --2- 172.21.15.47:0/1464249095 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd894101940 0x7fd8941991c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd88800c9b0 tx=0x7fd88800ccc0 comp rx=0 tx=0).stop 2024-04-17T05:59:54.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:54.219+0000 7fd8995de700 1 -- 172.21.15.47:0/1464249095 reap_dead start 2024-04-17T05:59:54.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:54.219+0000 7fd89b041700 1 -- 172.21.15.47:0/1464249095 shutdown_connections 2024-04-17T05:59:54.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:54.219+0000 7fd89b041700 1 -- 172.21.15.47:0/1464249095 >> 172.21.15.47:0/1464249095 conn(0x7fd8940788c0 msgr2=0x7fd89406b360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:59:54.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:54.219+0000 7fd89b041700 1 -- 172.21.15.47:0/1464249095 shutdown_connections 2024-04-17T05:59:54.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:54.219+0000 7fd89b041700 1 -- 172.21.15.47:0/1464249095 wait complete. 2024-04-17T05:59:54.931 INFO:teuthology.orchestra.run.smithi047.stdout:201863462959 2024-04-17T05:59:54.931 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 565db2d8-fc7e-11ee-bc91-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-04-17T05:59:55.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:55.441+0000 7f381ba50700 1 -- 172.21.15.47:0/1574160014 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3814101d20 msgr2=0x7f3814105d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:55.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:55.441+0000 7f381ba50700 1 --2- 172.21.15.47:0/1574160014 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3814101d20 0x7f3814105d20 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f38040099c0 tx=0x7f3804009cd0 comp rx=0 tx=0).stop 2024-04-17T05:59:55.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:55.441+0000 7f381ba50700 1 -- 172.21.15.47:0/1574160014 shutdown_connections 2024-04-17T05:59:55.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:55.441+0000 7f381ba50700 1 --2- 172.21.15.47:0/1574160014 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3814106400 0x7f381410dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:55.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:55.441+0000 7f381ba50700 1 --2- 172.21.15.47:0/1574160014 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3814101370 0x7f3814101750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:55.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:55.441+0000 7f381ba50700 1 --2- 172.21.15.47:0/1574160014 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3814101d20 0x7f3814105d20 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:55.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:55.441+0000 7f381ba50700 1 -- 172.21.15.47:0/1574160014 >> 172.21.15.47:0/1574160014 conn(0x7f3814077fc0 msgr2=0x7f38140ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:59:55.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:55.441+0000 7f381ba50700 1 -- 172.21.15.47:0/1574160014 shutdown_connections 2024-04-17T05:59:55.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:55.441+0000 7f381ba50700 1 -- 172.21.15.47:0/1574160014 wait complete. 2024-04-17T05:59:55.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:55.442+0000 7f381ba50700 1 Processor -- start 2024-04-17T05:59:55.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:55.442+0000 7f381ba50700 1 -- start start 2024-04-17T05:59:55.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:55.442+0000 7f381ba50700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3814101370 0x7f38141991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:55.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:55.442+0000 7f381ba50700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3814101d20 0x7f3814199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:55.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:55.442+0000 7f381ba50700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3814106400 0x7f381419dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:55.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:55.442+0000 7f381ba50700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f3814111c50 con 0x7f3814101370 2024-04-17T05:59:55.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:55.442+0000 7f381ba50700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3814111ad0 con 0x7f3814101d20 2024-04-17T05:59:55.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:55.442+0000 7f381ba50700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3814111dd0 con 0x7f3814106400 2024-04-17T05:59:55.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:55.443+0000 7f38197ec700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3814101370 0x7f38141991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:55.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:55.443+0000 7f38197ec700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3814101370 0x7f38141991e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57146/0 (socket says 172.21.15.47:57146) 2024-04-17T05:59:55.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:55.443+0000 7f38197ec700 1 -- 172.21.15.47:0/955072535 learned_addr learned my addr 172.21.15.47:0/955072535 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:59:55.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:55.443+0000 7f3819fed700 1 --2- 172.21.15.47:0/955072535 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3814106400 0x7f381419dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:55.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:55.443+0000 7f3818feb700 1 --2- 172.21.15.47:0/955072535 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3814101d20 0x7f3814199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:55.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:55.443+0000 7f38197ec700 1 -- 172.21.15.47:0/955072535 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3814106400 msgr2=0x7f381419dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:55.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:55.443+0000 7f38197ec700 1 --2- 172.21.15.47:0/955072535 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3814106400 0x7f381419dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:55.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:55.443+0000 7f38197ec700 1 -- 172.21.15.47:0/955072535 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3814101d20 msgr2=0x7f3814199720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:55.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:55.443+0000 7f38197ec700 1 --2- 172.21.15.47:0/955072535 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3814101d20 0x7f3814199720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:55.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:55.443+0000 7f38197ec700 1 -- 172.21.15.47:0/955072535 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f381419e1f0 con 0x7f3814101370 2024-04-17T05:59:55.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:55.443+0000 7f38197ec700 1 --2- 172.21.15.47:0/955072535 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3814101370 0x7f38141991e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f381000d980 tx=0x7f381000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:59:55.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:55.443+0000 7f380a7fc700 1 -- 172.21.15.47:0/955072535 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3810016070 con 0x7f3814101370 2024-04-17T05:59:55.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:55.443+0000 7f380a7fc700 1 -- 172.21.15.47:0/955072535 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3810004d10 con 0x7f3814101370 2024-04-17T05:59:55.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:55.443+0000 7f381ba50700 1 -- 172.21.15.47:0/955072535 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f381419e4e0 con 0x7f3814101370 2024-04-17T05:59:55.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:55.443+0000 7f380a7fc700 1 -- 172.21.15.47:0/955072535 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3810005670 con 0x7f3814101370 2024-04-17T05:59:55.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:55.443+0000 7f381ba50700 1 -- 172.21.15.47:0/955072535 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38141a5d60 con 0x7f3814101370 2024-04-17T05:59:55.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:55.444+0000 7f381ba50700 1 -- 172.21.15.47:0/955072535 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f38140631a0 con 0x7f3814101370 2024-04-17T05:59:55.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:55.446+0000 7f380a7fc700 1 -- 172.21.15.47:0/955072535 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f3810004750 con 0x7f3814101370 2024-04-17T05:59:55.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:55.446+0000 7f380a7fc700 1 --2- 172.21.15.47:0/955072535 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f38000a3cd0 0x7f38000a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:55.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:55.446+0000 7f3818feb700 1 --2- 172.21.15.47:0/955072535 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f38000a3cd0 0x7f38000a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:55.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:55.446+0000 7f380a7fc700 1 -- 172.21.15.47:0/955072535 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f38100d07d0 con 0x7f3814101370 2024-04-17T05:59:55.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:55.447+0000 7f380a7fc700 1 --2- 172.21.15.47:0/955072535 >> [v2:172.21.15.47:6826/728359554,v1:172.21.15.47:6827/728359554] conn(0x7f38000ad4b0 0x7f38000af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:55.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:55.447+0000 7f380a7fc700 1 -- 172.21.15.47:0/955072535 --> [v2:172.21.15.47:6826/728359554,v1:172.21.15.47:6827/728359554] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f38000aff80 con 0x7f38000ad4b0 2024-04-17T05:59:55.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:55.447+0000 7f380a7fc700 1 -- 172.21.15.47:0/955072535 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f38100d0cb0 con 0x7f3814101370 2024-04-17T05:59:55.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:55.447+0000 7f3819fed700 1 --2- 172.21.15.47:0/955072535 >> [v2:172.21.15.47:6826/728359554,v1:172.21.15.47:6827/728359554] conn(0x7f38000ad4b0 0x7f38000af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:55.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:55.447+0000 7f3818feb700 1 --2- 172.21.15.47:0/955072535 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f38000a3cd0 0x7f38000a6190 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3804000c00 tx=0x7f3804005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:59:55.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:55.447+0000 7f3819fed700 1 --2- 172.21.15.47:0/955072535 >> [v2:172.21.15.47:6826/728359554,v1:172.21.15.47:6827/728359554] conn(0x7f38000ad4b0 0x7f38000af8d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:59:55.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:55.448+0000 7f380a7fc700 1 -- 172.21.15.47:0/955072535 <== osd.9 v2:172.21.15.47:6826/728359554 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f38000aff80 con 0x7f38000ad4b0 2024-04-17T05:59:55.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:55.467+0000 7f381ba50700 1 -- 172.21.15.47:0/955072535 --> [v2:172.21.15.47:6826/728359554,v1:172.21.15.47:6827/728359554] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f38140635a0 con 0x7f38000ad4b0 2024-04-17T05:59:55.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:55.467+0000 7f380a7fc700 1 -- 172.21.15.47:0/955072535 <== osd.9 v2:172.21.15.47:6826/728359554 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f38140635a0 con 0x7f38000ad4b0 2024-04-17T05:59:55.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:55.467+0000 7f381ba50700 1 -- 172.21.15.47:0/955072535 >> [v2:172.21.15.47:6826/728359554,v1:172.21.15.47:6827/728359554] conn(0x7f38000ad4b0 msgr2=0x7f38000af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:55.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:55.467+0000 7f381ba50700 1 --2- 172.21.15.47:0/955072535 >> [v2:172.21.15.47:6826/728359554,v1:172.21.15.47:6827/728359554] conn(0x7f38000ad4b0 0x7f38000af8d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:55.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:55.467+0000 7f381ba50700 1 -- 172.21.15.47:0/955072535 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f38000a3cd0 msgr2=0x7f38000a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:55.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:55.467+0000 7f381ba50700 1 --2- 172.21.15.47:0/955072535 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f38000a3cd0 0x7f38000a6190 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3804000c00 tx=0x7f3804005fb0 comp rx=0 tx=0).stop 2024-04-17T05:59:55.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:55.467+0000 7f381ba50700 1 -- 172.21.15.47:0/955072535 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3814101370 msgr2=0x7f38141991e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:55.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:55.467+0000 7f381ba50700 1 --2- 172.21.15.47:0/955072535 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3814101370 0x7f38141991e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f381000d980 tx=0x7f381000dc90 comp rx=0 tx=0).stop 2024-04-17T05:59:55.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:55.467+0000 7f3819fed700 1 -- 172.21.15.47:0/955072535 reap_dead start 2024-04-17T05:59:55.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:55.467+0000 7f381ba50700 1 -- 172.21.15.47:0/955072535 shutdown_connections 2024-04-17T05:59:55.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:55.468+0000 7f381ba50700 1 -- 172.21.15.47:0/955072535 >> 172.21.15.47:0/955072535 conn(0x7f3814077fc0 msgr2=0x7f381410eb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:59:55.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:55.468+0000 7f381ba50700 1 -- 172.21.15.47:0/955072535 shutdown_connections 2024-04-17T05:59:55.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:55.468+0000 7f381ba50700 1 -- 172.21.15.47:0/955072535 wait complete. 2024-04-17T05:59:55.935 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:55 smithi047 ceph-mon[32223]: pgmap v306: 97 pgs: 97 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-04-17T05:59:55.935 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:55 smithi098 ceph-mon[36516]: pgmap v306: 97 pgs: 97 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-04-17T05:59:56.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:55 smithi103 ceph-mon[36243]: pgmap v306: 97 pgs: 97 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-04-17T05:59:56.365 INFO:teuthology.orchestra.run.smithi047.stdout:292057776160 2024-04-17T05:59:56.366 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 565db2d8-fc7e-11ee-bc91-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-04-17T05:59:56.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:56.549+0000 7f303f4e0700 1 -- 172.21.15.47:0/2526998899 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f303806a760 msgr2=0x7f303806abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:56.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:56.549+0000 7f303f4e0700 1 --2- 172.21.15.47:0/2526998899 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f303806a760 0x7f303806abc0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f3028009990 tx=0x7f3028009ca0 comp rx=0 tx=0).stop 2024-04-17T05:59:56.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:56.549+0000 7f303f4e0700 1 -- 172.21.15.47:0/2526998899 shutdown_connections 2024-04-17T05:59:56.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:56.549+0000 7f303f4e0700 1 --2- 172.21.15.47:0/2526998899 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f303806b100 0x7f303810f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:56.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:56.549+0000 7f303f4e0700 1 --2- 172.21.15.47:0/2526998899 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3038108a40 0x7f3038108e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:56.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:56.549+0000 7f303f4e0700 1 --2- 172.21.15.47:0/2526998899 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f303806a760 0x7f303806abc0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:56.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:56.549+0000 7f303f4e0700 1 -- 172.21.15.47:0/2526998899 >> 172.21.15.47:0/2526998899 conn(0x7f3038078900 msgr2=0x7f3038078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:59:56.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:56.549+0000 7f303f4e0700 1 -- 172.21.15.47:0/2526998899 shutdown_connections 2024-04-17T05:59:56.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:56.549+0000 7f303f4e0700 1 -- 172.21.15.47:0/2526998899 wait complete. 2024-04-17T05:59:56.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:56.550+0000 7f303f4e0700 1 Processor -- start 2024-04-17T05:59:56.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:56.550+0000 7f303f4e0700 1 -- start start 2024-04-17T05:59:56.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:56.550+0000 7f303f4e0700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f303806a760 0x7f3038199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:56.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:56.550+0000 7f303f4e0700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f303806b100 0x7f3038199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:56.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:56.550+0000 7f303f4e0700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3038108a40 0x7f303819dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:56.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:56.550+0000 7f303f4e0700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f3038112d30 con 0x7f3038108a40 2024-04-17T05:59:56.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:56.550+0000 7f303f4e0700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3038112bb0 con 0x7f303806b100 2024-04-17T05:59:56.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:56.550+0000 7f303f4e0700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3038112eb0 con 0x7f303806a760 2024-04-17T05:59:56.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:56.550+0000 7f303da7d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3038108a40 0x7f303819dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:56.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:56.550+0000 7f303da7d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3038108a40 0x7f303819dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57166/0 (socket says 172.21.15.47:57166) 2024-04-17T05:59:56.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:56.550+0000 7f303da7d700 1 -- 172.21.15.47:0/3718920487 learned_addr learned my addr 172.21.15.47:0/3718920487 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:59:56.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:56.550+0000 7f303ca7b700 1 --2- 172.21.15.47:0/3718920487 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f303806b100 0x7f3038199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:56.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:56.550+0000 7f303d27c700 1 --2- 172.21.15.47:0/3718920487 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f303806a760 0x7f3038199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:56.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:56.551+0000 7f303da7d700 1 -- 172.21.15.47:0/3718920487 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f303806a760 msgr2=0x7f3038199240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:56.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:56.551+0000 7f303da7d700 1 --2- 172.21.15.47:0/3718920487 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f303806a760 0x7f3038199240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:56.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:56.551+0000 7f303da7d700 1 -- 172.21.15.47:0/3718920487 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f303806b100 msgr2=0x7f3038199780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:56.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:56.551+0000 7f303da7d700 1 --2- 172.21.15.47:0/3718920487 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f303806b100 0x7f3038199780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:56.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:56.551+0000 7f303da7d700 1 -- 172.21.15.47:0/3718920487 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f303819e200 con 0x7f3038108a40 2024-04-17T05:59:56.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:56.551+0000 7f303ca7b700 1 --2- 172.21.15.47:0/3718920487 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f303806b100 0x7f3038199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T05:59:56.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:56.551+0000 7f303da7d700 1 --2- 172.21.15.47:0/3718920487 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3038108a40 0x7f303819dac0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f3030007ee0 tx=0x7f303000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:59:56.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:56.551+0000 7f302e7fc700 1 -- 172.21.15.47:0/3718920487 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f303000dd20 con 0x7f3038108a40 2024-04-17T05:59:56.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:56.551+0000 7f302e7fc700 1 -- 172.21.15.47:0/3718920487 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3030004020 con 0x7f3038108a40 2024-04-17T05:59:56.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:56.551+0000 7f303f4e0700 1 -- 172.21.15.47:0/3718920487 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f303819e4f0 con 0x7f3038108a40 2024-04-17T05:59:56.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:56.551+0000 7f302e7fc700 1 -- 172.21.15.47:0/3718920487 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3030015660 con 0x7f3038108a40 2024-04-17T05:59:56.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:56.551+0000 7f303f4e0700 1 -- 172.21.15.47:0/3718920487 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30381a5d70 con 0x7f3038108a40 2024-04-17T05:59:56.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:56.552+0000 7f303f4e0700 1 -- 172.21.15.47:0/3718920487 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f30380631e0 con 0x7f3038108a40 2024-04-17T05:59:56.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:56.554+0000 7f302e7fc700 1 -- 172.21.15.47:0/3718920487 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f30300041d0 con 0x7f3038108a40 2024-04-17T05:59:56.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:56.554+0000 7f302e7fc700 1 --2- 172.21.15.47:0/3718920487 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f30240a3cd0 0x7f30240a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:56.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:56.554+0000 7f302e7fc700 1 -- 172.21.15.47:0/3718920487 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f30300cfde0 con 0x7f3038108a40 2024-04-17T05:59:56.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:56.555+0000 7f303d27c700 1 --2- 172.21.15.47:0/3718920487 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f30240a3cd0 0x7f30240a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:56.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:56.555+0000 7f302e7fc700 1 --2- 172.21.15.47:0/3718920487 >> [v2:172.21.15.98:6824/468205689,v1:172.21.15.98:6825/468205689] conn(0x7f30240ad4b0 0x7f30240af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:56.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:56.555+0000 7f302e7fc700 1 -- 172.21.15.47:0/3718920487 --> [v2:172.21.15.98:6824/468205689,v1:172.21.15.98:6825/468205689] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f30240aff80 con 0x7f30240ad4b0 2024-04-17T05:59:56.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:56.555+0000 7f302e7fc700 1 -- 172.21.15.47:0/3718920487 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f30300d2530 con 0x7f3038108a40 2024-04-17T05:59:56.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:56.555+0000 7f303ca7b700 1 --2- 172.21.15.47:0/3718920487 >> [v2:172.21.15.98:6824/468205689,v1:172.21.15.98:6825/468205689] conn(0x7f30240ad4b0 0x7f30240af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:56.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:56.555+0000 7f303d27c700 1 --2- 172.21.15.47:0/3718920487 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f30240a3cd0 0x7f30240a6190 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3034009be0 tx=0x7f3034009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:59:56.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:56.555+0000 7f303ca7b700 1 --2- 172.21.15.47:0/3718920487 >> [v2:172.21.15.98:6824/468205689,v1:172.21.15.98:6825/468205689] conn(0x7f30240ad4b0 0x7f30240af8d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:59:56.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:56.556+0000 7f302e7fc700 1 -- 172.21.15.47:0/3718920487 <== osd.10 v2:172.21.15.98:6824/468205689 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f30240aff80 con 0x7f30240ad4b0 2024-04-17T05:59:56.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:56.573+0000 7f303f4e0700 1 -- 172.21.15.47:0/3718920487 --> [v2:172.21.15.98:6824/468205689,v1:172.21.15.98:6825/468205689] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f30380635e0 con 0x7f30240ad4b0 2024-04-17T05:59:56.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:56.573+0000 7f302e7fc700 1 -- 172.21.15.47:0/3718920487 <== osd.10 v2:172.21.15.98:6824/468205689 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f30380635e0 con 0x7f30240ad4b0 2024-04-17T05:59:56.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:56.573+0000 7f303f4e0700 1 -- 172.21.15.47:0/3718920487 >> [v2:172.21.15.98:6824/468205689,v1:172.21.15.98:6825/468205689] conn(0x7f30240ad4b0 msgr2=0x7f30240af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:56.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:56.573+0000 7f303f4e0700 1 --2- 172.21.15.47:0/3718920487 >> [v2:172.21.15.98:6824/468205689,v1:172.21.15.98:6825/468205689] conn(0x7f30240ad4b0 0x7f30240af8d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:56.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:56.573+0000 7f303f4e0700 1 -- 172.21.15.47:0/3718920487 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f30240a3cd0 msgr2=0x7f30240a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:56.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:56.573+0000 7f303f4e0700 1 --2- 172.21.15.47:0/3718920487 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f30240a3cd0 0x7f30240a6190 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3034009be0 tx=0x7f3034009260 comp rx=0 tx=0).stop 2024-04-17T05:59:56.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:56.573+0000 7f303f4e0700 1 -- 172.21.15.47:0/3718920487 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3038108a40 msgr2=0x7f303819dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:56.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:56.573+0000 7f303f4e0700 1 --2- 172.21.15.47:0/3718920487 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3038108a40 0x7f303819dac0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f3030007ee0 tx=0x7f303000d3b0 comp rx=0 tx=0).stop 2024-04-17T05:59:56.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:56.573+0000 7f303da7d700 1 -- 172.21.15.47:0/3718920487 reap_dead start 2024-04-17T05:59:56.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:56.574+0000 7f303f4e0700 1 -- 172.21.15.47:0/3718920487 shutdown_connections 2024-04-17T05:59:56.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:56.574+0000 7f303f4e0700 1 -- 172.21.15.47:0/3718920487 >> 172.21.15.47:0/3718920487 conn(0x7f3038078900 msgr2=0x7f30380ffa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:59:56.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:56.574+0000 7f303f4e0700 1 -- 172.21.15.47:0/3718920487 shutdown_connections 2024-04-17T05:59:56.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:56.574+0000 7f303f4e0700 1 -- 172.21.15.47:0/3718920487 wait complete. 2024-04-17T05:59:57.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.225+0000 7f8f20368700 1 -- 172.21.15.47:0/2094976623 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8f1806bb80 msgr2=0x7f8f1810f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:57.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.225+0000 7f8f20368700 1 --2- 172.21.15.47:0/2094976623 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8f1806bb80 0x7f8f1810f910 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f8f1000b0d0 tx=0x7f8f1000b3e0 comp rx=0 tx=0).stop 2024-04-17T05:59:57.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.226+0000 7f8f20368700 1 -- 172.21.15.47:0/2094976623 shutdown_connections 2024-04-17T05:59:57.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.226+0000 7f8f20368700 1 --2- 172.21.15.47:0/2094976623 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8f1806bb80 0x7f8f1810f910 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:57.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.226+0000 7f8f20368700 1 --2- 172.21.15.47:0/2094976623 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f18108a60 0x7f8f18108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:57.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.226+0000 7f8f20368700 1 --2- 172.21.15.47:0/2094976623 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f1806b1c0 0x7f8f1806b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:57.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.226+0000 7f8f20368700 1 -- 172.21.15.47:0/2094976623 >> 172.21.15.47:0/2094976623 conn(0x7f8f180779a0 msgr2=0x7f8f18077db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:59:57.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.226+0000 7f8f20368700 1 -- 172.21.15.47:0/2094976623 shutdown_connections 2024-04-17T05:59:57.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.226+0000 7f8f20368700 1 -- 172.21.15.47:0/2094976623 wait complete. 2024-04-17T05:59:57.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.226+0000 7f8f20368700 1 Processor -- start 2024-04-17T05:59:57.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.226+0000 7f8f20368700 1 -- start start 2024-04-17T05:59:57.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.226+0000 7f8f20368700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f1806b1c0 0x7f8f18199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:57.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.226+0000 7f8f20368700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8f1806bb80 0x7f8f181996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:57.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.226+0000 7f8f20368700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f18108a60 0x7f8f1819d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:57.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.226+0000 7f8f20368700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f8f18112d70 con 0x7f8f1806bb80 2024-04-17T05:59:57.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.226+0000 7f8f20368700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8f18112bf0 con 0x7f8f1806b1c0 2024-04-17T05:59:57.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.226+0000 7f8f20368700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8f18112ef0 con 0x7f8f18108a60 2024-04-17T05:59:57.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.227+0000 7f8f1d903700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8f1806bb80 0x7f8f181996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:57.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.227+0000 7f8f1d903700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8f1806bb80 0x7f8f181996a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57176/0 (socket says 172.21.15.47:57176) 2024-04-17T05:59:57.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.227+0000 7f8f1d903700 1 -- 172.21.15.47:0/1983717423 learned_addr learned my addr 172.21.15.47:0/1983717423 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:59:57.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.227+0000 7f8f1e104700 1 --2- 172.21.15.47:0/1983717423 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f1806b1c0 0x7f8f18199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:57.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.227+0000 7f8f1e905700 1 --2- 172.21.15.47:0/1983717423 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f18108a60 0x7f8f1819d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:57.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.227+0000 7f8f1d903700 1 -- 172.21.15.47:0/1983717423 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f18108a60 msgr2=0x7f8f1819d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:57.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.227+0000 7f8f1d903700 1 --2- 172.21.15.47:0/1983717423 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f18108a60 0x7f8f1819d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:57.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.227+0000 7f8f1d903700 1 -- 172.21.15.47:0/1983717423 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f1806b1c0 msgr2=0x7f8f18199160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:57.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.227+0000 7f8f1d903700 1 --2- 172.21.15.47:0/1983717423 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f1806b1c0 0x7f8f18199160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:57.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.227+0000 7f8f1d903700 1 -- 172.21.15.47:0/1983717423 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f1819e0d0 con 0x7f8f1806bb80 2024-04-17T05:59:57.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.227+0000 7f8f1d903700 1 --2- 172.21.15.47:0/1983717423 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8f1806bb80 0x7f8f181996a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f8f0800d980 tx=0x7f8f0800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:59:57.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.227+0000 7f8f0f7fe700 1 -- 172.21.15.47:0/1983717423 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f08016070 con 0x7f8f1806bb80 2024-04-17T05:59:57.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.227+0000 7f8f0f7fe700 1 -- 172.21.15.47:0/1983717423 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f08004d10 con 0x7f8f1806bb80 2024-04-17T05:59:57.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.227+0000 7f8f20368700 1 -- 172.21.15.47:0/1983717423 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f1819e3c0 con 0x7f8f1806bb80 2024-04-17T05:59:57.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.227+0000 7f8f0f7fe700 1 -- 172.21.15.47:0/1983717423 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f080055f0 con 0x7f8f1806bb80 2024-04-17T05:59:57.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.227+0000 7f8f20368700 1 -- 172.21.15.47:0/1983717423 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f181a5c40 con 0x7f8f1806bb80 2024-04-17T05:59:57.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.228+0000 7f8f20368700 1 -- 172.21.15.47:0/1983717423 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8f180631e0 con 0x7f8f1806bb80 2024-04-17T05:59:57.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.229+0000 7f8f0f7fe700 1 -- 172.21.15.47:0/1983717423 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f8f08004750 con 0x7f8f1806bb80 2024-04-17T05:59:57.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.230+0000 7f8f0f7fe700 1 --2- 172.21.15.47:0/1983717423 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f8f040a3cd0 0x7f8f040a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:57.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.230+0000 7f8f1e104700 1 --2- 172.21.15.47:0/1983717423 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f8f040a3cd0 0x7f8f040a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:57.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.230+0000 7f8f0f7fe700 1 -- 172.21.15.47:0/1983717423 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f8f080d09f0 con 0x7f8f1806bb80 2024-04-17T05:59:57.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.230+0000 7f8f0f7fe700 1 --2- 172.21.15.47:0/1983717423 >> [v2:172.21.15.98:6808/3060482485,v1:172.21.15.98:6809/3060482485] conn(0x7f8f040ad4b0 0x7f8f040af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:57.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.230+0000 7f8f0f7fe700 1 -- 172.21.15.47:0/1983717423 --> [v2:172.21.15.98:6808/3060482485,v1:172.21.15.98:6809/3060482485] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8f08015ec0 con 0x7f8f040ad4b0 2024-04-17T05:59:57.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.230+0000 7f8f0f7fe700 1 -- 172.21.15.47:0/1983717423 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f8f080d0da0 con 0x7f8f1806bb80 2024-04-17T05:59:57.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.231+0000 7f8f1e104700 1 --2- 172.21.15.47:0/1983717423 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f8f040a3cd0 0x7f8f040a6190 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f8f1400abd0 tx=0x7f8f1400a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:59:57.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.231+0000 7f8f1e905700 1 --2- 172.21.15.47:0/1983717423 >> [v2:172.21.15.98:6808/3060482485,v1:172.21.15.98:6809/3060482485] conn(0x7f8f040ad4b0 0x7f8f040af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:57.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.231+0000 7f8f1e905700 1 --2- 172.21.15.47:0/1983717423 >> [v2:172.21.15.98:6808/3060482485,v1:172.21.15.98:6809/3060482485] conn(0x7f8f040ad4b0 0x7f8f040af8d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:59:57.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.232+0000 7f8f0f7fe700 1 -- 172.21.15.47:0/1983717423 <== osd.4 v2:172.21.15.98:6808/3060482485 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8f08015ec0 con 0x7f8f040ad4b0 2024-04-17T05:59:57.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.248+0000 7f8f20368700 1 -- 172.21.15.47:0/1983717423 --> [v2:172.21.15.98:6808/3060482485,v1:172.21.15.98:6809/3060482485] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8f18063660 con 0x7f8f040ad4b0 2024-04-17T05:59:57.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.249+0000 7f8f0f7fe700 1 -- 172.21.15.47:0/1983717423 <== osd.4 v2:172.21.15.98:6808/3060482485 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f8f18063660 con 0x7f8f040ad4b0 2024-04-17T05:59:57.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.249+0000 7f8f20368700 1 -- 172.21.15.47:0/1983717423 >> [v2:172.21.15.98:6808/3060482485,v1:172.21.15.98:6809/3060482485] conn(0x7f8f040ad4b0 msgr2=0x7f8f040af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:57.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.249+0000 7f8f20368700 1 --2- 172.21.15.47:0/1983717423 >> [v2:172.21.15.98:6808/3060482485,v1:172.21.15.98:6809/3060482485] conn(0x7f8f040ad4b0 0x7f8f040af8d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:57.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.249+0000 7f8f20368700 1 -- 172.21.15.47:0/1983717423 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f8f040a3cd0 msgr2=0x7f8f040a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:57.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.249+0000 7f8f20368700 1 --2- 172.21.15.47:0/1983717423 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f8f040a3cd0 0x7f8f040a6190 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f8f1400abd0 tx=0x7f8f1400a380 comp rx=0 tx=0).stop 2024-04-17T05:59:57.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.249+0000 7f8f20368700 1 -- 172.21.15.47:0/1983717423 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8f1806bb80 msgr2=0x7f8f181996a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:57.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.249+0000 7f8f20368700 1 --2- 172.21.15.47:0/1983717423 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8f1806bb80 0x7f8f181996a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f8f0800d980 tx=0x7f8f0800dc90 comp rx=0 tx=0).stop 2024-04-17T05:59:57.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.249+0000 7f8f1e905700 1 -- 172.21.15.47:0/1983717423 reap_dead start 2024-04-17T05:59:57.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.249+0000 7f8f20368700 1 -- 172.21.15.47:0/1983717423 shutdown_connections 2024-04-17T05:59:57.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.249+0000 7f8f20368700 1 -- 172.21.15.47:0/1983717423 >> 172.21.15.47:0/1983717423 conn(0x7f8f180779a0 msgr2=0x7f8f181004c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:59:57.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.250+0000 7f8f20368700 1 -- 172.21.15.47:0/1983717423 shutdown_connections 2024-04-17T05:59:57.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.250+0000 7f8f20368700 1 -- 172.21.15.47:0/1983717423 wait complete. 2024-04-17T05:59:57.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.669+0000 7f27568ad700 1 -- 172.21.15.47:0/507940508 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2750107930 msgr2=0x7f2750107d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:57.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.669+0000 7f27568ad700 1 --2- 172.21.15.47:0/507940508 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2750107930 0x7f2750107d10 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f2740009990 tx=0x7f2740009ca0 comp rx=0 tx=0).stop 2024-04-17T05:59:57.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.669+0000 7f27568ad700 1 -- 172.21.15.47:0/507940508 shutdown_connections 2024-04-17T05:59:57.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.669+0000 7f27568ad700 1 --2- 172.21.15.47:0/507940508 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2750102340 0x7f275010e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:57.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.669+0000 7f27568ad700 1 --2- 172.21.15.47:0/507940508 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2750107930 0x7f2750107d10 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:57.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.669+0000 7f27568ad700 1 --2- 172.21.15.47:0/507940508 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2750101980 0x7f2750101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:57.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.669+0000 7f27568ad700 1 -- 172.21.15.47:0/507940508 >> 172.21.15.47:0/507940508 conn(0x7f2750078900 msgr2=0x7f2750078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:59:57.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.669+0000 7f27568ad700 1 -- 172.21.15.47:0/507940508 shutdown_connections 2024-04-17T05:59:57.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.669+0000 7f27568ad700 1 -- 172.21.15.47:0/507940508 wait complete. 2024-04-17T05:59:57.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.670+0000 7f27568ad700 1 Processor -- start 2024-04-17T05:59:57.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.670+0000 7f27568ad700 1 -- start start 2024-04-17T05:59:57.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.670+0000 7f27568ad700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2750101980 0x7f275019a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:57.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.670+0000 7f27568ad700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2750102340 0x7f275019a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:57.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.670+0000 7f27568ad700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2750107930 0x7f27501a5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:57.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.670+0000 7f27568ad700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f2750111c60 con 0x7f2750107930 2024-04-17T05:59:57.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.670+0000 7f27568ad700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2750111ae0 con 0x7f2750101980 2024-04-17T05:59:57.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.670+0000 7f27568ad700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2750111de0 con 0x7f2750102340 2024-04-17T05:59:57.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.671+0000 7f2754e4a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2750107930 0x7f27501a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:57.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.671+0000 7f2754e4a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2750107930 0x7f27501a5120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57186/0 (socket says 172.21.15.47:57186) 2024-04-17T05:59:57.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.671+0000 7f2754e4a700 1 -- 172.21.15.47:0/2964342770 learned_addr learned my addr 172.21.15.47:0/2964342770 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T05:59:57.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.671+0000 7f274f7fe700 1 --2- 172.21.15.47:0/2964342770 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2750102340 0x7f275019a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:57.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.671+0000 7f274ffff700 1 --2- 172.21.15.47:0/2964342770 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2750101980 0x7f275019a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:57.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.671+0000 7f2754e4a700 1 -- 172.21.15.47:0/2964342770 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2750102340 msgr2=0x7f275019a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:57.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.671+0000 7f2754e4a700 1 --2- 172.21.15.47:0/2964342770 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2750102340 0x7f275019a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:57.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.671+0000 7f2754e4a700 1 -- 172.21.15.47:0/2964342770 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2750101980 msgr2=0x7f275019a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:57.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.671+0000 7f2754e4a700 1 --2- 172.21.15.47:0/2964342770 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2750101980 0x7f275019a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:57.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.671+0000 7f2754e4a700 1 -- 172.21.15.47:0/2964342770 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27501a5660 con 0x7f2750107930 2024-04-17T05:59:57.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.671+0000 7f2754e4a700 1 --2- 172.21.15.47:0/2964342770 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2750107930 0x7f27501a5120 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f274800ada0 tx=0x7f274800c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:59:57.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.671+0000 7f274d7fa700 1 -- 172.21.15.47:0/2964342770 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2748017020 con 0x7f2750107930 2024-04-17T05:59:57.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.671+0000 7f274d7fa700 1 -- 172.21.15.47:0/2964342770 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f274800cdb0 con 0x7f2750107930 2024-04-17T05:59:57.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.671+0000 7f27568ad700 1 -- 172.21.15.47:0/2964342770 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27501a5890 con 0x7f2750107930 2024-04-17T05:59:57.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.671+0000 7f274d7fa700 1 -- 172.21.15.47:0/2964342770 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27480053e0 con 0x7f2750107930 2024-04-17T05:59:57.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.671+0000 7f27568ad700 1 -- 172.21.15.47:0/2964342770 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27501a5dd0 con 0x7f2750107930 2024-04-17T05:59:57.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.672+0000 7f27568ad700 1 -- 172.21.15.47:0/2964342770 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f27500630e0 con 0x7f2750107930 2024-04-17T05:59:57.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.673+0000 7f274d7fa700 1 -- 172.21.15.47:0/2964342770 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f274800c8d0 con 0x7f2750107930 2024-04-17T05:59:57.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.674+0000 7f274d7fa700 1 --2- 172.21.15.47:0/2964342770 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f27380a3d50 0x7f27380a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:57.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.674+0000 7f274ffff700 1 --2- 172.21.15.47:0/2964342770 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f27380a3d50 0x7f27380a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:57.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.674+0000 7f274d7fa700 1 -- 172.21.15.47:0/2964342770 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f27480d0410 con 0x7f2750107930 2024-04-17T05:59:57.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.674+0000 7f274d7fa700 1 --2- 172.21.15.47:0/2964342770 >> [v2:172.21.15.98:6800/1180430954,v1:172.21.15.98:6801/1180430954] conn(0x7f27380ad530 0x7f27380af950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T05:59:57.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.674+0000 7f274d7fa700 1 -- 172.21.15.47:0/2964342770 --> [v2:172.21.15.98:6800/1180430954,v1:172.21.15.98:6801/1180430954] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f274800fe50 con 0x7f27380ad530 2024-04-17T05:59:57.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.674+0000 7f274d7fa700 1 -- 172.21.15.47:0/2964342770 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f27480d08f0 con 0x7f2750107930 2024-04-17T05:59:57.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.675+0000 7f274ffff700 1 --2- 172.21.15.47:0/2964342770 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f27380a3d50 0x7f27380a6210 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f274000b2e0 tx=0x7f2740005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:59:57.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.675+0000 7f274f7fe700 1 --2- 172.21.15.47:0/2964342770 >> [v2:172.21.15.98:6800/1180430954,v1:172.21.15.98:6801/1180430954] conn(0x7f27380ad530 0x7f27380af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T05:59:57.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.675+0000 7f274f7fe700 1 --2- 172.21.15.47:0/2964342770 >> [v2:172.21.15.98:6800/1180430954,v1:172.21.15.98:6801/1180430954] conn(0x7f27380ad530 0x7f27380af950 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T05:59:57.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.676+0000 7f274d7fa700 1 -- 172.21.15.47:0/2964342770 <== osd.1 v2:172.21.15.98:6800/1180430954 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f274800fe50 con 0x7f27380ad530 2024-04-17T05:59:57.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.693+0000 7f27568ad700 1 -- 172.21.15.47:0/2964342770 --> [v2:172.21.15.98:6800/1180430954,v1:172.21.15.98:6801/1180430954] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f27500634e0 con 0x7f27380ad530 2024-04-17T05:59:57.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.694+0000 7f274d7fa700 1 -- 172.21.15.47:0/2964342770 <== osd.1 v2:172.21.15.98:6800/1180430954 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f27500634e0 con 0x7f27380ad530 2024-04-17T05:59:57.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.694+0000 7f27568ad700 1 -- 172.21.15.47:0/2964342770 >> [v2:172.21.15.98:6800/1180430954,v1:172.21.15.98:6801/1180430954] conn(0x7f27380ad530 msgr2=0x7f27380af950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:57.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.694+0000 7f27568ad700 1 --2- 172.21.15.47:0/2964342770 >> [v2:172.21.15.98:6800/1180430954,v1:172.21.15.98:6801/1180430954] conn(0x7f27380ad530 0x7f27380af950 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T05:59:57.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.694+0000 7f27568ad700 1 -- 172.21.15.47:0/2964342770 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f27380a3d50 msgr2=0x7f27380a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:57.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.694+0000 7f27568ad700 1 --2- 172.21.15.47:0/2964342770 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f27380a3d50 0x7f27380a6210 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f274000b2e0 tx=0x7f2740005960 comp rx=0 tx=0).stop 2024-04-17T05:59:57.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.694+0000 7f27568ad700 1 -- 172.21.15.47:0/2964342770 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2750107930 msgr2=0x7f27501a5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T05:59:57.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.694+0000 7f27568ad700 1 --2- 172.21.15.47:0/2964342770 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2750107930 0x7f27501a5120 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f274800ada0 tx=0x7f274800c330 comp rx=0 tx=0).stop 2024-04-17T05:59:57.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.694+0000 7f2754e4a700 1 -- 172.21.15.47:0/2964342770 reap_dead start 2024-04-17T05:59:57.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.695+0000 7f27568ad700 1 -- 172.21.15.47:0/2964342770 shutdown_connections 2024-04-17T05:59:57.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.695+0000 7f27568ad700 1 -- 172.21.15.47:0/2964342770 >> 172.21.15.47:0/2964342770 conn(0x7f2750078900 msgr2=0x7f27500ffa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T05:59:57.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.695+0000 7f27568ad700 1 -- 172.21.15.47:0/2964342770 shutdown_connections 2024-04-17T05:59:57.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T05:59:57.695+0000 7f27568ad700 1 -- 172.21.15.47:0/2964342770 wait complete. 2024-04-17T05:59:57.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:57 smithi047 ceph-mon[32223]: pgmap v307: 97 pgs: 97 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-17T05:59:58.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:57 smithi103 ceph-mon[36243]: pgmap v307: 97 pgs: 97 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-17T05:59:58.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:57 smithi098 ceph-mon[36516]: pgmap v307: 97 pgs: 97 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-17T05:59:58.486 INFO:teuthology.orchestra.run.smithi047.stdout:111669149758 2024-04-17T05:59:58.486 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 565db2d8-fc7e-11ee-bc91-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-04-17T05:59:59.602 INFO:teuthology.orchestra.run.smithi047.stdout:313532612635 2024-04-17T05:59:59.613 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 565db2d8-fc7e-11ee-bc91-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-04-17T05:59:59.750 INFO:teuthology.orchestra.run.smithi047.stdout:34359738444 2024-04-17T05:59:59.750 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 565db2d8-fc7e-11ee-bc91-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-04-17T05:59:59.830 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 05:59:59 smithi047 ceph-mon[32223]: pgmap v308: 97 pgs: 97 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-04-17T06:00:00.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 05:59:59 smithi103 ceph-mon[36243]: pgmap v308: 97 pgs: 97 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-04-17T06:00:00.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 05:59:59 smithi098 ceph-mon[36516]: pgmap v308: 97 pgs: 97 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-04-17T06:00:00.652 INFO:teuthology.orchestra.run.smithi047.stdout:146028888122 2024-04-17T06:00:00.653 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 565db2d8-fc7e-11ee-bc91-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-04-17T06:00:00.869 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T06:00:00.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:00:00 smithi047 ceph-mon[32223]: overall HEALTH_OK 2024-04-17T06:00:01.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:00:00 smithi103 ceph-mon[36243]: overall HEALTH_OK 2024-04-17T06:00:01.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:00:00 smithi098 ceph-mon[36516]: overall HEALTH_OK 2024-04-17T06:00:01.433 INFO:teuthology.orchestra.run.smithi047.stdout:55834574920 2024-04-17T06:00:01.433 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 565db2d8-fc7e-11ee-bc91-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-04-17T06:00:01.922 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:00:01 smithi047 ceph-mon[32223]: pgmap v309: 97 pgs: 97 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-17T06:00:02.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:00:01 smithi103 ceph-mon[36243]: pgmap v309: 97 pgs: 97 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-17T06:00:02.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:00:01 smithi098 ceph-mon[36516]: pgmap v309: 97 pgs: 97 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-17T06:00:02.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.270+0000 7f7fccb8a700 1 -- 172.21.15.47:0/1667317880 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7fc8100fa0 msgr2=0x7f7fc8101420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:02.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.270+0000 7f7fccb8a700 1 --2- 172.21.15.47:0/1667317880 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7fc8100fa0 0x7f7fc8101420 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f7fb80099c0 tx=0x7f7fb8009cd0 comp rx=0 tx=0).stop 2024-04-17T06:00:02.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.270+0000 7f7fccb8a700 1 -- 172.21.15.47:0/1667317880 shutdown_connections 2024-04-17T06:00:02.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.270+0000 7f7fccb8a700 1 --2- 172.21.15.47:0/1667317880 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7fc8101960 0x7f7fc810f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:02.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.270+0000 7f7fccb8a700 1 --2- 172.21.15.47:0/1667317880 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7fc8107a70 0x7f7fc8107e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:02.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.270+0000 7f7fccb8a700 1 --2- 172.21.15.47:0/1667317880 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7fc8100fa0 0x7f7fc8101420 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:02.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.270+0000 7f7fccb8a700 1 -- 172.21.15.47:0/1667317880 >> 172.21.15.47:0/1667317880 conn(0x7f7fc8077f50 msgr2=0x7f7fc80ff2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:00:02.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.270+0000 7f7fccb8a700 1 -- 172.21.15.47:0/1667317880 shutdown_connections 2024-04-17T06:00:02.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.270+0000 7f7fccb8a700 1 -- 172.21.15.47:0/1667317880 wait complete. 2024-04-17T06:00:02.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.271+0000 7f7fccb8a700 1 Processor -- start 2024-04-17T06:00:02.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.271+0000 7f7fccb8a700 1 -- start start 2024-04-17T06:00:02.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.271+0000 7f7fccb8a700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7fc8100fa0 0x7f7fc80740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:02.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.271+0000 7f7fccb8a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7fc8101960 0x7f7fc8074630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:02.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.271+0000 7f7fccb8a700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7fc8107a70 0x7f7fc806d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:02.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.271+0000 7f7fccb8a700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc8112ed0 con 0x7f7fc8101960 2024-04-17T06:00:02.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.271+0000 7f7fccb8a700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc8112d50 con 0x7f7fc8107a70 2024-04-17T06:00:02.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.271+0000 7f7fc5d9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7fc8101960 0x7f7fc8074630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:02.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.271+0000 7f7fc5d9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7fc8101960 0x7f7fc8074630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57206/0 (socket says 172.21.15.47:57206) 2024-04-17T06:00:02.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.271+0000 7f7fc5d9b700 1 -- 172.21.15.47:0/3023075229 learned_addr learned my addr 172.21.15.47:0/3023075229 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:00:02.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.271+0000 7f7fccb8a700 1 -- 172.21.15.47:0/3023075229 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc8113050 con 0x7f7fc8100fa0 2024-04-17T06:00:02.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.271+0000 7f7fc6d9d700 1 --2- 172.21.15.47:0/3023075229 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7fc8107a70 0x7f7fc806d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:02.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.271+0000 7f7fc659c700 1 --2- 172.21.15.47:0/3023075229 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7fc8100fa0 0x7f7fc80740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:02.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.272+0000 7f7fc5d9b700 1 -- 172.21.15.47:0/3023075229 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7fc8100fa0 msgr2=0x7f7fc80740f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:02.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.272+0000 7f7fc5d9b700 1 --2- 172.21.15.47:0/3023075229 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7fc8100fa0 0x7f7fc80740f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:02.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.272+0000 7f7fc5d9b700 1 -- 172.21.15.47:0/3023075229 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7fc8107a70 msgr2=0x7f7fc806d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:02.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.272+0000 7f7fc5d9b700 1 --2- 172.21.15.47:0/3023075229 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7fc8107a70 0x7f7fc806d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:02.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.272+0000 7f7fc5d9b700 1 -- 172.21.15.47:0/3023075229 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fc806dc10 con 0x7f7fc8101960 2024-04-17T06:00:02.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.272+0000 7f7fc659c700 1 --2- 172.21.15.47:0/3023075229 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7fc8100fa0 0x7f7fc80740f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:00:02.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.272+0000 7f7fc5d9b700 1 --2- 172.21.15.47:0/3023075229 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7fc8101960 0x7f7fc8074630 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f7fb8009fd0 tx=0x7f7fb8003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:00:02.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.272+0000 7f7fbf7fe700 1 -- 172.21.15.47:0/3023075229 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fb8027070 con 0x7f7fc8101960 2024-04-17T06:00:02.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.272+0000 7f7fbf7fe700 1 -- 172.21.15.47:0/3023075229 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fb80040d0 con 0x7f7fc8101960 2024-04-17T06:00:02.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.272+0000 7f7fbf7fe700 1 -- 172.21.15.47:0/3023075229 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fb800f460 con 0x7f7fc8101960 2024-04-17T06:00:02.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.273+0000 7f7fccb8a700 1 -- 172.21.15.47:0/3023075229 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fc806dea0 con 0x7f7fc8101960 2024-04-17T06:00:02.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.273+0000 7f7fccb8a700 1 -- 172.21.15.47:0/3023075229 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fc8071df0 con 0x7f7fc8101960 2024-04-17T06:00:02.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.274+0000 7f7fbd7fa700 1 -- 172.21.15.47:0/3023075229 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7fc8072380 con 0x7f7fc8101960 2024-04-17T06:00:02.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.276+0000 7f7fbf7fe700 1 -- 172.21.15.47:0/3023075229 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f7fb802a070 con 0x7f7fc8101960 2024-04-17T06:00:02.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.277+0000 7f7fbf7fe700 1 --2- 172.21.15.47:0/3023075229 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f7fb40a3d50 0x7f7fb40a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:02.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.277+0000 7f7fc659c700 1 --2- 172.21.15.47:0/3023075229 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f7fb40a3d50 0x7f7fb40a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:02.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.277+0000 7f7fbf7fe700 1 -- 172.21.15.47:0/3023075229 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f7fb80d1db0 con 0x7f7fc8101960 2024-04-17T06:00:02.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.277+0000 7f7fbf7fe700 1 --2- 172.21.15.47:0/3023075229 >> [v2:172.21.15.103:6808/4181363567,v1:172.21.15.103:6809/4181363567] conn(0x7f7fb40ad530 0x7f7fb40af950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:02.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.277+0000 7f7fbf7fe700 1 -- 172.21.15.47:0/3023075229 --> [v2:172.21.15.103:6808/4181363567,v1:172.21.15.103:6809/4181363567] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7fb40b0000 con 0x7f7fb40ad530 2024-04-17T06:00:02.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.277+0000 7f7fbf7fe700 1 -- 172.21.15.47:0/3023075229 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f7fb80d2290 con 0x7f7fc8101960 2024-04-17T06:00:02.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.277+0000 7f7fc659c700 1 --2- 172.21.15.47:0/3023075229 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f7fb40a3d50 0x7f7fb40a6210 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f7fb0005950 tx=0x7f7fb0005f50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:00:02.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.278+0000 7f7fc6d9d700 1 --2- 172.21.15.47:0/3023075229 >> [v2:172.21.15.103:6808/4181363567,v1:172.21.15.103:6809/4181363567] conn(0x7f7fb40ad530 0x7f7fb40af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:02.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.278+0000 7f7fc6d9d700 1 --2- 172.21.15.47:0/3023075229 >> [v2:172.21.15.103:6808/4181363567,v1:172.21.15.103:6809/4181363567] conn(0x7f7fb40ad530 0x7f7fb40af950 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:00:02.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.279+0000 7f7fbf7fe700 1 -- 172.21.15.47:0/3023075229 <== osd.5 v2:172.21.15.103:6808/4181363567 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7fb40b0000 con 0x7f7fb40ad530 2024-04-17T06:00:02.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.296+0000 7f7fbd7fa700 1 -- 172.21.15.47:0/3023075229 --> [v2:172.21.15.103:6808/4181363567,v1:172.21.15.103:6809/4181363567] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7fc80727d0 con 0x7f7fb40ad530 2024-04-17T06:00:02.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.296+0000 7f7fbf7fe700 1 -- 172.21.15.47:0/3023075229 <== osd.5 v2:172.21.15.103:6808/4181363567 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f7fc80727d0 con 0x7f7fb40ad530 2024-04-17T06:00:02.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.297+0000 7f7fbd7fa700 1 -- 172.21.15.47:0/3023075229 >> [v2:172.21.15.103:6808/4181363567,v1:172.21.15.103:6809/4181363567] conn(0x7f7fb40ad530 msgr2=0x7f7fb40af950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:02.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.297+0000 7f7fbd7fa700 1 --2- 172.21.15.47:0/3023075229 >> [v2:172.21.15.103:6808/4181363567,v1:172.21.15.103:6809/4181363567] conn(0x7f7fb40ad530 0x7f7fb40af950 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:02.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.297+0000 7f7fbd7fa700 1 -- 172.21.15.47:0/3023075229 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f7fb40a3d50 msgr2=0x7f7fb40a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:02.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.297+0000 7f7fbd7fa700 1 --2- 172.21.15.47:0/3023075229 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f7fb40a3d50 0x7f7fb40a6210 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f7fb0005950 tx=0x7f7fb0005f50 comp rx=0 tx=0).stop 2024-04-17T06:00:02.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.297+0000 7f7fbd7fa700 1 -- 172.21.15.47:0/3023075229 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7fc8101960 msgr2=0x7f7fc8074630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:02.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.297+0000 7f7fbd7fa700 1 --2- 172.21.15.47:0/3023075229 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7fc8101960 0x7f7fc8074630 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f7fb8009fd0 tx=0x7f7fb8003680 comp rx=0 tx=0).stop 2024-04-17T06:00:02.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.297+0000 7f7fc6d9d700 1 -- 172.21.15.47:0/3023075229 reap_dead start 2024-04-17T06:00:02.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.297+0000 7f7fbd7fa700 1 -- 172.21.15.47:0/3023075229 shutdown_connections 2024-04-17T06:00:02.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.297+0000 7f7fbd7fa700 1 -- 172.21.15.47:0/3023075229 >> 172.21.15.47:0/3023075229 conn(0x7f7fc8077f50 msgr2=0x7f7fc8105980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:00:02.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.297+0000 7f7fbd7fa700 1 -- 172.21.15.47:0/3023075229 shutdown_connections 2024-04-17T06:00:02.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.298+0000 7f7fbd7fa700 1 -- 172.21.15.47:0/3023075229 wait complete. 2024-04-17T06:00:02.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.476+0000 7f1b219e6700 1 -- 172.21.15.47:0/2984797815 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b1c06b990 msgr2=0x7f1b1c10f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:02.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.476+0000 7f1b219e6700 1 --2- 172.21.15.47:0/2984797815 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b1c06b990 0x7f1b1c10f900 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f1b1400b0a0 tx=0x7f1b1400b3b0 comp rx=0 tx=0).stop 2024-04-17T06:00:02.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.476+0000 7f1b219e6700 1 -- 172.21.15.47:0/2984797815 shutdown_connections 2024-04-17T06:00:02.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.476+0000 7f1b219e6700 1 --2- 172.21.15.47:0/2984797815 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b1c06b990 0x7f1b1c10f900 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:02.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.476+0000 7f1b219e6700 1 --2- 172.21.15.47:0/2984797815 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1b1c108a30 0x7f1b1c108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:02.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.476+0000 7f1b219e6700 1 --2- 172.21.15.47:0/2984797815 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1b1c06aff0 0x7f1b1c06b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:02.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.476+0000 7f1b219e6700 1 -- 172.21.15.47:0/2984797815 >> 172.21.15.47:0/2984797815 conn(0x7f1b1c0788c0 msgr2=0x7f1b1c078cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:00:02.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.476+0000 7f1b219e6700 1 -- 172.21.15.47:0/2984797815 shutdown_connections 2024-04-17T06:00:02.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.476+0000 7f1b219e6700 1 -- 172.21.15.47:0/2984797815 wait complete. 2024-04-17T06:00:02.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.476+0000 7f1b219e6700 1 Processor -- start 2024-04-17T06:00:02.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.476+0000 7f1b219e6700 1 -- start start 2024-04-17T06:00:02.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.477+0000 7f1b219e6700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b1c06aff0 0x7f1b1c199210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:02.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.477+0000 7f1b219e6700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1b1c06b990 0x7f1b1c199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:02.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.477+0000 7f1b219e6700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1b1c108a30 0x7f1b1c19da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:02.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.477+0000 7f1b219e6700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f1b1c112d20 con 0x7f1b1c06aff0 2024-04-17T06:00:02.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.477+0000 7f1b219e6700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1b1c112ba0 con 0x7f1b1c06b990 2024-04-17T06:00:02.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.477+0000 7f1b219e6700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1b1c112ea0 con 0x7f1b1c108a30 2024-04-17T06:00:02.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.477+0000 7f1b1affd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b1c06aff0 0x7f1b1c199210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:02.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.477+0000 7f1b1affd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b1c06aff0 0x7f1b1c199210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57218/0 (socket says 172.21.15.47:57218) 2024-04-17T06:00:02.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.477+0000 7f1b1affd700 1 -- 172.21.15.47:0/1289394346 learned_addr learned my addr 172.21.15.47:0/1289394346 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:00:02.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.477+0000 7f1b1a7fc700 1 --2- 172.21.15.47:0/1289394346 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1b1c06b990 0x7f1b1c199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:02.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.477+0000 7f1b1affd700 1 -- 172.21.15.47:0/1289394346 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1b1c108a30 msgr2=0x7f1b1c19da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:02.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.477+0000 7f1b1affd700 1 --2- 172.21.15.47:0/1289394346 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1b1c108a30 0x7f1b1c19da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:02.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.477+0000 7f1b1affd700 1 -- 172.21.15.47:0/1289394346 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1b1c06b990 msgr2=0x7f1b1c199750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:02.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.477+0000 7f1b1affd700 1 --2- 172.21.15.47:0/1289394346 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1b1c06b990 0x7f1b1c199750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:02.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.477+0000 7f1b1affd700 1 -- 172.21.15.47:0/1289394346 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b1c19e1d0 con 0x7f1b1c06aff0 2024-04-17T06:00:02.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.477+0000 7f1b1affd700 1 --2- 172.21.15.47:0/1289394346 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b1c06aff0 0x7f1b1c199210 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c00d980 tx=0x7f1b0c00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:00:02.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.477+0000 7f1b209e4700 1 -- 172.21.15.47:0/1289394346 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b0c017070 con 0x7f1b1c06aff0 2024-04-17T06:00:02.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.477+0000 7f1b209e4700 1 -- 172.21.15.47:0/1289394346 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b0c004d10 con 0x7f1b1c06aff0 2024-04-17T06:00:02.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.477+0000 7f1b219e6700 1 -- 172.21.15.47:0/1289394346 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b1c19e4c0 con 0x7f1b1c06aff0 2024-04-17T06:00:02.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.477+0000 7f1b209e4700 1 -- 172.21.15.47:0/1289394346 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b0c0055c0 con 0x7f1b1c06aff0 2024-04-17T06:00:02.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.477+0000 7f1b219e6700 1 -- 172.21.15.47:0/1289394346 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b1c1a5d40 con 0x7f1b1c06aff0 2024-04-17T06:00:02.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.478+0000 7f1b219e6700 1 -- 172.21.15.47:0/1289394346 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1b1c0631e0 con 0x7f1b1c06aff0 2024-04-17T06:00:02.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.480+0000 7f1b209e4700 1 -- 172.21.15.47:0/1289394346 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f1b0c004750 con 0x7f1b1c06aff0 2024-04-17T06:00:02.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.481+0000 7f1b209e4700 1 --2- 172.21.15.47:0/1289394346 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f1b040a3cd0 0x7f1b040a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:02.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.481+0000 7f1b1a7fc700 1 --2- 172.21.15.47:0/1289394346 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f1b040a3cd0 0x7f1b040a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:02.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.481+0000 7f1b209e4700 1 -- 172.21.15.47:0/1289394346 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f1b0c0d0820 con 0x7f1b1c06aff0 2024-04-17T06:00:02.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.481+0000 7f1b209e4700 1 --2- 172.21.15.47:0/1289394346 >> [v2:172.21.15.103:6800/494676216,v1:172.21.15.103:6801/494676216] conn(0x7f1b040ad4b0 0x7f1b040af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:02.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.481+0000 7f1b209e4700 1 -- 172.21.15.47:0/1289394346 --> [v2:172.21.15.103:6800/494676216,v1:172.21.15.103:6801/494676216] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1b040aff80 con 0x7f1b040ad4b0 2024-04-17T06:00:02.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.481+0000 7f1b209e4700 1 -- 172.21.15.47:0/1289394346 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f1b0c0d0d00 con 0x7f1b1c06aff0 2024-04-17T06:00:02.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.481+0000 7f1b1a7fc700 1 --2- 172.21.15.47:0/1289394346 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f1b040a3cd0 0x7f1b040a6190 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1b10005950 tx=0x7f1b100058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:00:02.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.482+0000 7f1b1b7fe700 1 --2- 172.21.15.47:0/1289394346 >> [v2:172.21.15.103:6800/494676216,v1:172.21.15.103:6801/494676216] conn(0x7f1b040ad4b0 0x7f1b040af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:02.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.482+0000 7f1b1b7fe700 1 --2- 172.21.15.47:0/1289394346 >> [v2:172.21.15.103:6800/494676216,v1:172.21.15.103:6801/494676216] conn(0x7f1b040ad4b0 0x7f1b040af8d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:00:02.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.483+0000 7f1b209e4700 1 -- 172.21.15.47:0/1289394346 <== osd.2 v2:172.21.15.103:6800/494676216 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1b040aff80 con 0x7f1b040ad4b0 2024-04-17T06:00:02.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.499+0000 7f1b219e6700 1 -- 172.21.15.47:0/1289394346 --> [v2:172.21.15.103:6800/494676216,v1:172.21.15.103:6801/494676216] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1b1c063660 con 0x7f1b040ad4b0 2024-04-17T06:00:02.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.499+0000 7f1b209e4700 1 -- 172.21.15.47:0/1289394346 <== osd.2 v2:172.21.15.103:6800/494676216 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f1b1c063660 con 0x7f1b040ad4b0 2024-04-17T06:00:02.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.500+0000 7f1b219e6700 1 -- 172.21.15.47:0/1289394346 >> [v2:172.21.15.103:6800/494676216,v1:172.21.15.103:6801/494676216] conn(0x7f1b040ad4b0 msgr2=0x7f1b040af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:02.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.500+0000 7f1b219e6700 1 --2- 172.21.15.47:0/1289394346 >> [v2:172.21.15.103:6800/494676216,v1:172.21.15.103:6801/494676216] conn(0x7f1b040ad4b0 0x7f1b040af8d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:02.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.500+0000 7f1b219e6700 1 -- 172.21.15.47:0/1289394346 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f1b040a3cd0 msgr2=0x7f1b040a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:02.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.500+0000 7f1b219e6700 1 --2- 172.21.15.47:0/1289394346 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f1b040a3cd0 0x7f1b040a6190 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1b10005950 tx=0x7f1b100058e0 comp rx=0 tx=0).stop 2024-04-17T06:00:02.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.500+0000 7f1b219e6700 1 -- 172.21.15.47:0/1289394346 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b1c06aff0 msgr2=0x7f1b1c199210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:02.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.500+0000 7f1b219e6700 1 --2- 172.21.15.47:0/1289394346 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b1c06aff0 0x7f1b1c199210 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c00d980 tx=0x7f1b0c00dd40 comp rx=0 tx=0).stop 2024-04-17T06:00:02.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.500+0000 7f1b1b7fe700 1 -- 172.21.15.47:0/1289394346 reap_dead start 2024-04-17T06:00:02.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.500+0000 7f1b219e6700 1 -- 172.21.15.47:0/1289394346 shutdown_connections 2024-04-17T06:00:02.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.500+0000 7f1b219e6700 1 -- 172.21.15.47:0/1289394346 >> 172.21.15.47:0/1289394346 conn(0x7f1b1c0788c0 msgr2=0x7f1b1c0ff990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:00:02.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.500+0000 7f1b219e6700 1 -- 172.21.15.47:0/1289394346 shutdown_connections 2024-04-17T06:00:02.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:02.500+0000 7f1b219e6700 1 -- 172.21.15.47:0/1289394346 wait complete. 2024-04-17T06:00:02.739 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T06:00:02.962 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T06:00:02.966 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T06:00:03.404 INFO:teuthology.orchestra.run.smithi047.stdout:77309411397 2024-04-17T06:00:03.405 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 565db2d8-fc7e-11ee-bc91-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-04-17T06:00:04.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:00:03 smithi103 ceph-mon[36243]: pgmap v310: 97 pgs: 97 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-17T06:00:04.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:00:03 smithi098 ceph-mon[36516]: pgmap v310: 97 pgs: 97 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-17T06:00:04.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:00:03 smithi047 ceph-mon[32223]: pgmap v310: 97 pgs: 97 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-17T06:00:06.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:00:05 smithi103 ceph-mon[36243]: pgmap v311: 97 pgs: 97 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-17T06:00:06.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:00:05 smithi098 ceph-mon[36516]: pgmap v311: 97 pgs: 97 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-17T06:00:06.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:00:05 smithi047 ceph-mon[32223]: pgmap v311: 97 pgs: 97 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-17T06:00:06.908 INFO:teuthology.orchestra.run.smithi047.stdout:167503724597 2024-04-17T06:00:06.908 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 565db2d8-fc7e-11ee-bc91-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-04-17T06:00:08.007 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:00:07 smithi103 ceph-mon[36243]: pgmap v312: 97 pgs: 97 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-17T06:00:08.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:00:07 smithi098 ceph-mon[36516]: pgmap v312: 97 pgs: 97 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-17T06:00:08.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:00:07 smithi047 ceph-mon[32223]: pgmap v312: 97 pgs: 97 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-17T06:00:08.268 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T06:00:08.269 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T06:00:08.269 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T06:00:08.270 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T06:00:08.272 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T06:00:08.275 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T06:00:08.882 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T06:00:08.883 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T06:00:09.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:09.823+0000 7fb6c36b9700 1 -- 172.21.15.47:0/3546729213 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6bc108a40 msgr2=0x7fb6bc108e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:09.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:09.823+0000 7fb6c36b9700 1 --2- 172.21.15.47:0/3546729213 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6bc108a40 0x7fb6bc108e20 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fb6b8009990 tx=0x7fb6b8009ca0 comp rx=0 tx=0).stop 2024-04-17T06:00:09.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:09.823+0000 7fb6c36b9700 1 -- 172.21.15.47:0/3546729213 shutdown_connections 2024-04-17T06:00:09.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:09.823+0000 7fb6c36b9700 1 --2- 172.21.15.47:0/3546729213 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb6bc06b100 0x7fb6bc10f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:09.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:09.823+0000 7fb6c36b9700 1 --2- 172.21.15.47:0/3546729213 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6bc108a40 0x7fb6bc108e20 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:09.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:09.823+0000 7fb6c36b9700 1 --2- 172.21.15.47:0/3546729213 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb6bc06a760 0x7fb6bc06abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:09.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:09.823+0000 7fb6c36b9700 1 -- 172.21.15.47:0/3546729213 >> 172.21.15.47:0/3546729213 conn(0x7fb6bc078900 msgr2=0x7fb6bc078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:00:09.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:09.823+0000 7fb6c36b9700 1 -- 172.21.15.47:0/3546729213 shutdown_connections 2024-04-17T06:00:09.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:09.823+0000 7fb6c36b9700 1 -- 172.21.15.47:0/3546729213 wait complete. 2024-04-17T06:00:09.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:09.824+0000 7fb6c36b9700 1 Processor -- start 2024-04-17T06:00:09.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:09.824+0000 7fb6c36b9700 1 -- start start 2024-04-17T06:00:09.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:09.824+0000 7fb6c36b9700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb6bc06a760 0x7fb6bc199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:09.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:09.824+0000 7fb6c36b9700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb6bc06b100 0x7fb6bc1997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:09.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:09.824+0000 7fb6c36b9700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6bc108a40 0x7fb6bc19dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:09.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:09.824+0000 7fb6c36b9700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb6bc112d30 con 0x7fb6bc108a40 2024-04-17T06:00:09.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:09.824+0000 7fb6c36b9700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb6bc112bb0 con 0x7fb6bc06a760 2024-04-17T06:00:09.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:09.824+0000 7fb6c36b9700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb6bc112eb0 con 0x7fb6bc06b100 2024-04-17T06:00:09.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:09.824+0000 7fb6c1c56700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6bc108a40 0x7fb6bc19dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:09.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:09.824+0000 7fb6c1c56700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6bc108a40 0x7fb6bc19dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54452/0 (socket says 172.21.15.47:54452) 2024-04-17T06:00:09.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:09.824+0000 7fb6c1c56700 1 -- 172.21.15.47:0/2992135790 learned_addr learned my addr 172.21.15.47:0/2992135790 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:00:09.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:09.824+0000 7fb6c0c54700 1 --2- 172.21.15.47:0/2992135790 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb6bc06b100 0x7fb6bc1997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:09.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:09.825+0000 7fb6c1455700 1 --2- 172.21.15.47:0/2992135790 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb6bc06a760 0x7fb6bc199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:09.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:09.825+0000 7fb6c1c56700 1 -- 172.21.15.47:0/2992135790 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb6bc06b100 msgr2=0x7fb6bc1997a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:09.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:09.825+0000 7fb6c1c56700 1 --2- 172.21.15.47:0/2992135790 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb6bc06b100 0x7fb6bc1997a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:09.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:09.825+0000 7fb6c1c56700 1 -- 172.21.15.47:0/2992135790 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb6bc06a760 msgr2=0x7fb6bc199260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:09.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:09.825+0000 7fb6c1c56700 1 --2- 172.21.15.47:0/2992135790 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb6bc06a760 0x7fb6bc199260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:09.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:09.825+0000 7fb6c1c56700 1 -- 172.21.15.47:0/2992135790 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6bc19e220 con 0x7fb6bc108a40 2024-04-17T06:00:09.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:09.825+0000 7fb6c1455700 1 --2- 172.21.15.47:0/2992135790 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb6bc06a760 0x7fb6bc199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:00:09.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:09.825+0000 7fb6c1c56700 1 --2- 172.21.15.47:0/2992135790 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6bc108a40 0x7fb6bc19dae0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fb6b400ef30 tx=0x7fb6b400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:00:09.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:09.825+0000 7fb6ae7fc700 1 -- 172.21.15.47:0/2992135790 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6b40092e0 con 0x7fb6bc108a40 2024-04-17T06:00:09.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:09.825+0000 7fb6c36b9700 1 -- 172.21.15.47:0/2992135790 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6bc19e4b0 con 0x7fb6bc108a40 2024-04-17T06:00:09.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:09.825+0000 7fb6ae7fc700 1 -- 172.21.15.47:0/2992135790 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6b4004bb0 con 0x7fb6bc108a40 2024-04-17T06:00:09.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:09.825+0000 7fb6ae7fc700 1 -- 172.21.15.47:0/2992135790 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6b4004280 con 0x7fb6bc108a40 2024-04-17T06:00:09.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:09.825+0000 7fb6c36b9700 1 -- 172.21.15.47:0/2992135790 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6bc1a5d90 con 0x7fb6bc108a40 2024-04-17T06:00:09.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:09.826+0000 7fb6c36b9700 1 -- 172.21.15.47:0/2992135790 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6bc069740 con 0x7fb6bc108a40 2024-04-17T06:00:09.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:09.827+0000 7fb6ae7fc700 1 -- 172.21.15.47:0/2992135790 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fb6b401b030 con 0x7fb6bc108a40 2024-04-17T06:00:09.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:09.828+0000 7fb6ae7fc700 1 --2- 172.21.15.47:0/2992135790 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fb6a80a3cd0 0x7fb6a80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:09.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:09.828+0000 7fb6c1455700 1 --2- 172.21.15.47:0/2992135790 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fb6a80a3cd0 0x7fb6a80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:09.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:09.828+0000 7fb6ae7fc700 1 -- 172.21.15.47:0/2992135790 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fb6b40cfe50 con 0x7fb6bc108a40 2024-04-17T06:00:09.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:09.829+0000 7fb6c1455700 1 --2- 172.21.15.47:0/2992135790 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fb6a80a3cd0 0x7fb6a80a6190 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb6b800b2e0 tx=0x7fb6b801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:00:09.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:09.833+0000 7fb6ae7fc700 1 -- 172.21.15.47:0/2992135790 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fb6b4022020 con 0x7fb6bc108a40 2024-04-17T06:00:10.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:00:09 smithi103 ceph-mon[36243]: pgmap v313: 97 pgs: 97 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-04-17T06:00:10.038 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:00:09 smithi047 ceph-mon[32223]: pgmap v313: 97 pgs: 97 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-04-17T06:00:10.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:10.047+0000 7fb6c36b9700 1 -- 172.21.15.47:0/2992135790 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fb6bc02df90 con 0x7fb6bc108a40 2024-04-17T06:00:10.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:10.048+0000 7fb6ae7fc700 1 -- 172.21.15.47:0/2992135790 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fb6b4010040 con 0x7fb6bc108a40 2024-04-17T06:00:10.046 INFO:teuthology.orchestra.run.smithi047.stdout:201863462962 2024-04-17T06:00:10.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:10.051+0000 7fb6c36b9700 1 -- 172.21.15.47:0/2992135790 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fb6a80a3cd0 msgr2=0x7fb6a80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:10.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:10.051+0000 7fb6c36b9700 1 --2- 172.21.15.47:0/2992135790 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fb6a80a3cd0 0x7fb6a80a6190 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb6b800b2e0 tx=0x7fb6b801a040 comp rx=0 tx=0).stop 2024-04-17T06:00:10.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:10.051+0000 7fb6c36b9700 1 -- 172.21.15.47:0/2992135790 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6bc108a40 msgr2=0x7fb6bc19dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:10.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:10.051+0000 7fb6c36b9700 1 --2- 172.21.15.47:0/2992135790 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6bc108a40 0x7fb6bc19dae0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fb6b400ef30 tx=0x7fb6b400c3a0 comp rx=0 tx=0).stop 2024-04-17T06:00:10.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:10.051+0000 7fb6c36b9700 1 -- 172.21.15.47:0/2992135790 shutdown_connections 2024-04-17T06:00:10.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:10.051+0000 7fb6c36b9700 1 --2- 172.21.15.47:0/2992135790 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fb6a80a3cd0 0x7fb6a80a6190 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:10.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:10.051+0000 7fb6c36b9700 1 --2- 172.21.15.47:0/2992135790 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6bc108a40 0x7fb6bc19dae0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:10.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:10.051+0000 7fb6c36b9700 1 --2- 172.21.15.47:0/2992135790 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb6bc06a760 0x7fb6bc199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:10.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:10.051+0000 7fb6c36b9700 1 --2- 172.21.15.47:0/2992135790 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb6bc06b100 0x7fb6bc1997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:10.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:10.051+0000 7fb6c36b9700 1 -- 172.21.15.47:0/2992135790 >> 172.21.15.47:0/2992135790 conn(0x7fb6bc078900 msgr2=0x7fb6bc0ffa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:00:10.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:10.051+0000 7fb6c36b9700 1 -- 172.21.15.47:0/2992135790 shutdown_connections 2024-04-17T06:00:10.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:10.051+0000 7fb6c36b9700 1 -- 172.21.15.47:0/2992135790 wait complete. 2024-04-17T06:00:10.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:00:09 smithi098 ceph-mon[36516]: pgmap v313: 97 pgs: 97 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-04-17T06:00:10.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:00:10 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/2992135790' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-17T06:00:11.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:00:10 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/2992135790' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-17T06:00:11.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:00:10 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/2992135790' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-17T06:00:11.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:00:11 smithi047 ceph-mon[32223]: pgmap v314: 97 pgs: 97 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-04-17T06:00:12.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:00:11 smithi103 ceph-mon[36243]: pgmap v314: 97 pgs: 97 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-04-17T06:00:12.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:00:11 smithi098 ceph-mon[36516]: pgmap v314: 97 pgs: 97 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-04-17T06:00:12.319 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462959 got 201863462962 for osd.6 2024-04-17T06:00:12.320 DEBUG:teuthology.parallel:result is None 2024-04-17T06:00:12.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:12.517+0000 7f3e64750700 1 -- 172.21.15.47:0/2897184332 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3e5c108a60 msgr2=0x7f3e5c108e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:12.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:12.517+0000 7f3e64750700 1 --2- 172.21.15.47:0/2897184332 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3e5c108a60 0x7f3e5c108e40 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f3e58009990 tx=0x7f3e58009ca0 comp rx=0 tx=0).stop 2024-04-17T06:00:12.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:12.517+0000 7f3e64750700 1 -- 172.21.15.47:0/2897184332 shutdown_connections 2024-04-17T06:00:12.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:12.517+0000 7f3e64750700 1 --2- 172.21.15.47:0/2897184332 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3e5c103460 0x7f3e5c10f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:12.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:12.517+0000 7f3e64750700 1 --2- 172.21.15.47:0/2897184332 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3e5c108a60 0x7f3e5c108e40 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:12.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:12.517+0000 7f3e64750700 1 --2- 172.21.15.47:0/2897184332 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3e5c102aa0 0x7f3e5c102f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:12.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:12.517+0000 7f3e64750700 1 -- 172.21.15.47:0/2897184332 >> 172.21.15.47:0/2897184332 conn(0x7f3e5c079bb0 msgr2=0x7f3e5c079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:00:12.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:12.517+0000 7f3e64750700 1 -- 172.21.15.47:0/2897184332 shutdown_connections 2024-04-17T06:00:12.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:12.518+0000 7f3e64750700 1 -- 172.21.15.47:0/2897184332 wait complete. 2024-04-17T06:00:12.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:12.518+0000 7f3e64750700 1 Processor -- start 2024-04-17T06:00:12.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:12.518+0000 7f3e64750700 1 -- start start 2024-04-17T06:00:12.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:12.518+0000 7f3e64750700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3e5c102aa0 0x7f3e5c199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:12.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:12.518+0000 7f3e64750700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3e5c103460 0x7f3e5c199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:12.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:12.518+0000 7f3e64750700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3e5c108a60 0x7f3e5c19da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:12.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:12.518+0000 7f3e64750700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f3e5c112da0 con 0x7f3e5c108a60 2024-04-17T06:00:12.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:12.518+0000 7f3e64750700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3e5c112c20 con 0x7f3e5c102aa0 2024-04-17T06:00:12.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:12.518+0000 7f3e64750700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3e5c112f20 con 0x7f3e5c103460 2024-04-17T06:00:12.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:12.518+0000 7f3e62ced700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3e5c108a60 0x7f3e5c19da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:12.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:12.518+0000 7f3e62ced700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3e5c108a60 0x7f3e5c19da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54466/0 (socket says 172.21.15.47:54466) 2024-04-17T06:00:12.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:12.518+0000 7f3e62ced700 1 -- 172.21.15.47:0/2278024038 learned_addr learned my addr 172.21.15.47:0/2278024038 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:00:12.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:12.519+0000 7f3e61ceb700 1 --2- 172.21.15.47:0/2278024038 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3e5c103460 0x7f3e5c199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:12.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:12.519+0000 7f3e624ec700 1 --2- 172.21.15.47:0/2278024038 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3e5c102aa0 0x7f3e5c199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:12.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:12.519+0000 7f3e62ced700 1 -- 172.21.15.47:0/2278024038 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3e5c103460 msgr2=0x7f3e5c199670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:12.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:12.519+0000 7f3e62ced700 1 --2- 172.21.15.47:0/2278024038 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3e5c103460 0x7f3e5c199670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:12.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:12.519+0000 7f3e62ced700 1 -- 172.21.15.47:0/2278024038 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3e5c102aa0 msgr2=0x7f3e5c199130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:12.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:12.519+0000 7f3e62ced700 1 --2- 172.21.15.47:0/2278024038 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3e5c102aa0 0x7f3e5c199130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:12.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:12.519+0000 7f3e62ced700 1 -- 172.21.15.47:0/2278024038 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e5c19e140 con 0x7f3e5c108a60 2024-04-17T06:00:12.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:12.519+0000 7f3e624ec700 1 --2- 172.21.15.47:0/2278024038 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3e5c102aa0 0x7f3e5c199130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:00:12.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:12.519+0000 7f3e62ced700 1 --2- 172.21.15.47:0/2278024038 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3e5c108a60 0x7f3e5c19da00 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f3e54007d70 tx=0x7f3e5400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:00:12.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:12.519+0000 7f3e537fe700 1 -- 172.21.15.47:0/2278024038 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e54016020 con 0x7f3e5c108a60 2024-04-17T06:00:12.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:12.519+0000 7f3e537fe700 1 -- 172.21.15.47:0/2278024038 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e5400ad40 con 0x7f3e5c108a60 2024-04-17T06:00:12.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:12.519+0000 7f3e64750700 1 -- 172.21.15.47:0/2278024038 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e5c19e430 con 0x7f3e5c108a60 2024-04-17T06:00:12.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:12.519+0000 7f3e537fe700 1 -- 172.21.15.47:0/2278024038 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e54011600 con 0x7f3e5c108a60 2024-04-17T06:00:12.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:12.519+0000 7f3e64750700 1 -- 172.21.15.47:0/2278024038 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e5c1a5cb0 con 0x7f3e5c108a60 2024-04-17T06:00:12.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:12.520+0000 7f3e64750700 1 -- 172.21.15.47:0/2278024038 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e5c069740 con 0x7f3e5c108a60 2024-04-17T06:00:12.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:12.521+0000 7f3e537fe700 1 -- 172.21.15.47:0/2278024038 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f3e54019020 con 0x7f3e5c108a60 2024-04-17T06:00:12.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:12.522+0000 7f3e537fe700 1 --2- 172.21.15.47:0/2278024038 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f3e480a3d50 0x7f3e480a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:12.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:12.522+0000 7f3e624ec700 1 --2- 172.21.15.47:0/2278024038 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f3e480a3d50 0x7f3e480a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:12.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:12.522+0000 7f3e537fe700 1 -- 172.21.15.47:0/2278024038 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f3e540d0580 con 0x7f3e5c108a60 2024-04-17T06:00:12.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:12.522+0000 7f3e624ec700 1 --2- 172.21.15.47:0/2278024038 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f3e480a3d50 0x7f3e480a6210 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f3e5800b5c0 tx=0x7f3e58005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:00:12.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:12.526+0000 7f3e537fe700 1 -- 172.21.15.47:0/2278024038 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f3e54020020 con 0x7f3e5c108a60 2024-04-17T06:00:12.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:12.743+0000 7f3e64750700 1 -- 172.21.15.47:0/2278024038 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f3e5c1a5ee0 con 0x7f3e5c108a60 2024-04-17T06:00:12.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:12.743+0000 7f3e537fe700 1 -- 172.21.15.47:0/2278024038 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f3e5409d0c0 con 0x7f3e5c108a60 2024-04-17T06:00:12.741 INFO:teuthology.orchestra.run.smithi047.stdout:257698037800 2024-04-17T06:00:12.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:12.745+0000 7f3e64750700 1 -- 172.21.15.47:0/2278024038 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f3e480a3d50 msgr2=0x7f3e480a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:12.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:12.745+0000 7f3e64750700 1 --2- 172.21.15.47:0/2278024038 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f3e480a3d50 0x7f3e480a6210 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f3e5800b5c0 tx=0x7f3e58005fb0 comp rx=0 tx=0).stop 2024-04-17T06:00:12.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:12.745+0000 7f3e64750700 1 -- 172.21.15.47:0/2278024038 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3e5c108a60 msgr2=0x7f3e5c19da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:12.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:12.745+0000 7f3e64750700 1 --2- 172.21.15.47:0/2278024038 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3e5c108a60 0x7f3e5c19da00 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f3e54007d70 tx=0x7f3e5400a340 comp rx=0 tx=0).stop 2024-04-17T06:00:12.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:12.745+0000 7f3e64750700 1 -- 172.21.15.47:0/2278024038 shutdown_connections 2024-04-17T06:00:12.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:12.745+0000 7f3e64750700 1 --2- 172.21.15.47:0/2278024038 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f3e480a3d50 0x7f3e480a6210 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:12.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:12.745+0000 7f3e64750700 1 --2- 172.21.15.47:0/2278024038 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3e5c108a60 0x7f3e5c19da00 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:12.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:12.745+0000 7f3e64750700 1 --2- 172.21.15.47:0/2278024038 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3e5c102aa0 0x7f3e5c199130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:12.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:12.745+0000 7f3e64750700 1 --2- 172.21.15.47:0/2278024038 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3e5c103460 0x7f3e5c199670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:12.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:12.745+0000 7f3e64750700 1 -- 172.21.15.47:0/2278024038 >> 172.21.15.47:0/2278024038 conn(0x7f3e5c079bb0 msgr2=0x7f3e5c101570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:00:12.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:12.746+0000 7f3e64750700 1 -- 172.21.15.47:0/2278024038 shutdown_connections 2024-04-17T06:00:12.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:12.746+0000 7f3e64750700 1 -- 172.21.15.47:0/2278024038 wait complete. 2024-04-17T06:00:12.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:00:12 smithi047 ceph-mon[32223]: pgmap v315: 97 pgs: 97 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-17T06:00:13.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:00:12 smithi103 ceph-mon[36243]: pgmap v315: 97 pgs: 97 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-17T06:00:13.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:00:12 smithi098 ceph-mon[36516]: pgmap v315: 97 pgs: 97 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-17T06:00:13.820 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:00:13 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/2278024038' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-17T06:00:13.820 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:00:13 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T06:00:13.820 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:00:13 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-04-17T06:00:13.820 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:00:13 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2024-04-17T06:00:13.820 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:00:13 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-04-17T06:00:13.820 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:00:13 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-04-17T06:00:13.820 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:00:13 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.b", "id": [3, 9]}]: dispatch 2024-04-17T06:00:13.820 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:00:13 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]: dispatch 2024-04-17T06:00:13.820 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:00:13 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]: dispatch 2024-04-17T06:00:13.821 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:00:13 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2024-04-17T06:00:14.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:00:13 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/2278024038' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-17T06:00:14.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:00:13 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T06:00:14.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:00:13 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-04-17T06:00:14.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:00:13 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2024-04-17T06:00:14.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:00:13 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-04-17T06:00:14.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:00:13 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-04-17T06:00:14.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:00:13 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.b", "id": [3, 9]}]: dispatch 2024-04-17T06:00:14.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:00:13 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]: dispatch 2024-04-17T06:00:14.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:00:13 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]: dispatch 2024-04-17T06:00:14.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:00:13 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2024-04-17T06:00:14.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:00:13 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/2278024038' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-17T06:00:14.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:00:13 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T06:00:14.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:00:13 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-04-17T06:00:14.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:00:13 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2024-04-17T06:00:14.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:00:13 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-04-17T06:00:14.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:00:13 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-04-17T06:00:14.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:00:13 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.b", "id": [3, 9]}]: dispatch 2024-04-17T06:00:14.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:00:13 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]: dispatch 2024-04-17T06:00:14.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:00:13 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]: dispatch 2024-04-17T06:00:14.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:00:13 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2024-04-17T06:00:14.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:14.524+0000 7f70172ac700 1 -- 172.21.15.47:0/395758870 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7010102aa0 msgr2=0x7f7010102f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:14.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:14.524+0000 7f70172ac700 1 --2- 172.21.15.47:0/395758870 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7010102aa0 0x7f7010102f20 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f7000009990 tx=0x7f7000009ca0 comp rx=0 tx=0).stop 2024-04-17T06:00:14.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:14.524+0000 7f70172ac700 1 -- 172.21.15.47:0/395758870 shutdown_connections 2024-04-17T06:00:14.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:14.524+0000 7f70172ac700 1 --2- 172.21.15.47:0/395758870 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7010103460 0x7f701010f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:14.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:14.524+0000 7f70172ac700 1 --2- 172.21.15.47:0/395758870 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7010108a60 0x7f7010108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:14.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:14.524+0000 7f70172ac700 1 --2- 172.21.15.47:0/395758870 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7010102aa0 0x7f7010102f20 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:14.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:14.524+0000 7f70172ac700 1 -- 172.21.15.47:0/395758870 >> 172.21.15.47:0/395758870 conn(0x7f7010079bb0 msgr2=0x7f7010079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:00:14.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:14.524+0000 7f70172ac700 1 -- 172.21.15.47:0/395758870 shutdown_connections 2024-04-17T06:00:14.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:14.525+0000 7f70172ac700 1 -- 172.21.15.47:0/395758870 wait complete. 2024-04-17T06:00:14.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:14.525+0000 7f70172ac700 1 Processor -- start 2024-04-17T06:00:14.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:14.525+0000 7f70172ac700 1 -- start start 2024-04-17T06:00:14.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:14.525+0000 7f70172ac700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7010102aa0 0x7f701019b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:14.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:14.525+0000 7f70172ac700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7010103460 0x7f701019b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:14.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:14.525+0000 7f70172ac700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7010108a60 0x7f701019fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:14.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:14.525+0000 7f70172ac700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f7010112d60 con 0x7f7010108a60 2024-04-17T06:00:14.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:14.525+0000 7f70172ac700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f7010112be0 con 0x7f7010102aa0 2024-04-17T06:00:14.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:14.525+0000 7f70172ac700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7010112ee0 con 0x7f7010103460 2024-04-17T06:00:14.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:14.525+0000 7f7015849700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7010108a60 0x7f701019fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:14.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:14.526+0000 7f7015048700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7010102aa0 0x7f701019b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:14.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:14.526+0000 7f7015849700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7010108a60 0x7f701019fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54488/0 (socket says 172.21.15.47:54488) 2024-04-17T06:00:14.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:14.526+0000 7f7014847700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7010103460 0x7f701019b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:14.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:14.526+0000 7f7015048700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7010102aa0 0x7f701019b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.47:42150/0 (socket says 172.21.15.47:42150) 2024-04-17T06:00:14.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:14.526+0000 7f7015849700 1 -- 172.21.15.47:0/1979634291 learned_addr learned my addr 172.21.15.47:0/1979634291 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:00:14.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:14.526+0000 7f7015849700 1 -- 172.21.15.47:0/1979634291 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7010103460 msgr2=0x7f701019b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:14.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:14.526+0000 7f7015849700 1 --2- 172.21.15.47:0/1979634291 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7010103460 0x7f701019b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:14.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:14.526+0000 7f7015849700 1 -- 172.21.15.47:0/1979634291 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7010102aa0 msgr2=0x7f701019b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:14.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:14.526+0000 7f7015849700 1 --2- 172.21.15.47:0/1979634291 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7010102aa0 0x7f701019b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:14.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:14.526+0000 7f7015849700 1 -- 172.21.15.47:0/1979634291 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70101a03e0 con 0x7f7010108a60 2024-04-17T06:00:14.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:14.526+0000 7f7015849700 1 --2- 172.21.15.47:0/1979634291 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7010108a60 0x7f701019fca0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f7008007d70 tx=0x7f700800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:00:14.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:14.526+0000 7f70067fc700 1 -- 172.21.15.47:0/1979634291 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7008016020 con 0x7f7010108a60 2024-04-17T06:00:14.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:14.526+0000 7f70067fc700 1 -- 172.21.15.47:0/1979634291 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f700800ac90 con 0x7f7010108a60 2024-04-17T06:00:14.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:14.526+0000 7f70067fc700 1 -- 172.21.15.47:0/1979634291 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70080115c0 con 0x7f7010108a60 2024-04-17T06:00:14.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:14.526+0000 7f70172ac700 1 -- 172.21.15.47:0/1979634291 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70101a06d0 con 0x7f7010108a60 2024-04-17T06:00:14.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:14.526+0000 7f70172ac700 1 -- 172.21.15.47:0/1979634291 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70101a7f50 con 0x7f7010108a60 2024-04-17T06:00:14.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:14.527+0000 7f70172ac700 1 -- 172.21.15.47:0/1979634291 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7010069740 con 0x7f7010108a60 2024-04-17T06:00:14.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:14.528+0000 7f70067fc700 1 -- 172.21.15.47:0/1979634291 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f7008019020 con 0x7f7010108a60 2024-04-17T06:00:14.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:14.529+0000 7f70067fc700 1 --2- 172.21.15.47:0/1979634291 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f6ffc0a3cd0 0x7f6ffc0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:14.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:14.529+0000 7f7015048700 1 --2- 172.21.15.47:0/1979634291 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f6ffc0a3cd0 0x7f6ffc0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:14.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:14.529+0000 7f70067fc700 1 -- 172.21.15.47:0/1979634291 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f70080d0380 con 0x7f7010108a60 2024-04-17T06:00:14.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:14.530+0000 7f7015048700 1 --2- 172.21.15.47:0/1979634291 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f6ffc0a3cd0 0x7f6ffc0a6190 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f700c0045b0 tx=0x7f700c00b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:00:14.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:14.534+0000 7f70067fc700 1 -- 172.21.15.47:0/1979634291 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f7008020020 con 0x7f7010108a60 2024-04-17T06:00:14.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:14.752+0000 7f70172ac700 1 -- 172.21.15.47:0/1979634291 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f701019c730 con 0x7f7010108a60 2024-04-17T06:00:14.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:14.752+0000 7f70067fc700 1 -- 172.21.15.47:0/1979634291 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f701019c730 con 0x7f7010108a60 2024-04-17T06:00:14.750 INFO:teuthology.orchestra.run.smithi047.stdout:347892351001 2024-04-17T06:00:14.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:14.754+0000 7f70172ac700 1 -- 172.21.15.47:0/1979634291 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f6ffc0a3cd0 msgr2=0x7f6ffc0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:14.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:14.754+0000 7f70172ac700 1 --2- 172.21.15.47:0/1979634291 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f6ffc0a3cd0 0x7f6ffc0a6190 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f700c0045b0 tx=0x7f700c00b3c0 comp rx=0 tx=0).stop 2024-04-17T06:00:14.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:14.754+0000 7f70172ac700 1 -- 172.21.15.47:0/1979634291 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7010108a60 msgr2=0x7f701019fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:14.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:14.754+0000 7f70172ac700 1 --2- 172.21.15.47:0/1979634291 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7010108a60 0x7f701019fca0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f7008007d70 tx=0x7f700800a340 comp rx=0 tx=0).stop 2024-04-17T06:00:14.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:14.754+0000 7f70172ac700 1 -- 172.21.15.47:0/1979634291 shutdown_connections 2024-04-17T06:00:14.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:14.754+0000 7f70172ac700 1 --2- 172.21.15.47:0/1979634291 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f6ffc0a3cd0 0x7f6ffc0a6190 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:14.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:14.754+0000 7f70172ac700 1 --2- 172.21.15.47:0/1979634291 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7010108a60 0x7f701019fca0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:14.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:14.754+0000 7f70172ac700 1 --2- 172.21.15.47:0/1979634291 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7010102aa0 0x7f701019b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:14.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:14.754+0000 7f70172ac700 1 --2- 172.21.15.47:0/1979634291 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7010103460 0x7f701019b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:14.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:14.754+0000 7f70172ac700 1 -- 172.21.15.47:0/1979634291 >> 172.21.15.47:0/1979634291 conn(0x7f7010079bb0 msgr2=0x7f701010de40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:00:14.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:14.755+0000 7f70172ac700 1 -- 172.21.15.47:0/1979634291 shutdown_connections 2024-04-17T06:00:14.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:14.755+0000 7f70172ac700 1 -- 172.21.15.47:0/1979634291 wait complete. 2024-04-17T06:00:14.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:14.765+0000 7f0c68c6f700 1 -- 172.21.15.47:0/639543918 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c6406a760 msgr2=0x7f0c6406abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:14.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:14.765+0000 7f0c68c6f700 1 --2- 172.21.15.47:0/639543918 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c6406a760 0x7f0c6406abc0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f0c58009990 tx=0x7f0c58009ca0 comp rx=0 tx=0).stop 2024-04-17T06:00:14.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:14.766+0000 7f0c68c6f700 1 -- 172.21.15.47:0/639543918 shutdown_connections 2024-04-17T06:00:14.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:14.766+0000 7f0c68c6f700 1 --2- 172.21.15.47:0/639543918 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0c6406b100 0x7f0c6410f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:14.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:14.766+0000 7f0c68c6f700 1 --2- 172.21.15.47:0/639543918 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0c64108a40 0x7f0c64108e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:14.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:14.766+0000 7f0c68c6f700 1 --2- 172.21.15.47:0/639543918 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c6406a760 0x7f0c6406abc0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:14.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:14.766+0000 7f0c68c6f700 1 -- 172.21.15.47:0/639543918 >> 172.21.15.47:0/639543918 conn(0x7f0c64078900 msgr2=0x7f0c64078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:00:14.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:14.766+0000 7f0c68c6f700 1 -- 172.21.15.47:0/639543918 shutdown_connections 2024-04-17T06:00:14.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:14.766+0000 7f0c68c6f700 1 -- 172.21.15.47:0/639543918 wait complete. 2024-04-17T06:00:14.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:14.766+0000 7f0c68c6f700 1 Processor -- start 2024-04-17T06:00:14.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:14.766+0000 7f0c68c6f700 1 -- start start 2024-04-17T06:00:14.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:14.767+0000 7f0c68c6f700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0c6406a760 0x7f0c6419b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:14.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:14.767+0000 7f0c68c6f700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0c6406b100 0x7f0c6419ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:14.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:14.767+0000 7f0c68c6f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c64108a40 0x7f0c6419fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:14.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:14.767+0000 7f0c68c6f700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f0c64112d30 con 0x7f0c64108a40 2024-04-17T06:00:14.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:14.767+0000 7f0c68c6f700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0c64112bb0 con 0x7f0c6406b100 2024-04-17T06:00:14.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:14.767+0000 7f0c68c6f700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0c64112eb0 con 0x7f0c6406a760 2024-04-17T06:00:14.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:14.767+0000 7f0c62d9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c64108a40 0x7f0c6419fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:14.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:14.767+0000 7f0c61d9b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0c6406b100 0x7f0c6419ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:14.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:14.767+0000 7f0c61d9b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0c6406b100 0x7f0c6419ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.47:55948/0 (socket says 172.21.15.47:55948) 2024-04-17T06:00:14.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:14.767+0000 7f0c62d9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c64108a40 0x7f0c6419fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40480/0 (socket says 172.21.15.47:40480) 2024-04-17T06:00:14.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:14.767+0000 7f0c61d9b700 1 -- 172.21.15.47:0/4190702563 learned_addr learned my addr 172.21.15.47:0/4190702563 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:00:14.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:14.767+0000 7f0c6259c700 1 --2- 172.21.15.47:0/4190702563 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0c6406a760 0x7f0c6419b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:14.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:14.767+0000 7f0c61d9b700 1 -- 172.21.15.47:0/4190702563 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0c6406a760 msgr2=0x7f0c6419b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:14.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:14.767+0000 7f0c61d9b700 1 --2- 172.21.15.47:0/4190702563 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0c6406a760 0x7f0c6419b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:14.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:14.767+0000 7f0c61d9b700 1 -- 172.21.15.47:0/4190702563 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c64108a40 msgr2=0x7f0c6419fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:14.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:14.767+0000 7f0c61d9b700 1 --2- 172.21.15.47:0/4190702563 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c64108a40 0x7f0c6419fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:14.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:14.767+0000 7f0c61d9b700 1 -- 172.21.15.47:0/4190702563 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c641a0550 con 0x7f0c6406b100 2024-04-17T06:00:14.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:14.767+0000 7f0c61d9b700 1 --2- 172.21.15.47:0/4190702563 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0c6406b100 0x7f0c6419ba80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f0c58000c00 tx=0x7f0c5800bab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:00:14.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:14.768+0000 7f0c537fe700 1 -- 172.21.15.47:0/4190702563 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c58027070 con 0x7f0c6406b100 2024-04-17T06:00:14.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:14.768+0000 7f0c68c6f700 1 -- 172.21.15.47:0/4190702563 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c641a07e0 con 0x7f0c6406b100 2024-04-17T06:00:14.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:14.768+0000 7f0c537fe700 1 -- 172.21.15.47:0/4190702563 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c58003d70 con 0x7f0c6406b100 2024-04-17T06:00:14.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:14.768+0000 7f0c537fe700 1 -- 172.21.15.47:0/4190702563 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c5800f460 con 0x7f0c6406b100 2024-04-17T06:00:14.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:14.768+0000 7f0c68c6f700 1 -- 172.21.15.47:0/4190702563 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c641a80c0 con 0x7f0c6406b100 2024-04-17T06:00:14.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:14.769+0000 7f0c68c6f700 1 -- 172.21.15.47:0/4190702563 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c64069740 con 0x7f0c6406b100 2024-04-17T06:00:14.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:14.770+0000 7f0c537fe700 1 -- 172.21.15.47:0/4190702563 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f0c5802a070 con 0x7f0c6406b100 2024-04-17T06:00:14.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:14.770+0000 7f0c537fe700 1 --2- 172.21.15.47:0/4190702563 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f0c4c0a3d50 0x7f0c4c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:14.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:14.770+0000 7f0c6259c700 1 --2- 172.21.15.47:0/4190702563 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f0c4c0a3d50 0x7f0c4c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:14.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:14.770+0000 7f0c537fe700 1 -- 172.21.15.47:0/4190702563 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f0c580d1430 con 0x7f0c6406b100 2024-04-17T06:00:14.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:14.771+0000 7f0c6259c700 1 --2- 172.21.15.47:0/4190702563 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f0c4c0a3d50 0x7f0c4c0a6210 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0c54009be0 tx=0x7f0c54009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:00:14.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:14.775+0000 7f0c537fe700 1 -- 172.21.15.47:0/4190702563 <== mon.2 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f0c58013070 con 0x7f0c6406b100 2024-04-17T06:00:14.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:14.986+0000 7f0c68c6f700 1 -- 172.21.15.47:0/4190702563 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f0c6402df90 con 0x7f0c6406b100 2024-04-17T06:00:14.984 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:00:14 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-04-17T06:00:14.984 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:00:14 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-04-17T06:00:14.984 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:00:14 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-04-17T06:00:14.984 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:00:14 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-04-17T06:00:14.985 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:00:14 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.b", "id": [3, 9]}]': finished 2024-04-17T06:00:14.985 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:00:14 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]': finished 2024-04-17T06:00:14.985 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:00:14 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]': finished 2024-04-17T06:00:14.985 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:00:14 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2024-04-17T06:00:14.985 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:00:14 smithi047 ceph-mon[32223]: osdmap e87: 12 total, 12 up, 12 in 2024-04-17T06:00:14.985 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:00:14 smithi047 ceph-mon[32223]: pgmap v317: 97 pgs: 97 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-17T06:00:14.985 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:00:14 smithi047 ceph-mon[32223]: osdmap e88: 12 total, 12 up, 12 in 2024-04-17T06:00:14.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:14.991+0000 7f0c537fe700 1 -- 172.21.15.47:0/4190702563 <== mon.2 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f0c5809dcf0 con 0x7f0c6406b100 2024-04-17T06:00:14.989 INFO:teuthology.orchestra.run.smithi047.stdout:111669149762 2024-04-17T06:00:14.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:14.993+0000 7f0c68c6f700 1 -- 172.21.15.47:0/4190702563 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f0c4c0a3d50 msgr2=0x7f0c4c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:14.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:14.993+0000 7f0c68c6f700 1 --2- 172.21.15.47:0/4190702563 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f0c4c0a3d50 0x7f0c4c0a6210 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0c54009be0 tx=0x7f0c54009260 comp rx=0 tx=0).stop 2024-04-17T06:00:14.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:14.993+0000 7f0c68c6f700 1 -- 172.21.15.47:0/4190702563 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0c6406b100 msgr2=0x7f0c6419ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:14.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:14.993+0000 7f0c68c6f700 1 --2- 172.21.15.47:0/4190702563 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0c6406b100 0x7f0c6419ba80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f0c58000c00 tx=0x7f0c5800bab0 comp rx=0 tx=0).stop 2024-04-17T06:00:14.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:14.993+0000 7f0c68c6f700 1 -- 172.21.15.47:0/4190702563 shutdown_connections 2024-04-17T06:00:14.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:14.993+0000 7f0c68c6f700 1 --2- 172.21.15.47:0/4190702563 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f0c4c0a3d50 0x7f0c4c0a6210 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:14.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:14.993+0000 7f0c68c6f700 1 --2- 172.21.15.47:0/4190702563 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c64108a40 0x7f0c6419fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:14.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:14.993+0000 7f0c68c6f700 1 --2- 172.21.15.47:0/4190702563 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0c6406a760 0x7f0c6419b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:14.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:14.993+0000 7f0c68c6f700 1 --2- 172.21.15.47:0/4190702563 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0c6406b100 0x7f0c6419ba80 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:14.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:14.993+0000 7f0c68c6f700 1 -- 172.21.15.47:0/4190702563 >> 172.21.15.47:0/4190702563 conn(0x7f0c64078900 msgr2=0x7f0c6410dfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:00:14.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:14.993+0000 7f0c68c6f700 1 -- 172.21.15.47:0/4190702563 shutdown_connections 2024-04-17T06:00:14.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:14.993+0000 7f0c68c6f700 1 -- 172.21.15.47:0/4190702563 wait complete. 2024-04-17T06:00:15.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:15.034+0000 7f0ec2379700 1 -- 172.21.15.47:0/2940279243 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0ebc101980 msgr2=0x7f0ebc101e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:15.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:15.034+0000 7f0ec2379700 1 --2- 172.21.15.47:0/2940279243 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0ebc101980 0x7f0ebc101e00 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f0ea40099c0 tx=0x7f0ea4009cd0 comp rx=0 tx=0).stop 2024-04-17T06:00:15.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:15.034+0000 7f0ec2379700 1 -- 172.21.15.47:0/2940279243 shutdown_connections 2024-04-17T06:00:15.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:15.034+0000 7f0ec2379700 1 --2- 172.21.15.47:0/2940279243 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ebc102340 0x7f0ebc10e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:15.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:15.034+0000 7f0ec2379700 1 --2- 172.21.15.47:0/2940279243 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ebc1079d0 0x7f0ebc107db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:15.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:15.034+0000 7f0ec2379700 1 --2- 172.21.15.47:0/2940279243 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0ebc101980 0x7f0ebc101e00 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:15.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:15.034+0000 7f0ec2379700 1 -- 172.21.15.47:0/2940279243 >> 172.21.15.47:0/2940279243 conn(0x7f0ebc078990 msgr2=0x7f0ebc078da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:00:15.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:15.034+0000 7f0ec2379700 1 -- 172.21.15.47:0/2940279243 shutdown_connections 2024-04-17T06:00:15.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:15.034+0000 7f0ec2379700 1 -- 172.21.15.47:0/2940279243 wait complete. 2024-04-17T06:00:15.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:15.035+0000 7f0ec2379700 1 Processor -- start 2024-04-17T06:00:15.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:15.035+0000 7f0ec2379700 1 -- start start 2024-04-17T06:00:15.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:15.035+0000 7f0ec2379700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0ebc101980 0x7f0ebc19a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:15.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:15.035+0000 7f0ec2379700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ebc102340 0x7f0ebc19a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:15.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:15.035+0000 7f0ec2379700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ebc1079d0 0x7f0ebc1a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:15.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:15.035+0000 7f0ec2379700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f0ebc111cb0 con 0x7f0ebc101980 2024-04-17T06:00:15.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:15.035+0000 7f0ec2379700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0ebc111b30 con 0x7f0ebc1079d0 2024-04-17T06:00:15.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:15.035+0000 7f0ec2379700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0ebc111e30 con 0x7f0ebc102340 2024-04-17T06:00:15.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:15.035+0000 7f0ebbfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0ebc101980 0x7f0ebc19a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:15.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:15.035+0000 7f0ebbfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0ebc101980 0x7f0ebc19a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40494/0 (socket says 172.21.15.47:40494) 2024-04-17T06:00:15.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:15.035+0000 7f0ebbfff700 1 -- 172.21.15.47:0/2564804519 learned_addr learned my addr 172.21.15.47:0/2564804519 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:00:15.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:15.035+0000 7f0eb3fff700 1 --2- 172.21.15.47:0/2564804519 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ebc102340 0x7f0ebc19a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:15.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:15.035+0000 7f0ec0916700 1 --2- 172.21.15.47:0/2564804519 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ebc1079d0 0x7f0ebc1a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:15.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:15.036+0000 7f0ebbfff700 1 -- 172.21.15.47:0/2564804519 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ebc102340 msgr2=0x7f0ebc19a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:15.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:15.036+0000 7f0ebbfff700 1 --2- 172.21.15.47:0/2564804519 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ebc102340 0x7f0ebc19a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:15.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:15.036+0000 7f0ebbfff700 1 -- 172.21.15.47:0/2564804519 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ebc1079d0 msgr2=0x7f0ebc1a5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:15.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:15.036+0000 7f0ebbfff700 1 --2- 172.21.15.47:0/2564804519 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ebc1079d0 0x7f0ebc1a5150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:15.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:15.036+0000 7f0ebbfff700 1 -- 172.21.15.47:0/2564804519 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ebc1a5690 con 0x7f0ebc101980 2024-04-17T06:00:15.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:15.036+0000 7f0ebbfff700 1 --2- 172.21.15.47:0/2564804519 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0ebc101980 0x7f0ebc19a0d0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f0eac00e9c0 tx=0x7f0eac00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:00:15.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:15.036+0000 7f0eb9ffb700 1 -- 172.21.15.47:0/2564804519 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eac00cbd0 con 0x7f0ebc101980 2024-04-17T06:00:15.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:15.036+0000 7f0eb9ffb700 1 -- 172.21.15.47:0/2564804519 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0eac00ee50 con 0x7f0ebc101980 2024-04-17T06:00:15.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:15.036+0000 7f0eb9ffb700 1 -- 172.21.15.47:0/2564804519 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eac0055c0 con 0x7f0ebc101980 2024-04-17T06:00:15.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:15.036+0000 7f0ec2379700 1 -- 172.21.15.47:0/2564804519 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ebc197f20 con 0x7f0ebc101980 2024-04-17T06:00:15.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:15.036+0000 7f0ec2379700 1 -- 172.21.15.47:0/2564804519 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ebc1983e0 con 0x7f0ebc101980 2024-04-17T06:00:15.037 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:00:14 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-04-17T06:00:15.037 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:00:14 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-04-17T06:00:15.037 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:00:14 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-04-17T06:00:15.037 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:00:14 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-04-17T06:00:15.037 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:00:14 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.b", "id": [3, 9]}]': finished 2024-04-17T06:00:15.037 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:00:14 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]': finished 2024-04-17T06:00:15.037 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:00:14 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]': finished 2024-04-17T06:00:15.037 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:00:14 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2024-04-17T06:00:15.037 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:00:14 smithi103 ceph-mon[36243]: osdmap e87: 12 total, 12 up, 12 in 2024-04-17T06:00:15.037 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:00:14 smithi103 ceph-mon[36243]: pgmap v317: 97 pgs: 97 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-17T06:00:15.037 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:00:14 smithi103 ceph-mon[36243]: osdmap e88: 12 total, 12 up, 12 in 2024-04-17T06:00:15.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:15.037+0000 7f0ec2379700 1 -- 172.21.15.47:0/2564804519 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ebc1030a0 con 0x7f0ebc101980 2024-04-17T06:00:15.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:15.039+0000 7f0eb9ffb700 1 -- 172.21.15.47:0/2564804519 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f0eac004750 con 0x7f0ebc101980 2024-04-17T06:00:15.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:15.039+0000 7f0eb9ffb700 1 --2- 172.21.15.47:0/2564804519 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f0ea80a3cd0 0x7f0ea80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:15.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:15.039+0000 7f0eb3fff700 1 --2- 172.21.15.47:0/2564804519 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f0ea80a3cd0 0x7f0ea80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:15.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:15.039+0000 7f0eb9ffb700 1 -- 172.21.15.47:0/2564804519 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f0eac0d0ab0 con 0x7f0ebc101980 2024-04-17T06:00:15.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:15.040+0000 7f0eb3fff700 1 --2- 172.21.15.47:0/2564804519 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f0ea80a3cd0 0x7f0ea80a6190 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc06bc70 tx=0x7f0ea4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:00:15.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:15.043+0000 7f0eb9ffb700 1 -- 172.21.15.47:0/2564804519 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f0eac09d3d0 con 0x7f0ebc101980 2024-04-17T06:00:15.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:00:14 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-04-17T06:00:15.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:00:14 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-04-17T06:00:15.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:00:14 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-04-17T06:00:15.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:00:14 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-04-17T06:00:15.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:00:14 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.b", "id": [3, 9]}]': finished 2024-04-17T06:00:15.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:00:14 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]': finished 2024-04-17T06:00:15.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:00:14 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]': finished 2024-04-17T06:00:15.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:00:14 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2024-04-17T06:00:15.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:00:14 smithi098 ceph-mon[36516]: osdmap e87: 12 total, 12 up, 12 in 2024-04-17T06:00:15.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:00:14 smithi098 ceph-mon[36516]: pgmap v317: 97 pgs: 97 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-17T06:00:15.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:00:14 smithi098 ceph-mon[36516]: osdmap e88: 12 total, 12 up, 12 in 2024-04-17T06:00:15.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:15.259+0000 7f0ec2379700 1 -- 172.21.15.47:0/2564804519 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f0ebc02df90 con 0x7f0ebc101980 2024-04-17T06:00:15.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:15.260+0000 7f0eb9ffb700 1 -- 172.21.15.47:0/2564804519 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f0eac09ccd0 con 0x7f0ebc101980 2024-04-17T06:00:15.258 INFO:teuthology.orchestra.run.smithi047.stdout:223338299436 2024-04-17T06:00:15.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:15.262+0000 7f0ec2379700 1 -- 172.21.15.47:0/2564804519 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f0ea80a3cd0 msgr2=0x7f0ea80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:15.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:15.262+0000 7f0ec2379700 1 --2- 172.21.15.47:0/2564804519 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f0ea80a3cd0 0x7f0ea80a6190 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc06bc70 tx=0x7f0ea4005fb0 comp rx=0 tx=0).stop 2024-04-17T06:00:15.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:15.262+0000 7f0ec2379700 1 -- 172.21.15.47:0/2564804519 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0ebc101980 msgr2=0x7f0ebc19a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:15.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:15.262+0000 7f0ec2379700 1 --2- 172.21.15.47:0/2564804519 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0ebc101980 0x7f0ebc19a0d0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f0eac00e9c0 tx=0x7f0eac00ecd0 comp rx=0 tx=0).stop 2024-04-17T06:00:15.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:15.262+0000 7f0ec2379700 1 -- 172.21.15.47:0/2564804519 shutdown_connections 2024-04-17T06:00:15.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:15.262+0000 7f0ec2379700 1 --2- 172.21.15.47:0/2564804519 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f0ea80a3cd0 0x7f0ea80a6190 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:15.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:15.262+0000 7f0ec2379700 1 --2- 172.21.15.47:0/2564804519 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ebc1079d0 0x7f0ebc1a5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:15.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:15.262+0000 7f0ec2379700 1 --2- 172.21.15.47:0/2564804519 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0ebc101980 0x7f0ebc19a0d0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:15.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:15.262+0000 7f0ec2379700 1 --2- 172.21.15.47:0/2564804519 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ebc102340 0x7f0ebc19a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:15.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:15.262+0000 7f0ec2379700 1 -- 172.21.15.47:0/2564804519 >> 172.21.15.47:0/2564804519 conn(0x7f0ebc078990 msgr2=0x7f0ebc0ffad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:00:15.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:15.262+0000 7f0ec2379700 1 -- 172.21.15.47:0/2564804519 shutdown_connections 2024-04-17T06:00:15.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:15.262+0000 7f0ec2379700 1 -- 172.21.15.47:0/2564804519 wait complete. 2024-04-17T06:00:15.698 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149758 got 111669149762 for osd.3 2024-04-17T06:00:15.698 DEBUG:teuthology.parallel:result is None 2024-04-17T06:00:16.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:00:15 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/1979634291' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-17T06:00:16.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:00:15 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T06:00:16.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:00:15 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/4190702563' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-17T06:00:16.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:00:15 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T06:00:16.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:00:15 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T06:00:16.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:00:15 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T06:00:16.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:00:15 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/2564804519' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-17T06:00:16.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:00:15 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/1979634291' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-17T06:00:16.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:00:15 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T06:00:16.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:00:15 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/4190702563' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-17T06:00:16.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:00:15 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T06:00:16.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:00:15 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T06:00:16.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:00:15 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T06:00:16.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:00:15 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/2564804519' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-17T06:00:16.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:00:15 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/1979634291' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-17T06:00:16.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:00:15 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T06:00:16.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:00:15 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/4190702563' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-17T06:00:16.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:00:15 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T06:00:16.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:00:15 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T06:00:16.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:00:15 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T06:00:16.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:00:15 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/2564804519' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-17T06:00:16.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:00:16 smithi047 ceph-mon[32223]: pgmap v319: 97 pgs: 2 peering, 95 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s; 13 B/s, 0 objects/s recovering 2024-04-17T06:00:17.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:00:16 smithi103 ceph-mon[36243]: pgmap v319: 97 pgs: 2 peering, 95 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s; 13 B/s, 0 objects/s recovering 2024-04-17T06:00:17.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:00:16 smithi098 ceph-mon[36516]: pgmap v319: 97 pgs: 2 peering, 95 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s; 13 B/s, 0 objects/s recovering 2024-04-17T06:00:17.190 INFO:tasks.cephadm.ceph_manager.ceph:need seq 257698037794 got 257698037800 for osd.8 2024-04-17T06:00:17.190 DEBUG:teuthology.parallel:result is None 2024-04-17T06:00:17.851 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:00:17 smithi047 ceph-mon[32223]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-04-17T06:00:18.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:00:17 smithi103 ceph-mon[36243]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-04-17T06:00:18.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:00:17 smithi098 ceph-mon[36516]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-04-17T06:00:18.725 INFO:tasks.cephadm.ceph_manager.ceph:need seq 347892350996 got 347892351001 for osd.11 2024-04-17T06:00:18.725 DEBUG:teuthology.parallel:result is None 2024-04-17T06:00:18.994 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:00:18 smithi047 ceph-mon[32223]: pgmap v320: 97 pgs: 2 peering, 95 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 3 op/s; 13 B/s, 0 objects/s recovering 2024-04-17T06:00:19.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:00:18 smithi103 ceph-mon[36243]: pgmap v320: 97 pgs: 2 peering, 95 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 3 op/s; 13 B/s, 0 objects/s recovering 2024-04-17T06:00:19.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:00:18 smithi098 ceph-mon[36516]: pgmap v320: 97 pgs: 2 peering, 95 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 3 op/s; 13 B/s, 0 objects/s recovering 2024-04-17T06:00:19.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:00:19 smithi047 ceph-mon[32223]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-04-17T06:00:19.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:00:19 smithi047 ceph-mon[32223]: Cluster is now healthy 2024-04-17T06:00:20.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:00:19 smithi098 ceph-mon[36516]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-04-17T06:00:20.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:00:19 smithi098 ceph-mon[36516]: Cluster is now healthy 2024-04-17T06:00:20.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:00:19 smithi103 ceph-mon[36243]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-04-17T06:00:20.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:00:19 smithi103 ceph-mon[36243]: Cluster is now healthy 2024-04-17T06:00:20.991 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:00:20 smithi047 ceph-mon[32223]: pgmap v321: 97 pgs: 97 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s; 7.3 KiB/s, 1 objects/s recovering 2024-04-17T06:00:21.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:21.004+0000 7f5b71919700 1 -- 172.21.15.47:0/3170192068 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5b6c103490 msgr2=0x7f5b6c10f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:21.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:21.004+0000 7f5b71919700 1 --2- 172.21.15.47:0/3170192068 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5b6c103490 0x7f5b6c10f8e0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f5b6400b0a0 tx=0x7f5b6400b3b0 comp rx=0 tx=0).stop 2024-04-17T06:00:21.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:21.004+0000 7f5b71919700 1 -- 172.21.15.47:0/3170192068 shutdown_connections 2024-04-17T06:00:21.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:21.004+0000 7f5b71919700 1 --2- 172.21.15.47:0/3170192068 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5b6c103490 0x7f5b6c10f8e0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:21.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:21.004+0000 7f5b71919700 1 --2- 172.21.15.47:0/3170192068 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5b6c108a80 0x7f5b6c108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:21.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:21.004+0000 7f5b71919700 1 --2- 172.21.15.47:0/3170192068 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b6c102ad0 0x7f5b6c102f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:21.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:21.004+0000 7f5b71919700 1 -- 172.21.15.47:0/3170192068 >> 172.21.15.47:0/3170192068 conn(0x7f5b6c079bc0 msgr2=0x7f5b6c079fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:00:21.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:21.004+0000 7f5b71919700 1 -- 172.21.15.47:0/3170192068 shutdown_connections 2024-04-17T06:00:21.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:21.004+0000 7f5b71919700 1 -- 172.21.15.47:0/3170192068 wait complete. 2024-04-17T06:00:21.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:21.005+0000 7f5b71919700 1 Processor -- start 2024-04-17T06:00:21.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:21.005+0000 7f5b71919700 1 -- start start 2024-04-17T06:00:21.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:21.005+0000 7f5b71919700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5b6c102ad0 0x7f5b6c1991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:21.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:21.005+0000 7f5b71919700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b6c103490 0x7f5b6c199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:21.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:21.005+0000 7f5b71919700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5b6c108a80 0x7f5b6c19da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:21.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:21.005+0000 7f5b71919700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f5b6c112da0 con 0x7f5b6c102ad0 2024-04-17T06:00:21.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:21.005+0000 7f5b71919700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5b6c112c20 con 0x7f5b6c108a80 2024-04-17T06:00:21.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:21.005+0000 7f5b71919700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5b6c112f20 con 0x7f5b6c103490 2024-04-17T06:00:21.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:21.006+0000 7f5b6affd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5b6c102ad0 0x7f5b6c1991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:21.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:21.006+0000 7f5b6affd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5b6c102ad0 0x7f5b6c1991f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40514/0 (socket says 172.21.15.47:40514) 2024-04-17T06:00:21.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:21.006+0000 7f5b6affd700 1 -- 172.21.15.47:0/1873021959 learned_addr learned my addr 172.21.15.47:0/1873021959 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:00:21.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:21.006+0000 7f5b6a7fc700 1 --2- 172.21.15.47:0/1873021959 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b6c103490 0x7f5b6c199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:21.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:21.006+0000 7f5b6b7fe700 1 --2- 172.21.15.47:0/1873021959 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5b6c108a80 0x7f5b6c19da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:21.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:21.006+0000 7f5b6affd700 1 -- 172.21.15.47:0/1873021959 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b6c103490 msgr2=0x7f5b6c199730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:21.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:21.006+0000 7f5b6affd700 1 --2- 172.21.15.47:0/1873021959 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b6c103490 0x7f5b6c199730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:21.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:21.006+0000 7f5b6affd700 1 -- 172.21.15.47:0/1873021959 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5b6c108a80 msgr2=0x7f5b6c19da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:21.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:21.006+0000 7f5b6affd700 1 --2- 172.21.15.47:0/1873021959 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5b6c108a80 0x7f5b6c19da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:21.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:21.006+0000 7f5b6affd700 1 -- 172.21.15.47:0/1873021959 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b6c19e1b0 con 0x7f5b6c102ad0 2024-04-17T06:00:21.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:21.006+0000 7f5b6b7fe700 1 --2- 172.21.15.47:0/1873021959 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5b6c108a80 0x7f5b6c19da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:00:21.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:21.006+0000 7f5b6affd700 1 --2- 172.21.15.47:0/1873021959 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5b6c102ad0 0x7f5b6c1991f0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c00d950 tx=0x7f5b5c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:00:21.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:21.006+0000 7f5b70917700 1 -- 172.21.15.47:0/1873021959 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b5c016070 con 0x7f5b6c102ad0 2024-04-17T06:00:21.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:21.006+0000 7f5b71919700 1 -- 172.21.15.47:0/1873021959 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b6c19e4a0 con 0x7f5b6c102ad0 2024-04-17T06:00:21.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:21.006+0000 7f5b70917700 1 -- 172.21.15.47:0/1873021959 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b5c009e20 con 0x7f5b6c102ad0 2024-04-17T06:00:21.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:21.006+0000 7f5b70917700 1 -- 172.21.15.47:0/1873021959 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b5c00f620 con 0x7f5b6c102ad0 2024-04-17T06:00:21.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:21.006+0000 7f5b71919700 1 -- 172.21.15.47:0/1873021959 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b6c1a5d20 con 0x7f5b6c102ad0 2024-04-17T06:00:21.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:21.007+0000 7f5b71919700 1 -- 172.21.15.47:0/1873021959 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b6c069740 con 0x7f5b6c102ad0 2024-04-17T06:00:21.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:21.008+0000 7f5b70917700 1 -- 172.21.15.47:0/1873021959 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f5b5c019070 con 0x7f5b6c102ad0 2024-04-17T06:00:21.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:21.009+0000 7f5b70917700 1 --2- 172.21.15.47:0/1873021959 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f5b540a3d50 0x7f5b540a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:21.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:21.009+0000 7f5b6a7fc700 1 --2- 172.21.15.47:0/1873021959 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f5b540a3d50 0x7f5b540a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:21.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:21.009+0000 7f5b70917700 1 -- 172.21.15.47:0/1873021959 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f5b5c0d0140 con 0x7f5b6c102ad0 2024-04-17T06:00:21.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:21.010+0000 7f5b6a7fc700 1 --2- 172.21.15.47:0/1873021959 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f5b540a3d50 0x7f5b540a6210 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c07a170 tx=0x7f5b6000b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:00:21.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:21.013+0000 7f5b70917700 1 -- 172.21.15.47:0/1873021959 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f5b5c09c9d0 con 0x7f5b6c102ad0 2024-04-17T06:00:21.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:00:20 smithi098 ceph-mon[36516]: pgmap v321: 97 pgs: 97 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s; 7.3 KiB/s, 1 objects/s recovering 2024-04-17T06:00:21.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:21.222+0000 7f5b71919700 1 -- 172.21.15.47:0/1873021959 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f5b6c02df90 con 0x7f5b6c102ad0 2024-04-17T06:00:21.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:21.223+0000 7f5b70917700 1 -- 172.21.15.47:0/1873021959 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f5b5c09c2d0 con 0x7f5b6c102ad0 2024-04-17T06:00:21.221 INFO:teuthology.orchestra.run.smithi047.stdout:34359738449 2024-04-17T06:00:21.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:21.225+0000 7f5b71919700 1 -- 172.21.15.47:0/1873021959 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f5b540a3d50 msgr2=0x7f5b540a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:21.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:21.225+0000 7f5b71919700 1 --2- 172.21.15.47:0/1873021959 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f5b540a3d50 0x7f5b540a6210 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c07a170 tx=0x7f5b6000b3c0 comp rx=0 tx=0).stop 2024-04-17T06:00:21.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:21.225+0000 7f5b71919700 1 -- 172.21.15.47:0/1873021959 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5b6c102ad0 msgr2=0x7f5b6c1991f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:21.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:21.225+0000 7f5b71919700 1 --2- 172.21.15.47:0/1873021959 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5b6c102ad0 0x7f5b6c1991f0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c00d950 tx=0x7f5b5c00dc60 comp rx=0 tx=0).stop 2024-04-17T06:00:21.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:21.225+0000 7f5b71919700 1 -- 172.21.15.47:0/1873021959 shutdown_connections 2024-04-17T06:00:21.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:21.225+0000 7f5b71919700 1 --2- 172.21.15.47:0/1873021959 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f5b540a3d50 0x7f5b540a6210 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:21.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:21.225+0000 7f5b71919700 1 --2- 172.21.15.47:0/1873021959 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5b6c108a80 0x7f5b6c19da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:21.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:21.225+0000 7f5b71919700 1 --2- 172.21.15.47:0/1873021959 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5b6c102ad0 0x7f5b6c1991f0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:21.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:21.225+0000 7f5b71919700 1 --2- 172.21.15.47:0/1873021959 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b6c103490 0x7f5b6c199730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:21.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:21.225+0000 7f5b71919700 1 -- 172.21.15.47:0/1873021959 >> 172.21.15.47:0/1873021959 conn(0x7f5b6c079bc0 msgr2=0x7f5b6c1016b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:00:21.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:21.225+0000 7f5b71919700 1 -- 172.21.15.47:0/1873021959 shutdown_connections 2024-04-17T06:00:21.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:21.225+0000 7f5b71919700 1 -- 172.21.15.47:0/1873021959 wait complete. 2024-04-17T06:00:21.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:00:20 smithi103 ceph-mon[36243]: pgmap v321: 97 pgs: 97 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s; 7.3 KiB/s, 1 objects/s recovering 2024-04-17T06:00:22.077 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:00:21 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/1873021959' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-17T06:00:22.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:00:21 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/1873021959' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-17T06:00:22.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:00:21 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/1873021959' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-17T06:00:22.413 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738444 got 34359738449 for osd.0 2024-04-17T06:00:22.413 DEBUG:teuthology.parallel:result is None 2024-04-17T06:00:22.540 INFO:tasks.cephadm.ceph_manager.ceph:need seq 223338299431 got 223338299436 for osd.7 2024-04-17T06:00:22.540 DEBUG:teuthology.parallel:result is None 2024-04-17T06:00:23.041 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:00:22 smithi047 ceph-mon[32223]: pgmap v322: 97 pgs: 97 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s; 7.3 KiB/s, 1 objects/s recovering 2024-04-17T06:00:23.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:00:22 smithi098 ceph-mon[36516]: pgmap v322: 97 pgs: 97 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s; 7.3 KiB/s, 1 objects/s recovering 2024-04-17T06:00:23.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:22.833+0000 7f4a53b53700 1 -- 172.21.15.47:0/1672656212 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a4c102ae0 msgr2=0x7f4a4c102f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:23.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:22.833+0000 7f4a53b53700 1 --2- 172.21.15.47:0/1672656212 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a4c102ae0 0x7f4a4c102f60 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c009990 tx=0x7f4a3c009ca0 comp rx=0 tx=0).stop 2024-04-17T06:00:23.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:22.833+0000 7f4a53b53700 1 -- 172.21.15.47:0/1672656212 shutdown_connections 2024-04-17T06:00:23.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:22.833+0000 7f4a53b53700 1 --2- 172.21.15.47:0/1672656212 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4a4c1034a0 0x7f4a4c10f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:23.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:22.833+0000 7f4a53b53700 1 --2- 172.21.15.47:0/1672656212 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4a4c108ab0 0x7f4a4c108e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:23.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:22.833+0000 7f4a53b53700 1 --2- 172.21.15.47:0/1672656212 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a4c102ae0 0x7f4a4c102f60 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:23.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:22.833+0000 7f4a53b53700 1 -- 172.21.15.47:0/1672656212 >> 172.21.15.47:0/1672656212 conn(0x7f4a4c079c40 msgr2=0x7f4a4c07a050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:00:23.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:22.833+0000 7f4a53b53700 1 -- 172.21.15.47:0/1672656212 shutdown_connections 2024-04-17T06:00:23.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:22.833+0000 7f4a53b53700 1 -- 172.21.15.47:0/1672656212 wait complete. 2024-04-17T06:00:23.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:22.833+0000 7f4a53b53700 1 Processor -- start 2024-04-17T06:00:23.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:22.833+0000 7f4a53b53700 1 -- start start 2024-04-17T06:00:23.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:22.834+0000 7f4a53b53700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4a4c102ae0 0x7f4a4c074d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:23.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:22.834+0000 7f4a53b53700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a4c1034a0 0x7f4a4c075290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:23.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:22.834+0000 7f4a53b53700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4a4c108ab0 0x7f4a4c0757d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:23.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:22.834+0000 7f4a53b53700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f4a4c112e40 con 0x7f4a4c1034a0 2024-04-17T06:00:23.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:22.834+0000 7f4a53b53700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f4a4c112cc0 con 0x7f4a4c102ae0 2024-04-17T06:00:23.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:22.834+0000 7f4a53b53700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4a4c112fc0 con 0x7f4a4c108ab0 2024-04-17T06:00:23.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:22.834+0000 7f4a510ee700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a4c1034a0 0x7f4a4c075290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:23.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:22.834+0000 7f4a510ee700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a4c1034a0 0x7f4a4c075290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40536/0 (socket says 172.21.15.47:40536) 2024-04-17T06:00:23.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:22.834+0000 7f4a510ee700 1 -- 172.21.15.47:0/1027728034 learned_addr learned my addr 172.21.15.47:0/1027728034 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:00:23.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:22.834+0000 7f4a518ef700 1 --2- 172.21.15.47:0/1027728034 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4a4c102ae0 0x7f4a4c074d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:23.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:22.834+0000 7f4a520f0700 1 --2- 172.21.15.47:0/1027728034 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4a4c108ab0 0x7f4a4c0757d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:23.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:22.834+0000 7f4a510ee700 1 -- 172.21.15.47:0/1027728034 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4a4c108ab0 msgr2=0x7f4a4c0757d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:23.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:22.834+0000 7f4a510ee700 1 --2- 172.21.15.47:0/1027728034 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4a4c108ab0 0x7f4a4c0757d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:23.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:22.834+0000 7f4a510ee700 1 -- 172.21.15.47:0/1027728034 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4a4c102ae0 msgr2=0x7f4a4c074d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:23.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:22.834+0000 7f4a510ee700 1 --2- 172.21.15.47:0/1027728034 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4a4c102ae0 0x7f4a4c074d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:23.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:22.834+0000 7f4a510ee700 1 -- 172.21.15.47:0/1027728034 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a4c1a7780 con 0x7f4a4c1034a0 2024-04-17T06:00:23.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:22.834+0000 7f4a510ee700 1 --2- 172.21.15.47:0/1027728034 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a4c1034a0 0x7f4a4c075290 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c000c00 tx=0x7f4a3c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:00:23.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:22.834+0000 7f4a42ffd700 1 -- 172.21.15.47:0/1027728034 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a3c027070 con 0x7f4a4c1034a0 2024-04-17T06:00:23.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:22.834+0000 7f4a53b53700 1 -- 172.21.15.47:0/1027728034 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a4c1a7a10 con 0x7f4a4c1034a0 2024-04-17T06:00:23.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:22.834+0000 7f4a42ffd700 1 -- 172.21.15.47:0/1027728034 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a3c003780 con 0x7f4a4c1034a0 2024-04-17T06:00:23.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:22.834+0000 7f4a42ffd700 1 -- 172.21.15.47:0/1027728034 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a3c022dd0 con 0x7f4a4c1034a0 2024-04-17T06:00:23.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:22.834+0000 7f4a53b53700 1 -- 172.21.15.47:0/1027728034 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a4c1a7e60 con 0x7f4a4c1034a0 2024-04-17T06:00:23.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:22.835+0000 7f4a53b53700 1 -- 172.21.15.47:0/1027728034 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a4c10b040 con 0x7f4a4c1034a0 2024-04-17T06:00:23.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:22.837+0000 7f4a42ffd700 1 -- 172.21.15.47:0/1027728034 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f4a3c02a070 con 0x7f4a4c1034a0 2024-04-17T06:00:23.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:22.838+0000 7f4a42ffd700 1 --2- 172.21.15.47:0/1027728034 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f4a380a3cd0 0x7f4a380a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:23.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:22.838+0000 7f4a518ef700 1 --2- 172.21.15.47:0/1027728034 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f4a380a3cd0 0x7f4a380a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:23.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:22.838+0000 7f4a42ffd700 1 -- 172.21.15.47:0/1027728034 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f4a3c0d1600 con 0x7f4a4c1034a0 2024-04-17T06:00:23.162 INFO:teuthology.orchestra.run.smithi047.stdout:146028888126 2024-04-17T06:00:23.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:22.839+0000 7f4a518ef700 1 --2- 172.21.15.47:0/1027728034 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f4a380a3cd0 0x7f4a380a6190 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f4a48005950 tx=0x7f4a4800a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:00:23.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:22.842+0000 7f4a42ffd700 1 -- 172.21.15.47:0/1027728034 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f4a3c013070 con 0x7f4a4c1034a0 2024-04-17T06:00:23.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:23.050+0000 7f4a53b53700 1 -- 172.21.15.47:0/1027728034 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f4a4c10aa20 con 0x7f4a4c1034a0 2024-04-17T06:00:23.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:23.050+0000 7f4a42ffd700 1 -- 172.21.15.47:0/1027728034 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f4a3c09df50 con 0x7f4a4c1034a0 2024-04-17T06:00:23.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:23.053+0000 7f4a53b53700 1 -- 172.21.15.47:0/1027728034 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f4a380a3cd0 msgr2=0x7f4a380a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:23.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:23.053+0000 7f4a53b53700 1 --2- 172.21.15.47:0/1027728034 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f4a380a3cd0 0x7f4a380a6190 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f4a48005950 tx=0x7f4a4800a300 comp rx=0 tx=0).stop 2024-04-17T06:00:23.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:23.053+0000 7f4a53b53700 1 -- 172.21.15.47:0/1027728034 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a4c1034a0 msgr2=0x7f4a4c075290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:23.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:23.053+0000 7f4a53b53700 1 --2- 172.21.15.47:0/1027728034 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a4c1034a0 0x7f4a4c075290 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c000c00 tx=0x7f4a3c0047b0 comp rx=0 tx=0).stop 2024-04-17T06:00:23.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:23.053+0000 7f4a53b53700 1 -- 172.21.15.47:0/1027728034 shutdown_connections 2024-04-17T06:00:23.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:23.053+0000 7f4a53b53700 1 --2- 172.21.15.47:0/1027728034 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f4a380a3cd0 0x7f4a380a6190 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:23.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:23.053+0000 7f4a53b53700 1 --2- 172.21.15.47:0/1027728034 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4a4c108ab0 0x7f4a4c0757d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:23.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:23.053+0000 7f4a53b53700 1 --2- 172.21.15.47:0/1027728034 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4a4c102ae0 0x7f4a4c074d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:23.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:23.053+0000 7f4a53b53700 1 --2- 172.21.15.47:0/1027728034 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a4c1034a0 0x7f4a4c075290 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:23.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:23.053+0000 7f4a53b53700 1 -- 172.21.15.47:0/1027728034 >> 172.21.15.47:0/1027728034 conn(0x7f4a4c079c40 msgr2=0x7f4a4c100ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:00:23.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:23.053+0000 7f4a53b53700 1 -- 172.21.15.47:0/1027728034 shutdown_connections 2024-04-17T06:00:23.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:23.053+0000 7f4a53b53700 1 -- 172.21.15.47:0/1027728034 wait complete. 2024-04-17T06:00:23.185 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:00:22 smithi103 ceph-mon[36243]: pgmap v322: 97 pgs: 97 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s; 7.3 KiB/s, 1 objects/s recovering 2024-04-17T06:00:24.124 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:00:23 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/1027728034' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-17T06:00:24.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:00:23 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/1027728034' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-17T06:00:24.198 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:00:23 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/1027728034' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-17T06:00:25.020 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:00:24 smithi047 ceph-mon[32223]: pgmap v323: 97 pgs: 97 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s; 5.8 KiB/s, 1 objects/s recovering 2024-04-17T06:00:25.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:00:24 smithi098 ceph-mon[36516]: pgmap v323: 97 pgs: 97 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s; 5.8 KiB/s, 1 objects/s recovering 2024-04-17T06:00:25.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:25.021+0000 7fec9bb24700 1 -- 172.21.15.47:0/3449666367 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec94106410 msgr2=0x7fec9410dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:25.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:25.021+0000 7fec9bb24700 1 --2- 172.21.15.47:0/3449666367 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec94106410 0x7fec9410dfe0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fec8c00b0d0 tx=0x7fec8c00b3e0 comp rx=0 tx=0).stop 2024-04-17T06:00:25.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:25.021+0000 7fec9bb24700 1 -- 172.21.15.47:0/3449666367 shutdown_connections 2024-04-17T06:00:25.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:25.021+0000 7fec9bb24700 1 --2- 172.21.15.47:0/3449666367 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec94106410 0x7fec9410dfe0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:25.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:25.021+0000 7fec9bb24700 1 --2- 172.21.15.47:0/3449666367 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fec94101330 0x7fec94101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:25.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:25.021+0000 7fec9bb24700 1 --2- 172.21.15.47:0/3449666367 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fec94101ce0 0x7fec94105d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:25.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:25.021+0000 7fec9bb24700 1 -- 172.21.15.47:0/3449666367 >> 172.21.15.47:0/3449666367 conn(0x7fec94077f50 msgr2=0x7fec940ff2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:00:25.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:25.022+0000 7fec9bb24700 1 -- 172.21.15.47:0/3449666367 shutdown_connections 2024-04-17T06:00:25.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:25.022+0000 7fec9bb24700 1 -- 172.21.15.47:0/3449666367 wait complete. 2024-04-17T06:00:25.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:25.022+0000 7fec9bb24700 1 Processor -- start 2024-04-17T06:00:25.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:25.022+0000 7fec9bb24700 1 -- start start 2024-04-17T06:00:25.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:25.023+0000 7fec9bb24700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec94101330 0x7fec940fd390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:25.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:25.023+0000 7fec9bb24700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fec94101ce0 0x7fec940fd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:25.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:25.023+0000 7fec9bb24700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fec94106410 0x7fec940fde10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:25.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:25.023+0000 7fec9bb24700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fec94110b80 con 0x7fec94101330 2024-04-17T06:00:25.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:25.023+0000 7fec9bb24700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fec94110a00 con 0x7fec94101ce0 2024-04-17T06:00:25.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:25.023+0000 7fec9bb24700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fec94110d00 con 0x7fec94106410 2024-04-17T06:00:25.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:25.023+0000 7fec998c0700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec94101330 0x7fec940fd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:25.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:25.023+0000 7fec998c0700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec94101330 0x7fec940fd390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:42910/0 (socket says 172.21.15.47:42910) 2024-04-17T06:00:25.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:25.023+0000 7fec998c0700 1 -- 172.21.15.47:0/790971714 learned_addr learned my addr 172.21.15.47:0/790971714 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:00:25.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:25.023+0000 7fec9a0c1700 1 --2- 172.21.15.47:0/790971714 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fec94106410 0x7fec940fde10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:25.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:25.023+0000 7fec990bf700 1 --2- 172.21.15.47:0/790971714 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fec94101ce0 0x7fec940fd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:25.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:25.023+0000 7fec998c0700 1 -- 172.21.15.47:0/790971714 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fec94106410 msgr2=0x7fec940fde10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:25.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:25.023+0000 7fec998c0700 1 --2- 172.21.15.47:0/790971714 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fec94106410 0x7fec940fde10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:25.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:25.023+0000 7fec998c0700 1 -- 172.21.15.47:0/790971714 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fec94101ce0 msgr2=0x7fec940fd8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:25.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:25.023+0000 7fec998c0700 1 --2- 172.21.15.47:0/790971714 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fec94101ce0 0x7fec940fd8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:25.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:25.023+0000 7fec998c0700 1 -- 172.21.15.47:0/790971714 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec941a76d0 con 0x7fec94101330 2024-04-17T06:00:25.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:25.023+0000 7fec998c0700 1 --2- 172.21.15.47:0/790971714 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec94101330 0x7fec940fd390 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fec9000ddc0 tx=0x7fec9000a530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:00:25.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:25.023+0000 7fec8affd700 1 -- 172.21.15.47:0/790971714 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec9001b070 con 0x7fec94101330 2024-04-17T06:00:25.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:25.023+0000 7fec8affd700 1 -- 172.21.15.47:0/790971714 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec9000ad40 con 0x7fec94101330 2024-04-17T06:00:25.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:25.023+0000 7fec9bb24700 1 -- 172.21.15.47:0/790971714 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec941a79c0 con 0x7fec94101330 2024-04-17T06:00:25.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:25.023+0000 7fec8affd700 1 -- 172.21.15.47:0/790971714 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec90016860 con 0x7fec94101330 2024-04-17T06:00:25.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:25.023+0000 7fec9bb24700 1 -- 172.21.15.47:0/790971714 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec941a7f00 con 0x7fec94101330 2024-04-17T06:00:25.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:25.024+0000 7fec9bb24700 1 -- 172.21.15.47:0/790971714 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec94069740 con 0x7fec94101330 2024-04-17T06:00:25.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:25.026+0000 7fec8affd700 1 -- 172.21.15.47:0/790971714 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fec9001e070 con 0x7fec94101330 2024-04-17T06:00:25.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:25.026+0000 7fec8affd700 1 --2- 172.21.15.47:0/790971714 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fec800a3cd0 0x7fec800a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:25.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:25.026+0000 7fec990bf700 1 --2- 172.21.15.47:0/790971714 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fec800a3cd0 0x7fec800a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:25.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:25.026+0000 7fec8affd700 1 -- 172.21.15.47:0/790971714 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fec900cffd0 con 0x7fec94101330 2024-04-17T06:00:25.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:25.027+0000 7fec990bf700 1 --2- 172.21.15.47:0/790971714 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fec800a3cd0 0x7fec800a6190 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fec84005950 tx=0x7fec840058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:00:25.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:25.030+0000 7fec8affd700 1 -- 172.21.15.47:0/790971714 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fec9009c970 con 0x7fec94101330 2024-04-17T06:00:25.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:25.239+0000 7fec9bb24700 1 -- 172.21.15.47:0/790971714 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fec9402df90 con 0x7fec94101330 2024-04-17T06:00:25.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:25.239+0000 7fec8affd700 1 -- 172.21.15.47:0/790971714 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fec9009c270 con 0x7fec94101330 2024-04-17T06:00:25.238 INFO:teuthology.orchestra.run.smithi047.stdout:77309411401 2024-04-17T06:00:25.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:25.241+0000 7fec9bb24700 1 -- 172.21.15.47:0/790971714 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fec800a3cd0 msgr2=0x7fec800a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:25.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:25.241+0000 7fec9bb24700 1 --2- 172.21.15.47:0/790971714 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fec800a3cd0 0x7fec800a6190 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fec84005950 tx=0x7fec840058e0 comp rx=0 tx=0).stop 2024-04-17T06:00:25.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:25.241+0000 7fec9bb24700 1 -- 172.21.15.47:0/790971714 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec94101330 msgr2=0x7fec940fd390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:25.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:25.241+0000 7fec9bb24700 1 --2- 172.21.15.47:0/790971714 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec94101330 0x7fec940fd390 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fec9000ddc0 tx=0x7fec9000a530 comp rx=0 tx=0).stop 2024-04-17T06:00:25.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:25.241+0000 7fec9bb24700 1 -- 172.21.15.47:0/790971714 shutdown_connections 2024-04-17T06:00:25.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:25.241+0000 7fec9bb24700 1 --2- 172.21.15.47:0/790971714 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fec800a3cd0 0x7fec800a6190 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:25.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:25.241+0000 7fec9bb24700 1 --2- 172.21.15.47:0/790971714 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fec94106410 0x7fec940fde10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:25.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:25.241+0000 7fec9bb24700 1 --2- 172.21.15.47:0/790971714 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec94101330 0x7fec940fd390 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:25.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:25.241+0000 7fec9bb24700 1 --2- 172.21.15.47:0/790971714 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fec94101ce0 0x7fec940fd8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:25.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:25.241+0000 7fec9bb24700 1 -- 172.21.15.47:0/790971714 >> 172.21.15.47:0/790971714 conn(0x7fec94077f50 msgr2=0x7fec941047f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:00:25.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:25.241+0000 7fec9bb24700 1 -- 172.21.15.47:0/790971714 shutdown_connections 2024-04-17T06:00:25.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:25.241+0000 7fec9bb24700 1 -- 172.21.15.47:0/790971714 wait complete. 2024-04-17T06:00:25.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:00:24 smithi103 ceph-mon[36243]: pgmap v323: 97 pgs: 97 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s; 5.8 KiB/s, 1 objects/s recovering 2024-04-17T06:00:25.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:25.383+0000 7f599d3bf700 1 -- 172.21.15.47:0/1841263625 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5998102310 msgr2=0x7f599810e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:25.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:25.383+0000 7f599d3bf700 1 --2- 172.21.15.47:0/1841263625 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5998102310 0x7f599810e800 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f599000b0a0 tx=0x7f599000b3b0 comp rx=0 tx=0).stop 2024-04-17T06:00:25.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:25.383+0000 7f599d3bf700 1 -- 172.21.15.47:0/1841263625 shutdown_connections 2024-04-17T06:00:25.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:25.383+0000 7f599d3bf700 1 --2- 172.21.15.47:0/1841263625 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5998102310 0x7f599810e800 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:25.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:25.383+0000 7f599d3bf700 1 --2- 172.21.15.47:0/1841263625 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f59981079a0 0x7f5998107d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:25.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:25.383+0000 7f599d3bf700 1 --2- 172.21.15.47:0/1841263625 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5998101950 0x7f5998101dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:25.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:25.383+0000 7f599d3bf700 1 -- 172.21.15.47:0/1841263625 >> 172.21.15.47:0/1841263625 conn(0x7f5998078900 msgr2=0x7f5998078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:00:25.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:25.383+0000 7f599d3bf700 1 -- 172.21.15.47:0/1841263625 shutdown_connections 2024-04-17T06:00:25.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:25.383+0000 7f599d3bf700 1 -- 172.21.15.47:0/1841263625 wait complete. 2024-04-17T06:00:25.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:25.384+0000 7f599d3bf700 1 Processor -- start 2024-04-17T06:00:25.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:25.384+0000 7f599d3bf700 1 -- start start 2024-04-17T06:00:25.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:25.384+0000 7f599d3bf700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5998101950 0x7f599819a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:25.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:25.384+0000 7f599d3bf700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5998102310 0x7f599819a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:25.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:25.384+0000 7f599d3bf700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f59981079a0 0x7f59981a5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:25.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:25.384+0000 7f599d3bf700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f5998111c60 con 0x7f5998101950 2024-04-17T06:00:25.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:25.384+0000 7f599d3bf700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5998111ae0 con 0x7f59981079a0 2024-04-17T06:00:25.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:25.384+0000 7f599d3bf700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5998111de0 con 0x7f5998102310 2024-04-17T06:00:25.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:25.384+0000 7f5996ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5998101950 0x7f599819a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:25.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:25.384+0000 7f5996ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5998101950 0x7f599819a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:42938/0 (socket says 172.21.15.47:42938) 2024-04-17T06:00:25.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:25.384+0000 7f5996ffd700 1 -- 172.21.15.47:0/76547765 learned_addr learned my addr 172.21.15.47:0/76547765 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:00:25.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:25.384+0000 7f598e7fc700 1 --2- 172.21.15.47:0/76547765 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5998102310 0x7f599819a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:25.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:25.384+0000 7f59977fe700 1 --2- 172.21.15.47:0/76547765 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f59981079a0 0x7f59981a5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:25.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:25.385+0000 7f5996ffd700 1 -- 172.21.15.47:0/76547765 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5998102310 msgr2=0x7f599819a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:25.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:25.385+0000 7f5996ffd700 1 --2- 172.21.15.47:0/76547765 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5998102310 0x7f599819a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:25.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:25.385+0000 7f5996ffd700 1 -- 172.21.15.47:0/76547765 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f59981079a0 msgr2=0x7f59981a5080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:25.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:25.385+0000 7f5996ffd700 1 --2- 172.21.15.47:0/76547765 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f59981079a0 0x7f59981a5080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:25.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:25.385+0000 7f5996ffd700 1 -- 172.21.15.47:0/76547765 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59981a55c0 con 0x7f5998101950 2024-04-17T06:00:25.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:25.385+0000 7f598e7fc700 1 --2- 172.21.15.47:0/76547765 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5998102310 0x7f599819a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T06:00:25.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:25.385+0000 7f5996ffd700 1 --2- 172.21.15.47:0/76547765 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5998101950 0x7f599819a000 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f598800d950 tx=0x7f598800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:00:25.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:25.385+0000 7f5994ff9700 1 -- 172.21.15.47:0/76547765 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5988016070 con 0x7f5998101950 2024-04-17T06:00:25.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:25.385+0000 7f5994ff9700 1 -- 172.21.15.47:0/76547765 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f598800fd70 con 0x7f5998101950 2024-04-17T06:00:25.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:25.385+0000 7f599d3bf700 1 -- 172.21.15.47:0/76547765 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59981a57f0 con 0x7f5998101950 2024-04-17T06:00:25.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:25.385+0000 7f5994ff9700 1 -- 172.21.15.47:0/76547765 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f598800e610 con 0x7f5998101950 2024-04-17T06:00:25.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:25.385+0000 7f599d3bf700 1 -- 172.21.15.47:0/76547765 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59981a5d50 con 0x7f5998101950 2024-04-17T06:00:25.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:25.386+0000 7f599d3bf700 1 -- 172.21.15.47:0/76547765 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5998103070 con 0x7f5998101950 2024-04-17T06:00:25.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:25.387+0000 7f5994ff9700 1 -- 172.21.15.47:0/76547765 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f5988019070 con 0x7f5998101950 2024-04-17T06:00:25.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:25.388+0000 7f5994ff9700 1 --2- 172.21.15.47:0/76547765 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f59840a3d50 0x7f59840a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:25.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:25.388+0000 7f5994ff9700 1 -- 172.21.15.47:0/76547765 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f59880cfef0 con 0x7f5998101950 2024-04-17T06:00:25.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:25.388+0000 7f598e7fc700 1 --2- 172.21.15.47:0/76547765 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f59840a3d50 0x7f59840a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:25.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:25.388+0000 7f598e7fc700 1 --2- 172.21.15.47:0/76547765 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f59840a3d50 0x7f59840a6210 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5998078e70 tx=0x7f5980009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:00:25.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:25.393+0000 7f5994ff9700 1 -- 172.21.15.47:0/76547765 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f598809c780 con 0x7f5998101950 2024-04-17T06:00:25.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:25.606+0000 7f599d3bf700 1 -- 172.21.15.47:0/76547765 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f599802e150 con 0x7f5998101950 2024-04-17T06:00:25.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:25.607+0000 7f5994ff9700 1 -- 172.21.15.47:0/76547765 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f598809c080 con 0x7f5998101950 2024-04-17T06:00:25.605 INFO:teuthology.orchestra.run.smithi047.stdout:167503724601 2024-04-17T06:00:25.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:25.609+0000 7f599d3bf700 1 -- 172.21.15.47:0/76547765 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f59840a3d50 msgr2=0x7f59840a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:25.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:25.609+0000 7f599d3bf700 1 --2- 172.21.15.47:0/76547765 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f59840a3d50 0x7f59840a6210 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5998078e70 tx=0x7f5980009260 comp rx=0 tx=0).stop 2024-04-17T06:00:25.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:25.609+0000 7f599d3bf700 1 -- 172.21.15.47:0/76547765 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5998101950 msgr2=0x7f599819a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:25.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:25.609+0000 7f599d3bf700 1 --2- 172.21.15.47:0/76547765 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5998101950 0x7f599819a000 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f598800d950 tx=0x7f598800dc60 comp rx=0 tx=0).stop 2024-04-17T06:00:25.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:25.609+0000 7f599d3bf700 1 -- 172.21.15.47:0/76547765 shutdown_connections 2024-04-17T06:00:25.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:25.609+0000 7f599d3bf700 1 --2- 172.21.15.47:0/76547765 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f59840a3d50 0x7f59840a6210 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:25.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:25.609+0000 7f599d3bf700 1 --2- 172.21.15.47:0/76547765 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f59981079a0 0x7f59981a5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:25.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:25.609+0000 7f599d3bf700 1 --2- 172.21.15.47:0/76547765 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5998101950 0x7f599819a000 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:25.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:25.609+0000 7f599d3bf700 1 --2- 172.21.15.47:0/76547765 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5998102310 0x7f599819a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:25.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:25.609+0000 7f599d3bf700 1 -- 172.21.15.47:0/76547765 >> 172.21.15.47:0/76547765 conn(0x7f5998078900 msgr2=0x7f59980ff9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:00:25.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:25.609+0000 7f599d3bf700 1 -- 172.21.15.47:0/76547765 shutdown_connections 2024-04-17T06:00:25.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:25.609+0000 7f599d3bf700 1 -- 172.21.15.47:0/76547765 wait complete. 2024-04-17T06:00:25.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:00:25 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/790971714' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-17T06:00:25.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:00:25 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/76547765' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-17T06:00:26.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:26.030+0000 7f78f5a96700 1 -- 172.21.15.47:0/4020807693 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78f0102aa0 msgr2=0x7f78f0102f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:26.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:26.030+0000 7f78f5a96700 1 --2- 172.21.15.47:0/4020807693 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78f0102aa0 0x7f78f0102f20 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f78e4009990 tx=0x7f78e4009ca0 comp rx=0 tx=0).stop 2024-04-17T06:00:26.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:26.030+0000 7f78f5a96700 1 -- 172.21.15.47:0/4020807693 shutdown_connections 2024-04-17T06:00:26.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:26.030+0000 7f78f5a96700 1 --2- 172.21.15.47:0/4020807693 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f78f0103460 0x7f78f010f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:26.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:26.030+0000 7f78f5a96700 1 --2- 172.21.15.47:0/4020807693 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f78f0108a60 0x7f78f0108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:26.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:26.030+0000 7f78f5a96700 1 --2- 172.21.15.47:0/4020807693 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78f0102aa0 0x7f78f0102f20 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:26.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:26.030+0000 7f78f5a96700 1 -- 172.21.15.47:0/4020807693 >> 172.21.15.47:0/4020807693 conn(0x7f78f0079bb0 msgr2=0x7f78f0079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:00:26.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:26.030+0000 7f78f5a96700 1 -- 172.21.15.47:0/4020807693 shutdown_connections 2024-04-17T06:00:26.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:26.030+0000 7f78f5a96700 1 -- 172.21.15.47:0/4020807693 wait complete. 2024-04-17T06:00:26.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:26.030+0000 7f78f5a96700 1 Processor -- start 2024-04-17T06:00:26.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:26.030+0000 7f78f5a96700 1 -- start start 2024-04-17T06:00:26.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:26.031+0000 7f78f5a96700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f78f0102aa0 0x7f78f0199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:26.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:26.031+0000 7f78f5a96700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f78f0103460 0x7f78f0199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:26.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:26.031+0000 7f78f5a96700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78f0108a60 0x7f78f019dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:26.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:26.031+0000 7f78f5a96700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f78f0112d80 con 0x7f78f0108a60 2024-04-17T06:00:26.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:26.031+0000 7f78f5a96700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f78f0112c00 con 0x7f78f0103460 2024-04-17T06:00:26.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:26.031+0000 7f78f5a96700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f78f0112f00 con 0x7f78f0102aa0 2024-04-17T06:00:26.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:26.031+0000 7f78effff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78f0108a60 0x7f78f019dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:26.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:26.031+0000 7f78effff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78f0108a60 0x7f78f019dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:42950/0 (socket says 172.21.15.47:42950) 2024-04-17T06:00:26.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:26.031+0000 7f78effff700 1 -- 172.21.15.47:0/1664425966 learned_addr learned my addr 172.21.15.47:0/1664425966 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:00:26.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:26.031+0000 7f78ef7fe700 1 --2- 172.21.15.47:0/1664425966 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f78f0102aa0 0x7f78f0199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:26.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:26.031+0000 7f78eeffd700 1 --2- 172.21.15.47:0/1664425966 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f78f0103460 0x7f78f0199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:26.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:26.031+0000 7f78effff700 1 -- 172.21.15.47:0/1664425966 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f78f0102aa0 msgr2=0x7f78f0199230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:26.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:26.031+0000 7f78effff700 1 --2- 172.21.15.47:0/1664425966 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f78f0102aa0 0x7f78f0199230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:26.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:26.031+0000 7f78effff700 1 -- 172.21.15.47:0/1664425966 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f78f0103460 msgr2=0x7f78f0199770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:26.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:26.031+0000 7f78effff700 1 --2- 172.21.15.47:0/1664425966 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f78f0103460 0x7f78f0199770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:26.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:26.031+0000 7f78effff700 1 -- 172.21.15.47:0/1664425966 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78f019e1f0 con 0x7f78f0108a60 2024-04-17T06:00:26.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:26.031+0000 7f78ef7fe700 1 --2- 172.21.15.47:0/1664425966 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f78f0102aa0 0x7f78f0199230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T06:00:26.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:26.031+0000 7f78effff700 1 --2- 172.21.15.47:0/1664425966 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78f0108a60 0x7f78f019dab0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f78e800ef30 tx=0x7f78e800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:00:26.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:26.031+0000 7f78ecff9700 1 -- 172.21.15.47:0/1664425966 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78e80092e0 con 0x7f78f0108a60 2024-04-17T06:00:26.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:26.031+0000 7f78f5a96700 1 -- 172.21.15.47:0/1664425966 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78f019e480 con 0x7f78f0108a60 2024-04-17T06:00:26.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:26.032+0000 7f78ecff9700 1 -- 172.21.15.47:0/1664425966 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78e8004bb0 con 0x7f78f0108a60 2024-04-17T06:00:26.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:26.032+0000 7f78f5a96700 1 -- 172.21.15.47:0/1664425966 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78f01a5d60 con 0x7f78f0108a60 2024-04-17T06:00:26.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:26.032+0000 7f78ecff9700 1 -- 172.21.15.47:0/1664425966 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78e8004280 con 0x7f78f0108a60 2024-04-17T06:00:26.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:26.032+0000 7f78f5a96700 1 -- 172.21.15.47:0/1664425966 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78f0069740 con 0x7f78f0108a60 2024-04-17T06:00:26.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:26.034+0000 7f78ecff9700 1 -- 172.21.15.47:0/1664425966 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f78e801b030 con 0x7f78f0108a60 2024-04-17T06:00:26.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:26.034+0000 7f78ecff9700 1 --2- 172.21.15.47:0/1664425966 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f78d80a3d50 0x7f78d80a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:26.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:26.034+0000 7f78ef7fe700 1 --2- 172.21.15.47:0/1664425966 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f78d80a3d50 0x7f78d80a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:26.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:26.035+0000 7f78ecff9700 1 -- 172.21.15.47:0/1664425966 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f78e80d0920 con 0x7f78f0108a60 2024-04-17T06:00:26.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:26.035+0000 7f78ef7fe700 1 --2- 172.21.15.47:0/1664425966 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f78d80a3d50 0x7f78d80a6210 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f78e0005950 tx=0x7f78e000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:00:26.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:26.038+0000 7f78ecff9700 1 -- 172.21.15.47:0/1664425966 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f78e8022020 con 0x7f78f0108a60 2024-04-17T06:00:26.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:00:25 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/790971714' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-17T06:00:26.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:00:25 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/76547765' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-17T06:00:26.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:26.250+0000 7f78f5a96700 1 -- 172.21.15.47:0/1664425966 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f78f002df90 con 0x7f78f0108a60 2024-04-17T06:00:26.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:26.250+0000 7f78ecff9700 1 -- 172.21.15.47:0/1664425966 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f78e8010040 con 0x7f78f0108a60 2024-04-17T06:00:26.248 INFO:teuthology.orchestra.run.smithi047.stdout:292057776166 2024-04-17T06:00:26.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:26.250+0000 7fc239706700 1 -- 172.21.15.47:0/2458080911 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc234107e20 msgr2=0x7fc23410f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:26.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:26.250+0000 7fc239706700 1 --2- 172.21.15.47:0/2458080911 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc234107e20 0x7fc23410f9a0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fc22c00b0d0 tx=0x7fc22c00b3e0 comp rx=0 tx=0).stop 2024-04-17T06:00:26.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:26.250+0000 7fc239706700 1 -- 172.21.15.47:0/2458080911 shutdown_connections 2024-04-17T06:00:26.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:26.251+0000 7fc239706700 1 --2- 172.21.15.47:0/2458080911 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc234107e20 0x7fc23410f9a0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:26.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:26.251+0000 7fc239706700 1 --2- 172.21.15.47:0/2458080911 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc23406b380 0x7fc23406b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:26.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:26.251+0000 7fc239706700 1 --2- 172.21.15.47:0/2458080911 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc23406bd30 0x7fc2341076f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:26.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:26.251+0000 7fc239706700 1 -- 172.21.15.47:0/2458080911 >> 172.21.15.47:0/2458080911 conn(0x7fc234078900 msgr2=0x7fc234078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:00:26.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:26.251+0000 7fc239706700 1 -- 172.21.15.47:0/2458080911 shutdown_connections 2024-04-17T06:00:26.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:26.251+0000 7fc239706700 1 -- 172.21.15.47:0/2458080911 wait complete. 2024-04-17T06:00:26.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:26.251+0000 7fc239706700 1 Processor -- start 2024-04-17T06:00:26.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:26.251+0000 7fc239706700 1 -- start start 2024-04-17T06:00:26.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:26.252+0000 7fc239706700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc23406b380 0x7fc234199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:26.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:26.252+0000 7fc239706700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc23406bd30 0x7fc234199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:26.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:26.252+0000 7fc239706700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc234107e20 0x7fc23419da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:26.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:26.252+0000 7fc239706700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fc234112da0 con 0x7fc234107e20 2024-04-17T06:00:26.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:26.252+0000 7fc239706700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc234112c20 con 0x7fc23406b380 2024-04-17T06:00:26.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:26.252+0000 7fc239706700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc234112f20 con 0x7fc23406bd30 2024-04-17T06:00:26.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:26.252+0000 7fc2337fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc234107e20 0x7fc23419da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:26.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:26.252+0000 7fc2337fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc234107e20 0x7fc23419da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:42966/0 (socket says 172.21.15.47:42966) 2024-04-17T06:00:26.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:26.252+0000 7fc2337fe700 1 -- 172.21.15.47:0/2633685015 learned_addr learned my addr 172.21.15.47:0/2633685015 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:00:26.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:26.252+0000 7fc232ffd700 1 --2- 172.21.15.47:0/2633685015 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc23406b380 0x7fc234199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:26.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:26.252+0000 7fc2327fc700 1 --2- 172.21.15.47:0/2633685015 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc23406bd30 0x7fc234199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:26.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:26.252+0000 7fc2337fe700 1 -- 172.21.15.47:0/2633685015 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc23406bd30 msgr2=0x7fc234199670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:26.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:26.252+0000 7fc2337fe700 1 --2- 172.21.15.47:0/2633685015 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc23406bd30 0x7fc234199670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:26.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:26.252+0000 7fc2337fe700 1 -- 172.21.15.47:0/2633685015 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc23406b380 msgr2=0x7fc234199130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:26.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:26.252+0000 7fc2337fe700 1 --2- 172.21.15.47:0/2633685015 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc23406b380 0x7fc234199130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:26.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:26.252+0000 7fc2337fe700 1 -- 172.21.15.47:0/2633685015 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc23419e140 con 0x7fc234107e20 2024-04-17T06:00:26.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:26.252+0000 7fc2327fc700 1 --2- 172.21.15.47:0/2633685015 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc23406bd30 0x7fc234199670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T06:00:26.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:26.252+0000 7fc232ffd700 1 --2- 172.21.15.47:0/2633685015 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc23406b380 0x7fc234199130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T06:00:26.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:26.252+0000 7fc2337fe700 1 --2- 172.21.15.47:0/2633685015 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc234107e20 0x7fc23419da00 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fc22c00d010 tx=0x7fc22c0079a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:00:26.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:26.253+0000 7fc21bfff700 1 -- 172.21.15.47:0/2633685015 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc22c003c00 con 0x7fc234107e20 2024-04-17T06:00:26.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:26.253+0000 7fc21bfff700 1 -- 172.21.15.47:0/2633685015 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc22c022070 con 0x7fc234107e20 2024-04-17T06:00:26.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:26.253+0000 7fc239706700 1 -- 172.21.15.47:0/2633685015 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc23419e3d0 con 0x7fc234107e20 2024-04-17T06:00:26.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:26.253+0000 7fc21bfff700 1 -- 172.21.15.47:0/2633685015 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc22c01b6c0 con 0x7fc234107e20 2024-04-17T06:00:26.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:26.253+0000 7fc239706700 1 -- 172.21.15.47:0/2633685015 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2341a5cb0 con 0x7fc234107e20 2024-04-17T06:00:26.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:26.253+0000 7f78f5a96700 1 -- 172.21.15.47:0/1664425966 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f78d80a3d50 msgr2=0x7f78d80a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:26.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:26.253+0000 7f78f5a96700 1 --2- 172.21.15.47:0/1664425966 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f78d80a3d50 0x7f78d80a6210 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f78e0005950 tx=0x7f78e000b410 comp rx=0 tx=0).stop 2024-04-17T06:00:26.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:26.253+0000 7f78f5a96700 1 -- 172.21.15.47:0/1664425966 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78f0108a60 msgr2=0x7f78f019dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:26.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:26.253+0000 7f78f5a96700 1 --2- 172.21.15.47:0/1664425966 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78f0108a60 0x7f78f019dab0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f78e800ef30 tx=0x7f78e800c3a0 comp rx=0 tx=0).stop 2024-04-17T06:00:26.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:26.253+0000 7f78f5a96700 1 -- 172.21.15.47:0/1664425966 shutdown_connections 2024-04-17T06:00:26.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:26.253+0000 7f78f5a96700 1 --2- 172.21.15.47:0/1664425966 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f78d80a3d50 0x7f78d80a6210 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:26.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:26.253+0000 7f78f5a96700 1 --2- 172.21.15.47:0/1664425966 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78f0108a60 0x7f78f019dab0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:26.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:26.253+0000 7f78f5a96700 1 --2- 172.21.15.47:0/1664425966 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f78f0102aa0 0x7f78f0199230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:26.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:26.253+0000 7f78f5a96700 1 --2- 172.21.15.47:0/1664425966 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f78f0103460 0x7f78f0199770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:26.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:26.253+0000 7f78f5a96700 1 -- 172.21.15.47:0/1664425966 >> 172.21.15.47:0/1664425966 conn(0x7f78f0079bb0 msgr2=0x7f78f0101770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:00:26.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:26.253+0000 7f78f5a96700 1 -- 172.21.15.47:0/1664425966 shutdown_connections 2024-04-17T06:00:26.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:26.253+0000 7f78f5a96700 1 -- 172.21.15.47:0/1664425966 wait complete. 2024-04-17T06:00:26.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:26.254+0000 7fc239706700 1 -- 172.21.15.47:0/2633685015 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2341041a0 con 0x7fc234107e20 2024-04-17T06:00:26.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:26.256+0000 7fc21bfff700 1 -- 172.21.15.47:0/2633685015 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fc22c009ba0 con 0x7fc234107e20 2024-04-17T06:00:26.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:26.256+0000 7fc21bfff700 1 --2- 172.21.15.47:0/2633685015 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fc21c0a3cd0 0x7fc21c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:26.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:26.256+0000 7fc21bfff700 1 -- 172.21.15.47:0/2633685015 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fc22c0d1580 con 0x7fc234107e20 2024-04-17T06:00:26.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:26.256+0000 7fc232ffd700 1 --2- 172.21.15.47:0/2633685015 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fc21c0a3cd0 0x7fc21c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:26.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:26.257+0000 7fc232ffd700 1 --2- 172.21.15.47:0/2633685015 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fc21c0a3cd0 0x7fc21c0a6190 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc224011490 tx=0x7fc224009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:00:26.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:26.260+0000 7fc21bfff700 1 -- 172.21.15.47:0/2633685015 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fc22c02c020 con 0x7fc234107e20 2024-04-17T06:00:26.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:00:25 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/790971714' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-17T06:00:26.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:00:25 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/76547765' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-17T06:00:26.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:26.470+0000 7fc239706700 1 -- 172.21.15.47:0/2633685015 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fc23402df90 con 0x7fc234107e20 2024-04-17T06:00:26.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:26.470+0000 7fc21bfff700 1 -- 172.21.15.47:0/2633685015 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc22c01a3c0 con 0x7fc234107e20 2024-04-17T06:00:26.469 INFO:teuthology.orchestra.run.smithi047.stdout:55834574926 2024-04-17T06:00:26.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:26.473+0000 7fc239706700 1 -- 172.21.15.47:0/2633685015 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fc21c0a3cd0 msgr2=0x7fc21c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:26.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:26.473+0000 7fc239706700 1 --2- 172.21.15.47:0/2633685015 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fc21c0a3cd0 0x7fc21c0a6190 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc224011490 tx=0x7fc224009260 comp rx=0 tx=0).stop 2024-04-17T06:00:26.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:26.473+0000 7fc239706700 1 -- 172.21.15.47:0/2633685015 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc234107e20 msgr2=0x7fc23419da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:26.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:26.473+0000 7fc239706700 1 --2- 172.21.15.47:0/2633685015 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc234107e20 0x7fc23419da00 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fc22c00d010 tx=0x7fc22c0079a0 comp rx=0 tx=0).stop 2024-04-17T06:00:26.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:26.473+0000 7fc239706700 1 -- 172.21.15.47:0/2633685015 shutdown_connections 2024-04-17T06:00:26.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:26.473+0000 7fc239706700 1 --2- 172.21.15.47:0/2633685015 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fc21c0a3cd0 0x7fc21c0a6190 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:26.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:26.473+0000 7fc239706700 1 --2- 172.21.15.47:0/2633685015 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc234107e20 0x7fc23419da00 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:26.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:26.473+0000 7fc239706700 1 --2- 172.21.15.47:0/2633685015 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc23406b380 0x7fc234199130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:26.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:26.473+0000 7fc239706700 1 --2- 172.21.15.47:0/2633685015 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc23406bd30 0x7fc234199670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:26.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:26.473+0000 7fc239706700 1 -- 172.21.15.47:0/2633685015 >> 172.21.15.47:0/2633685015 conn(0x7fc234078900 msgr2=0x7fc2340ffe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:00:26.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:26.473+0000 7fc239706700 1 -- 172.21.15.47:0/2633685015 shutdown_connections 2024-04-17T06:00:26.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:26.473+0000 7fc239706700 1 -- 172.21.15.47:0/2633685015 wait complete. 2024-04-17T06:00:26.916 INFO:tasks.cephadm.ceph_manager.ceph:need seq 292057776160 got 292057776166 for osd.9 2024-04-17T06:00:26.916 DEBUG:teuthology.parallel:result is None 2024-04-17T06:00:26.974 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:00:26 smithi047 ceph-mon[32223]: pgmap v324: 97 pgs: 97 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s; 5.3 KiB/s, 1 objects/s recovering 2024-04-17T06:00:26.974 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:00:26 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/1664425966' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-17T06:00:26.975 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:00:26 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/2633685015' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-17T06:00:27.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:00:26 smithi098 ceph-mon[36516]: pgmap v324: 97 pgs: 97 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s; 5.3 KiB/s, 1 objects/s recovering 2024-04-17T06:00:27.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:00:26 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/1664425966' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-17T06:00:27.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:00:26 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/2633685015' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-17T06:00:27.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:00:26 smithi103 ceph-mon[36243]: pgmap v324: 97 pgs: 97 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s; 5.3 KiB/s, 1 objects/s recovering 2024-04-17T06:00:27.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:00:26 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/1664425966' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-17T06:00:27.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:00:26 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/2633685015' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-17T06:00:27.750 INFO:tasks.cephadm.ceph_manager.ceph:need seq 146028888122 got 146028888126 for osd.4 2024-04-17T06:00:27.750 DEBUG:teuthology.parallel:result is None 2024-04-17T06:00:28.180 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724597 got 167503724601 for osd.5 2024-04-17T06:00:28.180 DEBUG:teuthology.parallel:result is None 2024-04-17T06:00:28.853 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411397 got 77309411401 for osd.2 2024-04-17T06:00:28.854 DEBUG:teuthology.parallel:result is None 2024-04-17T06:00:29.092 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:00:28 smithi047 ceph-mon[32223]: pgmap v325: 97 pgs: 97 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 4.8 KiB/s, 0 objects/s recovering 2024-04-17T06:00:29.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:00:28 smithi098 ceph-mon[36516]: pgmap v325: 97 pgs: 97 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 4.8 KiB/s, 0 objects/s recovering 2024-04-17T06:00:29.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:00:28 smithi103 ceph-mon[36243]: pgmap v325: 97 pgs: 97 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 4.8 KiB/s, 0 objects/s recovering 2024-04-17T06:00:30.211 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574920 got 55834574926 for osd.1 2024-04-17T06:00:30.211 DEBUG:teuthology.parallel:result is None 2024-04-17T06:00:30.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:30.673+0000 7f6448950700 1 -- 172.21.15.47:0/4081575004 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6444101950 msgr2=0x7f6444101dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:30.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:30.673+0000 7f6448950700 1 --2- 172.21.15.47:0/4081575004 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6444101950 0x7f6444101dd0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f6438009990 tx=0x7f6438009ca0 comp rx=0 tx=0).stop 2024-04-17T06:00:30.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:30.673+0000 7f6448950700 1 -- 172.21.15.47:0/4081575004 shutdown_connections 2024-04-17T06:00:30.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:30.673+0000 7f6448950700 1 --2- 172.21.15.47:0/4081575004 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6444102310 0x7f644410e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:30.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:30.673+0000 7f6448950700 1 --2- 172.21.15.47:0/4081575004 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f64441079a0 0x7f6444107d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:30.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:30.673+0000 7f6448950700 1 --2- 172.21.15.47:0/4081575004 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6444101950 0x7f6444101dd0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:30.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:30.673+0000 7f6448950700 1 -- 172.21.15.47:0/4081575004 >> 172.21.15.47:0/4081575004 conn(0x7f6444078900 msgr2=0x7f6444078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:00:30.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:30.673+0000 7f6448950700 1 -- 172.21.15.47:0/4081575004 shutdown_connections 2024-04-17T06:00:30.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:30.673+0000 7f6448950700 1 -- 172.21.15.47:0/4081575004 wait complete. 2024-04-17T06:00:30.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:30.674+0000 7f6448950700 1 Processor -- start 2024-04-17T06:00:30.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:30.674+0000 7f6448950700 1 -- start start 2024-04-17T06:00:30.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:30.674+0000 7f6448950700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6444101950 0x7f6444103500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:30.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:30.674+0000 7f6448950700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6444102310 0x7f6444103a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:30.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:30.674+0000 7f6448950700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64441079a0 0x7f6444103f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:30.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:30.674+0000 7f6448950700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f6444111c50 con 0x7f64441079a0 2024-04-17T06:00:30.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:30.674+0000 7f6448950700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6444111ad0 con 0x7f6444102310 2024-04-17T06:00:30.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:30.674+0000 7f6448950700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6444111dd0 con 0x7f6444101950 2024-04-17T06:00:30.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:30.674+0000 7f6442d9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64441079a0 0x7f6444103f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:30.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:30.674+0000 7f6442d9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64441079a0 0x7f6444103f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:42988/0 (socket says 172.21.15.47:42988) 2024-04-17T06:00:30.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:30.674+0000 7f6442d9d700 1 -- 172.21.15.47:0/4017789181 learned_addr learned my addr 172.21.15.47:0/4017789181 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:00:30.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:30.674+0000 7f644259c700 1 --2- 172.21.15.47:0/4017789181 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6444101950 0x7f6444103500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:30.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:30.674+0000 7f6441d9b700 1 --2- 172.21.15.47:0/4017789181 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6444102310 0x7f6444103a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:30.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:30.674+0000 7f6442d9d700 1 -- 172.21.15.47:0/4017789181 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6444101950 msgr2=0x7f6444103500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:30.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:30.674+0000 7f6442d9d700 1 --2- 172.21.15.47:0/4017789181 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6444101950 0x7f6444103500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:30.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:30.674+0000 7f6442d9d700 1 -- 172.21.15.47:0/4017789181 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6444102310 msgr2=0x7f6444103a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:30.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:30.674+0000 7f6442d9d700 1 --2- 172.21.15.47:0/4017789181 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6444102310 0x7f6444103a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:30.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:30.674+0000 7f6442d9d700 1 -- 172.21.15.47:0/4017789181 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64441a7620 con 0x7f64441079a0 2024-04-17T06:00:30.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:30.674+0000 7f6441d9b700 1 --2- 172.21.15.47:0/4017789181 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6444102310 0x7f6444103a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:00:30.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:30.674+0000 7f6442d9d700 1 --2- 172.21.15.47:0/4017789181 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64441079a0 0x7f6444103f80 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f643c007ee0 tx=0x7f643c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:00:30.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:30.675+0000 7f64337fe700 1 -- 172.21.15.47:0/4017789181 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f643c00dbd0 con 0x7f64441079a0 2024-04-17T06:00:30.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:30.675+0000 7f64337fe700 1 -- 172.21.15.47:0/4017789181 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f643c00dd70 con 0x7f64441079a0 2024-04-17T06:00:30.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:30.675+0000 7f6448950700 1 -- 172.21.15.47:0/4017789181 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64441a7910 con 0x7f64441079a0 2024-04-17T06:00:30.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:30.675+0000 7f64337fe700 1 -- 172.21.15.47:0/4017789181 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f643c005a60 con 0x7f64441079a0 2024-04-17T06:00:30.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:30.675+0000 7f6448950700 1 -- 172.21.15.47:0/4017789181 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64441a7e50 con 0x7f64441079a0 2024-04-17T06:00:30.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:30.676+0000 7f6448950700 1 -- 172.21.15.47:0/4017789181 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6444069740 con 0x7f64441079a0 2024-04-17T06:00:30.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:30.678+0000 7f64337fe700 1 -- 172.21.15.47:0/4017789181 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f643c01a020 con 0x7f64441079a0 2024-04-17T06:00:30.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:30.678+0000 7f64337fe700 1 --2- 172.21.15.47:0/4017789181 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f642c0a3cd0 0x7f642c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:30.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:30.678+0000 7f644259c700 1 --2- 172.21.15.47:0/4017789181 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f642c0a3cd0 0x7f642c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:30.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:30.678+0000 7f64337fe700 1 -- 172.21.15.47:0/4017789181 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f643c0d02e0 con 0x7f64441079a0 2024-04-17T06:00:30.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:30.679+0000 7f644259c700 1 --2- 172.21.15.47:0/4017789181 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f642c0a3cd0 0x7f642c0a6190 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f643400abd0 tx=0x7f643400a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:00:30.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:30.682+0000 7f64337fe700 1 -- 172.21.15.47:0/4017789181 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f643c010070 con 0x7f64441079a0 2024-04-17T06:00:30.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:30.948+0000 7f6448950700 1 -- 172.21.15.47:0/4017789181 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f644402e150 con 0x7f64441079a0 2024-04-17T06:00:30.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:30.948+0000 7f64337fe700 1 -- 172.21.15.47:0/4017789181 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f643c09cb70 con 0x7f64441079a0 2024-04-17T06:00:30.946 INFO:teuthology.orchestra.run.smithi047.stdout:313532612642 2024-04-17T06:00:30.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:30.951+0000 7f6448950700 1 -- 172.21.15.47:0/4017789181 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f642c0a3cd0 msgr2=0x7f642c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:30.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:30.951+0000 7f6448950700 1 --2- 172.21.15.47:0/4017789181 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f642c0a3cd0 0x7f642c0a6190 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f643400abd0 tx=0x7f643400a380 comp rx=0 tx=0).stop 2024-04-17T06:00:30.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:30.951+0000 7f6448950700 1 -- 172.21.15.47:0/4017789181 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64441079a0 msgr2=0x7f6444103f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:30.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:30.951+0000 7f6448950700 1 --2- 172.21.15.47:0/4017789181 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64441079a0 0x7f6444103f80 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f643c007ee0 tx=0x7f643c00d3b0 comp rx=0 tx=0).stop 2024-04-17T06:00:30.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:30.951+0000 7f6448950700 1 -- 172.21.15.47:0/4017789181 shutdown_connections 2024-04-17T06:00:30.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:30.951+0000 7f6448950700 1 --2- 172.21.15.47:0/4017789181 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f642c0a3cd0 0x7f642c0a6190 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:30.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:30.951+0000 7f6448950700 1 --2- 172.21.15.47:0/4017789181 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64441079a0 0x7f6444103f80 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:30.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:30.951+0000 7f6448950700 1 --2- 172.21.15.47:0/4017789181 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6444101950 0x7f6444103500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:30.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:30.951+0000 7f6448950700 1 --2- 172.21.15.47:0/4017789181 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6444102310 0x7f6444103a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:30.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:30.951+0000 7f6448950700 1 -- 172.21.15.47:0/4017789181 >> 172.21.15.47:0/4017789181 conn(0x7f6444078900 msgr2=0x7f644406a9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:00:30.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:30.951+0000 7f6448950700 1 -- 172.21.15.47:0/4017789181 shutdown_connections 2024-04-17T06:00:30.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:30.951+0000 7f6448950700 1 -- 172.21.15.47:0/4017789181 wait complete. 2024-04-17T06:00:31.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:00:30 smithi098 ceph-mon[36516]: pgmap v326: 97 pgs: 97 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s; 4.8 KiB/s, 0 objects/s recovering 2024-04-17T06:00:31.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:00:30 smithi047 ceph-mon[32223]: pgmap v326: 97 pgs: 97 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s; 4.8 KiB/s, 0 objects/s recovering 2024-04-17T06:00:31.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:00:30 smithi103 ceph-mon[36243]: pgmap v326: 97 pgs: 97 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s; 4.8 KiB/s, 0 objects/s recovering 2024-04-17T06:00:31.654 INFO:tasks.cephadm.ceph_manager.ceph:need seq 313532612635 got 313532612642 for osd.10 2024-04-17T06:00:31.654 DEBUG:teuthology.parallel:result is None 2024-04-17T06:00:31.654 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-17T06:00:31.655 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 565db2d8-fc7e-11ee-bc91-c7b262605968 -- ceph pg dump --format=json 2024-04-17T06:00:31.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:00:31 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/4017789181' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-17T06:00:31.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:00:31 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T06:00:31.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:00:31 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T06:00:31.908 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T06:00:32.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:00:31 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/4017789181' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-17T06:00:32.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:00:31 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T06:00:32.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:00:31 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T06:00:32.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:00:31 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/4017789181' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-17T06:00:32.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:00:31 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T06:00:32.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:00:31 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T06:00:33.005 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:00:32 smithi047 ceph-mon[32223]: pgmap v327: 97 pgs: 97 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-17T06:00:33.005 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:00:32 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T06:00:33.005 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:00:32 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T06:00:33.005 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:00:32 smithi047 ceph-mon[32223]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T06:00:33.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:00:32 smithi098 ceph-mon[36516]: pgmap v327: 97 pgs: 97 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-17T06:00:33.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:00:32 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T06:00:33.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:00:32 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T06:00:33.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:00:32 smithi098 ceph-mon[36516]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T06:00:33.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:00:32 smithi103 ceph-mon[36243]: pgmap v327: 97 pgs: 97 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-17T06:00:33.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:00:32 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T06:00:33.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:00:32 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T06:00:33.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:00:32 smithi103 ceph-mon[36243]: from='mgr.14152 172.21.15.47:0/3551407755' entity='mgr.x' 2024-04-17T06:00:33.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:33.546+0000 7f6599233700 1 -- 172.21.15.47:0/4228488150 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6594107990 msgr2=0x7f6594107d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:33.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:33.546+0000 7f6599233700 1 --2- 172.21.15.47:0/4228488150 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6594107990 0x7f6594107d70 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f6584009990 tx=0x7f6584009ca0 comp rx=0 tx=0).stop 2024-04-17T06:00:33.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:33.546+0000 7f6599233700 1 -- 172.21.15.47:0/4228488150 shutdown_connections 2024-04-17T06:00:33.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:33.546+0000 7f6599233700 1 --2- 172.21.15.47:0/4228488150 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6594102300 0x7f659410e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:33.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:33.546+0000 7f6599233700 1 --2- 172.21.15.47:0/4228488150 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6594107990 0x7f6594107d70 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:33.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:33.546+0000 7f6599233700 1 --2- 172.21.15.47:0/4228488150 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6594101940 0x7f6594101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:33.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:33.546+0000 7f6599233700 1 -- 172.21.15.47:0/4228488150 >> 172.21.15.47:0/4228488150 conn(0x7f65940788c0 msgr2=0x7f6594078cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:00:33.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:33.546+0000 7f6599233700 1 -- 172.21.15.47:0/4228488150 shutdown_connections 2024-04-17T06:00:33.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:33.546+0000 7f6599233700 1 -- 172.21.15.47:0/4228488150 wait complete. 2024-04-17T06:00:33.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:33.547+0000 7f6599233700 1 Processor -- start 2024-04-17T06:00:33.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:33.547+0000 7f6599233700 1 -- start start 2024-04-17T06:00:33.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:33.547+0000 7f6599233700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6594101940 0x7f659419a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:33.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:33.547+0000 7f6599233700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6594102300 0x7f659419a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:33.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:33.547+0000 7f6599233700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6594107990 0x7f65941a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:33.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:33.547+0000 7f6599233700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f6594111cb0 con 0x7f6594102300 2024-04-17T06:00:33.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:33.547+0000 7f6599233700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6594111b30 con 0x7f6594101940 2024-04-17T06:00:33.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:33.547+0000 7f6599233700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6594111e30 con 0x7f6594107990 2024-04-17T06:00:33.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:33.547+0000 7f659259c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6594102300 0x7f659419a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:33.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:33.547+0000 7f659259c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6594102300 0x7f659419a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43014/0 (socket says 172.21.15.47:43014) 2024-04-17T06:00:33.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:33.548+0000 7f659259c700 1 -- 172.21.15.47:0/3677323874 learned_addr learned my addr 172.21.15.47:0/3677323874 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:00:33.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:33.548+0000 7f6592d9d700 1 --2- 172.21.15.47:0/3677323874 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6594101940 0x7f659419a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:33.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:33.548+0000 7f659359e700 1 --2- 172.21.15.47:0/3677323874 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6594107990 0x7f65941a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:33.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:33.548+0000 7f659259c700 1 -- 172.21.15.47:0/3677323874 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6594107990 msgr2=0x7f65941a5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:33.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:33.548+0000 7f659259c700 1 --2- 172.21.15.47:0/3677323874 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6594107990 0x7f65941a5150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:33.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:33.548+0000 7f659259c700 1 -- 172.21.15.47:0/3677323874 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6594101940 msgr2=0x7f659419a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:33.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:33.548+0000 7f659259c700 1 --2- 172.21.15.47:0/3677323874 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6594101940 0x7f659419a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:33.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:33.548+0000 7f659259c700 1 -- 172.21.15.47:0/3677323874 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65941a5690 con 0x7f6594102300 2024-04-17T06:00:33.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:33.548+0000 7f659359e700 1 --2- 172.21.15.47:0/3677323874 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6594107990 0x7f65941a5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:00:33.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:33.548+0000 7f6592d9d700 1 --2- 172.21.15.47:0/3677323874 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6594101940 0x7f659419a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:00:33.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:33.548+0000 7f659259c700 1 --2- 172.21.15.47:0/3677323874 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6594102300 0x7f659419a610 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f657c00b530 tx=0x7f657c00b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:00:33.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:33.548+0000 7f658bfff700 1 -- 172.21.15.47:0/3677323874 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f657c015020 con 0x7f6594102300 2024-04-17T06:00:33.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:33.548+0000 7f6599233700 1 -- 172.21.15.47:0/3677323874 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65941a5860 con 0x7f6594102300 2024-04-17T06:00:33.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:33.548+0000 7f658bfff700 1 -- 172.21.15.47:0/3677323874 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f657c011d10 con 0x7f6594102300 2024-04-17T06:00:33.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:33.548+0000 7f658bfff700 1 -- 172.21.15.47:0/3677323874 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f657c010e50 con 0x7f6594102300 2024-04-17T06:00:33.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:33.548+0000 7f6599233700 1 -- 172.21.15.47:0/3677323874 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65941a5d50 con 0x7f6594102300 2024-04-17T06:00:33.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:33.549+0000 7f6599233700 1 -- 172.21.15.47:0/3677323874 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6594069740 con 0x7f6594102300 2024-04-17T06:00:33.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:33.550+0000 7f658bfff700 1 -- 172.21.15.47:0/3677323874 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f657c018020 con 0x7f6594102300 2024-04-17T06:00:33.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:33.551+0000 7f658bfff700 1 --2- 172.21.15.47:0/3677323874 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f65800a3e70 0x7f65800a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:33.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:33.551+0000 7f6592d9d700 1 --2- 172.21.15.47:0/3677323874 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f65800a3e70 0x7f65800a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:33.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:33.551+0000 7f658bfff700 1 -- 172.21.15.47:0/3677323874 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f657c0cfd50 con 0x7f6594102300 2024-04-17T06:00:33.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:33.551+0000 7f6592d9d700 1 --2- 172.21.15.47:0/3677323874 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f65800a3e70 0x7f65800a6330 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f658400b5c0 tx=0x7f6584005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:00:33.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:33.556+0000 7f658bfff700 1 -- 172.21.15.47:0/3677323874 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f657c01f020 con 0x7f6594102300 2024-04-17T06:00:33.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:33.760+0000 7f6599233700 1 -- 172.21.15.47:0/3677323874 --> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6594062e90 con 0x7f65800a3e70 2024-04-17T06:00:33.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:33.767+0000 7f658bfff700 1 -- 172.21.15.47:0/3677323874 <== mgr.14152 v2:172.21.15.47:6800/3157902806 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258923 (secure 0 0 0) 0x7f6594062e90 con 0x7f65800a3e70 2024-04-17T06:00:33.769 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-17T06:00:33.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:33.770+0000 7f6599233700 1 -- 172.21.15.47:0/3677323874 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f65800a3e70 msgr2=0x7f65800a6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:33.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:33.770+0000 7f6599233700 1 --2- 172.21.15.47:0/3677323874 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f65800a3e70 0x7f65800a6330 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f658400b5c0 tx=0x7f6584005fb0 comp rx=0 tx=0).stop 2024-04-17T06:00:33.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:33.770+0000 7f6599233700 1 -- 172.21.15.47:0/3677323874 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6594102300 msgr2=0x7f659419a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:33.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:33.770+0000 7f6599233700 1 --2- 172.21.15.47:0/3677323874 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6594102300 0x7f659419a610 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f657c00b530 tx=0x7f657c00b8f0 comp rx=0 tx=0).stop 2024-04-17T06:00:33.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:33.770+0000 7f6599233700 1 -- 172.21.15.47:0/3677323874 shutdown_connections 2024-04-17T06:00:33.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:33.770+0000 7f6599233700 1 --2- 172.21.15.47:0/3677323874 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f65800a3e70 0x7f65800a6330 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:33.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:33.770+0000 7f6599233700 1 --2- 172.21.15.47:0/3677323874 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6594107990 0x7f65941a5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:33.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:33.770+0000 7f6599233700 1 --2- 172.21.15.47:0/3677323874 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6594101940 0x7f659419a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:33.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:33.770+0000 7f6599233700 1 --2- 172.21.15.47:0/3677323874 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6594102300 0x7f659419a610 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:33.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:33.770+0000 7f6599233700 1 -- 172.21.15.47:0/3677323874 >> 172.21.15.47:0/3677323874 conn(0x7f65940788c0 msgr2=0x7f65941006c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:00:33.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:33.770+0000 7f6599233700 1 -- 172.21.15.47:0/3677323874 shutdown_connections 2024-04-17T06:00:33.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:33.770+0000 7f6599233700 1 -- 172.21.15.47:0/3677323874 wait complete. 2024-04-17T06:00:33.772 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-04-17T06:00:34.408 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":328,"stamp":"2024-04-17T06:00:33.743668+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":654459,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":578,"num_read_kb":407,"num_write":339,"num_write_kb":3047,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":1240313,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":274,"ondisk_log_size":274,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":365000,"kb_used_data":11612,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124576824,"statfs":{"total":1151940427776,"available":1151566667776,"internally_reserved":0,"allocated":11890688,"data_stored":7467657,"data_compressed":121510,"data_compressed_allocated":155648,"data_compressed_original":1843200,"omap_allocated":19140,"internal_metadata":361674044},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":70,"num_read_kb":35,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003046"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.723843+0000","last_change":"2024-04-17T05:59:14.592927+0000","last_active":"2024-04-17T06:00:14.723843+0000","last_peered":"2024-04-17T06:00:14.723843+0000","last_clean":"2024-04-17T06:00:14.723843+0000","last_became_active":"2024-04-17T05:59:14.592036+0000","last_became_peered":"2024-04-17T05:59:14.592036+0000","last_unstale":"2024-04-17T06:00:14.723843+0000","last_undegraded":"2024-04-17T06:00:14.723843+0000","last_fullsized":"2024-04-17T06:00:14.723843+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:10:51.146397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.726024+0000","last_change":"2024-04-17T05:59:14.587862+0000","last_active":"2024-04-17T06:00:14.726024+0000","last_peered":"2024-04-17T06:00:14.726024+0000","last_clean":"2024-04-17T06:00:14.726024+0000","last_became_active":"2024-04-17T05:59:14.587553+0000","last_became_peered":"2024-04-17T05:59:14.587553+0000","last_unstale":"2024-04-17T06:00:14.726024+0000","last_undegraded":"2024-04-17T06:00:14.726024+0000","last_fullsized":"2024-04-17T06:00:14.726024+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:16:12.224247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.725576+0000","last_change":"2024-04-17T05:59:14.592223+0000","last_active":"2024-04-17T06:00:14.725576+0000","last_peered":"2024-04-17T06:00:14.725576+0000","last_clean":"2024-04-17T06:00:14.725576+0000","last_became_active":"2024-04-17T05:59:14.591892+0000","last_became_peered":"2024-04-17T05:59:14.591892+0000","last_unstale":"2024-04-17T06:00:14.725576+0000","last_undegraded":"2024-04-17T06:00:14.725576+0000","last_fullsized":"2024-04-17T06:00:14.725576+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:23:26.943368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:15.001748+0000","last_change":"2024-04-17T05:59:14.585607+0000","last_active":"2024-04-17T06:00:15.001748+0000","last_peered":"2024-04-17T06:00:15.001748+0000","last_clean":"2024-04-17T06:00:15.001748+0000","last_became_active":"2024-04-17T05:59:14.585255+0000","last_became_peered":"2024-04-17T05:59:14.585255+0000","last_unstale":"2024-04-17T06:00:15.001748+0000","last_undegraded":"2024-04-17T06:00:15.001748+0000","last_fullsized":"2024-04-17T06:00:15.001748+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:45:57.355035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.725535+0000","last_change":"2024-04-17T05:59:14.599703+0000","last_active":"2024-04-17T06:00:14.725535+0000","last_peered":"2024-04-17T06:00:14.725535+0000","last_clean":"2024-04-17T06:00:14.725535+0000","last_became_active":"2024-04-17T05:59:14.599305+0000","last_became_peered":"2024-04-17T05:59:14.599305+0000","last_unstale":"2024-04-17T06:00:14.725535+0000","last_undegraded":"2024-04-17T06:00:14.725535+0000","last_fullsized":"2024-04-17T06:00:14.725535+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:58:36.056590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.729429+0000","last_change":"2024-04-17T05:59:14.586176+0000","last_active":"2024-04-17T06:00:14.729429+0000","last_peered":"2024-04-17T06:00:14.729429+0000","last_clean":"2024-04-17T06:00:14.729429+0000","last_became_active":"2024-04-17T05:59:14.585850+0000","last_became_peered":"2024-04-17T05:59:14.585850+0000","last_unstale":"2024-04-17T06:00:14.729429+0000","last_undegraded":"2024-04-17T06:00:14.729429+0000","last_fullsized":"2024-04-17T06:00:14.729429+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:01:14.628048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.726720+0000","last_change":"2024-04-17T05:59:14.587750+0000","last_active":"2024-04-17T06:00:14.726720+0000","last_peered":"2024-04-17T06:00:14.726720+0000","last_clean":"2024-04-17T06:00:14.726720+0000","last_became_active":"2024-04-17T05:59:14.587394+0000","last_became_peered":"2024-04-17T05:59:14.587394+0000","last_unstale":"2024-04-17T06:00:14.726720+0000","last_undegraded":"2024-04-17T06:00:14.726720+0000","last_fullsized":"2024-04-17T06:00:14.726720+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:22:52.733877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:15.000910+0000","last_change":"2024-04-17T05:59:14.591458+0000","last_active":"2024-04-17T06:00:15.000910+0000","last_peered":"2024-04-17T06:00:15.000910+0000","last_clean":"2024-04-17T06:00:15.000910+0000","last_became_active":"2024-04-17T05:59:14.590898+0000","last_became_peered":"2024-04-17T05:59:14.590898+0000","last_unstale":"2024-04-17T06:00:15.000910+0000","last_undegraded":"2024-04-17T06:00:15.000910+0000","last_fullsized":"2024-04-17T06:00:15.000910+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:58:40.817928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.727959+0000","last_change":"2024-04-17T05:59:14.600199+0000","last_active":"2024-04-17T06:00:14.727959+0000","last_peered":"2024-04-17T06:00:14.727959+0000","last_clean":"2024-04-17T06:00:14.727959+0000","last_became_active":"2024-04-17T05:59:14.599286+0000","last_became_peered":"2024-04-17T05:59:14.599286+0000","last_unstale":"2024-04-17T06:00:14.727959+0000","last_undegraded":"2024-04-17T06:00:14.727959+0000","last_fullsized":"2024-04-17T06:00:14.727959+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:41:43.314103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.729872+0000","last_change":"2024-04-17T05:59:14.587999+0000","last_active":"2024-04-17T06:00:14.729872+0000","last_peered":"2024-04-17T06:00:14.729872+0000","last_clean":"2024-04-17T06:00:14.729872+0000","last_became_active":"2024-04-17T05:59:14.587655+0000","last_became_peered":"2024-04-17T05:59:14.587655+0000","last_unstale":"2024-04-17T06:00:14.729872+0000","last_undegraded":"2024-04-17T06:00:14.729872+0000","last_fullsized":"2024-04-17T06:00:14.729872+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:41:37.751776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.727096+0000","last_change":"2024-04-17T05:59:14.595003+0000","last_active":"2024-04-17T06:00:14.727096+0000","last_peered":"2024-04-17T06:00:14.727096+0000","last_clean":"2024-04-17T06:00:14.727096+0000","last_became_active":"2024-04-17T05:59:14.594424+0000","last_became_peered":"2024-04-17T05:59:14.594424+0000","last_unstale":"2024-04-17T06:00:14.727096+0000","last_undegraded":"2024-04-17T06:00:14.727096+0000","last_fullsized":"2024-04-17T06:00:14.727096+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:35:39.491074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:15.001695+0000","last_change":"2024-04-17T05:59:14.584957+0000","last_active":"2024-04-17T06:00:15.001695+0000","last_peered":"2024-04-17T06:00:15.001695+0000","last_clean":"2024-04-17T06:00:15.001695+0000","last_became_active":"2024-04-17T05:59:14.584625+0000","last_became_peered":"2024-04-17T05:59:14.584625+0000","last_unstale":"2024-04-17T06:00:15.001695+0000","last_undegraded":"2024-04-17T06:00:15.001695+0000","last_fullsized":"2024-04-17T06:00:15.001695+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:10:08.819831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.727596+0000","last_change":"2024-04-17T05:59:14.577633+0000","last_active":"2024-04-17T06:00:14.727596+0000","last_peered":"2024-04-17T06:00:14.727596+0000","last_clean":"2024-04-17T06:00:14.727596+0000","last_became_active":"2024-04-17T05:59:14.576608+0000","last_became_peered":"2024-04-17T05:59:14.576608+0000","last_unstale":"2024-04-17T06:00:14.727596+0000","last_undegraded":"2024-04-17T06:00:14.727596+0000","last_fullsized":"2024-04-17T06:00:14.727596+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:04:49.921958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.725330+0000","last_change":"2024-04-17T05:59:14.580912+0000","last_active":"2024-04-17T06:00:14.725330+0000","last_peered":"2024-04-17T06:00:14.725330+0000","last_clean":"2024-04-17T06:00:14.725330+0000","last_became_active":"2024-04-17T05:59:14.580500+0000","last_became_peered":"2024-04-17T05:59:14.580500+0000","last_unstale":"2024-04-17T06:00:14.725330+0000","last_undegraded":"2024-04-17T06:00:14.725330+0000","last_fullsized":"2024-04-17T06:00:14.725330+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:17:49.893499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.725925+0000","last_change":"2024-04-17T05:59:14.586109+0000","last_active":"2024-04-17T06:00:14.725925+0000","last_peered":"2024-04-17T06:00:14.725925+0000","last_clean":"2024-04-17T06:00:14.725925+0000","last_became_active":"2024-04-17T05:59:14.585667+0000","last_became_peered":"2024-04-17T05:59:14.585667+0000","last_unstale":"2024-04-17T06:00:14.725925+0000","last_undegraded":"2024-04-17T06:00:14.725925+0000","last_fullsized":"2024-04-17T06:00:14.725925+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:20:22.309678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.728517+0000","last_change":"2024-04-17T05:59:14.602921+0000","last_active":"2024-04-17T06:00:14.728517+0000","last_peered":"2024-04-17T06:00:14.728517+0000","last_clean":"2024-04-17T06:00:14.728517+0000","last_became_active":"2024-04-17T05:59:14.602589+0000","last_became_peered":"2024-04-17T05:59:14.602589+0000","last_unstale":"2024-04-17T06:00:14.728517+0000","last_undegraded":"2024-04-17T06:00:14.728517+0000","last_fullsized":"2024-04-17T06:00:14.728517+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:49:17.988067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.723932+0000","last_change":"2024-04-17T05:59:14.594201+0000","last_active":"2024-04-17T06:00:14.723932+0000","last_peered":"2024-04-17T06:00:14.723932+0000","last_clean":"2024-04-17T06:00:14.723932+0000","last_became_active":"2024-04-17T05:59:14.593850+0000","last_became_peered":"2024-04-17T05:59:14.593850+0000","last_unstale":"2024-04-17T06:00:14.723932+0000","last_undegraded":"2024-04-17T06:00:14.723932+0000","last_fullsized":"2024-04-17T06:00:14.723932+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:53:09.044324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.723715+0000","last_change":"2024-04-17T05:59:14.590439+0000","last_active":"2024-04-17T06:00:14.723715+0000","last_peered":"2024-04-17T06:00:14.723715+0000","last_clean":"2024-04-17T06:00:14.723715+0000","last_became_active":"2024-04-17T05:59:14.589844+0000","last_became_peered":"2024-04-17T05:59:14.589844+0000","last_unstale":"2024-04-17T06:00:14.723715+0000","last_undegraded":"2024-04-17T06:00:14.723715+0000","last_fullsized":"2024-04-17T06:00:14.723715+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:02:18.366048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.726966+0000","last_change":"2024-04-17T05:59:14.592297+0000","last_active":"2024-04-17T06:00:14.726966+0000","last_peered":"2024-04-17T06:00:14.726966+0000","last_clean":"2024-04-17T06:00:14.726966+0000","last_became_active":"2024-04-17T05:59:14.591976+0000","last_became_peered":"2024-04-17T05:59:14.591976+0000","last_unstale":"2024-04-17T06:00:14.726966+0000","last_undegraded":"2024-04-17T06:00:14.726966+0000","last_fullsized":"2024-04-17T06:00:14.726966+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:11:47.488231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.723707+0000","last_change":"2024-04-17T05:59:13.561616+0000","last_active":"2024-04-17T06:00:14.723707+0000","last_peered":"2024-04-17T06:00:14.723707+0000","last_clean":"2024-04-17T06:00:14.723707+0000","last_became_active":"2024-04-17T05:59:13.561298+0000","last_became_peered":"2024-04-17T05:59:13.561298+0000","last_unstale":"2024-04-17T06:00:14.723707+0000","last_undegraded":"2024-04-17T06:00:14.723707+0000","last_fullsized":"2024-04-17T06:00:14.723707+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:59:02.598774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:15.001431+0000","last_change":"2024-04-17T05:59:14.592023+0000","last_active":"2024-04-17T06:00:15.001431+0000","last_peered":"2024-04-17T06:00:15.001431+0000","last_clean":"2024-04-17T06:00:15.001431+0000","last_became_active":"2024-04-17T05:59:14.591707+0000","last_became_peered":"2024-04-17T05:59:14.591707+0000","last_unstale":"2024-04-17T06:00:15.001431+0000","last_undegraded":"2024-04-17T06:00:15.001431+0000","last_fullsized":"2024-04-17T06:00:15.001431+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:50:06.828184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:15.001551+0000","last_change":"2024-04-17T05:59:13.568073+0000","last_active":"2024-04-17T06:00:15.001551+0000","last_peered":"2024-04-17T06:00:15.001551+0000","last_clean":"2024-04-17T06:00:15.001551+0000","last_became_active":"2024-04-17T05:59:13.567751+0000","last_became_peered":"2024-04-17T05:59:13.567751+0000","last_unstale":"2024-04-17T06:00:15.001551+0000","last_undegraded":"2024-04-17T06:00:15.001551+0000","last_fullsized":"2024-04-17T06:00:15.001551+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:40:26.884308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.725947+0000","last_change":"2024-04-17T05:59:14.593344+0000","last_active":"2024-04-17T06:00:14.725947+0000","last_peered":"2024-04-17T06:00:14.725947+0000","last_clean":"2024-04-17T06:00:14.725947+0000","last_became_active":"2024-04-17T05:59:14.592946+0000","last_became_peered":"2024-04-17T05:59:14.592946+0000","last_unstale":"2024-04-17T06:00:14.725947+0000","last_undegraded":"2024-04-17T06:00:14.725947+0000","last_fullsized":"2024-04-17T06:00:14.725947+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:50:23.979412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.726609+0000","last_change":"2024-04-17T05:59:13.566639+0000","last_active":"2024-04-17T06:00:14.726609+0000","last_peered":"2024-04-17T06:00:14.726609+0000","last_clean":"2024-04-17T06:00:14.726609+0000","last_became_active":"2024-04-17T05:59:13.562766+0000","last_became_peered":"2024-04-17T05:59:13.562766+0000","last_unstale":"2024-04-17T06:00:14.726609+0000","last_undegraded":"2024-04-17T06:00:14.726609+0000","last_fullsized":"2024-04-17T06:00:14.726609+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:36:47.605106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.727742+0000","last_change":"2024-04-17T05:59:14.596055+0000","last_active":"2024-04-17T06:00:14.727742+0000","last_peered":"2024-04-17T06:00:14.727742+0000","last_clean":"2024-04-17T06:00:14.727742+0000","last_became_active":"2024-04-17T05:59:14.595672+0000","last_became_peered":"2024-04-17T05:59:14.595672+0000","last_unstale":"2024-04-17T06:00:14.727742+0000","last_undegraded":"2024-04-17T06:00:14.727742+0000","last_fullsized":"2024-04-17T06:00:14.727742+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:31:59.964502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.725808+0000","last_change":"2024-04-17T05:59:13.558715+0000","last_active":"2024-04-17T06:00:14.725808+0000","last_peered":"2024-04-17T06:00:14.725808+0000","last_clean":"2024-04-17T06:00:14.725808+0000","last_became_active":"2024-04-17T05:59:13.558377+0000","last_became_peered":"2024-04-17T05:59:13.558377+0000","last_unstale":"2024-04-17T06:00:14.725808+0000","last_undegraded":"2024-04-17T06:00:14.725808+0000","last_fullsized":"2024-04-17T06:00:14.725808+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:12:12.392655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.731769+0000","last_change":"2024-04-17T06:00:14.731769+0000","last_active":"2024-04-17T06:00:14.731769+0000","last_peered":"2024-04-17T06:00:14.731769+0000","last_clean":"2024-04-17T06:00:14.731769+0000","last_became_active":"2024-04-17T06:00:14.731483+0000","last_became_peered":"2024-04-17T06:00:14.731483+0000","last_unstale":"2024-04-17T06:00:14.731769+0000","last_undegraded":"2024-04-17T06:00:14.731769+0000","last_fullsized":"2024-04-17T06:00:14.731769+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:00:29.850454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'6","reported_seq":98,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:27.703739+0000","last_change":"2024-04-17T05:59:13.567654+0000","last_active":"2024-04-17T06:00:27.703739+0000","last_peered":"2024-04-17T06:00:27.703739+0000","last_clean":"2024-04-17T06:00:27.703739+0000","last_became_active":"2024-04-17T05:59:13.566474+0000","last_became_peered":"2024-04-17T05:59:13.566474+0000","last_unstale":"2024-04-17T06:00:27.703739+0000","last_undegraded":"2024-04-17T06:00:27.703739+0000","last_fullsized":"2024-04-17T06:00:27.703739+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:40:25.205832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":73,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.725594+0000","last_change":"2024-04-17T05:59:14.597145+0000","last_active":"2024-04-17T06:00:14.725594+0000","last_peered":"2024-04-17T06:00:14.725594+0000","last_clean":"2024-04-17T06:00:14.725594+0000","last_became_active":"2024-04-17T05:59:14.596802+0000","last_became_peered":"2024-04-17T05:59:14.596802+0000","last_unstale":"2024-04-17T06:00:14.725594+0000","last_undegraded":"2024-04-17T06:00:14.725594+0000","last_fullsized":"2024-04-17T06:00:14.725594+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:41:04.946052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.729325+0000","last_change":"2024-04-17T05:59:13.560100+0000","last_active":"2024-04-17T06:00:14.729325+0000","last_peered":"2024-04-17T06:00:14.729325+0000","last_clean":"2024-04-17T06:00:14.729325+0000","last_became_active":"2024-04-17T05:59:13.559769+0000","last_became_peered":"2024-04-17T05:59:13.559769+0000","last_unstale":"2024-04-17T06:00:14.729325+0000","last_undegraded":"2024-04-17T06:00:14.729325+0000","last_fullsized":"2024-04-17T06:00:14.729325+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:44:30.299028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.728753+0000","last_change":"2024-04-17T05:59:14.600733+0000","last_active":"2024-04-17T06:00:14.728753+0000","last_peered":"2024-04-17T06:00:14.728753+0000","last_clean":"2024-04-17T06:00:14.728753+0000","last_became_active":"2024-04-17T05:59:14.599273+0000","last_became_peered":"2024-04-17T05:59:14.599273+0000","last_unstale":"2024-04-17T06:00:14.728753+0000","last_undegraded":"2024-04-17T06:00:14.728753+0000","last_fullsized":"2024-04-17T06:00:14.728753+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:27:02.721452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.727705+0000","last_change":"2024-04-17T05:59:13.553000+0000","last_active":"2024-04-17T06:00:14.727705+0000","last_peered":"2024-04-17T06:00:14.727705+0000","last_clean":"2024-04-17T06:00:14.727705+0000","last_became_active":"2024-04-17T05:59:13.552469+0000","last_became_peered":"2024-04-17T05:59:13.552469+0000","last_unstale":"2024-04-17T06:00:14.727705+0000","last_undegraded":"2024-04-17T06:00:14.727705+0000","last_fullsized":"2024-04-17T06:00:14.727705+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:14:34.714179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:15.000886+0000","last_change":"2024-04-17T05:59:14.594792+0000","last_active":"2024-04-17T06:00:15.000886+0000","last_peered":"2024-04-17T06:00:15.000886+0000","last_clean":"2024-04-17T06:00:15.000886+0000","last_became_active":"2024-04-17T05:59:14.594436+0000","last_became_peered":"2024-04-17T05:59:14.594436+0000","last_unstale":"2024-04-17T06:00:15.000886+0000","last_undegraded":"2024-04-17T06:00:15.000886+0000","last_fullsized":"2024-04-17T06:00:15.000886+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:49:57.104050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.728365+0000","last_change":"2024-04-17T05:59:13.565352+0000","last_active":"2024-04-17T06:00:14.728365+0000","last_peered":"2024-04-17T06:00:14.728365+0000","last_clean":"2024-04-17T06:00:14.728365+0000","last_became_active":"2024-04-17T05:59:13.565046+0000","last_became_peered":"2024-04-17T05:59:13.565046+0000","last_unstale":"2024-04-17T06:00:14.728365+0000","last_undegraded":"2024-04-17T06:00:14.728365+0000","last_fullsized":"2024-04-17T06:00:14.728365+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:39:25.276782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.723409+0000","last_change":"2024-04-17T05:59:14.593925+0000","last_active":"2024-04-17T06:00:14.723409+0000","last_peered":"2024-04-17T06:00:14.723409+0000","last_clean":"2024-04-17T06:00:14.723409+0000","last_became_active":"2024-04-17T05:59:14.593535+0000","last_became_peered":"2024-04-17T05:59:14.593535+0000","last_unstale":"2024-04-17T06:00:14.723409+0000","last_undegraded":"2024-04-17T06:00:14.723409+0000","last_fullsized":"2024-04-17T06:00:14.723409+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:58:20.811999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'3","reported_seq":92,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:27.703195+0000","last_change":"2024-04-17T05:59:13.562281+0000","last_active":"2024-04-17T06:00:27.703195+0000","last_peered":"2024-04-17T06:00:27.703195+0000","last_clean":"2024-04-17T06:00:27.703195+0000","last_became_active":"2024-04-17T05:59:13.561931+0000","last_became_peered":"2024-04-17T05:59:13.561931+0000","last_unstale":"2024-04-17T06:00:27.703195+0000","last_undegraded":"2024-04-17T06:00:27.703195+0000","last_fullsized":"2024-04-17T06:00:27.703195+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:55:52.119923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":70,"num_read_kb":70,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.727303+0000","last_change":"2024-04-17T05:59:14.594708+0000","last_active":"2024-04-17T06:00:14.727303+0000","last_peered":"2024-04-17T06:00:14.727303+0000","last_clean":"2024-04-17T06:00:14.727303+0000","last_became_active":"2024-04-17T05:59:14.594206+0000","last_became_peered":"2024-04-17T05:59:14.594206+0000","last_unstale":"2024-04-17T06:00:14.727303+0000","last_undegraded":"2024-04-17T06:00:14.727303+0000","last_fullsized":"2024-04-17T06:00:14.727303+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:43:16.402659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.726993+0000","last_change":"2024-04-17T05:59:13.561056+0000","last_active":"2024-04-17T06:00:14.726993+0000","last_peered":"2024-04-17T06:00:14.726993+0000","last_clean":"2024-04-17T06:00:14.726993+0000","last_became_active":"2024-04-17T05:59:13.560670+0000","last_became_peered":"2024-04-17T05:59:13.560670+0000","last_unstale":"2024-04-17T06:00:14.726993+0000","last_undegraded":"2024-04-17T06:00:14.726993+0000","last_fullsized":"2024-04-17T06:00:14.726993+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:31:01.106013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.725926+0000","last_change":"2024-04-17T05:59:14.590243+0000","last_active":"2024-04-17T06:00:14.725926+0000","last_peered":"2024-04-17T06:00:14.725926+0000","last_clean":"2024-04-17T06:00:14.725926+0000","last_became_active":"2024-04-17T05:59:14.589911+0000","last_became_peered":"2024-04-17T05:59:14.589911+0000","last_unstale":"2024-04-17T06:00:14.725926+0000","last_undegraded":"2024-04-17T06:00:14.725926+0000","last_fullsized":"2024-04-17T06:00:14.725926+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:44:41.855928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.729590+0000","last_change":"2024-04-17T05:59:13.569983+0000","last_active":"2024-04-17T06:00:14.729590+0000","last_peered":"2024-04-17T06:00:14.729590+0000","last_clean":"2024-04-17T06:00:14.729590+0000","last_became_active":"2024-04-17T05:59:13.569667+0000","last_became_peered":"2024-04-17T05:59:13.569667+0000","last_unstale":"2024-04-17T06:00:14.729590+0000","last_undegraded":"2024-04-17T06:00:14.729590+0000","last_fullsized":"2024-04-17T06:00:14.729590+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:36:15.362530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.728126+0000","last_change":"2024-04-17T05:59:14.603404+0000","last_active":"2024-04-17T06:00:14.728126+0000","last_peered":"2024-04-17T06:00:14.728126+0000","last_clean":"2024-04-17T06:00:14.728126+0000","last_became_active":"2024-04-17T05:59:14.601593+0000","last_became_peered":"2024-04-17T05:59:14.601593+0000","last_unstale":"2024-04-17T06:00:14.728126+0000","last_undegraded":"2024-04-17T06:00:14.728126+0000","last_fullsized":"2024-04-17T06:00:14.728126+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:44:44.490496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'27","reported_seq":211,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:29.025476+0000","last_change":"2024-04-17T06:00:15.008166+0000","last_active":"2024-04-17T06:00:29.025476+0000","last_peered":"2024-04-17T06:00:29.025476+0000","last_clean":"2024-04-17T06:00:29.025476+0000","last_became_active":"2024-04-17T06:00:15.002373+0000","last_became_peered":"2024-04-17T06:00:15.002373+0000","last_unstale":"2024-04-17T06:00:29.025476+0000","last_undegraded":"2024-04-17T06:00:29.025476+0000","last_fullsized":"2024-04-17T06:00:29.025476+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:32:09.078832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59465,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":137,"num_read_kb":126,"num_write":28,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":59465,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.726358+0000","last_change":"2024-04-17T05:59:14.579304+0000","last_active":"2024-04-17T06:00:14.726358+0000","last_peered":"2024-04-17T06:00:14.726358+0000","last_clean":"2024-04-17T06:00:14.726358+0000","last_became_active":"2024-04-17T05:59:14.578948+0000","last_became_peered":"2024-04-17T05:59:14.578948+0000","last_unstale":"2024-04-17T06:00:14.726358+0000","last_undegraded":"2024-04-17T06:00:14.726358+0000","last_fullsized":"2024-04-17T06:00:14.726358+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:39:43.341725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.726288+0000","last_change":"2024-04-17T05:59:13.569534+0000","last_active":"2024-04-17T06:00:14.726288+0000","last_peered":"2024-04-17T06:00:14.726288+0000","last_clean":"2024-04-17T06:00:14.726288+0000","last_became_active":"2024-04-17T05:59:13.569041+0000","last_became_peered":"2024-04-17T05:59:13.569041+0000","last_unstale":"2024-04-17T06:00:14.726288+0000","last_undegraded":"2024-04-17T06:00:14.726288+0000","last_fullsized":"2024-04-17T06:00:14.726288+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:27:37.018696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.727562+0000","last_change":"2024-04-17T05:59:14.593820+0000","last_active":"2024-04-17T06:00:14.727562+0000","last_peered":"2024-04-17T06:00:14.727562+0000","last_clean":"2024-04-17T06:00:14.727562+0000","last_became_active":"2024-04-17T05:59:14.593473+0000","last_became_peered":"2024-04-17T05:59:14.593473+0000","last_unstale":"2024-04-17T06:00:14.727562+0000","last_undegraded":"2024-04-17T06:00:14.727562+0000","last_fullsized":"2024-04-17T06:00:14.727562+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:44:38.636980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.728888+0000","last_change":"2024-04-17T05:59:13.570124+0000","last_active":"2024-04-17T06:00:14.728888+0000","last_peered":"2024-04-17T06:00:14.728888+0000","last_clean":"2024-04-17T06:00:14.728888+0000","last_became_active":"2024-04-17T05:59:13.569778+0000","last_became_peered":"2024-04-17T05:59:13.569778+0000","last_unstale":"2024-04-17T06:00:14.728888+0000","last_undegraded":"2024-04-17T06:00:14.728888+0000","last_fullsized":"2024-04-17T06:00:14.728888+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:46:46.431207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.727119+0000","last_change":"2024-04-17T05:59:14.584863+0000","last_active":"2024-04-17T06:00:14.727119+0000","last_peered":"2024-04-17T06:00:14.727119+0000","last_clean":"2024-04-17T06:00:14.727119+0000","last_became_active":"2024-04-17T05:59:14.584554+0000","last_became_peered":"2024-04-17T05:59:14.584554+0000","last_unstale":"2024-04-17T06:00:14.727119+0000","last_undegraded":"2024-04-17T06:00:14.727119+0000","last_fullsized":"2024-04-17T06:00:14.727119+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:26:22.064337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'4","reported_seq":99,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:30.725456+0000","last_change":"2024-04-17T05:59:13.557625+0000","last_active":"2024-04-17T06:00:30.725456+0000","last_peered":"2024-04-17T06:00:30.725456+0000","last_clean":"2024-04-17T06:00:30.725456+0000","last_became_active":"2024-04-17T05:59:13.557307+0000","last_became_peered":"2024-04-17T05:59:13.557307+0000","last_unstale":"2024-04-17T06:00:30.725456+0000","last_undegraded":"2024-04-17T06:00:30.725456+0000","last_fullsized":"2024-04-17T06:00:30.725456+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:43:17.023049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":77,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.731050+0000","last_change":"2024-04-17T06:00:14.731050+0000","last_active":"2024-04-17T06:00:14.731050+0000","last_peered":"2024-04-17T06:00:14.731050+0000","last_clean":"2024-04-17T06:00:14.731050+0000","last_became_active":"2024-04-17T06:00:14.730653+0000","last_became_peered":"2024-04-17T06:00:14.730653+0000","last_unstale":"2024-04-17T06:00:14.731050+0000","last_undegraded":"2024-04-17T06:00:14.731050+0000","last_fullsized":"2024-04-17T06:00:14.731050+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:11:23.354930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.741811+0000","last_change":"2024-04-17T06:00:14.741383+0000","last_active":"2024-04-17T06:00:14.741811+0000","last_peered":"2024-04-17T06:00:14.741811+0000","last_clean":"2024-04-17T06:00:14.741811+0000","last_became_active":"2024-04-17T06:00:14.727133+0000","last_became_peered":"2024-04-17T06:00:14.727133+0000","last_unstale":"2024-04-17T06:00:14.741811+0000","last_undegraded":"2024-04-17T06:00:14.741811+0000","last_fullsized":"2024-04-17T06:00:14.741811+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:27:46.912019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.728623+0000","last_change":"2024-04-17T05:59:14.594082+0000","last_active":"2024-04-17T06:00:14.728623+0000","last_peered":"2024-04-17T06:00:14.728623+0000","last_clean":"2024-04-17T06:00:14.728623+0000","last_became_active":"2024-04-17T05:59:14.593754+0000","last_became_peered":"2024-04-17T05:59:14.593754+0000","last_unstale":"2024-04-17T06:00:14.728623+0000","last_undegraded":"2024-04-17T06:00:14.728623+0000","last_fullsized":"2024-04-17T06:00:14.728623+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:36:28.752766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.726112+0000","last_change":"2024-04-17T05:59:13.563470+0000","last_active":"2024-04-17T06:00:14.726112+0000","last_peered":"2024-04-17T06:00:14.726112+0000","last_clean":"2024-04-17T06:00:14.726112+0000","last_became_active":"2024-04-17T05:59:13.563067+0000","last_became_peered":"2024-04-17T05:59:13.563067+0000","last_unstale":"2024-04-17T06:00:14.726112+0000","last_undegraded":"2024-04-17T06:00:14.726112+0000","last_fullsized":"2024-04-17T06:00:14.726112+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:13:52.018604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.723699+0000","last_change":"2024-04-17T05:59:14.587339+0000","last_active":"2024-04-17T06:00:14.723699+0000","last_peered":"2024-04-17T06:00:14.723699+0000","last_clean":"2024-04-17T06:00:14.723699+0000","last_became_active":"2024-04-17T05:59:14.586956+0000","last_became_peered":"2024-04-17T05:59:14.586956+0000","last_unstale":"2024-04-17T06:00:14.723699+0000","last_undegraded":"2024-04-17T06:00:14.723699+0000","last_fullsized":"2024-04-17T06:00:14.723699+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:00:21.494832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.723479+0000","last_change":"2024-04-17T05:59:13.563915+0000","last_active":"2024-04-17T06:00:14.723479+0000","last_peered":"2024-04-17T06:00:14.723479+0000","last_clean":"2024-04-17T06:00:14.723479+0000","last_became_active":"2024-04-17T05:59:13.563218+0000","last_became_peered":"2024-04-17T05:59:13.563218+0000","last_unstale":"2024-04-17T06:00:14.723479+0000","last_undegraded":"2024-04-17T06:00:14.723479+0000","last_fullsized":"2024-04-17T06:00:14.723479+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:36:54.484278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"86'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.729467+0000","last_change":"2024-04-17T05:59:14.593168+0000","last_active":"2024-04-17T06:00:14.729467+0000","last_peered":"2024-04-17T06:00:14.729467+0000","last_clean":"2024-04-17T06:00:14.729467+0000","last_became_active":"2024-04-17T05:59:14.592744+0000","last_became_peered":"2024-04-17T05:59:14.592744+0000","last_unstale":"2024-04-17T06:00:14.729467+0000","last_undegraded":"2024-04-17T06:00:14.729467+0000","last_fullsized":"2024-04-17T06:00:14.729467+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:00:03.217864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.726881+0000","last_change":"2024-04-17T05:59:13.564703+0000","last_active":"2024-04-17T06:00:14.726881+0000","last_peered":"2024-04-17T06:00:14.726881+0000","last_clean":"2024-04-17T06:00:14.726881+0000","last_became_active":"2024-04-17T05:59:13.564263+0000","last_became_peered":"2024-04-17T05:59:13.564263+0000","last_unstale":"2024-04-17T06:00:14.726881+0000","last_undegraded":"2024-04-17T06:00:14.726881+0000","last_fullsized":"2024-04-17T06:00:14.726881+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:51:06.847989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.729842+0000","last_change":"2024-04-17T05:59:14.588074+0000","last_active":"2024-04-17T06:00:14.729842+0000","last_peered":"2024-04-17T06:00:14.729842+0000","last_clean":"2024-04-17T06:00:14.729842+0000","last_became_active":"2024-04-17T05:59:14.587691+0000","last_became_peered":"2024-04-17T05:59:14.587691+0000","last_unstale":"2024-04-17T06:00:14.729842+0000","last_undegraded":"2024-04-17T06:00:14.729842+0000","last_fullsized":"2024-04-17T06:00:14.729842+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:11:15.775356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.728510+0000","last_change":"2024-04-17T05:59:14.589178+0000","last_active":"2024-04-17T06:00:14.728510+0000","last_peered":"2024-04-17T06:00:14.728510+0000","last_clean":"2024-04-17T06:00:14.728510+0000","last_became_active":"2024-04-17T05:59:14.587351+0000","last_became_peered":"2024-04-17T05:59:14.587351+0000","last_unstale":"2024-04-17T06:00:14.728510+0000","last_undegraded":"2024-04-17T06:00:14.728510+0000","last_fullsized":"2024-04-17T06:00:14.728510+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:22:43.202731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'4","reported_seq":96,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:30.724745+0000","last_change":"2024-04-17T05:59:13.553754+0000","last_active":"2024-04-17T06:00:30.724745+0000","last_peered":"2024-04-17T06:00:30.724745+0000","last_clean":"2024-04-17T06:00:30.724745+0000","last_became_active":"2024-04-17T05:59:13.553437+0000","last_became_peered":"2024-04-17T05:59:13.553437+0000","last_unstale":"2024-04-17T06:00:30.724745+0000","last_undegraded":"2024-04-17T06:00:30.724745+0000","last_fullsized":"2024-04-17T06:00:30.724745+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:10:14.010264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":73,"num_read_kb":73,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:15.001280+0000","last_change":"2024-04-17T05:59:14.592074+0000","last_active":"2024-04-17T06:00:15.001280+0000","last_peered":"2024-04-17T06:00:15.001280+0000","last_clean":"2024-04-17T06:00:15.001280+0000","last_became_active":"2024-04-17T05:59:14.591695+0000","last_became_peered":"2024-04-17T05:59:14.591695+0000","last_unstale":"2024-04-17T06:00:15.001280+0000","last_undegraded":"2024-04-17T06:00:15.001280+0000","last_fullsized":"2024-04-17T06:00:15.001280+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:27:04.344257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:15.001167+0000","last_change":"2024-04-17T05:59:13.553916+0000","last_active":"2024-04-17T06:00:15.001167+0000","last_peered":"2024-04-17T06:00:15.001167+0000","last_clean":"2024-04-17T06:00:15.001167+0000","last_became_active":"2024-04-17T05:59:13.553597+0000","last_became_peered":"2024-04-17T05:59:13.553597+0000","last_unstale":"2024-04-17T06:00:15.001167+0000","last_undegraded":"2024-04-17T06:00:15.001167+0000","last_fullsized":"2024-04-17T06:00:15.001167+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:28:23.486188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":352,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.726650+0000","last_change":"2024-04-17T05:58:02.597504+0000","last_active":"2024-04-17T06:00:14.726650+0000","last_peered":"2024-04-17T06:00:14.726650+0000","last_clean":"2024-04-17T06:00:14.726650+0000","last_became_active":"2024-04-17T05:58:02.597086+0000","last_became_peered":"2024-04-17T05:58:02.597086+0000","last_unstale":"2024-04-17T06:00:14.726650+0000","last_undegraded":"2024-04-17T06:00:14.726650+0000","last_fullsized":"2024-04-17T06:00:14.726650+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:54:37.919204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:54:37.919204+0000","last_clean_scrub_stamp":"2024-04-17T05:54:37.919204+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:02:09.566418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.730339+0000","last_change":"2024-04-17T06:00:14.730339+0000","last_active":"2024-04-17T06:00:14.730339+0000","last_peered":"2024-04-17T06:00:14.730339+0000","last_clean":"2024-04-17T06:00:14.730339+0000","last_became_active":"2024-04-17T06:00:14.729872+0000","last_became_peered":"2024-04-17T06:00:14.729872+0000","last_unstale":"2024-04-17T06:00:14.730339+0000","last_undegraded":"2024-04-17T06:00:14.730339+0000","last_fullsized":"2024-04-17T06:00:14.730339+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:00:21.892184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.726839+0000","last_change":"2024-04-17T05:59:13.562426+0000","last_active":"2024-04-17T06:00:14.726839+0000","last_peered":"2024-04-17T06:00:14.726839+0000","last_clean":"2024-04-17T06:00:14.726839+0000","last_became_active":"2024-04-17T05:59:13.562133+0000","last_became_peered":"2024-04-17T05:59:13.562133+0000","last_unstale":"2024-04-17T06:00:14.726839+0000","last_undegraded":"2024-04-17T06:00:14.726839+0000","last_fullsized":"2024-04-17T06:00:14.726839+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:50:58.351924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.726439+0000","last_change":"2024-04-17T05:59:13.557783+0000","last_active":"2024-04-17T06:00:14.726439+0000","last_peered":"2024-04-17T06:00:14.726439+0000","last_clean":"2024-04-17T06:00:14.726439+0000","last_became_active":"2024-04-17T05:59:13.557485+0000","last_became_peered":"2024-04-17T05:59:13.557485+0000","last_unstale":"2024-04-17T06:00:14.726439+0000","last_undegraded":"2024-04-17T06:00:14.726439+0000","last_fullsized":"2024-04-17T06:00:14.726439+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:41:37.359553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.727610+0000","last_change":"2024-04-17T05:59:14.575993+0000","last_active":"2024-04-17T06:00:14.727610+0000","last_peered":"2024-04-17T06:00:14.727610+0000","last_clean":"2024-04-17T06:00:14.727610+0000","last_became_active":"2024-04-17T05:59:14.575654+0000","last_became_peered":"2024-04-17T05:59:14.575654+0000","last_unstale":"2024-04-17T06:00:14.727610+0000","last_undegraded":"2024-04-17T06:00:14.727610+0000","last_fullsized":"2024-04-17T06:00:14.727610+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:33:12.193747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'8","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.723773+0000","last_change":"2024-04-17T05:59:13.560256+0000","last_active":"2024-04-17T06:00:14.723773+0000","last_peered":"2024-04-17T06:00:14.723773+0000","last_clean":"2024-04-17T06:00:14.723773+0000","last_became_active":"2024-04-17T05:59:13.559938+0000","last_became_peered":"2024-04-17T05:59:13.559938+0000","last_unstale":"2024-04-17T06:00:14.723773+0000","last_undegraded":"2024-04-17T06:00:14.723773+0000","last_fullsized":"2024-04-17T06:00:14.723773+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:14:54.625298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:15.001565+0000","last_change":"2024-04-17T05:59:14.593955+0000","last_active":"2024-04-17T06:00:15.001565+0000","last_peered":"2024-04-17T06:00:15.001565+0000","last_clean":"2024-04-17T06:00:15.001565+0000","last_became_active":"2024-04-17T05:59:14.590626+0000","last_became_peered":"2024-04-17T05:59:14.590626+0000","last_unstale":"2024-04-17T06:00:15.001565+0000","last_undegraded":"2024-04-17T06:00:15.001565+0000","last_fullsized":"2024-04-17T06:00:15.001565+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:19:04.438634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:15.001228+0000","last_change":"2024-04-17T05:59:13.555734+0000","last_active":"2024-04-17T06:00:15.001228+0000","last_peered":"2024-04-17T06:00:15.001228+0000","last_clean":"2024-04-17T06:00:15.001228+0000","last_became_active":"2024-04-17T05:59:13.555284+0000","last_became_peered":"2024-04-17T05:59:13.555284+0000","last_unstale":"2024-04-17T06:00:15.001228+0000","last_undegraded":"2024-04-17T06:00:15.001228+0000","last_fullsized":"2024-04-17T06:00:15.001228+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:47:06.834332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.725806+0000","last_change":"2024-04-17T05:59:14.599078+0000","last_active":"2024-04-17T06:00:14.725806+0000","last_peered":"2024-04-17T06:00:14.725806+0000","last_clean":"2024-04-17T06:00:14.725806+0000","last_became_active":"2024-04-17T05:59:14.598670+0000","last_became_peered":"2024-04-17T05:59:14.598670+0000","last_unstale":"2024-04-17T06:00:14.725806+0000","last_undegraded":"2024-04-17T06:00:14.725806+0000","last_fullsized":"2024-04-17T06:00:14.725806+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:58:17.414704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.743749+0000","last_change":"2024-04-17T06:00:14.743749+0000","last_active":"2024-04-17T06:00:14.743749+0000","last_peered":"2024-04-17T06:00:14.743749+0000","last_clean":"2024-04-17T06:00:14.743749+0000","last_became_active":"2024-04-17T06:00:14.731352+0000","last_became_peered":"2024-04-17T06:00:14.731352+0000","last_unstale":"2024-04-17T06:00:14.743749+0000","last_undegraded":"2024-04-17T06:00:14.743749+0000","last_fullsized":"2024-04-17T06:00:14.743749+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:30:56.157829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.725286+0000","last_change":"2024-04-17T05:59:14.593450+0000","last_active":"2024-04-17T06:00:14.725286+0000","last_peered":"2024-04-17T06:00:14.725286+0000","last_clean":"2024-04-17T06:00:14.725286+0000","last_became_active":"2024-04-17T05:59:14.593104+0000","last_became_peered":"2024-04-17T05:59:14.593104+0000","last_unstale":"2024-04-17T06:00:14.725286+0000","last_undegraded":"2024-04-17T06:00:14.725286+0000","last_fullsized":"2024-04-17T06:00:14.725286+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:16:20.488110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:15.001104+0000","last_change":"2024-04-17T05:59:13.557330+0000","last_active":"2024-04-17T06:00:15.001104+0000","last_peered":"2024-04-17T06:00:15.001104+0000","last_clean":"2024-04-17T06:00:15.001104+0000","last_became_active":"2024-04-17T05:59:13.556970+0000","last_became_peered":"2024-04-17T05:59:13.556970+0000","last_unstale":"2024-04-17T06:00:15.001104+0000","last_undegraded":"2024-04-17T06:00:15.001104+0000","last_fullsized":"2024-04-17T06:00:15.001104+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:41:46.643845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.723595+0000","last_change":"2024-04-17T05:59:14.582281+0000","last_active":"2024-04-17T06:00:14.723595+0000","last_peered":"2024-04-17T06:00:14.723595+0000","last_clean":"2024-04-17T06:00:14.723595+0000","last_became_active":"2024-04-17T05:59:14.581922+0000","last_became_peered":"2024-04-17T05:59:14.581922+0000","last_unstale":"2024-04-17T06:00:14.723595+0000","last_undegraded":"2024-04-17T06:00:14.723595+0000","last_fullsized":"2024-04-17T06:00:14.723595+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:53:10.415786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.726097+0000","last_change":"2024-04-17T05:59:13.566514+0000","last_active":"2024-04-17T06:00:14.726097+0000","last_peered":"2024-04-17T06:00:14.726097+0000","last_clean":"2024-04-17T06:00:14.726097+0000","last_became_active":"2024-04-17T05:59:13.566004+0000","last_became_peered":"2024-04-17T05:59:13.566004+0000","last_unstale":"2024-04-17T06:00:14.726097+0000","last_undegraded":"2024-04-17T06:00:14.726097+0000","last_fullsized":"2024-04-17T06:00:14.726097+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:20:13.891422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.732919+0000","last_change":"2024-04-17T06:00:14.731515+0000","last_active":"2024-04-17T06:00:14.732919+0000","last_peered":"2024-04-17T06:00:14.732919+0000","last_clean":"2024-04-17T06:00:14.732919+0000","last_became_active":"2024-04-17T06:00:14.731153+0000","last_became_peered":"2024-04-17T06:00:14.731153+0000","last_unstale":"2024-04-17T06:00:14.732919+0000","last_undegraded":"2024-04-17T06:00:14.732919+0000","last_fullsized":"2024-04-17T06:00:14.732919+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:37:42.248717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,10],"acting":[9,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.725420+0000","last_change":"2024-04-17T05:59:13.562531+0000","last_active":"2024-04-17T06:00:14.725420+0000","last_peered":"2024-04-17T06:00:14.725420+0000","last_clean":"2024-04-17T06:00:14.725420+0000","last_became_active":"2024-04-17T05:59:13.562148+0000","last_became_peered":"2024-04-17T05:59:13.562148+0000","last_unstale":"2024-04-17T06:00:14.725420+0000","last_undegraded":"2024-04-17T06:00:14.725420+0000","last_fullsized":"2024-04-17T06:00:14.725420+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:10:14.407871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.727091+0000","last_change":"2024-04-17T05:59:14.596371+0000","last_active":"2024-04-17T06:00:14.727091+0000","last_peered":"2024-04-17T06:00:14.727091+0000","last_clean":"2024-04-17T06:00:14.727091+0000","last_became_active":"2024-04-17T05:59:14.596024+0000","last_became_peered":"2024-04-17T05:59:14.596024+0000","last_unstale":"2024-04-17T06:00:14.727091+0000","last_undegraded":"2024-04-17T06:00:14.727091+0000","last_fullsized":"2024-04-17T06:00:14.727091+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:06:14.916212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.726600+0000","last_change":"2024-04-17T05:59:13.569364+0000","last_active":"2024-04-17T06:00:14.726600+0000","last_peered":"2024-04-17T06:00:14.726600+0000","last_clean":"2024-04-17T06:00:14.726600+0000","last_became_active":"2024-04-17T05:59:13.569029+0000","last_became_peered":"2024-04-17T05:59:13.569029+0000","last_unstale":"2024-04-17T06:00:14.726600+0000","last_undegraded":"2024-04-17T06:00:14.726600+0000","last_fullsized":"2024-04-17T06:00:14.726600+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:58:24.629274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.727216+0000","last_change":"2024-04-17T05:59:14.593088+0000","last_active":"2024-04-17T06:00:14.727216+0000","last_peered":"2024-04-17T06:00:14.727216+0000","last_clean":"2024-04-17T06:00:14.727216+0000","last_became_active":"2024-04-17T05:59:14.592755+0000","last_became_peered":"2024-04-17T05:59:14.592755+0000","last_unstale":"2024-04-17T06:00:14.727216+0000","last_undegraded":"2024-04-17T06:00:14.727216+0000","last_fullsized":"2024-04-17T06:00:14.727216+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:27:14.503323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.728685+0000","last_change":"2024-04-17T05:59:13.569773+0000","last_active":"2024-04-17T06:00:14.728685+0000","last_peered":"2024-04-17T06:00:14.728685+0000","last_clean":"2024-04-17T06:00:14.728685+0000","last_became_active":"2024-04-17T05:59:13.569461+0000","last_became_peered":"2024-04-17T05:59:13.569461+0000","last_unstale":"2024-04-17T06:00:14.728685+0000","last_undegraded":"2024-04-17T06:00:14.728685+0000","last_fullsized":"2024-04-17T06:00:14.728685+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:21:13.441453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"86'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.726672+0000","last_change":"2024-04-17T05:59:14.588939+0000","last_active":"2024-04-17T06:00:14.726672+0000","last_peered":"2024-04-17T06:00:14.726672+0000","last_clean":"2024-04-17T06:00:14.726672+0000","last_became_active":"2024-04-17T05:59:14.588600+0000","last_became_peered":"2024-04-17T05:59:14.588600+0000","last_unstale":"2024-04-17T06:00:14.726672+0000","last_undegraded":"2024-04-17T06:00:14.726672+0000","last_fullsized":"2024-04-17T06:00:14.726672+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:07:47.865808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.727908+0000","last_change":"2024-04-17T05:59:13.560566+0000","last_active":"2024-04-17T06:00:14.727908+0000","last_peered":"2024-04-17T06:00:14.727908+0000","last_clean":"2024-04-17T06:00:14.727908+0000","last_became_active":"2024-04-17T05:59:13.560228+0000","last_became_peered":"2024-04-17T05:59:13.560228+0000","last_unstale":"2024-04-17T06:00:14.727908+0000","last_undegraded":"2024-04-17T06:00:14.727908+0000","last_fullsized":"2024-04-17T06:00:14.727908+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:13:59.436598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.727159+0000","last_change":"2024-04-17T05:59:14.583621+0000","last_active":"2024-04-17T06:00:14.727159+0000","last_peered":"2024-04-17T06:00:14.727159+0000","last_clean":"2024-04-17T06:00:14.727159+0000","last_became_active":"2024-04-17T05:59:14.583199+0000","last_became_peered":"2024-04-17T05:59:14.583199+0000","last_unstale":"2024-04-17T06:00:14.727159+0000","last_undegraded":"2024-04-17T06:00:14.727159+0000","last_fullsized":"2024-04-17T06:00:14.727159+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:53:12.516661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.723804+0000","last_change":"2024-04-17T05:59:14.587989+0000","last_active":"2024-04-17T06:00:14.723804+0000","last_peered":"2024-04-17T06:00:14.723804+0000","last_clean":"2024-04-17T06:00:14.723804+0000","last_became_active":"2024-04-17T05:59:14.586686+0000","last_became_peered":"2024-04-17T05:59:14.586686+0000","last_unstale":"2024-04-17T06:00:14.723804+0000","last_undegraded":"2024-04-17T06:00:14.723804+0000","last_fullsized":"2024-04-17T06:00:14.723804+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:21:23.227328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.726127+0000","last_change":"2024-04-17T05:59:14.588567+0000","last_active":"2024-04-17T06:00:14.726127+0000","last_peered":"2024-04-17T06:00:14.726127+0000","last_clean":"2024-04-17T06:00:14.726127+0000","last_became_active":"2024-04-17T05:59:14.588207+0000","last_became_peered":"2024-04-17T05:59:14.588207+0000","last_unstale":"2024-04-17T06:00:14.726127+0000","last_undegraded":"2024-04-17T06:00:14.726127+0000","last_fullsized":"2024-04-17T06:00:14.726127+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:44:26.094334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.725983+0000","last_change":"2024-04-17T05:59:14.593885+0000","last_active":"2024-04-17T06:00:14.725983+0000","last_peered":"2024-04-17T06:00:14.725983+0000","last_clean":"2024-04-17T06:00:14.725983+0000","last_became_active":"2024-04-17T05:59:14.593483+0000","last_became_peered":"2024-04-17T05:59:14.593483+0000","last_unstale":"2024-04-17T06:00:14.725983+0000","last_undegraded":"2024-04-17T06:00:14.725983+0000","last_fullsized":"2024-04-17T06:00:14.725983+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:22:41.837363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.729047+0000","last_change":"2024-04-17T06:00:14.729047+0000","last_active":"2024-04-17T06:00:14.729047+0000","last_peered":"2024-04-17T06:00:14.729047+0000","last_clean":"2024-04-17T06:00:14.729047+0000","last_became_active":"2024-04-17T06:00:14.728675+0000","last_became_peered":"2024-04-17T06:00:14.728675+0000","last_unstale":"2024-04-17T06:00:14.729047+0000","last_undegraded":"2024-04-17T06:00:14.729047+0000","last_fullsized":"2024-04-17T06:00:14.729047+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:05:42.244001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.727516+0000","last_change":"2024-04-17T05:59:14.583266+0000","last_active":"2024-04-17T06:00:14.727516+0000","last_peered":"2024-04-17T06:00:14.727516+0000","last_clean":"2024-04-17T06:00:14.727516+0000","last_became_active":"2024-04-17T05:59:14.582949+0000","last_became_peered":"2024-04-17T05:59:14.582949+0000","last_unstale":"2024-04-17T06:00:14.727516+0000","last_undegraded":"2024-04-17T06:00:14.727516+0000","last_fullsized":"2024-04-17T06:00:14.727516+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:49:14.740168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.723697+0000","last_change":"2024-04-17T05:59:14.587046+0000","last_active":"2024-04-17T06:00:14.723697+0000","last_peered":"2024-04-17T06:00:14.723697+0000","last_clean":"2024-04-17T06:00:14.723697+0000","last_became_active":"2024-04-17T05:59:14.586653+0000","last_became_peered":"2024-04-17T05:59:14.586653+0000","last_unstale":"2024-04-17T06:00:14.723697+0000","last_undegraded":"2024-04-17T06:00:14.723697+0000","last_fullsized":"2024-04-17T06:00:14.723697+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:11:57.171410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.726118+0000","last_change":"2024-04-17T05:59:14.589670+0000","last_active":"2024-04-17T06:00:14.726118+0000","last_peered":"2024-04-17T06:00:14.726118+0000","last_clean":"2024-04-17T06:00:14.726118+0000","last_became_active":"2024-04-17T05:59:14.589003+0000","last_became_peered":"2024-04-17T05:59:14.589003+0000","last_unstale":"2024-04-17T06:00:14.726118+0000","last_undegraded":"2024-04-17T06:00:14.726118+0000","last_fullsized":"2024-04-17T06:00:14.726118+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:21:19.574178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:15.001527+0000","last_change":"2024-04-17T05:59:14.594107+0000","last_active":"2024-04-17T06:00:15.001527+0000","last_peered":"2024-04-17T06:00:15.001527+0000","last_clean":"2024-04-17T06:00:15.001527+0000","last_became_active":"2024-04-17T05:59:14.593799+0000","last_became_peered":"2024-04-17T05:59:14.593799+0000","last_unstale":"2024-04-17T06:00:15.001527+0000","last_undegraded":"2024-04-17T06:00:15.001527+0000","last_fullsized":"2024-04-17T06:00:15.001527+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:23:45.222667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.728363+0000","last_change":"2024-04-17T05:59:14.587671+0000","last_active":"2024-04-17T06:00:14.728363+0000","last_peered":"2024-04-17T06:00:14.728363+0000","last_clean":"2024-04-17T06:00:14.728363+0000","last_became_active":"2024-04-17T05:59:14.586473+0000","last_became_peered":"2024-04-17T05:59:14.586473+0000","last_unstale":"2024-04-17T06:00:14.728363+0000","last_undegraded":"2024-04-17T06:00:14.728363+0000","last_fullsized":"2024-04-17T06:00:14.728363+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:31:42.156354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:15.001635+0000","last_change":"2024-04-17T05:59:14.586961+0000","last_active":"2024-04-17T06:00:15.001635+0000","last_peered":"2024-04-17T06:00:15.001635+0000","last_clean":"2024-04-17T06:00:15.001635+0000","last_became_active":"2024-04-17T05:59:14.586601+0000","last_became_peered":"2024-04-17T05:59:14.586601+0000","last_unstale":"2024-04-17T06:00:15.001635+0000","last_undegraded":"2024-04-17T06:00:15.001635+0000","last_fullsized":"2024-04-17T06:00:15.001635+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:21:58.851523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.727691+0000","last_change":"2024-04-17T05:59:14.601239+0000","last_active":"2024-04-17T06:00:14.727691+0000","last_peered":"2024-04-17T06:00:14.727691+0000","last_clean":"2024-04-17T06:00:14.727691+0000","last_became_active":"2024-04-17T05:59:14.600852+0000","last_became_peered":"2024-04-17T05:59:14.600852+0000","last_unstale":"2024-04-17T06:00:14.727691+0000","last_undegraded":"2024-04-17T06:00:14.727691+0000","last_fullsized":"2024-04-17T06:00:14.727691+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:14:02.112456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.727704+0000","last_change":"2024-04-17T05:59:14.579952+0000","last_active":"2024-04-17T06:00:14.727704+0000","last_peered":"2024-04-17T06:00:14.727704+0000","last_clean":"2024-04-17T06:00:14.727704+0000","last_became_active":"2024-04-17T05:59:14.579625+0000","last_became_peered":"2024-04-17T05:59:14.579625+0000","last_unstale":"2024-04-17T06:00:14.727704+0000","last_undegraded":"2024-04-17T06:00:14.727704+0000","last_fullsized":"2024-04-17T06:00:14.727704+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:22:32.709762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.723643+0000","last_change":"2024-04-17T05:59:14.585187+0000","last_active":"2024-04-17T06:00:14.723643+0000","last_peered":"2024-04-17T06:00:14.723643+0000","last_clean":"2024-04-17T06:00:14.723643+0000","last_became_active":"2024-04-17T05:59:14.584820+0000","last_became_peered":"2024-04-17T05:59:14.584820+0000","last_unstale":"2024-04-17T06:00:14.723643+0000","last_undegraded":"2024-04-17T06:00:14.723643+0000","last_fullsized":"2024-04-17T06:00:14.723643+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:53:58.945885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":249153,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":63857,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":442,"num_read_kb":288,"num_write":156,"num_write_kb":107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":59577,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":364544,"data_stored":191571,"data_compressed":13327,"data_compressed_allocated":20480,"data_compressed_original":73728,"omap_allocated":0,"internal_metadata":0},"log_size":135,"ondisk_log_size":135,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351005,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":958464,"data_stored":459384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612642,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32256,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712896,"statfs":{"total":95995035648,"available":95962005504,"internally_reserved":0,"allocated":1097728,"data_stored":1110609,"data_compressed":37928,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776168,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36284,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708868,"statfs":{"total":95995035648,"available":95957880832,"internally_reserved":0,"allocated":966656,"data_stored":459508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":36174273},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037805,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":958464,"data_stored":459316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299440,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1015808,"data_stored":1051749,"data_compressed":36061,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462967,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":954368,"data_stored":458897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574928,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1011712,"data_stored":518155,"data_compressed":1867,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738452,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36296,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708856,"statfs":{"total":95995035648,"available":95957868544,"internally_reserved":0,"allocated":974848,"data_stored":460168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411403,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":970752,"data_stored":460431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149766,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":970752,"data_stored":459208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888129,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":983040,"data_stored":519190,"data_compressed":9593,"data_compressed_allocated":12288,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724603,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1028096,"data_stored":1051042,"data_compressed":36061,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34424,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34424,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36061,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36061,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36061,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36061,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36061,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":59465,"data_compressed":1867,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":60500,"data_compressed":9593,"data_compressed_allocated":12288,"data_compressed_original":57344,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":61434,"data_compressed":1867,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T06:00:34.413 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 565db2d8-fc7e-11ee-bc91-c7b262605968 -- ceph pg dump --format=json 2024-04-17T06:00:34.664 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T06:00:35.049 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:00:34 smithi047 ceph-mon[32223]: pgmap v328: 97 pgs: 97 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-17T06:00:35.050 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:00:34 smithi047 ceph-mon[32223]: from='client.14994 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-17T06:00:35.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:00:34 smithi098 ceph-mon[36516]: pgmap v328: 97 pgs: 97 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-17T06:00:35.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:00:34 smithi098 ceph-mon[36516]: from='client.14994 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-17T06:00:35.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:00:34 smithi103 ceph-mon[36243]: pgmap v328: 97 pgs: 97 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-17T06:00:35.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:00:34 smithi103 ceph-mon[36243]: from='client.14994 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-17T06:00:36.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:36.372+0000 7f0ce628d700 1 -- 172.21.15.47:0/2456043077 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0ce0100fa0 msgr2=0x7f0ce0101420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:36.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:36.372+0000 7f0ce628d700 1 --2- 172.21.15.47:0/2456043077 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0ce0100fa0 0x7f0ce0101420 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f0cd00099c0 tx=0x7f0cd0009cd0 comp rx=0 tx=0).stop 2024-04-17T06:00:36.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:36.372+0000 7f0ce628d700 1 -- 172.21.15.47:0/2456043077 shutdown_connections 2024-04-17T06:00:36.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:36.372+0000 7f0ce628d700 1 --2- 172.21.15.47:0/2456043077 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ce0101960 0x7f0ce010f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:36.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:36.372+0000 7f0ce628d700 1 --2- 172.21.15.47:0/2456043077 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ce0107a70 0x7f0ce0107e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:36.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:36.372+0000 7f0ce628d700 1 --2- 172.21.15.47:0/2456043077 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0ce0100fa0 0x7f0ce0101420 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:36.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:36.372+0000 7f0ce628d700 1 -- 172.21.15.47:0/2456043077 >> 172.21.15.47:0/2456043077 conn(0x7f0ce0077f50 msgr2=0x7f0ce00ff2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:00:36.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:36.372+0000 7f0ce628d700 1 -- 172.21.15.47:0/2456043077 shutdown_connections 2024-04-17T06:00:36.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:36.372+0000 7f0ce628d700 1 -- 172.21.15.47:0/2456043077 wait complete. 2024-04-17T06:00:36.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:36.373+0000 7f0ce628d700 1 Processor -- start 2024-04-17T06:00:36.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:36.373+0000 7f0ce628d700 1 -- start start 2024-04-17T06:00:36.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:36.374+0000 7f0ce628d700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ce0100fa0 0x7f0ce01992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:36.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:36.374+0000 7f0ce628d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ce0101960 0x7f0ce01997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:36.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:36.374+0000 7f0ce628d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0ce0107a70 0x7f0ce019db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:36.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:36.374+0000 7f0ce628d700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f0ce0112ed0 con 0x7f0ce0107a70 2024-04-17T06:00:36.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:36.374+0000 7f0ce628d700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0ce0112d50 con 0x7f0ce0101960 2024-04-17T06:00:36.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:36.374+0000 7f0ce628d700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0ce0113050 con 0x7f0ce0100fa0 2024-04-17T06:00:36.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:36.374+0000 7f0ce482a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0ce0107a70 0x7f0ce019db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:36.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:36.374+0000 7f0ce482a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0ce0107a70 0x7f0ce019db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57262/0 (socket says 172.21.15.47:57262) 2024-04-17T06:00:36.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:36.374+0000 7f0ce482a700 1 -- 172.21.15.47:0/4220691390 learned_addr learned my addr 172.21.15.47:0/4220691390 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:00:36.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:36.374+0000 7f0cdf7fe700 1 --2- 172.21.15.47:0/4220691390 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ce0101960 0x7f0ce01997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:36.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:36.374+0000 7f0cdffff700 1 --2- 172.21.15.47:0/4220691390 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ce0100fa0 0x7f0ce01992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:36.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:36.374+0000 7f0ce482a700 1 -- 172.21.15.47:0/4220691390 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ce0100fa0 msgr2=0x7f0ce01992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:36.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:36.374+0000 7f0ce482a700 1 --2- 172.21.15.47:0/4220691390 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ce0100fa0 0x7f0ce01992a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:36.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:36.374+0000 7f0ce482a700 1 -- 172.21.15.47:0/4220691390 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ce0101960 msgr2=0x7f0ce01997e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:36.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:36.374+0000 7f0ce482a700 1 --2- 172.21.15.47:0/4220691390 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ce0101960 0x7f0ce01997e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:36.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:36.374+0000 7f0ce482a700 1 -- 172.21.15.47:0/4220691390 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ce019e2b0 con 0x7f0ce0107a70 2024-04-17T06:00:36.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:36.374+0000 7f0cdffff700 1 --2- 172.21.15.47:0/4220691390 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ce0100fa0 0x7f0ce01992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:00:36.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:36.374+0000 7f0ce482a700 1 --2- 172.21.15.47:0/4220691390 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0ce0107a70 0x7f0ce019db70 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f0cd8007f10 tx=0x7f0cd800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:00:36.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:36.374+0000 7f0cdd7fa700 1 -- 172.21.15.47:0/4220691390 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd800ddd0 con 0x7f0ce0107a70 2024-04-17T06:00:36.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:36.374+0000 7f0cdd7fa700 1 -- 172.21.15.47:0/4220691390 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0cd8004020 con 0x7f0ce0107a70 2024-04-17T06:00:36.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:36.374+0000 7f0ce628d700 1 -- 172.21.15.47:0/4220691390 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ce019e5a0 con 0x7f0ce0107a70 2024-04-17T06:00:36.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:36.374+0000 7f0cdd7fa700 1 -- 172.21.15.47:0/4220691390 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd8015660 con 0x7f0ce0107a70 2024-04-17T06:00:36.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:36.374+0000 7f0ce628d700 1 -- 172.21.15.47:0/4220691390 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ce01a5e20 con 0x7f0ce0107a70 2024-04-17T06:00:36.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:36.376+0000 7f0ce628d700 1 -- 172.21.15.47:0/4220691390 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ce0069740 con 0x7f0ce0107a70 2024-04-17T06:00:36.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:36.376+0000 7f0cdd7fa700 1 -- 172.21.15.47:0/4220691390 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f0cd80041d0 con 0x7f0ce0107a70 2024-04-17T06:00:36.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:36.377+0000 7f0cdd7fa700 1 --2- 172.21.15.47:0/4220691390 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f0ccc0a3d50 0x7f0ccc0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:36.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:36.377+0000 7f0cdffff700 1 --2- 172.21.15.47:0/4220691390 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f0ccc0a3d50 0x7f0ccc0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:36.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:36.377+0000 7f0cdd7fa700 1 -- 172.21.15.47:0/4220691390 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f0cd80cff60 con 0x7f0ce0107a70 2024-04-17T06:00:36.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:36.378+0000 7f0cdffff700 1 --2- 172.21.15.47:0/4220691390 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f0ccc0a3d50 0x7f0ccc0a6210 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0cc8005fd0 tx=0x7f0cc8005d40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:00:36.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:36.382+0000 7f0cdd7fa700 1 -- 172.21.15.47:0/4220691390 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f0cd8010070 con 0x7f0ce0107a70 2024-04-17T06:00:36.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:36.588+0000 7f0ce628d700 1 -- 172.21.15.47:0/4220691390 --> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0ce019a600 con 0x7f0ccc0a3d50 2024-04-17T06:00:36.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:36.595+0000 7f0cdd7fa700 1 -- 172.21.15.47:0/4220691390 <== mgr.14152 v2:172.21.15.47:6800/3157902806 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258924 (secure 0 0 0) 0x7f0ce019a600 con 0x7f0ccc0a3d50 2024-04-17T06:00:36.597 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-17T06:00:36.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:36.599+0000 7f0ce628d700 1 -- 172.21.15.47:0/4220691390 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f0ccc0a3d50 msgr2=0x7f0ccc0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:36.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:36.599+0000 7f0ce628d700 1 --2- 172.21.15.47:0/4220691390 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f0ccc0a3d50 0x7f0ccc0a6210 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0cc8005fd0 tx=0x7f0cc8005d40 comp rx=0 tx=0).stop 2024-04-17T06:00:36.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:36.599+0000 7f0ce628d700 1 -- 172.21.15.47:0/4220691390 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0ce0107a70 msgr2=0x7f0ce019db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:36.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:36.599+0000 7f0ce628d700 1 --2- 172.21.15.47:0/4220691390 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0ce0107a70 0x7f0ce019db70 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f0cd8007f10 tx=0x7f0cd800d3b0 comp rx=0 tx=0).stop 2024-04-17T06:00:36.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:36.599+0000 7f0ce628d700 1 -- 172.21.15.47:0/4220691390 shutdown_connections 2024-04-17T06:00:36.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:36.599+0000 7f0ce628d700 1 --2- 172.21.15.47:0/4220691390 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f0ccc0a3d50 0x7f0ccc0a6210 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:36.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:36.599+0000 7f0ce628d700 1 --2- 172.21.15.47:0/4220691390 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0ce0107a70 0x7f0ce019db70 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:36.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:36.599+0000 7f0ce628d700 1 --2- 172.21.15.47:0/4220691390 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ce0100fa0 0x7f0ce01992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:36.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:36.599+0000 7f0ce628d700 1 --2- 172.21.15.47:0/4220691390 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ce0101960 0x7f0ce01997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:36.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:36.599+0000 7f0ce628d700 1 -- 172.21.15.47:0/4220691390 >> 172.21.15.47:0/4220691390 conn(0x7f0ce0077f50 msgr2=0x7f0ce00fd500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:00:36.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:36.599+0000 7f0ce628d700 1 -- 172.21.15.47:0/4220691390 shutdown_connections 2024-04-17T06:00:36.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:36.599+0000 7f0ce628d700 1 -- 172.21.15.47:0/4220691390 wait complete. 2024-04-17T06:00:36.600 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-04-17T06:00:37.008 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:00:36 smithi047 ceph-mon[32223]: pgmap v329: 97 pgs: 97 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-17T06:00:37.008 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:00:36 smithi047 ceph-mon[32223]: from='client.15000 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-17T06:00:37.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:00:36 smithi098 ceph-mon[36516]: pgmap v329: 97 pgs: 97 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-17T06:00:37.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:00:36 smithi098 ceph-mon[36516]: from='client.15000 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-17T06:00:37.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:00:36 smithi103 ceph-mon[36243]: pgmap v329: 97 pgs: 97 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-17T06:00:37.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:00:36 smithi103 ceph-mon[36243]: from='client.15000 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-17T06:00:37.310 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":329,"stamp":"2024-04-17T06:00:35.744160+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":654459,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":598,"num_read_kb":417,"num_write":339,"num_write_kb":3047,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":1240313,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":274,"ondisk_log_size":274,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":365000,"kb_used_data":11612,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124576824,"statfs":{"total":1151940427776,"available":1151566667776,"internally_reserved":0,"allocated":11890688,"data_stored":7467657,"data_compressed":121510,"data_compressed_allocated":155648,"data_compressed_original":1843200,"omap_allocated":19140,"internal_metadata":361674044},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":80,"num_read_kb":40,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003136"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.723843+0000","last_change":"2024-04-17T05:59:14.592927+0000","last_active":"2024-04-17T06:00:14.723843+0000","last_peered":"2024-04-17T06:00:14.723843+0000","last_clean":"2024-04-17T06:00:14.723843+0000","last_became_active":"2024-04-17T05:59:14.592036+0000","last_became_peered":"2024-04-17T05:59:14.592036+0000","last_unstale":"2024-04-17T06:00:14.723843+0000","last_undegraded":"2024-04-17T06:00:14.723843+0000","last_fullsized":"2024-04-17T06:00:14.723843+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:10:51.146397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.726024+0000","last_change":"2024-04-17T05:59:14.587862+0000","last_active":"2024-04-17T06:00:14.726024+0000","last_peered":"2024-04-17T06:00:14.726024+0000","last_clean":"2024-04-17T06:00:14.726024+0000","last_became_active":"2024-04-17T05:59:14.587553+0000","last_became_peered":"2024-04-17T05:59:14.587553+0000","last_unstale":"2024-04-17T06:00:14.726024+0000","last_undegraded":"2024-04-17T06:00:14.726024+0000","last_fullsized":"2024-04-17T06:00:14.726024+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:16:12.224247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.725576+0000","last_change":"2024-04-17T05:59:14.592223+0000","last_active":"2024-04-17T06:00:14.725576+0000","last_peered":"2024-04-17T06:00:14.725576+0000","last_clean":"2024-04-17T06:00:14.725576+0000","last_became_active":"2024-04-17T05:59:14.591892+0000","last_became_peered":"2024-04-17T05:59:14.591892+0000","last_unstale":"2024-04-17T06:00:14.725576+0000","last_undegraded":"2024-04-17T06:00:14.725576+0000","last_fullsized":"2024-04-17T06:00:14.725576+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:23:26.943368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:15.001748+0000","last_change":"2024-04-17T05:59:14.585607+0000","last_active":"2024-04-17T06:00:15.001748+0000","last_peered":"2024-04-17T06:00:15.001748+0000","last_clean":"2024-04-17T06:00:15.001748+0000","last_became_active":"2024-04-17T05:59:14.585255+0000","last_became_peered":"2024-04-17T05:59:14.585255+0000","last_unstale":"2024-04-17T06:00:15.001748+0000","last_undegraded":"2024-04-17T06:00:15.001748+0000","last_fullsized":"2024-04-17T06:00:15.001748+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:45:57.355035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.725535+0000","last_change":"2024-04-17T05:59:14.599703+0000","last_active":"2024-04-17T06:00:14.725535+0000","last_peered":"2024-04-17T06:00:14.725535+0000","last_clean":"2024-04-17T06:00:14.725535+0000","last_became_active":"2024-04-17T05:59:14.599305+0000","last_became_peered":"2024-04-17T05:59:14.599305+0000","last_unstale":"2024-04-17T06:00:14.725535+0000","last_undegraded":"2024-04-17T06:00:14.725535+0000","last_fullsized":"2024-04-17T06:00:14.725535+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:58:36.056590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.729429+0000","last_change":"2024-04-17T05:59:14.586176+0000","last_active":"2024-04-17T06:00:14.729429+0000","last_peered":"2024-04-17T06:00:14.729429+0000","last_clean":"2024-04-17T06:00:14.729429+0000","last_became_active":"2024-04-17T05:59:14.585850+0000","last_became_peered":"2024-04-17T05:59:14.585850+0000","last_unstale":"2024-04-17T06:00:14.729429+0000","last_undegraded":"2024-04-17T06:00:14.729429+0000","last_fullsized":"2024-04-17T06:00:14.729429+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:01:14.628048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.726720+0000","last_change":"2024-04-17T05:59:14.587750+0000","last_active":"2024-04-17T06:00:14.726720+0000","last_peered":"2024-04-17T06:00:14.726720+0000","last_clean":"2024-04-17T06:00:14.726720+0000","last_became_active":"2024-04-17T05:59:14.587394+0000","last_became_peered":"2024-04-17T05:59:14.587394+0000","last_unstale":"2024-04-17T06:00:14.726720+0000","last_undegraded":"2024-04-17T06:00:14.726720+0000","last_fullsized":"2024-04-17T06:00:14.726720+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:22:52.733877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:15.000910+0000","last_change":"2024-04-17T05:59:14.591458+0000","last_active":"2024-04-17T06:00:15.000910+0000","last_peered":"2024-04-17T06:00:15.000910+0000","last_clean":"2024-04-17T06:00:15.000910+0000","last_became_active":"2024-04-17T05:59:14.590898+0000","last_became_peered":"2024-04-17T05:59:14.590898+0000","last_unstale":"2024-04-17T06:00:15.000910+0000","last_undegraded":"2024-04-17T06:00:15.000910+0000","last_fullsized":"2024-04-17T06:00:15.000910+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:58:40.817928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.727959+0000","last_change":"2024-04-17T05:59:14.600199+0000","last_active":"2024-04-17T06:00:14.727959+0000","last_peered":"2024-04-17T06:00:14.727959+0000","last_clean":"2024-04-17T06:00:14.727959+0000","last_became_active":"2024-04-17T05:59:14.599286+0000","last_became_peered":"2024-04-17T05:59:14.599286+0000","last_unstale":"2024-04-17T06:00:14.727959+0000","last_undegraded":"2024-04-17T06:00:14.727959+0000","last_fullsized":"2024-04-17T06:00:14.727959+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:41:43.314103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.729872+0000","last_change":"2024-04-17T05:59:14.587999+0000","last_active":"2024-04-17T06:00:14.729872+0000","last_peered":"2024-04-17T06:00:14.729872+0000","last_clean":"2024-04-17T06:00:14.729872+0000","last_became_active":"2024-04-17T05:59:14.587655+0000","last_became_peered":"2024-04-17T05:59:14.587655+0000","last_unstale":"2024-04-17T06:00:14.729872+0000","last_undegraded":"2024-04-17T06:00:14.729872+0000","last_fullsized":"2024-04-17T06:00:14.729872+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:41:37.751776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.727096+0000","last_change":"2024-04-17T05:59:14.595003+0000","last_active":"2024-04-17T06:00:14.727096+0000","last_peered":"2024-04-17T06:00:14.727096+0000","last_clean":"2024-04-17T06:00:14.727096+0000","last_became_active":"2024-04-17T05:59:14.594424+0000","last_became_peered":"2024-04-17T05:59:14.594424+0000","last_unstale":"2024-04-17T06:00:14.727096+0000","last_undegraded":"2024-04-17T06:00:14.727096+0000","last_fullsized":"2024-04-17T06:00:14.727096+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:35:39.491074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:15.001695+0000","last_change":"2024-04-17T05:59:14.584957+0000","last_active":"2024-04-17T06:00:15.001695+0000","last_peered":"2024-04-17T06:00:15.001695+0000","last_clean":"2024-04-17T06:00:15.001695+0000","last_became_active":"2024-04-17T05:59:14.584625+0000","last_became_peered":"2024-04-17T05:59:14.584625+0000","last_unstale":"2024-04-17T06:00:15.001695+0000","last_undegraded":"2024-04-17T06:00:15.001695+0000","last_fullsized":"2024-04-17T06:00:15.001695+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:10:08.819831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.727596+0000","last_change":"2024-04-17T05:59:14.577633+0000","last_active":"2024-04-17T06:00:14.727596+0000","last_peered":"2024-04-17T06:00:14.727596+0000","last_clean":"2024-04-17T06:00:14.727596+0000","last_became_active":"2024-04-17T05:59:14.576608+0000","last_became_peered":"2024-04-17T05:59:14.576608+0000","last_unstale":"2024-04-17T06:00:14.727596+0000","last_undegraded":"2024-04-17T06:00:14.727596+0000","last_fullsized":"2024-04-17T06:00:14.727596+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:04:49.921958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.725330+0000","last_change":"2024-04-17T05:59:14.580912+0000","last_active":"2024-04-17T06:00:14.725330+0000","last_peered":"2024-04-17T06:00:14.725330+0000","last_clean":"2024-04-17T06:00:14.725330+0000","last_became_active":"2024-04-17T05:59:14.580500+0000","last_became_peered":"2024-04-17T05:59:14.580500+0000","last_unstale":"2024-04-17T06:00:14.725330+0000","last_undegraded":"2024-04-17T06:00:14.725330+0000","last_fullsized":"2024-04-17T06:00:14.725330+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:17:49.893499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.725925+0000","last_change":"2024-04-17T05:59:14.586109+0000","last_active":"2024-04-17T06:00:14.725925+0000","last_peered":"2024-04-17T06:00:14.725925+0000","last_clean":"2024-04-17T06:00:14.725925+0000","last_became_active":"2024-04-17T05:59:14.585667+0000","last_became_peered":"2024-04-17T05:59:14.585667+0000","last_unstale":"2024-04-17T06:00:14.725925+0000","last_undegraded":"2024-04-17T06:00:14.725925+0000","last_fullsized":"2024-04-17T06:00:14.725925+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:20:22.309678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.728517+0000","last_change":"2024-04-17T05:59:14.602921+0000","last_active":"2024-04-17T06:00:14.728517+0000","last_peered":"2024-04-17T06:00:14.728517+0000","last_clean":"2024-04-17T06:00:14.728517+0000","last_became_active":"2024-04-17T05:59:14.602589+0000","last_became_peered":"2024-04-17T05:59:14.602589+0000","last_unstale":"2024-04-17T06:00:14.728517+0000","last_undegraded":"2024-04-17T06:00:14.728517+0000","last_fullsized":"2024-04-17T06:00:14.728517+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:49:17.988067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.723932+0000","last_change":"2024-04-17T05:59:14.594201+0000","last_active":"2024-04-17T06:00:14.723932+0000","last_peered":"2024-04-17T06:00:14.723932+0000","last_clean":"2024-04-17T06:00:14.723932+0000","last_became_active":"2024-04-17T05:59:14.593850+0000","last_became_peered":"2024-04-17T05:59:14.593850+0000","last_unstale":"2024-04-17T06:00:14.723932+0000","last_undegraded":"2024-04-17T06:00:14.723932+0000","last_fullsized":"2024-04-17T06:00:14.723932+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:53:09.044324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.723715+0000","last_change":"2024-04-17T05:59:14.590439+0000","last_active":"2024-04-17T06:00:14.723715+0000","last_peered":"2024-04-17T06:00:14.723715+0000","last_clean":"2024-04-17T06:00:14.723715+0000","last_became_active":"2024-04-17T05:59:14.589844+0000","last_became_peered":"2024-04-17T05:59:14.589844+0000","last_unstale":"2024-04-17T06:00:14.723715+0000","last_undegraded":"2024-04-17T06:00:14.723715+0000","last_fullsized":"2024-04-17T06:00:14.723715+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:02:18.366048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.726966+0000","last_change":"2024-04-17T05:59:14.592297+0000","last_active":"2024-04-17T06:00:14.726966+0000","last_peered":"2024-04-17T06:00:14.726966+0000","last_clean":"2024-04-17T06:00:14.726966+0000","last_became_active":"2024-04-17T05:59:14.591976+0000","last_became_peered":"2024-04-17T05:59:14.591976+0000","last_unstale":"2024-04-17T06:00:14.726966+0000","last_undegraded":"2024-04-17T06:00:14.726966+0000","last_fullsized":"2024-04-17T06:00:14.726966+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:11:47.488231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.723707+0000","last_change":"2024-04-17T05:59:13.561616+0000","last_active":"2024-04-17T06:00:14.723707+0000","last_peered":"2024-04-17T06:00:14.723707+0000","last_clean":"2024-04-17T06:00:14.723707+0000","last_became_active":"2024-04-17T05:59:13.561298+0000","last_became_peered":"2024-04-17T05:59:13.561298+0000","last_unstale":"2024-04-17T06:00:14.723707+0000","last_undegraded":"2024-04-17T06:00:14.723707+0000","last_fullsized":"2024-04-17T06:00:14.723707+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:59:02.598774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:15.001431+0000","last_change":"2024-04-17T05:59:14.592023+0000","last_active":"2024-04-17T06:00:15.001431+0000","last_peered":"2024-04-17T06:00:15.001431+0000","last_clean":"2024-04-17T06:00:15.001431+0000","last_became_active":"2024-04-17T05:59:14.591707+0000","last_became_peered":"2024-04-17T05:59:14.591707+0000","last_unstale":"2024-04-17T06:00:15.001431+0000","last_undegraded":"2024-04-17T06:00:15.001431+0000","last_fullsized":"2024-04-17T06:00:15.001431+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:50:06.828184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:15.001551+0000","last_change":"2024-04-17T05:59:13.568073+0000","last_active":"2024-04-17T06:00:15.001551+0000","last_peered":"2024-04-17T06:00:15.001551+0000","last_clean":"2024-04-17T06:00:15.001551+0000","last_became_active":"2024-04-17T05:59:13.567751+0000","last_became_peered":"2024-04-17T05:59:13.567751+0000","last_unstale":"2024-04-17T06:00:15.001551+0000","last_undegraded":"2024-04-17T06:00:15.001551+0000","last_fullsized":"2024-04-17T06:00:15.001551+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:40:26.884308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.725947+0000","last_change":"2024-04-17T05:59:14.593344+0000","last_active":"2024-04-17T06:00:14.725947+0000","last_peered":"2024-04-17T06:00:14.725947+0000","last_clean":"2024-04-17T06:00:14.725947+0000","last_became_active":"2024-04-17T05:59:14.592946+0000","last_became_peered":"2024-04-17T05:59:14.592946+0000","last_unstale":"2024-04-17T06:00:14.725947+0000","last_undegraded":"2024-04-17T06:00:14.725947+0000","last_fullsized":"2024-04-17T06:00:14.725947+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:50:23.979412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.726609+0000","last_change":"2024-04-17T05:59:13.566639+0000","last_active":"2024-04-17T06:00:14.726609+0000","last_peered":"2024-04-17T06:00:14.726609+0000","last_clean":"2024-04-17T06:00:14.726609+0000","last_became_active":"2024-04-17T05:59:13.562766+0000","last_became_peered":"2024-04-17T05:59:13.562766+0000","last_unstale":"2024-04-17T06:00:14.726609+0000","last_undegraded":"2024-04-17T06:00:14.726609+0000","last_fullsized":"2024-04-17T06:00:14.726609+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:36:47.605106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.727742+0000","last_change":"2024-04-17T05:59:14.596055+0000","last_active":"2024-04-17T06:00:14.727742+0000","last_peered":"2024-04-17T06:00:14.727742+0000","last_clean":"2024-04-17T06:00:14.727742+0000","last_became_active":"2024-04-17T05:59:14.595672+0000","last_became_peered":"2024-04-17T05:59:14.595672+0000","last_unstale":"2024-04-17T06:00:14.727742+0000","last_undegraded":"2024-04-17T06:00:14.727742+0000","last_fullsized":"2024-04-17T06:00:14.727742+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:31:59.964502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.725808+0000","last_change":"2024-04-17T05:59:13.558715+0000","last_active":"2024-04-17T06:00:14.725808+0000","last_peered":"2024-04-17T06:00:14.725808+0000","last_clean":"2024-04-17T06:00:14.725808+0000","last_became_active":"2024-04-17T05:59:13.558377+0000","last_became_peered":"2024-04-17T05:59:13.558377+0000","last_unstale":"2024-04-17T06:00:14.725808+0000","last_undegraded":"2024-04-17T06:00:14.725808+0000","last_fullsized":"2024-04-17T06:00:14.725808+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:12:12.392655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.731769+0000","last_change":"2024-04-17T06:00:14.731769+0000","last_active":"2024-04-17T06:00:14.731769+0000","last_peered":"2024-04-17T06:00:14.731769+0000","last_clean":"2024-04-17T06:00:14.731769+0000","last_became_active":"2024-04-17T06:00:14.731483+0000","last_became_peered":"2024-04-17T06:00:14.731483+0000","last_unstale":"2024-04-17T06:00:14.731769+0000","last_undegraded":"2024-04-17T06:00:14.731769+0000","last_fullsized":"2024-04-17T06:00:14.731769+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:00:29.850454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'6","reported_seq":103,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:32.712412+0000","last_change":"2024-04-17T05:59:13.567654+0000","last_active":"2024-04-17T06:00:32.712412+0000","last_peered":"2024-04-17T06:00:32.712412+0000","last_clean":"2024-04-17T06:00:32.712412+0000","last_became_active":"2024-04-17T05:59:13.566474+0000","last_became_peered":"2024-04-17T05:59:13.566474+0000","last_unstale":"2024-04-17T06:00:32.712412+0000","last_undegraded":"2024-04-17T06:00:32.712412+0000","last_fullsized":"2024-04-17T06:00:32.712412+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:40:25.205832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":78,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.725594+0000","last_change":"2024-04-17T05:59:14.597145+0000","last_active":"2024-04-17T06:00:14.725594+0000","last_peered":"2024-04-17T06:00:14.725594+0000","last_clean":"2024-04-17T06:00:14.725594+0000","last_became_active":"2024-04-17T05:59:14.596802+0000","last_became_peered":"2024-04-17T05:59:14.596802+0000","last_unstale":"2024-04-17T06:00:14.725594+0000","last_undegraded":"2024-04-17T06:00:14.725594+0000","last_fullsized":"2024-04-17T06:00:14.725594+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:41:04.946052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.729325+0000","last_change":"2024-04-17T05:59:13.560100+0000","last_active":"2024-04-17T06:00:14.729325+0000","last_peered":"2024-04-17T06:00:14.729325+0000","last_clean":"2024-04-17T06:00:14.729325+0000","last_became_active":"2024-04-17T05:59:13.559769+0000","last_became_peered":"2024-04-17T05:59:13.559769+0000","last_unstale":"2024-04-17T06:00:14.729325+0000","last_undegraded":"2024-04-17T06:00:14.729325+0000","last_fullsized":"2024-04-17T06:00:14.729325+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:44:30.299028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.728753+0000","last_change":"2024-04-17T05:59:14.600733+0000","last_active":"2024-04-17T06:00:14.728753+0000","last_peered":"2024-04-17T06:00:14.728753+0000","last_clean":"2024-04-17T06:00:14.728753+0000","last_became_active":"2024-04-17T05:59:14.599273+0000","last_became_peered":"2024-04-17T05:59:14.599273+0000","last_unstale":"2024-04-17T06:00:14.728753+0000","last_undegraded":"2024-04-17T06:00:14.728753+0000","last_fullsized":"2024-04-17T06:00:14.728753+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:27:02.721452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.727705+0000","last_change":"2024-04-17T05:59:13.553000+0000","last_active":"2024-04-17T06:00:14.727705+0000","last_peered":"2024-04-17T06:00:14.727705+0000","last_clean":"2024-04-17T06:00:14.727705+0000","last_became_active":"2024-04-17T05:59:13.552469+0000","last_became_peered":"2024-04-17T05:59:13.552469+0000","last_unstale":"2024-04-17T06:00:14.727705+0000","last_undegraded":"2024-04-17T06:00:14.727705+0000","last_fullsized":"2024-04-17T06:00:14.727705+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:14:34.714179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:15.000886+0000","last_change":"2024-04-17T05:59:14.594792+0000","last_active":"2024-04-17T06:00:15.000886+0000","last_peered":"2024-04-17T06:00:15.000886+0000","last_clean":"2024-04-17T06:00:15.000886+0000","last_became_active":"2024-04-17T05:59:14.594436+0000","last_became_peered":"2024-04-17T05:59:14.594436+0000","last_unstale":"2024-04-17T06:00:15.000886+0000","last_undegraded":"2024-04-17T06:00:15.000886+0000","last_fullsized":"2024-04-17T06:00:15.000886+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:49:57.104050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.728365+0000","last_change":"2024-04-17T05:59:13.565352+0000","last_active":"2024-04-17T06:00:14.728365+0000","last_peered":"2024-04-17T06:00:14.728365+0000","last_clean":"2024-04-17T06:00:14.728365+0000","last_became_active":"2024-04-17T05:59:13.565046+0000","last_became_peered":"2024-04-17T05:59:13.565046+0000","last_unstale":"2024-04-17T06:00:14.728365+0000","last_undegraded":"2024-04-17T06:00:14.728365+0000","last_fullsized":"2024-04-17T06:00:14.728365+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:39:25.276782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.723409+0000","last_change":"2024-04-17T05:59:14.593925+0000","last_active":"2024-04-17T06:00:14.723409+0000","last_peered":"2024-04-17T06:00:14.723409+0000","last_clean":"2024-04-17T06:00:14.723409+0000","last_became_active":"2024-04-17T05:59:14.593535+0000","last_became_peered":"2024-04-17T05:59:14.593535+0000","last_unstale":"2024-04-17T06:00:14.723409+0000","last_undegraded":"2024-04-17T06:00:14.723409+0000","last_fullsized":"2024-04-17T06:00:14.723409+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:58:20.811999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'3","reported_seq":97,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:32.711853+0000","last_change":"2024-04-17T05:59:13.562281+0000","last_active":"2024-04-17T06:00:32.711853+0000","last_peered":"2024-04-17T06:00:32.711853+0000","last_clean":"2024-04-17T06:00:32.711853+0000","last_became_active":"2024-04-17T05:59:13.561931+0000","last_became_peered":"2024-04-17T05:59:13.561931+0000","last_unstale":"2024-04-17T06:00:32.711853+0000","last_undegraded":"2024-04-17T06:00:32.711853+0000","last_fullsized":"2024-04-17T06:00:32.711853+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:55:52.119923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":75,"num_read_kb":75,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.727303+0000","last_change":"2024-04-17T05:59:14.594708+0000","last_active":"2024-04-17T06:00:14.727303+0000","last_peered":"2024-04-17T06:00:14.727303+0000","last_clean":"2024-04-17T06:00:14.727303+0000","last_became_active":"2024-04-17T05:59:14.594206+0000","last_became_peered":"2024-04-17T05:59:14.594206+0000","last_unstale":"2024-04-17T06:00:14.727303+0000","last_undegraded":"2024-04-17T06:00:14.727303+0000","last_fullsized":"2024-04-17T06:00:14.727303+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:43:16.402659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.726993+0000","last_change":"2024-04-17T05:59:13.561056+0000","last_active":"2024-04-17T06:00:14.726993+0000","last_peered":"2024-04-17T06:00:14.726993+0000","last_clean":"2024-04-17T06:00:14.726993+0000","last_became_active":"2024-04-17T05:59:13.560670+0000","last_became_peered":"2024-04-17T05:59:13.560670+0000","last_unstale":"2024-04-17T06:00:14.726993+0000","last_undegraded":"2024-04-17T06:00:14.726993+0000","last_fullsized":"2024-04-17T06:00:14.726993+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:31:01.106013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.725926+0000","last_change":"2024-04-17T05:59:14.590243+0000","last_active":"2024-04-17T06:00:14.725926+0000","last_peered":"2024-04-17T06:00:14.725926+0000","last_clean":"2024-04-17T06:00:14.725926+0000","last_became_active":"2024-04-17T05:59:14.589911+0000","last_became_peered":"2024-04-17T05:59:14.589911+0000","last_unstale":"2024-04-17T06:00:14.725926+0000","last_undegraded":"2024-04-17T06:00:14.725926+0000","last_fullsized":"2024-04-17T06:00:14.725926+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:44:41.855928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.729590+0000","last_change":"2024-04-17T05:59:13.569983+0000","last_active":"2024-04-17T06:00:14.729590+0000","last_peered":"2024-04-17T06:00:14.729590+0000","last_clean":"2024-04-17T06:00:14.729590+0000","last_became_active":"2024-04-17T05:59:13.569667+0000","last_became_peered":"2024-04-17T05:59:13.569667+0000","last_unstale":"2024-04-17T06:00:14.729590+0000","last_undegraded":"2024-04-17T06:00:14.729590+0000","last_fullsized":"2024-04-17T06:00:14.729590+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:36:15.362530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.728126+0000","last_change":"2024-04-17T05:59:14.603404+0000","last_active":"2024-04-17T06:00:14.728126+0000","last_peered":"2024-04-17T06:00:14.728126+0000","last_clean":"2024-04-17T06:00:14.728126+0000","last_became_active":"2024-04-17T05:59:14.601593+0000","last_became_peered":"2024-04-17T05:59:14.601593+0000","last_unstale":"2024-04-17T06:00:14.728126+0000","last_undegraded":"2024-04-17T06:00:14.728126+0000","last_fullsized":"2024-04-17T06:00:14.728126+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:44:44.490496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'27","reported_seq":221,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:34.032987+0000","last_change":"2024-04-17T06:00:15.008166+0000","last_active":"2024-04-17T06:00:34.032987+0000","last_peered":"2024-04-17T06:00:34.032987+0000","last_clean":"2024-04-17T06:00:34.032987+0000","last_became_active":"2024-04-17T06:00:15.002373+0000","last_became_peered":"2024-04-17T06:00:15.002373+0000","last_unstale":"2024-04-17T06:00:34.032987+0000","last_undegraded":"2024-04-17T06:00:34.032987+0000","last_fullsized":"2024-04-17T06:00:34.032987+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:32:09.078832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59465,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":147,"num_read_kb":131,"num_write":28,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":59465,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.726358+0000","last_change":"2024-04-17T05:59:14.579304+0000","last_active":"2024-04-17T06:00:14.726358+0000","last_peered":"2024-04-17T06:00:14.726358+0000","last_clean":"2024-04-17T06:00:14.726358+0000","last_became_active":"2024-04-17T05:59:14.578948+0000","last_became_peered":"2024-04-17T05:59:14.578948+0000","last_unstale":"2024-04-17T06:00:14.726358+0000","last_undegraded":"2024-04-17T06:00:14.726358+0000","last_fullsized":"2024-04-17T06:00:14.726358+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:39:43.341725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.726288+0000","last_change":"2024-04-17T05:59:13.569534+0000","last_active":"2024-04-17T06:00:14.726288+0000","last_peered":"2024-04-17T06:00:14.726288+0000","last_clean":"2024-04-17T06:00:14.726288+0000","last_became_active":"2024-04-17T05:59:13.569041+0000","last_became_peered":"2024-04-17T05:59:13.569041+0000","last_unstale":"2024-04-17T06:00:14.726288+0000","last_undegraded":"2024-04-17T06:00:14.726288+0000","last_fullsized":"2024-04-17T06:00:14.726288+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:27:37.018696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.727562+0000","last_change":"2024-04-17T05:59:14.593820+0000","last_active":"2024-04-17T06:00:14.727562+0000","last_peered":"2024-04-17T06:00:14.727562+0000","last_clean":"2024-04-17T06:00:14.727562+0000","last_became_active":"2024-04-17T05:59:14.593473+0000","last_became_peered":"2024-04-17T05:59:14.593473+0000","last_unstale":"2024-04-17T06:00:14.727562+0000","last_undegraded":"2024-04-17T06:00:14.727562+0000","last_fullsized":"2024-04-17T06:00:14.727562+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:44:38.636980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.728888+0000","last_change":"2024-04-17T05:59:13.570124+0000","last_active":"2024-04-17T06:00:14.728888+0000","last_peered":"2024-04-17T06:00:14.728888+0000","last_clean":"2024-04-17T06:00:14.728888+0000","last_became_active":"2024-04-17T05:59:13.569778+0000","last_became_peered":"2024-04-17T05:59:13.569778+0000","last_unstale":"2024-04-17T06:00:14.728888+0000","last_undegraded":"2024-04-17T06:00:14.728888+0000","last_fullsized":"2024-04-17T06:00:14.728888+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:46:46.431207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.727119+0000","last_change":"2024-04-17T05:59:14.584863+0000","last_active":"2024-04-17T06:00:14.727119+0000","last_peered":"2024-04-17T06:00:14.727119+0000","last_clean":"2024-04-17T06:00:14.727119+0000","last_became_active":"2024-04-17T05:59:14.584554+0000","last_became_peered":"2024-04-17T05:59:14.584554+0000","last_unstale":"2024-04-17T06:00:14.727119+0000","last_undegraded":"2024-04-17T06:00:14.727119+0000","last_fullsized":"2024-04-17T06:00:14.727119+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:26:22.064337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'4","reported_seq":99,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:30.725456+0000","last_change":"2024-04-17T05:59:13.557625+0000","last_active":"2024-04-17T06:00:30.725456+0000","last_peered":"2024-04-17T06:00:30.725456+0000","last_clean":"2024-04-17T06:00:30.725456+0000","last_became_active":"2024-04-17T05:59:13.557307+0000","last_became_peered":"2024-04-17T05:59:13.557307+0000","last_unstale":"2024-04-17T06:00:30.725456+0000","last_undegraded":"2024-04-17T06:00:30.725456+0000","last_fullsized":"2024-04-17T06:00:30.725456+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:43:17.023049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":77,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.731050+0000","last_change":"2024-04-17T06:00:14.731050+0000","last_active":"2024-04-17T06:00:14.731050+0000","last_peered":"2024-04-17T06:00:14.731050+0000","last_clean":"2024-04-17T06:00:14.731050+0000","last_became_active":"2024-04-17T06:00:14.730653+0000","last_became_peered":"2024-04-17T06:00:14.730653+0000","last_unstale":"2024-04-17T06:00:14.731050+0000","last_undegraded":"2024-04-17T06:00:14.731050+0000","last_fullsized":"2024-04-17T06:00:14.731050+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:11:23.354930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.741811+0000","last_change":"2024-04-17T06:00:14.741383+0000","last_active":"2024-04-17T06:00:14.741811+0000","last_peered":"2024-04-17T06:00:14.741811+0000","last_clean":"2024-04-17T06:00:14.741811+0000","last_became_active":"2024-04-17T06:00:14.727133+0000","last_became_peered":"2024-04-17T06:00:14.727133+0000","last_unstale":"2024-04-17T06:00:14.741811+0000","last_undegraded":"2024-04-17T06:00:14.741811+0000","last_fullsized":"2024-04-17T06:00:14.741811+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:27:46.912019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.728623+0000","last_change":"2024-04-17T05:59:14.594082+0000","last_active":"2024-04-17T06:00:14.728623+0000","last_peered":"2024-04-17T06:00:14.728623+0000","last_clean":"2024-04-17T06:00:14.728623+0000","last_became_active":"2024-04-17T05:59:14.593754+0000","last_became_peered":"2024-04-17T05:59:14.593754+0000","last_unstale":"2024-04-17T06:00:14.728623+0000","last_undegraded":"2024-04-17T06:00:14.728623+0000","last_fullsized":"2024-04-17T06:00:14.728623+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:36:28.752766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.726112+0000","last_change":"2024-04-17T05:59:13.563470+0000","last_active":"2024-04-17T06:00:14.726112+0000","last_peered":"2024-04-17T06:00:14.726112+0000","last_clean":"2024-04-17T06:00:14.726112+0000","last_became_active":"2024-04-17T05:59:13.563067+0000","last_became_peered":"2024-04-17T05:59:13.563067+0000","last_unstale":"2024-04-17T06:00:14.726112+0000","last_undegraded":"2024-04-17T06:00:14.726112+0000","last_fullsized":"2024-04-17T06:00:14.726112+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:13:52.018604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.723699+0000","last_change":"2024-04-17T05:59:14.587339+0000","last_active":"2024-04-17T06:00:14.723699+0000","last_peered":"2024-04-17T06:00:14.723699+0000","last_clean":"2024-04-17T06:00:14.723699+0000","last_became_active":"2024-04-17T05:59:14.586956+0000","last_became_peered":"2024-04-17T05:59:14.586956+0000","last_unstale":"2024-04-17T06:00:14.723699+0000","last_undegraded":"2024-04-17T06:00:14.723699+0000","last_fullsized":"2024-04-17T06:00:14.723699+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:00:21.494832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.723479+0000","last_change":"2024-04-17T05:59:13.563915+0000","last_active":"2024-04-17T06:00:14.723479+0000","last_peered":"2024-04-17T06:00:14.723479+0000","last_clean":"2024-04-17T06:00:14.723479+0000","last_became_active":"2024-04-17T05:59:13.563218+0000","last_became_peered":"2024-04-17T05:59:13.563218+0000","last_unstale":"2024-04-17T06:00:14.723479+0000","last_undegraded":"2024-04-17T06:00:14.723479+0000","last_fullsized":"2024-04-17T06:00:14.723479+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:36:54.484278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"86'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.729467+0000","last_change":"2024-04-17T05:59:14.593168+0000","last_active":"2024-04-17T06:00:14.729467+0000","last_peered":"2024-04-17T06:00:14.729467+0000","last_clean":"2024-04-17T06:00:14.729467+0000","last_became_active":"2024-04-17T05:59:14.592744+0000","last_became_peered":"2024-04-17T05:59:14.592744+0000","last_unstale":"2024-04-17T06:00:14.729467+0000","last_undegraded":"2024-04-17T06:00:14.729467+0000","last_fullsized":"2024-04-17T06:00:14.729467+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:00:03.217864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.726881+0000","last_change":"2024-04-17T05:59:13.564703+0000","last_active":"2024-04-17T06:00:14.726881+0000","last_peered":"2024-04-17T06:00:14.726881+0000","last_clean":"2024-04-17T06:00:14.726881+0000","last_became_active":"2024-04-17T05:59:13.564263+0000","last_became_peered":"2024-04-17T05:59:13.564263+0000","last_unstale":"2024-04-17T06:00:14.726881+0000","last_undegraded":"2024-04-17T06:00:14.726881+0000","last_fullsized":"2024-04-17T06:00:14.726881+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:51:06.847989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.729842+0000","last_change":"2024-04-17T05:59:14.588074+0000","last_active":"2024-04-17T06:00:14.729842+0000","last_peered":"2024-04-17T06:00:14.729842+0000","last_clean":"2024-04-17T06:00:14.729842+0000","last_became_active":"2024-04-17T05:59:14.587691+0000","last_became_peered":"2024-04-17T05:59:14.587691+0000","last_unstale":"2024-04-17T06:00:14.729842+0000","last_undegraded":"2024-04-17T06:00:14.729842+0000","last_fullsized":"2024-04-17T06:00:14.729842+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:11:15.775356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.728510+0000","last_change":"2024-04-17T05:59:14.589178+0000","last_active":"2024-04-17T06:00:14.728510+0000","last_peered":"2024-04-17T06:00:14.728510+0000","last_clean":"2024-04-17T06:00:14.728510+0000","last_became_active":"2024-04-17T05:59:14.587351+0000","last_became_peered":"2024-04-17T05:59:14.587351+0000","last_unstale":"2024-04-17T06:00:14.728510+0000","last_undegraded":"2024-04-17T06:00:14.728510+0000","last_fullsized":"2024-04-17T06:00:14.728510+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:22:43.202731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'4","reported_seq":96,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:30.724745+0000","last_change":"2024-04-17T05:59:13.553754+0000","last_active":"2024-04-17T06:00:30.724745+0000","last_peered":"2024-04-17T06:00:30.724745+0000","last_clean":"2024-04-17T06:00:30.724745+0000","last_became_active":"2024-04-17T05:59:13.553437+0000","last_became_peered":"2024-04-17T05:59:13.553437+0000","last_unstale":"2024-04-17T06:00:30.724745+0000","last_undegraded":"2024-04-17T06:00:30.724745+0000","last_fullsized":"2024-04-17T06:00:30.724745+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:10:14.010264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":73,"num_read_kb":73,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:15.001280+0000","last_change":"2024-04-17T05:59:14.592074+0000","last_active":"2024-04-17T06:00:15.001280+0000","last_peered":"2024-04-17T06:00:15.001280+0000","last_clean":"2024-04-17T06:00:15.001280+0000","last_became_active":"2024-04-17T05:59:14.591695+0000","last_became_peered":"2024-04-17T05:59:14.591695+0000","last_unstale":"2024-04-17T06:00:15.001280+0000","last_undegraded":"2024-04-17T06:00:15.001280+0000","last_fullsized":"2024-04-17T06:00:15.001280+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:27:04.344257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:15.001167+0000","last_change":"2024-04-17T05:59:13.553916+0000","last_active":"2024-04-17T06:00:15.001167+0000","last_peered":"2024-04-17T06:00:15.001167+0000","last_clean":"2024-04-17T06:00:15.001167+0000","last_became_active":"2024-04-17T05:59:13.553597+0000","last_became_peered":"2024-04-17T05:59:13.553597+0000","last_unstale":"2024-04-17T06:00:15.001167+0000","last_undegraded":"2024-04-17T06:00:15.001167+0000","last_fullsized":"2024-04-17T06:00:15.001167+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:28:23.486188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":352,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.726650+0000","last_change":"2024-04-17T05:58:02.597504+0000","last_active":"2024-04-17T06:00:14.726650+0000","last_peered":"2024-04-17T06:00:14.726650+0000","last_clean":"2024-04-17T06:00:14.726650+0000","last_became_active":"2024-04-17T05:58:02.597086+0000","last_became_peered":"2024-04-17T05:58:02.597086+0000","last_unstale":"2024-04-17T06:00:14.726650+0000","last_undegraded":"2024-04-17T06:00:14.726650+0000","last_fullsized":"2024-04-17T06:00:14.726650+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:54:37.919204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:54:37.919204+0000","last_clean_scrub_stamp":"2024-04-17T05:54:37.919204+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:02:09.566418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.730339+0000","last_change":"2024-04-17T06:00:14.730339+0000","last_active":"2024-04-17T06:00:14.730339+0000","last_peered":"2024-04-17T06:00:14.730339+0000","last_clean":"2024-04-17T06:00:14.730339+0000","last_became_active":"2024-04-17T06:00:14.729872+0000","last_became_peered":"2024-04-17T06:00:14.729872+0000","last_unstale":"2024-04-17T06:00:14.730339+0000","last_undegraded":"2024-04-17T06:00:14.730339+0000","last_fullsized":"2024-04-17T06:00:14.730339+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:00:21.892184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.726839+0000","last_change":"2024-04-17T05:59:13.562426+0000","last_active":"2024-04-17T06:00:14.726839+0000","last_peered":"2024-04-17T06:00:14.726839+0000","last_clean":"2024-04-17T06:00:14.726839+0000","last_became_active":"2024-04-17T05:59:13.562133+0000","last_became_peered":"2024-04-17T05:59:13.562133+0000","last_unstale":"2024-04-17T06:00:14.726839+0000","last_undegraded":"2024-04-17T06:00:14.726839+0000","last_fullsized":"2024-04-17T06:00:14.726839+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:50:58.351924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.726439+0000","last_change":"2024-04-17T05:59:13.557783+0000","last_active":"2024-04-17T06:00:14.726439+0000","last_peered":"2024-04-17T06:00:14.726439+0000","last_clean":"2024-04-17T06:00:14.726439+0000","last_became_active":"2024-04-17T05:59:13.557485+0000","last_became_peered":"2024-04-17T05:59:13.557485+0000","last_unstale":"2024-04-17T06:00:14.726439+0000","last_undegraded":"2024-04-17T06:00:14.726439+0000","last_fullsized":"2024-04-17T06:00:14.726439+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:41:37.359553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.727610+0000","last_change":"2024-04-17T05:59:14.575993+0000","last_active":"2024-04-17T06:00:14.727610+0000","last_peered":"2024-04-17T06:00:14.727610+0000","last_clean":"2024-04-17T06:00:14.727610+0000","last_became_active":"2024-04-17T05:59:14.575654+0000","last_became_peered":"2024-04-17T05:59:14.575654+0000","last_unstale":"2024-04-17T06:00:14.727610+0000","last_undegraded":"2024-04-17T06:00:14.727610+0000","last_fullsized":"2024-04-17T06:00:14.727610+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:33:12.193747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'8","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.723773+0000","last_change":"2024-04-17T05:59:13.560256+0000","last_active":"2024-04-17T06:00:14.723773+0000","last_peered":"2024-04-17T06:00:14.723773+0000","last_clean":"2024-04-17T06:00:14.723773+0000","last_became_active":"2024-04-17T05:59:13.559938+0000","last_became_peered":"2024-04-17T05:59:13.559938+0000","last_unstale":"2024-04-17T06:00:14.723773+0000","last_undegraded":"2024-04-17T06:00:14.723773+0000","last_fullsized":"2024-04-17T06:00:14.723773+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:14:54.625298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:15.001565+0000","last_change":"2024-04-17T05:59:14.593955+0000","last_active":"2024-04-17T06:00:15.001565+0000","last_peered":"2024-04-17T06:00:15.001565+0000","last_clean":"2024-04-17T06:00:15.001565+0000","last_became_active":"2024-04-17T05:59:14.590626+0000","last_became_peered":"2024-04-17T05:59:14.590626+0000","last_unstale":"2024-04-17T06:00:15.001565+0000","last_undegraded":"2024-04-17T06:00:15.001565+0000","last_fullsized":"2024-04-17T06:00:15.001565+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:19:04.438634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:15.001228+0000","last_change":"2024-04-17T05:59:13.555734+0000","last_active":"2024-04-17T06:00:15.001228+0000","last_peered":"2024-04-17T06:00:15.001228+0000","last_clean":"2024-04-17T06:00:15.001228+0000","last_became_active":"2024-04-17T05:59:13.555284+0000","last_became_peered":"2024-04-17T05:59:13.555284+0000","last_unstale":"2024-04-17T06:00:15.001228+0000","last_undegraded":"2024-04-17T06:00:15.001228+0000","last_fullsized":"2024-04-17T06:00:15.001228+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:47:06.834332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.725806+0000","last_change":"2024-04-17T05:59:14.599078+0000","last_active":"2024-04-17T06:00:14.725806+0000","last_peered":"2024-04-17T06:00:14.725806+0000","last_clean":"2024-04-17T06:00:14.725806+0000","last_became_active":"2024-04-17T05:59:14.598670+0000","last_became_peered":"2024-04-17T05:59:14.598670+0000","last_unstale":"2024-04-17T06:00:14.725806+0000","last_undegraded":"2024-04-17T06:00:14.725806+0000","last_fullsized":"2024-04-17T06:00:14.725806+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:58:17.414704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.743749+0000","last_change":"2024-04-17T06:00:14.743749+0000","last_active":"2024-04-17T06:00:14.743749+0000","last_peered":"2024-04-17T06:00:14.743749+0000","last_clean":"2024-04-17T06:00:14.743749+0000","last_became_active":"2024-04-17T06:00:14.731352+0000","last_became_peered":"2024-04-17T06:00:14.731352+0000","last_unstale":"2024-04-17T06:00:14.743749+0000","last_undegraded":"2024-04-17T06:00:14.743749+0000","last_fullsized":"2024-04-17T06:00:14.743749+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:30:56.157829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.725286+0000","last_change":"2024-04-17T05:59:14.593450+0000","last_active":"2024-04-17T06:00:14.725286+0000","last_peered":"2024-04-17T06:00:14.725286+0000","last_clean":"2024-04-17T06:00:14.725286+0000","last_became_active":"2024-04-17T05:59:14.593104+0000","last_became_peered":"2024-04-17T05:59:14.593104+0000","last_unstale":"2024-04-17T06:00:14.725286+0000","last_undegraded":"2024-04-17T06:00:14.725286+0000","last_fullsized":"2024-04-17T06:00:14.725286+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:16:20.488110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:15.001104+0000","last_change":"2024-04-17T05:59:13.557330+0000","last_active":"2024-04-17T06:00:15.001104+0000","last_peered":"2024-04-17T06:00:15.001104+0000","last_clean":"2024-04-17T06:00:15.001104+0000","last_became_active":"2024-04-17T05:59:13.556970+0000","last_became_peered":"2024-04-17T05:59:13.556970+0000","last_unstale":"2024-04-17T06:00:15.001104+0000","last_undegraded":"2024-04-17T06:00:15.001104+0000","last_fullsized":"2024-04-17T06:00:15.001104+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:41:46.643845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.723595+0000","last_change":"2024-04-17T05:59:14.582281+0000","last_active":"2024-04-17T06:00:14.723595+0000","last_peered":"2024-04-17T06:00:14.723595+0000","last_clean":"2024-04-17T06:00:14.723595+0000","last_became_active":"2024-04-17T05:59:14.581922+0000","last_became_peered":"2024-04-17T05:59:14.581922+0000","last_unstale":"2024-04-17T06:00:14.723595+0000","last_undegraded":"2024-04-17T06:00:14.723595+0000","last_fullsized":"2024-04-17T06:00:14.723595+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:53:10.415786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.726097+0000","last_change":"2024-04-17T05:59:13.566514+0000","last_active":"2024-04-17T06:00:14.726097+0000","last_peered":"2024-04-17T06:00:14.726097+0000","last_clean":"2024-04-17T06:00:14.726097+0000","last_became_active":"2024-04-17T05:59:13.566004+0000","last_became_peered":"2024-04-17T05:59:13.566004+0000","last_unstale":"2024-04-17T06:00:14.726097+0000","last_undegraded":"2024-04-17T06:00:14.726097+0000","last_fullsized":"2024-04-17T06:00:14.726097+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:20:13.891422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.732919+0000","last_change":"2024-04-17T06:00:14.731515+0000","last_active":"2024-04-17T06:00:14.732919+0000","last_peered":"2024-04-17T06:00:14.732919+0000","last_clean":"2024-04-17T06:00:14.732919+0000","last_became_active":"2024-04-17T06:00:14.731153+0000","last_became_peered":"2024-04-17T06:00:14.731153+0000","last_unstale":"2024-04-17T06:00:14.732919+0000","last_undegraded":"2024-04-17T06:00:14.732919+0000","last_fullsized":"2024-04-17T06:00:14.732919+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:37:42.248717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,10],"acting":[9,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.725420+0000","last_change":"2024-04-17T05:59:13.562531+0000","last_active":"2024-04-17T06:00:14.725420+0000","last_peered":"2024-04-17T06:00:14.725420+0000","last_clean":"2024-04-17T06:00:14.725420+0000","last_became_active":"2024-04-17T05:59:13.562148+0000","last_became_peered":"2024-04-17T05:59:13.562148+0000","last_unstale":"2024-04-17T06:00:14.725420+0000","last_undegraded":"2024-04-17T06:00:14.725420+0000","last_fullsized":"2024-04-17T06:00:14.725420+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:10:14.407871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.727091+0000","last_change":"2024-04-17T05:59:14.596371+0000","last_active":"2024-04-17T06:00:14.727091+0000","last_peered":"2024-04-17T06:00:14.727091+0000","last_clean":"2024-04-17T06:00:14.727091+0000","last_became_active":"2024-04-17T05:59:14.596024+0000","last_became_peered":"2024-04-17T05:59:14.596024+0000","last_unstale":"2024-04-17T06:00:14.727091+0000","last_undegraded":"2024-04-17T06:00:14.727091+0000","last_fullsized":"2024-04-17T06:00:14.727091+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:06:14.916212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.726600+0000","last_change":"2024-04-17T05:59:13.569364+0000","last_active":"2024-04-17T06:00:14.726600+0000","last_peered":"2024-04-17T06:00:14.726600+0000","last_clean":"2024-04-17T06:00:14.726600+0000","last_became_active":"2024-04-17T05:59:13.569029+0000","last_became_peered":"2024-04-17T05:59:13.569029+0000","last_unstale":"2024-04-17T06:00:14.726600+0000","last_undegraded":"2024-04-17T06:00:14.726600+0000","last_fullsized":"2024-04-17T06:00:14.726600+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:58:24.629274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.727216+0000","last_change":"2024-04-17T05:59:14.593088+0000","last_active":"2024-04-17T06:00:14.727216+0000","last_peered":"2024-04-17T06:00:14.727216+0000","last_clean":"2024-04-17T06:00:14.727216+0000","last_became_active":"2024-04-17T05:59:14.592755+0000","last_became_peered":"2024-04-17T05:59:14.592755+0000","last_unstale":"2024-04-17T06:00:14.727216+0000","last_undegraded":"2024-04-17T06:00:14.727216+0000","last_fullsized":"2024-04-17T06:00:14.727216+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:27:14.503323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.728685+0000","last_change":"2024-04-17T05:59:13.569773+0000","last_active":"2024-04-17T06:00:14.728685+0000","last_peered":"2024-04-17T06:00:14.728685+0000","last_clean":"2024-04-17T06:00:14.728685+0000","last_became_active":"2024-04-17T05:59:13.569461+0000","last_became_peered":"2024-04-17T05:59:13.569461+0000","last_unstale":"2024-04-17T06:00:14.728685+0000","last_undegraded":"2024-04-17T06:00:14.728685+0000","last_fullsized":"2024-04-17T06:00:14.728685+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:21:13.441453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"86'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.726672+0000","last_change":"2024-04-17T05:59:14.588939+0000","last_active":"2024-04-17T06:00:14.726672+0000","last_peered":"2024-04-17T06:00:14.726672+0000","last_clean":"2024-04-17T06:00:14.726672+0000","last_became_active":"2024-04-17T05:59:14.588600+0000","last_became_peered":"2024-04-17T05:59:14.588600+0000","last_unstale":"2024-04-17T06:00:14.726672+0000","last_undegraded":"2024-04-17T06:00:14.726672+0000","last_fullsized":"2024-04-17T06:00:14.726672+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:07:47.865808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.727908+0000","last_change":"2024-04-17T05:59:13.560566+0000","last_active":"2024-04-17T06:00:14.727908+0000","last_peered":"2024-04-17T06:00:14.727908+0000","last_clean":"2024-04-17T06:00:14.727908+0000","last_became_active":"2024-04-17T05:59:13.560228+0000","last_became_peered":"2024-04-17T05:59:13.560228+0000","last_unstale":"2024-04-17T06:00:14.727908+0000","last_undegraded":"2024-04-17T06:00:14.727908+0000","last_fullsized":"2024-04-17T06:00:14.727908+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:12.537873+0000","last_clean_scrub_stamp":"2024-04-17T05:59:12.537873+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:13:59.436598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.727159+0000","last_change":"2024-04-17T05:59:14.583621+0000","last_active":"2024-04-17T06:00:14.727159+0000","last_peered":"2024-04-17T06:00:14.727159+0000","last_clean":"2024-04-17T06:00:14.727159+0000","last_became_active":"2024-04-17T05:59:14.583199+0000","last_became_peered":"2024-04-17T05:59:14.583199+0000","last_unstale":"2024-04-17T06:00:14.727159+0000","last_undegraded":"2024-04-17T06:00:14.727159+0000","last_fullsized":"2024-04-17T06:00:14.727159+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:53:12.516661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.723804+0000","last_change":"2024-04-17T05:59:14.587989+0000","last_active":"2024-04-17T06:00:14.723804+0000","last_peered":"2024-04-17T06:00:14.723804+0000","last_clean":"2024-04-17T06:00:14.723804+0000","last_became_active":"2024-04-17T05:59:14.586686+0000","last_became_peered":"2024-04-17T05:59:14.586686+0000","last_unstale":"2024-04-17T06:00:14.723804+0000","last_undegraded":"2024-04-17T06:00:14.723804+0000","last_fullsized":"2024-04-17T06:00:14.723804+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:21:23.227328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.726127+0000","last_change":"2024-04-17T05:59:14.588567+0000","last_active":"2024-04-17T06:00:14.726127+0000","last_peered":"2024-04-17T06:00:14.726127+0000","last_clean":"2024-04-17T06:00:14.726127+0000","last_became_active":"2024-04-17T05:59:14.588207+0000","last_became_peered":"2024-04-17T05:59:14.588207+0000","last_unstale":"2024-04-17T06:00:14.726127+0000","last_undegraded":"2024-04-17T06:00:14.726127+0000","last_fullsized":"2024-04-17T06:00:14.726127+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:44:26.094334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.725983+0000","last_change":"2024-04-17T05:59:14.593885+0000","last_active":"2024-04-17T06:00:14.725983+0000","last_peered":"2024-04-17T06:00:14.725983+0000","last_clean":"2024-04-17T06:00:14.725983+0000","last_became_active":"2024-04-17T05:59:14.593483+0000","last_became_peered":"2024-04-17T05:59:14.593483+0000","last_unstale":"2024-04-17T06:00:14.725983+0000","last_undegraded":"2024-04-17T06:00:14.725983+0000","last_fullsized":"2024-04-17T06:00:14.725983+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:22:41.837363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.729047+0000","last_change":"2024-04-17T06:00:14.729047+0000","last_active":"2024-04-17T06:00:14.729047+0000","last_peered":"2024-04-17T06:00:14.729047+0000","last_clean":"2024-04-17T06:00:14.729047+0000","last_became_active":"2024-04-17T06:00:14.728675+0000","last_became_peered":"2024-04-17T06:00:14.728675+0000","last_unstale":"2024-04-17T06:00:14.729047+0000","last_undegraded":"2024-04-17T06:00:14.729047+0000","last_fullsized":"2024-04-17T06:00:14.729047+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:05:42.244001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.727516+0000","last_change":"2024-04-17T05:59:14.583266+0000","last_active":"2024-04-17T06:00:14.727516+0000","last_peered":"2024-04-17T06:00:14.727516+0000","last_clean":"2024-04-17T06:00:14.727516+0000","last_became_active":"2024-04-17T05:59:14.582949+0000","last_became_peered":"2024-04-17T05:59:14.582949+0000","last_unstale":"2024-04-17T06:00:14.727516+0000","last_undegraded":"2024-04-17T06:00:14.727516+0000","last_fullsized":"2024-04-17T06:00:14.727516+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:49:14.740168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.723697+0000","last_change":"2024-04-17T05:59:14.587046+0000","last_active":"2024-04-17T06:00:14.723697+0000","last_peered":"2024-04-17T06:00:14.723697+0000","last_clean":"2024-04-17T06:00:14.723697+0000","last_became_active":"2024-04-17T05:59:14.586653+0000","last_became_peered":"2024-04-17T05:59:14.586653+0000","last_unstale":"2024-04-17T06:00:14.723697+0000","last_undegraded":"2024-04-17T06:00:14.723697+0000","last_fullsized":"2024-04-17T06:00:14.723697+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:11:57.171410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.726118+0000","last_change":"2024-04-17T05:59:14.589670+0000","last_active":"2024-04-17T06:00:14.726118+0000","last_peered":"2024-04-17T06:00:14.726118+0000","last_clean":"2024-04-17T06:00:14.726118+0000","last_became_active":"2024-04-17T05:59:14.589003+0000","last_became_peered":"2024-04-17T05:59:14.589003+0000","last_unstale":"2024-04-17T06:00:14.726118+0000","last_undegraded":"2024-04-17T06:00:14.726118+0000","last_fullsized":"2024-04-17T06:00:14.726118+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:21:19.574178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:15.001527+0000","last_change":"2024-04-17T05:59:14.594107+0000","last_active":"2024-04-17T06:00:15.001527+0000","last_peered":"2024-04-17T06:00:15.001527+0000","last_clean":"2024-04-17T06:00:15.001527+0000","last_became_active":"2024-04-17T05:59:14.593799+0000","last_became_peered":"2024-04-17T05:59:14.593799+0000","last_unstale":"2024-04-17T06:00:15.001527+0000","last_undegraded":"2024-04-17T06:00:15.001527+0000","last_fullsized":"2024-04-17T06:00:15.001527+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:23:45.222667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.728363+0000","last_change":"2024-04-17T05:59:14.587671+0000","last_active":"2024-04-17T06:00:14.728363+0000","last_peered":"2024-04-17T06:00:14.728363+0000","last_clean":"2024-04-17T06:00:14.728363+0000","last_became_active":"2024-04-17T05:59:14.586473+0000","last_became_peered":"2024-04-17T05:59:14.586473+0000","last_unstale":"2024-04-17T06:00:14.728363+0000","last_undegraded":"2024-04-17T06:00:14.728363+0000","last_fullsized":"2024-04-17T06:00:14.728363+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:31:42.156354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:15.001635+0000","last_change":"2024-04-17T05:59:14.586961+0000","last_active":"2024-04-17T06:00:15.001635+0000","last_peered":"2024-04-17T06:00:15.001635+0000","last_clean":"2024-04-17T06:00:15.001635+0000","last_became_active":"2024-04-17T05:59:14.586601+0000","last_became_peered":"2024-04-17T05:59:14.586601+0000","last_unstale":"2024-04-17T06:00:15.001635+0000","last_undegraded":"2024-04-17T06:00:15.001635+0000","last_fullsized":"2024-04-17T06:00:15.001635+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:21:58.851523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.727691+0000","last_change":"2024-04-17T05:59:14.601239+0000","last_active":"2024-04-17T06:00:14.727691+0000","last_peered":"2024-04-17T06:00:14.727691+0000","last_clean":"2024-04-17T06:00:14.727691+0000","last_became_active":"2024-04-17T05:59:14.600852+0000","last_became_peered":"2024-04-17T05:59:14.600852+0000","last_unstale":"2024-04-17T06:00:14.727691+0000","last_undegraded":"2024-04-17T06:00:14.727691+0000","last_fullsized":"2024-04-17T06:00:14.727691+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:14:02.112456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.727704+0000","last_change":"2024-04-17T05:59:14.579952+0000","last_active":"2024-04-17T06:00:14.727704+0000","last_peered":"2024-04-17T06:00:14.727704+0000","last_clean":"2024-04-17T06:00:14.727704+0000","last_became_active":"2024-04-17T05:59:14.579625+0000","last_became_peered":"2024-04-17T05:59:14.579625+0000","last_unstale":"2024-04-17T06:00:14.727704+0000","last_undegraded":"2024-04-17T06:00:14.727704+0000","last_fullsized":"2024-04-17T06:00:14.727704+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:22:32.709762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-17T06:00:14.723643+0000","last_change":"2024-04-17T05:59:14.585187+0000","last_active":"2024-04-17T06:00:14.723643+0000","last_peered":"2024-04-17T06:00:14.723643+0000","last_clean":"2024-04-17T06:00:14.723643+0000","last_became_active":"2024-04-17T05:59:14.584820+0000","last_became_peered":"2024-04-17T05:59:14.584820+0000","last_unstale":"2024-04-17T06:00:14.723643+0000","last_undegraded":"2024-04-17T06:00:14.723643+0000","last_fullsized":"2024-04-17T06:00:14.723643+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:59:13.542518+0000","last_clean_scrub_stamp":"2024-04-17T05:59:13.542518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:53:58.945885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":249153,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":63857,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":462,"num_read_kb":298,"num_write":156,"num_write_kb":107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":59577,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":364544,"data_stored":191571,"data_compressed":13327,"data_compressed_allocated":20480,"data_compressed_original":73728,"omap_allocated":0,"internal_metadata":0},"log_size":135,"ondisk_log_size":135,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351006,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":958464,"data_stored":459384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612643,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32256,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712896,"statfs":{"total":95995035648,"available":95962005504,"internally_reserved":0,"allocated":1097728,"data_stored":1110609,"data_compressed":37928,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776168,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36284,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708868,"statfs":{"total":95995035648,"available":95957880832,"internally_reserved":0,"allocated":966656,"data_stored":459508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":36174273},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037805,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":958464,"data_stored":459316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299440,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1015808,"data_stored":1051749,"data_compressed":36061,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462967,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":954368,"data_stored":458897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574928,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1011712,"data_stored":518155,"data_compressed":1867,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738453,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36296,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708856,"statfs":{"total":95995035648,"available":95957868544,"internally_reserved":0,"allocated":974848,"data_stored":460168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411403,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":970752,"data_stored":460431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149766,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":970752,"data_stored":459208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888129,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":983040,"data_stored":519190,"data_compressed":9593,"data_compressed_allocated":12288,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724604,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1028096,"data_stored":1051042,"data_compressed":36061,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34424,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34424,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36061,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36061,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36061,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36061,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36061,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":59465,"data_compressed":1867,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":60500,"data_compressed":9593,"data_compressed_allocated":12288,"data_compressed_original":57344,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":61434,"data_compressed":1867,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T06:00:37.314 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-17T06:00:37.314 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-17T06:00:37.314 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-04-17T06:00:37.314 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 565db2d8-fc7e-11ee-bc91-c7b262605968 -- ceph health --format=json 2024-04-17T06:00:37.567 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T06:00:39.027 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:00:38 smithi047 ceph-mon[32223]: pgmap v330: 97 pgs: 97 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-17T06:00:39.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:00:38 smithi098 ceph-mon[36516]: pgmap v330: 97 pgs: 97 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-17T06:00:39.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:39.285+0000 7f56ab480700 1 -- 172.21.15.47:0/2443151808 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56a4101d30 msgr2=0x7f56a41021b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:39.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:39.285+0000 7f56ab480700 1 --2- 172.21.15.47:0/2443151808 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56a4101d30 0x7f56a41021b0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f5694009990 tx=0x7f5694009ca0 comp rx=0 tx=0).stop 2024-04-17T06:00:39.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:39.285+0000 7f56ab480700 1 -- 172.21.15.47:0/2443151808 shutdown_connections 2024-04-17T06:00:39.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:39.285+0000 7f56ab480700 1 --2- 172.21.15.47:0/2443151808 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f56a41026f0 0x7f56a410e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:39.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:39.285+0000 7f56ab480700 1 --2- 172.21.15.47:0/2443151808 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f56a41069a0 0x7f56a4106d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:39.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:39.285+0000 7f56ab480700 1 --2- 172.21.15.47:0/2443151808 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56a4101d30 0x7f56a41021b0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:39.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:39.285+0000 7f56ab480700 1 -- 172.21.15.47:0/2443151808 >> 172.21.15.47:0/2443151808 conn(0x7f56a4078900 msgr2=0x7f56a4078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:00:39.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:39.285+0000 7f56ab480700 1 -- 172.21.15.47:0/2443151808 shutdown_connections 2024-04-17T06:00:39.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:39.285+0000 7f56ab480700 1 -- 172.21.15.47:0/2443151808 wait complete. 2024-04-17T06:00:39.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:39.285+0000 7f56ab480700 1 Processor -- start 2024-04-17T06:00:39.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:39.286+0000 7f56ab480700 1 -- start start 2024-04-17T06:00:39.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:39.286+0000 7f56ab480700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56a4101d30 0x7f56a419b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:39.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:39.286+0000 7f56ab480700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f56a41026f0 0x7f56a419b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:39.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:39.286+0000 7f56ab480700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f56a41069a0 0x7f56a419fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:39.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:39.286+0000 7f56ab480700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f56a4111c50 con 0x7f56a4101d30 2024-04-17T06:00:39.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:39.286+0000 7f56ab480700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f56a4111ad0 con 0x7f56a41069a0 2024-04-17T06:00:39.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:39.286+0000 7f56ab480700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f56a4111dd0 con 0x7f56a41026f0 2024-04-17T06:00:39.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:39.286+0000 7f56a921c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56a4101d30 0x7f56a419b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:39.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:39.286+0000 7f56a921c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56a4101d30 0x7f56a419b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57266/0 (socket says 172.21.15.47:57266) 2024-04-17T06:00:39.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:39.286+0000 7f56a921c700 1 -- 172.21.15.47:0/4220949555 learned_addr learned my addr 172.21.15.47:0/4220949555 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:00:39.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:39.286+0000 7f56a8a1b700 1 --2- 172.21.15.47:0/4220949555 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f56a41026f0 0x7f56a419b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:39.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:39.286+0000 7f56a921c700 1 -- 172.21.15.47:0/4220949555 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f56a41026f0 msgr2=0x7f56a419b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:39.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:39.286+0000 7f56a921c700 1 --2- 172.21.15.47:0/4220949555 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f56a41026f0 0x7f56a419b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:39.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:39.286+0000 7f56a921c700 1 -- 172.21.15.47:0/4220949555 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f56a41069a0 msgr2=0x7f56a419fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:39.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:39.286+0000 7f56a921c700 1 --2- 172.21.15.47:0/4220949555 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f56a41069a0 0x7f56a419fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:39.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:39.286+0000 7f56a921c700 1 -- 172.21.15.47:0/4220949555 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56a41a0350 con 0x7f56a4101d30 2024-04-17T06:00:39.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:39.287+0000 7f56a921c700 1 --2- 172.21.15.47:0/4220949555 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56a4101d30 0x7f56a419b340 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f56a000c920 tx=0x7f56a000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:00:39.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:39.287+0000 7f569a7fc700 1 -- 172.21.15.47:0/4220949555 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56a0007b00 con 0x7f56a4101d30 2024-04-17T06:00:39.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:39.287+0000 7f56ab480700 1 -- 172.21.15.47:0/4220949555 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56a41a0640 con 0x7f56a4101d30 2024-04-17T06:00:39.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:39.287+0000 7f569a7fc700 1 -- 172.21.15.47:0/4220949555 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56a0004d10 con 0x7f56a4101d30 2024-04-17T06:00:39.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:39.287+0000 7f569a7fc700 1 -- 172.21.15.47:0/4220949555 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56a0005630 con 0x7f56a4101d30 2024-04-17T06:00:39.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:39.287+0000 7f56ab480700 1 -- 172.21.15.47:0/4220949555 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56a41a7ec0 con 0x7f56a4101d30 2024-04-17T06:00:39.288 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:00:38 smithi103 ceph-mon[36243]: pgmap v330: 97 pgs: 97 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-17T06:00:39.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:39.288+0000 7f56ab480700 1 -- 172.21.15.47:0/4220949555 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56a4103090 con 0x7f56a4101d30 2024-04-17T06:00:39.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:39.289+0000 7f569a7fc700 1 -- 172.21.15.47:0/4220949555 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f56a0004750 con 0x7f56a4101d30 2024-04-17T06:00:39.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:39.290+0000 7f569a7fc700 1 --2- 172.21.15.47:0/4220949555 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f56900a3cd0 0x7f56900a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:39.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:39.290+0000 7f56a8a1b700 1 --2- 172.21.15.47:0/4220949555 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f56900a3cd0 0x7f56900a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:39.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:39.290+0000 7f569a7fc700 1 -- 172.21.15.47:0/4220949555 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f56a00d0460 con 0x7f56a4101d30 2024-04-17T06:00:39.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:39.290+0000 7f56a8a1b700 1 --2- 172.21.15.47:0/4220949555 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f56900a3cd0 0x7f56900a6190 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f569400b5c0 tx=0x7f5694005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:00:39.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:39.295+0000 7f569a7fc700 1 -- 172.21.15.47:0/4220949555 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f56a0014030 con 0x7f56a4101d30 2024-04-17T06:00:39.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:39.551+0000 7f56ab480700 1 -- 172.21.15.47:0/4220949555 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f56a4069740 con 0x7f56a4101d30 2024-04-17T06:00:39.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:39.551+0000 7f569a7fc700 1 -- 172.21.15.47:0/4220949555 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f56a009ccf0 con 0x7f56a4101d30 2024-04-17T06:00:39.550 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-17T06:00:39.550 INFO:teuthology.orchestra.run.smithi047.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-17T06:00:39.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:39.553+0000 7f56ab480700 1 -- 172.21.15.47:0/4220949555 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f56900a3cd0 msgr2=0x7f56900a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:39.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:39.553+0000 7f56ab480700 1 --2- 172.21.15.47:0/4220949555 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f56900a3cd0 0x7f56900a6190 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f569400b5c0 tx=0x7f5694005960 comp rx=0 tx=0).stop 2024-04-17T06:00:39.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:39.553+0000 7f56ab480700 1 -- 172.21.15.47:0/4220949555 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56a4101d30 msgr2=0x7f56a419b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:39.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:39.553+0000 7f56ab480700 1 --2- 172.21.15.47:0/4220949555 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56a4101d30 0x7f56a419b340 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f56a000c920 tx=0x7f56a000cc30 comp rx=0 tx=0).stop 2024-04-17T06:00:39.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:39.554+0000 7f56ab480700 1 -- 172.21.15.47:0/4220949555 shutdown_connections 2024-04-17T06:00:39.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:39.554+0000 7f56ab480700 1 --2- 172.21.15.47:0/4220949555 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f56900a3cd0 0x7f56900a6190 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:39.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:39.554+0000 7f56ab480700 1 --2- 172.21.15.47:0/4220949555 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f56a41069a0 0x7f56a419fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:39.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:39.554+0000 7f56ab480700 1 --2- 172.21.15.47:0/4220949555 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56a4101d30 0x7f56a419b340 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:39.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:39.554+0000 7f56ab480700 1 --2- 172.21.15.47:0/4220949555 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f56a41026f0 0x7f56a419b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:39.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:39.554+0000 7f56ab480700 1 -- 172.21.15.47:0/4220949555 >> 172.21.15.47:0/4220949555 conn(0x7f56a4078900 msgr2=0x7f56a406b120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:00:39.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:39.554+0000 7f56ab480700 1 -- 172.21.15.47:0/4220949555 shutdown_connections 2024-04-17T06:00:39.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:39.554+0000 7f56ab480700 1 -- 172.21.15.47:0/4220949555 wait complete. 2024-04-17T06:00:40.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:00:39 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/4220949555' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-17T06:00:40.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:00:39 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/4220949555' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-17T06:00:40.270 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-04-17T06:00:40.270 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T06:00:40.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:00:39 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/4220949555' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-17T06:00:40.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:40.381+0000 7f24b3a7e640 1 -- 172.21.15.47:0/3407692269 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f24ac1029b0 msgr2=0x7f24ac1092d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:40.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:40.381+0000 7f24b3a7e640 1 --2- 172.21.15.47:0/3407692269 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f24ac1029b0 0x7f24ac1092d0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f24a4009ca0 tx=0x7f24a402c0f0 comp rx=0 tx=0).stop 2024-04-17T06:00:40.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:40.381+0000 7f24b3a7e640 1 -- 172.21.15.47:0/3407692269 shutdown_connections 2024-04-17T06:00:40.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:40.381+0000 7f24b3a7e640 1 --2- 172.21.15.47:0/3407692269 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f24ac1029b0 0x7f24ac1092d0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:40.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:40.381+0000 7f24b3a7e640 1 --2- 172.21.15.47:0/3407692269 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f24ac101ff0 0x7f24ac102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:40.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:40.381+0000 7f24b3a7e640 1 --2- 172.21.15.47:0/3407692269 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f24ac100ea0 0x7f24ac1012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:40.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:40.381+0000 7f24b3a7e640 1 -- 172.21.15.47:0/3407692269 >> 172.21.15.47:0/3407692269 conn(0x7f24ac0fc410 msgr2=0x7f24ac0fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:00:40.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:40.381+0000 7f24b3a7e640 1 -- 172.21.15.47:0/3407692269 shutdown_connections 2024-04-17T06:00:40.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:40.381+0000 7f24b3a7e640 1 -- 172.21.15.47:0/3407692269 wait complete. 2024-04-17T06:00:40.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:40.382+0000 7f24b3a7e640 1 Processor -- start 2024-04-17T06:00:40.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:40.382+0000 7f24b3a7e640 1 -- start start 2024-04-17T06:00:40.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:40.382+0000 7f24b3a7e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f24ac100ea0 0x7f24ac1971a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:40.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:40.382+0000 7f24b3a7e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f24ac101ff0 0x7f24ac1976e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:40.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:40.382+0000 7f24b3a7e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f24ac1029b0 0x7f24ac19e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:40.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:40.382+0000 7f24b3a7e640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f24ac10cbc0 con 0x7f24ac100ea0 2024-04-17T06:00:40.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:40.382+0000 7f24b3a7e640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f24ac10ca40 con 0x7f24ac1029b0 2024-04-17T06:00:40.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:40.382+0000 7f24b3a7e640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f24ac10cd40 con 0x7f24ac101ff0 2024-04-17T06:00:40.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:40.382+0000 7f24b181a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f24ac100ea0 0x7f24ac1971a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:40.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:40.382+0000 7f24b181a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f24ac100ea0 0x7f24ac1971a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57290/0 (socket says 172.21.15.47:57290) 2024-04-17T06:00:40.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:40.382+0000 7f24b181a640 1 -- 172.21.15.47:0/2750484881 learned_addr learned my addr 172.21.15.47:0/2750484881 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:00:40.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:40.383+0000 7f24b201b640 1 --2- 172.21.15.47:0/2750484881 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f24ac1029b0 0x7f24ac19e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:40.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:40.383+0000 7f24b1019640 1 --2- 172.21.15.47:0/2750484881 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f24ac101ff0 0x7f24ac1976e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:40.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:40.383+0000 7f24b181a640 1 -- 172.21.15.47:0/2750484881 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f24ac101ff0 msgr2=0x7f24ac1976e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:40.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:40.383+0000 7f24b181a640 1 --2- 172.21.15.47:0/2750484881 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f24ac101ff0 0x7f24ac1976e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:40.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:40.383+0000 7f24b181a640 1 -- 172.21.15.47:0/2750484881 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f24ac1029b0 msgr2=0x7f24ac19e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:40.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:40.383+0000 7f24b181a640 1 --2- 172.21.15.47:0/2750484881 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f24ac1029b0 0x7f24ac19e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:40.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:40.383+0000 7f24b181a640 1 -- 172.21.15.47:0/2750484881 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24ac19edc0 con 0x7f24ac100ea0 2024-04-17T06:00:40.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:40.383+0000 7f24b201b640 1 --2- 172.21.15.47:0/2750484881 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f24ac1029b0 0x7f24ac19e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:00:40.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:40.383+0000 7f24b181a640 1 --2- 172.21.15.47:0/2750484881 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f24ac100ea0 0x7f24ac1971a0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f24a800d950 tx=0x7f24a800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:00:40.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:40.383+0000 7f249effd640 1 -- 172.21.15.47:0/2750484881 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a8017070 con 0x7f24ac100ea0 2024-04-17T06:00:40.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:40.383+0000 7f249effd640 1 -- 172.21.15.47:0/2750484881 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24a8009c30 con 0x7f24ac100ea0 2024-04-17T06:00:40.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:40.383+0000 7f24b3a7e640 1 -- 172.21.15.47:0/2750484881 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24ac19f0b0 con 0x7f24ac100ea0 2024-04-17T06:00:40.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:40.383+0000 7f249effd640 1 -- 172.21.15.47:0/2750484881 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a800f6b0 con 0x7f24ac100ea0 2024-04-17T06:00:40.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:40.383+0000 7f24b3a7e640 1 -- 172.21.15.47:0/2750484881 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24ac19f600 con 0x7f24ac100ea0 2024-04-17T06:00:40.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:40.384+0000 7f24b3a7e640 1 -- 172.21.15.47:0/2750484881 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24ac108150 con 0x7f24ac100ea0 2024-04-17T06:00:40.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:40.386+0000 7f249effd640 1 -- 172.21.15.47:0/2750484881 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f24a80109d0 con 0x7f24ac100ea0 2024-04-17T06:00:40.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:40.387+0000 7f249effd640 1 --2- 172.21.15.47:0/2750484881 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f24980a3c40 0x7f24980a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:40.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:40.387+0000 7f249effd640 1 -- 172.21.15.47:0/2750484881 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f24a80d0300 con 0x7f24ac100ea0 2024-04-17T06:00:40.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:40.387+0000 7f24b1019640 1 --2- 172.21.15.47:0/2750484881 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f24980a3c40 0x7f24980a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:40.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:40.387+0000 7f24b1019640 1 --2- 172.21.15.47:0/2750484881 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f24980a3c40 0x7f24980a6100 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f24ac1986b0 tx=0x7f24a0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:00:40.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:40.390+0000 7f249effd640 1 -- 172.21.15.47:0/2750484881 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f24a8009de0 con 0x7f24ac100ea0 2024-04-17T06:00:40.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:40.578+0000 7f24b3a7e640 1 -- 172.21.15.47:0/2750484881 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f24ac062b90 con 0x7f24ac100ea0 2024-04-17T06:00:40.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:40.580+0000 7f249effd640 1 -- 172.21.15.47:0/2750484881 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22602 (secure 0 0 0) 0x7f24a809cb30 con 0x7f24ac100ea0 2024-04-17T06:00:40.579 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-17T06:00:40.579 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":88,"fsid":"565db2d8-fc7e-11ee-bc91-c7b262605968","created":"2024-04-17T05:51:19.888865+0000","modified":"2024-04-17T06:00:14.720609+0000","last_up_change":"2024-04-17T05:58:24.380131+0000","last_in_change":"2024-04-17T05:58:04.418236+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T05:54:37.734306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T05:59:12.458010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T05:59:12.901339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8453f300-533d-4736-a343-55619c08e966","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":1290127544},{"type":"v1","addr":"172.21.15.47:6803","nonce":1290127544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":1290127544},{"type":"v1","addr":"172.21.15.47:6805","nonce":1290127544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":1290127544},{"type":"v1","addr":"172.21.15.47:6809","nonce":1290127544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":1290127544},{"type":"v1","addr":"172.21.15.47:6807","nonce":1290127544}]},"public_addr":"172.21.15.47:6803/1290127544","cluster_addr":"172.21.15.47:6805/1290127544","heartbeat_back_addr":"172.21.15.47:6809/1290127544","heartbeat_front_addr":"172.21.15.47:6807/1290127544","state":["exists","up"]},{"osd":1,"uuid":"80b0f03e-c1bc-4658-8028-1e6b8eedcc47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":1180430954},{"type":"v1","addr":"172.21.15.98:6801","nonce":1180430954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":1180430954},{"type":"v1","addr":"172.21.15.98:6803","nonce":1180430954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":1180430954},{"type":"v1","addr":"172.21.15.98:6807","nonce":1180430954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":1180430954},{"type":"v1","addr":"172.21.15.98:6805","nonce":1180430954}]},"public_addr":"172.21.15.98:6801/1180430954","cluster_addr":"172.21.15.98:6803/1180430954","heartbeat_back_addr":"172.21.15.98:6807/1180430954","heartbeat_front_addr":"172.21.15.98:6805/1180430954","state":["exists","up"]},{"osd":2,"uuid":"0fc9d383-05b4-498d-b530-fccf7aeddd7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":494676216},{"type":"v1","addr":"172.21.15.103:6801","nonce":494676216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":494676216},{"type":"v1","addr":"172.21.15.103:6803","nonce":494676216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":494676216},{"type":"v1","addr":"172.21.15.103:6807","nonce":494676216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":494676216},{"type":"v1","addr":"172.21.15.103:6805","nonce":494676216}]},"public_addr":"172.21.15.103:6801/494676216","cluster_addr":"172.21.15.103:6803/494676216","heartbeat_back_addr":"172.21.15.103:6807/494676216","heartbeat_front_addr":"172.21.15.103:6805/494676216","state":["exists","up"]},{"osd":3,"uuid":"f6e0b0d5-0fb3-42d0-9e84-f7cfef21a998","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":1998369447},{"type":"v1","addr":"172.21.15.47:6811","nonce":1998369447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":1998369447},{"type":"v1","addr":"172.21.15.47:6813","nonce":1998369447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":1998369447},{"type":"v1","addr":"172.21.15.47:6817","nonce":1998369447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":1998369447},{"type":"v1","addr":"172.21.15.47:6815","nonce":1998369447}]},"public_addr":"172.21.15.47:6811/1998369447","cluster_addr":"172.21.15.47:6813/1998369447","heartbeat_back_addr":"172.21.15.47:6817/1998369447","heartbeat_front_addr":"172.21.15.47:6815/1998369447","state":["exists","up"]},{"osd":4,"uuid":"2c26217e-02df-466d-bc01-0149d4aec3c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":3060482485},{"type":"v1","addr":"172.21.15.98:6809","nonce":3060482485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":3060482485},{"type":"v1","addr":"172.21.15.98:6811","nonce":3060482485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":3060482485},{"type":"v1","addr":"172.21.15.98:6815","nonce":3060482485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":3060482485},{"type":"v1","addr":"172.21.15.98:6813","nonce":3060482485}]},"public_addr":"172.21.15.98:6809/3060482485","cluster_addr":"172.21.15.98:6811/3060482485","heartbeat_back_addr":"172.21.15.98:6815/3060482485","heartbeat_front_addr":"172.21.15.98:6813/3060482485","state":["exists","up"]},{"osd":5,"uuid":"740bf3e0-bea8-496b-8d6f-e42d0b017258","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":4181363567},{"type":"v1","addr":"172.21.15.103:6809","nonce":4181363567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":4181363567},{"type":"v1","addr":"172.21.15.103:6811","nonce":4181363567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":4181363567},{"type":"v1","addr":"172.21.15.103:6815","nonce":4181363567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":4181363567},{"type":"v1","addr":"172.21.15.103:6813","nonce":4181363567}]},"public_addr":"172.21.15.103:6809/4181363567","cluster_addr":"172.21.15.103:6811/4181363567","heartbeat_back_addr":"172.21.15.103:6815/4181363567","heartbeat_front_addr":"172.21.15.103:6813/4181363567","state":["exists","up"]},{"osd":6,"uuid":"356a6255-2c07-4f0d-a233-26be04c92b38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":2517529668},{"type":"v1","addr":"172.21.15.47:6819","nonce":2517529668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2517529668},{"type":"v1","addr":"172.21.15.47:6821","nonce":2517529668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2517529668},{"type":"v1","addr":"172.21.15.47:6825","nonce":2517529668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2517529668},{"type":"v1","addr":"172.21.15.47:6823","nonce":2517529668}]},"public_addr":"172.21.15.47:6819/2517529668","cluster_addr":"172.21.15.47:6821/2517529668","heartbeat_back_addr":"172.21.15.47:6825/2517529668","heartbeat_front_addr":"172.21.15.47:6823/2517529668","state":["exists","up"]},{"osd":7,"uuid":"349d0120-8290-417d-a800-5a8985f554f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":4225498496},{"type":"v1","addr":"172.21.15.98:6817","nonce":4225498496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":4225498496},{"type":"v1","addr":"172.21.15.98:6819","nonce":4225498496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":4225498496},{"type":"v1","addr":"172.21.15.98:6823","nonce":4225498496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":4225498496},{"type":"v1","addr":"172.21.15.98:6821","nonce":4225498496}]},"public_addr":"172.21.15.98:6817/4225498496","cluster_addr":"172.21.15.98:6819/4225498496","heartbeat_back_addr":"172.21.15.98:6823/4225498496","heartbeat_front_addr":"172.21.15.98:6821/4225498496","state":["exists","up"]},{"osd":8,"uuid":"cee4712f-321b-42c0-92ba-e15144c00f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":3433905520},{"type":"v1","addr":"172.21.15.103:6817","nonce":3433905520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":3433905520},{"type":"v1","addr":"172.21.15.103:6819","nonce":3433905520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":3433905520},{"type":"v1","addr":"172.21.15.103:6823","nonce":3433905520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":3433905520},{"type":"v1","addr":"172.21.15.103:6821","nonce":3433905520}]},"public_addr":"172.21.15.103:6817/3433905520","cluster_addr":"172.21.15.103:6819/3433905520","heartbeat_back_addr":"172.21.15.103:6823/3433905520","heartbeat_front_addr":"172.21.15.103:6821/3433905520","state":["exists","up"]},{"osd":9,"uuid":"63fc73c1-5a22-487d-80b8-9894e2ddd207","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":728359554},{"type":"v1","addr":"172.21.15.47:6827","nonce":728359554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":728359554},{"type":"v1","addr":"172.21.15.47:6829","nonce":728359554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":728359554},{"type":"v1","addr":"172.21.15.47:6833","nonce":728359554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":728359554},{"type":"v1","addr":"172.21.15.47:6831","nonce":728359554}]},"public_addr":"172.21.15.47:6827/728359554","cluster_addr":"172.21.15.47:6829/728359554","heartbeat_back_addr":"172.21.15.47:6833/728359554","heartbeat_front_addr":"172.21.15.47:6831/728359554","state":["exists","up"]},{"osd":10,"uuid":"9565bcc3-62ee-48eb-a07a-03a81e0cc8e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":468205689},{"type":"v1","addr":"172.21.15.98:6825","nonce":468205689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":468205689},{"type":"v1","addr":"172.21.15.98:6827","nonce":468205689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":468205689},{"type":"v1","addr":"172.21.15.98:6831","nonce":468205689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":468205689},{"type":"v1","addr":"172.21.15.98:6829","nonce":468205689}]},"public_addr":"172.21.15.98:6825/468205689","cluster_addr":"172.21.15.98:6827/468205689","heartbeat_back_addr":"172.21.15.98:6831/468205689","heartbeat_front_addr":"172.21.15.98:6829/468205689","state":["exists","up"]},{"osd":11,"uuid":"2e85e8d9-bf6c-4a12-b124-a4690dd61108","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":2639831013},{"type":"v1","addr":"172.21.15.103:6825","nonce":2639831013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":2639831013},{"type":"v1","addr":"172.21.15.103:6827","nonce":2639831013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":2639831013},{"type":"v1","addr":"172.21.15.103:6831","nonce":2639831013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":2639831013},{"type":"v1","addr":"172.21.15.103:6829","nonce":2639831013}]},"public_addr":"172.21.15.103:6825/2639831013","cluster_addr":"172.21.15.103:6827/2639831013","heartbeat_back_addr":"172.21.15.103:6831/2639831013","heartbeat_front_addr":"172.21.15.103:6829/2639831013","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:53:47.366438+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:54:10.953559+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:54:34.320332+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:54:59.751360+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:55:23.982052+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:55:48.425017+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:56:14.793368+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:56:40.440830+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:57:04.623680+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:57:31.215262+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:57:57.040346+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:58:22.671572+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:6801/3632157175":"2024-04-18T05:52:13.645775+0000","172.21.15.47:0/4288219269":"2024-04-18T05:52:13.645775+0000","172.21.15.47:0/2584487544":"2024-04-18T05:52:13.645775+0000","172.21.15.47:6801/953064786":"2024-04-18T05:51:45.979351+0000","172.21.15.47:0/3484500927":"2024-04-18T05:51:45.979351+0000","172.21.15.47:6800/3632157175":"2024-04-18T05:52:13.645775+0000","172.21.15.47:0/112268670":"2024-04-18T05:51:45.979351+0000","172.21.15.47:0/1638423256":"2024-04-18T05:51:45.979351+0000","172.21.15.47:6800/953064786":"2024-04-18T05:51:45.979351+0000","172.21.15.47:0/849479010":"2024-04-18T05:52:13.645775+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T06:00:40.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:40.582+0000 7f24b3a7e640 1 -- 172.21.15.47:0/2750484881 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f24980a3c40 msgr2=0x7f24980a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:40.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:40.582+0000 7f24b3a7e640 1 --2- 172.21.15.47:0/2750484881 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f24980a3c40 0x7f24980a6100 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f24ac1986b0 tx=0x7f24a0008040 comp rx=0 tx=0).stop 2024-04-17T06:00:40.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:40.582+0000 7f24b3a7e640 1 -- 172.21.15.47:0/2750484881 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f24ac100ea0 msgr2=0x7f24ac1971a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:40.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:40.582+0000 7f24b3a7e640 1 --2- 172.21.15.47:0/2750484881 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f24ac100ea0 0x7f24ac1971a0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f24a800d950 tx=0x7f24a800de10 comp rx=0 tx=0).stop 2024-04-17T06:00:40.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:40.582+0000 7f24b3a7e640 1 -- 172.21.15.47:0/2750484881 shutdown_connections 2024-04-17T06:00:40.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:40.582+0000 7f24b3a7e640 1 --2- 172.21.15.47:0/2750484881 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f24980a3c40 0x7f24980a6100 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:40.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:40.582+0000 7f24b3a7e640 1 --2- 172.21.15.47:0/2750484881 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f24ac1029b0 0x7f24ac19e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:40.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:40.582+0000 7f24b3a7e640 1 --2- 172.21.15.47:0/2750484881 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f24ac101ff0 0x7f24ac1976e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:40.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:40.582+0000 7f24b3a7e640 1 --2- 172.21.15.47:0/2750484881 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f24ac100ea0 0x7f24ac1971a0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:40.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:40.582+0000 7f24b3a7e640 1 -- 172.21.15.47:0/2750484881 >> 172.21.15.47:0/2750484881 conn(0x7f24ac0fc410 msgr2=0x7f24ac105320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:00:40.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:40.582+0000 7f24b3a7e640 1 -- 172.21.15.47:0/2750484881 shutdown_connections 2024-04-17T06:00:40.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:40.582+0000 7f24b3a7e640 1 -- 172.21.15.47:0/2750484881 wait complete. 2024-04-17T06:00:40.593 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T05:54:37.734306+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T05:59:12.458010+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T05:59:12.901339+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T06:00:40.593 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T06:00:40.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:40.703+0000 7f0c35c77640 1 -- 172.21.15.47:0/490642628 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c30100ea0 msgr2=0x7f0c301012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:40.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:40.703+0000 7f0c35c77640 1 --2- 172.21.15.47:0/490642628 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c30100ea0 0x7f0c301012c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f0c20009900 tx=0x7f0c2002bc30 comp rx=0 tx=0).stop 2024-04-17T06:00:40.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:40.703+0000 7f0c35c77640 1 -- 172.21.15.47:0/490642628 shutdown_connections 2024-04-17T06:00:40.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:40.703+0000 7f0c35c77640 1 --2- 172.21.15.47:0/490642628 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0c301029b0 0x7f0c301092d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:40.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:40.703+0000 7f0c35c77640 1 --2- 172.21.15.47:0/490642628 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0c30101ff0 0x7f0c30102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:40.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:40.703+0000 7f0c35c77640 1 --2- 172.21.15.47:0/490642628 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c30100ea0 0x7f0c301012c0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:40.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:40.703+0000 7f0c35c77640 1 -- 172.21.15.47:0/490642628 >> 172.21.15.47:0/490642628 conn(0x7f0c300fc410 msgr2=0x7f0c300fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:00:40.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:40.703+0000 7f0c35c77640 1 -- 172.21.15.47:0/490642628 shutdown_connections 2024-04-17T06:00:40.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:40.703+0000 7f0c35c77640 1 -- 172.21.15.47:0/490642628 wait complete. 2024-04-17T06:00:40.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:40.704+0000 7f0c35c77640 1 Processor -- start 2024-04-17T06:00:40.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:40.704+0000 7f0c35c77640 1 -- start start 2024-04-17T06:00:40.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:40.704+0000 7f0c35c77640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c30100ea0 0x7f0c30198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:40.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:40.704+0000 7f0c35c77640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0c30101ff0 0x7f0c30198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:40.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:40.704+0000 7f0c35c77640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0c301029b0 0x7f0c3019f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:40.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:40.704+0000 7f0c35c77640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f0c3010cb20 con 0x7f0c30100ea0 2024-04-17T06:00:40.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:40.704+0000 7f0c35c77640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0c3010c9a0 con 0x7f0c30101ff0 2024-04-17T06:00:40.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:40.704+0000 7f0c35c77640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0c3010cca0 con 0x7f0c301029b0 2024-04-17T06:00:40.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:40.704+0000 7f0c2f7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c30100ea0 0x7f0c30198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:40.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:40.704+0000 7f0c2f7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c30100ea0 0x7f0c30198340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57308/0 (socket says 172.21.15.47:57308) 2024-04-17T06:00:40.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:40.704+0000 7f0c2f7fe640 1 -- 172.21.15.47:0/407353268 learned_addr learned my addr 172.21.15.47:0/407353268 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:00:40.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:40.704+0000 7f0c2f7fe640 1 -- 172.21.15.47:0/407353268 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0c301029b0 msgr2=0x7f0c3019f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:40.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:40.704+0000 7f0c27fff640 1 --2- 172.21.15.47:0/407353268 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0c30101ff0 0x7f0c30198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:40.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:40.704+0000 7f0c2ffff640 1 --2- 172.21.15.47:0/407353268 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0c301029b0 0x7f0c3019f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:40.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:40.704+0000 7f0c2f7fe640 1 --2- 172.21.15.47:0/407353268 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0c301029b0 0x7f0c3019f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:40.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:40.704+0000 7f0c2f7fe640 1 -- 172.21.15.47:0/407353268 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0c30101ff0 msgr2=0x7f0c30198880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:40.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:40.704+0000 7f0c2f7fe640 1 --2- 172.21.15.47:0/407353268 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0c30101ff0 0x7f0c30198880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:40.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:40.704+0000 7f0c2f7fe640 1 -- 172.21.15.47:0/407353268 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c3019ff60 con 0x7f0c30100ea0 2024-04-17T06:00:40.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:40.704+0000 7f0c2ffff640 1 --2- 172.21.15.47:0/407353268 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0c301029b0 0x7f0c3019f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:00:40.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:40.704+0000 7f0c27fff640 1 --2- 172.21.15.47:0/407353268 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0c30101ff0 0x7f0c30198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:00:40.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:40.704+0000 7f0c2f7fe640 1 --2- 172.21.15.47:0/407353268 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c30100ea0 0x7f0c30198340 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f0c20009230 tx=0x7f0c20005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:00:40.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:40.705+0000 7f0c2d7fa640 1 -- 172.21.15.47:0/407353268 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c20041070 con 0x7f0c30100ea0 2024-04-17T06:00:40.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:40.705+0000 7f0c2d7fa640 1 -- 172.21.15.47:0/407353268 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c200054e0 con 0x7f0c30100ea0 2024-04-17T06:00:40.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:40.705+0000 7f0c35c77640 1 -- 172.21.15.47:0/407353268 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c301a01f0 con 0x7f0c30100ea0 2024-04-17T06:00:40.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:40.705+0000 7f0c2d7fa640 1 -- 172.21.15.47:0/407353268 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c20034440 con 0x7f0c30100ea0 2024-04-17T06:00:40.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:40.705+0000 7f0c35c77640 1 -- 172.21.15.47:0/407353268 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c301a06e0 con 0x7f0c30100ea0 2024-04-17T06:00:40.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:40.706+0000 7f0c35c77640 1 -- 172.21.15.47:0/407353268 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c301a0a00 con 0x7f0c30100ea0 2024-04-17T06:00:40.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:40.707+0000 7f0c2d7fa640 1 -- 172.21.15.47:0/407353268 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f0c20044070 con 0x7f0c30100ea0 2024-04-17T06:00:40.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:40.708+0000 7f0c2d7fa640 1 --2- 172.21.15.47:0/407353268 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f0c1c0a3c40 0x7f0c1c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:40.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:40.708+0000 7f0c27fff640 1 --2- 172.21.15.47:0/407353268 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f0c1c0a3c40 0x7f0c1c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:40.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:40.708+0000 7f0c2d7fa640 1 -- 172.21.15.47:0/407353268 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f0c200f21f0 con 0x7f0c30100ea0 2024-04-17T06:00:40.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:40.708+0000 7f0c27fff640 1 --2- 172.21.15.47:0/407353268 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f0c1c0a3c40 0x7f0c1c0a6100 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f0c18005d00 tx=0x7f0c18005ef0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:00:40.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:40.712+0000 7f0c2d7fa640 1 -- 172.21.15.47:0/407353268 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f0c200beb90 con 0x7f0c30100ea0 2024-04-17T06:00:40.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:40.918+0000 7f0c35c77640 1 -- 172.21.15.47:0/407353268 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0c301a0bd0 con 0x7f0c30100ea0 2024-04-17T06:00:40.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:40.918+0000 7f0c2d7fa640 1 -- 172.21.15.47:0/407353268 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7f0c200be490 con 0x7f0c30100ea0 2024-04-17T06:00:40.917 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-04-17T06:00:40.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:40.920+0000 7f0c35c77640 1 -- 172.21.15.47:0/407353268 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f0c1c0a3c40 msgr2=0x7f0c1c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:40.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:40.920+0000 7f0c35c77640 1 --2- 172.21.15.47:0/407353268 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f0c1c0a3c40 0x7f0c1c0a6100 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f0c18005d00 tx=0x7f0c18005ef0 comp rx=0 tx=0).stop 2024-04-17T06:00:40.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:40.920+0000 7f0c35c77640 1 -- 172.21.15.47:0/407353268 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c30100ea0 msgr2=0x7f0c30198340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:40.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:40.920+0000 7f0c35c77640 1 --2- 172.21.15.47:0/407353268 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c30100ea0 0x7f0c30198340 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f0c20009230 tx=0x7f0c20005dd0 comp rx=0 tx=0).stop 2024-04-17T06:00:40.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:40.920+0000 7f0c35c77640 1 -- 172.21.15.47:0/407353268 shutdown_connections 2024-04-17T06:00:40.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:40.920+0000 7f0c35c77640 1 --2- 172.21.15.47:0/407353268 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f0c1c0a3c40 0x7f0c1c0a6100 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:40.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:40.920+0000 7f0c35c77640 1 --2- 172.21.15.47:0/407353268 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0c301029b0 0x7f0c3019f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:40.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:40.920+0000 7f0c35c77640 1 --2- 172.21.15.47:0/407353268 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0c30101ff0 0x7f0c30198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:40.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:40.920+0000 7f0c35c77640 1 --2- 172.21.15.47:0/407353268 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c30100ea0 0x7f0c30198340 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:40.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:40.920+0000 7f0c35c77640 1 -- 172.21.15.47:0/407353268 >> 172.21.15.47:0/407353268 conn(0x7f0c300fc410 msgr2=0x7f0c300fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:00:40.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:40.920+0000 7f0c35c77640 1 -- 172.21.15.47:0/407353268 shutdown_connections 2024-04-17T06:00:40.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:40.920+0000 7f0c35c77640 1 -- 172.21.15.47:0/407353268 wait complete. 2024-04-17T06:00:40.930 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T06:00:41.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.040+0000 7f9e2662a640 1 -- 172.21.15.47:0/4228066622 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9e20102ac0 msgr2=0x7f9e20104eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:41.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.040+0000 7f9e2662a640 1 --2- 172.21.15.47:0/4228066622 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9e20102ac0 0x7f9e20104eb0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f9e10009900 tx=0x7f9e1002bc30 comp rx=0 tx=0).stop 2024-04-17T06:00:41.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.040+0000 7f9e2662a640 1 -- 172.21.15.47:0/4228066622 shutdown_connections 2024-04-17T06:00:41.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.040+0000 7f9e2662a640 1 --2- 172.21.15.47:0/4228066622 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9e20107d20 0x7f9e2010a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:41.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.040+0000 7f9e2662a640 1 --2- 172.21.15.47:0/4228066622 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9e201053f0 0x7f9e201077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:41.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.040+0000 7f9e2662a640 1 --2- 172.21.15.47:0/4228066622 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9e20102ac0 0x7f9e20104eb0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:41.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.040+0000 7f9e2662a640 1 -- 172.21.15.47:0/4228066622 >> 172.21.15.47:0/4228066622 conn(0x7f9e200fc410 msgr2=0x7f9e200fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:00:41.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.040+0000 7f9e2662a640 1 -- 172.21.15.47:0/4228066622 shutdown_connections 2024-04-17T06:00:41.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.040+0000 7f9e2662a640 1 -- 172.21.15.47:0/4228066622 wait complete. 2024-04-17T06:00:41.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.041+0000 7f9e2662a640 1 Processor -- start 2024-04-17T06:00:41.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.041+0000 7f9e2662a640 1 -- start start 2024-04-17T06:00:41.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.041+0000 7f9e2662a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9e20102ac0 0x7f9e201983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:41.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.041+0000 7f9e2662a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9e201053f0 0x7f9e20198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:41.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.041+0000 7f9e2662a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9e20107d20 0x7f9e2019f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:41.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.041+0000 7f9e2662a640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f9e2010dd10 con 0x7f9e20107d20 2024-04-17T06:00:41.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.041+0000 7f9e2662a640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9e2010db90 con 0x7f9e201053f0 2024-04-17T06:00:41.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.041+0000 7f9e2662a640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9e2010de90 con 0x7f9e20102ac0 2024-04-17T06:00:41.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.041+0000 7f9e24bc7640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9e20107d20 0x7f9e2019f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:41.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.041+0000 7f9e24bc7640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9e20107d20 0x7f9e2019f990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57330/0 (socket says 172.21.15.47:57330) 2024-04-17T06:00:41.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.041+0000 7f9e24bc7640 1 -- 172.21.15.47:0/190461638 learned_addr learned my addr 172.21.15.47:0/190461638 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:00:41.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.041+0000 7f9e1ffff640 1 --2- 172.21.15.47:0/190461638 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9e20102ac0 0x7f9e201983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:41.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.041+0000 7f9e1f7fe640 1 --2- 172.21.15.47:0/190461638 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9e201053f0 0x7f9e20198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:41.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.042+0000 7f9e24bc7640 1 -- 172.21.15.47:0/190461638 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9e20102ac0 msgr2=0x7f9e201983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:41.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.042+0000 7f9e24bc7640 1 --2- 172.21.15.47:0/190461638 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9e20102ac0 0x7f9e201983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:41.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.042+0000 7f9e24bc7640 1 -- 172.21.15.47:0/190461638 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9e201053f0 msgr2=0x7f9e20198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:41.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.042+0000 7f9e24bc7640 1 --2- 172.21.15.47:0/190461638 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9e201053f0 0x7f9e20198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:41.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.042+0000 7f9e24bc7640 1 -- 172.21.15.47:0/190461638 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e201a0000 con 0x7f9e20107d20 2024-04-17T06:00:41.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.042+0000 7f9e24bc7640 1 --2- 172.21.15.47:0/190461638 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9e20107d20 0x7f9e2019f990 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9e1800b970 tx=0x7f9e1800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:00:41.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.042+0000 7f9e1d7fa640 1 -- 172.21.15.47:0/190461638 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e18019070 con 0x7f9e20107d20 2024-04-17T06:00:41.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.042+0000 7f9e1d7fa640 1 -- 172.21.15.47:0/190461638 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e18014440 con 0x7f9e20107d20 2024-04-17T06:00:41.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.042+0000 7f9e2662a640 1 -- 172.21.15.47:0/190461638 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e201a02f0 con 0x7f9e20107d20 2024-04-17T06:00:41.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.042+0000 7f9e1d7fa640 1 -- 172.21.15.47:0/190461638 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e180075d0 con 0x7f9e20107d20 2024-04-17T06:00:41.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.042+0000 7f9e2662a640 1 -- 172.21.15.47:0/190461638 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e201a07f0 con 0x7f9e20107d20 2024-04-17T06:00:41.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.043+0000 7f9e2662a640 1 -- 172.21.15.47:0/190461638 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e200fdf00 con 0x7f9e20107d20 2024-04-17T06:00:41.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.044+0000 7f9e1d7fa640 1 -- 172.21.15.47:0/190461638 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f9e1801c070 con 0x7f9e20107d20 2024-04-17T06:00:41.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.045+0000 7f9e1d7fa640 1 --2- 172.21.15.47:0/190461638 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f9e080a3c40 0x7f9e080a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:41.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.045+0000 7f9e1d7fa640 1 -- 172.21.15.47:0/190461638 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f9e180cf9b0 con 0x7f9e20107d20 2024-04-17T06:00:41.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.045+0000 7f9e1ffff640 1 --2- 172.21.15.47:0/190461638 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f9e080a3c40 0x7f9e080a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:41.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.045+0000 7f9e1ffff640 1 --2- 172.21.15.47:0/190461638 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f9e080a3c40 0x7f9e080a6100 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9e10005ab0 tx=0x7f9e10004f10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:00:41.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.050+0000 7f9e1d7fa640 1 -- 172.21.15.47:0/190461638 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f9e1809bd00 con 0x7f9e20107d20 2024-04-17T06:00:41.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:00:40 smithi098 ceph-mon[36516]: pgmap v331: 97 pgs: 97 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-17T06:00:41.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:00:40 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/2750484881' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T06:00:41.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:00:40 smithi047 ceph-mon[32223]: pgmap v331: 97 pgs: 97 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-17T06:00:41.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:00:40 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/2750484881' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T06:00:41.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.236+0000 7f9e2662a640 1 -- 172.21.15.47:0/190461638 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f9e20062b60 con 0x7f9e20107d20 2024-04-17T06:00:41.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.236+0000 7f9e1d7fa640 1 -- 172.21.15.47:0/190461638 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7f9e180a0000 con 0x7f9e20107d20 2024-04-17T06:00:41.235 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-04-17T06:00:41.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.238+0000 7f9e2662a640 1 -- 172.21.15.47:0/190461638 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f9e080a3c40 msgr2=0x7f9e080a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:41.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.238+0000 7f9e2662a640 1 --2- 172.21.15.47:0/190461638 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f9e080a3c40 0x7f9e080a6100 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9e10005ab0 tx=0x7f9e10004f10 comp rx=0 tx=0).stop 2024-04-17T06:00:41.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.238+0000 7f9e2662a640 1 -- 172.21.15.47:0/190461638 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9e20107d20 msgr2=0x7f9e2019f990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:41.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.238+0000 7f9e2662a640 1 --2- 172.21.15.47:0/190461638 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9e20107d20 0x7f9e2019f990 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9e1800b970 tx=0x7f9e1800bd70 comp rx=0 tx=0).stop 2024-04-17T06:00:41.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.238+0000 7f9e2662a640 1 -- 172.21.15.47:0/190461638 shutdown_connections 2024-04-17T06:00:41.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.238+0000 7f9e2662a640 1 --2- 172.21.15.47:0/190461638 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f9e080a3c40 0x7f9e080a6100 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:41.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.238+0000 7f9e2662a640 1 --2- 172.21.15.47:0/190461638 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9e20107d20 0x7f9e2019f990 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:41.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.238+0000 7f9e2662a640 1 --2- 172.21.15.47:0/190461638 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9e201053f0 0x7f9e20198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:41.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.238+0000 7f9e2662a640 1 --2- 172.21.15.47:0/190461638 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9e20102ac0 0x7f9e201983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:41.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.238+0000 7f9e2662a640 1 -- 172.21.15.47:0/190461638 >> 172.21.15.47:0/190461638 conn(0x7f9e200fc410 msgr2=0x7f9e20103540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:00:41.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.238+0000 7f9e2662a640 1 -- 172.21.15.47:0/190461638 shutdown_connections 2024-04-17T06:00:41.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.238+0000 7f9e2662a640 1 -- 172.21.15.47:0/190461638 wait complete. 2024-04-17T06:00:41.248 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T06:00:41.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:00:40 smithi103 ceph-mon[36243]: pgmap v331: 97 pgs: 97 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-17T06:00:41.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:00:40 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/2750484881' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T06:00:41.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.358+0000 7f56cd247640 1 -- 172.21.15.47:0/2721178672 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56c81031b0 msgr2=0x7f56c8103630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:41.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.358+0000 7f56cd247640 1 --2- 172.21.15.47:0/2721178672 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56c81031b0 0x7f56c8103630 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f56b0009900 tx=0x7f56b002bc30 comp rx=0 tx=0).stop 2024-04-17T06:00:41.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.358+0000 7f56cd247640 1 -- 172.21.15.47:0/2721178672 shutdown_connections 2024-04-17T06:00:41.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.358+0000 7f56cd247640 1 --2- 172.21.15.47:0/2721178672 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f56c8103b70 0x7f56c810a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:41.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.358+0000 7f56cd247640 1 --2- 172.21.15.47:0/2721178672 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56c81031b0 0x7f56c8103630 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:41.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.358+0000 7f56cd247640 1 --2- 172.21.15.47:0/2721178672 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f56c8101fc0 0x7f56c81023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:41.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.358+0000 7f56cd247640 1 -- 172.21.15.47:0/2721178672 >> 172.21.15.47:0/2721178672 conn(0x7f56c80fd530 msgr2=0x7f56c80ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:00:41.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.358+0000 7f56cd247640 1 -- 172.21.15.47:0/2721178672 shutdown_connections 2024-04-17T06:00:41.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.358+0000 7f56cd247640 1 -- 172.21.15.47:0/2721178672 wait complete. 2024-04-17T06:00:41.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.359+0000 7f56cd247640 1 Processor -- start 2024-04-17T06:00:41.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.359+0000 7f56cd247640 1 -- start start 2024-04-17T06:00:41.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.359+0000 7f56cd247640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f56c8101fc0 0x7f56c81982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:41.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.359+0000 7f56cd247640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56c81031b0 0x7f56c8198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:41.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.359+0000 7f56cd247640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f56c8103b70 0x7f56c819f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:41.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.359+0000 7f56cd247640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f56c810dc50 con 0x7f56c81031b0 2024-04-17T06:00:41.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.359+0000 7f56cd247640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f56c810dad0 con 0x7f56c8101fc0 2024-04-17T06:00:41.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.359+0000 7f56cd247640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f56c810ddd0 con 0x7f56c8103b70 2024-04-17T06:00:41.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.359+0000 7f56c659c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56c81031b0 0x7f56c8198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:41.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.359+0000 7f56c659c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56c81031b0 0x7f56c8198820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57350/0 (socket says 172.21.15.47:57350) 2024-04-17T06:00:41.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.359+0000 7f56c659c640 1 -- 172.21.15.47:0/2018627491 learned_addr learned my addr 172.21.15.47:0/2018627491 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:00:41.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.360+0000 7f56c759e640 1 --2- 172.21.15.47:0/2018627491 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f56c8103b70 0x7f56c819f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:41.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.360+0000 7f56c6d9d640 1 --2- 172.21.15.47:0/2018627491 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f56c8101fc0 0x7f56c81982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:41.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.360+0000 7f56c659c640 1 -- 172.21.15.47:0/2018627491 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f56c8103b70 msgr2=0x7f56c819f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:41.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.360+0000 7f56c659c640 1 --2- 172.21.15.47:0/2018627491 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f56c8103b70 0x7f56c819f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:41.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.360+0000 7f56c659c640 1 -- 172.21.15.47:0/2018627491 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f56c8101fc0 msgr2=0x7f56c81982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:41.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.360+0000 7f56c659c640 1 --2- 172.21.15.47:0/2018627491 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f56c8101fc0 0x7f56c81982e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:41.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.360+0000 7f56c659c640 1 -- 172.21.15.47:0/2018627491 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56c819ff00 con 0x7f56c81031b0 2024-04-17T06:00:41.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.360+0000 7f56c759e640 1 --2- 172.21.15.47:0/2018627491 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f56c8103b70 0x7f56c819f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:00:41.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.360+0000 7f56c659c640 1 --2- 172.21.15.47:0/2018627491 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56c81031b0 0x7f56c8198820 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f56b0009230 tx=0x7f56b0005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:00:41.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.360+0000 7f56bffff640 1 -- 172.21.15.47:0/2018627491 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b0041070 con 0x7f56c81031b0 2024-04-17T06:00:41.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.360+0000 7f56cd247640 1 -- 172.21.15.47:0/2018627491 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56c81a0190 con 0x7f56c81031b0 2024-04-17T06:00:41.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.360+0000 7f56bffff640 1 -- 172.21.15.47:0/2018627491 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56b00054e0 con 0x7f56c81031b0 2024-04-17T06:00:41.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.360+0000 7f56bffff640 1 -- 172.21.15.47:0/2018627491 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b0034440 con 0x7f56c81031b0 2024-04-17T06:00:41.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.360+0000 7f56cd247640 1 -- 172.21.15.47:0/2018627491 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56c81a06d0 con 0x7f56c81031b0 2024-04-17T06:00:41.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.361+0000 7f56cd247640 1 -- 172.21.15.47:0/2018627491 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56c8109330 con 0x7f56c81031b0 2024-04-17T06:00:41.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.362+0000 7f56bffff640 1 -- 172.21.15.47:0/2018627491 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f56b0044070 con 0x7f56c81031b0 2024-04-17T06:00:41.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.363+0000 7f56bffff640 1 --2- 172.21.15.47:0/2018627491 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f56b40a3c40 0x7f56b40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:41.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.363+0000 7f56c6d9d640 1 --2- 172.21.15.47:0/2018627491 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f56b40a3c40 0x7f56b40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:41.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.363+0000 7f56bffff640 1 -- 172.21.15.47:0/2018627491 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f56b00f2250 con 0x7f56c81031b0 2024-04-17T06:00:41.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.363+0000 7f56c6d9d640 1 --2- 172.21.15.47:0/2018627491 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f56b40a3c40 0x7f56b40a6100 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f56c0005c40 tx=0x7f56c0005bd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:00:41.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.368+0000 7f56bffff640 1 -- 172.21.15.47:0/2018627491 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f56b00beb90 con 0x7f56c81031b0 2024-04-17T06:00:41.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.555+0000 7f56cd247640 1 -- 172.21.15.47:0/2018627491 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f56c802d5d0 con 0x7f56c81031b0 2024-04-17T06:00:41.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.555+0000 7f56bffff640 1 -- 172.21.15.47:0/2018627491 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7f56b00be490 con 0x7f56c81031b0 2024-04-17T06:00:41.554 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-04-17T06:00:41.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.557+0000 7f56cd247640 1 -- 172.21.15.47:0/2018627491 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f56b40a3c40 msgr2=0x7f56b40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:41.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.557+0000 7f56cd247640 1 --2- 172.21.15.47:0/2018627491 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f56b40a3c40 0x7f56b40a6100 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f56c0005c40 tx=0x7f56c0005bd0 comp rx=0 tx=0).stop 2024-04-17T06:00:41.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.557+0000 7f56cd247640 1 -- 172.21.15.47:0/2018627491 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56c81031b0 msgr2=0x7f56c8198820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:41.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.557+0000 7f56cd247640 1 --2- 172.21.15.47:0/2018627491 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56c81031b0 0x7f56c8198820 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f56b0009230 tx=0x7f56b0005dd0 comp rx=0 tx=0).stop 2024-04-17T06:00:41.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.557+0000 7f56cd247640 1 -- 172.21.15.47:0/2018627491 shutdown_connections 2024-04-17T06:00:41.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.557+0000 7f56cd247640 1 --2- 172.21.15.47:0/2018627491 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f56b40a3c40 0x7f56b40a6100 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:41.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.557+0000 7f56cd247640 1 --2- 172.21.15.47:0/2018627491 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f56c8103b70 0x7f56c819f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:41.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.557+0000 7f56cd247640 1 --2- 172.21.15.47:0/2018627491 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56c81031b0 0x7f56c8198820 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:41.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.557+0000 7f56cd247640 1 --2- 172.21.15.47:0/2018627491 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f56c8101fc0 0x7f56c81982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:41.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.557+0000 7f56cd247640 1 -- 172.21.15.47:0/2018627491 >> 172.21.15.47:0/2018627491 conn(0x7f56c80fd530 msgr2=0x7f56c8106500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:00:41.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.557+0000 7f56cd247640 1 -- 172.21.15.47:0/2018627491 shutdown_connections 2024-04-17T06:00:41.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.557+0000 7f56cd247640 1 -- 172.21.15.47:0/2018627491 wait complete. 2024-04-17T06:00:41.568 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T06:00:41.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.674+0000 7f9228140640 1 -- 172.21.15.47:0/4135215322 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9220101170 msgr2=0x7f92201015d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:41.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.674+0000 7f9228140640 1 --2- 172.21.15.47:0/4135215322 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9220101170 0x7f92201015d0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f9214009900 tx=0x7f921402bc30 comp rx=0 tx=0).stop 2024-04-17T06:00:41.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.674+0000 7f9228140640 1 -- 172.21.15.47:0/4135215322 shutdown_connections 2024-04-17T06:00:41.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.674+0000 7f9228140640 1 --2- 172.21.15.47:0/4135215322 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9220101b10 0x7f9220101f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:41.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.674+0000 7f9228140640 1 --2- 172.21.15.47:0/4135215322 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9220101170 0x7f92201015d0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:41.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.674+0000 7f9228140640 1 --2- 172.21.15.47:0/4135215322 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9220100810 0x7f9220100c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:41.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.674+0000 7f9228140640 1 -- 172.21.15.47:0/4135215322 >> 172.21.15.47:0/4135215322 conn(0x7f92200fc410 msgr2=0x7f92200fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:00:41.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.674+0000 7f9228140640 1 -- 172.21.15.47:0/4135215322 shutdown_connections 2024-04-17T06:00:41.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.674+0000 7f9228140640 1 -- 172.21.15.47:0/4135215322 wait complete. 2024-04-17T06:00:41.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.675+0000 7f9228140640 1 Processor -- start 2024-04-17T06:00:41.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.675+0000 7f9228140640 1 -- start start 2024-04-17T06:00:41.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.675+0000 7f9228140640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9220100810 0x7f922019a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:41.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.675+0000 7f9228140640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9220101170 0x7f922019aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:41.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.675+0000 7f9228140640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9220101b10 0x7f92201a1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:41.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.675+0000 7f9228140640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f922010feb0 con 0x7f9220101b10 2024-04-17T06:00:41.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.675+0000 7f9228140640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f922010fd30 con 0x7f9220100810 2024-04-17T06:00:41.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.675+0000 7f9228140640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9220110030 con 0x7f9220101170 2024-04-17T06:00:41.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.675+0000 7f92266dd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9220101b10 0x7f92201a1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:41.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.675+0000 7f92266dd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9220101b10 0x7f92201a1ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57364/0 (socket says 172.21.15.47:57364) 2024-04-17T06:00:41.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.675+0000 7f92266dd640 1 -- 172.21.15.47:0/937185109 learned_addr learned my addr 172.21.15.47:0/937185109 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:00:41.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.675+0000 7f9225edc640 1 --2- 172.21.15.47:0/937185109 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9220100810 0x7f922019a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:41.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.675+0000 7f92256db640 1 --2- 172.21.15.47:0/937185109 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9220101170 0x7f922019aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:41.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.676+0000 7f92266dd640 1 -- 172.21.15.47:0/937185109 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9220101170 msgr2=0x7f922019aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:41.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.676+0000 7f92266dd640 1 --2- 172.21.15.47:0/937185109 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9220101170 0x7f922019aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:41.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.676+0000 7f92266dd640 1 -- 172.21.15.47:0/937185109 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9220100810 msgr2=0x7f922019a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:41.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.676+0000 7f92266dd640 1 --2- 172.21.15.47:0/937185109 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9220100810 0x7f922019a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:41.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.676+0000 7f92266dd640 1 -- 172.21.15.47:0/937185109 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92201a2140 con 0x7f9220101b10 2024-04-17T06:00:41.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.676+0000 7f92256db640 1 --2- 172.21.15.47:0/937185109 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9220101170 0x7f922019aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:00:41.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.676+0000 7f92266dd640 1 --2- 172.21.15.47:0/937185109 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9220101b10 0x7f92201a1ad0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f921800c970 tx=0x7f921800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:00:41.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.676+0000 7f921effd640 1 -- 172.21.15.47:0/937185109 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9218007bb0 con 0x7f9220101b10 2024-04-17T06:00:41.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.676+0000 7f921effd640 1 -- 172.21.15.47:0/937185109 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9218007d50 con 0x7f9220101b10 2024-04-17T06:00:41.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.676+0000 7f9228140640 1 -- 172.21.15.47:0/937185109 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92201a2430 con 0x7f9220101b10 2024-04-17T06:00:41.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.676+0000 7f921effd640 1 -- 172.21.15.47:0/937185109 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f921800f450 con 0x7f9220101b10 2024-04-17T06:00:41.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.676+0000 7f9228140640 1 -- 172.21.15.47:0/937185109 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92201a2900 con 0x7f9220101b10 2024-04-17T06:00:41.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.677+0000 7f9228140640 1 -- 172.21.15.47:0/937185109 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f922010b690 con 0x7f9220101b10 2024-04-17T06:00:41.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.678+0000 7f921effd640 1 -- 172.21.15.47:0/937185109 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f9218004f60 con 0x7f9220101b10 2024-04-17T06:00:41.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.679+0000 7f921effd640 1 --2- 172.21.15.47:0/937185109 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f92080a3c40 0x7f92080a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:41.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.679+0000 7f9225edc640 1 --2- 172.21.15.47:0/937185109 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f92080a3c40 0x7f92080a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:41.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.679+0000 7f921effd640 1 -- 172.21.15.47:0/937185109 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f92180d0350 con 0x7f9220101b10 2024-04-17T06:00:41.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.679+0000 7f9225edc640 1 --2- 172.21.15.47:0/937185109 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f92080a3c40 0x7f92080a6100 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f921000a400 tx=0x7f921000d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:00:41.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.683+0000 7f921effd640 1 -- 172.21.15.47:0/937185109 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f9218014030 con 0x7f9220101b10 2024-04-17T06:00:41.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.911+0000 7f9228140640 1 -- 172.21.15.47:0/937185109 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9220062b60 con 0x7f9220101b10 2024-04-17T06:00:41.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.912+0000 7f921effd640 1 -- 172.21.15.47:0/937185109 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11881 (secure 0 0 0) 0x7f921809cb80 con 0x7f9220101b10 2024-04-17T06:00:41.911 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-17T06:00:41.911 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3189958720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3189958720},{"type":"v1","addr":"172.21.15.47:6835","nonce":3189958720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14595,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/3143529973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":3143529973},{"type":"v1","addr":"172.21.15.47:6837","nonce":3143529973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14652,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/2375823701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":2375823701},{"type":"v1","addr":"172.21.15.47:6839","nonce":2375823701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14664,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6841/3334309157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":3334309157},{"type":"v1","addr":"172.21.15.47:6841","nonce":3334309157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/2398363754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":2398363754},{"type":"v1","addr":"172.21.15.103:6833","nonce":2398363754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/384473493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":384473493},{"type":"v1","addr":"172.21.15.98:6833","nonce":384473493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T05:59:13.922176+0000","modified":"2024-04-17T05:59:19.598261+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24518,"mds_1":24479,"mds_2":24466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6835/270543522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":270543522},{"type":"v1","addr":"172.21.15.103:6835","nonce":270543522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6837/116936024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":116936024},{"type":"v1","addr":"172.21.15.103:6837","nonce":116936024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6835/1966500278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":1966500278},{"type":"v1","addr":"172.21.15.98:6835","nonce":1966500278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.103:6839/1467157889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":1467157889},{"type":"v1","addr":"172.21.15.103:6839","nonce":1467157889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.98:6837/1117440719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1117440719},{"type":"v1","addr":"172.21.15.98:6837","nonce":1117440719}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6839/2841287917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":2841287917},{"type":"v1","addr":"172.21.15.98:6839","nonce":2841287917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24518,"qdb_cluster":[24479,24518,24466]},"id":1}]} 2024-04-17T06:00:41.911 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 21 2024-04-17T06:00:41.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.914+0000 7f9228140640 1 -- 172.21.15.47:0/937185109 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f92080a3c40 msgr2=0x7f92080a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:41.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.914+0000 7f9228140640 1 --2- 172.21.15.47:0/937185109 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f92080a3c40 0x7f92080a6100 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f921000a400 tx=0x7f921000d040 comp rx=0 tx=0).stop 2024-04-17T06:00:41.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.914+0000 7f9228140640 1 -- 172.21.15.47:0/937185109 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9220101b10 msgr2=0x7f92201a1ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:41.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.914+0000 7f9228140640 1 --2- 172.21.15.47:0/937185109 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9220101b10 0x7f92201a1ad0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f921800c970 tx=0x7f921800ce30 comp rx=0 tx=0).stop 2024-04-17T06:00:41.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.914+0000 7f9228140640 1 -- 172.21.15.47:0/937185109 shutdown_connections 2024-04-17T06:00:41.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.914+0000 7f9228140640 1 --2- 172.21.15.47:0/937185109 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f92080a3c40 0x7f92080a6100 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:41.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.914+0000 7f9228140640 1 --2- 172.21.15.47:0/937185109 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9220101b10 0x7f92201a1ad0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:41.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.914+0000 7f9228140640 1 --2- 172.21.15.47:0/937185109 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9220101170 0x7f922019aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:41.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.914+0000 7f9228140640 1 --2- 172.21.15.47:0/937185109 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9220100810 0x7f922019a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:41.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.914+0000 7f9228140640 1 -- 172.21.15.47:0/937185109 >> 172.21.15.47:0/937185109 conn(0x7f92200fc410 msgr2=0x7f92200fdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:00:41.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.914+0000 7f9228140640 1 -- 172.21.15.47:0/937185109 shutdown_connections 2024-04-17T06:00:41.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:41.914+0000 7f9228140640 1 -- 172.21.15.47:0/937185109 wait complete. 2024-04-17T06:00:41.924 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T06:00:42.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.031+0000 7efefaf52640 1 -- 172.21.15.47:0/893648549 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efef4100030 msgr2=0x7efef41004b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:42.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.031+0000 7efefaf52640 1 --2- 172.21.15.47:0/893648549 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efef4100030 0x7efef41004b0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7efee0009900 tx=0x7efee002bc30 comp rx=0 tx=0).stop 2024-04-17T06:00:42.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.031+0000 7efefaf52640 1 -- 172.21.15.47:0/893648549 shutdown_connections 2024-04-17T06:00:42.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.031+0000 7efefaf52640 1 --2- 172.21.15.47:0/893648549 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efef41009f0 0x7efef410a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:42.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.031+0000 7efefaf52640 1 --2- 172.21.15.47:0/893648549 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efef4100030 0x7efef41004b0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:42.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.031+0000 7efefaf52640 1 --2- 172.21.15.47:0/893648549 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efef41035c0 0x7efef41039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:42.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.031+0000 7efefaf52640 1 -- 172.21.15.47:0/893648549 >> 172.21.15.47:0/893648549 conn(0x7efef40fbbd0 msgr2=0x7efef40fdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:00:42.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.031+0000 7efefaf52640 1 -- 172.21.15.47:0/893648549 shutdown_connections 2024-04-17T06:00:42.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.031+0000 7efefaf52640 1 -- 172.21.15.47:0/893648549 wait complete. 2024-04-17T06:00:42.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.031+0000 7efefaf52640 1 Processor -- start 2024-04-17T06:00:42.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.032+0000 7efefaf52640 1 -- start start 2024-04-17T06:00:42.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.032+0000 7efefaf52640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efef4100030 0x7efef4198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:42.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.032+0000 7efefaf52640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efef41009f0 0x7efef41987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:42.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.032+0000 7efefaf52640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efef41035c0 0x7efef419f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:42.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.032+0000 7efefaf52640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7efef410dca0 con 0x7efef41035c0 2024-04-17T06:00:42.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.032+0000 7efefaf52640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7efef410db20 con 0x7efef4100030 2024-04-17T06:00:42.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.032+0000 7efefaf52640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7efef410de20 con 0x7efef41009f0 2024-04-17T06:00:42.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.032+0000 7efef94ef640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efef41035c0 0x7efef419f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:42.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.032+0000 7efef94ef640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efef41035c0 0x7efef419f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57380/0 (socket says 172.21.15.47:57380) 2024-04-17T06:00:42.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.032+0000 7efef94ef640 1 -- 172.21.15.47:0/290675453 learned_addr learned my addr 172.21.15.47:0/290675453 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:00:42.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.032+0000 7efef8cee640 1 --2- 172.21.15.47:0/290675453 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efef4100030 0x7efef4198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:42.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.032+0000 7efef3fff640 1 --2- 172.21.15.47:0/290675453 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efef41009f0 0x7efef41987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:42.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.032+0000 7efef94ef640 1 -- 172.21.15.47:0/290675453 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efef41009f0 msgr2=0x7efef41987d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:42.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.032+0000 7efef94ef640 1 --2- 172.21.15.47:0/290675453 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efef41009f0 0x7efef41987d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:42.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.032+0000 7efef94ef640 1 -- 172.21.15.47:0/290675453 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efef4100030 msgr2=0x7efef4198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:42.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.032+0000 7efef94ef640 1 --2- 172.21.15.47:0/290675453 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efef4100030 0x7efef4198290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:42.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.032+0000 7efef94ef640 1 -- 172.21.15.47:0/290675453 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efef419feb0 con 0x7efef41035c0 2024-04-17T06:00:42.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.032+0000 7efef94ef640 1 --2- 172.21.15.47:0/290675453 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efef41035c0 0x7efef419f840 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7efeec00b970 tx=0x7efeec00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:00:42.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.032+0000 7efef1ffb640 1 -- 172.21.15.47:0/290675453 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeec019070 con 0x7efef41035c0 2024-04-17T06:00:42.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.032+0000 7efef1ffb640 1 -- 172.21.15.47:0/290675453 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efeec00cb40 con 0x7efef41035c0 2024-04-17T06:00:42.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.032+0000 7efefaf52640 1 -- 172.21.15.47:0/290675453 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efef41a01a0 con 0x7efef41035c0 2024-04-17T06:00:42.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.032+0000 7efef1ffb640 1 -- 172.21.15.47:0/290675453 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeec007650 con 0x7efef41035c0 2024-04-17T06:00:42.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.033+0000 7efefaf52640 1 -- 172.21.15.47:0/290675453 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efef41a0740 con 0x7efef41035c0 2024-04-17T06:00:42.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.033+0000 7efefaf52640 1 -- 172.21.15.47:0/290675453 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efef4109420 con 0x7efef41035c0 2024-04-17T06:00:42.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.035+0000 7efef1ffb640 1 -- 172.21.15.47:0/290675453 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7efeec01c070 con 0x7efef41035c0 2024-04-17T06:00:42.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.035+0000 7efef1ffb640 1 --2- 172.21.15.47:0/290675453 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7efee40a3cc0 0x7efee40a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:42.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.035+0000 7efef8cee640 1 --2- 172.21.15.47:0/290675453 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7efee40a3cc0 0x7efee40a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:42.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.035+0000 7efef1ffb640 1 -- 172.21.15.47:0/290675453 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7efeec0cfa60 con 0x7efef41035c0 2024-04-17T06:00:42.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.036+0000 7efef8cee640 1 --2- 172.21.15.47:0/290675453 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7efee40a3cc0 0x7efee40a6180 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7efee80058c0 tx=0x7efee8005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:00:42.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.040+0000 7efef1ffb640 1 -- 172.21.15.47:0/290675453 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7efeec09c3a0 con 0x7efef41035c0 2024-04-17T06:00:42.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:00:41 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/407353268' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-17T06:00:42.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:00:41 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/190461638' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-17T06:00:42.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:00:41 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/2018627491' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-17T06:00:42.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:00:41 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/407353268' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-17T06:00:42.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:00:41 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/190461638' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-17T06:00:42.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:00:41 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/2018627491' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-17T06:00:42.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.227+0000 7efefaf52640 1 -- 172.21.15.47:0/290675453 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efef4062b60 con 0x7efef41035c0 2024-04-17T06:00:42.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.228+0000 7efef1ffb640 1 -- 172.21.15.47:0/290675453 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22602 (secure 0 0 0) 0x7efeec09bca0 con 0x7efef41035c0 2024-04-17T06:00:42.227 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-17T06:00:42.227 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":88,"fsid":"565db2d8-fc7e-11ee-bc91-c7b262605968","created":"2024-04-17T05:51:19.888865+0000","modified":"2024-04-17T06:00:14.720609+0000","last_up_change":"2024-04-17T05:58:24.380131+0000","last_in_change":"2024-04-17T05:58:04.418236+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T05:54:37.734306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T05:59:12.458010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T05:59:12.901339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8453f300-533d-4736-a343-55619c08e966","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":1290127544},{"type":"v1","addr":"172.21.15.47:6803","nonce":1290127544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":1290127544},{"type":"v1","addr":"172.21.15.47:6805","nonce":1290127544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":1290127544},{"type":"v1","addr":"172.21.15.47:6809","nonce":1290127544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":1290127544},{"type":"v1","addr":"172.21.15.47:6807","nonce":1290127544}]},"public_addr":"172.21.15.47:6803/1290127544","cluster_addr":"172.21.15.47:6805/1290127544","heartbeat_back_addr":"172.21.15.47:6809/1290127544","heartbeat_front_addr":"172.21.15.47:6807/1290127544","state":["exists","up"]},{"osd":1,"uuid":"80b0f03e-c1bc-4658-8028-1e6b8eedcc47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":1180430954},{"type":"v1","addr":"172.21.15.98:6801","nonce":1180430954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":1180430954},{"type":"v1","addr":"172.21.15.98:6803","nonce":1180430954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":1180430954},{"type":"v1","addr":"172.21.15.98:6807","nonce":1180430954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":1180430954},{"type":"v1","addr":"172.21.15.98:6805","nonce":1180430954}]},"public_addr":"172.21.15.98:6801/1180430954","cluster_addr":"172.21.15.98:6803/1180430954","heartbeat_back_addr":"172.21.15.98:6807/1180430954","heartbeat_front_addr":"172.21.15.98:6805/1180430954","state":["exists","up"]},{"osd":2,"uuid":"0fc9d383-05b4-498d-b530-fccf7aeddd7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":494676216},{"type":"v1","addr":"172.21.15.103:6801","nonce":494676216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":494676216},{"type":"v1","addr":"172.21.15.103:6803","nonce":494676216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":494676216},{"type":"v1","addr":"172.21.15.103:6807","nonce":494676216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":494676216},{"type":"v1","addr":"172.21.15.103:6805","nonce":494676216}]},"public_addr":"172.21.15.103:6801/494676216","cluster_addr":"172.21.15.103:6803/494676216","heartbeat_back_addr":"172.21.15.103:6807/494676216","heartbeat_front_addr":"172.21.15.103:6805/494676216","state":["exists","up"]},{"osd":3,"uuid":"f6e0b0d5-0fb3-42d0-9e84-f7cfef21a998","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":1998369447},{"type":"v1","addr":"172.21.15.47:6811","nonce":1998369447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":1998369447},{"type":"v1","addr":"172.21.15.47:6813","nonce":1998369447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":1998369447},{"type":"v1","addr":"172.21.15.47:6817","nonce":1998369447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":1998369447},{"type":"v1","addr":"172.21.15.47:6815","nonce":1998369447}]},"public_addr":"172.21.15.47:6811/1998369447","cluster_addr":"172.21.15.47:6813/1998369447","heartbeat_back_addr":"172.21.15.47:6817/1998369447","heartbeat_front_addr":"172.21.15.47:6815/1998369447","state":["exists","up"]},{"osd":4,"uuid":"2c26217e-02df-466d-bc01-0149d4aec3c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":3060482485},{"type":"v1","addr":"172.21.15.98:6809","nonce":3060482485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":3060482485},{"type":"v1","addr":"172.21.15.98:6811","nonce":3060482485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":3060482485},{"type":"v1","addr":"172.21.15.98:6815","nonce":3060482485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":3060482485},{"type":"v1","addr":"172.21.15.98:6813","nonce":3060482485}]},"public_addr":"172.21.15.98:6809/3060482485","cluster_addr":"172.21.15.98:6811/3060482485","heartbeat_back_addr":"172.21.15.98:6815/3060482485","heartbeat_front_addr":"172.21.15.98:6813/3060482485","state":["exists","up"]},{"osd":5,"uuid":"740bf3e0-bea8-496b-8d6f-e42d0b017258","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":4181363567},{"type":"v1","addr":"172.21.15.103:6809","nonce":4181363567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":4181363567},{"type":"v1","addr":"172.21.15.103:6811","nonce":4181363567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":4181363567},{"type":"v1","addr":"172.21.15.103:6815","nonce":4181363567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":4181363567},{"type":"v1","addr":"172.21.15.103:6813","nonce":4181363567}]},"public_addr":"172.21.15.103:6809/4181363567","cluster_addr":"172.21.15.103:6811/4181363567","heartbeat_back_addr":"172.21.15.103:6815/4181363567","heartbeat_front_addr":"172.21.15.103:6813/4181363567","state":["exists","up"]},{"osd":6,"uuid":"356a6255-2c07-4f0d-a233-26be04c92b38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":2517529668},{"type":"v1","addr":"172.21.15.47:6819","nonce":2517529668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2517529668},{"type":"v1","addr":"172.21.15.47:6821","nonce":2517529668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2517529668},{"type":"v1","addr":"172.21.15.47:6825","nonce":2517529668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2517529668},{"type":"v1","addr":"172.21.15.47:6823","nonce":2517529668}]},"public_addr":"172.21.15.47:6819/2517529668","cluster_addr":"172.21.15.47:6821/2517529668","heartbeat_back_addr":"172.21.15.47:6825/2517529668","heartbeat_front_addr":"172.21.15.47:6823/2517529668","state":["exists","up"]},{"osd":7,"uuid":"349d0120-8290-417d-a800-5a8985f554f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":4225498496},{"type":"v1","addr":"172.21.15.98:6817","nonce":4225498496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":4225498496},{"type":"v1","addr":"172.21.15.98:6819","nonce":4225498496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":4225498496},{"type":"v1","addr":"172.21.15.98:6823","nonce":4225498496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":4225498496},{"type":"v1","addr":"172.21.15.98:6821","nonce":4225498496}]},"public_addr":"172.21.15.98:6817/4225498496","cluster_addr":"172.21.15.98:6819/4225498496","heartbeat_back_addr":"172.21.15.98:6823/4225498496","heartbeat_front_addr":"172.21.15.98:6821/4225498496","state":["exists","up"]},{"osd":8,"uuid":"cee4712f-321b-42c0-92ba-e15144c00f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":3433905520},{"type":"v1","addr":"172.21.15.103:6817","nonce":3433905520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":3433905520},{"type":"v1","addr":"172.21.15.103:6819","nonce":3433905520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":3433905520},{"type":"v1","addr":"172.21.15.103:6823","nonce":3433905520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":3433905520},{"type":"v1","addr":"172.21.15.103:6821","nonce":3433905520}]},"public_addr":"172.21.15.103:6817/3433905520","cluster_addr":"172.21.15.103:6819/3433905520","heartbeat_back_addr":"172.21.15.103:6823/3433905520","heartbeat_front_addr":"172.21.15.103:6821/3433905520","state":["exists","up"]},{"osd":9,"uuid":"63fc73c1-5a22-487d-80b8-9894e2ddd207","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":728359554},{"type":"v1","addr":"172.21.15.47:6827","nonce":728359554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":728359554},{"type":"v1","addr":"172.21.15.47:6829","nonce":728359554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":728359554},{"type":"v1","addr":"172.21.15.47:6833","nonce":728359554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":728359554},{"type":"v1","addr":"172.21.15.47:6831","nonce":728359554}]},"public_addr":"172.21.15.47:6827/728359554","cluster_addr":"172.21.15.47:6829/728359554","heartbeat_back_addr":"172.21.15.47:6833/728359554","heartbeat_front_addr":"172.21.15.47:6831/728359554","state":["exists","up"]},{"osd":10,"uuid":"9565bcc3-62ee-48eb-a07a-03a81e0cc8e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":468205689},{"type":"v1","addr":"172.21.15.98:6825","nonce":468205689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":468205689},{"type":"v1","addr":"172.21.15.98:6827","nonce":468205689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":468205689},{"type":"v1","addr":"172.21.15.98:6831","nonce":468205689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":468205689},{"type":"v1","addr":"172.21.15.98:6829","nonce":468205689}]},"public_addr":"172.21.15.98:6825/468205689","cluster_addr":"172.21.15.98:6827/468205689","heartbeat_back_addr":"172.21.15.98:6831/468205689","heartbeat_front_addr":"172.21.15.98:6829/468205689","state":["exists","up"]},{"osd":11,"uuid":"2e85e8d9-bf6c-4a12-b124-a4690dd61108","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":2639831013},{"type":"v1","addr":"172.21.15.103:6825","nonce":2639831013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":2639831013},{"type":"v1","addr":"172.21.15.103:6827","nonce":2639831013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":2639831013},{"type":"v1","addr":"172.21.15.103:6831","nonce":2639831013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":2639831013},{"type":"v1","addr":"172.21.15.103:6829","nonce":2639831013}]},"public_addr":"172.21.15.103:6825/2639831013","cluster_addr":"172.21.15.103:6827/2639831013","heartbeat_back_addr":"172.21.15.103:6831/2639831013","heartbeat_front_addr":"172.21.15.103:6829/2639831013","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:53:47.366438+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:54:10.953559+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:54:34.320332+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:54:59.751360+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:55:23.982052+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:55:48.425017+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:56:14.793368+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:56:40.440830+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:57:04.623680+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:57:31.215262+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:57:57.040346+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:58:22.671572+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:6801/3632157175":"2024-04-18T05:52:13.645775+0000","172.21.15.47:0/4288219269":"2024-04-18T05:52:13.645775+0000","172.21.15.47:0/2584487544":"2024-04-18T05:52:13.645775+0000","172.21.15.47:6801/953064786":"2024-04-18T05:51:45.979351+0000","172.21.15.47:0/3484500927":"2024-04-18T05:51:45.979351+0000","172.21.15.47:6800/3632157175":"2024-04-18T05:52:13.645775+0000","172.21.15.47:0/112268670":"2024-04-18T05:51:45.979351+0000","172.21.15.47:0/1638423256":"2024-04-18T05:51:45.979351+0000","172.21.15.47:6800/953064786":"2024-04-18T05:51:45.979351+0000","172.21.15.47:0/849479010":"2024-04-18T05:52:13.645775+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T06:00:42.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.231+0000 7efefaf52640 1 -- 172.21.15.47:0/290675453 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7efee40a3cc0 msgr2=0x7efee40a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:42.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.231+0000 7efefaf52640 1 --2- 172.21.15.47:0/290675453 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7efee40a3cc0 0x7efee40a6180 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7efee80058c0 tx=0x7efee8005850 comp rx=0 tx=0).stop 2024-04-17T06:00:42.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.231+0000 7efefaf52640 1 -- 172.21.15.47:0/290675453 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efef41035c0 msgr2=0x7efef419f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:42.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.231+0000 7efefaf52640 1 --2- 172.21.15.47:0/290675453 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efef41035c0 0x7efef419f840 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7efeec00b970 tx=0x7efeec00be30 comp rx=0 tx=0).stop 2024-04-17T06:00:42.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.231+0000 7efefaf52640 1 -- 172.21.15.47:0/290675453 shutdown_connections 2024-04-17T06:00:42.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.231+0000 7efefaf52640 1 --2- 172.21.15.47:0/290675453 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7efee40a3cc0 0x7efee40a6180 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:42.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.231+0000 7efefaf52640 1 --2- 172.21.15.47:0/290675453 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efef41035c0 0x7efef419f840 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:42.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.231+0000 7efefaf52640 1 --2- 172.21.15.47:0/290675453 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efef41009f0 0x7efef41987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:42.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.231+0000 7efefaf52640 1 --2- 172.21.15.47:0/290675453 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efef4100030 0x7efef4198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:42.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.231+0000 7efefaf52640 1 -- 172.21.15.47:0/290675453 >> 172.21.15.47:0/290675453 conn(0x7efef40fbbd0 msgr2=0x7efef41065f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:00:42.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.231+0000 7efefaf52640 1 -- 172.21.15.47:0/290675453 shutdown_connections 2024-04-17T06:00:42.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.231+0000 7efefaf52640 1 -- 172.21.15.47:0/290675453 wait complete. 2024-04-17T06:00:42.241 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T05:54:37.734306+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T05:59:12.458010+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T05:59:12.901339+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T06:00:42.241 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T06:00:42.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:00:41 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/407353268' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-17T06:00:42.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:00:41 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/190461638' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-17T06:00:42.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:00:41 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/2018627491' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-17T06:00:42.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.351+0000 7fe965da8640 1 -- 172.21.15.47:0/2173798848 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe960103c00 msgr2=0x7fe96010a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:42.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.351+0000 7fe965da8640 1 --2- 172.21.15.47:0/2173798848 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe960103c00 0x7fe96010a490 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fe958009cd0 tx=0x7fe95802c0f0 comp rx=0 tx=0).stop 2024-04-17T06:00:42.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.351+0000 7fe965da8640 1 -- 172.21.15.47:0/2173798848 shutdown_connections 2024-04-17T06:00:42.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.351+0000 7fe965da8640 1 --2- 172.21.15.47:0/2173798848 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe960103c00 0x7fe96010a490 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:42.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.351+0000 7fe965da8640 1 --2- 172.21.15.47:0/2173798848 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe960103240 0x7fe9601036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:42.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.351+0000 7fe965da8640 1 --2- 172.21.15.47:0/2173798848 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe960100270 0x7fe960100690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:42.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.351+0000 7fe965da8640 1 -- 172.21.15.47:0/2173798848 >> 172.21.15.47:0/2173798848 conn(0x7fe9600fbba0 msgr2=0x7fe9600fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:00:42.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.351+0000 7fe965da8640 1 -- 172.21.15.47:0/2173798848 shutdown_connections 2024-04-17T06:00:42.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.351+0000 7fe965da8640 1 -- 172.21.15.47:0/2173798848 wait complete. 2024-04-17T06:00:42.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.352+0000 7fe965da8640 1 Processor -- start 2024-04-17T06:00:42.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.352+0000 7fe965da8640 1 -- start start 2024-04-17T06:00:42.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.352+0000 7fe965da8640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe960100270 0x7fe960198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:42.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.352+0000 7fe965da8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe960103240 0x7fe9601988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:42.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.352+0000 7fe965da8640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe960103c00 0x7fe96019f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:42.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.352+0000 7fe965da8640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fe96010dd40 con 0x7fe960103c00 2024-04-17T06:00:42.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.352+0000 7fe965da8640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe96010dbc0 con 0x7fe960100270 2024-04-17T06:00:42.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.352+0000 7fe965da8640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe96010dec0 con 0x7fe960103240 2024-04-17T06:00:42.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.352+0000 7fe95ffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe960103c00 0x7fe96019f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:42.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.352+0000 7fe95ffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe960103c00 0x7fe96019f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57402/0 (socket says 172.21.15.47:57402) 2024-04-17T06:00:42.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.352+0000 7fe95ffff640 1 -- 172.21.15.47:0/4181596047 learned_addr learned my addr 172.21.15.47:0/4181596047 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:00:42.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.352+0000 7fe95f7fe640 1 --2- 172.21.15.47:0/4181596047 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe960100270 0x7fe960198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:42.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.352+0000 7fe95effd640 1 --2- 172.21.15.47:0/4181596047 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe960103240 0x7fe9601988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:42.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.352+0000 7fe95ffff640 1 -- 172.21.15.47:0/4181596047 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe960103240 msgr2=0x7fe9601988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:42.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.352+0000 7fe95ffff640 1 --2- 172.21.15.47:0/4181596047 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe960103240 0x7fe9601988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:42.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.352+0000 7fe95ffff640 1 -- 172.21.15.47:0/4181596047 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe960100270 msgr2=0x7fe960198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:42.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.352+0000 7fe95ffff640 1 --2- 172.21.15.47:0/4181596047 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe960100270 0x7fe960198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:42.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.352+0000 7fe95ffff640 1 -- 172.21.15.47:0/4181596047 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe96019ff80 con 0x7fe960103c00 2024-04-17T06:00:42.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.352+0000 7fe95f7fe640 1 --2- 172.21.15.47:0/4181596047 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe960100270 0x7fe960198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:00:42.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.352+0000 7fe95effd640 1 --2- 172.21.15.47:0/4181596047 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe960103240 0x7fe9601988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:00:42.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.353+0000 7fe95ffff640 1 --2- 172.21.15.47:0/4181596047 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe960103c00 0x7fe96019f910 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fe958009ca0 tx=0x7fe958035d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:00:42.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.353+0000 7fe95cff9640 1 -- 172.21.15.47:0/4181596047 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe958039040 con 0x7fe960103c00 2024-04-17T06:00:42.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.353+0000 7fe95cff9640 1 -- 172.21.15.47:0/4181596047 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe958044070 con 0x7fe960103c00 2024-04-17T06:00:42.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.353+0000 7fe965da8640 1 -- 172.21.15.47:0/4181596047 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9601a0210 con 0x7fe960103c00 2024-04-17T06:00:42.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.353+0000 7fe95cff9640 1 -- 172.21.15.47:0/4181596047 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9580045a0 con 0x7fe960103c00 2024-04-17T06:00:42.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.353+0000 7fe965da8640 1 -- 172.21.15.47:0/4181596047 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9601a0780 con 0x7fe960103c00 2024-04-17T06:00:42.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.354+0000 7fe965da8640 1 -- 172.21.15.47:0/4181596047 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe960109370 con 0x7fe960103c00 2024-04-17T06:00:42.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.355+0000 7fe95cff9640 1 -- 172.21.15.47:0/4181596047 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe958003970 con 0x7fe960103c00 2024-04-17T06:00:42.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.356+0000 7fe95cff9640 1 --2- 172.21.15.47:0/4181596047 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fe9480a3c40 0x7fe9480a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:42.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.356+0000 7fe95f7fe640 1 --2- 172.21.15.47:0/4181596047 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fe9480a3c40 0x7fe9480a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:42.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.356+0000 7fe95cff9640 1 -- 172.21.15.47:0/4181596047 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fe9580f2cb0 con 0x7fe960103c00 2024-04-17T06:00:42.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.356+0000 7fe95f7fe640 1 --2- 172.21.15.47:0/4181596047 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fe9480a3c40 0x7fe9480a6100 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe950006fd0 tx=0x7fe950008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:00:42.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.360+0000 7fe95cff9640 1 -- 172.21.15.47:0/4181596047 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fe9580bf5f0 con 0x7fe960103c00 2024-04-17T06:00:42.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.546+0000 7fe965da8640 1 -- 172.21.15.47:0/4181596047 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe960062b60 con 0x7fe960103c00 2024-04-17T06:00:42.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.546+0000 7fe95cff9640 1 -- 172.21.15.47:0/4181596047 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7fe9580beef0 con 0x7fe960103c00 2024-04-17T06:00:42.545 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-04-17T06:00:42.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.549+0000 7fe965da8640 1 -- 172.21.15.47:0/4181596047 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fe9480a3c40 msgr2=0x7fe9480a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:42.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.549+0000 7fe965da8640 1 --2- 172.21.15.47:0/4181596047 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fe9480a3c40 0x7fe9480a6100 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe950006fd0 tx=0x7fe950008040 comp rx=0 tx=0).stop 2024-04-17T06:00:42.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.549+0000 7fe965da8640 1 -- 172.21.15.47:0/4181596047 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe960103c00 msgr2=0x7fe96019f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:42.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.549+0000 7fe965da8640 1 --2- 172.21.15.47:0/4181596047 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe960103c00 0x7fe96019f910 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fe958009ca0 tx=0x7fe958035d90 comp rx=0 tx=0).stop 2024-04-17T06:00:42.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.549+0000 7fe965da8640 1 -- 172.21.15.47:0/4181596047 shutdown_connections 2024-04-17T06:00:42.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.549+0000 7fe965da8640 1 --2- 172.21.15.47:0/4181596047 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fe9480a3c40 0x7fe9480a6100 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:42.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.549+0000 7fe965da8640 1 --2- 172.21.15.47:0/4181596047 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe960103c00 0x7fe96019f910 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:42.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.549+0000 7fe965da8640 1 --2- 172.21.15.47:0/4181596047 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe960103240 0x7fe9601988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:42.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.549+0000 7fe965da8640 1 --2- 172.21.15.47:0/4181596047 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe960100270 0x7fe960198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:42.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.549+0000 7fe965da8640 1 -- 172.21.15.47:0/4181596047 >> 172.21.15.47:0/4181596047 conn(0x7fe9600fbba0 msgr2=0x7fe960106540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:00:42.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.549+0000 7fe965da8640 1 -- 172.21.15.47:0/4181596047 shutdown_connections 2024-04-17T06:00:42.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.549+0000 7fe965da8640 1 -- 172.21.15.47:0/4181596047 wait complete. 2024-04-17T06:00:42.559 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T06:00:42.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.670+0000 7fa1084d9640 1 -- 172.21.15.47:0/3452733893 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa100107c80 msgr2=0x7fa10010a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:42.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.670+0000 7fa1084d9640 1 --2- 172.21.15.47:0/3452733893 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa100107c80 0x7fa10010a0d0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fa0f8009ca0 tx=0x7fa0f802c0f0 comp rx=0 tx=0).stop 2024-04-17T06:00:42.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.670+0000 7fa1084d9640 1 -- 172.21.15.47:0/3452733893 shutdown_connections 2024-04-17T06:00:42.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.670+0000 7fa1084d9640 1 --2- 172.21.15.47:0/3452733893 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa100107c80 0x7fa10010a0d0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:42.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.670+0000 7fa1084d9640 1 --2- 172.21.15.47:0/3452733893 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa100105350 0x7fa100107740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:42.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.670+0000 7fa1084d9640 1 --2- 172.21.15.47:0/3452733893 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa100102a20 0x7fa100104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:42.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.670+0000 7fa1084d9640 1 -- 172.21.15.47:0/3452733893 >> 172.21.15.47:0/3452733893 conn(0x7fa1000fc410 msgr2=0x7fa1000fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:00:42.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.670+0000 7fa1084d9640 1 -- 172.21.15.47:0/3452733893 shutdown_connections 2024-04-17T06:00:42.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.670+0000 7fa1084d9640 1 -- 172.21.15.47:0/3452733893 wait complete. 2024-04-17T06:00:42.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.671+0000 7fa1084d9640 1 Processor -- start 2024-04-17T06:00:42.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.671+0000 7fa1084d9640 1 -- start start 2024-04-17T06:00:42.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.671+0000 7fa1084d9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa100102a20 0x7fa100198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:42.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.671+0000 7fa1084d9640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa100105350 0x7fa100198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:42.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.671+0000 7fa1084d9640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa100107c80 0x7fa10019f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:42.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.671+0000 7fa1084d9640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa10010dc30 con 0x7fa100105350 2024-04-17T06:00:42.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.671+0000 7fa1084d9640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa10010dab0 con 0x7fa100107c80 2024-04-17T06:00:42.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.671+0000 7fa1084d9640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa10010ddb0 con 0x7fa100102a20 2024-04-17T06:00:42.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.671+0000 7fa105a74640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa100105350 0x7fa100198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:42.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.671+0000 7fa105a74640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa100105350 0x7fa100198780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57408/0 (socket says 172.21.15.47:57408) 2024-04-17T06:00:42.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.671+0000 7fa105a74640 1 -- 172.21.15.47:0/2973704640 learned_addr learned my addr 172.21.15.47:0/2973704640 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:00:42.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.671+0000 7fa106275640 1 --2- 172.21.15.47:0/2973704640 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa100102a20 0x7fa100198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:42.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.672+0000 7fa106a76640 1 --2- 172.21.15.47:0/2973704640 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa100107c80 0x7fa10019f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:42.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.672+0000 7fa105a74640 1 -- 172.21.15.47:0/2973704640 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa100102a20 msgr2=0x7fa100198240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:42.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.672+0000 7fa105a74640 1 --2- 172.21.15.47:0/2973704640 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa100102a20 0x7fa100198240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:42.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.672+0000 7fa105a74640 1 -- 172.21.15.47:0/2973704640 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa100107c80 msgr2=0x7fa10019f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:42.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.672+0000 7fa105a74640 1 --2- 172.21.15.47:0/2973704640 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa100107c80 0x7fa10019f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:42.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.672+0000 7fa105a74640 1 -- 172.21.15.47:0/2973704640 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa10019fe60 con 0x7fa100105350 2024-04-17T06:00:42.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.672+0000 7fa105a74640 1 --2- 172.21.15.47:0/2973704640 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa100105350 0x7fa100198780 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fa0f000c970 tx=0x7fa0f000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:00:42.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.672+0000 7fa0f77fe640 1 -- 172.21.15.47:0/2973704640 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f0014070 con 0x7fa100105350 2024-04-17T06:00:42.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.672+0000 7fa0f77fe640 1 -- 172.21.15.47:0/2973704640 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0f0004440 con 0x7fa100105350 2024-04-17T06:00:42.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.672+0000 7fa1084d9640 1 -- 172.21.15.47:0/2973704640 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1001a0150 con 0x7fa100105350 2024-04-17T06:00:42.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.672+0000 7fa0f77fe640 1 -- 172.21.15.47:0/2973704640 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f000f450 con 0x7fa100105350 2024-04-17T06:00:42.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.672+0000 7fa1084d9640 1 -- 172.21.15.47:0/2973704640 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1001a06a0 con 0x7fa100105350 2024-04-17T06:00:42.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.673+0000 7fa1084d9640 1 -- 172.21.15.47:0/2973704640 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1000fdf00 con 0x7fa100105350 2024-04-17T06:00:42.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.674+0000 7fa0f77fe640 1 -- 172.21.15.47:0/2973704640 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fa0f0017070 con 0x7fa100105350 2024-04-17T06:00:42.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.675+0000 7fa0f77fe640 1 --2- 172.21.15.47:0/2973704640 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fa0ec0a3c40 0x7fa0ec0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:42.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.675+0000 7fa106275640 1 --2- 172.21.15.47:0/2973704640 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fa0ec0a3c40 0x7fa0ec0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:42.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.675+0000 7fa0f77fe640 1 -- 172.21.15.47:0/2973704640 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fa0f00d03a0 con 0x7fa100105350 2024-04-17T06:00:42.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.676+0000 7fa106275640 1 --2- 172.21.15.47:0/2973704640 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fa0ec0a3c40 0x7fa0ec0a6100 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc005fd0 tx=0x7fa0fc00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:00:42.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.680+0000 7fa0f77fe640 1 -- 172.21.15.47:0/2973704640 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fa0f001e020 con 0x7fa100105350 2024-04-17T06:00:42.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.868+0000 7fa1084d9640 1 -- 172.21.15.47:0/2973704640 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa10002d820 con 0x7fa100105350 2024-04-17T06:00:42.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.868+0000 7fa0f77fe640 1 -- 172.21.15.47:0/2973704640 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7fa0f009cc30 con 0x7fa100105350 2024-04-17T06:00:42.868 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-04-17T06:00:42.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.870+0000 7fa1084d9640 1 -- 172.21.15.47:0/2973704640 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fa0ec0a3c40 msgr2=0x7fa0ec0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:42.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.870+0000 7fa1084d9640 1 --2- 172.21.15.47:0/2973704640 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fa0ec0a3c40 0x7fa0ec0a6100 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc005fd0 tx=0x7fa0fc00b390 comp rx=0 tx=0).stop 2024-04-17T06:00:42.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.870+0000 7fa1084d9640 1 -- 172.21.15.47:0/2973704640 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa100105350 msgr2=0x7fa100198780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:42.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.870+0000 7fa1084d9640 1 --2- 172.21.15.47:0/2973704640 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa100105350 0x7fa100198780 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fa0f000c970 tx=0x7fa0f000ce30 comp rx=0 tx=0).stop 2024-04-17T06:00:42.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.870+0000 7fa1084d9640 1 -- 172.21.15.47:0/2973704640 shutdown_connections 2024-04-17T06:00:42.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.870+0000 7fa1084d9640 1 --2- 172.21.15.47:0/2973704640 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fa0ec0a3c40 0x7fa0ec0a6100 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:42.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.870+0000 7fa1084d9640 1 --2- 172.21.15.47:0/2973704640 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa100107c80 0x7fa10019f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:42.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.870+0000 7fa1084d9640 1 --2- 172.21.15.47:0/2973704640 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa100105350 0x7fa100198780 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:42.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.870+0000 7fa1084d9640 1 --2- 172.21.15.47:0/2973704640 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa100102a20 0x7fa100198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:42.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.870+0000 7fa1084d9640 1 -- 172.21.15.47:0/2973704640 >> 172.21.15.47:0/2973704640 conn(0x7fa1000fc410 msgr2=0x7fa100103360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:00:42.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.871+0000 7fa1084d9640 1 -- 172.21.15.47:0/2973704640 shutdown_connections 2024-04-17T06:00:42.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.871+0000 7fa1084d9640 1 -- 172.21.15.47:0/2973704640 wait complete. 2024-04-17T06:00:42.881 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T06:00:42.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.991+0000 7f8b5db0e640 1 -- 172.21.15.47:0/3813784824 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b58101820 msgr2=0x7f8b58101ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:42.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.991+0000 7f8b5db0e640 1 --2- 172.21.15.47:0/3813784824 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b58101820 0x7f8b58101ca0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c009900 tx=0x7f8b4c02bc30 comp rx=0 tx=0).stop 2024-04-17T06:00:42.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.991+0000 7f8b5db0e640 1 -- 172.21.15.47:0/3813784824 shutdown_connections 2024-04-17T06:00:42.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.991+0000 7f8b5db0e640 1 --2- 172.21.15.47:0/3813784824 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8b581021e0 0x7f8b58108a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:42.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.991+0000 7f8b5db0e640 1 --2- 172.21.15.47:0/3813784824 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b58101820 0x7f8b58101ca0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:42.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.991+0000 7f8b5db0e640 1 --2- 172.21.15.47:0/3813784824 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8b58100630 0x7f8b58100a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:42.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.991+0000 7f8b5db0e640 1 -- 172.21.15.47:0/3813784824 >> 172.21.15.47:0/3813784824 conn(0x7f8b580fbba0 msgr2=0x7f8b580fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:00:42.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.991+0000 7f8b5db0e640 1 -- 172.21.15.47:0/3813784824 shutdown_connections 2024-04-17T06:00:42.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.991+0000 7f8b5db0e640 1 -- 172.21.15.47:0/3813784824 wait complete. 2024-04-17T06:00:42.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.992+0000 7f8b5db0e640 1 Processor -- start 2024-04-17T06:00:42.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.992+0000 7f8b5db0e640 1 -- start start 2024-04-17T06:00:42.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.992+0000 7f8b5db0e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8b58100630 0x7f8b581983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:42.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.992+0000 7f8b5db0e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8b58101820 0x7f8b58198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:42.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.992+0000 7f8b5db0e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b581021e0 0x7f8b5819f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:42.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.992+0000 7f8b5db0e640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f8b5810dcc0 con 0x7f8b581021e0 2024-04-17T06:00:42.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.992+0000 7f8b5db0e640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8b5810db40 con 0x7f8b58100630 2024-04-17T06:00:42.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.992+0000 7f8b5db0e640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8b5810de40 con 0x7f8b58101820 2024-04-17T06:00:42.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.992+0000 7f8b57fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b581021e0 0x7f8b5819f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:42.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.992+0000 7f8b57fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b581021e0 0x7f8b5819f980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57414/0 (socket says 172.21.15.47:57414) 2024-04-17T06:00:42.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.992+0000 7f8b57fff640 1 -- 172.21.15.47:0/1403990944 learned_addr learned my addr 172.21.15.47:0/1403990944 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:00:42.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.993+0000 7f8b56ffd640 1 --2- 172.21.15.47:0/1403990944 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8b58101820 0x7f8b58198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:42.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.993+0000 7f8b577fe640 1 --2- 172.21.15.47:0/1403990944 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8b58100630 0x7f8b581983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:42.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.993+0000 7f8b57fff640 1 -- 172.21.15.47:0/1403990944 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8b58101820 msgr2=0x7f8b58198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:42.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.993+0000 7f8b57fff640 1 --2- 172.21.15.47:0/1403990944 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8b58101820 0x7f8b58198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:42.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.993+0000 7f8b57fff640 1 -- 172.21.15.47:0/1403990944 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8b58100630 msgr2=0x7f8b581983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:42.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.993+0000 7f8b57fff640 1 --2- 172.21.15.47:0/1403990944 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8b58100630 0x7f8b581983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:42.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.993+0000 7f8b57fff640 1 -- 172.21.15.47:0/1403990944 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b5819fff0 con 0x7f8b581021e0 2024-04-17T06:00:42.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.993+0000 7f8b57fff640 1 --2- 172.21.15.47:0/1403990944 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b581021e0 0x7f8b5819f980 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f8b5000eaa0 tx=0x7f8b5000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:00:42.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.993+0000 7f8b54ff9640 1 -- 172.21.15.47:0/1403990944 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b5000cdb0 con 0x7f8b581021e0 2024-04-17T06:00:42.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.993+0000 7f8b54ff9640 1 -- 172.21.15.47:0/1403990944 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b50005020 con 0x7f8b581021e0 2024-04-17T06:00:42.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.993+0000 7f8b5db0e640 1 -- 172.21.15.47:0/1403990944 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b581a02e0 con 0x7f8b581021e0 2024-04-17T06:00:42.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.993+0000 7f8b54ff9640 1 -- 172.21.15.47:0/1403990944 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b500116b0 con 0x7f8b581021e0 2024-04-17T06:00:42.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.993+0000 7f8b5db0e640 1 -- 172.21.15.47:0/1403990944 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b581a07e0 con 0x7f8b581021e0 2024-04-17T06:00:42.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.994+0000 7f8b5db0e640 1 -- 172.21.15.47:0/1403990944 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b581078f0 con 0x7f8b581021e0 2024-04-17T06:00:42.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.995+0000 7f8b54ff9640 1 -- 172.21.15.47:0/1403990944 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f8b50005560 con 0x7f8b581021e0 2024-04-17T06:00:42.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.996+0000 7f8b54ff9640 1 --2- 172.21.15.47:0/1403990944 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f8b400a3c40 0x7f8b400a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:42.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.996+0000 7f8b577fe640 1 --2- 172.21.15.47:0/1403990944 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f8b400a3c40 0x7f8b400a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:42.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.996+0000 7f8b54ff9640 1 -- 172.21.15.47:0/1403990944 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f8b500d08d0 con 0x7f8b581021e0 2024-04-17T06:00:42.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:42.997+0000 7f8b577fe640 1 --2- 172.21.15.47:0/1403990944 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f8b400a3c40 0x7f8b400a6100 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8b58101680 tx=0x7f8b48008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:00:42.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.001+0000 7f8b54ff9640 1 -- 172.21.15.47:0/1403990944 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f8b50016030 con 0x7f8b581021e0 2024-04-17T06:00:43.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:00:42 smithi098 ceph-mon[36516]: pgmap v332: 97 pgs: 97 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-17T06:00:43.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:00:42 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/937185109' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T06:00:43.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:00:42 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/290675453' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T06:00:43.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:00:42 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/4181596047' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-17T06:00:43.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:00:42 smithi047 ceph-mon[32223]: pgmap v332: 97 pgs: 97 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-17T06:00:43.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:00:42 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/937185109' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T06:00:43.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:00:42 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/290675453' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T06:00:43.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:00:42 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/4181596047' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-17T06:00:43.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.187+0000 7f8b5db0e640 1 -- 172.21.15.47:0/1403990944 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f8b58062b60 con 0x7f8b581021e0 2024-04-17T06:00:43.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.188+0000 7f8b54ff9640 1 -- 172.21.15.47:0/1403990944 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7f8b5009d100 con 0x7f8b581021e0 2024-04-17T06:00:43.186 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-04-17T06:00:43.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.190+0000 7f8b5db0e640 1 -- 172.21.15.47:0/1403990944 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f8b400a3c40 msgr2=0x7f8b400a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:43.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.190+0000 7f8b5db0e640 1 --2- 172.21.15.47:0/1403990944 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f8b400a3c40 0x7f8b400a6100 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8b58101680 tx=0x7f8b48008040 comp rx=0 tx=0).stop 2024-04-17T06:00:43.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.190+0000 7f8b5db0e640 1 -- 172.21.15.47:0/1403990944 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b581021e0 msgr2=0x7f8b5819f980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:43.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.190+0000 7f8b5db0e640 1 --2- 172.21.15.47:0/1403990944 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b581021e0 0x7f8b5819f980 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f8b5000eaa0 tx=0x7f8b5000ef60 comp rx=0 tx=0).stop 2024-04-17T06:00:43.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.190+0000 7f8b5db0e640 1 -- 172.21.15.47:0/1403990944 shutdown_connections 2024-04-17T06:00:43.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.190+0000 7f8b5db0e640 1 --2- 172.21.15.47:0/1403990944 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f8b400a3c40 0x7f8b400a6100 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:43.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.190+0000 7f8b5db0e640 1 --2- 172.21.15.47:0/1403990944 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b581021e0 0x7f8b5819f980 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:43.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.190+0000 7f8b5db0e640 1 --2- 172.21.15.47:0/1403990944 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8b58101820 0x7f8b58198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:43.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.190+0000 7f8b5db0e640 1 --2- 172.21.15.47:0/1403990944 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8b58100630 0x7f8b581983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:43.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.190+0000 7f8b5db0e640 1 -- 172.21.15.47:0/1403990944 >> 172.21.15.47:0/1403990944 conn(0x7f8b580fbba0 msgr2=0x7f8b58104ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:00:43.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.190+0000 7f8b5db0e640 1 -- 172.21.15.47:0/1403990944 shutdown_connections 2024-04-17T06:00:43.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.190+0000 7f8b5db0e640 1 -- 172.21.15.47:0/1403990944 wait complete. 2024-04-17T06:00:43.200 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T06:00:43.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:00:42 smithi103 ceph-mon[36243]: pgmap v332: 97 pgs: 97 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-17T06:00:43.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:00:42 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/937185109' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T06:00:43.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:00:42 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/290675453' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T06:00:43.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:00:42 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/4181596047' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-17T06:00:43.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.308+0000 7fcaf364a640 1 -- 172.21.15.47:0/4198484285 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcaec105350 msgr2=0x7fcaec107740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:43.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.308+0000 7fcaf364a640 1 --2- 172.21.15.47:0/4198484285 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcaec105350 0x7fcaec107740 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fcadc009900 tx=0x7fcadc02bc30 comp rx=0 tx=0).stop 2024-04-17T06:00:43.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.308+0000 7fcaf364a640 1 -- 172.21.15.47:0/4198484285 shutdown_connections 2024-04-17T06:00:43.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.308+0000 7fcaf364a640 1 --2- 172.21.15.47:0/4198484285 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcaec107c80 0x7fcaec10a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:43.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.308+0000 7fcaf364a640 1 --2- 172.21.15.47:0/4198484285 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcaec105350 0x7fcaec107740 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:43.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.308+0000 7fcaf364a640 1 --2- 172.21.15.47:0/4198484285 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcaec102a20 0x7fcaec104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:43.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.308+0000 7fcaf364a640 1 -- 172.21.15.47:0/4198484285 >> 172.21.15.47:0/4198484285 conn(0x7fcaec0fc410 msgr2=0x7fcaec0fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:00:43.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.308+0000 7fcaf364a640 1 -- 172.21.15.47:0/4198484285 shutdown_connections 2024-04-17T06:00:43.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.308+0000 7fcaf364a640 1 -- 172.21.15.47:0/4198484285 wait complete. 2024-04-17T06:00:43.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.308+0000 7fcaf364a640 1 Processor -- start 2024-04-17T06:00:43.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.308+0000 7fcaf364a640 1 -- start start 2024-04-17T06:00:43.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.308+0000 7fcaf364a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcaec102a20 0x7fcaec1982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:43.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.308+0000 7fcaf364a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcaec105350 0x7fcaec198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:43.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.308+0000 7fcaf364a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcaec107c80 0x7fcaec19f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:43.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.308+0000 7fcaf364a640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fcaec10dc40 con 0x7fcaec102a20 2024-04-17T06:00:43.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.308+0000 7fcaf364a640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fcaec10dac0 con 0x7fcaec105350 2024-04-17T06:00:43.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.308+0000 7fcaf364a640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcaec10ddc0 con 0x7fcaec107c80 2024-04-17T06:00:43.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.309+0000 7fcaf13e6640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcaec102a20 0x7fcaec1982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:43.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.309+0000 7fcaf13e6640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcaec102a20 0x7fcaec1982e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57432/0 (socket says 172.21.15.47:57432) 2024-04-17T06:00:43.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.309+0000 7fcaf13e6640 1 -- 172.21.15.47:0/3965002100 learned_addr learned my addr 172.21.15.47:0/3965002100 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:00:43.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.309+0000 7fcaf0be5640 1 --2- 172.21.15.47:0/3965002100 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcaec105350 0x7fcaec198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:43.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.309+0000 7fcaf1be7640 1 --2- 172.21.15.47:0/3965002100 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcaec107c80 0x7fcaec19f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:43.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.309+0000 7fcaf13e6640 1 -- 172.21.15.47:0/3965002100 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcaec107c80 msgr2=0x7fcaec19f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:43.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.309+0000 7fcaf13e6640 1 --2- 172.21.15.47:0/3965002100 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcaec107c80 0x7fcaec19f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:43.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.309+0000 7fcaf13e6640 1 -- 172.21.15.47:0/3965002100 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcaec105350 msgr2=0x7fcaec198820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:43.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.309+0000 7fcaf13e6640 1 --2- 172.21.15.47:0/3965002100 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcaec105350 0x7fcaec198820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:43.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.309+0000 7fcaf13e6640 1 -- 172.21.15.47:0/3965002100 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaec19ff00 con 0x7fcaec102a20 2024-04-17T06:00:43.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.309+0000 7fcaf0be5640 1 --2- 172.21.15.47:0/3965002100 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcaec105350 0x7fcaec198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:00:43.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.309+0000 7fcaf1be7640 1 --2- 172.21.15.47:0/3965002100 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcaec107c80 0x7fcaec19f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:00:43.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.309+0000 7fcaf13e6640 1 --2- 172.21.15.47:0/3965002100 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcaec102a20 0x7fcaec1982e0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fcae800d950 tx=0x7fcae800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:00:43.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.309+0000 7fcae27fc640 1 -- 172.21.15.47:0/3965002100 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae8017030 con 0x7fcaec102a20 2024-04-17T06:00:43.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.309+0000 7fcae27fc640 1 -- 172.21.15.47:0/3965002100 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcae8004020 con 0x7fcaec102a20 2024-04-17T06:00:43.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.309+0000 7fcaf364a640 1 -- 172.21.15.47:0/3965002100 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcaec1a01f0 con 0x7fcaec102a20 2024-04-17T06:00:43.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.309+0000 7fcae27fc640 1 -- 172.21.15.47:0/3965002100 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae80047a0 con 0x7fcaec102a20 2024-04-17T06:00:43.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.309+0000 7fcaf364a640 1 -- 172.21.15.47:0/3965002100 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcaec1a0710 con 0x7fcaec102a20 2024-04-17T06:00:43.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.310+0000 7fcaf364a640 1 -- 172.21.15.47:0/3965002100 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcaec0fdf00 con 0x7fcaec102a20 2024-04-17T06:00:43.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.311+0000 7fcae27fc640 1 -- 172.21.15.47:0/3965002100 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fcae80041d0 con 0x7fcaec102a20 2024-04-17T06:00:43.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.312+0000 7fcae27fc640 1 --2- 172.21.15.47:0/3965002100 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fcad80a3c40 0x7fcad80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:43.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.312+0000 7fcaf0be5640 1 --2- 172.21.15.47:0/3965002100 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fcad80a3c40 0x7fcad80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:43.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.312+0000 7fcae27fc640 1 -- 172.21.15.47:0/3965002100 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fcae80d0130 con 0x7fcaec102a20 2024-04-17T06:00:43.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.312+0000 7fcaf0be5640 1 --2- 172.21.15.47:0/3965002100 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fcad80a3c40 0x7fcad80a6100 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fcadc02c170 tx=0x7fcadc02df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:00:43.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.317+0000 7fcae27fc640 1 -- 172.21.15.47:0/3965002100 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fcae809c960 con 0x7fcaec102a20 2024-04-17T06:00:43.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.541+0000 7fcaf364a640 1 -- 172.21.15.47:0/3965002100 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcaec02d5d0 con 0x7fcaec102a20 2024-04-17T06:00:43.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.542+0000 7fcae27fc640 1 -- 172.21.15.47:0/3965002100 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11881 (secure 0 0 0) 0x7fcae809c260 con 0x7fcaec102a20 2024-04-17T06:00:43.541 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-17T06:00:43.541 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3189958720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3189958720},{"type":"v1","addr":"172.21.15.47:6835","nonce":3189958720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14595,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/3143529973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":3143529973},{"type":"v1","addr":"172.21.15.47:6837","nonce":3143529973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14652,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/2375823701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":2375823701},{"type":"v1","addr":"172.21.15.47:6839","nonce":2375823701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14664,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6841/3334309157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":3334309157},{"type":"v1","addr":"172.21.15.47:6841","nonce":3334309157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/2398363754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":2398363754},{"type":"v1","addr":"172.21.15.103:6833","nonce":2398363754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/384473493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":384473493},{"type":"v1","addr":"172.21.15.98:6833","nonce":384473493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T05:59:13.922176+0000","modified":"2024-04-17T05:59:19.598261+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24518,"mds_1":24479,"mds_2":24466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6835/270543522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":270543522},{"type":"v1","addr":"172.21.15.103:6835","nonce":270543522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6837/116936024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":116936024},{"type":"v1","addr":"172.21.15.103:6837","nonce":116936024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6835/1966500278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":1966500278},{"type":"v1","addr":"172.21.15.98:6835","nonce":1966500278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.103:6839/1467157889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":1467157889},{"type":"v1","addr":"172.21.15.103:6839","nonce":1467157889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.98:6837/1117440719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1117440719},{"type":"v1","addr":"172.21.15.98:6837","nonce":1117440719}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6839/2841287917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":2841287917},{"type":"v1","addr":"172.21.15.98:6839","nonce":2841287917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24518,"qdb_cluster":[24479,24518,24466]},"id":1}]} 2024-04-17T06:00:43.541 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 21 2024-04-17T06:00:43.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.544+0000 7fcaf364a640 1 -- 172.21.15.47:0/3965002100 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fcad80a3c40 msgr2=0x7fcad80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:43.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.544+0000 7fcaf364a640 1 --2- 172.21.15.47:0/3965002100 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fcad80a3c40 0x7fcad80a6100 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fcadc02c170 tx=0x7fcadc02df90 comp rx=0 tx=0).stop 2024-04-17T06:00:43.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.544+0000 7fcaf364a640 1 -- 172.21.15.47:0/3965002100 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcaec102a20 msgr2=0x7fcaec1982e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:43.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.544+0000 7fcaf364a640 1 --2- 172.21.15.47:0/3965002100 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcaec102a20 0x7fcaec1982e0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fcae800d950 tx=0x7fcae800de10 comp rx=0 tx=0).stop 2024-04-17T06:00:43.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.544+0000 7fcaf364a640 1 -- 172.21.15.47:0/3965002100 shutdown_connections 2024-04-17T06:00:43.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.544+0000 7fcaf364a640 1 --2- 172.21.15.47:0/3965002100 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fcad80a3c40 0x7fcad80a6100 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:43.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.544+0000 7fcaf364a640 1 --2- 172.21.15.47:0/3965002100 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcaec107c80 0x7fcaec19f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:43.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.544+0000 7fcaf364a640 1 --2- 172.21.15.47:0/3965002100 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcaec105350 0x7fcaec198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:43.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.544+0000 7fcaf364a640 1 --2- 172.21.15.47:0/3965002100 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcaec102a20 0x7fcaec1982e0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:43.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.544+0000 7fcaf364a640 1 -- 172.21.15.47:0/3965002100 >> 172.21.15.47:0/3965002100 conn(0x7fcaec0fc410 msgr2=0x7fcaec103340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:00:43.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.544+0000 7fcaf364a640 1 -- 172.21.15.47:0/3965002100 shutdown_connections 2024-04-17T06:00:43.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.545+0000 7fcaf364a640 1 -- 172.21.15.47:0/3965002100 wait complete. 2024-04-17T06:00:43.555 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T06:00:43.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.662+0000 7fa406060640 1 -- 172.21.15.47:0/164758980 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa400101fd0 msgr2=0x7fa4001023f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:43.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.662+0000 7fa406060640 1 --2- 172.21.15.47:0/164758980 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa400101fd0 0x7fa4001023f0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fa3f0009ca0 tx=0x7fa3f002c0f0 comp rx=0 tx=0).stop 2024-04-17T06:00:43.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.662+0000 7fa406060640 1 -- 172.21.15.47:0/164758980 shutdown_connections 2024-04-17T06:00:43.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.662+0000 7fa406060640 1 --2- 172.21.15.47:0/164758980 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa400103b80 0x7fa40010a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:43.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.662+0000 7fa406060640 1 --2- 172.21.15.47:0/164758980 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa4001031c0 0x7fa400103640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:43.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.662+0000 7fa406060640 1 --2- 172.21.15.47:0/164758980 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa400101fd0 0x7fa4001023f0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:43.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.662+0000 7fa406060640 1 -- 172.21.15.47:0/164758980 >> 172.21.15.47:0/164758980 conn(0x7fa4000fd540 msgr2=0x7fa4000ff980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:00:43.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.662+0000 7fa406060640 1 -- 172.21.15.47:0/164758980 shutdown_connections 2024-04-17T06:00:43.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.662+0000 7fa406060640 1 -- 172.21.15.47:0/164758980 wait complete. 2024-04-17T06:00:43.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.662+0000 7fa406060640 1 Processor -- start 2024-04-17T06:00:43.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.662+0000 7fa406060640 1 -- start start 2024-04-17T06:00:43.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.663+0000 7fa406060640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa400101fd0 0x7fa40019a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:43.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.663+0000 7fa406060640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4001031c0 0x7fa40019ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:43.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.663+0000 7fa406060640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa400103b80 0x7fa4001a1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:43.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.663+0000 7fa406060640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa40010dd20 con 0x7fa4001031c0 2024-04-17T06:00:43.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.663+0000 7fa406060640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa40010dba0 con 0x7fa400103b80 2024-04-17T06:00:43.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.663+0000 7fa406060640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa40010dea0 con 0x7fa400101fd0 2024-04-17T06:00:43.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.663+0000 7fa3feffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4001031c0 0x7fa40019ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:43.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.663+0000 7fa3feffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4001031c0 0x7fa40019ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57438/0 (socket says 172.21.15.47:57438) 2024-04-17T06:00:43.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.663+0000 7fa3feffd640 1 -- 172.21.15.47:0/3460394627 learned_addr learned my addr 172.21.15.47:0/3460394627 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:00:43.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.663+0000 7fa3ff7fe640 1 --2- 172.21.15.47:0/3460394627 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa400101fd0 0x7fa40019a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:43.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.663+0000 7fa3fffff640 1 --2- 172.21.15.47:0/3460394627 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa400103b80 0x7fa4001a1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:43.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.663+0000 7fa3feffd640 1 -- 172.21.15.47:0/3460394627 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa400101fd0 msgr2=0x7fa40019a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:43.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.663+0000 7fa3feffd640 1 --2- 172.21.15.47:0/3460394627 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa400101fd0 0x7fa40019a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:43.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.663+0000 7fa3feffd640 1 -- 172.21.15.47:0/3460394627 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa400103b80 msgr2=0x7fa4001a1bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:43.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.663+0000 7fa3feffd640 1 --2- 172.21.15.47:0/3460394627 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa400103b80 0x7fa4001a1bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:43.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.663+0000 7fa3feffd640 1 -- 172.21.15.47:0/3460394627 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4001a2220 con 0x7fa4001031c0 2024-04-17T06:00:43.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.663+0000 7fa3feffd640 1 --2- 172.21.15.47:0/3460394627 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4001031c0 0x7fa40019ab40 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa3e800b4a0 tx=0x7fa3e800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:00:43.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.663+0000 7fa3fcff9640 1 -- 172.21.15.47:0/3460394627 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e8014070 con 0x7fa4001031c0 2024-04-17T06:00:43.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.663+0000 7fa3fcff9640 1 -- 172.21.15.47:0/3460394627 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3e8010af0 con 0x7fa4001031c0 2024-04-17T06:00:43.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.663+0000 7fa406060640 1 -- 172.21.15.47:0/3460394627 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4001a2510 con 0x7fa4001031c0 2024-04-17T06:00:43.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.663+0000 7fa3fcff9640 1 -- 172.21.15.47:0/3460394627 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e8017d10 con 0x7fa4001031c0 2024-04-17T06:00:43.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.663+0000 7fa406060640 1 -- 172.21.15.47:0/3460394627 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4001a2b30 con 0x7fa4001031c0 2024-04-17T06:00:43.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.664+0000 7fa406060640 1 -- 172.21.15.47:0/3460394627 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4001a2df0 con 0x7fa4001031c0 2024-04-17T06:00:43.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.666+0000 7fa3fcff9640 1 -- 172.21.15.47:0/3460394627 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fa3e8017070 con 0x7fa4001031c0 2024-04-17T06:00:43.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.666+0000 7fa3fcff9640 1 --2- 172.21.15.47:0/3460394627 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fa3ec0a3cc0 0x7fa3ec0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:43.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.667+0000 7fa3fcff9640 1 -- 172.21.15.47:0/3460394627 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fa3e80cfc50 con 0x7fa4001031c0 2024-04-17T06:00:43.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.667+0000 7fa3ff7fe640 1 --2- 172.21.15.47:0/3460394627 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fa3ec0a3cc0 0x7fa3ec0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:43.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.667+0000 7fa3ff7fe640 1 --2- 172.21.15.47:0/3460394627 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fa3ec0a3cc0 0x7fa3ec0a6180 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa3f002c700 tx=0x7fa3f002cca0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:00:43.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.670+0000 7fa3fcff9640 1 -- 172.21.15.47:0/3460394627 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fa3e809c5f0 con 0x7fa4001031c0 2024-04-17T06:00:43.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.867+0000 7fa3fcff9640 1 -- 172.21.15.47:0/3460394627 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fa3e8010ca0 con 0x7fa4001031c0 2024-04-17T06:00:43.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.891+0000 7fa406060640 1 -- 172.21.15.47:0/3460394627 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa4001a2fc0 con 0x7fa4001031c0 2024-04-17T06:00:43.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.892+0000 7fa3fcff9640 1 -- 172.21.15.47:0/3460394627 <== mon.0 v2:172.21.15.47:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22602 (secure 0 0 0) 0x7fa3e80c85c0 con 0x7fa4001031c0 2024-04-17T06:00:43.891 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-17T06:00:43.891 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":88,"fsid":"565db2d8-fc7e-11ee-bc91-c7b262605968","created":"2024-04-17T05:51:19.888865+0000","modified":"2024-04-17T06:00:14.720609+0000","last_up_change":"2024-04-17T05:58:24.380131+0000","last_in_change":"2024-04-17T05:58:04.418236+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T05:54:37.734306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T05:59:12.458010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T05:59:12.901339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8453f300-533d-4736-a343-55619c08e966","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":1290127544},{"type":"v1","addr":"172.21.15.47:6803","nonce":1290127544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":1290127544},{"type":"v1","addr":"172.21.15.47:6805","nonce":1290127544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":1290127544},{"type":"v1","addr":"172.21.15.47:6809","nonce":1290127544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":1290127544},{"type":"v1","addr":"172.21.15.47:6807","nonce":1290127544}]},"public_addr":"172.21.15.47:6803/1290127544","cluster_addr":"172.21.15.47:6805/1290127544","heartbeat_back_addr":"172.21.15.47:6809/1290127544","heartbeat_front_addr":"172.21.15.47:6807/1290127544","state":["exists","up"]},{"osd":1,"uuid":"80b0f03e-c1bc-4658-8028-1e6b8eedcc47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":1180430954},{"type":"v1","addr":"172.21.15.98:6801","nonce":1180430954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":1180430954},{"type":"v1","addr":"172.21.15.98:6803","nonce":1180430954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":1180430954},{"type":"v1","addr":"172.21.15.98:6807","nonce":1180430954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":1180430954},{"type":"v1","addr":"172.21.15.98:6805","nonce":1180430954}]},"public_addr":"172.21.15.98:6801/1180430954","cluster_addr":"172.21.15.98:6803/1180430954","heartbeat_back_addr":"172.21.15.98:6807/1180430954","heartbeat_front_addr":"172.21.15.98:6805/1180430954","state":["exists","up"]},{"osd":2,"uuid":"0fc9d383-05b4-498d-b530-fccf7aeddd7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":494676216},{"type":"v1","addr":"172.21.15.103:6801","nonce":494676216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":494676216},{"type":"v1","addr":"172.21.15.103:6803","nonce":494676216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":494676216},{"type":"v1","addr":"172.21.15.103:6807","nonce":494676216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":494676216},{"type":"v1","addr":"172.21.15.103:6805","nonce":494676216}]},"public_addr":"172.21.15.103:6801/494676216","cluster_addr":"172.21.15.103:6803/494676216","heartbeat_back_addr":"172.21.15.103:6807/494676216","heartbeat_front_addr":"172.21.15.103:6805/494676216","state":["exists","up"]},{"osd":3,"uuid":"f6e0b0d5-0fb3-42d0-9e84-f7cfef21a998","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":1998369447},{"type":"v1","addr":"172.21.15.47:6811","nonce":1998369447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":1998369447},{"type":"v1","addr":"172.21.15.47:6813","nonce":1998369447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":1998369447},{"type":"v1","addr":"172.21.15.47:6817","nonce":1998369447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":1998369447},{"type":"v1","addr":"172.21.15.47:6815","nonce":1998369447}]},"public_addr":"172.21.15.47:6811/1998369447","cluster_addr":"172.21.15.47:6813/1998369447","heartbeat_back_addr":"172.21.15.47:6817/1998369447","heartbeat_front_addr":"172.21.15.47:6815/1998369447","state":["exists","up"]},{"osd":4,"uuid":"2c26217e-02df-466d-bc01-0149d4aec3c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":3060482485},{"type":"v1","addr":"172.21.15.98:6809","nonce":3060482485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":3060482485},{"type":"v1","addr":"172.21.15.98:6811","nonce":3060482485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":3060482485},{"type":"v1","addr":"172.21.15.98:6815","nonce":3060482485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":3060482485},{"type":"v1","addr":"172.21.15.98:6813","nonce":3060482485}]},"public_addr":"172.21.15.98:6809/3060482485","cluster_addr":"172.21.15.98:6811/3060482485","heartbeat_back_addr":"172.21.15.98:6815/3060482485","heartbeat_front_addr":"172.21.15.98:6813/3060482485","state":["exists","up"]},{"osd":5,"uuid":"740bf3e0-bea8-496b-8d6f-e42d0b017258","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":4181363567},{"type":"v1","addr":"172.21.15.103:6809","nonce":4181363567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":4181363567},{"type":"v1","addr":"172.21.15.103:6811","nonce":4181363567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":4181363567},{"type":"v1","addr":"172.21.15.103:6815","nonce":4181363567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":4181363567},{"type":"v1","addr":"172.21.15.103:6813","nonce":4181363567}]},"public_addr":"172.21.15.103:6809/4181363567","cluster_addr":"172.21.15.103:6811/4181363567","heartbeat_back_addr":"172.21.15.103:6815/4181363567","heartbeat_front_addr":"172.21.15.103:6813/4181363567","state":["exists","up"]},{"osd":6,"uuid":"356a6255-2c07-4f0d-a233-26be04c92b38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":2517529668},{"type":"v1","addr":"172.21.15.47:6819","nonce":2517529668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2517529668},{"type":"v1","addr":"172.21.15.47:6821","nonce":2517529668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2517529668},{"type":"v1","addr":"172.21.15.47:6825","nonce":2517529668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2517529668},{"type":"v1","addr":"172.21.15.47:6823","nonce":2517529668}]},"public_addr":"172.21.15.47:6819/2517529668","cluster_addr":"172.21.15.47:6821/2517529668","heartbeat_back_addr":"172.21.15.47:6825/2517529668","heartbeat_front_addr":"172.21.15.47:6823/2517529668","state":["exists","up"]},{"osd":7,"uuid":"349d0120-8290-417d-a800-5a8985f554f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":4225498496},{"type":"v1","addr":"172.21.15.98:6817","nonce":4225498496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":4225498496},{"type":"v1","addr":"172.21.15.98:6819","nonce":4225498496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":4225498496},{"type":"v1","addr":"172.21.15.98:6823","nonce":4225498496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":4225498496},{"type":"v1","addr":"172.21.15.98:6821","nonce":4225498496}]},"public_addr":"172.21.15.98:6817/4225498496","cluster_addr":"172.21.15.98:6819/4225498496","heartbeat_back_addr":"172.21.15.98:6823/4225498496","heartbeat_front_addr":"172.21.15.98:6821/4225498496","state":["exists","up"]},{"osd":8,"uuid":"cee4712f-321b-42c0-92ba-e15144c00f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":3433905520},{"type":"v1","addr":"172.21.15.103:6817","nonce":3433905520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":3433905520},{"type":"v1","addr":"172.21.15.103:6819","nonce":3433905520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":3433905520},{"type":"v1","addr":"172.21.15.103:6823","nonce":3433905520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":3433905520},{"type":"v1","addr":"172.21.15.103:6821","nonce":3433905520}]},"public_addr":"172.21.15.103:6817/3433905520","cluster_addr":"172.21.15.103:6819/3433905520","heartbeat_back_addr":"172.21.15.103:6823/3433905520","heartbeat_front_addr":"172.21.15.103:6821/3433905520","state":["exists","up"]},{"osd":9,"uuid":"63fc73c1-5a22-487d-80b8-9894e2ddd207","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":728359554},{"type":"v1","addr":"172.21.15.47:6827","nonce":728359554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":728359554},{"type":"v1","addr":"172.21.15.47:6829","nonce":728359554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":728359554},{"type":"v1","addr":"172.21.15.47:6833","nonce":728359554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":728359554},{"type":"v1","addr":"172.21.15.47:6831","nonce":728359554}]},"public_addr":"172.21.15.47:6827/728359554","cluster_addr":"172.21.15.47:6829/728359554","heartbeat_back_addr":"172.21.15.47:6833/728359554","heartbeat_front_addr":"172.21.15.47:6831/728359554","state":["exists","up"]},{"osd":10,"uuid":"9565bcc3-62ee-48eb-a07a-03a81e0cc8e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":468205689},{"type":"v1","addr":"172.21.15.98:6825","nonce":468205689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":468205689},{"type":"v1","addr":"172.21.15.98:6827","nonce":468205689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":468205689},{"type":"v1","addr":"172.21.15.98:6831","nonce":468205689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":468205689},{"type":"v1","addr":"172.21.15.98:6829","nonce":468205689}]},"public_addr":"172.21.15.98:6825/468205689","cluster_addr":"172.21.15.98:6827/468205689","heartbeat_back_addr":"172.21.15.98:6831/468205689","heartbeat_front_addr":"172.21.15.98:6829/468205689","state":["exists","up"]},{"osd":11,"uuid":"2e85e8d9-bf6c-4a12-b124-a4690dd61108","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":2639831013},{"type":"v1","addr":"172.21.15.103:6825","nonce":2639831013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":2639831013},{"type":"v1","addr":"172.21.15.103:6827","nonce":2639831013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":2639831013},{"type":"v1","addr":"172.21.15.103:6831","nonce":2639831013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":2639831013},{"type":"v1","addr":"172.21.15.103:6829","nonce":2639831013}]},"public_addr":"172.21.15.103:6825/2639831013","cluster_addr":"172.21.15.103:6827/2639831013","heartbeat_back_addr":"172.21.15.103:6831/2639831013","heartbeat_front_addr":"172.21.15.103:6829/2639831013","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:53:47.366438+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:54:10.953559+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:54:34.320332+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:54:59.751360+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:55:23.982052+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:55:48.425017+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:56:14.793368+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:56:40.440830+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:57:04.623680+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:57:31.215262+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:57:57.040346+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:58:22.671572+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:6801/3632157175":"2024-04-18T05:52:13.645775+0000","172.21.15.47:0/4288219269":"2024-04-18T05:52:13.645775+0000","172.21.15.47:0/2584487544":"2024-04-18T05:52:13.645775+0000","172.21.15.47:6801/953064786":"2024-04-18T05:51:45.979351+0000","172.21.15.47:0/3484500927":"2024-04-18T05:51:45.979351+0000","172.21.15.47:6800/3632157175":"2024-04-18T05:52:13.645775+0000","172.21.15.47:0/112268670":"2024-04-18T05:51:45.979351+0000","172.21.15.47:0/1638423256":"2024-04-18T05:51:45.979351+0000","172.21.15.47:6800/953064786":"2024-04-18T05:51:45.979351+0000","172.21.15.47:0/849479010":"2024-04-18T05:52:13.645775+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T06:00:43.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.894+0000 7fa406060640 1 -- 172.21.15.47:0/3460394627 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fa3ec0a3cc0 msgr2=0x7fa3ec0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:43.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.894+0000 7fa406060640 1 --2- 172.21.15.47:0/3460394627 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fa3ec0a3cc0 0x7fa3ec0a6180 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa3f002c700 tx=0x7fa3f002cca0 comp rx=0 tx=0).stop 2024-04-17T06:00:43.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.894+0000 7fa406060640 1 -- 172.21.15.47:0/3460394627 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4001031c0 msgr2=0x7fa40019ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:43.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.894+0000 7fa406060640 1 --2- 172.21.15.47:0/3460394627 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4001031c0 0x7fa40019ab40 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa3e800b4a0 tx=0x7fa3e800b960 comp rx=0 tx=0).stop 2024-04-17T06:00:43.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.894+0000 7fa406060640 1 -- 172.21.15.47:0/3460394627 shutdown_connections 2024-04-17T06:00:43.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.894+0000 7fa406060640 1 --2- 172.21.15.47:0/3460394627 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fa3ec0a3cc0 0x7fa3ec0a6180 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:43.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.894+0000 7fa406060640 1 --2- 172.21.15.47:0/3460394627 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa400103b80 0x7fa4001a1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:43.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.894+0000 7fa406060640 1 --2- 172.21.15.47:0/3460394627 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4001031c0 0x7fa40019ab40 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:43.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.894+0000 7fa406060640 1 --2- 172.21.15.47:0/3460394627 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa400101fd0 0x7fa40019a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:43.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.894+0000 7fa406060640 1 -- 172.21.15.47:0/3460394627 >> 172.21.15.47:0/3460394627 conn(0x7fa4000fd540 msgr2=0x7fa400074190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:00:43.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.894+0000 7fa406060640 1 -- 172.21.15.47:0/3460394627 shutdown_connections 2024-04-17T06:00:43.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:43.894+0000 7fa406060640 1 -- 172.21.15.47:0/3460394627 wait complete. 2024-04-17T06:00:43.905 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T06:00:44.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.012+0000 7ff7d7c2a640 1 -- 172.21.15.47:0/1306030365 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7d0100ea0 msgr2=0x7ff7d01012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:44.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.012+0000 7ff7d7c2a640 1 --2- 172.21.15.47:0/1306030365 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7d0100ea0 0x7ff7d01012c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7ff7c4009900 tx=0x7ff7c402bc30 comp rx=0 tx=0).stop 2024-04-17T06:00:44.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.012+0000 7ff7d7c2a640 1 -- 172.21.15.47:0/1306030365 shutdown_connections 2024-04-17T06:00:44.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.012+0000 7ff7d7c2a640 1 --2- 172.21.15.47:0/1306030365 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff7d01029b0 0x7ff7d01092d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:44.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.012+0000 7ff7d7c2a640 1 --2- 172.21.15.47:0/1306030365 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff7d0101ff0 0x7ff7d0102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:44.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.012+0000 7ff7d7c2a640 1 --2- 172.21.15.47:0/1306030365 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7d0100ea0 0x7ff7d01012c0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:44.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.012+0000 7ff7d7c2a640 1 -- 172.21.15.47:0/1306030365 >> 172.21.15.47:0/1306030365 conn(0x7ff7d00fc410 msgr2=0x7ff7d00fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:00:44.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.012+0000 7ff7d7c2a640 1 -- 172.21.15.47:0/1306030365 shutdown_connections 2024-04-17T06:00:44.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.012+0000 7ff7d7c2a640 1 -- 172.21.15.47:0/1306030365 wait complete. 2024-04-17T06:00:44.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.012+0000 7ff7d7c2a640 1 Processor -- start 2024-04-17T06:00:44.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.012+0000 7ff7d7c2a640 1 -- start start 2024-04-17T06:00:44.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.012+0000 7ff7d7c2a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7d0100ea0 0x7ff7d0069c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:44.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.013+0000 7ff7d7c2a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff7d0101ff0 0x7ff7d019d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:44.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.013+0000 7ff7d7c2a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff7d01029b0 0x7ff7d019f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:44.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.013+0000 7ff7d7c2a640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff7d010cba0 con 0x7ff7d0100ea0 2024-04-17T06:00:44.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.013+0000 7ff7d7c2a640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff7d010ca20 con 0x7ff7d01029b0 2024-04-17T06:00:44.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.013+0000 7ff7d7c2a640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff7d010cd20 con 0x7ff7d0101ff0 2024-04-17T06:00:44.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.013+0000 7ff7d59c6640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7d0100ea0 0x7ff7d0069c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:44.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.013+0000 7ff7d59c6640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7d0100ea0 0x7ff7d0069c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57460/0 (socket says 172.21.15.47:57460) 2024-04-17T06:00:44.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.013+0000 7ff7d59c6640 1 -- 172.21.15.47:0/1594618039 learned_addr learned my addr 172.21.15.47:0/1594618039 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:00:44.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.013+0000 7ff7d51c5640 1 --2- 172.21.15.47:0/1594618039 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff7d0101ff0 0x7ff7d019d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:44.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.013+0000 7ff7d59c6640 1 -- 172.21.15.47:0/1594618039 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff7d0101ff0 msgr2=0x7ff7d019d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:44.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.013+0000 7ff7d59c6640 1 --2- 172.21.15.47:0/1594618039 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff7d0101ff0 0x7ff7d019d000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:44.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.013+0000 7ff7d59c6640 1 -- 172.21.15.47:0/1594618039 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff7d01029b0 msgr2=0x7ff7d019f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:44.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.013+0000 7ff7d59c6640 1 --2- 172.21.15.47:0/1594618039 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff7d01029b0 0x7ff7d019f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:44.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.013+0000 7ff7d59c6640 1 -- 172.21.15.47:0/1594618039 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7d019ff30 con 0x7ff7d0100ea0 2024-04-17T06:00:44.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.013+0000 7ff7d59c6640 1 --2- 172.21.15.47:0/1594618039 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7d0100ea0 0x7ff7d0069c60 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7ff7c4009230 tx=0x7ff7c4004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:00:44.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.013+0000 7ff7c2ffd640 1 -- 172.21.15.47:0/1594618039 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7c4041070 con 0x7ff7d0100ea0 2024-04-17T06:00:44.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.013+0000 7ff7c2ffd640 1 -- 172.21.15.47:0/1594618039 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7c40054e0 con 0x7ff7d0100ea0 2024-04-17T06:00:44.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.013+0000 7ff7d7c2a640 1 -- 172.21.15.47:0/1594618039 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7d01a0160 con 0x7ff7d0100ea0 2024-04-17T06:00:44.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.013+0000 7ff7c2ffd640 1 -- 172.21.15.47:0/1594618039 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7c40346a0 con 0x7ff7d0100ea0 2024-04-17T06:00:44.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.013+0000 7ff7d7c2a640 1 -- 172.21.15.47:0/1594618039 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7d01a0650 con 0x7ff7d0100ea0 2024-04-17T06:00:44.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.014+0000 7ff7d7c2a640 1 -- 172.21.15.47:0/1594618039 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7d0108150 con 0x7ff7d0100ea0 2024-04-17T06:00:44.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.015+0000 7ff7c2ffd640 1 -- 172.21.15.47:0/1594618039 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7ff7c4044070 con 0x7ff7d0100ea0 2024-04-17T06:00:44.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.016+0000 7ff7c2ffd640 1 --2- 172.21.15.47:0/1594618039 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7ff7bc0a3bb0 0x7ff7bc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:44.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.016+0000 7ff7d51c5640 1 --2- 172.21.15.47:0/1594618039 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7ff7bc0a3bb0 0x7ff7bc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:44.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.017+0000 7ff7c2ffd640 1 -- 172.21.15.47:0/1594618039 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7ff7c40f2200 con 0x7ff7d0100ea0 2024-04-17T06:00:44.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.017+0000 7ff7d51c5640 1 --2- 172.21.15.47:0/1594618039 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7ff7bc0a3bb0 0x7ff7bc0a6070 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff7cc005fd0 tx=0x7ff7cc005ce0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:00:44.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.021+0000 7ff7c2ffd640 1 -- 172.21.15.47:0/1594618039 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7ff7c40bebf0 con 0x7ff7d0100ea0 2024-04-17T06:00:44.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:00:43 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/2973704640' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-17T06:00:44.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:00:43 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/1403990944' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-17T06:00:44.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:00:43 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/3965002100' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T06:00:44.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:00:43 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/2973704640' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-17T06:00:44.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:00:43 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/1403990944' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-17T06:00:44.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:00:43 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/3965002100' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T06:00:44.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.247+0000 7ff7d7c2a640 1 -- 172.21.15.47:0/1594618039 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff7d002d5d0 con 0x7ff7d0100ea0 2024-04-17T06:00:44.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.248+0000 7ff7c2ffd640 1 -- 172.21.15.47:0/1594618039 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11881 (secure 0 0 0) 0x7ff7c40be4f0 con 0x7ff7d0100ea0 2024-04-17T06:00:44.247 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-17T06:00:44.247 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3189958720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3189958720},{"type":"v1","addr":"172.21.15.47:6835","nonce":3189958720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14595,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/3143529973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":3143529973},{"type":"v1","addr":"172.21.15.47:6837","nonce":3143529973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14652,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/2375823701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":2375823701},{"type":"v1","addr":"172.21.15.47:6839","nonce":2375823701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14664,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6841/3334309157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":3334309157},{"type":"v1","addr":"172.21.15.47:6841","nonce":3334309157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/2398363754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":2398363754},{"type":"v1","addr":"172.21.15.103:6833","nonce":2398363754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/384473493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":384473493},{"type":"v1","addr":"172.21.15.98:6833","nonce":384473493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T05:59:13.922176+0000","modified":"2024-04-17T05:59:19.598261+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24518,"mds_1":24479,"mds_2":24466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6835/270543522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":270543522},{"type":"v1","addr":"172.21.15.103:6835","nonce":270543522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6837/116936024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":116936024},{"type":"v1","addr":"172.21.15.103:6837","nonce":116936024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6835/1966500278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":1966500278},{"type":"v1","addr":"172.21.15.98:6835","nonce":1966500278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.103:6839/1467157889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":1467157889},{"type":"v1","addr":"172.21.15.103:6839","nonce":1467157889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.98:6837/1117440719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1117440719},{"type":"v1","addr":"172.21.15.98:6837","nonce":1117440719}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6839/2841287917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":2841287917},{"type":"v1","addr":"172.21.15.98:6839","nonce":2841287917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24518,"qdb_cluster":[24479,24518,24466]},"id":1}]} 2024-04-17T06:00:44.248 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 21 2024-04-17T06:00:44.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.250+0000 7ff7d7c2a640 1 -- 172.21.15.47:0/1594618039 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7ff7bc0a3bb0 msgr2=0x7ff7bc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:44.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.250+0000 7ff7d7c2a640 1 --2- 172.21.15.47:0/1594618039 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7ff7bc0a3bb0 0x7ff7bc0a6070 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff7cc005fd0 tx=0x7ff7cc005ce0 comp rx=0 tx=0).stop 2024-04-17T06:00:44.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.250+0000 7ff7d7c2a640 1 -- 172.21.15.47:0/1594618039 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7d0100ea0 msgr2=0x7ff7d0069c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:44.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.250+0000 7ff7d7c2a640 1 --2- 172.21.15.47:0/1594618039 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7d0100ea0 0x7ff7d0069c60 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7ff7c4009230 tx=0x7ff7c4004040 comp rx=0 tx=0).stop 2024-04-17T06:00:44.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.250+0000 7ff7d7c2a640 1 -- 172.21.15.47:0/1594618039 shutdown_connections 2024-04-17T06:00:44.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.250+0000 7ff7d7c2a640 1 --2- 172.21.15.47:0/1594618039 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7ff7bc0a3bb0 0x7ff7bc0a6070 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:44.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.250+0000 7ff7d7c2a640 1 --2- 172.21.15.47:0/1594618039 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff7d01029b0 0x7ff7d019f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:44.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.250+0000 7ff7d7c2a640 1 --2- 172.21.15.47:0/1594618039 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff7d0101ff0 0x7ff7d019d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:44.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.250+0000 7ff7d7c2a640 1 --2- 172.21.15.47:0/1594618039 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7d0100ea0 0x7ff7d0069c60 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:44.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.250+0000 7ff7d7c2a640 1 -- 172.21.15.47:0/1594618039 >> 172.21.15.47:0/1594618039 conn(0x7ff7d00fc410 msgr2=0x7ff7d0105320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:00:44.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.251+0000 7ff7d7c2a640 1 -- 172.21.15.47:0/1594618039 shutdown_connections 2024-04-17T06:00:44.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.251+0000 7ff7d7c2a640 1 -- 172.21.15.47:0/1594618039 wait complete. 2024-04-17T06:00:44.261 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 21, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T05:59:13.922176+0000', 'modified': '2024-04-17T05:59:19.598261+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'snap_rstat': True, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24518, 'mds_1': 24479, 'mds_2': 24466}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24436': {'gid': 24436, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.103:6835/270543522', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6834', 'nonce': 270543522}, {'type': 'v1', 'addr': '172.21.15.103:6835', 'nonce': 270543522}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24448': {'gid': 24448, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.103:6837/116936024', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6836', 'nonce': 116936024}, {'type': 'v1', 'addr': '172.21.15.103:6837', 'nonce': 116936024}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24449': {'gid': 24449, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.98:6835/1966500278', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6834', 'nonce': 1966500278}, {'type': 'v1', 'addr': '172.21.15.98:6835', 'nonce': 1966500278}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24466': {'gid': 24466, 'name': 'i', 'rank': 2, 'incarnation': 20, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.103:6839/1467157889', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6838', 'nonce': 1467157889}, {'type': 'v1', 'addr': '172.21.15.103:6839', 'nonce': 1467157889}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24479': {'gid': 24479, 'name': 'h', 'rank': 1, 'incarnation': 18, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.98:6837/1117440719', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6836', 'nonce': 1117440719}, {'type': 'v1', 'addr': '172.21.15.98:6837', 'nonce': 1117440719}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24518': {'gid': 24518, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.98:6839/2841287917', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6838', 'nonce': 2841287917}, {'type': 'v1', 'addr': '172.21.15.98:6839', 'nonce': 2841287917}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24518, 'qdb_cluster': [24479, 24518, 24466]} 2024-04-17T06:00:44.261 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-04-17T06:00:44.261 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-17T06:00:44.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:00:43 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/2973704640' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-17T06:00:44.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:00:43 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/1403990944' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-17T06:00:44.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:00:43 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/3965002100' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T06:00:44.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.369+0000 7fc1a6467640 1 -- 172.21.15.47:0/27028109 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc1a0102a10 msgr2=0x7fc1a0109330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:44.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.369+0000 7fc1a6467640 1 --2- 172.21.15.47:0/27028109 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc1a0102a10 0x7fc1a0109330 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fc198009ca0 tx=0x7fc19802c0f0 comp rx=0 tx=0).stop 2024-04-17T06:00:44.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.369+0000 7fc1a6467640 1 -- 172.21.15.47:0/27028109 shutdown_connections 2024-04-17T06:00:44.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.369+0000 7fc1a6467640 1 --2- 172.21.15.47:0/27028109 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc1a0102a10 0x7fc1a0109330 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:44.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.369+0000 7fc1a6467640 1 --2- 172.21.15.47:0/27028109 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc1a0102050 0x7fc1a01024d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:44.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.369+0000 7fc1a6467640 1 --2- 172.21.15.47:0/27028109 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc1a0100e70 0x7fc1a0101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:44.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.369+0000 7fc1a6467640 1 -- 172.21.15.47:0/27028109 >> 172.21.15.47:0/27028109 conn(0x7fc1a00fc450 msgr2=0x7fc1a00fe870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:00:44.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.369+0000 7fc1a6467640 1 -- 172.21.15.47:0/27028109 shutdown_connections 2024-04-17T06:00:44.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.369+0000 7fc1a6467640 1 -- 172.21.15.47:0/27028109 wait complete. 2024-04-17T06:00:44.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.370+0000 7fc1a6467640 1 Processor -- start 2024-04-17T06:00:44.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.370+0000 7fc1a6467640 1 -- start start 2024-04-17T06:00:44.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.370+0000 7fc1a6467640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc1a0100e70 0x7fc1a0198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:44.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.370+0000 7fc1a6467640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc1a0102050 0x7fc1a0198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:44.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.370+0000 7fc1a6467640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc1a0102a10 0x7fc1a019f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:44.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.370+0000 7fc1a6467640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a010dc90 con 0x7fc1a0102a10 2024-04-17T06:00:44.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.370+0000 7fc1a6467640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a010db10 con 0x7fc1a0100e70 2024-04-17T06:00:44.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.370+0000 7fc1a6467640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a010de10 con 0x7fc1a0102050 2024-04-17T06:00:44.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.370+0000 7fc1a4a04640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc1a0102a10 0x7fc1a019f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:44.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.370+0000 7fc1a4a04640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc1a0102a10 0x7fc1a019f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57482/0 (socket says 172.21.15.47:57482) 2024-04-17T06:00:44.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.370+0000 7fc1a4a04640 1 -- 172.21.15.47:0/496585993 learned_addr learned my addr 172.21.15.47:0/496585993 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:00:44.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.370+0000 7fc19f7fe640 1 --2- 172.21.15.47:0/496585993 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc1a0102050 0x7fc1a0198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:44.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.370+0000 7fc19ffff640 1 --2- 172.21.15.47:0/496585993 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc1a0100e70 0x7fc1a0198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:44.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.370+0000 7fc1a4a04640 1 -- 172.21.15.47:0/496585993 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc1a0102050 msgr2=0x7fc1a0198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:44.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.370+0000 7fc1a4a04640 1 --2- 172.21.15.47:0/496585993 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc1a0102050 0x7fc1a0198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:44.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.370+0000 7fc1a4a04640 1 -- 172.21.15.47:0/496585993 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc1a0100e70 msgr2=0x7fc1a0198210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:44.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.370+0000 7fc1a4a04640 1 --2- 172.21.15.47:0/496585993 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc1a0100e70 0x7fc1a0198210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:44.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.370+0000 7fc1a4a04640 1 -- 172.21.15.47:0/496585993 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1a019fe30 con 0x7fc1a0102a10 2024-04-17T06:00:44.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.370+0000 7fc19ffff640 1 --2- 172.21.15.47:0/496585993 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc1a0100e70 0x7fc1a0198210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:00:44.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.371+0000 7fc1a4a04640 1 --2- 172.21.15.47:0/496585993 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc1a0102a10 0x7fc1a019f7c0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fc19802c700 tx=0x7fc198032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:00:44.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.371+0000 7fc19d7fa640 1 -- 172.21.15.47:0/496585993 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc198039040 con 0x7fc1a0102a10 2024-04-17T06:00:44.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.371+0000 7fc19d7fa640 1 -- 172.21.15.47:0/496585993 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc198044070 con 0x7fc1a0102a10 2024-04-17T06:00:44.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.371+0000 7fc19d7fa640 1 -- 172.21.15.47:0/496585993 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1980042a0 con 0x7fc1a0102a10 2024-04-17T06:00:44.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.371+0000 7fc1a6467640 1 -- 172.21.15.47:0/496585993 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1a01a00c0 con 0x7fc1a0102a10 2024-04-17T06:00:44.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.371+0000 7fc1a6467640 1 -- 172.21.15.47:0/496585993 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1a01a0600 con 0x7fc1a0102a10 2024-04-17T06:00:44.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.373+0000 7fc19d7fa640 1 -- 172.21.15.47:0/496585993 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fc198003970 con 0x7fc1a0102a10 2024-04-17T06:00:44.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.374+0000 7fc1a904d740 1 Processor -- start 2024-04-17T06:00:44.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.374+0000 7fc1a904d740 1 -- start start 2024-04-17T06:00:44.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.374+0000 7fc1a904d740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561fdd6b1ca0 0x561fdd6b20c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:44.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.374+0000 7fc1a904d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fdd6b2690 0x561fdd6b7190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:44.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.374+0000 7fc1a904d740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561fdd6b7800 0x561fdd6b9c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:44.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.374+0000 7fc1a904d740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x561fdd5c5240 con 0x561fdd6b7800 2024-04-17T06:00:44.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.374+0000 7fc1a904d740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x561fdd6229f0 con 0x561fdd6b1ca0 2024-04-17T06:00:44.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.374+0000 7fc1a904d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561fdd5dd9e0 con 0x561fdd6b2690 2024-04-17T06:00:44.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.374+0000 7fc19d7fa640 1 --2- 172.21.15.47:0/496585993 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fc1880a3c30 0x7fc1880a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:44.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.374+0000 7fc19f7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561fdd6b7800 0x561fdd6b9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:44.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.374+0000 7fc19f7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561fdd6b7800 0x561fdd6b9c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57494/0 (socket says 172.21.15.47:57494) 2024-04-17T06:00:44.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.374+0000 7fc19f7fe640 1 -- 172.21.15.47:0/60265575 learned_addr learned my addr 172.21.15.47:0/60265575 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:00:44.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.374+0000 7fc1a4a04640 1 --2- 172.21.15.47:0/496585993 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fc1880a3c30 0x7fc1880a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:44.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.374+0000 7fc19ffff640 1 --2- 172.21.15.47:0/60265575 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fdd6b2690 0x561fdd6b7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:44.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.374+0000 7fc19f7fe640 1 --2- 172.21.15.47:0/60265575 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561fdd6b1ca0 0x561fdd6b20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:44.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.374+0000 7fc19d7fa640 1 -- 172.21.15.47:0/496585993 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fc1980f2890 con 0x7fc1a0102a10 2024-04-17T06:00:44.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.374+0000 7fc19f7fe640 1 -- 172.21.15.47:0/60265575 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fdd6b2690 msgr2=0x561fdd6b7190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:00:44.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.374+0000 7fc19f7fe640 1 --2- 172.21.15.47:0/60265575 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fdd6b2690 0x561fdd6b7190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:44.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.374+0000 7fc19f7fe640 1 -- 172.21.15.47:0/60265575 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561fdd6b1ca0 msgr2=0x561fdd6b20c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:00:44.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.374+0000 7fc19f7fe640 1 --2- 172.21.15.47:0/60265575 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561fdd6b1ca0 0x561fdd6b20c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:44.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.374+0000 7fc19f7fe640 1 -- 172.21.15.47:0/60265575 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561fdd5d0bc0 con 0x561fdd6b7800 2024-04-17T06:00:44.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.375+0000 7fc19f7fe640 1 --2- 172.21.15.47:0/60265575 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561fdd6b7800 0x561fdd6b9c90 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fc19400b6c0 tx=0x7fc19400b650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d146b6b1adfd187 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:00:44.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.375+0000 7fc1a4a04640 1 --2- 172.21.15.47:0/496585993 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fc1880a3c30 0x7fc1880a60f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc1980b96b0 tx=0x7fc1980e9c90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:00:44.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.375+0000 7fc1867fc640 1 -- 172.21.15.47:0/60265575 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc19401e640 con 0x561fdd6b7800 2024-04-17T06:00:44.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.375+0000 7fc1867fc640 1 -- 172.21.15.47:0/60265575 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc194010910 con 0x561fdd6b7800 2024-04-17T06:00:44.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.375+0000 7fc1867fc640 1 -- 172.21.15.47:0/60265575 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc19401d820 con 0x561fdd6b7800 2024-04-17T06:00:44.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.376+0000 7fc1a904d740 1 -- 172.21.15.47:0/60265575 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561fdd6b7800 msgr2=0x561fdd6b9c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:44.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.376+0000 7fc1a904d740 1 --2- 172.21.15.47:0/60265575 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561fdd6b7800 0x561fdd6b9c90 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fc19400b6c0 tx=0x7fc19400b650 comp rx=0 tx=0).stop 2024-04-17T06:00:44.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.376+0000 7fc1a904d740 1 -- 172.21.15.47:0/60265575 shutdown_connections 2024-04-17T06:00:44.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.376+0000 7fc1a904d740 1 --2- 172.21.15.47:0/60265575 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561fdd6b7800 0x561fdd6b9c90 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:44.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.376+0000 7fc1a904d740 1 --2- 172.21.15.47:0/60265575 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fdd6b2690 0x561fdd6b7190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:44.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.376+0000 7fc1a904d740 1 --2- 172.21.15.47:0/60265575 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561fdd6b1ca0 0x561fdd6b20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:44.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.376+0000 7fc1a904d740 1 -- 172.21.15.47:0/60265575 >> 172.21.15.47:0/60265575 conn(0x561fdd6ad210 msgr2=0x561fdd6ad610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:00:44.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.376+0000 7fc1a904d740 1 -- 172.21.15.47:0/60265575 shutdown_connections 2024-04-17T06:00:44.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.376+0000 7fc1a904d740 1 -- 172.21.15.47:0/60265575 wait complete. 2024-04-17T06:00:44.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.376+0000 7fc1a904d740 1 -- start start 2024-04-17T06:00:44.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.376+0000 7fc1a904d740 1 Processor -- start 2024-04-17T06:00:44.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.376+0000 7fc1a904d740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561fdd6b1ca0 0x561fdd6bbec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:44.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.376+0000 7fc1a904d740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561fdd6b2690 0x561fdd6bc400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:44.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.376+0000 7fc1a904d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fdd6b7800 0x561fdd6c3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:44.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.376+0000 7fc1a904d740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x561fdd61c0f0 con 0x561fdd6b2690 2024-04-17T06:00:44.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.376+0000 7fc1a904d740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x561fdd616dd0 con 0x561fdd6b1ca0 2024-04-17T06:00:44.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.376+0000 7fc1a904d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561fdd624cd0 con 0x561fdd6b7800 2024-04-17T06:00:44.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.376+0000 7fc1a904d740 10 client.0 _ll_register_callbacks cb 0x561fdd60d570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:00:44.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.377+0000 7fc19ffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561fdd6b2690 0x561fdd6bc400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:44.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.377+0000 7fc19ffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561fdd6b2690 0x561fdd6bc400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57504/0 (socket says 172.21.15.47:57504) 2024-04-17T06:00:44.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.377+0000 7fc19ffff640 1 -- 172.21.15.47:0/3134335321 learned_addr learned my addr 172.21.15.47:0/3134335321 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:00:44.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.377+0000 7fc19f7fe640 1 --2- 172.21.15.47:0/3134335321 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fdd6b7800 0x561fdd6c3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:44.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.377+0000 7fc19f7fe640 1 --2- 172.21.15.47:0/3134335321 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561fdd6b1ca0 0x561fdd6bbec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:44.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.377+0000 7fc19ffff640 1 -- 172.21.15.47:0/3134335321 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fdd6b7800 msgr2=0x561fdd6c3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:00:44.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.377+0000 7fc19ffff640 1 --2- 172.21.15.47:0/3134335321 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fdd6b7800 0x561fdd6c3440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:44.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.377+0000 7fc19ffff640 1 -- 172.21.15.47:0/3134335321 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561fdd6b1ca0 msgr2=0x561fdd6bbec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:00:44.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.377+0000 7fc19ffff640 1 --2- 172.21.15.47:0/3134335321 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561fdd6b1ca0 0x561fdd6bbec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:44.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.377+0000 7fc19ffff640 1 -- 172.21.15.47:0/3134335321 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561fdd32a2d0 con 0x561fdd6b2690 2024-04-17T06:00:44.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.377+0000 7fc19f7fe640 1 --2- 172.21.15.47:0/3134335321 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561fdd6b1ca0 0x561fdd6bbec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:00:44.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.377+0000 7fc19ffff640 1 --2- 172.21.15.47:0/3134335321 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561fdd6b2690 0x561fdd6bc400 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fc19000bf20 tx=0x7fc1900128e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2112cbd12dbb82a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:00:44.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.377+0000 7fc17f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-17T06:00:44.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.377+0000 7fc17f7fe640 1 -- 172.21.15.47:0/3134335321 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc190010960 con 0x561fdd6b2690 2024-04-17T06:00:44.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.377+0000 7fc17f7fe640 1 -- 172.21.15.47:0/3134335321 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fc190010b00 con 0x561fdd6b2690 2024-04-17T06:00:44.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.377+0000 7fc1a904d740 1 -- 172.21.15.47:0/3134335321 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc194010910 con 0x561fdd6b2690 2024-04-17T06:00:44.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.377+0000 7fc17f7fe640 1 -- 172.21.15.47:0/3134335321 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fc190010d00 con 0x561fdd6b2690 2024-04-17T06:00:44.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.377+0000 7fc1a904d740 10 client.15090 fetch_fsmap learned FSMap version 21 2024-04-17T06:00:44.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.377+0000 7fc1a904d740 1 -- 172.21.15.47:0/3134335321 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x561fdd3f46b0 con 0x561fdd6b2690 2024-04-17T06:00:44.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.378+0000 7fc17f7fe640 1 -- 172.21.15.47:0/3134335321 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc194010910 con 0x561fdd6b2690 2024-04-17T06:00:44.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.378+0000 7fc17f7fe640 1 -- 172.21.15.47:0/3134335321 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 21) ==== 8339+0+0 (secure 0 0 0) 0x7fc19001b3a0 con 0x561fdd6b2690 2024-04-17T06:00:44.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.378+0000 7fc1a904d740 10 client.15090 fetch_fsmap finished waiting for FSMap version 21 2024-04-17T06:00:44.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.378+0000 7fc1a904d740 10 client.15090 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-17T06:00:44.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.378+0000 7fc1a904d740 20 client.15090 populate_metadata read hostname 'smithi047' 2024-04-17T06:00:44.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.378+0000 7fc1a904d740 1 --2- 172.21.15.47:0/3134335321 >> [v2:172.21.15.103:6838/1467157889,v1:172.21.15.103:6839/1467157889] conn(0x561fdd6c8c50 0x561fdd6ca0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:44.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.378+0000 7fc1a904d740 4 client.15090 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T06:00:44.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.378+0000 7fc1a904d740 1 -- 172.21.15.47:0/3134335321 --> [v2:172.21.15.103:6838/1467157889,v1:172.21.15.103:6839/1467157889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561fdd59e4c0 con 0x561fdd6c8c50 2024-04-17T06:00:44.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.378+0000 7fc19f7fe640 1 --2- 172.21.15.47:0/3134335321 >> [v2:172.21.15.103:6838/1467157889,v1:172.21.15.103:6839/1467157889] conn(0x561fdd6c8c50 0x561fdd6ca0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:44.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.378+0000 7fc19f7fe640 1 --2- 172.21.15.47:0/3134335321 >> [v2:172.21.15.103:6838/1467157889,v1:172.21.15.103:6839/1467157889] conn(0x561fdd6c8c50 0x561fdd6ca0c0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=68ee9a086dcf246 server_cookie=64326322bf4e887 in_seq=0 out_seq=0 2024-04-17T06:00:44.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.378+0000 7fc17f7fe640 10 client.15090 ms_handle_connect on v2:172.21.15.103:6838/1467157889 2024-04-17T06:00:44.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.379+0000 7fc17f7fe640 1 -- 172.21.15.47:0/3134335321 <== mds.2 v2:172.21.15.103:6838/1467157889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561fdd59e4c0 con 0x561fdd6c8c50 2024-04-17T06:00:44.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.379+0000 7fc17f7fe640 10 client.15090 handle_command_reply: tid=0 2024-04-17T06:00:44.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.379+0000 7fc1a904d740 1 client.15090 shutdown 2024-04-17T06:00:44.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.380+0000 7fc1a904d740 1 -- 172.21.15.47:0/3134335321 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561fdd6b2690 msgr2=0x561fdd6bc400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:44.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.380+0000 7fc1a904d740 1 --2- 172.21.15.47:0/3134335321 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561fdd6b2690 0x561fdd6bc400 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fc19000bf20 tx=0x7fc1900128e0 comp rx=0 tx=0).stop 2024-04-17T06:00:44.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.380+0000 7fc1a904d740 1 -- 172.21.15.47:0/3134335321 shutdown_connections 2024-04-17T06:00:44.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.380+0000 7fc1a904d740 1 --2- 172.21.15.47:0/3134335321 >> [v2:172.21.15.103:6838/1467157889,v1:172.21.15.103:6839/1467157889] conn(0x561fdd6c8c50 0x561fdd6ca0c0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:44.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.380+0000 7fc1a904d740 1 --2- 172.21.15.47:0/3134335321 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fdd6b7800 0x561fdd6c3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:44.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.380+0000 7fc1a904d740 1 --2- 172.21.15.47:0/3134335321 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561fdd6b2690 0x561fdd6bc400 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:44.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.380+0000 7fc17f7fe640 0 client.15090 ms_handle_reset on v2:172.21.15.103:6838/1467157889 2024-04-17T06:00:44.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.380+0000 7fc1a904d740 1 --2- 172.21.15.47:0/3134335321 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561fdd6b1ca0 0x561fdd6bbec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:44.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.380+0000 7fc1a904d740 1 -- 172.21.15.47:0/3134335321 >> 172.21.15.47:0/3134335321 conn(0x561fdd6ad210 msgr2=0x561fdd6ac350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:00:44.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.380+0000 7fc1a904d740 1 -- 172.21.15.47:0/3134335321 shutdown_connections 2024-04-17T06:00:44.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.380+0000 7fc1a904d740 1 -- 172.21.15.47:0/3134335321 wait complete. 2024-04-17T06:00:44.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.380+0000 7fc1a904d740 20 client.15090 trim_cache size 0 max 16384 2024-04-17T06:00:44.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.398+0000 7fc1a904d740 1 Processor -- start 2024-04-17T06:00:44.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.398+0000 7fc1a904d740 1 -- start start 2024-04-17T06:00:44.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.398+0000 7fc1a904d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fdd6b1ca0 0x561fdd6b9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:44.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.398+0000 7fc1a904d740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561fdd6b2690 0x561fdd6a7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:44.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.398+0000 7fc1a904d740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561fdd6b7800 0x561fdd6ad5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:44.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.398+0000 7fc1a904d740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x561fdd6c8400 con 0x561fdd6b7800 2024-04-17T06:00:44.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.398+0000 7fc1a904d740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x561fdd6c8ad0 con 0x561fdd6b2690 2024-04-17T06:00:44.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.398+0000 7fc1a904d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561fdd6c91a0 con 0x561fdd6b1ca0 2024-04-17T06:00:44.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.398+0000 7fc19f7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561fdd6b7800 0x561fdd6ad5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:44.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.398+0000 7fc19f7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561fdd6b7800 0x561fdd6ad5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57520/0 (socket says 172.21.15.47:57520) 2024-04-17T06:00:44.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.398+0000 7fc19f7fe640 1 -- 172.21.15.47:0/2247632238 learned_addr learned my addr 172.21.15.47:0/2247632238 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:00:44.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.398+0000 7fc19f7fe640 1 --2- 172.21.15.47:0/2247632238 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fdd6b1ca0 0x561fdd6b9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:44.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.399+0000 7fc19ffff640 1 --2- 172.21.15.47:0/2247632238 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561fdd6b2690 0x561fdd6a7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:44.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.399+0000 7fc19f7fe640 1 -- 172.21.15.47:0/2247632238 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fdd6b1ca0 msgr2=0x561fdd6b9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:00:44.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.399+0000 7fc19f7fe640 1 --2- 172.21.15.47:0/2247632238 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fdd6b1ca0 0x561fdd6b9bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:44.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.399+0000 7fc19f7fe640 1 -- 172.21.15.47:0/2247632238 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561fdd6b2690 msgr2=0x561fdd6a7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:00:44.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.399+0000 7fc19f7fe640 1 --2- 172.21.15.47:0/2247632238 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561fdd6b2690 0x561fdd6a7fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:44.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.399+0000 7fc19f7fe640 1 -- 172.21.15.47:0/2247632238 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561fdd6b2ea0 con 0x561fdd6b7800 2024-04-17T06:00:44.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.399+0000 7fc19f7fe640 1 --2- 172.21.15.47:0/2247632238 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561fdd6b7800 0x561fdd6ad5f0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fc19400bf40 tx=0x7fc194005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f7b37e227397e1b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:00:44.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.399+0000 7fc185ffb640 1 -- 172.21.15.47:0/2247632238 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc194023040 con 0x561fdd6b7800 2024-04-17T06:00:44.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.399+0000 7fc185ffb640 1 -- 172.21.15.47:0/2247632238 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc194010430 con 0x561fdd6b7800 2024-04-17T06:00:44.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.399+0000 7fc185ffb640 1 -- 172.21.15.47:0/2247632238 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc194005320 con 0x561fdd6b7800 2024-04-17T06:00:44.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.400+0000 7fc1a904d740 1 -- 172.21.15.47:0/2247632238 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561fdd6b7800 msgr2=0x561fdd6ad5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:44.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.400+0000 7fc1a904d740 1 --2- 172.21.15.47:0/2247632238 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561fdd6b7800 0x561fdd6ad5f0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fc19400bf40 tx=0x7fc194005850 comp rx=0 tx=0).stop 2024-04-17T06:00:44.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.400+0000 7fc1a904d740 1 -- 172.21.15.47:0/2247632238 shutdown_connections 2024-04-17T06:00:44.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.400+0000 7fc1a904d740 1 --2- 172.21.15.47:0/2247632238 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561fdd6b7800 0x561fdd6ad5f0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:44.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.400+0000 7fc1a904d740 1 --2- 172.21.15.47:0/2247632238 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561fdd6b2690 0x561fdd6a7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:44.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.400+0000 7fc1a904d740 1 --2- 172.21.15.47:0/2247632238 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fdd6b1ca0 0x561fdd6b9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:44.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.400+0000 7fc1a904d740 1 -- 172.21.15.47:0/2247632238 >> 172.21.15.47:0/2247632238 conn(0x561fdd6ad210 msgr2=0x561fdd6baa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:00:44.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.400+0000 7fc1a904d740 1 -- 172.21.15.47:0/2247632238 shutdown_connections 2024-04-17T06:00:44.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.400+0000 7fc1a904d740 1 -- 172.21.15.47:0/2247632238 wait complete. 2024-04-17T06:00:44.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.400+0000 7fc1a904d740 1 -- start start 2024-04-17T06:00:44.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.400+0000 7fc1a904d740 1 Processor -- start 2024-04-17T06:00:44.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.400+0000 7fc1a904d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fdd6b1ca0 0x561fdd6be5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:44.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.400+0000 7fc1a904d740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561fdd6b2690 0x561fdd6beb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:44.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.400+0000 7fc1a904d740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561fdd6b7800 0x561fdd6c1b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:44.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.400+0000 7fc1a904d740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x561fdd6c9f40 con 0x561fdd6b7800 2024-04-17T06:00:44.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.400+0000 7fc1a904d740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x561fdd6c9870 con 0x561fdd6b2690 2024-04-17T06:00:44.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.400+0000 7fc1a904d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561fdd6ca610 con 0x561fdd6b1ca0 2024-04-17T06:00:44.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.400+0000 7fc1a904d740 10 client.0 _ll_register_callbacks cb 0x7fc19000b840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:00:44.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.401+0000 7fc19f7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561fdd6b7800 0x561fdd6c1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:44.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.401+0000 7fc19f7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561fdd6b7800 0x561fdd6c1b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57522/0 (socket says 172.21.15.47:57522) 2024-04-17T06:00:44.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.401+0000 7fc19f7fe640 1 -- 172.21.15.47:0/3895278475 learned_addr learned my addr 172.21.15.47:0/3895278475 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:00:44.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.401+0000 7fc19ffff640 1 --2- 172.21.15.47:0/3895278475 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561fdd6b2690 0x561fdd6beb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:44.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.401+0000 7fc19f7fe640 1 --2- 172.21.15.47:0/3895278475 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fdd6b1ca0 0x561fdd6be5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:44.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.401+0000 7fc19f7fe640 1 -- 172.21.15.47:0/3895278475 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fdd6b1ca0 msgr2=0x561fdd6be5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:00:44.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.401+0000 7fc19f7fe640 1 --2- 172.21.15.47:0/3895278475 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fdd6b1ca0 0x561fdd6be5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:44.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.401+0000 7fc19f7fe640 1 -- 172.21.15.47:0/3895278475 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561fdd6b2690 msgr2=0x561fdd6beb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:00:44.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.401+0000 7fc19f7fe640 1 --2- 172.21.15.47:0/3895278475 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561fdd6b2690 0x561fdd6beb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:44.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.401+0000 7fc19f7fe640 1 -- 172.21.15.47:0/3895278475 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561fdd6c20f0 con 0x561fdd6b7800 2024-04-17T06:00:44.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.401+0000 7fc19f7fe640 1 --2- 172.21.15.47:0/3895278475 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561fdd6b7800 0x561fdd6c1b00 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fc19400bcb0 tx=0x7fc194031040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c23ab4a22f6a748c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:00:44.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.401+0000 7fc17f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-17T06:00:44.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.401+0000 7fc17f7fe640 1 -- 172.21.15.47:0/3895278475 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc194040060 con 0x561fdd6b7800 2024-04-17T06:00:44.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.401+0000 7fc1a904d740 1 -- 172.21.15.47:0/3895278475 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc194010430 con 0x561fdd6b7800 2024-04-17T06:00:44.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.401+0000 7fc17f7fe640 1 -- 172.21.15.47:0/3895278475 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fc1940105e0 con 0x561fdd6b7800 2024-04-17T06:00:44.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.401+0000 7fc17f7fe640 1 -- 172.21.15.47:0/3895278475 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fc1940107e0 con 0x561fdd6b7800 2024-04-17T06:00:44.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.401+0000 7fc1a904d740 10 client.15096 fetch_fsmap learned FSMap version 21 2024-04-17T06:00:44.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.401+0000 7fc1a904d740 1 -- 172.21.15.47:0/3895278475 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x561fdd72f110 con 0x561fdd6b7800 2024-04-17T06:00:44.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.402+0000 7fc17f7fe640 1 -- 172.21.15.47:0/3895278475 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc194010430 con 0x561fdd6b7800 2024-04-17T06:00:44.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.402+0000 7fc17f7fe640 1 -- 172.21.15.47:0/3895278475 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 21) ==== 8339+0+0 (secure 0 0 0) 0x7fc194037070 con 0x561fdd6b7800 2024-04-17T06:00:44.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.402+0000 7fc1a904d740 10 client.15096 fetch_fsmap finished waiting for FSMap version 21 2024-04-17T06:00:44.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.402+0000 7fc1a904d740 10 client.15096 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-17T06:00:44.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.402+0000 7fc1a904d740 20 client.15096 populate_metadata read hostname 'smithi047' 2024-04-17T06:00:44.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.402+0000 7fc1a904d740 1 --2- 172.21.15.47:0/3895278475 >> [v2:172.21.15.103:6838/1467157889,v1:172.21.15.103:6839/1467157889] conn(0x561fdd7335a0 0x561fdd7322b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:44.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.402+0000 7fc1a904d740 4 client.15096 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-17T06:00:44.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.402+0000 7fc1a904d740 1 -- 172.21.15.47:0/3895278475 --> [v2:172.21.15.103:6838/1467157889,v1:172.21.15.103:6839/1467157889] -- command(tid 0: {"prefix": "status"}) -- 0x561fdd6254e0 con 0x561fdd7335a0 2024-04-17T06:00:44.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.402+0000 7fc19ffff640 1 --2- 172.21.15.47:0/3895278475 >> [v2:172.21.15.103:6838/1467157889,v1:172.21.15.103:6839/1467157889] conn(0x561fdd7335a0 0x561fdd7322b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:44.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.402+0000 7fc17f7fe640 10 client.15096 ms_handle_connect on v2:172.21.15.103:6838/1467157889 2024-04-17T06:00:44.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.402+0000 7fc19ffff640 1 --2- 172.21.15.47:0/3895278475 >> [v2:172.21.15.103:6838/1467157889,v1:172.21.15.103:6839/1467157889] conn(0x561fdd7335a0 0x561fdd7322b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5e69f20c817dc48e server_cookie=ef6a27e2871c36b0 in_seq=0 out_seq=0 2024-04-17T06:00:44.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.403+0000 7fc17f7fe640 1 -- 172.21.15.47:0/3895278475 <== mds.2 v2:172.21.15.103:6838/1467157889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x561fdd6254e0 con 0x561fdd7335a0 2024-04-17T06:00:44.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.403+0000 7fc17f7fe640 10 client.15096 handle_command_reply: tid=0 2024-04-17T06:00:44.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.403+0000 7fc1a904d740 1 client.15096 shutdown 2024-04-17T06:00:44.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.403+0000 7fc1a904d740 1 -- 172.21.15.47:0/3895278475 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561fdd6b7800 msgr2=0x561fdd6c1b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:44.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.403+0000 7fc1a904d740 1 --2- 172.21.15.47:0/3895278475 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561fdd6b7800 0x561fdd6c1b00 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fc19400bcb0 tx=0x7fc194031040 comp rx=0 tx=0).stop 2024-04-17T06:00:44.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.403+0000 7fc1a904d740 1 -- 172.21.15.47:0/3895278475 shutdown_connections 2024-04-17T06:00:44.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.403+0000 7fc1a904d740 1 --2- 172.21.15.47:0/3895278475 >> [v2:172.21.15.103:6838/1467157889,v1:172.21.15.103:6839/1467157889] conn(0x561fdd7335a0 0x561fdd7322b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:44.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.403+0000 7fc1a904d740 1 --2- 172.21.15.47:0/3895278475 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561fdd6b7800 0x561fdd6c1b00 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:44.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.403+0000 7fc1a904d740 1 --2- 172.21.15.47:0/3895278475 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561fdd6b2690 0x561fdd6beb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:44.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.403+0000 7fc17f7fe640 0 client.15096 ms_handle_reset on v2:172.21.15.103:6838/1467157889 2024-04-17T06:00:44.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.403+0000 7fc1a904d740 1 --2- 172.21.15.47:0/3895278475 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fdd6b1ca0 0x561fdd6be5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:44.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.403+0000 7fc1a904d740 1 -- 172.21.15.47:0/3895278475 >> 172.21.15.47:0/3895278475 conn(0x561fdd6ad210 msgr2=0x561fdd6a7fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:00:44.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.403+0000 7fc1a904d740 1 -- 172.21.15.47:0/3895278475 shutdown_connections 2024-04-17T06:00:44.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.403+0000 7fc1a904d740 1 -- 172.21.15.47:0/3895278475 wait complete. 2024-04-17T06:00:44.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.403+0000 7fc1a904d740 20 client.15096 trim_cache size 0 max 16384 2024-04-17T06:00:44.409 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-17T06:00:44.409 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_fsid": "565db2d8-fc7e-11ee-bc91-c7b262605968", 2024-04-17T06:00:44.409 INFO:teuthology.orchestra.run.smithi047.stdout: "whoami": 2, 2024-04-17T06:00:44.409 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 24466, 2024-04-17T06:00:44.409 INFO:teuthology.orchestra.run.smithi047.stdout: "want_state": "up:active", 2024-04-17T06:00:44.409 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "up:active", 2024-04-17T06:00:44.409 INFO:teuthology.orchestra.run.smithi047.stdout: "fs_name": "cephfs", 2024-04-17T06:00:44.409 INFO:teuthology.orchestra.run.smithi047.stdout: "rank_uptime": 85.780932379000006, 2024-04-17T06:00:44.409 INFO:teuthology.orchestra.run.smithi047.stdout: "mdsmap_epoch": 21, 2024-04-17T06:00:44.409 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch": 88, 2024-04-17T06:00:44.409 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch_barrier": 86, 2024-04-17T06:00:44.409 INFO:teuthology.orchestra.run.smithi047.stdout: "uptime": 104.47082208400001 2024-04-17T06:00:44.409 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-17T06:00:44.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.403+0000 7fc1a6467640 1 -- 172.21.15.47:0/496585993 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fc1880a3c30 msgr2=0x7fc1880a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:44.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.403+0000 7fc1a6467640 1 --2- 172.21.15.47:0/496585993 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fc1880a3c30 0x7fc1880a60f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc1980b96b0 tx=0x7fc1980e9c90 comp rx=0 tx=0).stop 2024-04-17T06:00:44.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.403+0000 7fc1a6467640 1 -- 172.21.15.47:0/496585993 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc1a0102a10 msgr2=0x7fc1a019f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:44.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.403+0000 7fc1a6467640 1 --2- 172.21.15.47:0/496585993 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc1a0102a10 0x7fc1a019f7c0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fc19802c700 tx=0x7fc198032f20 comp rx=0 tx=0).stop 2024-04-17T06:00:44.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.404+0000 7fc1a6467640 1 -- 172.21.15.47:0/496585993 shutdown_connections 2024-04-17T06:00:44.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.404+0000 7fc1a6467640 1 --2- 172.21.15.47:0/496585993 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fc1880a3c30 0x7fc1880a60f0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:44.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.404+0000 7fc1a6467640 1 --2- 172.21.15.47:0/496585993 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc1a0102a10 0x7fc1a019f7c0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:44.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.404+0000 7fc1a6467640 1 --2- 172.21.15.47:0/496585993 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc1a0102050 0x7fc1a0198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:44.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.404+0000 7fc1a6467640 1 --2- 172.21.15.47:0/496585993 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc1a0100e70 0x7fc1a0198210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:44.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.404+0000 7fc1a6467640 1 -- 172.21.15.47:0/496585993 >> 172.21.15.47:0/496585993 conn(0x7fc1a00fc450 msgr2=0x7fc1a01053e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:00:44.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.404+0000 7fc1a6467640 1 -- 172.21.15.47:0/496585993 shutdown_connections 2024-04-17T06:00:44.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.404+0000 7fc1a6467640 1 -- 172.21.15.47:0/496585993 wait complete. 2024-04-17T06:00:44.413 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-04-17T06:00:44.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.521+0000 7fecc35bd640 1 -- 172.21.15.47:0/614206767 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fecbc107ca0 msgr2=0x7fecbc10a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:44.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.521+0000 7fecc35bd640 1 --2- 172.21.15.47:0/614206767 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fecbc107ca0 0x7fecbc10a0f0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fecb4009ca0 tx=0x7fecb402c0f0 comp rx=0 tx=0).stop 2024-04-17T06:00:44.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.521+0000 7fecc35bd640 1 -- 172.21.15.47:0/614206767 shutdown_connections 2024-04-17T06:00:44.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.521+0000 7fecc35bd640 1 --2- 172.21.15.47:0/614206767 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fecbc107ca0 0x7fecbc10a0f0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:44.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.521+0000 7fecc35bd640 1 --2- 172.21.15.47:0/614206767 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fecbc105370 0x7fecbc107760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:44.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.521+0000 7fecc35bd640 1 --2- 172.21.15.47:0/614206767 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fecbc102a40 0x7fecbc104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:44.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.521+0000 7fecc35bd640 1 -- 172.21.15.47:0/614206767 >> 172.21.15.47:0/614206767 conn(0x7fecbc0fc450 msgr2=0x7fecbc0fe870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:00:44.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.522+0000 7fecc35bd640 1 -- 172.21.15.47:0/614206767 shutdown_connections 2024-04-17T06:00:44.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.522+0000 7fecc35bd640 1 -- 172.21.15.47:0/614206767 wait complete. 2024-04-17T06:00:44.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.522+0000 7fecc35bd640 1 Processor -- start 2024-04-17T06:00:44.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.522+0000 7fecc35bd640 1 -- start start 2024-04-17T06:00:44.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.522+0000 7fecc35bd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fecbc102a40 0x7fecbc1023b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:44.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.522+0000 7fecc35bd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fecbc105370 0x7fecbc1009f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:44.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.522+0000 7fecc35bd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fecbc107ca0 0x7fecbc100f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:44.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.522+0000 7fecc35bd640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fecbc10dcb0 con 0x7fecbc107ca0 2024-04-17T06:00:44.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.522+0000 7fecc35bd640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fecbc10db30 con 0x7fecbc105370 2024-04-17T06:00:44.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.522+0000 7fecc35bd640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fecbc10de30 con 0x7fecbc102a40 2024-04-17T06:00:44.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.522+0000 7fecc1b5a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fecbc107ca0 0x7fecbc100f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:44.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.522+0000 7fecc1b5a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fecbc107ca0 0x7fecbc100f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57532/0 (socket says 172.21.15.47:57532) 2024-04-17T06:00:44.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.522+0000 7fecc1b5a640 1 -- 172.21.15.47:0/3783321072 learned_addr learned my addr 172.21.15.47:0/3783321072 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:00:44.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.523+0000 7fecc0b58640 1 --2- 172.21.15.47:0/3783321072 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fecbc105370 0x7fecbc1009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:44.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.523+0000 7fecc1359640 1 --2- 172.21.15.47:0/3783321072 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fecbc102a40 0x7fecbc1023b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:44.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.523+0000 7fecc1b5a640 1 -- 172.21.15.47:0/3783321072 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fecbc102a40 msgr2=0x7fecbc1023b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:44.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.523+0000 7fecc1b5a640 1 --2- 172.21.15.47:0/3783321072 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fecbc102a40 0x7fecbc1023b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:44.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.523+0000 7fecc1b5a640 1 -- 172.21.15.47:0/3783321072 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fecbc105370 msgr2=0x7fecbc1009f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:44.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.523+0000 7fecc1b5a640 1 --2- 172.21.15.47:0/3783321072 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fecbc105370 0x7fecbc1009f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:44.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.523+0000 7fecc1b5a640 1 -- 172.21.15.47:0/3783321072 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecbc1017c0 con 0x7fecbc107ca0 2024-04-17T06:00:44.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.523+0000 7fecc1b5a640 1 --2- 172.21.15.47:0/3783321072 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fecbc107ca0 0x7fecbc100f30 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fecb4037040 tx=0x7fecb4032f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:00:44.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.523+0000 7fecae7fc640 1 -- 172.21.15.47:0/3783321072 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb4039040 con 0x7fecbc107ca0 2024-04-17T06:00:44.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.523+0000 7fecae7fc640 1 -- 172.21.15.47:0/3783321072 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fecb4044070 con 0x7fecbc107ca0 2024-04-17T06:00:44.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.523+0000 7fecc35bd640 1 -- 172.21.15.47:0/3783321072 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecbc1a23b0 con 0x7fecbc107ca0 2024-04-17T06:00:44.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.523+0000 7fecae7fc640 1 -- 172.21.15.47:0/3783321072 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb4004330 con 0x7fecbc107ca0 2024-04-17T06:00:44.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.523+0000 7fecc35bd640 1 -- 172.21.15.47:0/3783321072 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecbc1a2950 con 0x7fecbc107ca0 2024-04-17T06:00:44.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.525+0000 7fecae7fc640 1 -- 172.21.15.47:0/3783321072 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fecb4003a00 con 0x7fecbc107ca0 2024-04-17T06:00:44.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.526+0000 7fecc61a9740 1 Processor -- start 2024-04-17T06:00:44.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.526+0000 7fecc61a9740 1 -- start start 2024-04-17T06:00:44.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.526+0000 7fecc61a9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da8667dca0 0x55da8667e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:44.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.526+0000 7fecc61a9740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55da8667e690 0x55da86683190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:44.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.526+0000 7fecc61a9740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da86683800 0x55da86685c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:44.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.526+0000 7fecc61a9740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55da86591240 con 0x55da86683800 2024-04-17T06:00:44.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.526+0000 7fecc61a9740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55da865ee9f0 con 0x55da8667e690 2024-04-17T06:00:44.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.526+0000 7fecc61a9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55da865a99e0 con 0x55da8667dca0 2024-04-17T06:00:44.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.526+0000 7fecc0b58640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da86683800 0x55da86685c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:44.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.526+0000 7fecc0b58640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da86683800 0x55da86685c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57542/0 (socket says 172.21.15.47:57542) 2024-04-17T06:00:44.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.526+0000 7fecc0b58640 1 -- 172.21.15.47:0/45162659 learned_addr learned my addr 172.21.15.47:0/45162659 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:00:44.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.526+0000 7fecae7fc640 1 --2- 172.21.15.47:0/3783321072 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7feca80a3c30 0x7feca80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:44.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.526+0000 7fecc1359640 1 --2- 172.21.15.47:0/45162659 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55da8667e690 0x55da86683190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:44.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.526+0000 7fecc0b58640 1 --2- 172.21.15.47:0/45162659 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da8667dca0 0x55da8667e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:44.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.526+0000 7fecc1b5a640 1 --2- 172.21.15.47:0/3783321072 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7feca80a3c30 0x7feca80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:44.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.526+0000 7fecc0b58640 1 -- 172.21.15.47:0/45162659 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da8667dca0 msgr2=0x55da8667e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:00:44.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.526+0000 7fecc0b58640 1 --2- 172.21.15.47:0/45162659 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da8667dca0 0x55da8667e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:44.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.526+0000 7fecc0b58640 1 -- 172.21.15.47:0/45162659 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55da8667e690 msgr2=0x55da86683190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:00:44.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.526+0000 7fecc0b58640 1 --2- 172.21.15.47:0/45162659 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55da8667e690 0x55da86683190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:44.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.526+0000 7fecc0b58640 1 -- 172.21.15.47:0/45162659 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da8659cbc0 con 0x55da86683800 2024-04-17T06:00:44.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.526+0000 7fecae7fc640 1 -- 172.21.15.47:0/3783321072 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fecb40f2c80 con 0x7fecbc107ca0 2024-04-17T06:00:44.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.527+0000 7fecc0b58640 1 --2- 172.21.15.47:0/45162659 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da86683800 0x55da86685c90 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fecb8005fd0 tx=0x7fecb800e610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d712aa53b891a562 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:00:44.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.527+0000 7fecc1b5a640 1 --2- 172.21.15.47:0/3783321072 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7feca80a3c30 0x7feca80a60f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fecb40bec70 tx=0x7fecb40ea080 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:00:44.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.527+0000 7feca77fe640 1 -- 172.21.15.47:0/45162659 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb8018070 con 0x55da86683800 2024-04-17T06:00:44.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.527+0000 7feca77fe640 1 -- 172.21.15.47:0/45162659 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fecb80047a0 con 0x55da86683800 2024-04-17T06:00:44.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.527+0000 7feca77fe640 1 -- 172.21.15.47:0/45162659 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb8010b10 con 0x55da86683800 2024-04-17T06:00:44.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.527+0000 7fecc61a9740 1 -- 172.21.15.47:0/45162659 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da86683800 msgr2=0x55da86685c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:44.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.527+0000 7fecc61a9740 1 --2- 172.21.15.47:0/45162659 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da86683800 0x55da86685c90 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fecb8005fd0 tx=0x7fecb800e610 comp rx=0 tx=0).stop 2024-04-17T06:00:44.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.528+0000 7fecc61a9740 1 -- 172.21.15.47:0/45162659 shutdown_connections 2024-04-17T06:00:44.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.528+0000 7fecc61a9740 1 --2- 172.21.15.47:0/45162659 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da86683800 0x55da86685c90 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:44.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.528+0000 7fecc61a9740 1 --2- 172.21.15.47:0/45162659 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55da8667e690 0x55da86683190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:44.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.528+0000 7fecc61a9740 1 --2- 172.21.15.47:0/45162659 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da8667dca0 0x55da8667e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:44.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.528+0000 7fecc61a9740 1 -- 172.21.15.47:0/45162659 >> 172.21.15.47:0/45162659 conn(0x55da86679210 msgr2=0x55da86679610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:00:44.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.528+0000 7fecc61a9740 1 -- 172.21.15.47:0/45162659 shutdown_connections 2024-04-17T06:00:44.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.528+0000 7fecc61a9740 1 -- 172.21.15.47:0/45162659 wait complete. 2024-04-17T06:00:44.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.528+0000 7fecc61a9740 1 -- start start 2024-04-17T06:00:44.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.528+0000 7fecc61a9740 1 Processor -- start 2024-04-17T06:00:44.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.528+0000 7fecc61a9740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da8667dca0 0x55da86687f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:44.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.528+0000 7fecc61a9740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55da8667e690 0x55da86688480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:44.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.528+0000 7fecc61a9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da86683800 0x55da8668f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:44.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.528+0000 7fecc61a9740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55da865e80f0 con 0x55da8667dca0 2024-04-17T06:00:44.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.528+0000 7fecc61a9740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55da865e2dd0 con 0x55da8667e690 2024-04-17T06:00:44.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.528+0000 7fecc61a9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55da865f0cd0 con 0x55da86683800 2024-04-17T06:00:44.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.528+0000 7fecc61a9740 10 client.0 _ll_register_callbacks cb 0x55da865d9570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:00:44.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.528+0000 7fecc0b58640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da8667dca0 0x55da86687f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:44.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.528+0000 7fecc0b58640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da8667dca0 0x55da86687f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53286/0 (socket says 172.21.15.47:53286) 2024-04-17T06:00:44.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.528+0000 7fecc0b58640 1 -- 172.21.15.47:0/3756537525 learned_addr learned my addr 172.21.15.47:0/3756537525 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:00:44.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.529+0000 7fecc0b58640 1 --2- 172.21.15.47:0/3756537525 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da86683800 0x55da8668f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:44.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.529+0000 7fecc1359640 1 --2- 172.21.15.47:0/3756537525 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55da8667e690 0x55da86688480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:44.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.529+0000 7fecc0b58640 1 -- 172.21.15.47:0/3756537525 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da86683800 msgr2=0x55da8668f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:00:44.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.529+0000 7fecc0b58640 1 --2- 172.21.15.47:0/3756537525 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da86683800 0x55da8668f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:44.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.529+0000 7fecc0b58640 1 -- 172.21.15.47:0/3756537525 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55da8667e690 msgr2=0x55da86688480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:00:44.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.529+0000 7fecc0b58640 1 --2- 172.21.15.47:0/3756537525 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55da8667e690 0x55da86688480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:44.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.529+0000 7fecc0b58640 1 -- 172.21.15.47:0/3756537525 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da862f62d0 con 0x55da8667dca0 2024-04-17T06:00:44.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.529+0000 7fecc0b58640 1 --2- 172.21.15.47:0/3756537525 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da8667dca0 0x55da86687f40 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fecb800eec0 tx=0x7fecb8005ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14d27946b8e6ba89 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:00:44.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.529+0000 7feca4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-17T06:00:44.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.529+0000 7feca4ff9640 1 -- 172.21.15.47:0/3756537525 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb8018070 con 0x55da8667dca0 2024-04-17T06:00:44.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.529+0000 7feca4ff9640 1 -- 172.21.15.47:0/3756537525 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fecb8010a90 con 0x55da8667dca0 2024-04-17T06:00:44.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.529+0000 7fecc61a9740 1 -- 172.21.15.47:0/3756537525 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fecb80047a0 con 0x55da8667dca0 2024-04-17T06:00:44.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.529+0000 7feca4ff9640 1 -- 172.21.15.47:0/3756537525 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fecb8031050 con 0x55da8667dca0 2024-04-17T06:00:44.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.529+0000 7fecc61a9740 10 client.15108 fetch_fsmap learned FSMap version 21 2024-04-17T06:00:44.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.529+0000 7fecc61a9740 1 -- 172.21.15.47:0/3756537525 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55da863c06b0 con 0x55da8667dca0 2024-04-17T06:00:44.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.530+0000 7feca4ff9640 1 -- 172.21.15.47:0/3756537525 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fecb80047a0 con 0x55da8667dca0 2024-04-17T06:00:44.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.530+0000 7feca4ff9640 1 -- 172.21.15.47:0/3756537525 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 21) ==== 8339+0+0 (secure 0 0 0) 0x7fecb801cab0 con 0x55da8667dca0 2024-04-17T06:00:44.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.530+0000 7fecc61a9740 10 client.15108 fetch_fsmap finished waiting for FSMap version 21 2024-04-17T06:00:44.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.530+0000 7fecc61a9740 10 client.15108 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-17T06:00:44.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.530+0000 7fecc61a9740 20 client.15108 populate_metadata read hostname 'smithi047' 2024-04-17T06:00:44.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.530+0000 7fecc61a9740 1 --2- 172.21.15.47:0/3756537525 >> [v2:172.21.15.98:6836/1117440719,v1:172.21.15.98:6837/1117440719] conn(0x55da866948e0 0x55da86695d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:44.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.530+0000 7fecc61a9740 4 client.15108 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T06:00:44.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.530+0000 7fecc61a9740 1 -- 172.21.15.47:0/3756537525 --> [v2:172.21.15.98:6836/1117440719,v1:172.21.15.98:6837/1117440719] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55da8656a4c0 con 0x55da866948e0 2024-04-17T06:00:44.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.530+0000 7fecc1359640 1 --2- 172.21.15.47:0/3756537525 >> [v2:172.21.15.98:6836/1117440719,v1:172.21.15.98:6837/1117440719] conn(0x55da866948e0 0x55da86695d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:44.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.530+0000 7feca4ff9640 10 client.15108 ms_handle_connect on v2:172.21.15.98:6836/1117440719 2024-04-17T06:00:44.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.530+0000 7fecc1359640 1 --2- 172.21.15.47:0/3756537525 >> [v2:172.21.15.98:6836/1117440719,v1:172.21.15.98:6837/1117440719] conn(0x55da866948e0 0x55da86695d50 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9c492dfc9fb22dfd server_cookie=2f4039dc862d4301 in_seq=0 out_seq=0 2024-04-17T06:00:44.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.531+0000 7feca4ff9640 1 -- 172.21.15.47:0/3756537525 <== mds.1 v2:172.21.15.98:6836/1117440719 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55da8656a4c0 con 0x55da866948e0 2024-04-17T06:00:44.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.531+0000 7feca4ff9640 10 client.15108 handle_command_reply: tid=0 2024-04-17T06:00:44.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.531+0000 7fecc61a9740 1 client.15108 shutdown 2024-04-17T06:00:44.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.531+0000 7fecc61a9740 1 -- 172.21.15.47:0/3756537525 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da8667dca0 msgr2=0x55da86687f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:44.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.531+0000 7fecc61a9740 1 --2- 172.21.15.47:0/3756537525 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da8667dca0 0x55da86687f40 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fecb800eec0 tx=0x7fecb8005ce0 comp rx=0 tx=0).stop 2024-04-17T06:00:44.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.531+0000 7fecc61a9740 1 -- 172.21.15.47:0/3756537525 shutdown_connections 2024-04-17T06:00:44.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.531+0000 7fecc61a9740 1 --2- 172.21.15.47:0/3756537525 >> [v2:172.21.15.98:6836/1117440719,v1:172.21.15.98:6837/1117440719] conn(0x55da866948e0 0x55da86695d50 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:44.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.531+0000 7fecc61a9740 1 --2- 172.21.15.47:0/3756537525 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da86683800 0x55da8668f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:44.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.531+0000 7fecc61a9740 1 --2- 172.21.15.47:0/3756537525 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55da8667e690 0x55da86688480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:44.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.531+0000 7feca4ff9640 0 client.15108 ms_handle_reset on v2:172.21.15.98:6836/1117440719 2024-04-17T06:00:44.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.531+0000 7fecc61a9740 1 --2- 172.21.15.47:0/3756537525 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da8667dca0 0x55da86687f40 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:44.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.531+0000 7fecc61a9740 1 -- 172.21.15.47:0/3756537525 >> 172.21.15.47:0/3756537525 conn(0x55da86679210 msgr2=0x55da86678350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:00:44.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.532+0000 7fecc61a9740 1 -- 172.21.15.47:0/3756537525 shutdown_connections 2024-04-17T06:00:44.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.532+0000 7fecc61a9740 1 -- 172.21.15.47:0/3756537525 wait complete. 2024-04-17T06:00:44.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.532+0000 7fecc61a9740 20 client.15108 trim_cache size 0 max 16384 2024-04-17T06:00:44.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.550+0000 7fecc61a9740 1 Processor -- start 2024-04-17T06:00:44.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.550+0000 7fecc61a9740 1 -- start start 2024-04-17T06:00:44.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.550+0000 7fecc61a9740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da8667dca0 0x55da866795f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:44.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.550+0000 7fecc61a9740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55da8667e690 0x55da8667fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:44.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.550+0000 7fecc61a9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da86683800 0x55da866800d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:44.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.550+0000 7fecc61a9740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55da86694090 con 0x55da8667dca0 2024-04-17T06:00:44.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.550+0000 7fecc61a9740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55da86694760 con 0x55da8667e690 2024-04-17T06:00:44.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.550+0000 7fecc61a9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55da86694e30 con 0x55da86683800 2024-04-17T06:00:44.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.550+0000 7fecc0b58640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da8667dca0 0x55da866795f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:44.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.550+0000 7fecc0b58640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da8667dca0 0x55da866795f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53294/0 (socket says 172.21.15.47:53294) 2024-04-17T06:00:44.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.550+0000 7fecc0b58640 1 -- 172.21.15.47:0/3968751268 learned_addr learned my addr 172.21.15.47:0/3968751268 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:00:44.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.551+0000 7fecc0b58640 1 -- 172.21.15.47:0/3968751268 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da86683800 msgr2=0x55da866800d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:00:44.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.551+0000 7fecc0b58640 1 --2- 172.21.15.47:0/3968751268 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da86683800 0x55da866800d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:44.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.551+0000 7fecc0b58640 1 -- 172.21.15.47:0/3968751268 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55da8667e690 msgr2=0x55da8667fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:00:44.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.551+0000 7fecc0b58640 1 --2- 172.21.15.47:0/3968751268 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55da8667e690 0x55da8667fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:44.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.551+0000 7fecc0b58640 1 -- 172.21.15.47:0/3968751268 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da86682e80 con 0x55da8667dca0 2024-04-17T06:00:44.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.551+0000 7fecc0b58640 1 --2- 172.21.15.47:0/3968751268 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da8667dca0 0x55da866795f0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fecb801ce80 tx=0x7fecb8005a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2df1cad702b15993 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:00:44.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.551+0000 7feca6ffd640 1 -- 172.21.15.47:0/3968751268 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb802e070 con 0x55da8667dca0 2024-04-17T06:00:44.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.551+0000 7feca6ffd640 1 -- 172.21.15.47:0/3968751268 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fecb800e590 con 0x55da8667dca0 2024-04-17T06:00:44.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.551+0000 7feca6ffd640 1 -- 172.21.15.47:0/3968751268 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb8020030 con 0x55da8667dca0 2024-04-17T06:00:44.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.552+0000 7fecc61a9740 1 -- 172.21.15.47:0/3968751268 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da8667dca0 msgr2=0x55da866795f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:44.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.552+0000 7fecc61a9740 1 --2- 172.21.15.47:0/3968751268 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da8667dca0 0x55da866795f0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fecb801ce80 tx=0x7fecb8005a90 comp rx=0 tx=0).stop 2024-04-17T06:00:44.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.552+0000 7fecc61a9740 1 -- 172.21.15.47:0/3968751268 shutdown_connections 2024-04-17T06:00:44.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.552+0000 7fecc61a9740 1 --2- 172.21.15.47:0/3968751268 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da86683800 0x55da866800d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:44.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.552+0000 7fecc61a9740 1 --2- 172.21.15.47:0/3968751268 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55da8667e690 0x55da8667fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:44.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.552+0000 7fecc61a9740 1 --2- 172.21.15.47:0/3968751268 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da8667dca0 0x55da866795f0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:44.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.552+0000 7fecc61a9740 1 -- 172.21.15.47:0/3968751268 >> 172.21.15.47:0/3968751268 conn(0x55da86679210 msgr2=0x55da86673fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:00:44.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.552+0000 7fecc61a9740 1 -- 172.21.15.47:0/3968751268 shutdown_connections 2024-04-17T06:00:44.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.552+0000 7fecc61a9740 1 -- 172.21.15.47:0/3968751268 wait complete. 2024-04-17T06:00:44.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.552+0000 7fecc61a9740 1 -- start start 2024-04-17T06:00:44.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.552+0000 7fecc61a9740 1 Processor -- start 2024-04-17T06:00:44.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.552+0000 7fecc61a9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da8667dca0 0x55da8668c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:44.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.552+0000 7fecc61a9740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da8667e690 0x55da866f6870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:44.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.552+0000 7fecc61a9740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55da86683800 0x55da866f8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:44.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.552+0000 7fecc61a9740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55da86695bd0 con 0x55da8667e690 2024-04-17T06:00:44.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.552+0000 7fecc61a9740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55da86695500 con 0x55da86683800 2024-04-17T06:00:44.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.552+0000 7fecc61a9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55da866962a0 con 0x55da8667dca0 2024-04-17T06:00:44.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.552+0000 7fecc61a9740 10 client.0 _ll_register_callbacks cb 0x7fecb000a0d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:00:44.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.552+0000 7fecc1359640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da8667e690 0x55da866f6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:44.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.552+0000 7fecc1359640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da8667e690 0x55da866f6870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53302/0 (socket says 172.21.15.47:53302) 2024-04-17T06:00:44.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.553+0000 7fecc1359640 1 -- 172.21.15.47:0/2700306331 learned_addr learned my addr 172.21.15.47:0/2700306331 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:00:44.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.553+0000 7fecc0b58640 1 --2- 172.21.15.47:0/2700306331 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da8667dca0 0x55da8668c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:44.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.553+0000 7fecc0b58640 1 --2- 172.21.15.47:0/2700306331 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55da86683800 0x55da866f8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:44.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.553+0000 7fecc1359640 1 -- 172.21.15.47:0/2700306331 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da8667dca0 msgr2=0x55da8668c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:00:44.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.553+0000 7fecc1359640 1 --2- 172.21.15.47:0/2700306331 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da8667dca0 0x55da8668c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:44.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.553+0000 7fecc1359640 1 -- 172.21.15.47:0/2700306331 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55da86683800 msgr2=0x55da866f8dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:00:44.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.553+0000 7fecc1359640 1 --2- 172.21.15.47:0/2700306331 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55da86683800 0x55da866f8dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:44.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.553+0000 7fecc1359640 1 -- 172.21.15.47:0/2700306331 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da866f9300 con 0x55da8667e690 2024-04-17T06:00:44.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.553+0000 7fecc0b58640 1 --2- 172.21.15.47:0/2700306331 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55da86683800 0x55da866f8dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:00:44.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.553+0000 7fecc1359640 1 --2- 172.21.15.47:0/2700306331 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da8667e690 0x55da866f6870 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fecb0000c00 tx=0x7fecb001cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee93bdfc10f0a5b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:00:44.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.553+0000 7feca4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-17T06:00:44.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.553+0000 7feca4ff9640 1 -- 172.21.15.47:0/2700306331 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb0005c90 con 0x55da8667e690 2024-04-17T06:00:44.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.553+0000 7feca4ff9640 1 -- 172.21.15.47:0/2700306331 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fecb00169a0 con 0x55da8667e690 2024-04-17T06:00:44.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.553+0000 7fecc61a9740 1 -- 172.21.15.47:0/2700306331 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fecb800e590 con 0x55da8667e690 2024-04-17T06:00:44.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.553+0000 7feca4ff9640 1 -- 172.21.15.47:0/2700306331 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fecb002c050 con 0x55da8667e690 2024-04-17T06:00:44.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.553+0000 7fecc61a9740 10 client.15114 fetch_fsmap learned FSMap version 21 2024-04-17T06:00:44.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.553+0000 7fecc61a9740 1 -- 172.21.15.47:0/2700306331 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55da866fa6a0 con 0x55da8667e690 2024-04-17T06:00:44.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.553+0000 7feca4ff9640 1 -- 172.21.15.47:0/2700306331 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fecb800e590 con 0x55da8667e690 2024-04-17T06:00:44.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.554+0000 7feca4ff9640 1 -- 172.21.15.47:0/2700306331 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 21) ==== 8339+0+0 (secure 0 0 0) 0x7fecb0006050 con 0x55da8667e690 2024-04-17T06:00:44.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.554+0000 7fecc61a9740 10 client.15114 fetch_fsmap finished waiting for FSMap version 21 2024-04-17T06:00:44.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.554+0000 7fecc61a9740 10 client.15114 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-17T06:00:44.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.554+0000 7fecc61a9740 20 client.15114 populate_metadata read hostname 'smithi047' 2024-04-17T06:00:44.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.554+0000 7fecc61a9740 1 --2- 172.21.15.47:0/2700306331 >> [v2:172.21.15.98:6836/1117440719,v1:172.21.15.98:6837/1117440719] conn(0x55da866ff280 0x55da866fdf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:44.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.554+0000 7fecc61a9740 4 client.15114 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-17T06:00:44.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.554+0000 7fecc61a9740 1 -- 172.21.15.47:0/2700306331 --> [v2:172.21.15.98:6836/1117440719,v1:172.21.15.98:6837/1117440719] -- command(tid 0: {"prefix": "status"}) -- 0x55da865f14e0 con 0x55da866ff280 2024-04-17T06:00:44.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.554+0000 7fecc0b58640 1 --2- 172.21.15.47:0/2700306331 >> [v2:172.21.15.98:6836/1117440719,v1:172.21.15.98:6837/1117440719] conn(0x55da866ff280 0x55da866fdf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:44.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.554+0000 7feca4ff9640 10 client.15114 ms_handle_connect on v2:172.21.15.98:6836/1117440719 2024-04-17T06:00:44.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.554+0000 7fecc0b58640 1 --2- 172.21.15.47:0/2700306331 >> [v2:172.21.15.98:6836/1117440719,v1:172.21.15.98:6837/1117440719] conn(0x55da866ff280 0x55da866fdf90 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f87184bce7c0e69c server_cookie=55c57c8cd9d409b in_seq=0 out_seq=0 2024-04-17T06:00:44.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.555+0000 7feca4ff9640 1 -- 172.21.15.47:0/2700306331 <== mds.1 v2:172.21.15.98:6836/1117440719 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55da865f14e0 con 0x55da866ff280 2024-04-17T06:00:44.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.555+0000 7feca4ff9640 10 client.15114 handle_command_reply: tid=0 2024-04-17T06:00:44.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.555+0000 7fecc61a9740 1 client.15114 shutdown 2024-04-17T06:00:44.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.555+0000 7fecc61a9740 1 -- 172.21.15.47:0/2700306331 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da8667e690 msgr2=0x55da866f6870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:44.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.555+0000 7fecc61a9740 1 --2- 172.21.15.47:0/2700306331 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da8667e690 0x55da866f6870 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fecb0000c00 tx=0x7fecb001cf20 comp rx=0 tx=0).stop 2024-04-17T06:00:44.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.555+0000 7fecc61a9740 1 -- 172.21.15.47:0/2700306331 shutdown_connections 2024-04-17T06:00:44.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.555+0000 7fecc61a9740 1 --2- 172.21.15.47:0/2700306331 >> [v2:172.21.15.98:6836/1117440719,v1:172.21.15.98:6837/1117440719] conn(0x55da866ff280 0x55da866fdf90 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:44.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.555+0000 7fecc61a9740 1 --2- 172.21.15.47:0/2700306331 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55da86683800 0x55da866f8dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:44.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.555+0000 7fecc61a9740 1 --2- 172.21.15.47:0/2700306331 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da8667e690 0x55da866f6870 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:44.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.555+0000 7feca4ff9640 0 client.15114 ms_handle_reset on v2:172.21.15.98:6836/1117440719 2024-04-17T06:00:44.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.555+0000 7fecc61a9740 1 --2- 172.21.15.47:0/2700306331 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da8667dca0 0x55da8668c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:44.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.555+0000 7fecc61a9740 1 -- 172.21.15.47:0/2700306331 >> 172.21.15.47:0/2700306331 conn(0x55da86679210 msgr2=0x55da866795f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:00:44.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.555+0000 7fecc61a9740 1 -- 172.21.15.47:0/2700306331 shutdown_connections 2024-04-17T06:00:44.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.555+0000 7fecc61a9740 1 -- 172.21.15.47:0/2700306331 wait complete. 2024-04-17T06:00:44.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.555+0000 7fecc61a9740 20 client.15114 trim_cache size 0 max 16384 2024-04-17T06:00:44.561 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-17T06:00:44.561 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_fsid": "565db2d8-fc7e-11ee-bc91-c7b262605968", 2024-04-17T06:00:44.561 INFO:teuthology.orchestra.run.smithi047.stdout: "whoami": 1, 2024-04-17T06:00:44.561 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 24479, 2024-04-17T06:00:44.561 INFO:teuthology.orchestra.run.smithi047.stdout: "want_state": "up:active", 2024-04-17T06:00:44.561 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "up:active", 2024-04-17T06:00:44.561 INFO:teuthology.orchestra.run.smithi047.stdout: "fs_name": "cephfs", 2024-04-17T06:00:44.561 INFO:teuthology.orchestra.run.smithi047.stdout: "rank_uptime": 86.939906363000006, 2024-04-17T06:00:44.562 INFO:teuthology.orchestra.run.smithi047.stdout: "mdsmap_epoch": 21, 2024-04-17T06:00:44.562 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch": 88, 2024-04-17T06:00:44.562 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch_barrier": 86, 2024-04-17T06:00:44.562 INFO:teuthology.orchestra.run.smithi047.stdout: "uptime": 107.396896504 2024-04-17T06:00:44.562 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-17T06:00:44.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.555+0000 7fecc35bd640 1 -- 172.21.15.47:0/3783321072 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7feca80a3c30 msgr2=0x7feca80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:44.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.555+0000 7fecc35bd640 1 --2- 172.21.15.47:0/3783321072 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7feca80a3c30 0x7feca80a60f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fecb40bec70 tx=0x7fecb40ea080 comp rx=0 tx=0).stop 2024-04-17T06:00:44.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.555+0000 7fecc35bd640 1 -- 172.21.15.47:0/3783321072 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fecbc107ca0 msgr2=0x7fecbc100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:44.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.555+0000 7fecc35bd640 1 --2- 172.21.15.47:0/3783321072 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fecbc107ca0 0x7fecbc100f30 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fecb4037040 tx=0x7fecb4032f90 comp rx=0 tx=0).stop 2024-04-17T06:00:44.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.555+0000 7fecc35bd640 1 -- 172.21.15.47:0/3783321072 shutdown_connections 2024-04-17T06:00:44.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.555+0000 7fecc35bd640 1 --2- 172.21.15.47:0/3783321072 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7feca80a3c30 0x7feca80a60f0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:44.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.555+0000 7fecc35bd640 1 --2- 172.21.15.47:0/3783321072 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fecbc107ca0 0x7fecbc100f30 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:44.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.555+0000 7fecc35bd640 1 --2- 172.21.15.47:0/3783321072 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fecbc105370 0x7fecbc1009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:44.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.555+0000 7fecc35bd640 1 --2- 172.21.15.47:0/3783321072 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fecbc102a40 0x7fecbc1023b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:44.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.555+0000 7fecc35bd640 1 -- 172.21.15.47:0/3783321072 >> 172.21.15.47:0/3783321072 conn(0x7fecbc0fc450 msgr2=0x7fecbc103380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:00:44.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.556+0000 7fecc35bd640 1 -- 172.21.15.47:0/3783321072 shutdown_connections 2024-04-17T06:00:44.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.556+0000 7fecc35bd640 1 -- 172.21.15.47:0/3783321072 wait complete. 2024-04-17T06:00:44.565 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-17T06:00:44.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.675+0000 7f8601359640 1 -- 172.21.15.47:0/3818092654 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85fc102a80 msgr2=0x7f85fc104e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:44.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.675+0000 7f8601359640 1 --2- 172.21.15.47:0/3818092654 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85fc102a80 0x7f85fc104e70 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f85ec009900 tx=0x7f85ec02bc30 comp rx=0 tx=0).stop 2024-04-17T06:00:44.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.675+0000 7f8601359640 1 -- 172.21.15.47:0/3818092654 shutdown_connections 2024-04-17T06:00:44.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.675+0000 7f8601359640 1 --2- 172.21.15.47:0/3818092654 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f85fc107ce0 0x7f85fc10a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:44.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.675+0000 7f8601359640 1 --2- 172.21.15.47:0/3818092654 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f85fc1053b0 0x7f85fc1077a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:44.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.675+0000 7f8601359640 1 --2- 172.21.15.47:0/3818092654 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85fc102a80 0x7f85fc104e70 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:44.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.675+0000 7f8601359640 1 -- 172.21.15.47:0/3818092654 >> 172.21.15.47:0/3818092654 conn(0x7f85fc0fc450 msgr2=0x7f85fc0fe870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:00:44.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.675+0000 7f8601359640 1 -- 172.21.15.47:0/3818092654 shutdown_connections 2024-04-17T06:00:44.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.675+0000 7f8601359640 1 -- 172.21.15.47:0/3818092654 wait complete. 2024-04-17T06:00:44.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.676+0000 7f8601359640 1 Processor -- start 2024-04-17T06:00:44.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.676+0000 7f8601359640 1 -- start start 2024-04-17T06:00:44.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.676+0000 7f8601359640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f85fc102a80 0x7f85fc1983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:44.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.676+0000 7f8601359640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f85fc1053b0 0x7f85fc1988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:44.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.676+0000 7f8601359640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85fc107ce0 0x7f85fc19f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:44.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.676+0000 7f8601359640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f85fc10dcb0 con 0x7f85fc107ce0 2024-04-17T06:00:44.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.676+0000 7f8601359640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f85fc10db30 con 0x7f85fc102a80 2024-04-17T06:00:44.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.676+0000 7f8601359640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f85fc10de30 con 0x7f85fc1053b0 2024-04-17T06:00:44.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.676+0000 7f85fb7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85fc107ce0 0x7f85fc19f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:44.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.676+0000 7f85fb7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85fc107ce0 0x7f85fc19f960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53320/0 (socket says 172.21.15.47:53320) 2024-04-17T06:00:44.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.676+0000 7f85fb7fe640 1 -- 172.21.15.47:0/4238570587 learned_addr learned my addr 172.21.15.47:0/4238570587 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:00:44.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.676+0000 7f85faffd640 1 --2- 172.21.15.47:0/4238570587 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f85fc102a80 0x7f85fc1983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:44.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.676+0000 7f85fa7fc640 1 --2- 172.21.15.47:0/4238570587 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f85fc1053b0 0x7f85fc1988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:44.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.676+0000 7f85fb7fe640 1 -- 172.21.15.47:0/4238570587 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f85fc1053b0 msgr2=0x7f85fc1988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:44.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.676+0000 7f85fb7fe640 1 --2- 172.21.15.47:0/4238570587 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f85fc1053b0 0x7f85fc1988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:44.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.676+0000 7f85fb7fe640 1 -- 172.21.15.47:0/4238570587 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f85fc102a80 msgr2=0x7f85fc1983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:44.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.676+0000 7f85fb7fe640 1 --2- 172.21.15.47:0/4238570587 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f85fc102a80 0x7f85fc1983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:44.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.676+0000 7f85fb7fe640 1 -- 172.21.15.47:0/4238570587 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85fc19ffd0 con 0x7f85fc107ce0 2024-04-17T06:00:44.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.676+0000 7f85faffd640 1 --2- 172.21.15.47:0/4238570587 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f85fc102a80 0x7f85fc1983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:00:44.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.676+0000 7f85fa7fc640 1 --2- 172.21.15.47:0/4238570587 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f85fc1053b0 0x7f85fc1988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:00:44.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.677+0000 7f85fb7fe640 1 --2- 172.21.15.47:0/4238570587 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85fc107ce0 0x7f85fc19f960 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f85f400b970 tx=0x7f85f400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:00:44.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.677+0000 7f85e3fff640 1 -- 172.21.15.47:0/4238570587 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f4019070 con 0x7f85fc107ce0 2024-04-17T06:00:44.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.677+0000 7f8601359640 1 -- 172.21.15.47:0/4238570587 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85fc1a02c0 con 0x7f85fc107ce0 2024-04-17T06:00:44.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.677+0000 7f85e3fff640 1 -- 172.21.15.47:0/4238570587 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85f400cb40 con 0x7f85fc107ce0 2024-04-17T06:00:44.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.677+0000 7f85e3fff640 1 -- 172.21.15.47:0/4238570587 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f4007650 con 0x7f85fc107ce0 2024-04-17T06:00:44.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.677+0000 7f8601359640 1 -- 172.21.15.47:0/4238570587 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85fc1a0810 con 0x7f85fc107ce0 2024-04-17T06:00:44.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.679+0000 7f85e3fff640 1 -- 172.21.15.47:0/4238570587 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f85f401c070 con 0x7f85fc107ce0 2024-04-17T06:00:44.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.679+0000 7f8604021740 1 Processor -- start 2024-04-17T06:00:44.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.679+0000 7f8604021740 1 -- start start 2024-04-17T06:00:44.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.680+0000 7f85e3fff640 1 --2- 172.21.15.47:0/4238570587 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f85e40a3bb0 0x7f85e40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:44.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.680+0000 7f8604021740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da5e437ca0 0x55da5e4380c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:44.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.680+0000 7f8604021740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da5e438690 0x55da5e43d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:44.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.680+0000 7f8604021740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55da5e43d800 0x55da5e43fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:44.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.680+0000 7f8604021740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55da5e34b240 con 0x55da5e437ca0 2024-04-17T06:00:44.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.680+0000 7f8604021740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55da5e3a89f0 con 0x55da5e43d800 2024-04-17T06:00:44.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.680+0000 7f8604021740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55da5e3639e0 con 0x55da5e438690 2024-04-17T06:00:44.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.680+0000 7f85fa7fc640 1 --2- 172.21.15.47:0/4238570587 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f85e40a3bb0 0x7f85e40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:44.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.680+0000 7f85faffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da5e437ca0 0x55da5e4380c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:44.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.680+0000 7f85faffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da5e437ca0 0x55da5e4380c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53322/0 (socket says 172.21.15.47:53322) 2024-04-17T06:00:44.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.680+0000 7f85faffd640 1 -- 172.21.15.47:0/2298439055 learned_addr learned my addr 172.21.15.47:0/2298439055 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:00:44.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.680+0000 7f85e3fff640 1 -- 172.21.15.47:0/4238570587 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f85f40cfa50 con 0x7f85fc107ce0 2024-04-17T06:00:44.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.680+0000 7f85fa7fc640 1 --2- 172.21.15.47:0/2298439055 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da5e438690 0x55da5e43d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:44.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.680+0000 7f85fb7fe640 1 --2- 172.21.15.47:0/2298439055 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55da5e43d800 0x55da5e43fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:44.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.680+0000 7f85faffd640 1 -- 172.21.15.47:0/2298439055 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da5e438690 msgr2=0x55da5e43d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:00:44.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.680+0000 7f85faffd640 1 --2- 172.21.15.47:0/2298439055 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da5e438690 0x55da5e43d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:44.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.680+0000 7f85faffd640 1 -- 172.21.15.47:0/2298439055 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55da5e43d800 msgr2=0x55da5e43fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:00:44.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.680+0000 7f85faffd640 1 --2- 172.21.15.47:0/2298439055 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55da5e43d800 0x55da5e43fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:44.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.680+0000 7f85faffd640 1 -- 172.21.15.47:0/2298439055 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da5e356bc0 con 0x55da5e437ca0 2024-04-17T06:00:44.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.680+0000 7f85faffd640 1 --2- 172.21.15.47:0/2298439055 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da5e437ca0 0x55da5e4380c0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f85ec036040 tx=0x7f85ec005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d04ea3b4748350f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:00:44.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.680+0000 7f85fa7fc640 1 --2- 172.21.15.47:0/4238570587 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f85e40a3bb0 0x7f85e40a6070 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f85f0005fd0 tx=0x7f85f0014840 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:00:44.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.680+0000 7f85e17fa640 1 -- 172.21.15.47:0/2298439055 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85ec02cc80 con 0x55da5e437ca0 2024-04-17T06:00:44.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.680+0000 7f85e17fa640 1 -- 172.21.15.47:0/2298439055 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85ec005000 con 0x55da5e437ca0 2024-04-17T06:00:44.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.680+0000 7f85e17fa640 1 -- 172.21.15.47:0/2298439055 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85ec0049d0 con 0x55da5e437ca0 2024-04-17T06:00:44.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.681+0000 7f8604021740 1 -- 172.21.15.47:0/2298439055 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da5e437ca0 msgr2=0x55da5e4380c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:44.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.681+0000 7f8604021740 1 --2- 172.21.15.47:0/2298439055 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da5e437ca0 0x55da5e4380c0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f85ec036040 tx=0x7f85ec005dd0 comp rx=0 tx=0).stop 2024-04-17T06:00:44.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.681+0000 7f8604021740 1 -- 172.21.15.47:0/2298439055 shutdown_connections 2024-04-17T06:00:44.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.681+0000 7f8604021740 1 --2- 172.21.15.47:0/2298439055 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55da5e43d800 0x55da5e43fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:44.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.681+0000 7f8604021740 1 --2- 172.21.15.47:0/2298439055 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da5e438690 0x55da5e43d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:44.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.681+0000 7f8604021740 1 --2- 172.21.15.47:0/2298439055 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da5e437ca0 0x55da5e4380c0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:44.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.681+0000 7f8604021740 1 -- 172.21.15.47:0/2298439055 >> 172.21.15.47:0/2298439055 conn(0x55da5e433210 msgr2=0x55da5e433610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:00:44.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.681+0000 7f8604021740 1 -- 172.21.15.47:0/2298439055 shutdown_connections 2024-04-17T06:00:44.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.681+0000 7f8604021740 1 -- 172.21.15.47:0/2298439055 wait complete. 2024-04-17T06:00:44.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.681+0000 7f8604021740 1 -- start start 2024-04-17T06:00:44.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.682+0000 7f8604021740 1 Processor -- start 2024-04-17T06:00:44.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.682+0000 7f8604021740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55da5e437ca0 0x55da5e441e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:44.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.682+0000 7f8604021740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da5e438690 0x55da5e442370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:44.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.682+0000 7f8604021740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da5e43d800 0x55da5e4493b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:44.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.682+0000 7f8604021740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55da5e3a20f0 con 0x55da5e43d800 2024-04-17T06:00:44.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.682+0000 7f8604021740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55da5e39cdd0 con 0x55da5e437ca0 2024-04-17T06:00:44.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.682+0000 7f8604021740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55da5e3aacd0 con 0x55da5e438690 2024-04-17T06:00:44.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.682+0000 7f8604021740 10 client.0 _ll_register_callbacks cb 0x55da5e393570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:00:44.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.682+0000 7f85fb7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da5e43d800 0x55da5e4493b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:44.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.682+0000 7f85fb7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da5e43d800 0x55da5e4493b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53338/0 (socket says 172.21.15.47:53338) 2024-04-17T06:00:44.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.682+0000 7f85fb7fe640 1 -- 172.21.15.47:0/927942137 learned_addr learned my addr 172.21.15.47:0/927942137 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:00:44.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.682+0000 7f85faffd640 1 --2- 172.21.15.47:0/927942137 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55da5e437ca0 0x55da5e441e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:44.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.682+0000 7f85fb7fe640 1 -- 172.21.15.47:0/927942137 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da5e438690 msgr2=0x55da5e442370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:00:44.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.682+0000 7f85fa7fc640 1 --2- 172.21.15.47:0/927942137 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da5e438690 0x55da5e442370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:44.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.682+0000 7f85fb7fe640 1 --2- 172.21.15.47:0/927942137 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da5e438690 0x55da5e442370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:44.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.682+0000 7f85fb7fe640 1 -- 172.21.15.47:0/927942137 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55da5e437ca0 msgr2=0x55da5e441e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:00:44.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.682+0000 7f85fb7fe640 1 --2- 172.21.15.47:0/927942137 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55da5e437ca0 0x55da5e441e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:44.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.682+0000 7f85fb7fe640 1 -- 172.21.15.47:0/927942137 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da5e0b02d0 con 0x55da5e43d800 2024-04-17T06:00:44.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.682+0000 7f85fa7fc640 1 --2- 172.21.15.47:0/927942137 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da5e438690 0x55da5e442370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:00:44.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.682+0000 7f85fb7fe640 1 --2- 172.21.15.47:0/927942137 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da5e43d800 0x55da5e4493b0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f85f40981a0 tx=0x7f85f40791d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4c38bc3d10e7944 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:00:44.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.682+0000 7f85deff5640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-17T06:00:44.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.682+0000 7f85deff5640 1 -- 172.21.15.47:0/927942137 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f4078fa0 con 0x55da5e43d800 2024-04-17T06:00:44.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.682+0000 7f85deff5640 1 -- 172.21.15.47:0/927942137 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f85f40983b0 con 0x55da5e43d800 2024-04-17T06:00:44.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.682+0000 7f8604021740 1 -- 172.21.15.47:0/927942137 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85ec005000 con 0x55da5e43d800 2024-04-17T06:00:44.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.683+0000 7f85deff5640 1 -- 172.21.15.47:0/927942137 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f85f409bd40 con 0x55da5e43d800 2024-04-17T06:00:44.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.683+0000 7f8604021740 10 client.15126 fetch_fsmap learned FSMap version 21 2024-04-17T06:00:44.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.683+0000 7f8604021740 1 -- 172.21.15.47:0/927942137 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55da5e17a6b0 con 0x55da5e43d800 2024-04-17T06:00:44.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.683+0000 7f85deff5640 1 -- 172.21.15.47:0/927942137 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85ec005000 con 0x55da5e43d800 2024-04-17T06:00:44.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.683+0000 7f85deff5640 1 -- 172.21.15.47:0/927942137 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 21) ==== 8339+0+0 (secure 0 0 0) 0x7f85f4013720 con 0x55da5e43d800 2024-04-17T06:00:44.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.683+0000 7f8604021740 10 client.15126 fetch_fsmap finished waiting for FSMap version 21 2024-04-17T06:00:44.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.683+0000 7f8604021740 10 client.15126 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-17T06:00:44.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.683+0000 7f8604021740 20 client.15126 populate_metadata read hostname 'smithi047' 2024-04-17T06:00:44.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.683+0000 7f8604021740 1 --2- 172.21.15.47:0/927942137 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55da5e44eac0 0x55da5e44ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:44.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.683+0000 7f8604021740 4 client.15126 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T06:00:44.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.683+0000 7f8604021740 1 -- 172.21.15.47:0/927942137 --> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55da5e3244c0 con 0x55da5e44eac0 2024-04-17T06:00:44.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.683+0000 7f85faffd640 1 --2- 172.21.15.47:0/927942137 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55da5e44eac0 0x55da5e44ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:44.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.684+0000 7f85faffd640 1 --2- 172.21.15.47:0/927942137 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55da5e44eac0 0x55da5e44ff30 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d8c6f33e78020fe server_cookie=aba81fa426ccceac in_seq=0 out_seq=0 2024-04-17T06:00:44.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.684+0000 7f85deff5640 10 client.15126 ms_handle_connect on v2:172.21.15.98:6838/2841287917 2024-04-17T06:00:44.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.684+0000 7f85deff5640 1 -- 172.21.15.47:0/927942137 <== mds.0 v2:172.21.15.98:6838/2841287917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55da5e3244c0 con 0x55da5e44eac0 2024-04-17T06:00:44.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.684+0000 7f85deff5640 10 client.15126 handle_command_reply: tid=0 2024-04-17T06:00:44.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.684+0000 7f8604021740 1 client.15126 shutdown 2024-04-17T06:00:44.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.684+0000 7f8604021740 1 -- 172.21.15.47:0/927942137 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da5e43d800 msgr2=0x55da5e4493b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:44.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.684+0000 7f8604021740 1 --2- 172.21.15.47:0/927942137 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da5e43d800 0x55da5e4493b0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f85f40981a0 tx=0x7f85f40791d0 comp rx=0 tx=0).stop 2024-04-17T06:00:44.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.685+0000 7f8604021740 1 -- 172.21.15.47:0/927942137 shutdown_connections 2024-04-17T06:00:44.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.685+0000 7f8604021740 1 --2- 172.21.15.47:0/927942137 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55da5e44eac0 0x55da5e44ff30 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:44.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.685+0000 7f8604021740 1 --2- 172.21.15.47:0/927942137 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da5e43d800 0x55da5e4493b0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:44.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.685+0000 7f85deff5640 0 client.15126 ms_handle_reset on v2:172.21.15.98:6838/2841287917 2024-04-17T06:00:44.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.685+0000 7f8604021740 1 --2- 172.21.15.47:0/927942137 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da5e438690 0x55da5e442370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:44.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.685+0000 7f8604021740 1 --2- 172.21.15.47:0/927942137 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55da5e437ca0 0x55da5e441e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:44.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.685+0000 7f8604021740 1 -- 172.21.15.47:0/927942137 >> 172.21.15.47:0/927942137 conn(0x55da5e433210 msgr2=0x55da5e432350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:00:44.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.685+0000 7f8604021740 1 -- 172.21.15.47:0/927942137 shutdown_connections 2024-04-17T06:00:44.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.685+0000 7f8604021740 1 -- 172.21.15.47:0/927942137 wait complete. 2024-04-17T06:00:44.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.685+0000 7f8604021740 20 client.15126 trim_cache size 0 max 16384 2024-04-17T06:00:44.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.703+0000 7f8604021740 1 Processor -- start 2024-04-17T06:00:44.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.703+0000 7f8604021740 1 -- start start 2024-04-17T06:00:44.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.703+0000 7f8604021740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da5e437ca0 0x55da5e4335f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:44.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.703+0000 7f8604021740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da5e438690 0x55da5e449370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:44.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.703+0000 7f8604021740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55da5e43d800 0x55da5e431910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:44.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.703+0000 7f8604021740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55da5e44e270 con 0x55da5e437ca0 2024-04-17T06:00:44.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.703+0000 7f8604021740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55da5e44e940 con 0x55da5e43d800 2024-04-17T06:00:44.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.703+0000 7f8604021740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55da5e44f010 con 0x55da5e438690 2024-04-17T06:00:44.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.703+0000 7f85faffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da5e437ca0 0x55da5e4335f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:44.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.703+0000 7f85faffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da5e437ca0 0x55da5e4335f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53348/0 (socket says 172.21.15.47:53348) 2024-04-17T06:00:44.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.703+0000 7f85faffd640 1 -- 172.21.15.47:0/3544695231 learned_addr learned my addr 172.21.15.47:0/3544695231 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:00:44.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.703+0000 7f85fa7fc640 1 --2- 172.21.15.47:0/3544695231 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da5e438690 0x55da5e449370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:44.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.703+0000 7f85faffd640 1 -- 172.21.15.47:0/3544695231 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da5e438690 msgr2=0x55da5e449370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:00:44.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.703+0000 7f85faffd640 1 --2- 172.21.15.47:0/3544695231 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da5e438690 0x55da5e449370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:44.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.703+0000 7f85faffd640 1 -- 172.21.15.47:0/3544695231 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55da5e43d800 msgr2=0x55da5e431910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:00:44.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.703+0000 7f85faffd640 1 --2- 172.21.15.47:0/3544695231 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55da5e43d800 0x55da5e431910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:44.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.703+0000 7f85faffd640 1 -- 172.21.15.47:0/3544695231 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da5e4426e0 con 0x55da5e437ca0 2024-04-17T06:00:44.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.703+0000 7f85fa7fc640 1 --2- 172.21.15.47:0/3544695231 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da5e438690 0x55da5e449370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:00:44.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.703+0000 7f85faffd640 1 --2- 172.21.15.47:0/3544695231 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da5e437ca0 0x55da5e4335f0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f85ec009900 tx=0x7f85ec02bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc1ce367c8bb1446 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:00:44.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.704+0000 7f85e0ff9640 1 -- 172.21.15.47:0/3544695231 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85ec02c2b0 con 0x55da5e437ca0 2024-04-17T06:00:44.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.704+0000 7f85e0ff9640 1 -- 172.21.15.47:0/3544695231 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85ec005000 con 0x55da5e437ca0 2024-04-17T06:00:44.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.704+0000 7f85e0ff9640 1 -- 172.21.15.47:0/3544695231 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85ec004c30 con 0x55da5e437ca0 2024-04-17T06:00:44.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.704+0000 7f8604021740 1 -- 172.21.15.47:0/3544695231 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da5e437ca0 msgr2=0x55da5e4335f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:44.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.704+0000 7f8604021740 1 --2- 172.21.15.47:0/3544695231 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da5e437ca0 0x55da5e4335f0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f85ec009900 tx=0x7f85ec02bc30 comp rx=0 tx=0).stop 2024-04-17T06:00:44.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.704+0000 7f8604021740 1 -- 172.21.15.47:0/3544695231 shutdown_connections 2024-04-17T06:00:44.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.704+0000 7f8604021740 1 --2- 172.21.15.47:0/3544695231 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55da5e43d800 0x55da5e431910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:44.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.704+0000 7f8604021740 1 --2- 172.21.15.47:0/3544695231 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da5e438690 0x55da5e449370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:44.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.704+0000 7f8604021740 1 --2- 172.21.15.47:0/3544695231 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da5e437ca0 0x55da5e4335f0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:44.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.704+0000 7f8604021740 1 -- 172.21.15.47:0/3544695231 >> 172.21.15.47:0/3544695231 conn(0x55da5e433210 msgr2=0x55da5e44da30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:00:44.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.704+0000 7f8604021740 1 -- 172.21.15.47:0/3544695231 shutdown_connections 2024-04-17T06:00:44.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.704+0000 7f8604021740 1 -- 172.21.15.47:0/3544695231 wait complete. 2024-04-17T06:00:44.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.704+0000 7f8604021740 1 -- start start 2024-04-17T06:00:44.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.705+0000 7f8604021740 1 Processor -- start 2024-04-17T06:00:44.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.705+0000 7f8604021740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da5e437ca0 0x55da5e447360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:44.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.705+0000 7f8604021740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55da5e438690 0x55da5e4b0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:44.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.705+0000 7f8604021740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da5e43d800 0x55da5e4b2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:44.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.705+0000 7f8604021740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55da5e44fdb0 con 0x55da5e437ca0 2024-04-17T06:00:44.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.705+0000 7f8604021740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55da5e44f6e0 con 0x55da5e438690 2024-04-17T06:00:44.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.705+0000 7f8604021740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55da5e450480 con 0x55da5e43d800 2024-04-17T06:00:44.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.705+0000 7f8604021740 10 client.0 _ll_register_callbacks cb 0x7f85f4079590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:00:44.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.705+0000 7f85faffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da5e437ca0 0x55da5e447360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:44.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.705+0000 7f85faffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da5e437ca0 0x55da5e447360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53354/0 (socket says 172.21.15.47:53354) 2024-04-17T06:00:44.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.705+0000 7f85faffd640 1 -- 172.21.15.47:0/780303896 learned_addr learned my addr 172.21.15.47:0/780303896 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:00:44.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.705+0000 7f85fb7fe640 1 --2- 172.21.15.47:0/780303896 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da5e43d800 0x55da5e4b2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:44.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.705+0000 7f85fa7fc640 1 --2- 172.21.15.47:0/780303896 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55da5e438690 0x55da5e4b0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:44.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.705+0000 7f85faffd640 1 -- 172.21.15.47:0/780303896 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da5e43d800 msgr2=0x55da5e4b2fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:00:44.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.705+0000 7f85faffd640 1 --2- 172.21.15.47:0/780303896 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da5e43d800 0x55da5e4b2fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:44.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.705+0000 7f85faffd640 1 -- 172.21.15.47:0/780303896 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55da5e438690 msgr2=0x55da5e4b0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:00:44.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.705+0000 7f85faffd640 1 --2- 172.21.15.47:0/780303896 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55da5e438690 0x55da5e4b0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:44.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.705+0000 7f85faffd640 1 -- 172.21.15.47:0/780303896 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da5e4b34e0 con 0x55da5e437ca0 2024-04-17T06:00:44.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.705+0000 7f85fa7fc640 1 --2- 172.21.15.47:0/780303896 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55da5e438690 0x55da5e4b0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:00:44.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.705+0000 7f85faffd640 1 --2- 172.21.15.47:0/780303896 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da5e437ca0 0x55da5e447360 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f85ec02c7b0 tx=0x7f85ec02cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f7851bb4a9c4d49 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:00:44.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.705+0000 7f85deff5640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-17T06:00:44.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.705+0000 7f85deff5640 1 -- 172.21.15.47:0/780303896 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85ec0049b0 con 0x55da5e437ca0 2024-04-17T06:00:44.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.706+0000 7f85deff5640 1 -- 172.21.15.47:0/780303896 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f85ec004b50 con 0x55da5e437ca0 2024-04-17T06:00:44.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.706+0000 7f8604021740 1 -- 172.21.15.47:0/780303896 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85ec005000 con 0x55da5e437ca0 2024-04-17T06:00:44.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.706+0000 7f85deff5640 1 -- 172.21.15.47:0/780303896 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f85ec004d50 con 0x55da5e437ca0 2024-04-17T06:00:44.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.706+0000 7f8604021740 10 client.15132 fetch_fsmap learned FSMap version 21 2024-04-17T06:00:44.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.706+0000 7f8604021740 1 -- 172.21.15.47:0/780303896 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55da5e4b4630 con 0x55da5e437ca0 2024-04-17T06:00:44.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.706+0000 7f85deff5640 1 -- 172.21.15.47:0/780303896 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85ec005000 con 0x55da5e437ca0 2024-04-17T06:00:44.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.706+0000 7f85deff5640 1 -- 172.21.15.47:0/780303896 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 21) ==== 8339+0+0 (secure 0 0 0) 0x7f85ec003bb0 con 0x55da5e437ca0 2024-04-17T06:00:44.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.706+0000 7f8604021740 10 client.15132 fetch_fsmap finished waiting for FSMap version 21 2024-04-17T06:00:44.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.706+0000 7f8604021740 10 client.15132 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-17T06:00:44.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.706+0000 7f8604021740 20 client.15132 populate_metadata read hostname 'smithi047' 2024-04-17T06:00:44.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.706+0000 7f8604021740 1 --2- 172.21.15.47:0/780303896 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55da5e4b8ae0 0x55da5e4b55f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:44.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.706+0000 7f8604021740 4 client.15132 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-17T06:00:44.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.706+0000 7f8604021740 1 -- 172.21.15.47:0/780303896 --> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] -- command(tid 0: {"prefix": "status"}) -- 0x55da5e3ab4e0 con 0x55da5e4b8ae0 2024-04-17T06:00:44.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.706+0000 7f85fa7fc640 1 --2- 172.21.15.47:0/780303896 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55da5e4b8ae0 0x55da5e4b55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:44.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.707+0000 7f85deff5640 10 client.15132 ms_handle_connect on v2:172.21.15.98:6838/2841287917 2024-04-17T06:00:44.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.707+0000 7f85fa7fc640 1 --2- 172.21.15.47:0/780303896 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55da5e4b8ae0 0x55da5e4b55f0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a166a4f22e1d64e server_cookie=249d28958132e492 in_seq=0 out_seq=0 2024-04-17T06:00:44.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.707+0000 7f85deff5640 1 -- 172.21.15.47:0/780303896 <== mds.0 v2:172.21.15.98:6838/2841287917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55da5e3ab4e0 con 0x55da5e4b8ae0 2024-04-17T06:00:44.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.707+0000 7f85deff5640 10 client.15132 handle_command_reply: tid=0 2024-04-17T06:00:44.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.707+0000 7f8604021740 1 client.15132 shutdown 2024-04-17T06:00:44.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.707+0000 7f8604021740 1 -- 172.21.15.47:0/780303896 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da5e437ca0 msgr2=0x55da5e447360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:44.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.707+0000 7f8604021740 1 --2- 172.21.15.47:0/780303896 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da5e437ca0 0x55da5e447360 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f85ec02c7b0 tx=0x7f85ec02cc90 comp rx=0 tx=0).stop 2024-04-17T06:00:44.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.707+0000 7f8604021740 1 -- 172.21.15.47:0/780303896 shutdown_connections 2024-04-17T06:00:44.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.707+0000 7f8604021740 1 --2- 172.21.15.47:0/780303896 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55da5e4b8ae0 0x55da5e4b55f0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:44.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.707+0000 7f8604021740 1 --2- 172.21.15.47:0/780303896 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da5e43d800 0x55da5e4b2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:44.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.707+0000 7f8604021740 1 --2- 172.21.15.47:0/780303896 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55da5e438690 0x55da5e4b0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:44.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.707+0000 7f85deff5640 0 client.15132 ms_handle_reset on v2:172.21.15.98:6838/2841287917 2024-04-17T06:00:44.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.707+0000 7f8604021740 1 --2- 172.21.15.47:0/780303896 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da5e437ca0 0x55da5e447360 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:44.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.707+0000 7f8604021740 1 -- 172.21.15.47:0/780303896 >> 172.21.15.47:0/780303896 conn(0x55da5e433210 msgr2=0x55da5e4335f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:00:44.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.707+0000 7f8604021740 1 -- 172.21.15.47:0/780303896 shutdown_connections 2024-04-17T06:00:44.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.707+0000 7f8604021740 1 -- 172.21.15.47:0/780303896 wait complete. 2024-04-17T06:00:44.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.707+0000 7f8604021740 20 client.15132 trim_cache size 0 max 16384 2024-04-17T06:00:44.714 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-17T06:00:44.714 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_fsid": "565db2d8-fc7e-11ee-bc91-c7b262605968", 2024-04-17T06:00:44.714 INFO:teuthology.orchestra.run.smithi047.stdout: "whoami": 0, 2024-04-17T06:00:44.714 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 24518, 2024-04-17T06:00:44.714 INFO:teuthology.orchestra.run.smithi047.stdout: "want_state": "up:active", 2024-04-17T06:00:44.714 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "up:active", 2024-04-17T06:00:44.714 INFO:teuthology.orchestra.run.smithi047.stdout: "fs_name": "cephfs", 2024-04-17T06:00:44.714 INFO:teuthology.orchestra.run.smithi047.stdout: "rank_uptime": 90.143725238000002, 2024-04-17T06:00:44.714 INFO:teuthology.orchestra.run.smithi047.stdout: "mdsmap_epoch": 21, 2024-04-17T06:00:44.715 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch": 88, 2024-04-17T06:00:44.715 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch_barrier": 86, 2024-04-17T06:00:44.715 INFO:teuthology.orchestra.run.smithi047.stdout: "uptime": 96.191381402999994 2024-04-17T06:00:44.715 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-17T06:00:44.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.708+0000 7f8601359640 1 -- 172.21.15.47:0/4238570587 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f85e40a3bb0 msgr2=0x7f85e40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:44.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.708+0000 7f8601359640 1 --2- 172.21.15.47:0/4238570587 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f85e40a3bb0 0x7f85e40a6070 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f85f0005fd0 tx=0x7f85f0014840 comp rx=0 tx=0).stop 2024-04-17T06:00:44.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.708+0000 7f8601359640 1 -- 172.21.15.47:0/4238570587 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85fc107ce0 msgr2=0x7f85fc19f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:44.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.708+0000 7f8601359640 1 --2- 172.21.15.47:0/4238570587 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85fc107ce0 0x7f85fc19f960 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f85f400b970 tx=0x7f85f400be30 comp rx=0 tx=0).stop 2024-04-17T06:00:44.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.708+0000 7f8601359640 1 -- 172.21.15.47:0/4238570587 shutdown_connections 2024-04-17T06:00:44.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.708+0000 7f8601359640 1 --2- 172.21.15.47:0/4238570587 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f85e40a3bb0 0x7f85e40a6070 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:44.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.708+0000 7f8601359640 1 --2- 172.21.15.47:0/4238570587 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85fc107ce0 0x7f85fc19f960 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:44.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.708+0000 7f8601359640 1 --2- 172.21.15.47:0/4238570587 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f85fc1053b0 0x7f85fc1988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:44.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.708+0000 7f8601359640 1 --2- 172.21.15.47:0/4238570587 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f85fc102a80 0x7f85fc1983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:44.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.708+0000 7f8601359640 1 -- 172.21.15.47:0/4238570587 >> 172.21.15.47:0/4238570587 conn(0x7f85fc0fc450 msgr2=0x7f85fc1033c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:00:44.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.708+0000 7f8601359640 1 -- 172.21.15.47:0/4238570587 shutdown_connections 2024-04-17T06:00:44.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:44.708+0000 7f8601359640 1 -- 172.21.15.47:0/4238570587 wait complete. 2024-04-17T06:00:44.718 INFO:tasks.cephadm:Setup complete, yielding 2024-04-17T06:00:44.718 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-17T06:00:44.728 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi047.front.sepia.ceph.com 2024-04-17T06:00:44.729 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 565db2d8-fc7e-11ee-bc91-c7b262605968 -- bash -c 'ceph orch status' 2024-04-17T06:00:44.977 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T06:00:45.118 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:00:44 smithi047 ceph-mon[32223]: pgmap v333: 97 pgs: 97 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-17T06:00:45.118 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:00:44 smithi047 ceph-mon[32223]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-04-17T06:00:45.119 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:00:44 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/3460394627' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T06:00:45.119 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:00:44 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/1594618039' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T06:00:45.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:00:44 smithi098 ceph-mon[36516]: pgmap v333: 97 pgs: 97 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-17T06:00:45.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:00:44 smithi098 ceph-mon[36516]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-04-17T06:00:45.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:00:44 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/3460394627' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T06:00:45.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:00:44 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/1594618039' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T06:00:45.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:00:44 smithi103 ceph-mon[36243]: pgmap v333: 97 pgs: 97 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-17T06:00:45.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:00:44 smithi103 ceph-mon[36243]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-04-17T06:00:45.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:00:44 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/3460394627' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T06:00:45.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:00:44 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/1594618039' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T06:00:46.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:46.696+0000 7f8911045700 1 -- 172.21.15.47:0/3841691282 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f890c103540 msgr2=0x7f890c105930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:46.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:46.696+0000 7f8911045700 1 --2- 172.21.15.47:0/3841691282 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f890c103540 0x7f890c105930 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f88fc009990 tx=0x7f88fc009ca0 comp rx=0 tx=0).stop 2024-04-17T06:00:46.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:46.697+0000 7f8911045700 1 -- 172.21.15.47:0/3841691282 shutdown_connections 2024-04-17T06:00:46.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:46.697+0000 7f8911045700 1 --2- 172.21.15.47:0/3841691282 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f890c1087a0 0x7f890c10ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:46.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:46.697+0000 7f8911045700 1 --2- 172.21.15.47:0/3841691282 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f890c103540 0x7f890c105930 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:46.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:46.697+0000 7f8911045700 1 --2- 172.21.15.47:0/3841691282 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f890c105e70 0x7f890c108260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:46.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:46.697+0000 7f8911045700 1 -- 172.21.15.47:0/3841691282 >> 172.21.15.47:0/3841691282 conn(0x7f890c0fcf50 msgr2=0x7f890c0ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:00:46.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:46.697+0000 7f8911045700 1 -- 172.21.15.47:0/3841691282 shutdown_connections 2024-04-17T06:00:46.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:46.697+0000 7f8911045700 1 -- 172.21.15.47:0/3841691282 wait complete. 2024-04-17T06:00:46.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:46.697+0000 7f8911045700 1 Processor -- start 2024-04-17T06:00:46.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:46.698+0000 7f8911045700 1 -- start start 2024-04-17T06:00:46.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:46.698+0000 7f8911045700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f890c103540 0x7f890c102ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:46.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:46.698+0000 7f8911045700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f890c105e70 0x7f890c1014f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:46.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:46.698+0000 7f8911045700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f890c1087a0 0x7f890c101a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:46.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:46.698+0000 7f8911045700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f890c10e6b0 con 0x7f890c1087a0 2024-04-17T06:00:46.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:46.698+0000 7f8911045700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f890c10e530 con 0x7f890c103540 2024-04-17T06:00:46.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:46.698+0000 7f8911045700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f890c10e830 con 0x7f890c105e70 2024-04-17T06:00:46.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:46.698+0000 7f890b59e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f890c1087a0 0x7f890c101a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:46.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:46.698+0000 7f890b59e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f890c1087a0 0x7f890c101a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53374/0 (socket says 172.21.15.47:53374) 2024-04-17T06:00:46.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:46.698+0000 7f890b59e700 1 -- 172.21.15.47:0/1760453618 learned_addr learned my addr 172.21.15.47:0/1760453618 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:00:46.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:46.698+0000 7f890ad9d700 1 --2- 172.21.15.47:0/1760453618 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f890c103540 0x7f890c102ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:46.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:46.698+0000 7f890a59c700 1 --2- 172.21.15.47:0/1760453618 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f890c105e70 0x7f890c1014f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:46.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:46.698+0000 7f890b59e700 1 -- 172.21.15.47:0/1760453618 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f890c105e70 msgr2=0x7f890c1014f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:46.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:46.698+0000 7f890b59e700 1 --2- 172.21.15.47:0/1760453618 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f890c105e70 0x7f890c1014f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:46.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:46.698+0000 7f890b59e700 1 -- 172.21.15.47:0/1760453618 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f890c103540 msgr2=0x7f890c102ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:46.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:46.698+0000 7f890b59e700 1 --2- 172.21.15.47:0/1760453618 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f890c103540 0x7f890c102ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:46.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:46.698+0000 7f890b59e700 1 -- 172.21.15.47:0/1760453618 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f890c1022e0 con 0x7f890c1087a0 2024-04-17T06:00:46.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:46.698+0000 7f890a59c700 1 --2- 172.21.15.47:0/1760453618 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f890c105e70 0x7f890c1014f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:00:46.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:46.698+0000 7f890ad9d700 1 --2- 172.21.15.47:0/1760453618 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f890c103540 0x7f890c102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:00:46.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:46.699+0000 7f890b59e700 1 --2- 172.21.15.47:0/1760453618 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f890c1087a0 0x7f890c101a30 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f890400ada0 tx=0x7f890400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:00:46.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:46.699+0000 7f88f3fff700 1 -- 172.21.15.47:0/1760453618 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8904017020 con 0x7f890c1087a0 2024-04-17T06:00:46.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:46.699+0000 7f88f3fff700 1 -- 172.21.15.47:0/1760453618 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8904007e30 con 0x7f890c1087a0 2024-04-17T06:00:46.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:46.699+0000 7f8911045700 1 -- 172.21.15.47:0/1760453618 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f890c1a2ec0 con 0x7f890c1087a0 2024-04-17T06:00:46.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:46.699+0000 7f88f3fff700 1 -- 172.21.15.47:0/1760453618 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8904005500 con 0x7f890c1087a0 2024-04-17T06:00:46.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:46.699+0000 7f8911045700 1 -- 172.21.15.47:0/1760453618 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f890c1a3320 con 0x7f890c1087a0 2024-04-17T06:00:46.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:46.700+0000 7f8911045700 1 -- 172.21.15.47:0/1760453618 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f890c069740 con 0x7f890c1087a0 2024-04-17T06:00:46.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:46.701+0000 7f88f3fff700 1 -- 172.21.15.47:0/1760453618 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f8904004ad0 con 0x7f890c1087a0 2024-04-17T06:00:46.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:46.701+0000 7f88f3fff700 1 --2- 172.21.15.47:0/1760453618 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f88f40a3c40 0x7f88f40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:46.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:46.702+0000 7f890ad9d700 1 --2- 172.21.15.47:0/1760453618 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f88f40a3c40 0x7f88f40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:46.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:46.702+0000 7f88f3fff700 1 -- 172.21.15.47:0/1760453618 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f89040cfbd0 con 0x7f890c1087a0 2024-04-17T06:00:46.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:46.702+0000 7f890ad9d700 1 --2- 172.21.15.47:0/1760453618 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f88f40a3c40 0x7f88f40a6100 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f88fc00b5c0 tx=0x7f88fc005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:00:46.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:46.706+0000 7f88f3fff700 1 -- 172.21.15.47:0/1760453618 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f8904021020 con 0x7f890c1087a0 2024-04-17T06:00:46.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:46.932+0000 7f8911045700 1 -- 172.21.15.47:0/1760453618 --> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f890c062e90 con 0x7f88f40a3c40 2024-04-17T06:00:46.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:46.933+0000 7f88f3fff700 1 -- 172.21.15.47:0/1760453618 <== mgr.14152 v2:172.21.15.47:6800/3157902806 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f890c062e90 con 0x7f88f40a3c40 2024-04-17T06:00:46.932 INFO:teuthology.orchestra.run.smithi047.stdout:Backend: cephadm 2024-04-17T06:00:46.932 INFO:teuthology.orchestra.run.smithi047.stdout:Available: Yes 2024-04-17T06:00:46.932 INFO:teuthology.orchestra.run.smithi047.stdout:Paused: No 2024-04-17T06:00:46.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:46.935+0000 7f8911045700 1 -- 172.21.15.47:0/1760453618 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f88f40a3c40 msgr2=0x7f88f40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:46.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:46.935+0000 7f8911045700 1 --2- 172.21.15.47:0/1760453618 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f88f40a3c40 0x7f88f40a6100 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f88fc00b5c0 tx=0x7f88fc005fb0 comp rx=0 tx=0).stop 2024-04-17T06:00:46.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:46.935+0000 7f8911045700 1 -- 172.21.15.47:0/1760453618 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f890c1087a0 msgr2=0x7f890c101a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:46.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:46.935+0000 7f8911045700 1 --2- 172.21.15.47:0/1760453618 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f890c1087a0 0x7f890c101a30 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f890400ada0 tx=0x7f890400c330 comp rx=0 tx=0).stop 2024-04-17T06:00:46.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:46.935+0000 7f8911045700 1 -- 172.21.15.47:0/1760453618 shutdown_connections 2024-04-17T06:00:46.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:46.935+0000 7f8911045700 1 --2- 172.21.15.47:0/1760453618 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f88f40a3c40 0x7f88f40a6100 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:46.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:46.935+0000 7f8911045700 1 --2- 172.21.15.47:0/1760453618 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f890c1087a0 0x7f890c101a30 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:46.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:46.935+0000 7f8911045700 1 --2- 172.21.15.47:0/1760453618 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f890c103540 0x7f890c102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:46.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:46.935+0000 7f8911045700 1 --2- 172.21.15.47:0/1760453618 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f890c105e70 0x7f890c1014f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:46.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:46.935+0000 7f8911045700 1 -- 172.21.15.47:0/1760453618 >> 172.21.15.47:0/1760453618 conn(0x7f890c0fcf50 msgr2=0x7f890c103f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:00:46.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:46.935+0000 7f8911045700 1 -- 172.21.15.47:0/1760453618 shutdown_connections 2024-04-17T06:00:46.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:46.935+0000 7f8911045700 1 -- 172.21.15.47:0/1760453618 wait complete. 2024-04-17T06:00:47.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:00:46 smithi098 ceph-mon[36516]: pgmap v334: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-04-17T06:00:47.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:00:46 smithi047 ceph-mon[32223]: pgmap v334: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-04-17T06:00:47.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:00:46 smithi103 ceph-mon[36243]: pgmap v334: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-04-17T06:00:47.648 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 565db2d8-fc7e-11ee-bc91-c7b262605968 -- bash -c 'ceph orch ps' 2024-04-17T06:00:47.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:00:47 smithi047 ceph-mon[32223]: from='client.15138 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T06:00:47.899 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T06:00:48.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:00:47 smithi098 ceph-mon[36516]: from='client.15138 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T06:00:48.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:00:47 smithi103 ceph-mon[36243]: from='client.15138 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T06:00:49.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:00:48 smithi098 ceph-mon[36516]: pgmap v335: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 6 op/s 2024-04-17T06:00:49.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:00:48 smithi047 ceph-mon[32223]: pgmap v335: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 6 op/s 2024-04-17T06:00:49.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:00:48 smithi103 ceph-mon[36243]: pgmap v335: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 6 op/s 2024-04-17T06:00:49.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:49.678+0000 7f9aeec7f700 1 -- 172.21.15.47:0/1113331503 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ae8102ba0 msgr2=0x7f9ae8103000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:49.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:49.678+0000 7f9aeec7f700 1 --2- 172.21.15.47:0/1113331503 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ae8102ba0 0x7f9ae8103000 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f9ad40099c0 tx=0x7f9ad4009cd0 comp rx=0 tx=0).stop 2024-04-17T06:00:49.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:49.678+0000 7f9aeec7f700 1 -- 172.21.15.47:0/1113331503 shutdown_connections 2024-04-17T06:00:49.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:49.678+0000 7f9aeec7f700 1 --2- 172.21.15.47:0/1113331503 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9ae8103540 0x7f9ae8109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:49.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:49.678+0000 7f9aeec7f700 1 --2- 172.21.15.47:0/1113331503 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ae81019a0 0x7f9ae8101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:49.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:49.678+0000 7f9aeec7f700 1 --2- 172.21.15.47:0/1113331503 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ae8102ba0 0x7f9ae8103000 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:49.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:49.678+0000 7f9aeec7f700 1 -- 172.21.15.47:0/1113331503 >> 172.21.15.47:0/1113331503 conn(0x7f9ae80fcf40 msgr2=0x7f9ae80ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:00:49.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:49.679+0000 7f9aeec7f700 1 -- 172.21.15.47:0/1113331503 shutdown_connections 2024-04-17T06:00:49.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:49.679+0000 7f9aeec7f700 1 -- 172.21.15.47:0/1113331503 wait complete. 2024-04-17T06:00:49.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:49.679+0000 7f9aeec7f700 1 Processor -- start 2024-04-17T06:00:49.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:49.679+0000 7f9aeec7f700 1 -- start start 2024-04-17T06:00:49.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:49.679+0000 7f9aeec7f700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9ae81019a0 0x7f9ae819b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:49.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:49.679+0000 7f9aeec7f700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ae8102ba0 0x7f9ae819b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:49.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:49.679+0000 7f9aeec7f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ae8103540 0x7f9ae81a2660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:49.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:49.679+0000 7f9aeec7f700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f9ae810d680 con 0x7f9ae8103540 2024-04-17T06:00:49.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:49.679+0000 7f9aeec7f700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9ae810d500 con 0x7f9ae81019a0 2024-04-17T06:00:49.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:49.679+0000 7f9aeec7f700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9ae810d800 con 0x7f9ae8102ba0 2024-04-17T06:00:49.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:49.679+0000 7f9aed21c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ae8103540 0x7f9ae81a2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:49.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:49.679+0000 7f9aed21c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ae8103540 0x7f9ae81a2660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53400/0 (socket says 172.21.15.47:53400) 2024-04-17T06:00:49.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:49.679+0000 7f9aed21c700 1 -- 172.21.15.47:0/3222435956 learned_addr learned my addr 172.21.15.47:0/3222435956 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:00:49.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:49.680+0000 7f9aeca1b700 1 --2- 172.21.15.47:0/3222435956 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9ae81019a0 0x7f9ae819b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:49.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:49.680+0000 7f9ae7fff700 1 --2- 172.21.15.47:0/3222435956 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ae8102ba0 0x7f9ae819b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:49.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:49.680+0000 7f9aed21c700 1 -- 172.21.15.47:0/3222435956 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ae8102ba0 msgr2=0x7f9ae819b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:49.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:49.680+0000 7f9aed21c700 1 --2- 172.21.15.47:0/3222435956 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ae8102ba0 0x7f9ae819b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:49.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:49.680+0000 7f9aed21c700 1 -- 172.21.15.47:0/3222435956 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9ae81019a0 msgr2=0x7f9ae819b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:49.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:49.680+0000 7f9aed21c700 1 --2- 172.21.15.47:0/3222435956 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9ae81019a0 0x7f9ae819b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:49.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:49.680+0000 7f9aed21c700 1 -- 172.21.15.47:0/3222435956 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ae81a2d00 con 0x7f9ae8103540 2024-04-17T06:00:49.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:49.680+0000 7f9ae7fff700 1 --2- 172.21.15.47:0/3222435956 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ae8102ba0 0x7f9ae819b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:00:49.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:49.680+0000 7f9aed21c700 1 --2- 172.21.15.47:0/3222435956 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ae8103540 0x7f9ae81a2660 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f9ae000b9a0 tx=0x7f9ae000bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:00:49.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:49.680+0000 7f9ae5ffb700 1 -- 172.21.15.47:0/3222435956 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae0019070 con 0x7f9ae8103540 2024-04-17T06:00:49.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:49.680+0000 7f9ae5ffb700 1 -- 172.21.15.47:0/3222435956 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ae0014440 con 0x7f9ae8103540 2024-04-17T06:00:49.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:49.680+0000 7f9ae5ffb700 1 -- 172.21.15.47:0/3222435956 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae00075d0 con 0x7f9ae8103540 2024-04-17T06:00:49.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:49.680+0000 7f9aeec7f700 1 -- 172.21.15.47:0/3222435956 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ae81a2ff0 con 0x7f9ae8103540 2024-04-17T06:00:49.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:49.680+0000 7f9aeec7f700 1 -- 172.21.15.47:0/3222435956 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ae81a3570 con 0x7f9ae8103540 2024-04-17T06:00:49.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:49.681+0000 7f9aeec7f700 1 -- 172.21.15.47:0/3222435956 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ae8069740 con 0x7f9ae8103540 2024-04-17T06:00:49.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:49.682+0000 7f9ae5ffb700 1 -- 172.21.15.47:0/3222435956 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f9ae001c070 con 0x7f9ae8103540 2024-04-17T06:00:49.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:49.683+0000 7f9ae5ffb700 1 --2- 172.21.15.47:0/3222435956 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f9ad80a3cc0 0x7f9ad80a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:49.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:49.683+0000 7f9aeca1b700 1 --2- 172.21.15.47:0/3222435956 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f9ad80a3cc0 0x7f9ad80a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:49.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:49.683+0000 7f9ae5ffb700 1 -- 172.21.15.47:0/3222435956 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f9ae00cfc60 con 0x7f9ae8103540 2024-04-17T06:00:49.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:49.683+0000 7f9aeca1b700 1 --2- 172.21.15.47:0/3222435956 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f9ad80a3cc0 0x7f9ad80a6180 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f9adc006fd0 tx=0x7f9adc008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:00:49.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:49.687+0000 7f9ae5ffb700 1 -- 172.21.15.47:0/3222435956 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f9ae009c680 con 0x7f9ae8103540 2024-04-17T06:00:49.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:49.911+0000 7f9aeec7f700 1 -- 172.21.15.47:0/3222435956 --> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f9ae80ff140 con 0x7f9ad80a3cc0 2024-04-17T06:00:49.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:49.925+0000 7f9ae5ffb700 1 -- 172.21.15.47:0/3222435956 <== mgr.14152 v2:172.21.15.47:6800/3157902806 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4185 (secure 0 0 0) 0x7f9ae80ff140 con 0x7f9ad80a3cc0 2024-04-17T06:00:49.924 INFO:teuthology.orchestra.run.smithi047.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-17T06:00:49.924 INFO:teuthology.orchestra.run.smithi047.stdout:mds.a smithi047 running (104s) 18s ago 104s 12.9M - 19.0.0-3133-ga8c84f59 76bc6b91b726 ba359aa70ff5 2024-04-17T06:00:49.924 INFO:teuthology.orchestra.run.smithi047.stdout:mds.b smithi098 running (101s) 34s ago 101s 25.7M - 19.0.0-3133-ga8c84f59 76bc6b91b726 b45b802cb363 2024-04-17T06:00:49.924 INFO:teuthology.orchestra.run.smithi047.stdout:mds.c smithi103 running (116s) 34s ago 115s 18.3M - 19.0.0-3133-ga8c84f59 76bc6b91b726 957484b8ec18 2024-04-17T06:00:49.924 INFO:teuthology.orchestra.run.smithi047.stdout:mds.d smithi047 running (2m) 18s ago 2m 13.7M - 19.0.0-3133-ga8c84f59 76bc6b91b726 5dad8c0cc874 2024-04-17T06:00:49.924 INFO:teuthology.orchestra.run.smithi047.stdout:mds.e smithi098 running (2m) 34s ago 2m 17.1M - 19.0.0-3133-ga8c84f59 76bc6b91b726 aac8438892f4 2024-04-17T06:00:49.924 INFO:teuthology.orchestra.run.smithi047.stdout:mds.f smithi103 running (118s) 34s ago 118s 15.7M - 19.0.0-3133-ga8c84f59 76bc6b91b726 fa8c8ed24336 2024-04-17T06:00:49.924 INFO:teuthology.orchestra.run.smithi047.stdout:mds.g smithi047 running (107s) 18s ago 107s 15.2M - 19.0.0-3133-ga8c84f59 76bc6b91b726 9b688587ce62 2024-04-17T06:00:49.924 INFO:teuthology.orchestra.run.smithi047.stdout:mds.h smithi098 running (113s) 34s ago 112s 19.9M - 19.0.0-3133-ga8c84f59 76bc6b91b726 94c7c390af79 2024-04-17T06:00:49.924 INFO:teuthology.orchestra.run.smithi047.stdout:mds.i smithi103 running (110s) 34s ago 109s 16.5M - 19.0.0-3133-ga8c84f59 76bc6b91b726 e6268839d96a 2024-04-17T06:00:49.924 INFO:teuthology.orchestra.run.smithi047.stdout:mds.j smithi047 running (2m) 18s ago 2m 13.3M - 19.0.0-3133-ga8c84f59 76bc6b91b726 90546cf558c8 2024-04-17T06:00:49.925 INFO:teuthology.orchestra.run.smithi047.stdout:mds.k smithi098 running (2m) 34s ago 2m 13.9M - 19.0.0-3133-ga8c84f59 76bc6b91b726 3f6debd36fb7 2024-04-17T06:00:49.925 INFO:teuthology.orchestra.run.smithi047.stdout:mds.l smithi103 running (2m) 34s ago 2m 13.0M - 19.0.0-3133-ga8c84f59 76bc6b91b726 903cc67b7c0d 2024-04-17T06:00:49.925 INFO:teuthology.orchestra.run.smithi047.stdout:mgr.x smithi047 *:9283,8765 running (9m) 18s ago 9m 515M - 19.0.0-3133-ga8c84f59 76bc6b91b726 b8f280f6ff25 2024-04-17T06:00:49.925 INFO:teuthology.orchestra.run.smithi047.stdout:mgr.y smithi098 *:8443,8765 running (7m) 34s ago 7m 453M - 19.0.0-3133-ga8c84f59 76bc6b91b726 a400e56efbbf 2024-04-17T06:00:49.925 INFO:teuthology.orchestra.run.smithi047.stdout:mgr.z smithi103 *:8443,8765 running (7m) 34s ago 7m 452M - 19.0.0-3133-ga8c84f59 76bc6b91b726 6af7082b7f51 2024-04-17T06:00:49.925 INFO:teuthology.orchestra.run.smithi047.stdout:mon.a smithi047 running (9m) 18s ago 9m 51.5M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 d01e5baa55cc 2024-04-17T06:00:49.925 INFO:teuthology.orchestra.run.smithi047.stdout:mon.b smithi098 running (7m) 34s ago 7m 39.2M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 f85004f63c84 2024-04-17T06:00:49.925 INFO:teuthology.orchestra.run.smithi047.stdout:mon.c smithi103 running (7m) 34s ago 7m 37.9M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 382e56093343 2024-04-17T06:00:49.925 INFO:teuthology.orchestra.run.smithi047.stdout:osd.0 smithi047 running (7m) 18s ago 7m 56.5M 4273M 19.0.0-3133-ga8c84f59 76bc6b91b726 57ab3478e010 2024-04-17T06:00:49.925 INFO:teuthology.orchestra.run.smithi047.stdout:osd.1 smithi098 running (6m) 34s ago 6m 48.4M 4270M 19.0.0-3133-ga8c84f59 76bc6b91b726 fb341992a6ef 2024-04-17T06:00:49.925 INFO:teuthology.orchestra.run.smithi047.stdout:osd.2 smithi103 running (6m) 34s ago 6m 49.0M 4270M 19.0.0-3133-ga8c84f59 76bc6b91b726 02645ff29b18 2024-04-17T06:00:49.925 INFO:teuthology.orchestra.run.smithi047.stdout:osd.3 smithi047 running (5m) 18s ago 5m 43.1M 4273M 19.0.0-3133-ga8c84f59 76bc6b91b726 522e29f01f59 2024-04-17T06:00:49.925 INFO:teuthology.orchestra.run.smithi047.stdout:osd.4 smithi098 running (5m) 34s ago 5m 52.9M 4270M 19.0.0-3133-ga8c84f59 76bc6b91b726 523761e89cab 2024-04-17T06:00:49.925 INFO:teuthology.orchestra.run.smithi047.stdout:osd.5 smithi103 running (5m) 34s ago 5m 44.7M 4270M 19.0.0-3133-ga8c84f59 76bc6b91b726 8071cfefa25b 2024-04-17T06:00:49.926 INFO:teuthology.orchestra.run.smithi047.stdout:osd.6 smithi047 running (4m) 18s ago 4m 41.1M 4273M 19.0.0-3133-ga8c84f59 76bc6b91b726 b477be316c81 2024-04-17T06:00:49.926 INFO:teuthology.orchestra.run.smithi047.stdout:osd.7 smithi098 running (4m) 34s ago 4m 46.0M 4270M 19.0.0-3133-ga8c84f59 76bc6b91b726 2dbd1aae384d 2024-04-17T06:00:49.926 INFO:teuthology.orchestra.run.smithi047.stdout:osd.8 smithi103 running (3m) 34s ago 3m 46.0M 4270M 19.0.0-3133-ga8c84f59 76bc6b91b726 9186747c0e9e 2024-04-17T06:00:49.926 INFO:teuthology.orchestra.run.smithi047.stdout:osd.9 smithi047 running (3m) 18s ago 3m 52.0M 4273M 19.0.0-3133-ga8c84f59 76bc6b91b726 f37324656282 2024-04-17T06:00:49.926 INFO:teuthology.orchestra.run.smithi047.stdout:osd.10 smithi098 running (2m) 34s ago 2m 53.6M 4270M 19.0.0-3133-ga8c84f59 76bc6b91b726 904727f23cbf 2024-04-17T06:00:49.926 INFO:teuthology.orchestra.run.smithi047.stdout:osd.11 smithi103 running (2m) 34s ago 2m 51.7M 4270M 19.0.0-3133-ga8c84f59 76bc6b91b726 2dac99cfead4 2024-04-17T06:00:49.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:49.927+0000 7f9aeec7f700 1 -- 172.21.15.47:0/3222435956 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f9ad80a3cc0 msgr2=0x7f9ad80a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:49.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:49.927+0000 7f9aeec7f700 1 --2- 172.21.15.47:0/3222435956 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f9ad80a3cc0 0x7f9ad80a6180 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f9adc006fd0 tx=0x7f9adc008040 comp rx=0 tx=0).stop 2024-04-17T06:00:49.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:49.927+0000 7f9aeec7f700 1 -- 172.21.15.47:0/3222435956 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ae8103540 msgr2=0x7f9ae81a2660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:49.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:49.927+0000 7f9aeec7f700 1 --2- 172.21.15.47:0/3222435956 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ae8103540 0x7f9ae81a2660 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f9ae000b9a0 tx=0x7f9ae000bd60 comp rx=0 tx=0).stop 2024-04-17T06:00:49.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:49.927+0000 7f9aeec7f700 1 -- 172.21.15.47:0/3222435956 shutdown_connections 2024-04-17T06:00:49.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:49.927+0000 7f9aeec7f700 1 --2- 172.21.15.47:0/3222435956 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f9ad80a3cc0 0x7f9ad80a6180 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:49.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:49.927+0000 7f9aeec7f700 1 --2- 172.21.15.47:0/3222435956 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ae8103540 0x7f9ae81a2660 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:49.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:49.927+0000 7f9aeec7f700 1 --2- 172.21.15.47:0/3222435956 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9ae81019a0 0x7f9ae819b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:49.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:49.927+0000 7f9aeec7f700 1 --2- 172.21.15.47:0/3222435956 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ae8102ba0 0x7f9ae819b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:49.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:49.927+0000 7f9aeec7f700 1 -- 172.21.15.47:0/3222435956 >> 172.21.15.47:0/3222435956 conn(0x7f9ae80fcf40 msgr2=0x7f9ae80fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:00:49.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:49.927+0000 7f9aeec7f700 1 -- 172.21.15.47:0/3222435956 shutdown_connections 2024-04-17T06:00:49.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:49.927+0000 7f9aeec7f700 1 -- 172.21.15.47:0/3222435956 wait complete. 2024-04-17T06:00:50.749 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 565db2d8-fc7e-11ee-bc91-c7b262605968 -- bash -c 'ceph orch ls' 2024-04-17T06:00:50.999 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T06:00:51.121 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:00:50 smithi047 ceph-mon[32223]: pgmap v336: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 7 op/s 2024-04-17T06:00:51.121 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:00:50 smithi047 ceph-mon[32223]: from='client.15144 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T06:00:51.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:00:50 smithi098 ceph-mon[36516]: pgmap v336: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 7 op/s 2024-04-17T06:00:51.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:00:50 smithi098 ceph-mon[36516]: from='client.15144 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T06:00:51.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:00:50 smithi103 ceph-mon[36243]: pgmap v336: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 7 op/s 2024-04-17T06:00:51.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:00:50 smithi103 ceph-mon[36243]: from='client.15144 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T06:00:52.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:52.700+0000 7f3fa7147700 1 -- 172.21.15.47:0/4114228889 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3fa01019b0 msgr2=0x7f3fa0101dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:52.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:52.700+0000 7f3fa7147700 1 --2- 172.21.15.47:0/4114228889 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3fa01019b0 0x7f3fa0101dd0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f3f940099c0 tx=0x7f3f94009cd0 comp rx=0 tx=0).stop 2024-04-17T06:00:52.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:52.700+0000 7f3fa7147700 1 -- 172.21.15.47:0/4114228889 shutdown_connections 2024-04-17T06:00:52.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:52.700+0000 7f3fa7147700 1 --2- 172.21.15.47:0/4114228889 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3fa0103550 0x7f3fa0109e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:52.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:52.700+0000 7f3fa7147700 1 --2- 172.21.15.47:0/4114228889 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3fa01019b0 0x7f3fa0101dd0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:52.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:52.700+0000 7f3fa7147700 1 --2- 172.21.15.47:0/4114228889 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3fa0102bb0 0x7f3fa0103010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:52.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:52.700+0000 7f3fa7147700 1 -- 172.21.15.47:0/4114228889 >> 172.21.15.47:0/4114228889 conn(0x7f3fa00fcf50 msgr2=0x7f3fa00ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:00:52.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:52.700+0000 7f3fa7147700 1 -- 172.21.15.47:0/4114228889 shutdown_connections 2024-04-17T06:00:52.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:52.700+0000 7f3fa7147700 1 -- 172.21.15.47:0/4114228889 wait complete. 2024-04-17T06:00:52.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:52.701+0000 7f3fa7147700 1 Processor -- start 2024-04-17T06:00:52.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:52.701+0000 7f3fa7147700 1 -- start start 2024-04-17T06:00:52.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:52.701+0000 7f3fa7147700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3fa01019b0 0x7f3fa0198d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:52.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:52.701+0000 7f3fa7147700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3fa0102bb0 0x7f3fa0199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:52.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:52.701+0000 7f3fa7147700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3fa0103550 0x7f3fa01a02f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:52.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:52.701+0000 7f3fa7147700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f3fa010d5f0 con 0x7f3fa01019b0 2024-04-17T06:00:52.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:52.701+0000 7f3fa7147700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3fa010d470 con 0x7f3fa0102bb0 2024-04-17T06:00:52.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:52.701+0000 7f3fa7147700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3fa010d770 con 0x7f3fa0103550 2024-04-17T06:00:52.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:52.702+0000 7f3fa4ee3700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3fa01019b0 0x7f3fa0198d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:52.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:52.702+0000 7f3fa4ee3700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3fa01019b0 0x7f3fa0198d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53418/0 (socket says 172.21.15.47:53418) 2024-04-17T06:00:52.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:52.702+0000 7f3fa4ee3700 1 -- 172.21.15.47:0/359216992 learned_addr learned my addr 172.21.15.47:0/359216992 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:00:52.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:52.702+0000 7f3f9ffff700 1 --2- 172.21.15.47:0/359216992 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3fa0102bb0 0x7f3fa0199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:52.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:52.702+0000 7f3fa4ee3700 1 -- 172.21.15.47:0/359216992 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3fa0103550 msgr2=0x7f3fa01a02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:52.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:52.702+0000 7f3fa56e4700 1 --2- 172.21.15.47:0/359216992 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3fa0103550 0x7f3fa01a02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:52.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:52.702+0000 7f3fa4ee3700 1 --2- 172.21.15.47:0/359216992 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3fa0103550 0x7f3fa01a02f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:52.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:52.702+0000 7f3fa4ee3700 1 -- 172.21.15.47:0/359216992 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3fa0102bb0 msgr2=0x7f3fa0199270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:52.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:52.702+0000 7f3fa4ee3700 1 --2- 172.21.15.47:0/359216992 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3fa0102bb0 0x7f3fa0199270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:52.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:52.702+0000 7f3fa4ee3700 1 -- 172.21.15.47:0/359216992 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fa01a0990 con 0x7f3fa01019b0 2024-04-17T06:00:52.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:52.702+0000 7f3f9ffff700 1 --2- 172.21.15.47:0/359216992 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3fa0102bb0 0x7f3fa0199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:00:52.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:52.702+0000 7f3fa4ee3700 1 --2- 172.21.15.47:0/359216992 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3fa01019b0 0x7f3fa0198d30 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f3f94005f00 tx=0x7f3f94004630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:00:52.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:52.702+0000 7f3f9dffb700 1 -- 172.21.15.47:0/359216992 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f94027070 con 0x7f3fa01019b0 2024-04-17T06:00:52.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:52.702+0000 7f3fa7147700 1 -- 172.21.15.47:0/359216992 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fa01a0c20 con 0x7f3fa01019b0 2024-04-17T06:00:52.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:52.702+0000 7f3f9dffb700 1 -- 172.21.15.47:0/359216992 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f94022470 con 0x7f3fa01019b0 2024-04-17T06:00:52.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:52.702+0000 7f3f9dffb700 1 -- 172.21.15.47:0/359216992 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f9400f6e0 con 0x7f3fa01019b0 2024-04-17T06:00:52.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:52.703+0000 7f3fa7147700 1 -- 172.21.15.47:0/359216992 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fa01a1150 con 0x7f3fa01019b0 2024-04-17T06:00:52.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:52.704+0000 7f3fa7147700 1 -- 172.21.15.47:0/359216992 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fa0069740 con 0x7f3fa01019b0 2024-04-17T06:00:52.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:52.705+0000 7f3f9dffb700 1 -- 172.21.15.47:0/359216992 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f3f9402a070 con 0x7f3fa01019b0 2024-04-17T06:00:52.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:52.705+0000 7f3f9dffb700 1 --2- 172.21.15.47:0/359216992 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f3f900a3c40 0x7f3f900a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:52.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:52.706+0000 7f3f9ffff700 1 --2- 172.21.15.47:0/359216992 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f3f900a3c40 0x7f3f900a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:52.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:52.706+0000 7f3f9dffb700 1 -- 172.21.15.47:0/359216992 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f3f940d1430 con 0x7f3fa01019b0 2024-04-17T06:00:52.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:52.706+0000 7f3f9ffff700 1 --2- 172.21.15.47:0/359216992 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f3f900a3c40 0x7f3f900a6100 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c005fd0 tx=0x7f3f8c005e20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:00:52.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:52.709+0000 7f3f9dffb700 1 -- 172.21.15.47:0/359216992 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f3f94013070 con 0x7f3fa01019b0 2024-04-17T06:00:52.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:52.926+0000 7f3fa7147700 1 -- 172.21.15.47:0/359216992 --> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f3fa00ff110 con 0x7f3f900a3c40 2024-04-17T06:00:52.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:52.932+0000 7f3f9dffb700 1 -- 172.21.15.47:0/359216992 <== mgr.14152 v2:172.21.15.47:6800/3157902806 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f3fa00ff110 con 0x7f3f900a3c40 2024-04-17T06:00:52.931 INFO:teuthology.orchestra.run.smithi047.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-04-17T06:00:52.931 INFO:teuthology.orchestra.run.smithi047.stdout:mds.all 12/12 37s ago 2m smithi047=a;smithi047=d;smithi047=g;smithi047=j;smithi098=b;smithi098=e;smithi098=h;smithi098=k;smithi103=c;smithi103=f;smithi103=i;smithi103=l;count:12 2024-04-17T06:00:52.931 INFO:teuthology.orchestra.run.smithi047.stdout:mgr 3/3 37s ago 7m smithi047=x;smithi098=y;smithi103=z;count:3 2024-04-17T06:00:52.931 INFO:teuthology.orchestra.run.smithi047.stdout:mon 3/3 37s ago 7m smithi047:172.21.15.47=a;smithi098:172.21.15.98=b;smithi103:172.21.15.103=c;count:3 2024-04-17T06:00:52.931 INFO:teuthology.orchestra.run.smithi047.stdout:osd 12 37s ago - 2024-04-17T06:00:52.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:52.934+0000 7f3fa7147700 1 -- 172.21.15.47:0/359216992 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f3f900a3c40 msgr2=0x7f3f900a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:52.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:52.934+0000 7f3fa7147700 1 --2- 172.21.15.47:0/359216992 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f3f900a3c40 0x7f3f900a6100 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c005fd0 tx=0x7f3f8c005e20 comp rx=0 tx=0).stop 2024-04-17T06:00:52.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:52.934+0000 7f3fa7147700 1 -- 172.21.15.47:0/359216992 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3fa01019b0 msgr2=0x7f3fa0198d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:52.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:52.934+0000 7f3fa7147700 1 --2- 172.21.15.47:0/359216992 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3fa01019b0 0x7f3fa0198d30 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f3f94005f00 tx=0x7f3f94004630 comp rx=0 tx=0).stop 2024-04-17T06:00:52.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:52.934+0000 7f3fa7147700 1 -- 172.21.15.47:0/359216992 shutdown_connections 2024-04-17T06:00:52.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:52.934+0000 7f3fa7147700 1 --2- 172.21.15.47:0/359216992 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f3f900a3c40 0x7f3f900a6100 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:52.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:52.934+0000 7f3fa7147700 1 --2- 172.21.15.47:0/359216992 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3fa0103550 0x7f3fa01a02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:52.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:52.934+0000 7f3fa7147700 1 --2- 172.21.15.47:0/359216992 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3fa01019b0 0x7f3fa0198d30 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:52.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:52.934+0000 7f3fa7147700 1 --2- 172.21.15.47:0/359216992 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3fa0102bb0 0x7f3fa0199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:52.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:52.934+0000 7f3fa7147700 1 -- 172.21.15.47:0/359216992 >> 172.21.15.47:0/359216992 conn(0x7f3fa00fcf50 msgr2=0x7f3fa00fea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:00:52.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:52.934+0000 7f3fa7147700 1 -- 172.21.15.47:0/359216992 shutdown_connections 2024-04-17T06:00:52.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:52.934+0000 7f3fa7147700 1 -- 172.21.15.47:0/359216992 wait complete. 2024-04-17T06:00:53.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:00:52 smithi098 ceph-mon[36516]: pgmap v337: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-04-17T06:00:53.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:00:52 smithi047 ceph-mon[32223]: pgmap v337: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-04-17T06:00:53.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:00:52 smithi103 ceph-mon[36243]: pgmap v337: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-04-17T06:00:53.669 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 565db2d8-fc7e-11ee-bc91-c7b262605968 -- bash -c 'ceph orch host ls' 2024-04-17T06:00:53.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:00:53 smithi047 ceph-mon[32223]: from='client.15150 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T06:00:53.925 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T06:00:54.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:00:53 smithi098 ceph-mon[36516]: from='client.15150 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T06:00:54.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:00:53 smithi103 ceph-mon[36243]: from='client.15150 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T06:00:55.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:00:54 smithi098 ceph-mon[36516]: pgmap v338: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 6 op/s 2024-04-17T06:00:55.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:00:54 smithi047 ceph-mon[32223]: pgmap v338: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 6 op/s 2024-04-17T06:00:55.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:00:54 smithi103 ceph-mon[36243]: pgmap v338: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 6 op/s 2024-04-17T06:00:55.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:55.782+0000 7f3d977eb700 1 -- 172.21.15.47:0/1698629345 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3d90105e60 msgr2=0x7f3d90108250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:55.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:55.782+0000 7f3d977eb700 1 --2- 172.21.15.47:0/1698629345 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3d90105e60 0x7f3d90108250 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f3d84009990 tx=0x7f3d84009ca0 comp rx=0 tx=0).stop 2024-04-17T06:00:55.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:55.782+0000 7f3d977eb700 1 -- 172.21.15.47:0/1698629345 shutdown_connections 2024-04-17T06:00:55.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:55.782+0000 7f3d977eb700 1 --2- 172.21.15.47:0/1698629345 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3d90108790 0x7f3d9010ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:55.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:55.782+0000 7f3d977eb700 1 --2- 172.21.15.47:0/1698629345 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d90103530 0x7f3d90105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:55.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:55.782+0000 7f3d977eb700 1 --2- 172.21.15.47:0/1698629345 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3d90105e60 0x7f3d90108250 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:55.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:55.782+0000 7f3d977eb700 1 -- 172.21.15.47:0/1698629345 >> 172.21.15.47:0/1698629345 conn(0x7f3d900fcf40 msgr2=0x7f3d900ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:00:55.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:55.782+0000 7f3d977eb700 1 -- 172.21.15.47:0/1698629345 shutdown_connections 2024-04-17T06:00:55.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:55.782+0000 7f3d977eb700 1 -- 172.21.15.47:0/1698629345 wait complete. 2024-04-17T06:00:55.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:55.782+0000 7f3d977eb700 1 Processor -- start 2024-04-17T06:00:55.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:55.782+0000 7f3d977eb700 1 -- start start 2024-04-17T06:00:55.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:55.783+0000 7f3d977eb700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d90103530 0x7f3d90198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:55.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:55.783+0000 7f3d977eb700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3d90105e60 0x7f3d901991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:55.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:55.783+0000 7f3d977eb700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3d90108790 0x7f3d901a0230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:55.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:55.783+0000 7f3d977eb700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f3d9010e6a0 con 0x7f3d90105e60 2024-04-17T06:00:55.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:55.783+0000 7f3d977eb700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3d9010e520 con 0x7f3d90103530 2024-04-17T06:00:55.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:55.783+0000 7f3d977eb700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3d9010e820 con 0x7f3d90108790 2024-04-17T06:00:55.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:55.783+0000 7f3d94d86700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3d90105e60 0x7f3d901991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:55.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:55.783+0000 7f3d94d86700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3d90105e60 0x7f3d901991c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53804/0 (socket says 172.21.15.47:53804) 2024-04-17T06:00:55.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:55.783+0000 7f3d94d86700 1 -- 172.21.15.47:0/2115338665 learned_addr learned my addr 172.21.15.47:0/2115338665 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:00:55.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:55.783+0000 7f3d95587700 1 --2- 172.21.15.47:0/2115338665 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d90103530 0x7f3d90198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:55.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:55.783+0000 7f3d95d88700 1 --2- 172.21.15.47:0/2115338665 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3d90108790 0x7f3d901a0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:55.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:55.783+0000 7f3d94d86700 1 -- 172.21.15.47:0/2115338665 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3d90108790 msgr2=0x7f3d901a0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:55.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:55.783+0000 7f3d94d86700 1 --2- 172.21.15.47:0/2115338665 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3d90108790 0x7f3d901a0230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:55.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:55.783+0000 7f3d94d86700 1 -- 172.21.15.47:0/2115338665 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d90103530 msgr2=0x7f3d90198c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:55.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:55.783+0000 7f3d94d86700 1 --2- 172.21.15.47:0/2115338665 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d90103530 0x7f3d90198c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:55.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:55.783+0000 7f3d94d86700 1 -- 172.21.15.47:0/2115338665 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d901a0900 con 0x7f3d90105e60 2024-04-17T06:00:55.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:55.783+0000 7f3d94d86700 1 --2- 172.21.15.47:0/2115338665 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3d90105e60 0x7f3d901991c0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f3d84005910 tx=0x7f3d8400bbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:00:55.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:55.783+0000 7f3d8e7fc700 1 -- 172.21.15.47:0/2115338665 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d84026070 con 0x7f3d90105e60 2024-04-17T06:00:55.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:55.783+0000 7f3d8e7fc700 1 -- 172.21.15.47:0/2115338665 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d8400f460 con 0x7f3d90105e60 2024-04-17T06:00:55.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:55.784+0000 7f3d977eb700 1 -- 172.21.15.47:0/2115338665 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d901a0b90 con 0x7f3d90105e60 2024-04-17T06:00:55.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:55.784+0000 7f3d8e7fc700 1 -- 172.21.15.47:0/2115338665 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d84021670 con 0x7f3d90105e60 2024-04-17T06:00:55.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:55.784+0000 7f3d977eb700 1 -- 172.21.15.47:0/2115338665 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d901a0fe0 con 0x7f3d90105e60 2024-04-17T06:00:55.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:55.785+0000 7f3d977eb700 1 -- 172.21.15.47:0/2115338665 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d9002df90 con 0x7f3d90105e60 2024-04-17T06:00:55.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:55.786+0000 7f3d8e7fc700 1 -- 172.21.15.47:0/2115338665 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f3d84029070 con 0x7f3d90105e60 2024-04-17T06:00:55.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:55.786+0000 7f3d8e7fc700 1 --2- 172.21.15.47:0/2115338665 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f3d780a3c40 0x7f3d780a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:55.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:55.786+0000 7f3d95587700 1 --2- 172.21.15.47:0/2115338665 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f3d780a3c40 0x7f3d780a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:55.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:55.786+0000 7f3d8e7fc700 1 -- 172.21.15.47:0/2115338665 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f3d840d1230 con 0x7f3d90105e60 2024-04-17T06:00:55.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:55.787+0000 7f3d95587700 1 --2- 172.21.15.47:0/2115338665 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f3d780a3c40 0x7f3d780a6100 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f3d80009550 tx=0x7f3d800091b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:00:55.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:55.792+0000 7f3d8e7fc700 1 -- 172.21.15.47:0/2115338665 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f3d8405dff0 con 0x7f3d90105e60 2024-04-17T06:00:56.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:56.009+0000 7f3d977eb700 1 -- 172.21.15.47:0/2115338665 --> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f3d90062e90 con 0x7f3d780a3c40 2024-04-17T06:00:56.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:56.010+0000 7f3d8e7fc700 1 -- 172.21.15.47:0/2115338665 <== mgr.14152 v2:172.21.15.47:6800/3157902806 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f3d90062e90 con 0x7f3d780a3c40 2024-04-17T06:00:56.010 INFO:teuthology.orchestra.run.smithi047.stdout:HOST ADDR LABELS STATUS 2024-04-17T06:00:56.010 INFO:teuthology.orchestra.run.smithi047.stdout:smithi047 172.21.15.47 2024-04-17T06:00:56.010 INFO:teuthology.orchestra.run.smithi047.stdout:smithi098 172.21.15.98 2024-04-17T06:00:56.011 INFO:teuthology.orchestra.run.smithi047.stdout:smithi103 172.21.15.103 2024-04-17T06:00:56.011 INFO:teuthology.orchestra.run.smithi047.stdout:3 hosts in cluster 2024-04-17T06:00:56.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:56.013+0000 7f3d977eb700 1 -- 172.21.15.47:0/2115338665 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f3d780a3c40 msgr2=0x7f3d780a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:56.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:56.013+0000 7f3d977eb700 1 --2- 172.21.15.47:0/2115338665 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f3d780a3c40 0x7f3d780a6100 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f3d80009550 tx=0x7f3d800091b0 comp rx=0 tx=0).stop 2024-04-17T06:00:56.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:56.013+0000 7f3d977eb700 1 -- 172.21.15.47:0/2115338665 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3d90105e60 msgr2=0x7f3d901991c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:56.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:56.013+0000 7f3d977eb700 1 --2- 172.21.15.47:0/2115338665 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3d90105e60 0x7f3d901991c0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f3d84005910 tx=0x7f3d8400bbb0 comp rx=0 tx=0).stop 2024-04-17T06:00:56.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:56.013+0000 7f3d977eb700 1 -- 172.21.15.47:0/2115338665 shutdown_connections 2024-04-17T06:00:56.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:56.013+0000 7f3d977eb700 1 --2- 172.21.15.47:0/2115338665 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f3d780a3c40 0x7f3d780a6100 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:56.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:56.013+0000 7f3d977eb700 1 --2- 172.21.15.47:0/2115338665 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3d90108790 0x7f3d901a0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:56.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:56.013+0000 7f3d977eb700 1 --2- 172.21.15.47:0/2115338665 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d90103530 0x7f3d90198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:56.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:56.013+0000 7f3d977eb700 1 --2- 172.21.15.47:0/2115338665 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3d90105e60 0x7f3d901991c0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:56.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:56.013+0000 7f3d977eb700 1 -- 172.21.15.47:0/2115338665 >> 172.21.15.47:0/2115338665 conn(0x7f3d900fcf40 msgr2=0x7f3d90103df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:00:56.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:56.013+0000 7f3d977eb700 1 -- 172.21.15.47:0/2115338665 shutdown_connections 2024-04-17T06:00:56.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:56.013+0000 7f3d977eb700 1 -- 172.21.15.47:0/2115338665 wait complete. 2024-04-17T06:00:56.708 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 565db2d8-fc7e-11ee-bc91-c7b262605968 -- bash -c 'ceph orch device ls' 2024-04-17T06:00:56.958 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T06:00:57.080 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:00:56 smithi047 ceph-mon[32223]: pgmap v339: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 7 op/s 2024-04-17T06:00:57.080 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:00:56 smithi047 ceph-mon[32223]: from='client.15156 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T06:00:57.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:00:56 smithi098 ceph-mon[36516]: pgmap v339: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 7 op/s 2024-04-17T06:00:57.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:00:56 smithi098 ceph-mon[36516]: from='client.15156 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T06:00:57.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:00:56 smithi103 ceph-mon[36243]: pgmap v339: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 7 op/s 2024-04-17T06:00:57.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:00:56 smithi103 ceph-mon[36243]: from='client.15156 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T06:00:58.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:58.721+0000 7f8dcc424700 1 -- 172.21.15.47:0/2725917976 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8dc4103540 msgr2=0x7f8dc4109df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:58.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:58.721+0000 7f8dcc424700 1 --2- 172.21.15.47:0/2725917976 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8dc4103540 0x7f8dc4109df0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f8dbc00b0d0 tx=0x7f8dbc00b3e0 comp rx=0 tx=0).stop 2024-04-17T06:00:58.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:58.721+0000 7f8dcc424700 1 -- 172.21.15.47:0/2725917976 shutdown_connections 2024-04-17T06:00:58.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:58.721+0000 7f8dcc424700 1 --2- 172.21.15.47:0/2725917976 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8dc4103540 0x7f8dc4109df0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:58.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:58.721+0000 7f8dcc424700 1 --2- 172.21.15.47:0/2725917976 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8dc41019a0 0x7f8dc4101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:58.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:58.721+0000 7f8dcc424700 1 --2- 172.21.15.47:0/2725917976 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8dc4102ba0 0x7f8dc4103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:58.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:58.721+0000 7f8dcc424700 1 -- 172.21.15.47:0/2725917976 >> 172.21.15.47:0/2725917976 conn(0x7f8dc40fcf40 msgr2=0x7f8dc40ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:00:58.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:58.722+0000 7f8dcc424700 1 -- 172.21.15.47:0/2725917976 shutdown_connections 2024-04-17T06:00:58.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:58.722+0000 7f8dcc424700 1 -- 172.21.15.47:0/2725917976 wait complete. 2024-04-17T06:00:58.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:58.722+0000 7f8dcc424700 1 Processor -- start 2024-04-17T06:00:58.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:58.722+0000 7f8dcc424700 1 -- start start 2024-04-17T06:00:58.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:58.722+0000 7f8dcc424700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8dc41019a0 0x7f8dc41059b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:58.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:58.722+0000 7f8dcc424700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8dc4102ba0 0x7f8dc4104050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:58.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:58.723+0000 7f8dcc424700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8dc4103540 0x7f8dc4104590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:58.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:58.723+0000 7f8dcc424700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f8dc410d680 con 0x7f8dc4102ba0 2024-04-17T06:00:58.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:58.723+0000 7f8dcc424700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8dc410d500 con 0x7f8dc4103540 2024-04-17T06:00:58.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:58.723+0000 7f8dcc424700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8dc410d800 con 0x7f8dc41019a0 2024-04-17T06:00:58.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:58.723+0000 7f8dc99bf700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8dc4102ba0 0x7f8dc4104050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:58.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:58.723+0000 7f8dc99bf700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8dc4102ba0 0x7f8dc4104050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53816/0 (socket says 172.21.15.47:53816) 2024-04-17T06:00:58.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:58.723+0000 7f8dc99bf700 1 -- 172.21.15.47:0/1061250428 learned_addr learned my addr 172.21.15.47:0/1061250428 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:00:58.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:58.723+0000 7f8dca9c1700 1 --2- 172.21.15.47:0/1061250428 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8dc4103540 0x7f8dc4104590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:58.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:58.723+0000 7f8dca1c0700 1 --2- 172.21.15.47:0/1061250428 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8dc41019a0 0x7f8dc41059b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:58.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:58.723+0000 7f8dc99bf700 1 -- 172.21.15.47:0/1061250428 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8dc41019a0 msgr2=0x7f8dc41059b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:58.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:58.723+0000 7f8dc99bf700 1 --2- 172.21.15.47:0/1061250428 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8dc41019a0 0x7f8dc41059b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:58.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:58.723+0000 7f8dc99bf700 1 -- 172.21.15.47:0/1061250428 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8dc4103540 msgr2=0x7f8dc4104590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:58.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:58.723+0000 7f8dc99bf700 1 --2- 172.21.15.47:0/1061250428 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8dc4103540 0x7f8dc4104590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:58.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:58.723+0000 7f8dc99bf700 1 -- 172.21.15.47:0/1061250428 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dc4104e60 con 0x7f8dc4102ba0 2024-04-17T06:00:58.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:58.723+0000 7f8dc99bf700 1 --2- 172.21.15.47:0/1061250428 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8dc4102ba0 0x7f8dc4104050 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f8db400b530 tx=0x7f8db400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:00:58.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:58.723+0000 7f8dbb7fe700 1 -- 172.21.15.47:0/1061250428 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8db4015020 con 0x7f8dc4102ba0 2024-04-17T06:00:58.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:58.723+0000 7f8dbb7fe700 1 -- 172.21.15.47:0/1061250428 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8db4011d10 con 0x7f8dc4102ba0 2024-04-17T06:00:58.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:58.723+0000 7f8dbb7fe700 1 -- 172.21.15.47:0/1061250428 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8db400f2e0 con 0x7f8dc4102ba0 2024-04-17T06:00:58.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:58.723+0000 7f8dcc424700 1 -- 172.21.15.47:0/1061250428 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8dc41a2f50 con 0x7f8dc4102ba0 2024-04-17T06:00:58.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:58.724+0000 7f8dcc424700 1 -- 172.21.15.47:0/1061250428 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8dc41a33d0 con 0x7f8dc4102ba0 2024-04-17T06:00:58.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:58.724+0000 7f8dcc424700 1 -- 172.21.15.47:0/1061250428 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8dc4069740 con 0x7f8dc4102ba0 2024-04-17T06:00:58.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:58.726+0000 7f8dbb7fe700 1 -- 172.21.15.47:0/1061250428 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f8db4018020 con 0x7f8dc4102ba0 2024-04-17T06:00:58.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:58.726+0000 7f8dbb7fe700 1 --2- 172.21.15.47:0/1061250428 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f8db00a3c40 0x7f8db00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:00:58.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:58.727+0000 7f8dca1c0700 1 --2- 172.21.15.47:0/1061250428 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f8db00a3c40 0x7f8db00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:00:58.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:58.727+0000 7f8dbb7fe700 1 -- 172.21.15.47:0/1061250428 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f8db40cff90 con 0x7f8dc4102ba0 2024-04-17T06:00:58.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:58.727+0000 7f8dca1c0700 1 --2- 172.21.15.47:0/1061250428 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f8db00a3c40 0x7f8db00a6100 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8dc4102a00 tx=0x7f8dc0009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:00:58.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:58.730+0000 7f8dbb7fe700 1 -- 172.21.15.47:0/1061250428 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f8db401f020 con 0x7f8dc4102ba0 2024-04-17T06:00:58.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:58.949+0000 7f8dcc424700 1 -- 172.21.15.47:0/1061250428 --> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f8dc406b8f0 con 0x7f8db00a3c40 2024-04-17T06:00:58.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:58.951+0000 7f8dbb7fe700 1 -- 172.21.15.47:0/1061250428 <== mgr.14152 v2:172.21.15.47:6800/3157902806 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f8dc406b8f0 con 0x7f8db00a3c40 2024-04-17T06:00:58.948 INFO:teuthology.orchestra.run.smithi047.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-04-17T06:00:58.949 INFO:teuthology.orchestra.run.smithi047.stdout:smithi047 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB75130022480DGN 447G No 3m ago Has a FileSystem, LVM detected 2024-04-17T06:00:58.949 INFO:teuthology.orchestra.run.smithi047.stdout:smithi098 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT6204010H400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-04-17T06:00:58.949 INFO:teuthology.orchestra.run.smithi047.stdout:smithi103 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT6204012Z400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-04-17T06:00:58.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:58.953+0000 7f8dcc424700 1 -- 172.21.15.47:0/1061250428 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f8db00a3c40 msgr2=0x7f8db00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:58.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:58.953+0000 7f8dcc424700 1 --2- 172.21.15.47:0/1061250428 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f8db00a3c40 0x7f8db00a6100 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8dc4102a00 tx=0x7f8dc0009190 comp rx=0 tx=0).stop 2024-04-17T06:00:58.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:58.953+0000 7f8dcc424700 1 -- 172.21.15.47:0/1061250428 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8dc4102ba0 msgr2=0x7f8dc4104050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:00:58.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:58.953+0000 7f8dcc424700 1 --2- 172.21.15.47:0/1061250428 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8dc4102ba0 0x7f8dc4104050 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f8db400b530 tx=0x7f8db400b840 comp rx=0 tx=0).stop 2024-04-17T06:00:58.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:58.953+0000 7f8dcc424700 1 -- 172.21.15.47:0/1061250428 shutdown_connections 2024-04-17T06:00:58.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:58.953+0000 7f8dcc424700 1 --2- 172.21.15.47:0/1061250428 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f8db00a3c40 0x7f8db00a6100 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:58.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:58.953+0000 7f8dcc424700 1 --2- 172.21.15.47:0/1061250428 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8dc4103540 0x7f8dc4104590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:58.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:58.953+0000 7f8dcc424700 1 --2- 172.21.15.47:0/1061250428 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8dc41019a0 0x7f8dc41059b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:58.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:58.953+0000 7f8dcc424700 1 --2- 172.21.15.47:0/1061250428 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8dc4102ba0 0x7f8dc4104050 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:00:58.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:58.953+0000 7f8dcc424700 1 -- 172.21.15.47:0/1061250428 >> 172.21.15.47:0/1061250428 conn(0x7f8dc40fcf40 msgr2=0x7f8dc406b1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:00:58.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:58.953+0000 7f8dcc424700 1 -- 172.21.15.47:0/1061250428 shutdown_connections 2024-04-17T06:00:58.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:00:58.953+0000 7f8dcc424700 1 -- 172.21.15.47:0/1061250428 wait complete. 2024-04-17T06:00:59.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:00:58 smithi047 ceph-mon[32223]: pgmap v340: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 6 op/s 2024-04-17T06:00:59.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:00:58 smithi103 ceph-mon[36243]: pgmap v340: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 6 op/s 2024-04-17T06:00:59.382 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:00:58 smithi098 ceph-mon[36516]: pgmap v340: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 6 op/s 2024-04-17T06:00:59.641 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-17T06:00:59.651 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi047.front.sepia.ceph.com 2024-04-17T06:00:59.651 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 565db2d8-fc7e-11ee-bc91-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-17T06:00:59.904 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T06:01:00.167 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:00:59 smithi047 ceph-mon[32223]: from='client.15162 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T06:01:00.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:00:59 smithi103 ceph-mon[36243]: from='client.15162 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T06:01:00.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:00:59 smithi098 ceph-mon[36516]: from='client.15162 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T06:01:01.187 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:00 smithi047 ceph-mon[32223]: pgmap v341: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 0 B/s wr, 6 op/s 2024-04-17T06:01:01.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:00 smithi103 ceph-mon[36243]: pgmap v341: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 0 B/s wr, 6 op/s 2024-04-17T06:01:01.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:00 smithi098 ceph-mon[36516]: pgmap v341: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 0 B/s wr, 6 op/s 2024-04-17T06:01:01.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:01.727+0000 7fd6d33a5700 1 -- 172.21.15.47:0/2091424331 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6cc1087f0 msgr2=0x7fd6cc10ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:01.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:01.727+0000 7fd6d33a5700 1 --2- 172.21.15.47:0/2091424331 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6cc1087f0 0x7fd6cc10ac00 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fd6c400b0a0 tx=0x7fd6c400b3b0 comp rx=0 tx=0).stop 2024-04-17T06:01:01.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:01.727+0000 7fd6d33a5700 1 -- 172.21.15.47:0/2091424331 shutdown_connections 2024-04-17T06:01:01.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:01.727+0000 7fd6d33a5700 1 --2- 172.21.15.47:0/2091424331 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6cc1087f0 0x7fd6cc10ac00 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:01.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:01.727+0000 7fd6d33a5700 1 --2- 172.21.15.47:0/2091424331 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd6cc103590 0x7fd6cc105980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:01.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:01.727+0000 7fd6d33a5700 1 --2- 172.21.15.47:0/2091424331 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd6cc105ec0 0x7fd6cc1082b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:01.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:01.727+0000 7fd6d33a5700 1 -- 172.21.15.47:0/2091424331 >> 172.21.15.47:0/2091424331 conn(0x7fd6cc0fcf40 msgr2=0x7fd6cc0ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:01:01.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:01.727+0000 7fd6d33a5700 1 -- 172.21.15.47:0/2091424331 shutdown_connections 2024-04-17T06:01:01.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:01.727+0000 7fd6d33a5700 1 -- 172.21.15.47:0/2091424331 wait complete. 2024-04-17T06:01:01.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:01.728+0000 7fd6d33a5700 1 Processor -- start 2024-04-17T06:01:01.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:01.728+0000 7fd6d33a5700 1 -- start start 2024-04-17T06:01:01.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:01.728+0000 7fd6d33a5700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd6cc103590 0x7fd6cc102ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:01.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:01.728+0000 7fd6d33a5700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd6cc105ec0 0x7fd6cc101540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:01.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:01.728+0000 7fd6d33a5700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6cc1087f0 0x7fd6cc101a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:01.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:01.728+0000 7fd6d33a5700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fd6cc10e670 con 0x7fd6cc1087f0 2024-04-17T06:01:01.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:01.728+0000 7fd6d33a5700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd6cc10e4f0 con 0x7fd6cc103590 2024-04-17T06:01:01.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:01.728+0000 7fd6d33a5700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd6cc10e7f0 con 0x7fd6cc105ec0 2024-04-17T06:01:01.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:01.728+0000 7fd6d1942700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6cc1087f0 0x7fd6cc101a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:01.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:01.728+0000 7fd6d1942700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6cc1087f0 0x7fd6cc101a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53828/0 (socket says 172.21.15.47:53828) 2024-04-17T06:01:01.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:01.728+0000 7fd6d1942700 1 -- 172.21.15.47:0/4065857232 learned_addr learned my addr 172.21.15.47:0/4065857232 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:01:01.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:01.728+0000 7fd6d1141700 1 --2- 172.21.15.47:0/4065857232 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd6cc103590 0x7fd6cc102ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:01.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:01.728+0000 7fd6d0940700 1 --2- 172.21.15.47:0/4065857232 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd6cc105ec0 0x7fd6cc101540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:01.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:01.728+0000 7fd6d1942700 1 -- 172.21.15.47:0/4065857232 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd6cc105ec0 msgr2=0x7fd6cc101540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:01.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:01.729+0000 7fd6d1942700 1 --2- 172.21.15.47:0/4065857232 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd6cc105ec0 0x7fd6cc101540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:01.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:01.729+0000 7fd6d1942700 1 -- 172.21.15.47:0/4065857232 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd6cc103590 msgr2=0x7fd6cc102ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:01.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:01.729+0000 7fd6d1942700 1 --2- 172.21.15.47:0/4065857232 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd6cc103590 0x7fd6cc102ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:01.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:01.729+0000 7fd6d1942700 1 -- 172.21.15.47:0/4065857232 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6cc102330 con 0x7fd6cc1087f0 2024-04-17T06:01:01.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:01.729+0000 7fd6d0940700 1 --2- 172.21.15.47:0/4065857232 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd6cc105ec0 0x7fd6cc101540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:01:01.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:01.729+0000 7fd6d1141700 1 --2- 172.21.15.47:0/4065857232 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd6cc103590 0x7fd6cc102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:01:01.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:01.729+0000 7fd6d1942700 1 --2- 172.21.15.47:0/4065857232 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6cc1087f0 0x7fd6cc101a80 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fd6c4007f00 tx=0x7fd6c4004e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:01:01.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:01.729+0000 7fd6be7fc700 1 -- 172.21.15.47:0/4065857232 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6c4021070 con 0x7fd6cc1087f0 2024-04-17T06:01:01.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:01.729+0000 7fd6d33a5700 1 -- 172.21.15.47:0/4065857232 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6cc1a2d50 con 0x7fd6cc1087f0 2024-04-17T06:01:01.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:01.729+0000 7fd6be7fc700 1 -- 172.21.15.47:0/4065857232 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6c401d930 con 0x7fd6cc1087f0 2024-04-17T06:01:01.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:01.729+0000 7fd6be7fc700 1 -- 172.21.15.47:0/4065857232 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6c4005570 con 0x7fd6cc1087f0 2024-04-17T06:01:01.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:01.729+0000 7fd6d33a5700 1 -- 172.21.15.47:0/4065857232 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6cc1a31b0 con 0x7fd6cc1087f0 2024-04-17T06:01:01.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:01.730+0000 7fd6d33a5700 1 -- 172.21.15.47:0/4065857232 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6cc069740 con 0x7fd6cc1087f0 2024-04-17T06:01:01.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:01.731+0000 7fd6be7fc700 1 -- 172.21.15.47:0/4065857232 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fd6c4024070 con 0x7fd6cc1087f0 2024-04-17T06:01:01.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:01.732+0000 7fd6be7fc700 1 --2- 172.21.15.47:0/4065857232 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fd6b80a3c40 0x7fd6b80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:01.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:01.732+0000 7fd6d1141700 1 --2- 172.21.15.47:0/4065857232 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fd6b80a3c40 0x7fd6b80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:01.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:01.732+0000 7fd6be7fc700 1 -- 172.21.15.47:0/4065857232 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fd6c400f020 con 0x7fd6cc1087f0 2024-04-17T06:01:01.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:01.732+0000 7fd6d1141700 1 --2- 172.21.15.47:0/4065857232 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fd6b80a3c40 0x7fd6b80a6100 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fd6c8005950 tx=0x7fd6c8005f30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:01:01.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:01.737+0000 7fd6be7fc700 1 -- 172.21.15.47:0/4065857232 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fd6c402b020 con 0x7fd6cc1087f0 2024-04-17T06:01:01.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:01.993+0000 7fd6d33a5700 1 -- 172.21.15.47:0/4065857232 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fd6cc063660 con 0x7fd6cc1087f0 2024-04-17T06:01:01.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:01.993+0000 7fd6be7fc700 1 -- 172.21.15.47:0/4065857232 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 21 v21) ==== 76+0+3057 (secure 0 0 0) 0x7fd6c4024360 con 0x7fd6cc1087f0 2024-04-17T06:01:01.991 INFO:teuthology.orchestra.run.smithi047.stdout:e21 2024-04-17T06:01:01.991 INFO:teuthology.orchestra.run.smithi047.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-17T06:01:01.991 INFO:teuthology.orchestra.run.smithi047.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-17T06:01:01.991 INFO:teuthology.orchestra.run.smithi047.stdout:legacy client fscid: 1 2024-04-17T06:01:01.991 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-17T06:01:01.991 INFO:teuthology.orchestra.run.smithi047.stdout:Filesystem 'cephfs' (1) 2024-04-17T06:01:01.992 INFO:teuthology.orchestra.run.smithi047.stdout:fs_name cephfs 2024-04-17T06:01:01.992 INFO:teuthology.orchestra.run.smithi047.stdout:epoch 21 2024-04-17T06:01:01.992 INFO:teuthology.orchestra.run.smithi047.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-17T06:01:01.992 INFO:teuthology.orchestra.run.smithi047.stdout:created 2024-04-17T05:59:13.922176+0000 2024-04-17T06:01:01.992 INFO:teuthology.orchestra.run.smithi047.stdout:modified 2024-04-17T05:59:19.598261+0000 2024-04-17T06:01:01.992 INFO:teuthology.orchestra.run.smithi047.stdout:tableserver 0 2024-04-17T06:01:01.992 INFO:teuthology.orchestra.run.smithi047.stdout:root 0 2024-04-17T06:01:01.992 INFO:teuthology.orchestra.run.smithi047.stdout:session_timeout 300 2024-04-17T06:01:01.992 INFO:teuthology.orchestra.run.smithi047.stdout:session_autoclose 300 2024-04-17T06:01:01.992 INFO:teuthology.orchestra.run.smithi047.stdout:max_file_size 1099511627776 2024-04-17T06:01:01.992 INFO:teuthology.orchestra.run.smithi047.stdout:max_xattr_size 65536 2024-04-17T06:01:01.992 INFO:teuthology.orchestra.run.smithi047.stdout:required_client_features {} 2024-04-17T06:01:01.992 INFO:teuthology.orchestra.run.smithi047.stdout:last_failure 0 2024-04-17T06:01:01.992 INFO:teuthology.orchestra.run.smithi047.stdout:last_failure_osd_epoch 0 2024-04-17T06:01:01.992 INFO:teuthology.orchestra.run.smithi047.stdout:snap_rstat 1 2024-04-17T06:01:01.993 INFO:teuthology.orchestra.run.smithi047.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-17T06:01:01.993 INFO:teuthology.orchestra.run.smithi047.stdout:max_mds 3 2024-04-17T06:01:01.993 INFO:teuthology.orchestra.run.smithi047.stdout:in 0,1,2 2024-04-17T06:01:01.993 INFO:teuthology.orchestra.run.smithi047.stdout:up {0=24518,1=24479,2=24466} 2024-04-17T06:01:01.993 INFO:teuthology.orchestra.run.smithi047.stdout:failed 2024-04-17T06:01:01.993 INFO:teuthology.orchestra.run.smithi047.stdout:damaged 2024-04-17T06:01:01.993 INFO:teuthology.orchestra.run.smithi047.stdout:stopped 2024-04-17T06:01:01.993 INFO:teuthology.orchestra.run.smithi047.stdout:data_pools [3] 2024-04-17T06:01:01.993 INFO:teuthology.orchestra.run.smithi047.stdout:metadata_pool 2 2024-04-17T06:01:01.993 INFO:teuthology.orchestra.run.smithi047.stdout:inline_data disabled 2024-04-17T06:01:01.993 INFO:teuthology.orchestra.run.smithi047.stdout:balancer 2024-04-17T06:01:01.993 INFO:teuthology.orchestra.run.smithi047.stdout:bal_rank_mask -1 2024-04-17T06:01:01.993 INFO:teuthology.orchestra.run.smithi047.stdout:standby_count_wanted 0 2024-04-17T06:01:01.993 INFO:teuthology.orchestra.run.smithi047.stdout:qdb_cluster leader: 24518 members: 24479,24518,24466 2024-04-17T06:01:01.994 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.b{0:24518} state up:active seq 3 addr [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] compat {c=[1],r=[1],i=[1fff]}] 2024-04-17T06:01:01.994 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.e{0:24449} state up:standby-replay seq 1 addr [v2:172.21.15.98:6834/1966500278,v1:172.21.15.98:6835/1966500278] compat {c=[1],r=[1],i=[1fff]}] 2024-04-17T06:01:01.994 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.h{1:24479} state up:active seq 7 addr [v2:172.21.15.98:6836/1117440719,v1:172.21.15.98:6837/1117440719] compat {c=[1],r=[1],i=[1fff]}] 2024-04-17T06:01:01.994 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.c{1:24448} state up:standby-replay seq 1 addr [v2:172.21.15.103:6836/116936024,v1:172.21.15.103:6837/116936024] compat {c=[1],r=[1],i=[1fff]}] 2024-04-17T06:01:01.994 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.i{2:24466} state up:active seq 6 addr [v2:172.21.15.103:6838/1467157889,v1:172.21.15.103:6839/1467157889] compat {c=[1],r=[1],i=[1fff]}] 2024-04-17T06:01:01.994 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.f{2:24436} state up:standby-replay seq 1 addr [v2:172.21.15.103:6834/270543522,v1:172.21.15.103:6835/270543522] compat {c=[1],r=[1],i=[1fff]}] 2024-04-17T06:01:01.994 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-17T06:01:01.994 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-17T06:01:01.994 INFO:teuthology.orchestra.run.smithi047.stdout:Standby daemons: 2024-04-17T06:01:01.994 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-17T06:01:01.994 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.j{-1:14583} state up:standby seq 1 addr [v2:172.21.15.47:6834/3189958720,v1:172.21.15.47:6835/3189958720] compat {c=[1],r=[1],i=[1fff]}] 2024-04-17T06:01:01.994 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.d{-1:14595} state up:standby seq 1 addr [v2:172.21.15.47:6836/3143529973,v1:172.21.15.47:6837/3143529973] compat {c=[1],r=[1],i=[1fff]}] 2024-04-17T06:01:01.994 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.g{-1:14652} state up:standby seq 1 addr [v2:172.21.15.47:6838/2375823701,v1:172.21.15.47:6839/2375823701] compat {c=[1],r=[1],i=[1fff]}] 2024-04-17T06:01:01.995 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.a{-1:14664} state up:standby seq 1 addr [v2:172.21.15.47:6840/3334309157,v1:172.21.15.47:6841/3334309157] compat {c=[1],r=[1],i=[1fff]}] 2024-04-17T06:01:01.995 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.l{-1:24394} state up:standby seq 1 addr [v2:172.21.15.103:6832/2398363754,v1:172.21.15.103:6833/2398363754] compat {c=[1],r=[1],i=[1fff]}] 2024-04-17T06:01:01.995 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.k{-1:24416} state up:standby seq 1 addr [v2:172.21.15.98:6832/384473493,v1:172.21.15.98:6833/384473493] compat {c=[1],r=[1],i=[1fff]}] 2024-04-17T06:01:01.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:01.995+0000 7fd6d33a5700 1 -- 172.21.15.47:0/4065857232 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fd6b80a3c40 msgr2=0x7fd6b80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:01.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:01.995+0000 7fd6d33a5700 1 --2- 172.21.15.47:0/4065857232 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fd6b80a3c40 0x7fd6b80a6100 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fd6c8005950 tx=0x7fd6c8005f30 comp rx=0 tx=0).stop 2024-04-17T06:01:01.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:01.995+0000 7fd6d33a5700 1 -- 172.21.15.47:0/4065857232 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6cc1087f0 msgr2=0x7fd6cc101a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:01.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:01.995+0000 7fd6d33a5700 1 --2- 172.21.15.47:0/4065857232 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6cc1087f0 0x7fd6cc101a80 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fd6c4007f00 tx=0x7fd6c4004e20 comp rx=0 tx=0).stop 2024-04-17T06:01:01.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:01.995+0000 7fd6d33a5700 1 -- 172.21.15.47:0/4065857232 shutdown_connections 2024-04-17T06:01:01.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:01.995+0000 7fd6d33a5700 1 --2- 172.21.15.47:0/4065857232 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fd6b80a3c40 0x7fd6b80a6100 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:01.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:01.995+0000 7fd6d33a5700 1 --2- 172.21.15.47:0/4065857232 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6cc1087f0 0x7fd6cc101a80 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:01.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:01.995+0000 7fd6d33a5700 1 --2- 172.21.15.47:0/4065857232 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd6cc103590 0x7fd6cc102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:01.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:01.995+0000 7fd6d33a5700 1 --2- 172.21.15.47:0/4065857232 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd6cc105ec0 0x7fd6cc101540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:01.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:01.995+0000 7fd6d33a5700 1 -- 172.21.15.47:0/4065857232 >> 172.21.15.47:0/4065857232 conn(0x7fd6cc0fcf40 msgr2=0x7fd6cc103ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:01:01.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:01.996+0000 7fd6d33a5700 1 -- 172.21.15.47:0/4065857232 shutdown_connections 2024-04-17T06:01:01.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:01.996+0000 7fd6d33a5700 1 -- 172.21.15.47:0/4065857232 wait complete. 2024-04-17T06:01:01.996 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 21 2024-04-17T06:01:02.822 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 565db2d8-fc7e-11ee-bc91-c7b262605968 -- bash -c 'ceph osd dump' 2024-04-17T06:01:03.078 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T06:01:03.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:02 smithi047 ceph-mon[32223]: pgmap v342: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-17T06:01:03.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:02 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/4065857232' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-17T06:01:03.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:02 smithi103 ceph-mon[36243]: pgmap v342: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-17T06:01:03.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:02 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/4065857232' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-17T06:01:03.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:02 smithi098 ceph-mon[36516]: pgmap v342: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-17T06:01:03.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:02 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/4065857232' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-17T06:01:04.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:04.890+0000 7f2217376700 1 -- 172.21.15.47:0/2431395549 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2210103cb0 msgr2=0x7f2210104110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:04.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:04.890+0000 7f2217376700 1 --2- 172.21.15.47:0/2431395549 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2210103cb0 0x7f2210104110 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f22000099c0 tx=0x7f2200009cd0 comp rx=0 tx=0).stop 2024-04-17T06:01:04.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:04.890+0000 7f2217376700 1 -- 172.21.15.47:0/2431395549 shutdown_connections 2024-04-17T06:01:04.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:04.890+0000 7f2217376700 1 --2- 172.21.15.47:0/2431395549 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2210104650 0x7f221010af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:04.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:04.890+0000 7f2217376700 1 --2- 172.21.15.47:0/2431395549 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2210102ab0 0x7f2210102ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:04.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:04.890+0000 7f2217376700 1 --2- 172.21.15.47:0/2431395549 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2210103cb0 0x7f2210104110 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:04.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:04.890+0000 7f2217376700 1 -- 172.21.15.47:0/2431395549 >> 172.21.15.47:0/2431395549 conn(0x7f22100fe050 msgr2=0x7f2210100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:01:04.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:04.890+0000 7f2217376700 1 -- 172.21.15.47:0/2431395549 shutdown_connections 2024-04-17T06:01:04.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:04.890+0000 7f2217376700 1 -- 172.21.15.47:0/2431395549 wait complete. 2024-04-17T06:01:04.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:04.890+0000 7f2217376700 1 Processor -- start 2024-04-17T06:01:04.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:04.891+0000 7f2217376700 1 -- start start 2024-04-17T06:01:04.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:04.891+0000 7f2217376700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2210102ab0 0x7f221019aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:04.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:04.891+0000 7f2217376700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2210103cb0 0x7f221019b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:04.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:04.891+0000 7f2217376700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2210104650 0x7f22101a24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:04.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:04.891+0000 7f2217376700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f221010e690 con 0x7f2210102ab0 2024-04-17T06:01:04.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:04.891+0000 7f2217376700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f221010e510 con 0x7f2210104650 2024-04-17T06:01:04.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:04.891+0000 7f2217376700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f221010e810 con 0x7f2210103cb0 2024-04-17T06:01:04.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:04.891+0000 7f2215112700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2210102ab0 0x7f221019aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:04.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:04.891+0000 7f2214911700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2210103cb0 0x7f221019b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:04.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:04.891+0000 7f2215112700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2210102ab0 0x7f221019aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40262/0 (socket says 172.21.15.47:40262) 2024-04-17T06:01:04.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:04.891+0000 7f2214911700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2210103cb0 0x7f221019b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.47:55834/0 (socket says 172.21.15.47:55834) 2024-04-17T06:01:04.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:04.891+0000 7f2215112700 1 -- 172.21.15.47:0/4024504948 learned_addr learned my addr 172.21.15.47:0/4024504948 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:01:04.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:04.891+0000 7f2215913700 1 --2- 172.21.15.47:0/4024504948 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2210104650 0x7f22101a24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:04.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:04.891+0000 7f2215112700 1 -- 172.21.15.47:0/4024504948 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2210103cb0 msgr2=0x7f221019b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:04.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:04.891+0000 7f2215112700 1 --2- 172.21.15.47:0/4024504948 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2210103cb0 0x7f221019b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:04.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:04.891+0000 7f2215112700 1 -- 172.21.15.47:0/4024504948 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2210104650 msgr2=0x7f22101a24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:04.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:04.891+0000 7f2215112700 1 --2- 172.21.15.47:0/4024504948 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2210104650 0x7f22101a24a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:04.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:04.891+0000 7f2215112700 1 -- 172.21.15.47:0/4024504948 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22101a2b40 con 0x7f2210102ab0 2024-04-17T06:01:04.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:04.891+0000 7f2215913700 1 --2- 172.21.15.47:0/4024504948 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2210104650 0x7f22101a24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:01:04.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:04.891+0000 7f2214911700 1 --2- 172.21.15.47:0/4024504948 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2210103cb0 0x7f221019b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T06:01:04.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:04.892+0000 7f2215112700 1 --2- 172.21.15.47:0/4024504948 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2210102ab0 0x7f221019aee0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f220c00d9b0 tx=0x7f220c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:01:04.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:04.892+0000 7f22067fc700 1 -- 172.21.15.47:0/4024504948 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f220c017070 con 0x7f2210102ab0 2024-04-17T06:01:04.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:04.892+0000 7f22067fc700 1 -- 172.21.15.47:0/4024504948 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f220c009e10 con 0x7f2210102ab0 2024-04-17T06:01:04.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:04.892+0000 7f2217376700 1 -- 172.21.15.47:0/4024504948 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22101a2e30 con 0x7f2210102ab0 2024-04-17T06:01:04.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:04.892+0000 7f22067fc700 1 -- 172.21.15.47:0/4024504948 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f220c00f620 con 0x7f2210102ab0 2024-04-17T06:01:04.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:04.892+0000 7f2217376700 1 -- 172.21.15.47:0/4024504948 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22101a3280 con 0x7f2210102ab0 2024-04-17T06:01:04.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:04.893+0000 7f2217376700 1 -- 172.21.15.47:0/4024504948 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2210069770 con 0x7f2210102ab0 2024-04-17T06:01:04.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:04.895+0000 7f22067fc700 1 -- 172.21.15.47:0/4024504948 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f220c009930 con 0x7f2210102ab0 2024-04-17T06:01:04.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:04.895+0000 7f22067fc700 1 --2- 172.21.15.47:0/4024504948 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f21fc0a3cc0 0x7f21fc0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:04.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:04.896+0000 7f2214911700 1 --2- 172.21.15.47:0/4024504948 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f21fc0a3cc0 0x7f21fc0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:04.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:04.896+0000 7f22067fc700 1 -- 172.21.15.47:0/4024504948 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f220c0cfee0 con 0x7f2210102ab0 2024-04-17T06:01:04.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:04.896+0000 7f2214911700 1 --2- 172.21.15.47:0/4024504948 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f21fc0a3cc0 0x7f21fc0a6180 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2200000c00 tx=0x7f2200005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:01:04.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:04.898+0000 7f22067fc700 1 -- 172.21.15.47:0/4024504948 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f2210069770 con 0x7f2210102ab0 2024-04-17T06:01:05.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:05.103+0000 7f2217376700 1 -- 172.21.15.47:0/4024504948 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f22101a3850 con 0x7f2210102ab0 2024-04-17T06:01:05.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:05.104+0000 7f22067fc700 1 -- 172.21.15.47:0/4024504948 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v88) ==== 56+0+5265 (secure 0 0 0) 0x7f220c09c8f0 con 0x7f2210102ab0 2024-04-17T06:01:05.102 INFO:teuthology.orchestra.run.smithi047.stdout:epoch 88 2024-04-17T06:01:05.102 INFO:teuthology.orchestra.run.smithi047.stdout:fsid 565db2d8-fc7e-11ee-bc91-c7b262605968 2024-04-17T06:01:05.103 INFO:teuthology.orchestra.run.smithi047.stdout:created 2024-04-17T05:51:19.888865+0000 2024-04-17T06:01:05.103 INFO:teuthology.orchestra.run.smithi047.stdout:modified 2024-04-17T06:00:14.720609+0000 2024-04-17T06:01:05.103 INFO:teuthology.orchestra.run.smithi047.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-04-17T06:01:05.103 INFO:teuthology.orchestra.run.smithi047.stdout:crush_version 26 2024-04-17T06:01:05.103 INFO:teuthology.orchestra.run.smithi047.stdout:full_ratio 0.95 2024-04-17T06:01:05.103 INFO:teuthology.orchestra.run.smithi047.stdout:backfillfull_ratio 0.9 2024-04-17T06:01:05.103 INFO:teuthology.orchestra.run.smithi047.stdout:nearfull_ratio 0.85 2024-04-17T06:01:05.103 INFO:teuthology.orchestra.run.smithi047.stdout:require_min_compat_client luminous 2024-04-17T06:01:05.103 INFO:teuthology.orchestra.run.smithi047.stdout:min_compat_client luminous 2024-04-17T06:01:05.103 INFO:teuthology.orchestra.run.smithi047.stdout:require_osd_release squid 2024-04-17T06:01:05.103 INFO:teuthology.orchestra.run.smithi047.stdout:stretch_mode_enabled false 2024-04-17T06:01:05.103 INFO:teuthology.orchestra.run.smithi047.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-04-17T06:01:05.103 INFO:teuthology.orchestra.run.smithi047.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-04-17T06:01:05.104 INFO:teuthology.orchestra.run.smithi047.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 86 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-04-17T06:01:05.104 INFO:teuthology.orchestra.run.smithi047.stdout:max_osd 12 2024-04-17T06:01:05.104 INFO:teuthology.orchestra.run.smithi047.stdout:osd.0 up in weight 1 up_from 8 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.47:6802/1290127544,v1:172.21.15.47:6803/1290127544] [v2:172.21.15.47:6804/1290127544,v1:172.21.15.47:6805/1290127544] exists,up 8453f300-533d-4736-a343-55619c08e966 2024-04-17T06:01:05.104 INFO:teuthology.orchestra.run.smithi047.stdout:osd.1 up in weight 1 up_from 13 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.98:6800/1180430954,v1:172.21.15.98:6801/1180430954] [v2:172.21.15.98:6802/1180430954,v1:172.21.15.98:6803/1180430954] exists,up 80b0f03e-c1bc-4658-8028-1e6b8eedcc47 2024-04-17T06:01:05.104 INFO:teuthology.orchestra.run.smithi047.stdout:osd.2 up in weight 1 up_from 18 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.103:6800/494676216,v1:172.21.15.103:6801/494676216] [v2:172.21.15.103:6802/494676216,v1:172.21.15.103:6803/494676216] exists,up 0fc9d383-05b4-498d-b530-fccf7aeddd7c 2024-04-17T06:01:05.104 INFO:teuthology.orchestra.run.smithi047.stdout:osd.3 up in weight 1 up_from 26 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.47:6810/1998369447,v1:172.21.15.47:6811/1998369447] [v2:172.21.15.47:6812/1998369447,v1:172.21.15.47:6813/1998369447] exists,up f6e0b0d5-0fb3-42d0-9e84-f7cfef21a998 2024-04-17T06:01:05.104 INFO:teuthology.orchestra.run.smithi047.stdout:osd.4 up in weight 1 up_from 34 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.98:6808/3060482485,v1:172.21.15.98:6809/3060482485] [v2:172.21.15.98:6810/3060482485,v1:172.21.15.98:6811/3060482485] exists,up 2c26217e-02df-466d-bc01-0149d4aec3c5 2024-04-17T06:01:05.104 INFO:teuthology.orchestra.run.smithi047.stdout:osd.5 up in weight 1 up_from 39 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.103:6808/4181363567,v1:172.21.15.103:6809/4181363567] [v2:172.21.15.103:6810/4181363567,v1:172.21.15.103:6811/4181363567] exists,up 740bf3e0-bea8-496b-8d6f-e42d0b017258 2024-04-17T06:01:05.104 INFO:teuthology.orchestra.run.smithi047.stdout:osd.6 up in weight 1 up_from 47 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.47:6818/2517529668,v1:172.21.15.47:6819/2517529668] [v2:172.21.15.47:6820/2517529668,v1:172.21.15.47:6821/2517529668] exists,up 356a6255-2c07-4f0d-a233-26be04c92b38 2024-04-17T06:01:05.104 INFO:teuthology.orchestra.run.smithi047.stdout:osd.7 up in weight 1 up_from 52 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.98:6816/4225498496,v1:172.21.15.98:6817/4225498496] [v2:172.21.15.98:6818/4225498496,v1:172.21.15.98:6819/4225498496] exists,up 349d0120-8290-417d-a800-5a8985f554f0 2024-04-17T06:01:05.104 INFO:teuthology.orchestra.run.smithi047.stdout:osd.8 up in weight 1 up_from 60 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.103:6816/3433905520,v1:172.21.15.103:6817/3433905520] [v2:172.21.15.103:6818/3433905520,v1:172.21.15.103:6819/3433905520] exists,up cee4712f-321b-42c0-92ba-e15144c00f57 2024-04-17T06:01:05.104 INFO:teuthology.orchestra.run.smithi047.stdout:osd.9 up in weight 1 up_from 68 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.47:6826/728359554,v1:172.21.15.47:6827/728359554] [v2:172.21.15.47:6828/728359554,v1:172.21.15.47:6829/728359554] exists,up 63fc73c1-5a22-487d-80b8-9894e2ddd207 2024-04-17T06:01:05.104 INFO:teuthology.orchestra.run.smithi047.stdout:osd.10 up in weight 1 up_from 73 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.98:6824/468205689,v1:172.21.15.98:6825/468205689] [v2:172.21.15.98:6826/468205689,v1:172.21.15.98:6827/468205689] exists,up 9565bcc3-62ee-48eb-a07a-03a81e0cc8e5 2024-04-17T06:01:05.104 INFO:teuthology.orchestra.run.smithi047.stdout:osd.11 up in weight 1 up_from 81 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.103:6824/2639831013,v1:172.21.15.103:6825/2639831013] [v2:172.21.15.103:6826/2639831013,v1:172.21.15.103:6827/2639831013] exists,up 2e85e8d9-bf6c-4a12-b124-a4690dd61108 2024-04-17T06:01:05.105 INFO:teuthology.orchestra.run.smithi047.stdout:pg_upmap_items 2.8 [2,11] 2024-04-17T06:01:05.105 INFO:teuthology.orchestra.run.smithi047.stdout:pg_upmap_items 2.10 [8,11] 2024-04-17T06:01:05.105 INFO:teuthology.orchestra.run.smithi047.stdout:pg_upmap_items 2.14 [2,4] 2024-04-17T06:01:05.105 INFO:teuthology.orchestra.run.smithi047.stdout:pg_upmap_items 3.1 [4,9] 2024-04-17T06:01:05.105 INFO:teuthology.orchestra.run.smithi047.stdout:pg_upmap_items 3.b [3,9] 2024-04-17T06:01:05.105 INFO:teuthology.orchestra.run.smithi047.stdout:pg_upmap_items 3.10 [5,9] 2024-04-17T06:01:05.105 INFO:teuthology.orchestra.run.smithi047.stdout:pg_upmap_items 3.1d [4,6] 2024-04-17T06:01:05.105 INFO:teuthology.orchestra.run.smithi047.stdout:pg_upmap_items 3.23 [4,8] 2024-04-17T06:01:05.105 INFO:teuthology.orchestra.run.smithi047.stdout:blocklist 172.21.15.47:6801/3632157175 expires 2024-04-18T05:52:13.645775+0000 2024-04-17T06:01:05.105 INFO:teuthology.orchestra.run.smithi047.stdout:blocklist 172.21.15.47:0/4288219269 expires 2024-04-18T05:52:13.645775+0000 2024-04-17T06:01:05.105 INFO:teuthology.orchestra.run.smithi047.stdout:blocklist 172.21.15.47:0/2584487544 expires 2024-04-18T05:52:13.645775+0000 2024-04-17T06:01:05.105 INFO:teuthology.orchestra.run.smithi047.stdout:blocklist 172.21.15.47:6801/953064786 expires 2024-04-18T05:51:45.979351+0000 2024-04-17T06:01:05.105 INFO:teuthology.orchestra.run.smithi047.stdout:blocklist 172.21.15.47:0/3484500927 expires 2024-04-18T05:51:45.979351+0000 2024-04-17T06:01:05.105 INFO:teuthology.orchestra.run.smithi047.stdout:blocklist 172.21.15.47:6800/3632157175 expires 2024-04-18T05:52:13.645775+0000 2024-04-17T06:01:05.105 INFO:teuthology.orchestra.run.smithi047.stdout:blocklist 172.21.15.47:0/112268670 expires 2024-04-18T05:51:45.979351+0000 2024-04-17T06:01:05.105 INFO:teuthology.orchestra.run.smithi047.stdout:blocklist 172.21.15.47:0/1638423256 expires 2024-04-18T05:51:45.979351+0000 2024-04-17T06:01:05.105 INFO:teuthology.orchestra.run.smithi047.stdout:blocklist 172.21.15.47:6800/953064786 expires 2024-04-18T05:51:45.979351+0000 2024-04-17T06:01:05.106 INFO:teuthology.orchestra.run.smithi047.stdout:blocklist 172.21.15.47:0/849479010 expires 2024-04-18T05:52:13.645775+0000 2024-04-17T06:01:05.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:05.107+0000 7f2217376700 1 -- 172.21.15.47:0/4024504948 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f21fc0a3cc0 msgr2=0x7f21fc0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:05.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:05.107+0000 7f2217376700 1 --2- 172.21.15.47:0/4024504948 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f21fc0a3cc0 0x7f21fc0a6180 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2200000c00 tx=0x7f2200005fb0 comp rx=0 tx=0).stop 2024-04-17T06:01:05.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:05.107+0000 7f2217376700 1 -- 172.21.15.47:0/4024504948 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2210102ab0 msgr2=0x7f221019aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:05.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:05.107+0000 7f2217376700 1 --2- 172.21.15.47:0/4024504948 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2210102ab0 0x7f221019aee0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f220c00d9b0 tx=0x7f220c00dcc0 comp rx=0 tx=0).stop 2024-04-17T06:01:05.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:05.107+0000 7f2217376700 1 -- 172.21.15.47:0/4024504948 shutdown_connections 2024-04-17T06:01:05.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:05.107+0000 7f2217376700 1 --2- 172.21.15.47:0/4024504948 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f21fc0a3cc0 0x7f21fc0a6180 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:05.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:05.107+0000 7f2217376700 1 --2- 172.21.15.47:0/4024504948 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2210104650 0x7f22101a24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:05.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:05.107+0000 7f2217376700 1 --2- 172.21.15.47:0/4024504948 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2210102ab0 0x7f221019aee0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:05.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:05.107+0000 7f2217376700 1 --2- 172.21.15.47:0/4024504948 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2210103cb0 0x7f221019b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:05.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:05.107+0000 7f2217376700 1 -- 172.21.15.47:0/4024504948 >> 172.21.15.47:0/4024504948 conn(0x7f22100fe050 msgr2=0x7f2210074c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:01:05.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:05.107+0000 7f2217376700 1 -- 172.21.15.47:0/4024504948 shutdown_connections 2024-04-17T06:01:05.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:05.107+0000 7f2217376700 1 -- 172.21.15.47:0/4024504948 wait complete. 2024-04-17T06:01:05.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:04 smithi047 ceph-mon[32223]: pgmap v343: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-17T06:01:05.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:04 smithi103 ceph-mon[36243]: pgmap v343: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-17T06:01:05.382 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:04 smithi098 ceph-mon[36516]: pgmap v343: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-17T06:01:05.898 INFO:teuthology.run_tasks:Running task fs.ready... 2024-04-17T06:01:05.910 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T06:01:06.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.024+0000 7f8f4ccb1640 1 -- 172.21.15.47:0/881175995 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8f48103b70 msgr2=0x7f8f4810a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:06.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.024+0000 7f8f4ccb1640 1 --2- 172.21.15.47:0/881175995 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8f48103b70 0x7f8f4810a400 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f8f40009ca0 tx=0x7f8f4002c0f0 comp rx=0 tx=0).stop 2024-04-17T06:01:06.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.024+0000 7f8f4ccb1640 1 -- 172.21.15.47:0/881175995 shutdown_connections 2024-04-17T06:01:06.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.024+0000 7f8f4ccb1640 1 --2- 172.21.15.47:0/881175995 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8f48103b70 0x7f8f4810a400 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:06.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.024+0000 7f8f4ccb1640 1 --2- 172.21.15.47:0/881175995 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f481031b0 0x7f8f48103630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:06.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.024+0000 7f8f4ccb1640 1 --2- 172.21.15.47:0/881175995 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f48101fc0 0x7f8f481023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:06.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.024+0000 7f8f4ccb1640 1 -- 172.21.15.47:0/881175995 >> 172.21.15.47:0/881175995 conn(0x7f8f480fd510 msgr2=0x7f8f480ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:01:06.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.024+0000 7f8f4ccb1640 1 -- 172.21.15.47:0/881175995 shutdown_connections 2024-04-17T06:01:06.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.024+0000 7f8f4ccb1640 1 -- 172.21.15.47:0/881175995 wait complete. 2024-04-17T06:01:06.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.024+0000 7f8f4ccb1640 1 Processor -- start 2024-04-17T06:01:06.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.025+0000 7f8f4ccb1640 1 -- start start 2024-04-17T06:01:06.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.025+0000 7f8f4ccb1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f48101fc0 0x7f8f48075d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:06.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.025+0000 7f8f4ccb1640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8f481031b0 0x7f8f48074350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:06.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.025+0000 7f8f4ccb1640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f48103b70 0x7f8f48074890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:06.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.025+0000 7f8f4ccb1640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f8f4810dc10 con 0x7f8f481031b0 2024-04-17T06:01:06.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.025+0000 7f8f4ccb1640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8f4810da90 con 0x7f8f48103b70 2024-04-17T06:01:06.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.025+0000 7f8f4ccb1640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8f4810dd90 con 0x7f8f48101fc0 2024-04-17T06:01:06.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.025+0000 7f8f3dd9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8f481031b0 0x7f8f48074350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:06.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.025+0000 7f8f3dd9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8f481031b0 0x7f8f48074350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40278/0 (socket says 172.21.15.47:40278) 2024-04-17T06:01:06.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.025+0000 7f8f3dd9b640 1 -- 172.21.15.47:0/997581244 learned_addr learned my addr 172.21.15.47:0/997581244 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:01:06.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.025+0000 7f8f4659c640 1 --2- 172.21.15.47:0/997581244 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f48101fc0 0x7f8f48075d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:06.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.025+0000 7f8f46d9d640 1 --2- 172.21.15.47:0/997581244 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f48103b70 0x7f8f48074890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:06.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.025+0000 7f8f3dd9b640 1 -- 172.21.15.47:0/997581244 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f48101fc0 msgr2=0x7f8f48075d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:06.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.025+0000 7f8f3dd9b640 1 --2- 172.21.15.47:0/997581244 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f48101fc0 0x7f8f48075d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:06.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.025+0000 7f8f3dd9b640 1 -- 172.21.15.47:0/997581244 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f48103b70 msgr2=0x7f8f48074890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:06.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.025+0000 7f8f3dd9b640 1 --2- 172.21.15.47:0/997581244 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f48103b70 0x7f8f48074890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:06.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.025+0000 7f8f3dd9b640 1 -- 172.21.15.47:0/997581244 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f48075120 con 0x7f8f481031b0 2024-04-17T06:01:06.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.025+0000 7f8f3dd9b640 1 --2- 172.21.15.47:0/997581244 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8f481031b0 0x7f8f48074350 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f8f3000c890 tx=0x7f8f3000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:01:06.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.025+0000 7f8f3ffff640 1 -- 172.21.15.47:0/997581244 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f30004220 con 0x7f8f481031b0 2024-04-17T06:01:06.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.026+0000 7f8f4ccb1640 1 -- 172.21.15.47:0/997581244 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f481a2360 con 0x7f8f481031b0 2024-04-17T06:01:06.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.026+0000 7f8f3ffff640 1 -- 172.21.15.47:0/997581244 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f300043c0 con 0x7f8f481031b0 2024-04-17T06:01:06.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.026+0000 7f8f3ffff640 1 -- 172.21.15.47:0/997581244 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f30004a00 con 0x7f8f481031b0 2024-04-17T06:01:06.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.026+0000 7f8f4ccb1640 1 -- 172.21.15.47:0/997581244 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f481a2900 con 0x7f8f481031b0 2024-04-17T06:01:06.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.026+0000 7f8f4ccb1640 1 -- 172.21.15.47:0/997581244 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f48109210 con 0x7f8f481031b0 2024-04-17T06:01:06.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.028+0000 7f8f3ffff640 1 -- 172.21.15.47:0/997581244 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f8f300079b0 con 0x7f8f481031b0 2024-04-17T06:01:06.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.028+0000 7f8f3ffff640 1 --2- 172.21.15.47:0/997581244 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f8f340a3bb0 0x7f8f340a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:06.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.028+0000 7f8f3ffff640 1 -- 172.21.15.47:0/997581244 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f8f300cffe0 con 0x7f8f481031b0 2024-04-17T06:01:06.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.028+0000 7f8f4659c640 1 --2- 172.21.15.47:0/997581244 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f8f340a3bb0 0x7f8f340a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:06.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.029+0000 7f8f4659c640 1 --2- 172.21.15.47:0/997581244 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f8f340a3bb0 0x7f8f340a6070 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8f48103010 tx=0x7f8f38005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:01:06.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.033+0000 7f8f3ffff640 1 -- 172.21.15.47:0/997581244 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f8f30014030 con 0x7f8f481031b0 2024-04-17T06:01:06.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:05 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/4024504948' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-17T06:01:06.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.221+0000 7f8f4ccb1640 1 -- 172.21.15.47:0/997581244 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8f48062b60 con 0x7f8f481031b0 2024-04-17T06:01:06.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.223+0000 7f8f3ffff640 1 -- 172.21.15.47:0/997581244 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22602 (secure 0 0 0) 0x7f8f3009c380 con 0x7f8f481031b0 2024-04-17T06:01:06.221 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-17T06:01:06.221 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":88,"fsid":"565db2d8-fc7e-11ee-bc91-c7b262605968","created":"2024-04-17T05:51:19.888865+0000","modified":"2024-04-17T06:00:14.720609+0000","last_up_change":"2024-04-17T05:58:24.380131+0000","last_in_change":"2024-04-17T05:58:04.418236+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T05:54:37.734306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T05:59:12.458010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T05:59:12.901339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8453f300-533d-4736-a343-55619c08e966","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":1290127544},{"type":"v1","addr":"172.21.15.47:6803","nonce":1290127544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":1290127544},{"type":"v1","addr":"172.21.15.47:6805","nonce":1290127544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":1290127544},{"type":"v1","addr":"172.21.15.47:6809","nonce":1290127544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":1290127544},{"type":"v1","addr":"172.21.15.47:6807","nonce":1290127544}]},"public_addr":"172.21.15.47:6803/1290127544","cluster_addr":"172.21.15.47:6805/1290127544","heartbeat_back_addr":"172.21.15.47:6809/1290127544","heartbeat_front_addr":"172.21.15.47:6807/1290127544","state":["exists","up"]},{"osd":1,"uuid":"80b0f03e-c1bc-4658-8028-1e6b8eedcc47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":1180430954},{"type":"v1","addr":"172.21.15.98:6801","nonce":1180430954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":1180430954},{"type":"v1","addr":"172.21.15.98:6803","nonce":1180430954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":1180430954},{"type":"v1","addr":"172.21.15.98:6807","nonce":1180430954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":1180430954},{"type":"v1","addr":"172.21.15.98:6805","nonce":1180430954}]},"public_addr":"172.21.15.98:6801/1180430954","cluster_addr":"172.21.15.98:6803/1180430954","heartbeat_back_addr":"172.21.15.98:6807/1180430954","heartbeat_front_addr":"172.21.15.98:6805/1180430954","state":["exists","up"]},{"osd":2,"uuid":"0fc9d383-05b4-498d-b530-fccf7aeddd7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":494676216},{"type":"v1","addr":"172.21.15.103:6801","nonce":494676216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":494676216},{"type":"v1","addr":"172.21.15.103:6803","nonce":494676216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":494676216},{"type":"v1","addr":"172.21.15.103:6807","nonce":494676216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":494676216},{"type":"v1","addr":"172.21.15.103:6805","nonce":494676216}]},"public_addr":"172.21.15.103:6801/494676216","cluster_addr":"172.21.15.103:6803/494676216","heartbeat_back_addr":"172.21.15.103:6807/494676216","heartbeat_front_addr":"172.21.15.103:6805/494676216","state":["exists","up"]},{"osd":3,"uuid":"f6e0b0d5-0fb3-42d0-9e84-f7cfef21a998","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":1998369447},{"type":"v1","addr":"172.21.15.47:6811","nonce":1998369447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":1998369447},{"type":"v1","addr":"172.21.15.47:6813","nonce":1998369447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":1998369447},{"type":"v1","addr":"172.21.15.47:6817","nonce":1998369447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":1998369447},{"type":"v1","addr":"172.21.15.47:6815","nonce":1998369447}]},"public_addr":"172.21.15.47:6811/1998369447","cluster_addr":"172.21.15.47:6813/1998369447","heartbeat_back_addr":"172.21.15.47:6817/1998369447","heartbeat_front_addr":"172.21.15.47:6815/1998369447","state":["exists","up"]},{"osd":4,"uuid":"2c26217e-02df-466d-bc01-0149d4aec3c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":3060482485},{"type":"v1","addr":"172.21.15.98:6809","nonce":3060482485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":3060482485},{"type":"v1","addr":"172.21.15.98:6811","nonce":3060482485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":3060482485},{"type":"v1","addr":"172.21.15.98:6815","nonce":3060482485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":3060482485},{"type":"v1","addr":"172.21.15.98:6813","nonce":3060482485}]},"public_addr":"172.21.15.98:6809/3060482485","cluster_addr":"172.21.15.98:6811/3060482485","heartbeat_back_addr":"172.21.15.98:6815/3060482485","heartbeat_front_addr":"172.21.15.98:6813/3060482485","state":["exists","up"]},{"osd":5,"uuid":"740bf3e0-bea8-496b-8d6f-e42d0b017258","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":4181363567},{"type":"v1","addr":"172.21.15.103:6809","nonce":4181363567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":4181363567},{"type":"v1","addr":"172.21.15.103:6811","nonce":4181363567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":4181363567},{"type":"v1","addr":"172.21.15.103:6815","nonce":4181363567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":4181363567},{"type":"v1","addr":"172.21.15.103:6813","nonce":4181363567}]},"public_addr":"172.21.15.103:6809/4181363567","cluster_addr":"172.21.15.103:6811/4181363567","heartbeat_back_addr":"172.21.15.103:6815/4181363567","heartbeat_front_addr":"172.21.15.103:6813/4181363567","state":["exists","up"]},{"osd":6,"uuid":"356a6255-2c07-4f0d-a233-26be04c92b38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":2517529668},{"type":"v1","addr":"172.21.15.47:6819","nonce":2517529668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2517529668},{"type":"v1","addr":"172.21.15.47:6821","nonce":2517529668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2517529668},{"type":"v1","addr":"172.21.15.47:6825","nonce":2517529668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2517529668},{"type":"v1","addr":"172.21.15.47:6823","nonce":2517529668}]},"public_addr":"172.21.15.47:6819/2517529668","cluster_addr":"172.21.15.47:6821/2517529668","heartbeat_back_addr":"172.21.15.47:6825/2517529668","heartbeat_front_addr":"172.21.15.47:6823/2517529668","state":["exists","up"]},{"osd":7,"uuid":"349d0120-8290-417d-a800-5a8985f554f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":4225498496},{"type":"v1","addr":"172.21.15.98:6817","nonce":4225498496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":4225498496},{"type":"v1","addr":"172.21.15.98:6819","nonce":4225498496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":4225498496},{"type":"v1","addr":"172.21.15.98:6823","nonce":4225498496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":4225498496},{"type":"v1","addr":"172.21.15.98:6821","nonce":4225498496}]},"public_addr":"172.21.15.98:6817/4225498496","cluster_addr":"172.21.15.98:6819/4225498496","heartbeat_back_addr":"172.21.15.98:6823/4225498496","heartbeat_front_addr":"172.21.15.98:6821/4225498496","state":["exists","up"]},{"osd":8,"uuid":"cee4712f-321b-42c0-92ba-e15144c00f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":3433905520},{"type":"v1","addr":"172.21.15.103:6817","nonce":3433905520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":3433905520},{"type":"v1","addr":"172.21.15.103:6819","nonce":3433905520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":3433905520},{"type":"v1","addr":"172.21.15.103:6823","nonce":3433905520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":3433905520},{"type":"v1","addr":"172.21.15.103:6821","nonce":3433905520}]},"public_addr":"172.21.15.103:6817/3433905520","cluster_addr":"172.21.15.103:6819/3433905520","heartbeat_back_addr":"172.21.15.103:6823/3433905520","heartbeat_front_addr":"172.21.15.103:6821/3433905520","state":["exists","up"]},{"osd":9,"uuid":"63fc73c1-5a22-487d-80b8-9894e2ddd207","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":728359554},{"type":"v1","addr":"172.21.15.47:6827","nonce":728359554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":728359554},{"type":"v1","addr":"172.21.15.47:6829","nonce":728359554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":728359554},{"type":"v1","addr":"172.21.15.47:6833","nonce":728359554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":728359554},{"type":"v1","addr":"172.21.15.47:6831","nonce":728359554}]},"public_addr":"172.21.15.47:6827/728359554","cluster_addr":"172.21.15.47:6829/728359554","heartbeat_back_addr":"172.21.15.47:6833/728359554","heartbeat_front_addr":"172.21.15.47:6831/728359554","state":["exists","up"]},{"osd":10,"uuid":"9565bcc3-62ee-48eb-a07a-03a81e0cc8e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":468205689},{"type":"v1","addr":"172.21.15.98:6825","nonce":468205689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":468205689},{"type":"v1","addr":"172.21.15.98:6827","nonce":468205689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":468205689},{"type":"v1","addr":"172.21.15.98:6831","nonce":468205689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":468205689},{"type":"v1","addr":"172.21.15.98:6829","nonce":468205689}]},"public_addr":"172.21.15.98:6825/468205689","cluster_addr":"172.21.15.98:6827/468205689","heartbeat_back_addr":"172.21.15.98:6831/468205689","heartbeat_front_addr":"172.21.15.98:6829/468205689","state":["exists","up"]},{"osd":11,"uuid":"2e85e8d9-bf6c-4a12-b124-a4690dd61108","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":2639831013},{"type":"v1","addr":"172.21.15.103:6825","nonce":2639831013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":2639831013},{"type":"v1","addr":"172.21.15.103:6827","nonce":2639831013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":2639831013},{"type":"v1","addr":"172.21.15.103:6831","nonce":2639831013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":2639831013},{"type":"v1","addr":"172.21.15.103:6829","nonce":2639831013}]},"public_addr":"172.21.15.103:6825/2639831013","cluster_addr":"172.21.15.103:6827/2639831013","heartbeat_back_addr":"172.21.15.103:6831/2639831013","heartbeat_front_addr":"172.21.15.103:6829/2639831013","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:53:47.366438+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:54:10.953559+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:54:34.320332+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:54:59.751360+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:55:23.982052+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:55:48.425017+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:56:14.793368+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:56:40.440830+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:57:04.623680+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:57:31.215262+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:57:57.040346+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:58:22.671572+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:6801/3632157175":"2024-04-18T05:52:13.645775+0000","172.21.15.47:0/4288219269":"2024-04-18T05:52:13.645775+0000","172.21.15.47:0/2584487544":"2024-04-18T05:52:13.645775+0000","172.21.15.47:6801/953064786":"2024-04-18T05:51:45.979351+0000","172.21.15.47:0/3484500927":"2024-04-18T05:51:45.979351+0000","172.21.15.47:6800/3632157175":"2024-04-18T05:52:13.645775+0000","172.21.15.47:0/112268670":"2024-04-18T05:51:45.979351+0000","172.21.15.47:0/1638423256":"2024-04-18T05:51:45.979351+0000","172.21.15.47:6800/953064786":"2024-04-18T05:51:45.979351+0000","172.21.15.47:0/849479010":"2024-04-18T05:52:13.645775+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T06:01:06.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.225+0000 7f8f4ccb1640 1 -- 172.21.15.47:0/997581244 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f8f340a3bb0 msgr2=0x7f8f340a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:06.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.225+0000 7f8f4ccb1640 1 --2- 172.21.15.47:0/997581244 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f8f340a3bb0 0x7f8f340a6070 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8f48103010 tx=0x7f8f38005f90 comp rx=0 tx=0).stop 2024-04-17T06:01:06.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.225+0000 7f8f4ccb1640 1 -- 172.21.15.47:0/997581244 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8f481031b0 msgr2=0x7f8f48074350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:06.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.225+0000 7f8f4ccb1640 1 --2- 172.21.15.47:0/997581244 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8f481031b0 0x7f8f48074350 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f8f3000c890 tx=0x7f8f3000cd50 comp rx=0 tx=0).stop 2024-04-17T06:01:06.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.225+0000 7f8f4ccb1640 1 -- 172.21.15.47:0/997581244 shutdown_connections 2024-04-17T06:01:06.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.225+0000 7f8f4ccb1640 1 --2- 172.21.15.47:0/997581244 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f8f340a3bb0 0x7f8f340a6070 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:06.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.225+0000 7f8f4ccb1640 1 --2- 172.21.15.47:0/997581244 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f48103b70 0x7f8f48074890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:06.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.225+0000 7f8f4ccb1640 1 --2- 172.21.15.47:0/997581244 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8f481031b0 0x7f8f48074350 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:06.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.225+0000 7f8f4ccb1640 1 --2- 172.21.15.47:0/997581244 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f48101fc0 0x7f8f48075d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:06.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.225+0000 7f8f4ccb1640 1 -- 172.21.15.47:0/997581244 >> 172.21.15.47:0/997581244 conn(0x7f8f480fd510 msgr2=0x7f8f481063e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:01:06.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.225+0000 7f8f4ccb1640 1 -- 172.21.15.47:0/997581244 shutdown_connections 2024-04-17T06:01:06.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.225+0000 7f8f4ccb1640 1 -- 172.21.15.47:0/997581244 wait complete. 2024-04-17T06:01:06.235 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T05:54:37.734306+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T05:59:12.458010+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T05:59:12.901339+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T06:01:06.235 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T06:01:06.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:05 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/4024504948' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-17T06:01:06.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.346+0000 7f248bba8640 1 -- 172.21.15.47:0/2379313436 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2484102a90 msgr2=0x7f2484104e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:06.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.346+0000 7f248bba8640 1 --2- 172.21.15.47:0/2379313436 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2484102a90 0x7f2484104e80 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f2480009ca0 tx=0x7f248002c0f0 comp rx=0 tx=0).stop 2024-04-17T06:01:06.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.346+0000 7f248bba8640 1 -- 172.21.15.47:0/2379313436 shutdown_connections 2024-04-17T06:01:06.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.346+0000 7f248bba8640 1 --2- 172.21.15.47:0/2379313436 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2484107cf0 0x7f248410a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:06.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.346+0000 7f248bba8640 1 --2- 172.21.15.47:0/2379313436 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f24841053c0 0x7f24841077b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:06.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.346+0000 7f248bba8640 1 --2- 172.21.15.47:0/2379313436 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2484102a90 0x7f2484104e80 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:06.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.346+0000 7f248bba8640 1 -- 172.21.15.47:0/2379313436 >> 172.21.15.47:0/2379313436 conn(0x7f24840fc400 msgr2=0x7f24840fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:01:06.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.346+0000 7f248bba8640 1 -- 172.21.15.47:0/2379313436 shutdown_connections 2024-04-17T06:01:06.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.346+0000 7f248bba8640 1 -- 172.21.15.47:0/2379313436 wait complete. 2024-04-17T06:01:06.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.347+0000 7f248bba8640 1 Processor -- start 2024-04-17T06:01:06.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.347+0000 7f248bba8640 1 -- start start 2024-04-17T06:01:06.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.347+0000 7f248bba8640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2484102a90 0x7f24841982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:06.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.347+0000 7f248bba8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f24841053c0 0x7f2484198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:06.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.347+0000 7f248bba8640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2484107cf0 0x7f248419f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:06.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.347+0000 7f248bba8640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f248410dc20 con 0x7f2484102a90 2024-04-17T06:01:06.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.347+0000 7f248bba8640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f248410daa0 con 0x7f2484107cf0 2024-04-17T06:01:06.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.347+0000 7f248bba8640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f248410dda0 con 0x7f24841053c0 2024-04-17T06:01:06.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.347+0000 7f2489944640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2484102a90 0x7f24841982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:06.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.347+0000 7f2489944640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2484102a90 0x7f24841982f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40302/0 (socket says 172.21.15.47:40302) 2024-04-17T06:01:06.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.347+0000 7f2489944640 1 -- 172.21.15.47:0/4033231209 learned_addr learned my addr 172.21.15.47:0/4033231209 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:01:06.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.347+0000 7f2489143640 1 --2- 172.21.15.47:0/4033231209 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f24841053c0 0x7f2484198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:06.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.347+0000 7f248a145640 1 --2- 172.21.15.47:0/4033231209 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2484107cf0 0x7f248419f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:06.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.347+0000 7f2489944640 1 -- 172.21.15.47:0/4033231209 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f24841053c0 msgr2=0x7f2484198830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:06.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.347+0000 7f2489944640 1 --2- 172.21.15.47:0/4033231209 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f24841053c0 0x7f2484198830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:06.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.347+0000 7f2489944640 1 -- 172.21.15.47:0/4033231209 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2484107cf0 msgr2=0x7f248419f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:06.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.347+0000 7f2489944640 1 --2- 172.21.15.47:0/4033231209 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2484107cf0 0x7f248419f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:06.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.348+0000 7f2489944640 1 -- 172.21.15.47:0/4033231209 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f248419ff10 con 0x7f2484102a90 2024-04-17T06:01:06.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.348+0000 7f2489143640 1 --2- 172.21.15.47:0/4033231209 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f24841053c0 0x7f2484198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:01:06.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.348+0000 7f2489944640 1 --2- 172.21.15.47:0/4033231209 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2484102a90 0x7f24841982f0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f24800325f0 tx=0x7f2480032ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:01:06.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.348+0000 7f247affd640 1 -- 172.21.15.47:0/4033231209 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2480039040 con 0x7f2484102a90 2024-04-17T06:01:06.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.348+0000 7f247affd640 1 -- 172.21.15.47:0/4033231209 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2480044070 con 0x7f2484102a90 2024-04-17T06:01:06.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.348+0000 7f248bba8640 1 -- 172.21.15.47:0/4033231209 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24841a01a0 con 0x7f2484102a90 2024-04-17T06:01:06.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.348+0000 7f247affd640 1 -- 172.21.15.47:0/4033231209 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2480004280 con 0x7f2484102a90 2024-04-17T06:01:06.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.348+0000 7f248bba8640 1 -- 172.21.15.47:0/4033231209 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24841a06b0 con 0x7f2484102a90 2024-04-17T06:01:06.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.349+0000 7f248bba8640 1 -- 172.21.15.47:0/4033231209 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24840fdef0 con 0x7f2484102a90 2024-04-17T06:01:06.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.350+0000 7f247affd640 1 -- 172.21.15.47:0/4033231209 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f2480003970 con 0x7f2484102a90 2024-04-17T06:01:06.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.351+0000 7f247affd640 1 --2- 172.21.15.47:0/4033231209 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f24700a3c30 0x7f24700a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:06.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.351+0000 7f2489143640 1 --2- 172.21.15.47:0/4033231209 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f24700a3c30 0x7f24700a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:06.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.351+0000 7f247affd640 1 -- 172.21.15.47:0/4033231209 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f24800f2920 con 0x7f2484102a90 2024-04-17T06:01:06.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.351+0000 7f2489143640 1 --2- 172.21.15.47:0/4033231209 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f24700a3c30 0x7f24700a60f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f24740058c0 tx=0x7f2474005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:01:06.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.355+0000 7f247affd640 1 -- 172.21.15.47:0/4033231209 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f24800bf2b0 con 0x7f2484102a90 2024-04-17T06:01:06.382 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:05 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/4024504948' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-17T06:01:06.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.543+0000 7f248bba8640 1 -- 172.21.15.47:0/4033231209 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2484062b60 con 0x7f2484102a90 2024-04-17T06:01:06.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.543+0000 7f247affd640 1 -- 172.21.15.47:0/4033231209 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7f24800bebb0 con 0x7f2484102a90 2024-04-17T06:01:06.541 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-04-17T06:01:06.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.545+0000 7f248bba8640 1 -- 172.21.15.47:0/4033231209 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f24700a3c30 msgr2=0x7f24700a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:06.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.545+0000 7f248bba8640 1 --2- 172.21.15.47:0/4033231209 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f24700a3c30 0x7f24700a60f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f24740058c0 tx=0x7f2474005850 comp rx=0 tx=0).stop 2024-04-17T06:01:06.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.545+0000 7f248bba8640 1 -- 172.21.15.47:0/4033231209 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2484102a90 msgr2=0x7f24841982f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:06.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.545+0000 7f248bba8640 1 --2- 172.21.15.47:0/4033231209 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2484102a90 0x7f24841982f0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f24800325f0 tx=0x7f2480032ec0 comp rx=0 tx=0).stop 2024-04-17T06:01:06.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.545+0000 7f248bba8640 1 -- 172.21.15.47:0/4033231209 shutdown_connections 2024-04-17T06:01:06.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.545+0000 7f248bba8640 1 --2- 172.21.15.47:0/4033231209 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f24700a3c30 0x7f24700a60f0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:06.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.545+0000 7f248bba8640 1 --2- 172.21.15.47:0/4033231209 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2484107cf0 0x7f248419f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:06.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.545+0000 7f248bba8640 1 --2- 172.21.15.47:0/4033231209 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f24841053c0 0x7f2484198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:06.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.545+0000 7f248bba8640 1 --2- 172.21.15.47:0/4033231209 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2484102a90 0x7f24841982f0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:06.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.545+0000 7f248bba8640 1 -- 172.21.15.47:0/4033231209 >> 172.21.15.47:0/4033231209 conn(0x7f24840fc400 msgr2=0x7f24841034b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:01:06.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.545+0000 7f248bba8640 1 -- 172.21.15.47:0/4033231209 shutdown_connections 2024-04-17T06:01:06.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.545+0000 7f248bba8640 1 -- 172.21.15.47:0/4033231209 wait complete. 2024-04-17T06:01:06.555 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T06:01:06.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.667+0000 7fe2ab982640 1 -- 172.21.15.47:0/990068705 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe2a4105b30 msgr2=0x7fe2a4105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:06.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.667+0000 7fe2ab982640 1 --2- 172.21.15.47:0/990068705 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe2a4105b30 0x7fe2a4105fb0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fe29c009ca0 tx=0x7fe29c02c0f0 comp rx=0 tx=0).stop 2024-04-17T06:01:06.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.667+0000 7fe2ab982640 1 -- 172.21.15.47:0/990068705 shutdown_connections 2024-04-17T06:01:06.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.667+0000 7fe2ab982640 1 --2- 172.21.15.47:0/990068705 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe2a4105b30 0x7fe2a4105fb0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:06.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.667+0000 7fe2ab982640 1 --2- 172.21.15.47:0/990068705 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe2a4103180 0x7fe2a41055f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:06.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.667+0000 7fe2ab982640 1 --2- 172.21.15.47:0/990068705 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe2a4100810 0x7fe2a4102c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:06.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.667+0000 7fe2ab982640 1 -- 172.21.15.47:0/990068705 >> 172.21.15.47:0/990068705 conn(0x7fe2a40fc410 msgr2=0x7fe2a40fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:01:06.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.668+0000 7fe2ab982640 1 -- 172.21.15.47:0/990068705 shutdown_connections 2024-04-17T06:01:06.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.668+0000 7fe2ab982640 1 -- 172.21.15.47:0/990068705 wait complete. 2024-04-17T06:01:06.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.668+0000 7fe2ab982640 1 Processor -- start 2024-04-17T06:01:06.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.668+0000 7fe2ab982640 1 -- start start 2024-04-17T06:01:06.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.668+0000 7fe2ab982640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe2a4100810 0x7fe2a419e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:06.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.668+0000 7fe2ab982640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe2a4103180 0x7fe2a419eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:06.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.668+0000 7fe2ab982640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe2a4105b30 0x7fe2a41a5e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:06.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.668+0000 7fe2ab982640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fe2a41141f0 con 0x7fe2a4103180 2024-04-17T06:01:06.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.668+0000 7fe2ab982640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe2a4114070 con 0x7fe2a4105b30 2024-04-17T06:01:06.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.668+0000 7fe2ab982640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe2a4114370 con 0x7fe2a4100810 2024-04-17T06:01:06.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.668+0000 7fe2a8f1d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe2a4103180 0x7fe2a419eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:06.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.668+0000 7fe2a8f1d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe2a4103180 0x7fe2a419eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40316/0 (socket says 172.21.15.47:40316) 2024-04-17T06:01:06.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.668+0000 7fe2a8f1d640 1 -- 172.21.15.47:0/2273578948 learned_addr learned my addr 172.21.15.47:0/2273578948 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:01:06.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.669+0000 7fe2a971e640 1 --2- 172.21.15.47:0/2273578948 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe2a4100810 0x7fe2a419e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:06.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.669+0000 7fe2a8f1d640 1 -- 172.21.15.47:0/2273578948 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe2a4100810 msgr2=0x7fe2a419e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:06.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.669+0000 7fe2a8f1d640 1 --2- 172.21.15.47:0/2273578948 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe2a4100810 0x7fe2a419e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:06.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.669+0000 7fe2a8f1d640 1 -- 172.21.15.47:0/2273578948 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe2a4105b30 msgr2=0x7fe2a41a5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:06.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.669+0000 7fe2a9f1f640 1 --2- 172.21.15.47:0/2273578948 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe2a4105b30 0x7fe2a41a5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:06.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.669+0000 7fe2a8f1d640 1 --2- 172.21.15.47:0/2273578948 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe2a4105b30 0x7fe2a41a5e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:06.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.669+0000 7fe2a8f1d640 1 -- 172.21.15.47:0/2273578948 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2a41a6480 con 0x7fe2a4103180 2024-04-17T06:01:06.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.669+0000 7fe2a8f1d640 1 --2- 172.21.15.47:0/2273578948 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe2a4103180 0x7fe2a419eda0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fe29400c890 tx=0x7fe29400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:01:06.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.669+0000 7fe29a7fc640 1 -- 172.21.15.47:0/2273578948 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe294004220 con 0x7fe2a4103180 2024-04-17T06:01:06.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.669+0000 7fe2ab982640 1 -- 172.21.15.47:0/2273578948 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2a41a6770 con 0x7fe2a4103180 2024-04-17T06:01:06.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.669+0000 7fe29a7fc640 1 -- 172.21.15.47:0/2273578948 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2940043c0 con 0x7fe2a4103180 2024-04-17T06:01:06.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.669+0000 7fe29a7fc640 1 -- 172.21.15.47:0/2273578948 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe294004a00 con 0x7fe2a4103180 2024-04-17T06:01:06.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.669+0000 7fe2ab982640 1 -- 172.21.15.47:0/2273578948 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2a41a6d10 con 0x7fe2a4103180 2024-04-17T06:01:06.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.670+0000 7fe2ab982640 1 -- 172.21.15.47:0/2273578948 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2a40fdf00 con 0x7fe2a4103180 2024-04-17T06:01:06.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.671+0000 7fe29a7fc640 1 -- 172.21.15.47:0/2273578948 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fe2940079b0 con 0x7fe2a4103180 2024-04-17T06:01:06.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.672+0000 7fe29a7fc640 1 --2- 172.21.15.47:0/2273578948 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fe2900a3bb0 0x7fe2900a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:06.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.672+0000 7fe2a971e640 1 --2- 172.21.15.47:0/2273578948 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fe2900a3bb0 0x7fe2900a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:06.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.672+0000 7fe29a7fc640 1 -- 172.21.15.47:0/2273578948 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fe2940d0580 con 0x7fe2a4103180 2024-04-17T06:01:06.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.672+0000 7fe2a971e640 1 --2- 172.21.15.47:0/2273578948 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fe2900a3bb0 0x7fe2900a6070 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe2a0005d00 tx=0x7fe2a0005c50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:01:06.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.676+0000 7fe29a7fc640 1 -- 172.21.15.47:0/2273578948 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fe294014030 con 0x7fe2a4103180 2024-04-17T06:01:06.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.865+0000 7fe2ab982640 1 -- 172.21.15.47:0/2273578948 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe2a402d810 con 0x7fe2a4103180 2024-04-17T06:01:06.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.865+0000 7fe29a7fc640 1 -- 172.21.15.47:0/2273578948 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7fe29409cf10 con 0x7fe2a4103180 2024-04-17T06:01:06.863 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-04-17T06:01:06.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.867+0000 7fe2ab982640 1 -- 172.21.15.47:0/2273578948 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fe2900a3bb0 msgr2=0x7fe2900a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:06.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.867+0000 7fe2ab982640 1 --2- 172.21.15.47:0/2273578948 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fe2900a3bb0 0x7fe2900a6070 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe2a0005d00 tx=0x7fe2a0005c50 comp rx=0 tx=0).stop 2024-04-17T06:01:06.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.867+0000 7fe2ab982640 1 -- 172.21.15.47:0/2273578948 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe2a4103180 msgr2=0x7fe2a419eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:06.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.867+0000 7fe2ab982640 1 --2- 172.21.15.47:0/2273578948 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe2a4103180 0x7fe2a419eda0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fe29400c890 tx=0x7fe29400cd50 comp rx=0 tx=0).stop 2024-04-17T06:01:06.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.867+0000 7fe2ab982640 1 -- 172.21.15.47:0/2273578948 shutdown_connections 2024-04-17T06:01:06.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.867+0000 7fe2ab982640 1 --2- 172.21.15.47:0/2273578948 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fe2900a3bb0 0x7fe2900a6070 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:06.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.867+0000 7fe2ab982640 1 --2- 172.21.15.47:0/2273578948 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe2a4105b30 0x7fe2a41a5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:06.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.867+0000 7fe2ab982640 1 --2- 172.21.15.47:0/2273578948 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe2a4103180 0x7fe2a419eda0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:06.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.867+0000 7fe2ab982640 1 --2- 172.21.15.47:0/2273578948 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe2a4100810 0x7fe2a419e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:06.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.867+0000 7fe2ab982640 1 -- 172.21.15.47:0/2273578948 >> 172.21.15.47:0/2273578948 conn(0x7fe2a40fc410 msgr2=0x7fe2a4101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:01:06.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.867+0000 7fe2ab982640 1 -- 172.21.15.47:0/2273578948 shutdown_connections 2024-04-17T06:01:06.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.867+0000 7fe2ab982640 1 -- 172.21.15.47:0/2273578948 wait complete. 2024-04-17T06:01:06.876 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T06:01:06.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.987+0000 7f4ef58c5640 1 -- 172.21.15.47:0/2620509667 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ef0101ff0 msgr2=0x7f4ef0102470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:06.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.987+0000 7f4ef58c5640 1 --2- 172.21.15.47:0/2620509667 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ef0101ff0 0x7f4ef0102470 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f4ed8009900 tx=0x7f4ed802bc30 comp rx=0 tx=0).stop 2024-04-17T06:01:06.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.987+0000 7f4ef58c5640 1 -- 172.21.15.47:0/2620509667 shutdown_connections 2024-04-17T06:01:06.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.987+0000 7f4ef58c5640 1 --2- 172.21.15.47:0/2620509667 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4ef01029b0 0x7f4ef01092d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:06.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.987+0000 7f4ef58c5640 1 --2- 172.21.15.47:0/2620509667 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ef0101ff0 0x7f4ef0102470 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:06.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.988+0000 7f4ef58c5640 1 --2- 172.21.15.47:0/2620509667 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4ef0100ea0 0x7f4ef01012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:06.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.988+0000 7f4ef58c5640 1 -- 172.21.15.47:0/2620509667 >> 172.21.15.47:0/2620509667 conn(0x7f4ef00fc410 msgr2=0x7f4ef00fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:01:06.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.988+0000 7f4ef58c5640 1 -- 172.21.15.47:0/2620509667 shutdown_connections 2024-04-17T06:01:06.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.988+0000 7f4ef58c5640 1 -- 172.21.15.47:0/2620509667 wait complete. 2024-04-17T06:01:06.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.988+0000 7f4ef58c5640 1 Processor -- start 2024-04-17T06:01:06.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.988+0000 7f4ef58c5640 1 -- start start 2024-04-17T06:01:06.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.988+0000 7f4ef58c5640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4ef0100ea0 0x7f4ef0104f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:06.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.988+0000 7f4ef58c5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4ef0101ff0 0x7f4ef0103550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:06.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.988+0000 7f4ef58c5640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ef01029b0 0x7f4ef0103a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:06.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.988+0000 7f4ef58c5640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f4ef010cb30 con 0x7f4ef01029b0 2024-04-17T06:01:06.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.988+0000 7f4ef58c5640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f4ef010c9b0 con 0x7f4ef0100ea0 2024-04-17T06:01:06.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.988+0000 7f4ef58c5640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4ef010ccb0 con 0x7f4ef0101ff0 2024-04-17T06:01:06.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.989+0000 7f4eef7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ef01029b0 0x7f4ef0103a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:06.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.989+0000 7f4eef7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ef01029b0 0x7f4ef0103a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40328/0 (socket says 172.21.15.47:40328) 2024-04-17T06:01:06.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.989+0000 7f4eef7fe640 1 -- 172.21.15.47:0/4159838376 learned_addr learned my addr 172.21.15.47:0/4159838376 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:01:06.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.989+0000 7f4eeeffd640 1 --2- 172.21.15.47:0/4159838376 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4ef0100ea0 0x7f4ef0104f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:06.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.989+0000 7f4eee7fc640 1 --2- 172.21.15.47:0/4159838376 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4ef0101ff0 0x7f4ef0103550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:06.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.989+0000 7f4eef7fe640 1 -- 172.21.15.47:0/4159838376 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4ef0101ff0 msgr2=0x7f4ef0103550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:06.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.989+0000 7f4eef7fe640 1 --2- 172.21.15.47:0/4159838376 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4ef0101ff0 0x7f4ef0103550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:06.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.989+0000 7f4eef7fe640 1 -- 172.21.15.47:0/4159838376 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4ef0100ea0 msgr2=0x7f4ef0104f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:06.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.989+0000 7f4eef7fe640 1 --2- 172.21.15.47:0/4159838376 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4ef0100ea0 0x7f4ef0104f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:06.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.989+0000 7f4eef7fe640 1 -- 172.21.15.47:0/4159838376 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ef0104320 con 0x7f4ef01029b0 2024-04-17T06:01:06.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.989+0000 7f4eeeffd640 1 --2- 172.21.15.47:0/4159838376 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4ef0100ea0 0x7f4ef0104f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:01:06.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.989+0000 7f4eef7fe640 1 --2- 172.21.15.47:0/4159838376 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ef01029b0 0x7f4ef0103a90 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f4ee000d950 tx=0x7f4ee000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:01:06.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.989+0000 7f4ef48c3640 1 -- 172.21.15.47:0/4159838376 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee0017030 con 0x7f4ef01029b0 2024-04-17T06:01:06.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.989+0000 7f4ef48c3640 1 -- 172.21.15.47:0/4159838376 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ee0004020 con 0x7f4ef01029b0 2024-04-17T06:01:06.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.989+0000 7f4ef58c5640 1 -- 172.21.15.47:0/4159838376 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ef01a2440 con 0x7f4ef01029b0 2024-04-17T06:01:06.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.989+0000 7f4ef48c3640 1 -- 172.21.15.47:0/4159838376 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee0003c00 con 0x7f4ef01029b0 2024-04-17T06:01:06.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.989+0000 7f4ef58c5640 1 -- 172.21.15.47:0/4159838376 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ef01a2910 con 0x7f4ef01029b0 2024-04-17T06:01:06.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.990+0000 7f4ef58c5640 1 -- 172.21.15.47:0/4159838376 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ef0068d80 con 0x7f4ef01029b0 2024-04-17T06:01:06.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.991+0000 7f4ef48c3640 1 -- 172.21.15.47:0/4159838376 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f4ee001a070 con 0x7f4ef01029b0 2024-04-17T06:01:06.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.992+0000 7f4ef48c3640 1 --2- 172.21.15.47:0/4159838376 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f4edc0a3bb0 0x7f4edc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:06.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.992+0000 7f4eeeffd640 1 --2- 172.21.15.47:0/4159838376 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f4edc0a3bb0 0x7f4edc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:06.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.992+0000 7f4ef48c3640 1 -- 172.21.15.47:0/4159838376 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f4ee00cfd60 con 0x7f4ef01029b0 2024-04-17T06:01:06.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.992+0000 7f4eeeffd640 1 --2- 172.21.15.47:0/4159838376 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f4edc0a3bb0 0x7f4edc0a6070 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f4ee8005c40 tx=0x7f4ee800a430 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:01:06.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:06.996+0000 7f4ef48c3640 1 -- 172.21.15.47:0/4159838376 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f4ee0021070 con 0x7f4ef01029b0 2024-04-17T06:01:07.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:06 smithi047 ceph-mon[32223]: pgmap v344: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-17T06:01:07.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:06 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/997581244' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T06:01:07.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:06 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/4033231209' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-17T06:01:07.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:06 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/2273578948' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-17T06:01:07.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.182+0000 7f4ef58c5640 1 -- 172.21.15.47:0/4159838376 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f4ef01a2c30 con 0x7f4ef01029b0 2024-04-17T06:01:07.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.183+0000 7f4ef48c3640 1 -- 172.21.15.47:0/4159838376 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7f4ee00bc070 con 0x7f4ef01029b0 2024-04-17T06:01:07.181 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-04-17T06:01:07.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.184+0000 7f4ef58c5640 1 -- 172.21.15.47:0/4159838376 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f4edc0a3bb0 msgr2=0x7f4edc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:07.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.184+0000 7f4ef58c5640 1 --2- 172.21.15.47:0/4159838376 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f4edc0a3bb0 0x7f4edc0a6070 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f4ee8005c40 tx=0x7f4ee800a430 comp rx=0 tx=0).stop 2024-04-17T06:01:07.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.184+0000 7f4ef58c5640 1 -- 172.21.15.47:0/4159838376 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ef01029b0 msgr2=0x7f4ef0103a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:07.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.184+0000 7f4ef58c5640 1 --2- 172.21.15.47:0/4159838376 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ef01029b0 0x7f4ef0103a90 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f4ee000d950 tx=0x7f4ee000de10 comp rx=0 tx=0).stop 2024-04-17T06:01:07.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.185+0000 7f4ef58c5640 1 -- 172.21.15.47:0/4159838376 shutdown_connections 2024-04-17T06:01:07.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.185+0000 7f4ef58c5640 1 --2- 172.21.15.47:0/4159838376 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f4edc0a3bb0 0x7f4edc0a6070 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:07.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.185+0000 7f4ef58c5640 1 --2- 172.21.15.47:0/4159838376 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ef01029b0 0x7f4ef0103a90 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:07.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.185+0000 7f4ef58c5640 1 --2- 172.21.15.47:0/4159838376 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4ef0101ff0 0x7f4ef0103550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:07.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.185+0000 7f4ef58c5640 1 --2- 172.21.15.47:0/4159838376 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4ef0100ea0 0x7f4ef0104f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:07.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.185+0000 7f4ef58c5640 1 -- 172.21.15.47:0/4159838376 >> 172.21.15.47:0/4159838376 conn(0x7f4ef00fc410 msgr2=0x7f4ef00690f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:01:07.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.185+0000 7f4ef58c5640 1 -- 172.21.15.47:0/4159838376 shutdown_connections 2024-04-17T06:01:07.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.185+0000 7f4ef58c5640 1 -- 172.21.15.47:0/4159838376 wait complete. 2024-04-17T06:01:07.194 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T06:01:07.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:06 smithi103 ceph-mon[36243]: pgmap v344: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-17T06:01:07.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:06 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/997581244' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T06:01:07.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:06 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/4033231209' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-17T06:01:07.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:06 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/2273578948' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-17T06:01:07.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.302+0000 7efc3b8f8640 1 -- 172.21.15.47:0/2332087814 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efc34105360 msgr2=0x7efc34107750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:07.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.302+0000 7efc3b8f8640 1 --2- 172.21.15.47:0/2332087814 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efc34105360 0x7efc34107750 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7efc24009900 tx=0x7efc2402bc30 comp rx=0 tx=0).stop 2024-04-17T06:01:07.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.302+0000 7efc3b8f8640 1 -- 172.21.15.47:0/2332087814 shutdown_connections 2024-04-17T06:01:07.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.302+0000 7efc3b8f8640 1 --2- 172.21.15.47:0/2332087814 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efc34107c90 0x7efc3410a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:07.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.302+0000 7efc3b8f8640 1 --2- 172.21.15.47:0/2332087814 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efc34105360 0x7efc34107750 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:07.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.302+0000 7efc3b8f8640 1 --2- 172.21.15.47:0/2332087814 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efc34102a30 0x7efc34104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:07.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.302+0000 7efc3b8f8640 1 -- 172.21.15.47:0/2332087814 >> 172.21.15.47:0/2332087814 conn(0x7efc340fc440 msgr2=0x7efc340fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:01:07.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.302+0000 7efc3b8f8640 1 -- 172.21.15.47:0/2332087814 shutdown_connections 2024-04-17T06:01:07.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.302+0000 7efc3b8f8640 1 -- 172.21.15.47:0/2332087814 wait complete. 2024-04-17T06:01:07.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.303+0000 7efc3b8f8640 1 Processor -- start 2024-04-17T06:01:07.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.303+0000 7efc3b8f8640 1 -- start start 2024-04-17T06:01:07.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.303+0000 7efc3b8f8640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efc34102a30 0x7efc341023a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:07.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.303+0000 7efc3b8f8640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efc34105360 0x7efc341009e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:07.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.303+0000 7efc3b8f8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efc34107c90 0x7efc34100f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:07.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.303+0000 7efc3b8f8640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7efc3410dc40 con 0x7efc34105360 2024-04-17T06:01:07.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.303+0000 7efc3b8f8640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7efc3410dac0 con 0x7efc34102a30 2024-04-17T06:01:07.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.303+0000 7efc3b8f8640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7efc3410ddc0 con 0x7efc34107c90 2024-04-17T06:01:07.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.303+0000 7efc38e93640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efc34105360 0x7efc341009e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:07.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.303+0000 7efc38e93640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efc34105360 0x7efc341009e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40342/0 (socket says 172.21.15.47:40342) 2024-04-17T06:01:07.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.303+0000 7efc38e93640 1 -- 172.21.15.47:0/1135004166 learned_addr learned my addr 172.21.15.47:0/1135004166 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:01:07.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.303+0000 7efc39e95640 1 --2- 172.21.15.47:0/1135004166 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efc34107c90 0x7efc34100f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:07.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.303+0000 7efc39694640 1 --2- 172.21.15.47:0/1135004166 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efc34102a30 0x7efc341023a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:07.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.303+0000 7efc38e93640 1 -- 172.21.15.47:0/1135004166 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efc34107c90 msgr2=0x7efc34100f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:07.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.303+0000 7efc38e93640 1 --2- 172.21.15.47:0/1135004166 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efc34107c90 0x7efc34100f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:07.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.303+0000 7efc38e93640 1 -- 172.21.15.47:0/1135004166 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efc34102a30 msgr2=0x7efc341023a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:07.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.303+0000 7efc38e93640 1 --2- 172.21.15.47:0/1135004166 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efc34102a30 0x7efc341023a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:07.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.303+0000 7efc38e93640 1 -- 172.21.15.47:0/1135004166 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc341017b0 con 0x7efc34105360 2024-04-17T06:01:07.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.303+0000 7efc39e95640 1 --2- 172.21.15.47:0/1135004166 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efc34107c90 0x7efc34100f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:01:07.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.303+0000 7efc39694640 1 --2- 172.21.15.47:0/1135004166 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efc34102a30 0x7efc341023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:01:07.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.304+0000 7efc38e93640 1 --2- 172.21.15.47:0/1135004166 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efc34105360 0x7efc341009e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7efc24000c00 tx=0x7efc24004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:01:07.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.304+0000 7efc2a7fc640 1 -- 172.21.15.47:0/1135004166 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc24041070 con 0x7efc34105360 2024-04-17T06:01:07.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.304+0000 7efc2a7fc640 1 -- 172.21.15.47:0/1135004166 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc240054e0 con 0x7efc34105360 2024-04-17T06:01:07.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.304+0000 7efc3b8f8640 1 -- 172.21.15.47:0/1135004166 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc341a2390 con 0x7efc34105360 2024-04-17T06:01:07.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.304+0000 7efc2a7fc640 1 -- 172.21.15.47:0/1135004166 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc240346a0 con 0x7efc34105360 2024-04-17T06:01:07.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.304+0000 7efc3b8f8640 1 -- 172.21.15.47:0/1135004166 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc341a28e0 con 0x7efc34105360 2024-04-17T06:01:07.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.305+0000 7efc3b8f8640 1 -- 172.21.15.47:0/1135004166 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc340fdf30 con 0x7efc34105360 2024-04-17T06:01:07.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.306+0000 7efc2a7fc640 1 -- 172.21.15.47:0/1135004166 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7efc24044070 con 0x7efc34105360 2024-04-17T06:01:07.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.307+0000 7efc2a7fc640 1 --2- 172.21.15.47:0/1135004166 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7efc200a3bb0 0x7efc200a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:07.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.307+0000 7efc39694640 1 --2- 172.21.15.47:0/1135004166 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7efc200a3bb0 0x7efc200a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:07.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.307+0000 7efc2a7fc640 1 -- 172.21.15.47:0/1135004166 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7efc240f21f0 con 0x7efc34105360 2024-04-17T06:01:07.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.307+0000 7efc39694640 1 --2- 172.21.15.47:0/1135004166 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7efc200a3bb0 0x7efc200a6070 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7efc30005fd0 tx=0x7efc30005ea0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:01:07.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.311+0000 7efc2a7fc640 1 -- 172.21.15.47:0/1135004166 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7efc240beb80 con 0x7efc34105360 2024-04-17T06:01:07.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:06 smithi098 ceph-mon[36516]: pgmap v344: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-17T06:01:07.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:06 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/997581244' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T06:01:07.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:06 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/4033231209' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-17T06:01:07.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:06 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/2273578948' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-17T06:01:07.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.536+0000 7efc3b8f8640 1 -- 172.21.15.47:0/1135004166 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efc3402d5d0 con 0x7efc34105360 2024-04-17T06:01:07.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.536+0000 7efc2a7fc640 1 -- 172.21.15.47:0/1135004166 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11881 (secure 0 0 0) 0x7efc240be480 con 0x7efc34105360 2024-04-17T06:01:07.534 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-17T06:01:07.534 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3189958720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3189958720},{"type":"v1","addr":"172.21.15.47:6835","nonce":3189958720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14595,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/3143529973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":3143529973},{"type":"v1","addr":"172.21.15.47:6837","nonce":3143529973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14652,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/2375823701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":2375823701},{"type":"v1","addr":"172.21.15.47:6839","nonce":2375823701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14664,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6841/3334309157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":3334309157},{"type":"v1","addr":"172.21.15.47:6841","nonce":3334309157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/2398363754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":2398363754},{"type":"v1","addr":"172.21.15.103:6833","nonce":2398363754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/384473493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":384473493},{"type":"v1","addr":"172.21.15.98:6833","nonce":384473493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T05:59:13.922176+0000","modified":"2024-04-17T05:59:19.598261+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24518,"mds_1":24479,"mds_2":24466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6835/270543522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":270543522},{"type":"v1","addr":"172.21.15.103:6835","nonce":270543522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6837/116936024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":116936024},{"type":"v1","addr":"172.21.15.103:6837","nonce":116936024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6835/1966500278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":1966500278},{"type":"v1","addr":"172.21.15.98:6835","nonce":1966500278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.103:6839/1467157889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":1467157889},{"type":"v1","addr":"172.21.15.103:6839","nonce":1467157889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.98:6837/1117440719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1117440719},{"type":"v1","addr":"172.21.15.98:6837","nonce":1117440719}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6839/2841287917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":2841287917},{"type":"v1","addr":"172.21.15.98:6839","nonce":2841287917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24518,"qdb_cluster":[24479,24518,24466]},"id":1}]} 2024-04-17T06:01:07.535 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 21 2024-04-17T06:01:07.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.538+0000 7efc3b8f8640 1 -- 172.21.15.47:0/1135004166 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7efc200a3bb0 msgr2=0x7efc200a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:07.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.538+0000 7efc3b8f8640 1 --2- 172.21.15.47:0/1135004166 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7efc200a3bb0 0x7efc200a6070 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7efc30005fd0 tx=0x7efc30005ea0 comp rx=0 tx=0).stop 2024-04-17T06:01:07.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.538+0000 7efc3b8f8640 1 -- 172.21.15.47:0/1135004166 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efc34105360 msgr2=0x7efc341009e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:07.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.538+0000 7efc3b8f8640 1 --2- 172.21.15.47:0/1135004166 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efc34105360 0x7efc341009e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7efc24000c00 tx=0x7efc24004040 comp rx=0 tx=0).stop 2024-04-17T06:01:07.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.538+0000 7efc3b8f8640 1 -- 172.21.15.47:0/1135004166 shutdown_connections 2024-04-17T06:01:07.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.538+0000 7efc3b8f8640 1 --2- 172.21.15.47:0/1135004166 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7efc200a3bb0 0x7efc200a6070 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:07.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.538+0000 7efc3b8f8640 1 --2- 172.21.15.47:0/1135004166 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efc34107c90 0x7efc34100f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:07.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.538+0000 7efc3b8f8640 1 --2- 172.21.15.47:0/1135004166 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efc34105360 0x7efc341009e0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:07.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.538+0000 7efc3b8f8640 1 --2- 172.21.15.47:0/1135004166 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efc34102a30 0x7efc341023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:07.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.538+0000 7efc3b8f8640 1 -- 172.21.15.47:0/1135004166 >> 172.21.15.47:0/1135004166 conn(0x7efc340fc440 msgr2=0x7efc341032d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:01:07.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.538+0000 7efc3b8f8640 1 -- 172.21.15.47:0/1135004166 shutdown_connections 2024-04-17T06:01:07.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.538+0000 7efc3b8f8640 1 -- 172.21.15.47:0/1135004166 wait complete. 2024-04-17T06:01:07.548 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T06:01:07.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.655+0000 7fad3c8eb640 1 -- 172.21.15.47:0/2046605412 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad38101ff0 msgr2=0x7fad38102470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:07.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.655+0000 7fad3c8eb640 1 --2- 172.21.15.47:0/2046605412 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad38101ff0 0x7fad38102470 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fad2c009900 tx=0x7fad2c02bc30 comp rx=0 tx=0).stop 2024-04-17T06:01:07.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.656+0000 7fad3c8eb640 1 -- 172.21.15.47:0/2046605412 shutdown_connections 2024-04-17T06:01:07.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.656+0000 7fad3c8eb640 1 --2- 172.21.15.47:0/2046605412 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fad381029b0 0x7fad381092d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:07.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.656+0000 7fad3c8eb640 1 --2- 172.21.15.47:0/2046605412 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad38101ff0 0x7fad38102470 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:07.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.656+0000 7fad3c8eb640 1 --2- 172.21.15.47:0/2046605412 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad38100ea0 0x7fad381012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:07.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.656+0000 7fad3c8eb640 1 -- 172.21.15.47:0/2046605412 >> 172.21.15.47:0/2046605412 conn(0x7fad380fc410 msgr2=0x7fad380fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:01:07.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.656+0000 7fad3c8eb640 1 -- 172.21.15.47:0/2046605412 shutdown_connections 2024-04-17T06:01:07.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.656+0000 7fad3c8eb640 1 -- 172.21.15.47:0/2046605412 wait complete. 2024-04-17T06:01:07.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.656+0000 7fad3c8eb640 1 Processor -- start 2024-04-17T06:01:07.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.656+0000 7fad3c8eb640 1 -- start start 2024-04-17T06:01:07.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.657+0000 7fad3c8eb640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fad38100ea0 0x7fad381971d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:07.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.657+0000 7fad3c8eb640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad38101ff0 0x7fad38197710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:07.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.657+0000 7fad3c8eb640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad381029b0 0x7fad3819e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:07.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.657+0000 7fad3c8eb640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fad3810cb30 con 0x7fad38101ff0 2024-04-17T06:01:07.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.657+0000 7fad3c8eb640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fad3810c9b0 con 0x7fad381029b0 2024-04-17T06:01:07.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.657+0000 7fad3c8eb640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fad3810ccb0 con 0x7fad38100ea0 2024-04-17T06:01:07.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.657+0000 7fad35d9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad38101ff0 0x7fad38197710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:07.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.657+0000 7fad35d9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad38101ff0 0x7fad38197710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40372/0 (socket says 172.21.15.47:40372) 2024-04-17T06:01:07.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.657+0000 7fad35d9b640 1 -- 172.21.15.47:0/347643707 learned_addr learned my addr 172.21.15.47:0/347643707 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:01:07.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.657+0000 7fad3659c640 1 --2- 172.21.15.47:0/347643707 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fad38100ea0 0x7fad381971d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:07.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.657+0000 7fad36d9d640 1 --2- 172.21.15.47:0/347643707 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad381029b0 0x7fad3819e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:07.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.657+0000 7fad35d9b640 1 -- 172.21.15.47:0/347643707 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fad38100ea0 msgr2=0x7fad381971d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:07.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.657+0000 7fad35d9b640 1 --2- 172.21.15.47:0/347643707 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fad38100ea0 0x7fad381971d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:07.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.657+0000 7fad35d9b640 1 -- 172.21.15.47:0/347643707 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad381029b0 msgr2=0x7fad3819e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:07.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.657+0000 7fad35d9b640 1 --2- 172.21.15.47:0/347643707 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad381029b0 0x7fad3819e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:07.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.657+0000 7fad35d9b640 1 -- 172.21.15.47:0/347643707 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad3819edf0 con 0x7fad38101ff0 2024-04-17T06:01:07.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.657+0000 7fad3659c640 1 --2- 172.21.15.47:0/347643707 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fad38100ea0 0x7fad381971d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:01:07.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.657+0000 7fad36d9d640 1 --2- 172.21.15.47:0/347643707 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad381029b0 0x7fad3819e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:01:07.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.657+0000 7fad35d9b640 1 --2- 172.21.15.47:0/347643707 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad38101ff0 0x7fad38197710 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fad2c005ab0 tx=0x7fad2c005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:01:07.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.657+0000 7fad277fe640 1 -- 172.21.15.47:0/347643707 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad2c041070 con 0x7fad38101ff0 2024-04-17T06:01:07.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.657+0000 7fad3c8eb640 1 -- 172.21.15.47:0/347643707 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad3819f080 con 0x7fad38101ff0 2024-04-17T06:01:07.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.657+0000 7fad277fe640 1 -- 172.21.15.47:0/347643707 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad2c0054e0 con 0x7fad38101ff0 2024-04-17T06:01:07.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.657+0000 7fad277fe640 1 -- 172.21.15.47:0/347643707 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad2c034440 con 0x7fad38101ff0 2024-04-17T06:01:07.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.657+0000 7fad3c8eb640 1 -- 172.21.15.47:0/347643707 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad380690f0 con 0x7fad38101ff0 2024-04-17T06:01:07.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.658+0000 7fad3c8eb640 1 -- 172.21.15.47:0/347643707 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad38069b10 con 0x7fad38101ff0 2024-04-17T06:01:07.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.660+0000 7fad277fe640 1 -- 172.21.15.47:0/347643707 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fad2c044070 con 0x7fad38101ff0 2024-04-17T06:01:07.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.660+0000 7fad277fe640 1 --2- 172.21.15.47:0/347643707 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fad200a5dc0 0x7fad200a8280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:07.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.660+0000 7fad3659c640 1 --2- 172.21.15.47:0/347643707 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fad200a5dc0 0x7fad200a8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:07.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.660+0000 7fad277fe640 1 -- 172.21.15.47:0/347643707 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fad2c0f22e0 con 0x7fad38101ff0 2024-04-17T06:01:07.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.661+0000 7fad3659c640 1 --2- 172.21.15.47:0/347643707 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fad200a5dc0 0x7fad200a8280 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fad280058c0 tx=0x7fad28005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:01:07.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.664+0000 7fad277fe640 1 -- 172.21.15.47:0/347643707 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fad2c0becd0 con 0x7fad38101ff0 2024-04-17T06:01:07.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.852+0000 7fad3c8eb640 1 -- 172.21.15.47:0/347643707 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fad38107c40 con 0x7fad38101ff0 2024-04-17T06:01:07.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.854+0000 7fad277fe640 1 -- 172.21.15.47:0/347643707 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22602 (secure 0 0 0) 0x7fad2c0be5d0 con 0x7fad38101ff0 2024-04-17T06:01:07.852 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-17T06:01:07.852 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":88,"fsid":"565db2d8-fc7e-11ee-bc91-c7b262605968","created":"2024-04-17T05:51:19.888865+0000","modified":"2024-04-17T06:00:14.720609+0000","last_up_change":"2024-04-17T05:58:24.380131+0000","last_in_change":"2024-04-17T05:58:04.418236+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T05:54:37.734306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T05:59:12.458010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T05:59:12.901339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8453f300-533d-4736-a343-55619c08e966","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":1290127544},{"type":"v1","addr":"172.21.15.47:6803","nonce":1290127544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":1290127544},{"type":"v1","addr":"172.21.15.47:6805","nonce":1290127544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":1290127544},{"type":"v1","addr":"172.21.15.47:6809","nonce":1290127544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":1290127544},{"type":"v1","addr":"172.21.15.47:6807","nonce":1290127544}]},"public_addr":"172.21.15.47:6803/1290127544","cluster_addr":"172.21.15.47:6805/1290127544","heartbeat_back_addr":"172.21.15.47:6809/1290127544","heartbeat_front_addr":"172.21.15.47:6807/1290127544","state":["exists","up"]},{"osd":1,"uuid":"80b0f03e-c1bc-4658-8028-1e6b8eedcc47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":1180430954},{"type":"v1","addr":"172.21.15.98:6801","nonce":1180430954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":1180430954},{"type":"v1","addr":"172.21.15.98:6803","nonce":1180430954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":1180430954},{"type":"v1","addr":"172.21.15.98:6807","nonce":1180430954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":1180430954},{"type":"v1","addr":"172.21.15.98:6805","nonce":1180430954}]},"public_addr":"172.21.15.98:6801/1180430954","cluster_addr":"172.21.15.98:6803/1180430954","heartbeat_back_addr":"172.21.15.98:6807/1180430954","heartbeat_front_addr":"172.21.15.98:6805/1180430954","state":["exists","up"]},{"osd":2,"uuid":"0fc9d383-05b4-498d-b530-fccf7aeddd7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":494676216},{"type":"v1","addr":"172.21.15.103:6801","nonce":494676216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":494676216},{"type":"v1","addr":"172.21.15.103:6803","nonce":494676216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":494676216},{"type":"v1","addr":"172.21.15.103:6807","nonce":494676216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":494676216},{"type":"v1","addr":"172.21.15.103:6805","nonce":494676216}]},"public_addr":"172.21.15.103:6801/494676216","cluster_addr":"172.21.15.103:6803/494676216","heartbeat_back_addr":"172.21.15.103:6807/494676216","heartbeat_front_addr":"172.21.15.103:6805/494676216","state":["exists","up"]},{"osd":3,"uuid":"f6e0b0d5-0fb3-42d0-9e84-f7cfef21a998","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":1998369447},{"type":"v1","addr":"172.21.15.47:6811","nonce":1998369447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":1998369447},{"type":"v1","addr":"172.21.15.47:6813","nonce":1998369447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":1998369447},{"type":"v1","addr":"172.21.15.47:6817","nonce":1998369447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":1998369447},{"type":"v1","addr":"172.21.15.47:6815","nonce":1998369447}]},"public_addr":"172.21.15.47:6811/1998369447","cluster_addr":"172.21.15.47:6813/1998369447","heartbeat_back_addr":"172.21.15.47:6817/1998369447","heartbeat_front_addr":"172.21.15.47:6815/1998369447","state":["exists","up"]},{"osd":4,"uuid":"2c26217e-02df-466d-bc01-0149d4aec3c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":3060482485},{"type":"v1","addr":"172.21.15.98:6809","nonce":3060482485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":3060482485},{"type":"v1","addr":"172.21.15.98:6811","nonce":3060482485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":3060482485},{"type":"v1","addr":"172.21.15.98:6815","nonce":3060482485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":3060482485},{"type":"v1","addr":"172.21.15.98:6813","nonce":3060482485}]},"public_addr":"172.21.15.98:6809/3060482485","cluster_addr":"172.21.15.98:6811/3060482485","heartbeat_back_addr":"172.21.15.98:6815/3060482485","heartbeat_front_addr":"172.21.15.98:6813/3060482485","state":["exists","up"]},{"osd":5,"uuid":"740bf3e0-bea8-496b-8d6f-e42d0b017258","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":4181363567},{"type":"v1","addr":"172.21.15.103:6809","nonce":4181363567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":4181363567},{"type":"v1","addr":"172.21.15.103:6811","nonce":4181363567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":4181363567},{"type":"v1","addr":"172.21.15.103:6815","nonce":4181363567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":4181363567},{"type":"v1","addr":"172.21.15.103:6813","nonce":4181363567}]},"public_addr":"172.21.15.103:6809/4181363567","cluster_addr":"172.21.15.103:6811/4181363567","heartbeat_back_addr":"172.21.15.103:6815/4181363567","heartbeat_front_addr":"172.21.15.103:6813/4181363567","state":["exists","up"]},{"osd":6,"uuid":"356a6255-2c07-4f0d-a233-26be04c92b38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":2517529668},{"type":"v1","addr":"172.21.15.47:6819","nonce":2517529668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2517529668},{"type":"v1","addr":"172.21.15.47:6821","nonce":2517529668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2517529668},{"type":"v1","addr":"172.21.15.47:6825","nonce":2517529668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2517529668},{"type":"v1","addr":"172.21.15.47:6823","nonce":2517529668}]},"public_addr":"172.21.15.47:6819/2517529668","cluster_addr":"172.21.15.47:6821/2517529668","heartbeat_back_addr":"172.21.15.47:6825/2517529668","heartbeat_front_addr":"172.21.15.47:6823/2517529668","state":["exists","up"]},{"osd":7,"uuid":"349d0120-8290-417d-a800-5a8985f554f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":4225498496},{"type":"v1","addr":"172.21.15.98:6817","nonce":4225498496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":4225498496},{"type":"v1","addr":"172.21.15.98:6819","nonce":4225498496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":4225498496},{"type":"v1","addr":"172.21.15.98:6823","nonce":4225498496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":4225498496},{"type":"v1","addr":"172.21.15.98:6821","nonce":4225498496}]},"public_addr":"172.21.15.98:6817/4225498496","cluster_addr":"172.21.15.98:6819/4225498496","heartbeat_back_addr":"172.21.15.98:6823/4225498496","heartbeat_front_addr":"172.21.15.98:6821/4225498496","state":["exists","up"]},{"osd":8,"uuid":"cee4712f-321b-42c0-92ba-e15144c00f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":3433905520},{"type":"v1","addr":"172.21.15.103:6817","nonce":3433905520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":3433905520},{"type":"v1","addr":"172.21.15.103:6819","nonce":3433905520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":3433905520},{"type":"v1","addr":"172.21.15.103:6823","nonce":3433905520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":3433905520},{"type":"v1","addr":"172.21.15.103:6821","nonce":3433905520}]},"public_addr":"172.21.15.103:6817/3433905520","cluster_addr":"172.21.15.103:6819/3433905520","heartbeat_back_addr":"172.21.15.103:6823/3433905520","heartbeat_front_addr":"172.21.15.103:6821/3433905520","state":["exists","up"]},{"osd":9,"uuid":"63fc73c1-5a22-487d-80b8-9894e2ddd207","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":728359554},{"type":"v1","addr":"172.21.15.47:6827","nonce":728359554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":728359554},{"type":"v1","addr":"172.21.15.47:6829","nonce":728359554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":728359554},{"type":"v1","addr":"172.21.15.47:6833","nonce":728359554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":728359554},{"type":"v1","addr":"172.21.15.47:6831","nonce":728359554}]},"public_addr":"172.21.15.47:6827/728359554","cluster_addr":"172.21.15.47:6829/728359554","heartbeat_back_addr":"172.21.15.47:6833/728359554","heartbeat_front_addr":"172.21.15.47:6831/728359554","state":["exists","up"]},{"osd":10,"uuid":"9565bcc3-62ee-48eb-a07a-03a81e0cc8e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":468205689},{"type":"v1","addr":"172.21.15.98:6825","nonce":468205689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":468205689},{"type":"v1","addr":"172.21.15.98:6827","nonce":468205689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":468205689},{"type":"v1","addr":"172.21.15.98:6831","nonce":468205689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":468205689},{"type":"v1","addr":"172.21.15.98:6829","nonce":468205689}]},"public_addr":"172.21.15.98:6825/468205689","cluster_addr":"172.21.15.98:6827/468205689","heartbeat_back_addr":"172.21.15.98:6831/468205689","heartbeat_front_addr":"172.21.15.98:6829/468205689","state":["exists","up"]},{"osd":11,"uuid":"2e85e8d9-bf6c-4a12-b124-a4690dd61108","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":2639831013},{"type":"v1","addr":"172.21.15.103:6825","nonce":2639831013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":2639831013},{"type":"v1","addr":"172.21.15.103:6827","nonce":2639831013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":2639831013},{"type":"v1","addr":"172.21.15.103:6831","nonce":2639831013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":2639831013},{"type":"v1","addr":"172.21.15.103:6829","nonce":2639831013}]},"public_addr":"172.21.15.103:6825/2639831013","cluster_addr":"172.21.15.103:6827/2639831013","heartbeat_back_addr":"172.21.15.103:6831/2639831013","heartbeat_front_addr":"172.21.15.103:6829/2639831013","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:53:47.366438+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:54:10.953559+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:54:34.320332+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:54:59.751360+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:55:23.982052+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:55:48.425017+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:56:14.793368+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:56:40.440830+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:57:04.623680+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:57:31.215262+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:57:57.040346+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:58:22.671572+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:6801/3632157175":"2024-04-18T05:52:13.645775+0000","172.21.15.47:0/4288219269":"2024-04-18T05:52:13.645775+0000","172.21.15.47:0/2584487544":"2024-04-18T05:52:13.645775+0000","172.21.15.47:6801/953064786":"2024-04-18T05:51:45.979351+0000","172.21.15.47:0/3484500927":"2024-04-18T05:51:45.979351+0000","172.21.15.47:6800/3632157175":"2024-04-18T05:52:13.645775+0000","172.21.15.47:0/112268670":"2024-04-18T05:51:45.979351+0000","172.21.15.47:0/1638423256":"2024-04-18T05:51:45.979351+0000","172.21.15.47:6800/953064786":"2024-04-18T05:51:45.979351+0000","172.21.15.47:0/849479010":"2024-04-18T05:52:13.645775+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T06:01:07.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.856+0000 7fad3c8eb640 1 -- 172.21.15.47:0/347643707 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fad200a5dc0 msgr2=0x7fad200a8280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:07.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.856+0000 7fad3c8eb640 1 --2- 172.21.15.47:0/347643707 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fad200a5dc0 0x7fad200a8280 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fad280058c0 tx=0x7fad28005850 comp rx=0 tx=0).stop 2024-04-17T06:01:07.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.856+0000 7fad3c8eb640 1 -- 172.21.15.47:0/347643707 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad38101ff0 msgr2=0x7fad38197710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:07.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.856+0000 7fad3c8eb640 1 --2- 172.21.15.47:0/347643707 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad38101ff0 0x7fad38197710 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fad2c005ab0 tx=0x7fad2c005dc0 comp rx=0 tx=0).stop 2024-04-17T06:01:07.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.856+0000 7fad3c8eb640 1 -- 172.21.15.47:0/347643707 shutdown_connections 2024-04-17T06:01:07.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.856+0000 7fad3c8eb640 1 --2- 172.21.15.47:0/347643707 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fad200a5dc0 0x7fad200a8280 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:07.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.856+0000 7fad3c8eb640 1 --2- 172.21.15.47:0/347643707 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad381029b0 0x7fad3819e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:07.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.856+0000 7fad3c8eb640 1 --2- 172.21.15.47:0/347643707 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad38101ff0 0x7fad38197710 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:07.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.856+0000 7fad3c8eb640 1 --2- 172.21.15.47:0/347643707 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fad38100ea0 0x7fad381971d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:07.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.856+0000 7fad3c8eb640 1 -- 172.21.15.47:0/347643707 >> 172.21.15.47:0/347643707 conn(0x7fad380fc410 msgr2=0x7fad381053a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:01:07.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.856+0000 7fad3c8eb640 1 -- 172.21.15.47:0/347643707 shutdown_connections 2024-04-17T06:01:07.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.856+0000 7fad3c8eb640 1 -- 172.21.15.47:0/347643707 wait complete. 2024-04-17T06:01:07.865 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T05:54:37.734306+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T05:59:12.458010+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T05:59:12.901339+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T06:01:07.866 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T06:01:07.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.977+0000 7f452f0e4640 1 -- 172.21.15.47:0/1291442655 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4528101820 msgr2=0x7f4528101ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:07.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.977+0000 7f452f0e4640 1 --2- 172.21.15.47:0/1291442655 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4528101820 0x7f4528101ca0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f4518009ca0 tx=0x7f451802c0f0 comp rx=0 tx=0).stop 2024-04-17T06:01:07.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.977+0000 7f452f0e4640 1 -- 172.21.15.47:0/1291442655 shutdown_connections 2024-04-17T06:01:07.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.977+0000 7f452f0e4640 1 --2- 172.21.15.47:0/1291442655 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f45281021e0 0x7f4528108a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:07.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.977+0000 7f452f0e4640 1 --2- 172.21.15.47:0/1291442655 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4528101820 0x7f4528101ca0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:07.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.977+0000 7f452f0e4640 1 --2- 172.21.15.47:0/1291442655 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4528100630 0x7f4528100a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:07.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.977+0000 7f452f0e4640 1 -- 172.21.15.47:0/1291442655 >> 172.21.15.47:0/1291442655 conn(0x7f45280fbba0 msgr2=0x7f45280fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:01:07.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.977+0000 7f452f0e4640 1 -- 172.21.15.47:0/1291442655 shutdown_connections 2024-04-17T06:01:07.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.977+0000 7f452f0e4640 1 -- 172.21.15.47:0/1291442655 wait complete. 2024-04-17T06:01:07.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.978+0000 7f452f0e4640 1 Processor -- start 2024-04-17T06:01:07.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.978+0000 7f452f0e4640 1 -- start start 2024-04-17T06:01:07.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.978+0000 7f452f0e4640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4528100630 0x7f45281982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:07.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.978+0000 7f452f0e4640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4528101820 0x7f4528198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:07.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.978+0000 7f452f0e4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f45281021e0 0x7f452819f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:07.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.978+0000 7f452f0e4640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f452810cb40 con 0x7f4528101820 2024-04-17T06:01:07.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.978+0000 7f452f0e4640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f452810c9c0 con 0x7f4528100630 2024-04-17T06:01:07.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.978+0000 7f452f0e4640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f452810ccc0 con 0x7f45281021e0 2024-04-17T06:01:07.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.978+0000 7f4527fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4528101820 0x7f4528198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:07.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.978+0000 7f4527fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4528101820 0x7f4528198820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40390/0 (socket says 172.21.15.47:40390) 2024-04-17T06:01:07.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.978+0000 7f4527fff640 1 -- 172.21.15.47:0/2396646728 learned_addr learned my addr 172.21.15.47:0/2396646728 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:01:07.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.978+0000 7f452ce80640 1 --2- 172.21.15.47:0/2396646728 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4528100630 0x7f45281982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:07.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.978+0000 7f452d681640 1 --2- 172.21.15.47:0/2396646728 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f45281021e0 0x7f452819f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:07.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.978+0000 7f4527fff640 1 -- 172.21.15.47:0/2396646728 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f45281021e0 msgr2=0x7f452819f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:07.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.978+0000 7f4527fff640 1 --2- 172.21.15.47:0/2396646728 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f45281021e0 0x7f452819f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:07.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.978+0000 7f4527fff640 1 -- 172.21.15.47:0/2396646728 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4528100630 msgr2=0x7f45281982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:07.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.978+0000 7f4527fff640 1 --2- 172.21.15.47:0/2396646728 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4528100630 0x7f45281982e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:07.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.978+0000 7f4527fff640 1 -- 172.21.15.47:0/2396646728 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f452819ff00 con 0x7f4528101820 2024-04-17T06:01:07.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.978+0000 7f452ce80640 1 --2- 172.21.15.47:0/2396646728 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4528100630 0x7f45281982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:01:07.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.978+0000 7f452d681640 1 --2- 172.21.15.47:0/2396646728 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f45281021e0 0x7f452819f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:01:07.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.978+0000 7f4527fff640 1 --2- 172.21.15.47:0/2396646728 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4528101820 0x7f4528198820 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f4518009dc0 tx=0x7f45180093f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:01:07.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.979+0000 7f4525ffb640 1 -- 172.21.15.47:0/2396646728 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4518039040 con 0x7f4528101820 2024-04-17T06:01:07.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.979+0000 7f4525ffb640 1 -- 172.21.15.47:0/2396646728 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4518044070 con 0x7f4528101820 2024-04-17T06:01:07.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.979+0000 7f452f0e4640 1 -- 172.21.15.47:0/2396646728 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45281a0190 con 0x7f4528101820 2024-04-17T06:01:07.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.979+0000 7f4525ffb640 1 -- 172.21.15.47:0/2396646728 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4518004310 con 0x7f4528101820 2024-04-17T06:01:07.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.979+0000 7f452f0e4640 1 -- 172.21.15.47:0/2396646728 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45281a0680 con 0x7f4528101820 2024-04-17T06:01:07.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.980+0000 7f452f0e4640 1 -- 172.21.15.47:0/2396646728 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45281a08e0 con 0x7f4528101820 2024-04-17T06:01:07.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.981+0000 7f4525ffb640 1 -- 172.21.15.47:0/2396646728 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f4518003a00 con 0x7f4528101820 2024-04-17T06:01:07.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.981+0000 7f4525ffb640 1 --2- 172.21.15.47:0/2396646728 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f45140a3bb0 0x7f45140a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:07.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.982+0000 7f4525ffb640 1 -- 172.21.15.47:0/2396646728 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f45180f2900 con 0x7f4528101820 2024-04-17T06:01:07.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.982+0000 7f452ce80640 1 --2- 172.21.15.47:0/2396646728 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f45140a3bb0 0x7f45140a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:07.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.982+0000 7f452ce80640 1 --2- 172.21.15.47:0/2396646728 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f45140a3bb0 0x7f45140a6070 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f4528101680 tx=0x7f4510005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:01:07.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:07.986+0000 7f4525ffb640 1 -- 172.21.15.47:0/2396646728 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f45180bf320 con 0x7f4528101820 2024-04-17T06:01:08.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:07 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/4159838376' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-17T06:01:08.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:07 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/1135004166' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T06:01:08.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:07 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/347643707' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T06:01:08.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.170+0000 7f452f0e4640 1 -- 172.21.15.47:0/2396646728 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f45281a0ab0 con 0x7f4528101820 2024-04-17T06:01:08.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.171+0000 7f4525ffb640 1 -- 172.21.15.47:0/2396646728 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7f45180bec20 con 0x7f4528101820 2024-04-17T06:01:08.169 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-04-17T06:01:08.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.173+0000 7f452f0e4640 1 -- 172.21.15.47:0/2396646728 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f45140a3bb0 msgr2=0x7f45140a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:08.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.173+0000 7f452f0e4640 1 --2- 172.21.15.47:0/2396646728 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f45140a3bb0 0x7f45140a6070 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f4528101680 tx=0x7f4510005f90 comp rx=0 tx=0).stop 2024-04-17T06:01:08.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.173+0000 7f452f0e4640 1 -- 172.21.15.47:0/2396646728 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4528101820 msgr2=0x7f4528198820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:08.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.173+0000 7f452f0e4640 1 --2- 172.21.15.47:0/2396646728 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4528101820 0x7f4528198820 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f4518009dc0 tx=0x7f45180093f0 comp rx=0 tx=0).stop 2024-04-17T06:01:08.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.173+0000 7f452f0e4640 1 -- 172.21.15.47:0/2396646728 shutdown_connections 2024-04-17T06:01:08.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.173+0000 7f452f0e4640 1 --2- 172.21.15.47:0/2396646728 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f45140a3bb0 0x7f45140a6070 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:08.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.173+0000 7f452f0e4640 1 --2- 172.21.15.47:0/2396646728 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f45281021e0 0x7f452819f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:08.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.173+0000 7f452f0e4640 1 --2- 172.21.15.47:0/2396646728 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4528101820 0x7f4528198820 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:08.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.173+0000 7f452f0e4640 1 --2- 172.21.15.47:0/2396646728 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4528100630 0x7f45281982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:08.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.173+0000 7f452f0e4640 1 -- 172.21.15.47:0/2396646728 >> 172.21.15.47:0/2396646728 conn(0x7f45280fbba0 msgr2=0x7f45280fd6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:01:08.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.173+0000 7f452f0e4640 1 -- 172.21.15.47:0/2396646728 shutdown_connections 2024-04-17T06:01:08.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.173+0000 7f452f0e4640 1 -- 172.21.15.47:0/2396646728 wait complete. 2024-04-17T06:01:08.182 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T06:01:08.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:07 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/4159838376' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-17T06:01:08.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:07 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/1135004166' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T06:01:08.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:07 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/347643707' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T06:01:08.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.292+0000 7f2ec5888640 1 -- 172.21.15.47:0/2427708025 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2ec0101fb0 msgr2=0x7f2ec01023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:08.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.292+0000 7f2ec5888640 1 --2- 172.21.15.47:0/2427708025 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2ec0101fb0 0x7f2ec01023d0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f2eb8009ca0 tx=0x7f2eb802c0f0 comp rx=0 tx=0).stop 2024-04-17T06:01:08.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.293+0000 7f2ec5888640 1 -- 172.21.15.47:0/2427708025 shutdown_connections 2024-04-17T06:01:08.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.293+0000 7f2ec5888640 1 --2- 172.21.15.47:0/2427708025 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2ec0103b60 0x7f2ec010a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:08.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.293+0000 7f2ec5888640 1 --2- 172.21.15.47:0/2427708025 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2ec01031a0 0x7f2ec0103620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:08.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.293+0000 7f2ec5888640 1 --2- 172.21.15.47:0/2427708025 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2ec0101fb0 0x7f2ec01023d0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:08.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.293+0000 7f2ec5888640 1 -- 172.21.15.47:0/2427708025 >> 172.21.15.47:0/2427708025 conn(0x7f2ec00fd540 msgr2=0x7f2ec00ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:01:08.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.293+0000 7f2ec5888640 1 -- 172.21.15.47:0/2427708025 shutdown_connections 2024-04-17T06:01:08.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.293+0000 7f2ec5888640 1 -- 172.21.15.47:0/2427708025 wait complete. 2024-04-17T06:01:08.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.293+0000 7f2ec5888640 1 Processor -- start 2024-04-17T06:01:08.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.293+0000 7f2ec5888640 1 -- start start 2024-04-17T06:01:08.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.293+0000 7f2ec5888640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2ec0101fb0 0x7f2ec01982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:08.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.293+0000 7f2ec5888640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2ec01031a0 0x7f2ec0198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:08.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.294+0000 7f2ec5888640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2ec0103b60 0x7f2ec019f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:08.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.294+0000 7f2ec5888640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec010dc40 con 0x7f2ec01031a0 2024-04-17T06:01:08.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.294+0000 7f2ec5888640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec010dac0 con 0x7f2ec0101fb0 2024-04-17T06:01:08.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.294+0000 7f2ec5888640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec010ddc0 con 0x7f2ec0103b60 2024-04-17T06:01:08.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.294+0000 7f2ebe7fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2ec01031a0 0x7f2ec0198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:08.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.294+0000 7f2ebe7fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2ec01031a0 0x7f2ec0198830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40408/0 (socket says 172.21.15.47:40408) 2024-04-17T06:01:08.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.294+0000 7f2ebe7fc640 1 -- 172.21.15.47:0/244671915 learned_addr learned my addr 172.21.15.47:0/244671915 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:01:08.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.294+0000 7f2ebeffd640 1 --2- 172.21.15.47:0/244671915 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2ec0101fb0 0x7f2ec01982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:08.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.294+0000 7f2ebf7fe640 1 --2- 172.21.15.47:0/244671915 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2ec0103b60 0x7f2ec019f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:08.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.294+0000 7f2ebe7fc640 1 -- 172.21.15.47:0/244671915 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2ec0103b60 msgr2=0x7f2ec019f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:08.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.294+0000 7f2ebe7fc640 1 --2- 172.21.15.47:0/244671915 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2ec0103b60 0x7f2ec019f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:08.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.294+0000 7f2ebe7fc640 1 -- 172.21.15.47:0/244671915 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2ec0101fb0 msgr2=0x7f2ec01982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:08.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.294+0000 7f2ebe7fc640 1 --2- 172.21.15.47:0/244671915 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2ec0101fb0 0x7f2ec01982f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:08.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.294+0000 7f2ebe7fc640 1 -- 172.21.15.47:0/244671915 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ec019ff10 con 0x7f2ec01031a0 2024-04-17T06:01:08.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.294+0000 7f2ebe7fc640 1 --2- 172.21.15.47:0/244671915 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2ec01031a0 0x7f2ec0198830 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f2ea800d950 tx=0x7f2ea800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:01:08.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.294+0000 7f2ec4886640 1 -- 172.21.15.47:0/244671915 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ea8017070 con 0x7f2ec01031a0 2024-04-17T06:01:08.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.294+0000 7f2ec5888640 1 -- 172.21.15.47:0/244671915 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ec01a0200 con 0x7f2ec01031a0 2024-04-17T06:01:08.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.294+0000 7f2ec4886640 1 -- 172.21.15.47:0/244671915 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ea8009c30 con 0x7f2ec01031a0 2024-04-17T06:01:08.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.294+0000 7f2ec4886640 1 -- 172.21.15.47:0/244671915 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ea800f6b0 con 0x7f2ec01031a0 2024-04-17T06:01:08.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.294+0000 7f2ec5888640 1 -- 172.21.15.47:0/244671915 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ec01a07a0 con 0x7f2ec01031a0 2024-04-17T06:01:08.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.295+0000 7f2ec5888640 1 -- 172.21.15.47:0/244671915 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ec0109350 con 0x7f2ec01031a0 2024-04-17T06:01:08.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.296+0000 7f2ec4886640 1 -- 172.21.15.47:0/244671915 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f2ea80109d0 con 0x7f2ec01031a0 2024-04-17T06:01:08.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.297+0000 7f2ec4886640 1 --2- 172.21.15.47:0/244671915 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f2eac0a3bb0 0x7f2eac0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:08.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.297+0000 7f2ebeffd640 1 --2- 172.21.15.47:0/244671915 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f2eac0a3bb0 0x7f2eac0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:08.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.297+0000 7f2ec4886640 1 -- 172.21.15.47:0/244671915 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f2ea80cffd0 con 0x7f2ec01031a0 2024-04-17T06:01:08.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.298+0000 7f2ebeffd640 1 --2- 172.21.15.47:0/244671915 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f2eac0a3bb0 0x7f2eac0a6070 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f2eb802c700 tx=0x7f2eb8032590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:01:08.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.302+0000 7f2ec4886640 1 -- 172.21.15.47:0/244671915 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f2ea8009de0 con 0x7f2ec01031a0 2024-04-17T06:01:08.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:07 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/4159838376' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-17T06:01:08.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:07 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/1135004166' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T06:01:08.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:07 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/347643707' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T06:01:08.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.489+0000 7f2ec5888640 1 -- 172.21.15.47:0/244671915 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f2ec0062b60 con 0x7f2ec01031a0 2024-04-17T06:01:08.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.490+0000 7f2ec4886640 1 -- 172.21.15.47:0/244671915 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7f2ea809c310 con 0x7f2ec01031a0 2024-04-17T06:01:08.488 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-04-17T06:01:08.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.492+0000 7f2ec5888640 1 -- 172.21.15.47:0/244671915 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f2eac0a3bb0 msgr2=0x7f2eac0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:08.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.492+0000 7f2ec5888640 1 --2- 172.21.15.47:0/244671915 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f2eac0a3bb0 0x7f2eac0a6070 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f2eb802c700 tx=0x7f2eb8032590 comp rx=0 tx=0).stop 2024-04-17T06:01:08.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.492+0000 7f2ec5888640 1 -- 172.21.15.47:0/244671915 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2ec01031a0 msgr2=0x7f2ec0198830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:08.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.492+0000 7f2ec5888640 1 --2- 172.21.15.47:0/244671915 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2ec01031a0 0x7f2ec0198830 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f2ea800d950 tx=0x7f2ea800de10 comp rx=0 tx=0).stop 2024-04-17T06:01:08.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.492+0000 7f2ec5888640 1 -- 172.21.15.47:0/244671915 shutdown_connections 2024-04-17T06:01:08.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.492+0000 7f2ec5888640 1 --2- 172.21.15.47:0/244671915 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f2eac0a3bb0 0x7f2eac0a6070 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:08.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.492+0000 7f2ec5888640 1 --2- 172.21.15.47:0/244671915 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2ec0103b60 0x7f2ec019f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:08.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.492+0000 7f2ec5888640 1 --2- 172.21.15.47:0/244671915 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2ec01031a0 0x7f2ec0198830 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:08.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.492+0000 7f2ec5888640 1 --2- 172.21.15.47:0/244671915 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2ec0101fb0 0x7f2ec01982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:08.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.492+0000 7f2ec5888640 1 -- 172.21.15.47:0/244671915 >> 172.21.15.47:0/244671915 conn(0x7f2ec00fd540 msgr2=0x7f2ec0106520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:01:08.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.492+0000 7f2ec5888640 1 -- 172.21.15.47:0/244671915 shutdown_connections 2024-04-17T06:01:08.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.492+0000 7f2ec5888640 1 -- 172.21.15.47:0/244671915 wait complete. 2024-04-17T06:01:08.501 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T06:01:08.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.611+0000 7f4cc720c640 1 -- 172.21.15.47:0/2194904286 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4cc01053b0 msgr2=0x7f4cc01077a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:08.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.611+0000 7f4cc720c640 1 --2- 172.21.15.47:0/2194904286 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4cc01053b0 0x7f4cc01077a0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f4cac009900 tx=0x7f4cac02bc30 comp rx=0 tx=0).stop 2024-04-17T06:01:08.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.612+0000 7f4cc720c640 1 -- 172.21.15.47:0/2194904286 shutdown_connections 2024-04-17T06:01:08.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.612+0000 7f4cc720c640 1 --2- 172.21.15.47:0/2194904286 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4cc0107ce0 0x7f4cc010a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:08.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.612+0000 7f4cc720c640 1 --2- 172.21.15.47:0/2194904286 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4cc01053b0 0x7f4cc01077a0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:08.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.612+0000 7f4cc720c640 1 --2- 172.21.15.47:0/2194904286 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4cc0102a80 0x7f4cc0104e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:08.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.612+0000 7f4cc720c640 1 -- 172.21.15.47:0/2194904286 >> 172.21.15.47:0/2194904286 conn(0x7f4cc00fc410 msgr2=0x7f4cc00fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:01:08.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.612+0000 7f4cc720c640 1 -- 172.21.15.47:0/2194904286 shutdown_connections 2024-04-17T06:01:08.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.612+0000 7f4cc720c640 1 -- 172.21.15.47:0/2194904286 wait complete. 2024-04-17T06:01:08.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.612+0000 7f4cc720c640 1 Processor -- start 2024-04-17T06:01:08.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.612+0000 7f4cc720c640 1 -- start start 2024-04-17T06:01:08.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.612+0000 7f4cc720c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4cc0102a80 0x7f4cc0198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:08.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.612+0000 7f4cc720c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4cc01053b0 0x7f4cc0198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:08.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.612+0000 7f4cc720c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4cc0107ce0 0x7f4cc019f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:08.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.612+0000 7f4cc720c640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc010dc30 con 0x7f4cc0107ce0 2024-04-17T06:01:08.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.612+0000 7f4cc720c640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc010dab0 con 0x7f4cc0102a80 2024-04-17T06:01:08.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.612+0000 7f4cc720c640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc010ddb0 con 0x7f4cc01053b0 2024-04-17T06:01:08.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.612+0000 7f4cc57a9640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4cc0107ce0 0x7f4cc019f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:08.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.612+0000 7f4cc57a9640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4cc0107ce0 0x7f4cc019f8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40428/0 (socket says 172.21.15.47:40428) 2024-04-17T06:01:08.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.612+0000 7f4cc57a9640 1 -- 172.21.15.47:0/4173577339 learned_addr learned my addr 172.21.15.47:0/4173577339 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:01:08.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.613+0000 7f4cbffff640 1 --2- 172.21.15.47:0/4173577339 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4cc01053b0 0x7f4cc0198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:08.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.613+0000 7f4cc4fa8640 1 --2- 172.21.15.47:0/4173577339 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4cc0102a80 0x7f4cc0198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:08.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.613+0000 7f4cc57a9640 1 -- 172.21.15.47:0/4173577339 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4cc01053b0 msgr2=0x7f4cc0198880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:08.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.613+0000 7f4cc57a9640 1 --2- 172.21.15.47:0/4173577339 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4cc01053b0 0x7f4cc0198880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:08.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.613+0000 7f4cc57a9640 1 -- 172.21.15.47:0/4173577339 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4cc0102a80 msgr2=0x7f4cc0198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:08.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.613+0000 7f4cc57a9640 1 --2- 172.21.15.47:0/4173577339 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4cc0102a80 0x7f4cc0198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:08.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.613+0000 7f4cc57a9640 1 -- 172.21.15.47:0/4173577339 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cc019ff60 con 0x7f4cc0107ce0 2024-04-17T06:01:08.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.613+0000 7f4cc57a9640 1 --2- 172.21.15.47:0/4173577339 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4cc0107ce0 0x7f4cc019f8f0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f4cb800eb10 tx=0x7f4cb800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:01:08.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.613+0000 7f4cbdffb640 1 -- 172.21.15.47:0/4173577339 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cb8004b20 con 0x7f4cc0107ce0 2024-04-17T06:01:08.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.613+0000 7f4cbdffb640 1 -- 172.21.15.47:0/4173577339 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4cb8004cc0 con 0x7f4cc0107ce0 2024-04-17T06:01:08.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.613+0000 7f4cc720c640 1 -- 172.21.15.47:0/4173577339 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cc01a0250 con 0x7f4cc0107ce0 2024-04-17T06:01:08.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.613+0000 7f4cbdffb640 1 -- 172.21.15.47:0/4173577339 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cb8010760 con 0x7f4cc0107ce0 2024-04-17T06:01:08.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.613+0000 7f4cc720c640 1 -- 172.21.15.47:0/4173577339 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cc01a0720 con 0x7f4cc0107ce0 2024-04-17T06:01:08.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.614+0000 7f4cc720c640 1 -- 172.21.15.47:0/4173577339 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cc00fdf00 con 0x7f4cc0107ce0 2024-04-17T06:01:08.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.615+0000 7f4cbdffb640 1 -- 172.21.15.47:0/4173577339 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f4cb8005020 con 0x7f4cc0107ce0 2024-04-17T06:01:08.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.616+0000 7f4cbdffb640 1 --2- 172.21.15.47:0/4173577339 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f4cb00a3bb0 0x7f4cb00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:08.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.616+0000 7f4cbdffb640 1 -- 172.21.15.47:0/4173577339 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f4cb80d0ae0 con 0x7f4cc0107ce0 2024-04-17T06:01:08.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.616+0000 7f4cc4fa8640 1 --2- 172.21.15.47:0/4173577339 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f4cb00a3bb0 0x7f4cb00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:08.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.616+0000 7f4cc4fa8640 1 --2- 172.21.15.47:0/4173577339 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f4cb00a3bb0 0x7f4cb00a6070 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4cb4009b60 tx=0x7f4cb4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:01:08.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.621+0000 7f4cbdffb640 1 -- 172.21.15.47:0/4173577339 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f4cb809d4d0 con 0x7f4cc0107ce0 2024-04-17T06:01:08.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.807+0000 7f4cc720c640 1 -- 172.21.15.47:0/4173577339 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f4cc0062b60 con 0x7f4cc0107ce0 2024-04-17T06:01:08.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.807+0000 7f4cbdffb640 1 -- 172.21.15.47:0/4173577339 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7f4cb809cdd0 con 0x7f4cc0107ce0 2024-04-17T06:01:08.805 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-04-17T06:01:08.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.809+0000 7f4cc720c640 1 -- 172.21.15.47:0/4173577339 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f4cb00a3bb0 msgr2=0x7f4cb00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:08.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.809+0000 7f4cc720c640 1 --2- 172.21.15.47:0/4173577339 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f4cb00a3bb0 0x7f4cb00a6070 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4cb4009b60 tx=0x7f4cb4005f90 comp rx=0 tx=0).stop 2024-04-17T06:01:08.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.809+0000 7f4cc720c640 1 -- 172.21.15.47:0/4173577339 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4cc0107ce0 msgr2=0x7f4cc019f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:08.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.809+0000 7f4cc720c640 1 --2- 172.21.15.47:0/4173577339 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4cc0107ce0 0x7f4cc019f8f0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f4cb800eb10 tx=0x7f4cb800c2d0 comp rx=0 tx=0).stop 2024-04-17T06:01:08.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.809+0000 7f4cc720c640 1 -- 172.21.15.47:0/4173577339 shutdown_connections 2024-04-17T06:01:08.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.809+0000 7f4cc720c640 1 --2- 172.21.15.47:0/4173577339 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f4cb00a3bb0 0x7f4cb00a6070 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:08.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.809+0000 7f4cc720c640 1 --2- 172.21.15.47:0/4173577339 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4cc0107ce0 0x7f4cc019f8f0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:08.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.809+0000 7f4cc720c640 1 --2- 172.21.15.47:0/4173577339 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4cc01053b0 0x7f4cc0198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:08.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.809+0000 7f4cc720c640 1 --2- 172.21.15.47:0/4173577339 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4cc0102a80 0x7f4cc0198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:08.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.809+0000 7f4cc720c640 1 -- 172.21.15.47:0/4173577339 >> 172.21.15.47:0/4173577339 conn(0x7f4cc00fc410 msgr2=0x7f4cc0103600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:01:08.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.809+0000 7f4cc720c640 1 -- 172.21.15.47:0/4173577339 shutdown_connections 2024-04-17T06:01:08.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.809+0000 7f4cc720c640 1 -- 172.21.15.47:0/4173577339 wait complete. 2024-04-17T06:01:08.818 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T06:01:08.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.926+0000 7fae8cea5640 1 -- 172.21.15.47:0/191233232 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae88102a00 msgr2=0x7fae88109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:08.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.926+0000 7fae8cea5640 1 --2- 172.21.15.47:0/191233232 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae88102a00 0x7fae88109320 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fae80009ca0 tx=0x7fae8002c0f0 comp rx=0 tx=0).stop 2024-04-17T06:01:08.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.926+0000 7fae8cea5640 1 -- 172.21.15.47:0/191233232 shutdown_connections 2024-04-17T06:01:08.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.926+0000 7fae8cea5640 1 --2- 172.21.15.47:0/191233232 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae88102a00 0x7fae88109320 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:08.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.926+0000 7fae8cea5640 1 --2- 172.21.15.47:0/191233232 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fae88102040 0x7fae881024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:08.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.926+0000 7fae8cea5640 1 --2- 172.21.15.47:0/191233232 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae88100e60 0x7fae88101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:08.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.926+0000 7fae8cea5640 1 -- 172.21.15.47:0/191233232 >> 172.21.15.47:0/191233232 conn(0x7fae880fc440 msgr2=0x7fae880fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:01:08.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.926+0000 7fae8cea5640 1 -- 172.21.15.47:0/191233232 shutdown_connections 2024-04-17T06:01:08.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.926+0000 7fae8cea5640 1 -- 172.21.15.47:0/191233232 wait complete. 2024-04-17T06:01:08.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.927+0000 7fae8cea5640 1 Processor -- start 2024-04-17T06:01:08.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.927+0000 7fae8cea5640 1 -- start start 2024-04-17T06:01:08.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.927+0000 7fae8cea5640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae88100e60 0x7fae88198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:08.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.927+0000 7fae8cea5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fae88102040 0x7fae881987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:08.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.927+0000 7fae8cea5640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae88102a00 0x7fae8819f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:08.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.927+0000 7fae8cea5640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fae8810cb50 con 0x7fae88100e60 2024-04-17T06:01:08.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.927+0000 7fae8cea5640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fae8810c9d0 con 0x7fae88102a00 2024-04-17T06:01:08.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.927+0000 7fae8cea5640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fae8810ccd0 con 0x7fae88102040 2024-04-17T06:01:08.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.927+0000 7fae8659c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae88100e60 0x7fae88198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:08.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.927+0000 7fae8659c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae88100e60 0x7fae88198280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40442/0 (socket says 172.21.15.47:40442) 2024-04-17T06:01:08.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.927+0000 7fae8659c640 1 -- 172.21.15.47:0/2794484033 learned_addr learned my addr 172.21.15.47:0/2794484033 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:01:08.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.928+0000 7fae7ffff640 1 --2- 172.21.15.47:0/2794484033 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fae88102040 0x7fae881987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:08.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.928+0000 7fae86d9d640 1 --2- 172.21.15.47:0/2794484033 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae88102a00 0x7fae8819f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:08.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.928+0000 7fae8659c640 1 -- 172.21.15.47:0/2794484033 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fae88102040 msgr2=0x7fae881987c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:08.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.928+0000 7fae8659c640 1 --2- 172.21.15.47:0/2794484033 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fae88102040 0x7fae881987c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:08.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.928+0000 7fae8659c640 1 -- 172.21.15.47:0/2794484033 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae88102a00 msgr2=0x7fae8819f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:08.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.928+0000 7fae8659c640 1 --2- 172.21.15.47:0/2794484033 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae88102a00 0x7fae8819f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:08.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.928+0000 7fae8659c640 1 -- 172.21.15.47:0/2794484033 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae8819fea0 con 0x7fae88100e60 2024-04-17T06:01:08.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.928+0000 7fae86d9d640 1 --2- 172.21.15.47:0/2794484033 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae88102a00 0x7fae8819f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:01:08.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.928+0000 7fae8659c640 1 --2- 172.21.15.47:0/2794484033 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae88100e60 0x7fae88198280 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fae7800d770 tx=0x7fae7800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:01:08.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.928+0000 7fae7f7fe640 1 -- 172.21.15.47:0/2794484033 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae78017070 con 0x7fae88100e60 2024-04-17T06:01:08.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.928+0000 7fae7f7fe640 1 -- 172.21.15.47:0/2794484033 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae78009c30 con 0x7fae88100e60 2024-04-17T06:01:08.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.928+0000 7fae8cea5640 1 -- 172.21.15.47:0/2794484033 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae881a0190 con 0x7fae88100e60 2024-04-17T06:01:08.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.928+0000 7fae7f7fe640 1 -- 172.21.15.47:0/2794484033 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae78003dd0 con 0x7fae88100e60 2024-04-17T06:01:08.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.928+0000 7fae8cea5640 1 -- 172.21.15.47:0/2794484033 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae881a06e0 con 0x7fae88100e60 2024-04-17T06:01:08.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.929+0000 7fae8cea5640 1 -- 172.21.15.47:0/2794484033 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae881a0940 con 0x7fae88100e60 2024-04-17T06:01:08.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.930+0000 7fae7f7fe640 1 -- 172.21.15.47:0/2794484033 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fae78003fb0 con 0x7fae88100e60 2024-04-17T06:01:08.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.931+0000 7fae7f7fe640 1 --2- 172.21.15.47:0/2794484033 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fae740a3bb0 0x7fae740a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:08.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.931+0000 7fae7ffff640 1 --2- 172.21.15.47:0/2794484033 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fae740a3bb0 0x7fae740a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:08.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.931+0000 7fae7f7fe640 1 -- 172.21.15.47:0/2794484033 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fae780cfdc0 con 0x7fae88100e60 2024-04-17T06:01:08.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.931+0000 7fae7ffff640 1 --2- 172.21.15.47:0/2794484033 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fae740a3bb0 0x7fae740a6070 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fae88199790 tx=0x7fae7000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:01:08.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:08.936+0000 7fae7f7fe640 1 -- 172.21.15.47:0/2794484033 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fae7809c750 con 0x7fae88100e60 2024-04-17T06:01:09.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:08 smithi047 ceph-mon[32223]: pgmap v345: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-17T06:01:09.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:08 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/2396646728' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-17T06:01:09.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:08 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/244671915' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-17T06:01:09.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:08 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/4173577339' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-17T06:01:09.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.162+0000 7fae8cea5640 1 -- 172.21.15.47:0/2794484033 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fae881a10b0 con 0x7fae88100e60 2024-04-17T06:01:09.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.163+0000 7fae7f7fe640 1 -- 172.21.15.47:0/2794484033 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11881 (secure 0 0 0) 0x7fae78009de0 con 0x7fae88100e60 2024-04-17T06:01:09.161 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-17T06:01:09.161 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3189958720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3189958720},{"type":"v1","addr":"172.21.15.47:6835","nonce":3189958720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14595,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/3143529973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":3143529973},{"type":"v1","addr":"172.21.15.47:6837","nonce":3143529973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14652,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/2375823701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":2375823701},{"type":"v1","addr":"172.21.15.47:6839","nonce":2375823701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14664,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6841/3334309157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":3334309157},{"type":"v1","addr":"172.21.15.47:6841","nonce":3334309157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/2398363754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":2398363754},{"type":"v1","addr":"172.21.15.103:6833","nonce":2398363754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/384473493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":384473493},{"type":"v1","addr":"172.21.15.98:6833","nonce":384473493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T05:59:13.922176+0000","modified":"2024-04-17T05:59:19.598261+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24518,"mds_1":24479,"mds_2":24466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6835/270543522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":270543522},{"type":"v1","addr":"172.21.15.103:6835","nonce":270543522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6837/116936024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":116936024},{"type":"v1","addr":"172.21.15.103:6837","nonce":116936024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6835/1966500278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":1966500278},{"type":"v1","addr":"172.21.15.98:6835","nonce":1966500278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.103:6839/1467157889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":1467157889},{"type":"v1","addr":"172.21.15.103:6839","nonce":1467157889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.98:6837/1117440719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1117440719},{"type":"v1","addr":"172.21.15.98:6837","nonce":1117440719}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6839/2841287917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":2841287917},{"type":"v1","addr":"172.21.15.98:6839","nonce":2841287917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24518,"qdb_cluster":[24479,24518,24466]},"id":1}]} 2024-04-17T06:01:09.161 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 21 2024-04-17T06:01:09.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.165+0000 7fae8cea5640 1 -- 172.21.15.47:0/2794484033 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fae740a3bb0 msgr2=0x7fae740a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:09.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.165+0000 7fae8cea5640 1 --2- 172.21.15.47:0/2794484033 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fae740a3bb0 0x7fae740a6070 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fae88199790 tx=0x7fae7000b390 comp rx=0 tx=0).stop 2024-04-17T06:01:09.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.165+0000 7fae8cea5640 1 -- 172.21.15.47:0/2794484033 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae88100e60 msgr2=0x7fae88198280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:09.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.165+0000 7fae8cea5640 1 --2- 172.21.15.47:0/2794484033 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae88100e60 0x7fae88198280 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fae7800d770 tx=0x7fae7800dc30 comp rx=0 tx=0).stop 2024-04-17T06:01:09.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.165+0000 7fae8cea5640 1 -- 172.21.15.47:0/2794484033 shutdown_connections 2024-04-17T06:01:09.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.165+0000 7fae8cea5640 1 --2- 172.21.15.47:0/2794484033 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fae740a3bb0 0x7fae740a6070 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:09.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.165+0000 7fae8cea5640 1 --2- 172.21.15.47:0/2794484033 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae88102a00 0x7fae8819f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:09.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.165+0000 7fae8cea5640 1 --2- 172.21.15.47:0/2794484033 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fae88102040 0x7fae881987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:09.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.165+0000 7fae8cea5640 1 --2- 172.21.15.47:0/2794484033 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae88100e60 0x7fae88198280 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:09.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.165+0000 7fae8cea5640 1 -- 172.21.15.47:0/2794484033 >> 172.21.15.47:0/2794484033 conn(0x7fae880fc440 msgr2=0x7fae880fd910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:01:09.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.165+0000 7fae8cea5640 1 -- 172.21.15.47:0/2794484033 shutdown_connections 2024-04-17T06:01:09.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.165+0000 7fae8cea5640 1 -- 172.21.15.47:0/2794484033 wait complete. 2024-04-17T06:01:09.174 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T06:01:09.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.281+0000 7f8c487b5640 1 -- 172.21.15.47:0/3662739310 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8c401031b0 msgr2=0x7f8c40103630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:09.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.281+0000 7f8c487b5640 1 --2- 172.21.15.47:0/3662739310 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8c401031b0 0x7f8c40103630 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f8c34009900 tx=0x7f8c3402bc30 comp rx=0 tx=0).stop 2024-04-17T06:01:09.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.282+0000 7f8c487b5640 1 -- 172.21.15.47:0/3662739310 shutdown_connections 2024-04-17T06:01:09.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.282+0000 7f8c487b5640 1 --2- 172.21.15.47:0/3662739310 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8c40103b70 0x7f8c4010a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:09.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.282+0000 7f8c487b5640 1 --2- 172.21.15.47:0/3662739310 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8c401031b0 0x7f8c40103630 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:09.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.282+0000 7f8c487b5640 1 --2- 172.21.15.47:0/3662739310 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8c40101fc0 0x7f8c401023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:09.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.282+0000 7f8c487b5640 1 -- 172.21.15.47:0/3662739310 >> 172.21.15.47:0/3662739310 conn(0x7f8c400fd510 msgr2=0x7f8c400ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:01:09.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.282+0000 7f8c487b5640 1 -- 172.21.15.47:0/3662739310 shutdown_connections 2024-04-17T06:01:09.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.282+0000 7f8c487b5640 1 -- 172.21.15.47:0/3662739310 wait complete. 2024-04-17T06:01:09.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.282+0000 7f8c487b5640 1 Processor -- start 2024-04-17T06:01:09.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.282+0000 7f8c487b5640 1 -- start start 2024-04-17T06:01:09.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.282+0000 7f8c487b5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8c40101fc0 0x7f8c40198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:09.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.282+0000 7f8c487b5640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8c401031b0 0x7f8c40198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:09.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.282+0000 7f8c487b5640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8c40103b70 0x7f8c4019f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:09.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.282+0000 7f8c487b5640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f8c4010dc50 con 0x7f8c401031b0 2024-04-17T06:01:09.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.282+0000 7f8c487b5640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8c4010dad0 con 0x7f8c40103b70 2024-04-17T06:01:09.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.282+0000 7f8c487b5640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8c4010ddd0 con 0x7f8c40101fc0 2024-04-17T06:01:09.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.283+0000 7f8c45d50640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8c401031b0 0x7f8c40198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:09.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.283+0000 7f8c45d50640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8c401031b0 0x7f8c40198780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40472/0 (socket says 172.21.15.47:40472) 2024-04-17T06:01:09.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.283+0000 7f8c45d50640 1 -- 172.21.15.47:0/246711209 learned_addr learned my addr 172.21.15.47:0/246711209 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:01:09.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.283+0000 7f8c46551640 1 --2- 172.21.15.47:0/246711209 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8c40101fc0 0x7f8c40198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:09.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.283+0000 7f8c46d52640 1 --2- 172.21.15.47:0/246711209 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8c40103b70 0x7f8c4019f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:09.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.283+0000 7f8c45d50640 1 -- 172.21.15.47:0/246711209 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8c40101fc0 msgr2=0x7f8c40198240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:09.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.283+0000 7f8c45d50640 1 --2- 172.21.15.47:0/246711209 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8c40101fc0 0x7f8c40198240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:09.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.283+0000 7f8c45d50640 1 -- 172.21.15.47:0/246711209 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8c40103b70 msgr2=0x7f8c4019f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:09.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.283+0000 7f8c45d50640 1 --2- 172.21.15.47:0/246711209 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8c40103b70 0x7f8c4019f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:09.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.283+0000 7f8c45d50640 1 -- 172.21.15.47:0/246711209 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c4019fe60 con 0x7f8c401031b0 2024-04-17T06:01:09.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.283+0000 7f8c46d52640 1 --2- 172.21.15.47:0/246711209 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8c40103b70 0x7f8c4019f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:01:09.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.283+0000 7f8c45d50640 1 --2- 172.21.15.47:0/246711209 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8c401031b0 0x7f8c40198780 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f8c34009230 tx=0x7f8c34005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:01:09.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.283+0000 7f8c3f7fe640 1 -- 172.21.15.47:0/246711209 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c34041070 con 0x7f8c401031b0 2024-04-17T06:01:09.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.283+0000 7f8c3f7fe640 1 -- 172.21.15.47:0/246711209 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c340054e0 con 0x7f8c401031b0 2024-04-17T06:01:09.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.283+0000 7f8c487b5640 1 -- 172.21.15.47:0/246711209 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c401a00f0 con 0x7f8c401031b0 2024-04-17T06:01:09.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.283+0000 7f8c3f7fe640 1 -- 172.21.15.47:0/246711209 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c34034440 con 0x7f8c401031b0 2024-04-17T06:01:09.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.283+0000 7f8c487b5640 1 -- 172.21.15.47:0/246711209 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c401a0630 con 0x7f8c401031b0 2024-04-17T06:01:09.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.284+0000 7f8c487b5640 1 -- 172.21.15.47:0/246711209 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c401092e0 con 0x7f8c401031b0 2024-04-17T06:01:09.285 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:08 smithi103 ceph-mon[36243]: pgmap v345: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-17T06:01:09.285 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:08 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/2396646728' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-17T06:01:09.285 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:08 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/244671915' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-17T06:01:09.285 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:08 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/4173577339' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-17T06:01:09.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.285+0000 7f8c3f7fe640 1 -- 172.21.15.47:0/246711209 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f8c34044070 con 0x7f8c401031b0 2024-04-17T06:01:09.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.286+0000 7f8c3f7fe640 1 --2- 172.21.15.47:0/246711209 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f8c280a3c30 0x7f8c280a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:09.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.286+0000 7f8c46551640 1 --2- 172.21.15.47:0/246711209 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f8c280a3c30 0x7f8c280a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:09.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.286+0000 7f8c3f7fe640 1 -- 172.21.15.47:0/246711209 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f8c340f2090 con 0x7f8c401031b0 2024-04-17T06:01:09.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.286+0000 7f8c46551640 1 --2- 172.21.15.47:0/246711209 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f8c280a3c30 0x7f8c280a60f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f8c38005fd0 tx=0x7f8c3800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:01:09.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.291+0000 7f8c3f7fe640 1 -- 172.21.15.47:0/246711209 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f8c340bea80 con 0x7f8c401031b0 2024-04-17T06:01:09.382 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:08 smithi098 ceph-mon[36516]: pgmap v345: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-17T06:01:09.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:08 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/2396646728' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-17T06:01:09.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:08 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/244671915' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-17T06:01:09.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:08 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/4173577339' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-17T06:01:09.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.480+0000 7f8c487b5640 1 -- 172.21.15.47:0/246711209 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8c40062b60 con 0x7f8c401031b0 2024-04-17T06:01:09.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.482+0000 7f8c3f7fe640 1 -- 172.21.15.47:0/246711209 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22602 (secure 0 0 0) 0x7f8c340be380 con 0x7f8c401031b0 2024-04-17T06:01:09.480 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-17T06:01:09.480 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":88,"fsid":"565db2d8-fc7e-11ee-bc91-c7b262605968","created":"2024-04-17T05:51:19.888865+0000","modified":"2024-04-17T06:00:14.720609+0000","last_up_change":"2024-04-17T05:58:24.380131+0000","last_in_change":"2024-04-17T05:58:04.418236+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T05:54:37.734306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T05:59:12.458010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T05:59:12.901339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8453f300-533d-4736-a343-55619c08e966","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":1290127544},{"type":"v1","addr":"172.21.15.47:6803","nonce":1290127544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":1290127544},{"type":"v1","addr":"172.21.15.47:6805","nonce":1290127544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":1290127544},{"type":"v1","addr":"172.21.15.47:6809","nonce":1290127544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":1290127544},{"type":"v1","addr":"172.21.15.47:6807","nonce":1290127544}]},"public_addr":"172.21.15.47:6803/1290127544","cluster_addr":"172.21.15.47:6805/1290127544","heartbeat_back_addr":"172.21.15.47:6809/1290127544","heartbeat_front_addr":"172.21.15.47:6807/1290127544","state":["exists","up"]},{"osd":1,"uuid":"80b0f03e-c1bc-4658-8028-1e6b8eedcc47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":1180430954},{"type":"v1","addr":"172.21.15.98:6801","nonce":1180430954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":1180430954},{"type":"v1","addr":"172.21.15.98:6803","nonce":1180430954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":1180430954},{"type":"v1","addr":"172.21.15.98:6807","nonce":1180430954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":1180430954},{"type":"v1","addr":"172.21.15.98:6805","nonce":1180430954}]},"public_addr":"172.21.15.98:6801/1180430954","cluster_addr":"172.21.15.98:6803/1180430954","heartbeat_back_addr":"172.21.15.98:6807/1180430954","heartbeat_front_addr":"172.21.15.98:6805/1180430954","state":["exists","up"]},{"osd":2,"uuid":"0fc9d383-05b4-498d-b530-fccf7aeddd7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":494676216},{"type":"v1","addr":"172.21.15.103:6801","nonce":494676216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":494676216},{"type":"v1","addr":"172.21.15.103:6803","nonce":494676216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":494676216},{"type":"v1","addr":"172.21.15.103:6807","nonce":494676216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":494676216},{"type":"v1","addr":"172.21.15.103:6805","nonce":494676216}]},"public_addr":"172.21.15.103:6801/494676216","cluster_addr":"172.21.15.103:6803/494676216","heartbeat_back_addr":"172.21.15.103:6807/494676216","heartbeat_front_addr":"172.21.15.103:6805/494676216","state":["exists","up"]},{"osd":3,"uuid":"f6e0b0d5-0fb3-42d0-9e84-f7cfef21a998","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":1998369447},{"type":"v1","addr":"172.21.15.47:6811","nonce":1998369447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":1998369447},{"type":"v1","addr":"172.21.15.47:6813","nonce":1998369447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":1998369447},{"type":"v1","addr":"172.21.15.47:6817","nonce":1998369447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":1998369447},{"type":"v1","addr":"172.21.15.47:6815","nonce":1998369447}]},"public_addr":"172.21.15.47:6811/1998369447","cluster_addr":"172.21.15.47:6813/1998369447","heartbeat_back_addr":"172.21.15.47:6817/1998369447","heartbeat_front_addr":"172.21.15.47:6815/1998369447","state":["exists","up"]},{"osd":4,"uuid":"2c26217e-02df-466d-bc01-0149d4aec3c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":3060482485},{"type":"v1","addr":"172.21.15.98:6809","nonce":3060482485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":3060482485},{"type":"v1","addr":"172.21.15.98:6811","nonce":3060482485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":3060482485},{"type":"v1","addr":"172.21.15.98:6815","nonce":3060482485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":3060482485},{"type":"v1","addr":"172.21.15.98:6813","nonce":3060482485}]},"public_addr":"172.21.15.98:6809/3060482485","cluster_addr":"172.21.15.98:6811/3060482485","heartbeat_back_addr":"172.21.15.98:6815/3060482485","heartbeat_front_addr":"172.21.15.98:6813/3060482485","state":["exists","up"]},{"osd":5,"uuid":"740bf3e0-bea8-496b-8d6f-e42d0b017258","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":4181363567},{"type":"v1","addr":"172.21.15.103:6809","nonce":4181363567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":4181363567},{"type":"v1","addr":"172.21.15.103:6811","nonce":4181363567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":4181363567},{"type":"v1","addr":"172.21.15.103:6815","nonce":4181363567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":4181363567},{"type":"v1","addr":"172.21.15.103:6813","nonce":4181363567}]},"public_addr":"172.21.15.103:6809/4181363567","cluster_addr":"172.21.15.103:6811/4181363567","heartbeat_back_addr":"172.21.15.103:6815/4181363567","heartbeat_front_addr":"172.21.15.103:6813/4181363567","state":["exists","up"]},{"osd":6,"uuid":"356a6255-2c07-4f0d-a233-26be04c92b38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":2517529668},{"type":"v1","addr":"172.21.15.47:6819","nonce":2517529668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2517529668},{"type":"v1","addr":"172.21.15.47:6821","nonce":2517529668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2517529668},{"type":"v1","addr":"172.21.15.47:6825","nonce":2517529668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2517529668},{"type":"v1","addr":"172.21.15.47:6823","nonce":2517529668}]},"public_addr":"172.21.15.47:6819/2517529668","cluster_addr":"172.21.15.47:6821/2517529668","heartbeat_back_addr":"172.21.15.47:6825/2517529668","heartbeat_front_addr":"172.21.15.47:6823/2517529668","state":["exists","up"]},{"osd":7,"uuid":"349d0120-8290-417d-a800-5a8985f554f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":4225498496},{"type":"v1","addr":"172.21.15.98:6817","nonce":4225498496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":4225498496},{"type":"v1","addr":"172.21.15.98:6819","nonce":4225498496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":4225498496},{"type":"v1","addr":"172.21.15.98:6823","nonce":4225498496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":4225498496},{"type":"v1","addr":"172.21.15.98:6821","nonce":4225498496}]},"public_addr":"172.21.15.98:6817/4225498496","cluster_addr":"172.21.15.98:6819/4225498496","heartbeat_back_addr":"172.21.15.98:6823/4225498496","heartbeat_front_addr":"172.21.15.98:6821/4225498496","state":["exists","up"]},{"osd":8,"uuid":"cee4712f-321b-42c0-92ba-e15144c00f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":3433905520},{"type":"v1","addr":"172.21.15.103:6817","nonce":3433905520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":3433905520},{"type":"v1","addr":"172.21.15.103:6819","nonce":3433905520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":3433905520},{"type":"v1","addr":"172.21.15.103:6823","nonce":3433905520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":3433905520},{"type":"v1","addr":"172.21.15.103:6821","nonce":3433905520}]},"public_addr":"172.21.15.103:6817/3433905520","cluster_addr":"172.21.15.103:6819/3433905520","heartbeat_back_addr":"172.21.15.103:6823/3433905520","heartbeat_front_addr":"172.21.15.103:6821/3433905520","state":["exists","up"]},{"osd":9,"uuid":"63fc73c1-5a22-487d-80b8-9894e2ddd207","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":728359554},{"type":"v1","addr":"172.21.15.47:6827","nonce":728359554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":728359554},{"type":"v1","addr":"172.21.15.47:6829","nonce":728359554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":728359554},{"type":"v1","addr":"172.21.15.47:6833","nonce":728359554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":728359554},{"type":"v1","addr":"172.21.15.47:6831","nonce":728359554}]},"public_addr":"172.21.15.47:6827/728359554","cluster_addr":"172.21.15.47:6829/728359554","heartbeat_back_addr":"172.21.15.47:6833/728359554","heartbeat_front_addr":"172.21.15.47:6831/728359554","state":["exists","up"]},{"osd":10,"uuid":"9565bcc3-62ee-48eb-a07a-03a81e0cc8e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":468205689},{"type":"v1","addr":"172.21.15.98:6825","nonce":468205689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":468205689},{"type":"v1","addr":"172.21.15.98:6827","nonce":468205689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":468205689},{"type":"v1","addr":"172.21.15.98:6831","nonce":468205689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":468205689},{"type":"v1","addr":"172.21.15.98:6829","nonce":468205689}]},"public_addr":"172.21.15.98:6825/468205689","cluster_addr":"172.21.15.98:6827/468205689","heartbeat_back_addr":"172.21.15.98:6831/468205689","heartbeat_front_addr":"172.21.15.98:6829/468205689","state":["exists","up"]},{"osd":11,"uuid":"2e85e8d9-bf6c-4a12-b124-a4690dd61108","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":2639831013},{"type":"v1","addr":"172.21.15.103:6825","nonce":2639831013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":2639831013},{"type":"v1","addr":"172.21.15.103:6827","nonce":2639831013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":2639831013},{"type":"v1","addr":"172.21.15.103:6831","nonce":2639831013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":2639831013},{"type":"v1","addr":"172.21.15.103:6829","nonce":2639831013}]},"public_addr":"172.21.15.103:6825/2639831013","cluster_addr":"172.21.15.103:6827/2639831013","heartbeat_back_addr":"172.21.15.103:6831/2639831013","heartbeat_front_addr":"172.21.15.103:6829/2639831013","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:53:47.366438+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:54:10.953559+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:54:34.320332+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:54:59.751360+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:55:23.982052+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:55:48.425017+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:56:14.793368+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:56:40.440830+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:57:04.623680+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:57:31.215262+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:57:57.040346+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:58:22.671572+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:6801/3632157175":"2024-04-18T05:52:13.645775+0000","172.21.15.47:0/4288219269":"2024-04-18T05:52:13.645775+0000","172.21.15.47:0/2584487544":"2024-04-18T05:52:13.645775+0000","172.21.15.47:6801/953064786":"2024-04-18T05:51:45.979351+0000","172.21.15.47:0/3484500927":"2024-04-18T05:51:45.979351+0000","172.21.15.47:6800/3632157175":"2024-04-18T05:52:13.645775+0000","172.21.15.47:0/112268670":"2024-04-18T05:51:45.979351+0000","172.21.15.47:0/1638423256":"2024-04-18T05:51:45.979351+0000","172.21.15.47:6800/953064786":"2024-04-18T05:51:45.979351+0000","172.21.15.47:0/849479010":"2024-04-18T05:52:13.645775+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T06:01:09.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.484+0000 7f8c487b5640 1 -- 172.21.15.47:0/246711209 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f8c280a3c30 msgr2=0x7f8c280a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:09.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.484+0000 7f8c487b5640 1 --2- 172.21.15.47:0/246711209 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f8c280a3c30 0x7f8c280a60f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f8c38005fd0 tx=0x7f8c3800b390 comp rx=0 tx=0).stop 2024-04-17T06:01:09.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.484+0000 7f8c487b5640 1 -- 172.21.15.47:0/246711209 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8c401031b0 msgr2=0x7f8c40198780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:09.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.484+0000 7f8c487b5640 1 --2- 172.21.15.47:0/246711209 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8c401031b0 0x7f8c40198780 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f8c34009230 tx=0x7f8c34005dd0 comp rx=0 tx=0).stop 2024-04-17T06:01:09.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.484+0000 7f8c487b5640 1 -- 172.21.15.47:0/246711209 shutdown_connections 2024-04-17T06:01:09.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.484+0000 7f8c487b5640 1 --2- 172.21.15.47:0/246711209 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f8c280a3c30 0x7f8c280a60f0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:09.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.484+0000 7f8c487b5640 1 --2- 172.21.15.47:0/246711209 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8c40103b70 0x7f8c4019f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:09.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.484+0000 7f8c487b5640 1 --2- 172.21.15.47:0/246711209 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8c401031b0 0x7f8c40198780 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:09.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.484+0000 7f8c487b5640 1 --2- 172.21.15.47:0/246711209 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8c40101fc0 0x7f8c40198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:09.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.484+0000 7f8c487b5640 1 -- 172.21.15.47:0/246711209 >> 172.21.15.47:0/246711209 conn(0x7f8c400fd510 msgr2=0x7f8c401064b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:01:09.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.484+0000 7f8c487b5640 1 -- 172.21.15.47:0/246711209 shutdown_connections 2024-04-17T06:01:09.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.484+0000 7f8c487b5640 1 -- 172.21.15.47:0/246711209 wait complete. 2024-04-17T06:01:09.493 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 21, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T05:59:13.922176+0000', 'modified': '2024-04-17T05:59:19.598261+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'snap_rstat': True, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24518, 'mds_1': 24479, 'mds_2': 24466}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24436': {'gid': 24436, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.103:6835/270543522', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6834', 'nonce': 270543522}, {'type': 'v1', 'addr': '172.21.15.103:6835', 'nonce': 270543522}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24448': {'gid': 24448, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.103:6837/116936024', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6836', 'nonce': 116936024}, {'type': 'v1', 'addr': '172.21.15.103:6837', 'nonce': 116936024}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24449': {'gid': 24449, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.98:6835/1966500278', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6834', 'nonce': 1966500278}, {'type': 'v1', 'addr': '172.21.15.98:6835', 'nonce': 1966500278}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24466': {'gid': 24466, 'name': 'i', 'rank': 2, 'incarnation': 20, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.103:6839/1467157889', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6838', 'nonce': 1467157889}, {'type': 'v1', 'addr': '172.21.15.103:6839', 'nonce': 1467157889}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24479': {'gid': 24479, 'name': 'h', 'rank': 1, 'incarnation': 18, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.98:6837/1117440719', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6836', 'nonce': 1117440719}, {'type': 'v1', 'addr': '172.21.15.98:6837', 'nonce': 1117440719}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24518': {'gid': 24518, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.98:6839/2841287917', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6838', 'nonce': 2841287917}, {'type': 'v1', 'addr': '172.21.15.98:6839', 'nonce': 2841287917}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24518, 'qdb_cluster': [24479, 24518, 24466]} 2024-04-17T06:01:09.493 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-04-17T06:01:09.494 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-17T06:01:09.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.603+0000 7fbcfd3db640 1 -- 172.21.15.47:0/2866253758 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbcf8101fa0 msgr2=0x7fbcf81023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:09.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.603+0000 7fbcfd3db640 1 --2- 172.21.15.47:0/2866253758 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbcf8101fa0 0x7fbcf81023c0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fbce8009900 tx=0x7fbce802bc30 comp rx=0 tx=0).stop 2024-04-17T06:01:09.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.603+0000 7fbcfd3db640 1 -- 172.21.15.47:0/2866253758 shutdown_connections 2024-04-17T06:01:09.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.603+0000 7fbcfd3db640 1 --2- 172.21.15.47:0/2866253758 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbcf8103b50 0x7fbcf810a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:09.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.603+0000 7fbcfd3db640 1 --2- 172.21.15.47:0/2866253758 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbcf8103190 0x7fbcf8103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:09.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.603+0000 7fbcfd3db640 1 --2- 172.21.15.47:0/2866253758 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbcf8101fa0 0x7fbcf81023c0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:09.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.603+0000 7fbcfd3db640 1 -- 172.21.15.47:0/2866253758 >> 172.21.15.47:0/2866253758 conn(0x7fbcf80fd510 msgr2=0x7fbcf80ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:01:09.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.603+0000 7fbcfd3db640 1 -- 172.21.15.47:0/2866253758 shutdown_connections 2024-04-17T06:01:09.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.603+0000 7fbcfd3db640 1 -- 172.21.15.47:0/2866253758 wait complete. 2024-04-17T06:01:09.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.603+0000 7fbcfd3db640 1 Processor -- start 2024-04-17T06:01:09.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.603+0000 7fbcfd3db640 1 -- start start 2024-04-17T06:01:09.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.604+0000 7fbcfd3db640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbcf8101fa0 0x7fbcf81982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:09.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.604+0000 7fbcfd3db640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbcf8103190 0x7fbcf8198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:09.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.604+0000 7fbcfd3db640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbcf8103b50 0x7fbcf819f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:09.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.604+0000 7fbcfd3db640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf810dc70 con 0x7fbcf8101fa0 2024-04-17T06:01:09.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.604+0000 7fbcfd3db640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf810daf0 con 0x7fbcf8103b50 2024-04-17T06:01:09.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.604+0000 7fbcfd3db640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf810ddf0 con 0x7fbcf8103190 2024-04-17T06:01:09.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.604+0000 7fbcf6ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbcf8101fa0 0x7fbcf81982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:09.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.604+0000 7fbcf6ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbcf8101fa0 0x7fbcf81982f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40502/0 (socket says 172.21.15.47:40502) 2024-04-17T06:01:09.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.604+0000 7fbcf6ffd640 1 -- 172.21.15.47:0/3806807467 learned_addr learned my addr 172.21.15.47:0/3806807467 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:01:09.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.604+0000 7fbcf77fe640 1 --2- 172.21.15.47:0/3806807467 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbcf8103b50 0x7fbcf819f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:09.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.604+0000 7fbcf67fc640 1 --2- 172.21.15.47:0/3806807467 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbcf8103190 0x7fbcf8198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:09.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.604+0000 7fbcf6ffd640 1 -- 172.21.15.47:0/3806807467 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbcf8103190 msgr2=0x7fbcf8198830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:09.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.604+0000 7fbcf6ffd640 1 --2- 172.21.15.47:0/3806807467 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbcf8103190 0x7fbcf8198830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:09.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.604+0000 7fbcf6ffd640 1 -- 172.21.15.47:0/3806807467 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbcf8103b50 msgr2=0x7fbcf819f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:09.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.604+0000 7fbcf6ffd640 1 --2- 172.21.15.47:0/3806807467 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbcf8103b50 0x7fbcf819f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:09.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.604+0000 7fbcf6ffd640 1 -- 172.21.15.47:0/3806807467 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcf819ff10 con 0x7fbcf8101fa0 2024-04-17T06:01:09.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.604+0000 7fbcf67fc640 1 --2- 172.21.15.47:0/3806807467 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbcf8103190 0x7fbcf8198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:01:09.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.604+0000 7fbcf6ffd640 1 --2- 172.21.15.47:0/3806807467 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbcf8101fa0 0x7fbcf81982f0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fbce8000c00 tx=0x7fbce8004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:01:09.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.604+0000 7fbcdffff640 1 -- 172.21.15.47:0/3806807467 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbce8041070 con 0x7fbcf8101fa0 2024-04-17T06:01:09.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.604+0000 7fbcdffff640 1 -- 172.21.15.47:0/3806807467 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbce80054e0 con 0x7fbcf8101fa0 2024-04-17T06:01:09.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.604+0000 7fbcfd3db640 1 -- 172.21.15.47:0/3806807467 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcf81a01a0 con 0x7fbcf8101fa0 2024-04-17T06:01:09.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.604+0000 7fbcdffff640 1 -- 172.21.15.47:0/3806807467 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbce80346a0 con 0x7fbcf8101fa0 2024-04-17T06:01:09.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.604+0000 7fbcfd3db640 1 -- 172.21.15.47:0/3806807467 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcf81a0690 con 0x7fbcf8101fa0 2024-04-17T06:01:09.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.607+0000 7fbcdffff640 1 -- 172.21.15.47:0/3806807467 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fbce8044070 con 0x7fbcf8101fa0 2024-04-17T06:01:09.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.607+0000 7fbcfffca740 1 Processor -- start 2024-04-17T06:01:09.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.607+0000 7fbcfffca740 1 -- start start 2024-04-17T06:01:09.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.607+0000 7fbcfffca740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561fb0be1ca0 0x561fb0be20c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:09.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.607+0000 7fbcfffca740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561fb0be2690 0x561fb0be7190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:09.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.608+0000 7fbcfffca740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fb0be7800 0x561fb0be9c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:09.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.608+0000 7fbcfffca740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x561fb0af5240 con 0x561fb0be2690 2024-04-17T06:01:09.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.608+0000 7fbcfffca740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x561fb0b529f0 con 0x561fb0be1ca0 2024-04-17T06:01:09.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.608+0000 7fbcfffca740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561fb0b0d9e0 con 0x561fb0be7800 2024-04-17T06:01:09.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.608+0000 7fbcdffff640 1 --2- 172.21.15.47:0/3806807467 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fbce00a3bb0 0x7fbce00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:09.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.608+0000 7fbcf6ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561fb0be2690 0x561fb0be7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:09.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.608+0000 7fbcf6ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561fb0be2690 0x561fb0be7190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40506/0 (socket says 172.21.15.47:40506) 2024-04-17T06:01:09.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.608+0000 7fbcf6ffd640 1 -- 172.21.15.47:0/3846776863 learned_addr learned my addr 172.21.15.47:0/3846776863 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:01:09.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.608+0000 7fbcf77fe640 1 --2- 172.21.15.47:0/3846776863 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561fb0be1ca0 0x561fb0be20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:09.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.608+0000 7fbcf77fe640 1 --2- 172.21.15.47:0/3806807467 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fbce00a3bb0 0x7fbce00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:09.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.608+0000 7fbcf67fc640 1 --2- 172.21.15.47:0/3846776863 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fb0be7800 0x561fb0be9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:09.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.608+0000 7fbcdffff640 1 -- 172.21.15.47:0/3806807467 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fbce80f2180 con 0x7fbcf8101fa0 2024-04-17T06:01:09.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.608+0000 7fbcf6ffd640 1 -- 172.21.15.47:0/3846776863 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fb0be7800 msgr2=0x561fb0be9c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:01:09.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.608+0000 7fbcf6ffd640 1 --2- 172.21.15.47:0/3846776863 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fb0be7800 0x561fb0be9c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:09.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.608+0000 7fbcf6ffd640 1 -- 172.21.15.47:0/3846776863 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561fb0be1ca0 msgr2=0x561fb0be20c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:01:09.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.608+0000 7fbcf6ffd640 1 --2- 172.21.15.47:0/3846776863 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561fb0be1ca0 0x561fb0be20c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:09.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.608+0000 7fbcf6ffd640 1 -- 172.21.15.47:0/3846776863 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561fb0b00bc0 con 0x561fb0be2690 2024-04-17T06:01:09.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.608+0000 7fbcf77fe640 1 --2- 172.21.15.47:0/3846776863 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561fb0be1ca0 0x561fb0be20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T06:01:09.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.608+0000 7fbcf67fc640 1 --2- 172.21.15.47:0/3846776863 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fb0be7800 0x561fb0be9c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:01:09.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.608+0000 7fbcf6ffd640 1 --2- 172.21.15.47:0/3846776863 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561fb0be2690 0x561fb0be7190 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fbce800b1b0 tx=0x7fbce8049000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b475827a4bdba43 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:01:09.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.608+0000 7fbcf77fe640 1 --2- 172.21.15.47:0/3806807467 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fbce00a3bb0 0x7fbce00a6070 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fbcf0007fd0 tx=0x7fbcf0007520 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:01:09.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.608+0000 7fbcdd7fa640 1 -- 172.21.15.47:0/3846776863 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbce804b070 con 0x561fb0be2690 2024-04-17T06:01:09.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.608+0000 7fbcdd7fa640 1 -- 172.21.15.47:0/3846776863 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbce8005690 con 0x561fb0be2690 2024-04-17T06:01:09.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.608+0000 7fbcdd7fa640 1 -- 172.21.15.47:0/3846776863 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbce80efda0 con 0x561fb0be2690 2024-04-17T06:01:09.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.609+0000 7fbcfffca740 1 -- 172.21.15.47:0/3846776863 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561fb0be2690 msgr2=0x561fb0be7190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:09.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.609+0000 7fbcfffca740 1 --2- 172.21.15.47:0/3846776863 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561fb0be2690 0x561fb0be7190 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fbce800b1b0 tx=0x7fbce8049000 comp rx=0 tx=0).stop 2024-04-17T06:01:09.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.609+0000 7fbcfffca740 1 -- 172.21.15.47:0/3846776863 shutdown_connections 2024-04-17T06:01:09.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.609+0000 7fbcfffca740 1 --2- 172.21.15.47:0/3846776863 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fb0be7800 0x561fb0be9c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:09.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.609+0000 7fbcfffca740 1 --2- 172.21.15.47:0/3846776863 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561fb0be2690 0x561fb0be7190 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:09.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.609+0000 7fbcfffca740 1 --2- 172.21.15.47:0/3846776863 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561fb0be1ca0 0x561fb0be20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:09.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.609+0000 7fbcfffca740 1 -- 172.21.15.47:0/3846776863 >> 172.21.15.47:0/3846776863 conn(0x561fb0bdd210 msgr2=0x561fb0bdd610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:01:09.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.609+0000 7fbcfffca740 1 -- 172.21.15.47:0/3846776863 shutdown_connections 2024-04-17T06:01:09.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.609+0000 7fbcfffca740 1 -- 172.21.15.47:0/3846776863 wait complete. 2024-04-17T06:01:09.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.609+0000 7fbcfffca740 1 -- start start 2024-04-17T06:01:09.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.610+0000 7fbcfffca740 1 Processor -- start 2024-04-17T06:01:09.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.610+0000 7fbcfffca740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561fb0be1ca0 0x561fb0bebb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:09.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.610+0000 7fbcfffca740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fb0be2690 0x561fb0bec060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:09.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.610+0000 7fbcfffca740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561fb0be7800 0x561fb0bf30a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:09.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.610+0000 7fbcfffca740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x561fb0b4c0f0 con 0x561fb0be7800 2024-04-17T06:01:09.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.610+0000 7fbcfffca740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x561fb0b46dd0 con 0x561fb0be1ca0 2024-04-17T06:01:09.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.610+0000 7fbcfffca740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561fb0b54cd0 con 0x561fb0be2690 2024-04-17T06:01:09.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.610+0000 7fbcfffca740 10 client.0 _ll_register_callbacks cb 0x561fb0b3d570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:01:09.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.610+0000 7fbcf77fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561fb0be7800 0x561fb0bf30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:09.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.610+0000 7fbcf77fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561fb0be7800 0x561fb0bf30a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40518/0 (socket says 172.21.15.47:40518) 2024-04-17T06:01:09.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.610+0000 7fbcf77fe640 1 -- 172.21.15.47:0/1910893312 learned_addr learned my addr 172.21.15.47:0/1910893312 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:01:09.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.610+0000 7fbcf6ffd640 1 --2- 172.21.15.47:0/1910893312 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561fb0be1ca0 0x561fb0bebb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:09.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.610+0000 7fbcf67fc640 1 --2- 172.21.15.47:0/1910893312 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fb0be2690 0x561fb0bec060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:09.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.610+0000 7fbcf77fe640 1 -- 172.21.15.47:0/1910893312 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fb0be2690 msgr2=0x561fb0bec060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:01:09.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.610+0000 7fbcf77fe640 1 --2- 172.21.15.47:0/1910893312 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fb0be2690 0x561fb0bec060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:09.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.610+0000 7fbcf77fe640 1 -- 172.21.15.47:0/1910893312 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561fb0be1ca0 msgr2=0x561fb0bebb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:01:09.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.610+0000 7fbcf77fe640 1 --2- 172.21.15.47:0/1910893312 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561fb0be1ca0 0x561fb0bebb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:09.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.610+0000 7fbcf77fe640 1 -- 172.21.15.47:0/1910893312 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561fb085a2d0 con 0x561fb0be7800 2024-04-17T06:01:09.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.610+0000 7fbcf77fe640 1 --2- 172.21.15.47:0/1910893312 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561fb0be7800 0x561fb0bf30a0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fbcf0003c70 tx=0x7fbcf0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5762dcb4d985cb8e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:01:09.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.610+0000 7fbcdaff5640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-17T06:01:09.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.610+0000 7fbcdaff5640 1 -- 172.21.15.47:0/1910893312 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcf0025070 con 0x561fb0be7800 2024-04-17T06:01:09.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.611+0000 7fbcdaff5640 1 -- 172.21.15.47:0/1910893312 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fbcf000f890 con 0x561fb0be7800 2024-04-17T06:01:09.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.611+0000 7fbcfffca740 1 -- 172.21.15.47:0/1910893312 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561fb0ace4c0 con 0x561fb0be7800 2024-04-17T06:01:09.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.611+0000 7fbcdaff5640 1 -- 172.21.15.47:0/1910893312 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fbcf0031050 con 0x561fb0be7800 2024-04-17T06:01:09.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.611+0000 7fbcfffca740 10 client.15252 fetch_fsmap learned FSMap version 21 2024-04-17T06:01:09.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.611+0000 7fbcfffca740 1 -- 172.21.15.47:0/1910893312 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x561fb09246b0 con 0x561fb0be7800 2024-04-17T06:01:09.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.611+0000 7fbcdaff5640 1 -- 172.21.15.47:0/1910893312 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x561fb0ace4c0 con 0x561fb0be7800 2024-04-17T06:01:09.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.611+0000 7fbcdaff5640 1 -- 172.21.15.47:0/1910893312 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 21) ==== 8339+0+0 (secure 0 0 0) 0x7fbcf00219c0 con 0x561fb0be7800 2024-04-17T06:01:09.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.611+0000 7fbcfffca740 10 client.15252 fetch_fsmap finished waiting for FSMap version 21 2024-04-17T06:01:09.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.611+0000 7fbcfffca740 10 client.15252 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-17T06:01:09.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.611+0000 7fbcfffca740 20 client.15252 populate_metadata read hostname 'smithi047' 2024-04-17T06:01:09.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.611+0000 7fbcfffca740 1 --2- 172.21.15.47:0/1910893312 >> [v2:172.21.15.103:6838/1467157889,v1:172.21.15.103:6839/1467157889] conn(0x561fb0bf8860 0x561fb0bf9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:09.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.611+0000 7fbcfffca740 4 client.15252 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T06:01:09.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.611+0000 7fbcfffca740 1 -- 172.21.15.47:0/1910893312 --> [v2:172.21.15.103:6838/1467157889,v1:172.21.15.103:6839/1467157889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561fb0b554e0 con 0x561fb0bf8860 2024-04-17T06:01:09.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.611+0000 7fbcf6ffd640 1 --2- 172.21.15.47:0/1910893312 >> [v2:172.21.15.103:6838/1467157889,v1:172.21.15.103:6839/1467157889] conn(0x561fb0bf8860 0x561fb0bf9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:09.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.612+0000 7fbcdaff5640 10 client.15252 ms_handle_connect on v2:172.21.15.103:6838/1467157889 2024-04-17T06:01:09.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.612+0000 7fbcf6ffd640 1 --2- 172.21.15.47:0/1910893312 >> [v2:172.21.15.103:6838/1467157889,v1:172.21.15.103:6839/1467157889] conn(0x561fb0bf8860 0x561fb0bf9cd0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=22ce478a9e3d3489 server_cookie=ff8adea16e14d771 in_seq=0 out_seq=0 2024-04-17T06:01:09.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.613+0000 7fbcdaff5640 1 -- 172.21.15.47:0/1910893312 <== mds.2 v2:172.21.15.103:6838/1467157889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561fb0b554e0 con 0x561fb0bf8860 2024-04-17T06:01:09.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.613+0000 7fbcdaff5640 10 client.15252 handle_command_reply: tid=0 2024-04-17T06:01:09.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.613+0000 7fbcfffca740 1 client.15252 shutdown 2024-04-17T06:01:09.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.613+0000 7fbcfffca740 1 -- 172.21.15.47:0/1910893312 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561fb0be7800 msgr2=0x561fb0bf30a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:09.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.613+0000 7fbcfffca740 1 --2- 172.21.15.47:0/1910893312 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561fb0be7800 0x561fb0bf30a0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fbcf0003c70 tx=0x7fbcf0004a40 comp rx=0 tx=0).stop 2024-04-17T06:01:09.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.613+0000 7fbcfffca740 1 -- 172.21.15.47:0/1910893312 shutdown_connections 2024-04-17T06:01:09.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.613+0000 7fbcfffca740 1 --2- 172.21.15.47:0/1910893312 >> [v2:172.21.15.103:6838/1467157889,v1:172.21.15.103:6839/1467157889] conn(0x561fb0bf8860 0x561fb0bf9cd0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:09.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.613+0000 7fbcfffca740 1 --2- 172.21.15.47:0/1910893312 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561fb0be7800 0x561fb0bf30a0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:09.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.613+0000 7fbcfffca740 1 --2- 172.21.15.47:0/1910893312 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fb0be2690 0x561fb0bec060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:09.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.613+0000 7fbcdaff5640 0 client.15252 ms_handle_reset on v2:172.21.15.103:6838/1467157889 2024-04-17T06:01:09.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.613+0000 7fbcfffca740 1 --2- 172.21.15.47:0/1910893312 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561fb0be1ca0 0x561fb0bebb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:09.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.613+0000 7fbcfffca740 1 -- 172.21.15.47:0/1910893312 >> 172.21.15.47:0/1910893312 conn(0x561fb0bdd210 msgr2=0x561fb0bdc350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:01:09.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.613+0000 7fbcfffca740 1 -- 172.21.15.47:0/1910893312 shutdown_connections 2024-04-17T06:01:09.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.613+0000 7fbcfffca740 1 -- 172.21.15.47:0/1910893312 wait complete. 2024-04-17T06:01:09.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.613+0000 7fbcfffca740 20 client.15252 trim_cache size 0 max 16384 2024-04-17T06:01:09.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.631+0000 7fbcfffca740 1 Processor -- start 2024-04-17T06:01:09.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.632+0000 7fbcfffca740 1 -- start start 2024-04-17T06:01:09.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.632+0000 7fbcfffca740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561fb0be1ca0 0x561fb0bea670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:09.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.632+0000 7fbcfffca740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561fb0be2690 0x561fb0beabb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:09.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.632+0000 7fbcfffca740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fb0be7800 0x561fb0bf3060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:09.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.632+0000 7fbcfffca740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x561fb0bf8010 con 0x561fb0be2690 2024-04-17T06:01:09.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.632+0000 7fbcfffca740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x561fb0bf86e0 con 0x561fb0be1ca0 2024-04-17T06:01:09.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.632+0000 7fbcfffca740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561fb0bf8db0 con 0x561fb0be7800 2024-04-17T06:01:09.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.632+0000 7fbcf67fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561fb0be2690 0x561fb0beabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:09.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.632+0000 7fbcf67fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561fb0be2690 0x561fb0beabb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40528/0 (socket says 172.21.15.47:40528) 2024-04-17T06:01:09.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.632+0000 7fbcf67fc640 1 -- 172.21.15.47:0/202421143 learned_addr learned my addr 172.21.15.47:0/202421143 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:01:09.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.632+0000 7fbcf6ffd640 1 --2- 172.21.15.47:0/202421143 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561fb0be1ca0 0x561fb0bea670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:09.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.632+0000 7fbcf77fe640 1 --2- 172.21.15.47:0/202421143 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fb0be7800 0x561fb0bf3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:09.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.632+0000 7fbcf67fc640 1 -- 172.21.15.47:0/202421143 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fb0be7800 msgr2=0x561fb0bf3060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:01:09.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.632+0000 7fbcf67fc640 1 --2- 172.21.15.47:0/202421143 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fb0be7800 0x561fb0bf3060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:09.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.632+0000 7fbcf67fc640 1 -- 172.21.15.47:0/202421143 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561fb0be1ca0 msgr2=0x561fb0bea670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:01:09.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.632+0000 7fbcf67fc640 1 --2- 172.21.15.47:0/202421143 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561fb0be1ca0 0x561fb0bea670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:09.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.632+0000 7fbcf67fc640 1 -- 172.21.15.47:0/202421143 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561fb0be2a70 con 0x561fb0be2690 2024-04-17T06:01:09.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.632+0000 7fbcf77fe640 1 --2- 172.21.15.47:0/202421143 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fb0be7800 0x561fb0bf3060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:01:09.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.633+0000 7fbcf67fc640 1 --2- 172.21.15.47:0/202421143 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561fb0be2690 0x561fb0beabb0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fbcec00e8c0 tx=0x7fbcec00ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf7a94819370452e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:01:09.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.633+0000 7fbcdcff9640 1 -- 172.21.15.47:0/202421143 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcec0109e0 con 0x561fb0be2690 2024-04-17T06:01:09.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.633+0000 7fbcdcff9640 1 -- 172.21.15.47:0/202421143 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbcec010b80 con 0x561fb0be2690 2024-04-17T06:01:09.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.633+0000 7fbcdcff9640 1 -- 172.21.15.47:0/202421143 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcec01c660 con 0x561fb0be2690 2024-04-17T06:01:09.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.633+0000 7fbcfffca740 1 -- 172.21.15.47:0/202421143 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561fb0be2690 msgr2=0x561fb0beabb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:09.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.633+0000 7fbcfffca740 1 --2- 172.21.15.47:0/202421143 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561fb0be2690 0x561fb0beabb0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fbcec00e8c0 tx=0x7fbcec00ed80 comp rx=0 tx=0).stop 2024-04-17T06:01:09.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.634+0000 7fbcfffca740 1 -- 172.21.15.47:0/202421143 shutdown_connections 2024-04-17T06:01:09.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.634+0000 7fbcfffca740 1 --2- 172.21.15.47:0/202421143 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fb0be7800 0x561fb0bf3060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:09.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.634+0000 7fbcfffca740 1 --2- 172.21.15.47:0/202421143 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561fb0be2690 0x561fb0beabb0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:09.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.634+0000 7fbcfffca740 1 --2- 172.21.15.47:0/202421143 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561fb0be1ca0 0x561fb0bea670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:09.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.634+0000 7fbcfffca740 1 -- 172.21.15.47:0/202421143 >> 172.21.15.47:0/202421143 conn(0x561fb0bdd210 msgr2=0x561fb0bdd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:01:09.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.634+0000 7fbcfffca740 1 -- 172.21.15.47:0/202421143 shutdown_connections 2024-04-17T06:01:09.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.634+0000 7fbcfffca740 1 -- 172.21.15.47:0/202421143 wait complete. 2024-04-17T06:01:09.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.634+0000 7fbcfffca740 1 -- start start 2024-04-17T06:01:09.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.634+0000 7fbcfffca740 1 Processor -- start 2024-04-17T06:01:09.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.634+0000 7fbcfffca740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561fb0be1ca0 0x561fb0bee850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:09.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.634+0000 7fbcfffca740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561fb0be2690 0x561fb0beed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:09.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.634+0000 7fbcfffca740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fb0be7800 0x561fb0bf1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:09.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.634+0000 7fbcfffca740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x561fb0bf9b50 con 0x561fb0be1ca0 2024-04-17T06:01:09.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.634+0000 7fbcfffca740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x561fb0bf9480 con 0x561fb0be2690 2024-04-17T06:01:09.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.634+0000 7fbcfffca740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561fb0bfa220 con 0x561fb0be7800 2024-04-17T06:01:09.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.634+0000 7fbcfffca740 10 client.0 _ll_register_callbacks cb 0x7fbcf0007740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:01:09.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.634+0000 7fbcf6ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561fb0be1ca0 0x561fb0bee850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:09.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.634+0000 7fbcf6ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561fb0be1ca0 0x561fb0bee850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40534/0 (socket says 172.21.15.47:40534) 2024-04-17T06:01:09.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.634+0000 7fbcf6ffd640 1 -- 172.21.15.47:0/997272622 learned_addr learned my addr 172.21.15.47:0/997272622 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:01:09.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.634+0000 7fbcf67fc640 1 --2- 172.21.15.47:0/997272622 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561fb0be2690 0x561fb0beed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:09.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.634+0000 7fbcf77fe640 1 --2- 172.21.15.47:0/997272622 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fb0be7800 0x561fb0bf1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:09.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.635+0000 7fbcf6ffd640 1 -- 172.21.15.47:0/997272622 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fb0be7800 msgr2=0x561fb0bf1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:01:09.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.635+0000 7fbcf6ffd640 1 --2- 172.21.15.47:0/997272622 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fb0be7800 0x561fb0bf1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:09.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.635+0000 7fbcf6ffd640 1 -- 172.21.15.47:0/997272622 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561fb0be2690 msgr2=0x561fb0beed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:01:09.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.635+0000 7fbcf6ffd640 1 --2- 172.21.15.47:0/997272622 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561fb0be2690 0x561fb0beed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:09.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.635+0000 7fbcf6ffd640 1 -- 172.21.15.47:0/997272622 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561fb0bf2330 con 0x561fb0be1ca0 2024-04-17T06:01:09.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.635+0000 7fbcf67fc640 1 --2- 172.21.15.47:0/997272622 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561fb0be2690 0x561fb0beed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:01:09.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.635+0000 7fbcf6ffd640 1 --2- 172.21.15.47:0/997272622 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561fb0be1ca0 0x561fb0bee850 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fbce80ba150 tx=0x7fbce80ba810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6770917180f8d1a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:01:09.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.635+0000 7fbcdaff5640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-17T06:01:09.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.635+0000 7fbcdaff5640 1 -- 172.21.15.47:0/997272622 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbce80d7060 con 0x561fb0be1ca0 2024-04-17T06:01:09.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.635+0000 7fbcdaff5640 1 -- 172.21.15.47:0/997272622 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fbce80035f0 con 0x561fb0be1ca0 2024-04-17T06:01:09.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.635+0000 7fbcfffca740 1 -- 172.21.15.47:0/997272622 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbcec010b80 con 0x561fb0be1ca0 2024-04-17T06:01:09.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.635+0000 7fbcdaff5640 1 -- 172.21.15.47:0/997272622 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fbce804b040 con 0x561fb0be1ca0 2024-04-17T06:01:09.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.635+0000 7fbcfffca740 10 client.15258 fetch_fsmap learned FSMap version 21 2024-04-17T06:01:09.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.635+0000 7fbcfffca740 1 -- 172.21.15.47:0/997272622 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x561fb0c5e810 con 0x561fb0be1ca0 2024-04-17T06:01:09.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.635+0000 7fbcdaff5640 1 -- 172.21.15.47:0/997272622 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbcec010b80 con 0x561fb0be1ca0 2024-04-17T06:01:09.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.635+0000 7fbcdaff5640 1 -- 172.21.15.47:0/997272622 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 21) ==== 8339+0+0 (secure 0 0 0) 0x7fbce80efce0 con 0x561fb0be1ca0 2024-04-17T06:01:09.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.636+0000 7fbcfffca740 10 client.15258 fetch_fsmap finished waiting for FSMap version 21 2024-04-17T06:01:09.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.636+0000 7fbcfffca740 10 client.15258 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-17T06:01:09.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.636+0000 7fbcfffca740 20 client.15258 populate_metadata read hostname 'smithi047' 2024-04-17T06:01:09.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.636+0000 7fbcfffca740 1 --2- 172.21.15.47:0/997272622 >> [v2:172.21.15.103:6838/1467157889,v1:172.21.15.103:6839/1467157889] conn(0x561fb0c63250 0x561fb0c61f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:09.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.636+0000 7fbcfffca740 4 client.15258 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-17T06:01:09.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.636+0000 7fbcfffca740 1 -- 172.21.15.47:0/997272622 --> [v2:172.21.15.103:6838/1467157889,v1:172.21.15.103:6839/1467157889] -- command(tid 0: {"prefix": "status"}) -- 0x561fb0b4f350 con 0x561fb0c63250 2024-04-17T06:01:09.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.636+0000 7fbcf67fc640 1 --2- 172.21.15.47:0/997272622 >> [v2:172.21.15.103:6838/1467157889,v1:172.21.15.103:6839/1467157889] conn(0x561fb0c63250 0x561fb0c61f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:09.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.636+0000 7fbcdaff5640 10 client.15258 ms_handle_connect on v2:172.21.15.103:6838/1467157889 2024-04-17T06:01:09.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.636+0000 7fbcf67fc640 1 --2- 172.21.15.47:0/997272622 >> [v2:172.21.15.103:6838/1467157889,v1:172.21.15.103:6839/1467157889] conn(0x561fb0c63250 0x561fb0c61f60 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e72aa0c6229b8a7d server_cookie=cbca799684c28a88 in_seq=0 out_seq=0 2024-04-17T06:01:09.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.637+0000 7fbcdaff5640 1 -- 172.21.15.47:0/997272622 <== mds.2 v2:172.21.15.103:6838/1467157889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x561fb0b4f350 con 0x561fb0c63250 2024-04-17T06:01:09.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.637+0000 7fbcdaff5640 10 client.15258 handle_command_reply: tid=0 2024-04-17T06:01:09.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.637+0000 7fbcfffca740 1 client.15258 shutdown 2024-04-17T06:01:09.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.637+0000 7fbcfffca740 1 -- 172.21.15.47:0/997272622 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561fb0be1ca0 msgr2=0x561fb0bee850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:09.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.637+0000 7fbcfffca740 1 --2- 172.21.15.47:0/997272622 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561fb0be1ca0 0x561fb0bee850 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fbce80ba150 tx=0x7fbce80ba810 comp rx=0 tx=0).stop 2024-04-17T06:01:09.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.637+0000 7fbcfffca740 1 -- 172.21.15.47:0/997272622 shutdown_connections 2024-04-17T06:01:09.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.637+0000 7fbcfffca740 1 --2- 172.21.15.47:0/997272622 >> [v2:172.21.15.103:6838/1467157889,v1:172.21.15.103:6839/1467157889] conn(0x561fb0c63250 0x561fb0c61f60 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:09.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.637+0000 7fbcfffca740 1 --2- 172.21.15.47:0/997272622 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fb0be7800 0x561fb0bf1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:09.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.637+0000 7fbcdaff5640 0 client.15258 ms_handle_reset on v2:172.21.15.103:6838/1467157889 2024-04-17T06:01:09.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.637+0000 7fbcfffca740 1 --2- 172.21.15.47:0/997272622 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561fb0be2690 0x561fb0beed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:09.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.637+0000 7fbcfffca740 1 --2- 172.21.15.47:0/997272622 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561fb0be1ca0 0x561fb0bee850 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:09.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.637+0000 7fbcfffca740 1 -- 172.21.15.47:0/997272622 >> 172.21.15.47:0/997272622 conn(0x561fb0bdd210 msgr2=0x561fb0bea670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:01:09.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.637+0000 7fbcfffca740 1 -- 172.21.15.47:0/997272622 shutdown_connections 2024-04-17T06:01:09.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.637+0000 7fbcfffca740 1 -- 172.21.15.47:0/997272622 wait complete. 2024-04-17T06:01:09.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.637+0000 7fbcfffca740 20 client.15258 trim_cache size 0 max 16384 2024-04-17T06:01:09.643 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-17T06:01:09.643 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_fsid": "565db2d8-fc7e-11ee-bc91-c7b262605968", 2024-04-17T06:01:09.643 INFO:teuthology.orchestra.run.smithi047.stdout: "whoami": 2, 2024-04-17T06:01:09.643 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 24466, 2024-04-17T06:01:09.643 INFO:teuthology.orchestra.run.smithi047.stdout: "want_state": "up:active", 2024-04-17T06:01:09.643 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "up:active", 2024-04-17T06:01:09.643 INFO:teuthology.orchestra.run.smithi047.stdout: "fs_name": "cephfs", 2024-04-17T06:01:09.643 INFO:teuthology.orchestra.run.smithi047.stdout: "rank_uptime": 111.013979788, 2024-04-17T06:01:09.643 INFO:teuthology.orchestra.run.smithi047.stdout: "mdsmap_epoch": 21, 2024-04-17T06:01:09.643 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch": 88, 2024-04-17T06:01:09.643 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch_barrier": 86, 2024-04-17T06:01:09.643 INFO:teuthology.orchestra.run.smithi047.stdout: "uptime": 129.70387281699999 2024-04-17T06:01:09.643 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-17T06:01:09.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.638+0000 7fbcfd3db640 1 -- 172.21.15.47:0/3806807467 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fbce00a3bb0 msgr2=0x7fbce00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:09.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.638+0000 7fbcfd3db640 1 --2- 172.21.15.47:0/3806807467 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fbce00a3bb0 0x7fbce00a6070 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fbcf0007fd0 tx=0x7fbcf0007520 comp rx=0 tx=0).stop 2024-04-17T06:01:09.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.638+0000 7fbcfd3db640 1 -- 172.21.15.47:0/3806807467 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbcf8101fa0 msgr2=0x7fbcf81982f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:09.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.638+0000 7fbcfd3db640 1 --2- 172.21.15.47:0/3806807467 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbcf8101fa0 0x7fbcf81982f0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fbce8000c00 tx=0x7fbce8004040 comp rx=0 tx=0).stop 2024-04-17T06:01:09.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.638+0000 7fbcfd3db640 1 -- 172.21.15.47:0/3806807467 shutdown_connections 2024-04-17T06:01:09.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.638+0000 7fbcfd3db640 1 --2- 172.21.15.47:0/3806807467 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fbce00a3bb0 0x7fbce00a6070 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:09.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.638+0000 7fbcfd3db640 1 --2- 172.21.15.47:0/3806807467 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbcf8103b50 0x7fbcf819f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:09.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.638+0000 7fbcfd3db640 1 --2- 172.21.15.47:0/3806807467 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbcf8103190 0x7fbcf8198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:09.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.638+0000 7fbcfd3db640 1 --2- 172.21.15.47:0/3806807467 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbcf8101fa0 0x7fbcf81982f0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:09.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.638+0000 7fbcfd3db640 1 -- 172.21.15.47:0/3806807467 >> 172.21.15.47:0/3806807467 conn(0x7fbcf80fd510 msgr2=0x7fbcf8106510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:01:09.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.638+0000 7fbcfd3db640 1 -- 172.21.15.47:0/3806807467 shutdown_connections 2024-04-17T06:01:09.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.638+0000 7fbcfd3db640 1 -- 172.21.15.47:0/3806807467 wait complete. 2024-04-17T06:01:09.647 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-04-17T06:01:09.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.756+0000 7f03d422a640 1 -- 172.21.15.47:0/4199386219 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03cc101ff0 msgr2=0x7f03cc102470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:09.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.756+0000 7f03d422a640 1 --2- 172.21.15.47:0/4199386219 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03cc101ff0 0x7f03cc102470 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f03c0009900 tx=0x7f03c002bc30 comp rx=0 tx=0).stop 2024-04-17T06:01:09.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.756+0000 7f03d422a640 1 -- 172.21.15.47:0/4199386219 shutdown_connections 2024-04-17T06:01:09.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.756+0000 7f03d422a640 1 --2- 172.21.15.47:0/4199386219 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f03cc1029b0 0x7f03cc1092d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:09.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.756+0000 7f03d422a640 1 --2- 172.21.15.47:0/4199386219 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03cc101ff0 0x7f03cc102470 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:09.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.756+0000 7f03d422a640 1 --2- 172.21.15.47:0/4199386219 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03cc100ea0 0x7f03cc1012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:09.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.756+0000 7f03d422a640 1 -- 172.21.15.47:0/4199386219 >> 172.21.15.47:0/4199386219 conn(0x7f03cc0fc410 msgr2=0x7f03cc0fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:01:09.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.756+0000 7f03d422a640 1 -- 172.21.15.47:0/4199386219 shutdown_connections 2024-04-17T06:01:09.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.756+0000 7f03d422a640 1 -- 172.21.15.47:0/4199386219 wait complete. 2024-04-17T06:01:09.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.756+0000 7f03d422a640 1 Processor -- start 2024-04-17T06:01:09.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.756+0000 7f03d422a640 1 -- start start 2024-04-17T06:01:09.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.757+0000 7f03d422a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03cc100ea0 0x7f03cc1971a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:09.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.757+0000 7f03d422a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03cc101ff0 0x7f03cc1976e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:09.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.757+0000 7f03d422a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f03cc1029b0 0x7f03cc19e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:09.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.757+0000 7f03d422a640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f03cc10cb30 con 0x7f03cc100ea0 2024-04-17T06:01:09.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.757+0000 7f03d422a640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f03cc10c9b0 con 0x7f03cc1029b0 2024-04-17T06:01:09.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.757+0000 7f03d422a640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f03cc10ccb0 con 0x7f03cc101ff0 2024-04-17T06:01:09.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.757+0000 7f03d1fc6640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03cc100ea0 0x7f03cc1971a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:09.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.757+0000 7f03d1fc6640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03cc100ea0 0x7f03cc1971a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40552/0 (socket says 172.21.15.47:40552) 2024-04-17T06:01:09.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.757+0000 7f03d1fc6640 1 -- 172.21.15.47:0/4164383573 learned_addr learned my addr 172.21.15.47:0/4164383573 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:01:09.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.757+0000 7f03d17c5640 1 --2- 172.21.15.47:0/4164383573 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03cc101ff0 0x7f03cc1976e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:09.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.757+0000 7f03d27c7640 1 --2- 172.21.15.47:0/4164383573 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f03cc1029b0 0x7f03cc19e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:09.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.757+0000 7f03d1fc6640 1 -- 172.21.15.47:0/4164383573 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03cc101ff0 msgr2=0x7f03cc1976e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:09.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.757+0000 7f03d1fc6640 1 --2- 172.21.15.47:0/4164383573 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03cc101ff0 0x7f03cc1976e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:09.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.757+0000 7f03d1fc6640 1 -- 172.21.15.47:0/4164383573 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f03cc1029b0 msgr2=0x7f03cc19e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:09.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.757+0000 7f03d1fc6640 1 --2- 172.21.15.47:0/4164383573 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f03cc1029b0 0x7f03cc19e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:09.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.757+0000 7f03d1fc6640 1 -- 172.21.15.47:0/4164383573 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03cc19edc0 con 0x7f03cc100ea0 2024-04-17T06:01:09.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.757+0000 7f03d1fc6640 1 --2- 172.21.15.47:0/4164383573 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03cc100ea0 0x7f03cc1971a0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f03c800d950 tx=0x7f03c800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:01:09.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.757+0000 7f03beffd640 1 -- 172.21.15.47:0/4164383573 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c8017030 con 0x7f03cc100ea0 2024-04-17T06:01:09.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.757+0000 7f03beffd640 1 -- 172.21.15.47:0/4164383573 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03c8004020 con 0x7f03cc100ea0 2024-04-17T06:01:09.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.757+0000 7f03d422a640 1 -- 172.21.15.47:0/4164383573 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03cc19f0b0 con 0x7f03cc100ea0 2024-04-17T06:01:09.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.757+0000 7f03beffd640 1 -- 172.21.15.47:0/4164383573 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c80047a0 con 0x7f03cc100ea0 2024-04-17T06:01:09.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.757+0000 7f03d422a640 1 -- 172.21.15.47:0/4164383573 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03cc19f5d0 con 0x7f03cc100ea0 2024-04-17T06:01:09.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.760+0000 7f03beffd640 1 -- 172.21.15.47:0/4164383573 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f03c80041d0 con 0x7f03cc100ea0 2024-04-17T06:01:09.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.760+0000 7f03d6e15740 1 Processor -- start 2024-04-17T06:01:09.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.760+0000 7f03d6e15740 1 -- start start 2024-04-17T06:01:09.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.760+0000 7f03d6e15740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d18355eb0 0x558d183562d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:09.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.760+0000 7f03d6e15740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558d183568a0 0x558d1835b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:09.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.760+0000 7f03d6e15740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558d1835ba10 0x558d1835dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:09.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.760+0000 7f03d6e15740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x558d18267240 con 0x558d183568a0 2024-04-17T06:01:09.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.760+0000 7f03d6e15740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x558d182c49f0 con 0x558d1835ba10 2024-04-17T06:01:09.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.760+0000 7f03d6e15740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558d1827f9e0 con 0x558d18355eb0 2024-04-17T06:01:09.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.760+0000 7f03beffd640 1 --2- 172.21.15.47:0/4164383573 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f03b80a3bb0 0x7f03b80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:09.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.760+0000 7f03d1fc6640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558d183568a0 0x558d1835b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:09.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.760+0000 7f03d1fc6640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558d183568a0 0x558d1835b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40566/0 (socket says 172.21.15.47:40566) 2024-04-17T06:01:09.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.760+0000 7f03d1fc6640 1 -- 172.21.15.47:0/3503929828 learned_addr learned my addr 172.21.15.47:0/3503929828 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:01:09.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.761+0000 7f03d27c7640 1 --2- 172.21.15.47:0/4164383573 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f03b80a3bb0 0x7f03b80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:09.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.761+0000 7f03d17c5640 1 --2- 172.21.15.47:0/3503929828 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558d1835ba10 0x558d1835dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:09.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.761+0000 7f03d27c7640 1 --2- 172.21.15.47:0/3503929828 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d18355eb0 0x558d183562d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:09.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.761+0000 7f03beffd640 1 -- 172.21.15.47:0/4164383573 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f03c80cffa0 con 0x7f03cc100ea0 2024-04-17T06:01:09.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.761+0000 7f03d1fc6640 1 -- 172.21.15.47:0/3503929828 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d18355eb0 msgr2=0x558d183562d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:01:09.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.761+0000 7f03d1fc6640 1 --2- 172.21.15.47:0/3503929828 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d18355eb0 0x558d183562d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:09.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.761+0000 7f03d1fc6640 1 -- 172.21.15.47:0/3503929828 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558d1835ba10 msgr2=0x558d1835dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:01:09.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.761+0000 7f03d1fc6640 1 --2- 172.21.15.47:0/3503929828 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558d1835ba10 0x558d1835dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:09.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.761+0000 7f03d1fc6640 1 -- 172.21.15.47:0/3503929828 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d18272bc0 con 0x558d183568a0 2024-04-17T06:01:09.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.761+0000 7f03d17c5640 1 --2- 172.21.15.47:0/3503929828 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558d1835ba10 0x558d1835dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:01:09.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.761+0000 7f03d27c7640 1 --2- 172.21.15.47:0/3503929828 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d18355eb0 0x558d183562d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:01:09.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.761+0000 7f03d1fc6640 1 --2- 172.21.15.47:0/3503929828 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558d183568a0 0x558d1835b3a0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f03c80a4970 tx=0x7f03c809bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21fe9a7b9ca6805e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:01:09.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.761+0000 7f03d27c7640 1 --2- 172.21.15.47:0/4164383573 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f03b80a3bb0 0x7f03b80a6070 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x558d18357100 tx=0x7f03c4007390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:01:09.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.761+0000 7f03b7fff640 1 -- 172.21.15.47:0/3503929828 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c809c230 con 0x558d183568a0 2024-04-17T06:01:09.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.761+0000 7f03b7fff640 1 -- 172.21.15.47:0/3503929828 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03c80a05e0 con 0x558d183568a0 2024-04-17T06:01:09.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.761+0000 7f03b7fff640 1 -- 172.21.15.47:0/3503929828 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c80a0130 con 0x558d183568a0 2024-04-17T06:01:09.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.762+0000 7f03d6e15740 1 -- 172.21.15.47:0/3503929828 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558d183568a0 msgr2=0x558d1835b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:09.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.762+0000 7f03d6e15740 1 --2- 172.21.15.47:0/3503929828 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558d183568a0 0x558d1835b3a0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f03c80a4970 tx=0x7f03c809bea0 comp rx=0 tx=0).stop 2024-04-17T06:01:09.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.762+0000 7f03d6e15740 1 -- 172.21.15.47:0/3503929828 shutdown_connections 2024-04-17T06:01:09.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.762+0000 7f03d6e15740 1 --2- 172.21.15.47:0/3503929828 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558d1835ba10 0x558d1835dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:09.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.762+0000 7f03d6e15740 1 --2- 172.21.15.47:0/3503929828 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558d183568a0 0x558d1835b3a0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:09.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.762+0000 7f03d6e15740 1 --2- 172.21.15.47:0/3503929828 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d18355eb0 0x558d183562d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:09.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.762+0000 7f03d6e15740 1 -- 172.21.15.47:0/3503929828 >> 172.21.15.47:0/3503929828 conn(0x558d18351420 msgr2=0x558d18351820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:01:09.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.762+0000 7f03d6e15740 1 -- 172.21.15.47:0/3503929828 shutdown_connections 2024-04-17T06:01:09.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.762+0000 7f03d6e15740 1 -- 172.21.15.47:0/3503929828 wait complete. 2024-04-17T06:01:09.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.762+0000 7f03d6e15740 1 -- start start 2024-04-17T06:01:09.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.762+0000 7f03d6e15740 1 Processor -- start 2024-04-17T06:01:09.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.763+0000 7f03d6e15740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558d18355eb0 0x558d18360180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:09.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.763+0000 7f03d6e15740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558d183568a0 0x558d183606c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:09.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.763+0000 7f03d6e15740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d1835ba10 0x558d18367690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:09.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.763+0000 7f03d6e15740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x558d182be0f0 con 0x558d183568a0 2024-04-17T06:01:09.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.763+0000 7f03d6e15740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x558d182b8dd0 con 0x558d18355eb0 2024-04-17T06:01:09.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.763+0000 7f03d6e15740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558d182c6cd0 con 0x558d1835ba10 2024-04-17T06:01:09.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.763+0000 7f03d6e15740 10 client.0 _ll_register_callbacks cb 0x558d182af570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:01:09.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.763+0000 7f03d17c5640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558d183568a0 0x558d183606c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:09.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.763+0000 7f03d17c5640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558d183568a0 0x558d183606c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40574/0 (socket says 172.21.15.47:40574) 2024-04-17T06:01:09.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.763+0000 7f03d17c5640 1 -- 172.21.15.47:0/645635646 learned_addr learned my addr 172.21.15.47:0/645635646 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:01:09.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.763+0000 7f03d17c5640 1 -- 172.21.15.47:0/645635646 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d1835ba10 msgr2=0x558d18367690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:01:09.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.763+0000 7f03d17c5640 1 --2- 172.21.15.47:0/645635646 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d1835ba10 0x558d18367690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:09.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.763+0000 7f03d17c5640 1 -- 172.21.15.47:0/645635646 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558d18355eb0 msgr2=0x558d18360180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:01:09.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.763+0000 7f03d1fc6640 1 --2- 172.21.15.47:0/645635646 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558d18355eb0 0x558d18360180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:09.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.763+0000 7f03d17c5640 1 --2- 172.21.15.47:0/645635646 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558d18355eb0 0x558d18360180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:09.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.763+0000 7f03d17c5640 1 -- 172.21.15.47:0/645635646 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d17fcc2d0 con 0x558d183568a0 2024-04-17T06:01:09.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.763+0000 7f03d17c5640 1 --2- 172.21.15.47:0/645635646 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558d183568a0 0x558d183606c0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f03c0009900 tx=0x7f03c00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3ab81a012ed1a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:01:09.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.763+0000 7f03b57fa640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-17T06:01:09.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.764+0000 7f03b57fa640 1 -- 172.21.15.47:0/645635646 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c0005d50 con 0x558d183568a0 2024-04-17T06:01:09.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.764+0000 7f03b57fa640 1 -- 172.21.15.47:0/645635646 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f03c0005450 con 0x558d183568a0 2024-04-17T06:01:09.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.764+0000 7f03d6e15740 1 -- 172.21.15.47:0/645635646 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03c80a05e0 con 0x558d183568a0 2024-04-17T06:01:09.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.764+0000 7f03b57fa640 1 -- 172.21.15.47:0/645635646 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f03c00044e0 con 0x558d183568a0 2024-04-17T06:01:09.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.764+0000 7f03d6e15740 10 client.15270 fetch_fsmap learned FSMap version 21 2024-04-17T06:01:09.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.764+0000 7f03d6e15740 1 -- 172.21.15.47:0/645635646 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x558d180966b0 con 0x558d183568a0 2024-04-17T06:01:09.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.764+0000 7f03b57fa640 1 -- 172.21.15.47:0/645635646 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03c80a05e0 con 0x558d183568a0 2024-04-17T06:01:09.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.764+0000 7f03b57fa640 1 -- 172.21.15.47:0/645635646 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 21) ==== 8339+0+0 (secure 0 0 0) 0x7f03c0003a00 con 0x558d183568a0 2024-04-17T06:01:09.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.764+0000 7f03d6e15740 10 client.15270 fetch_fsmap finished waiting for FSMap version 21 2024-04-17T06:01:09.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.764+0000 7f03d6e15740 10 client.15270 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-17T06:01:09.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.764+0000 7f03d6e15740 20 client.15270 populate_metadata read hostname 'smithi047' 2024-04-17T06:01:09.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.764+0000 7f03d6e15740 1 --2- 172.21.15.47:0/645635646 >> [v2:172.21.15.98:6836/1117440719,v1:172.21.15.98:6837/1117440719] conn(0x558d1836ce00 0x558d1836e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:09.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.764+0000 7f03d6e15740 4 client.15270 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T06:01:09.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.764+0000 7f03d6e15740 1 -- 172.21.15.47:0/645635646 --> [v2:172.21.15.98:6836/1117440719,v1:172.21.15.98:6837/1117440719] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558d182404c0 con 0x558d1836ce00 2024-04-17T06:01:09.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.765+0000 7f03d1fc6640 1 --2- 172.21.15.47:0/645635646 >> [v2:172.21.15.98:6836/1117440719,v1:172.21.15.98:6837/1117440719] conn(0x558d1836ce00 0x558d1836e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:09.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.765+0000 7f03b57fa640 10 client.15270 ms_handle_connect on v2:172.21.15.98:6836/1117440719 2024-04-17T06:01:09.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.765+0000 7f03d1fc6640 1 --2- 172.21.15.47:0/645635646 >> [v2:172.21.15.98:6836/1117440719,v1:172.21.15.98:6837/1117440719] conn(0x558d1836ce00 0x558d1836e270 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=994bd5499e12ee2e server_cookie=fafd6872b9b6ddd5 in_seq=0 out_seq=0 2024-04-17T06:01:09.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.766+0000 7f03b57fa640 1 -- 172.21.15.47:0/645635646 <== mds.1 v2:172.21.15.98:6836/1117440719 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558d182404c0 con 0x558d1836ce00 2024-04-17T06:01:09.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.766+0000 7f03b57fa640 10 client.15270 handle_command_reply: tid=0 2024-04-17T06:01:09.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.766+0000 7f03d6e15740 1 client.15270 shutdown 2024-04-17T06:01:09.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.766+0000 7f03d6e15740 1 -- 172.21.15.47:0/645635646 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558d183568a0 msgr2=0x558d183606c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:09.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.766+0000 7f03d6e15740 1 --2- 172.21.15.47:0/645635646 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558d183568a0 0x558d183606c0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f03c0009900 tx=0x7f03c00042f0 comp rx=0 tx=0).stop 2024-04-17T06:01:09.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.766+0000 7f03d6e15740 1 -- 172.21.15.47:0/645635646 shutdown_connections 2024-04-17T06:01:09.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.766+0000 7f03d6e15740 1 --2- 172.21.15.47:0/645635646 >> [v2:172.21.15.98:6836/1117440719,v1:172.21.15.98:6837/1117440719] conn(0x558d1836ce00 0x558d1836e270 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:09.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.766+0000 7f03d6e15740 1 --2- 172.21.15.47:0/645635646 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d1835ba10 0x558d18367690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:09.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.766+0000 7f03d6e15740 1 --2- 172.21.15.47:0/645635646 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558d183568a0 0x558d183606c0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:09.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.766+0000 7f03d6e15740 1 --2- 172.21.15.47:0/645635646 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558d18355eb0 0x558d18360180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:09.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.766+0000 7f03b57fa640 0 client.15270 ms_handle_reset on v2:172.21.15.98:6836/1117440719 2024-04-17T06:01:09.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.766+0000 7f03d6e15740 1 -- 172.21.15.47:0/645635646 >> 172.21.15.47:0/645635646 conn(0x558d18351420 msgr2=0x558d1834e350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:01:09.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.766+0000 7f03d6e15740 1 -- 172.21.15.47:0/645635646 shutdown_connections 2024-04-17T06:01:09.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.766+0000 7f03d6e15740 1 -- 172.21.15.47:0/645635646 wait complete. 2024-04-17T06:01:09.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.766+0000 7f03d6e15740 20 client.15270 trim_cache size 0 max 16384 2024-04-17T06:01:09.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.784+0000 7f03d6e15740 1 Processor -- start 2024-04-17T06:01:09.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.784+0000 7f03d6e15740 1 -- start start 2024-04-17T06:01:09.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.784+0000 7f03d6e15740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558d18355eb0 0x558d1835f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:09.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.784+0000 7f03d6e15740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558d183568a0 0x558d18351800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:09.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.784+0000 7f03d6e15740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d1835ba10 0x558d18357e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:09.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.784+0000 7f03d6e15740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x558d1836c5b0 con 0x558d183568a0 2024-04-17T06:01:09.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.784+0000 7f03d6e15740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x558d1836cc80 con 0x558d18355eb0 2024-04-17T06:01:09.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.784+0000 7f03d6e15740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558d1836d350 con 0x558d1835ba10 2024-04-17T06:01:09.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.784+0000 7f03d17c5640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558d183568a0 0x558d18351800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:09.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.784+0000 7f03d17c5640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558d183568a0 0x558d18351800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40588/0 (socket says 172.21.15.47:40588) 2024-04-17T06:01:09.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.784+0000 7f03d17c5640 1 -- 172.21.15.47:0/3876675647 learned_addr learned my addr 172.21.15.47:0/3876675647 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:01:09.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.785+0000 7f03d1fc6640 1 --2- 172.21.15.47:0/3876675647 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558d18355eb0 0x558d1835f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:09.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.785+0000 7f03d17c5640 1 -- 172.21.15.47:0/3876675647 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d1835ba10 msgr2=0x558d18357e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:01:09.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.785+0000 7f03d17c5640 1 --2- 172.21.15.47:0/3876675647 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d1835ba10 0x558d18357e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:09.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.785+0000 7f03d17c5640 1 -- 172.21.15.47:0/3876675647 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558d18355eb0 msgr2=0x558d1835f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:01:09.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.785+0000 7f03d17c5640 1 --2- 172.21.15.47:0/3876675647 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558d18355eb0 0x558d1835f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:09.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.785+0000 7f03d17c5640 1 -- 172.21.15.47:0/3876675647 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d18355ca0 con 0x558d183568a0 2024-04-17T06:01:09.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.785+0000 7f03d17c5640 1 --2- 172.21.15.47:0/3876675647 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558d183568a0 0x558d18351800 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f03c00096c0 tx=0x7f03c0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d461da4ff7037577 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:01:09.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.785+0000 7f03b77fe640 1 -- 172.21.15.47:0/3876675647 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c0055070 con 0x558d183568a0 2024-04-17T06:01:09.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.785+0000 7f03b77fe640 1 -- 172.21.15.47:0/3876675647 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03c00035f0 con 0x558d183568a0 2024-04-17T06:01:09.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.785+0000 7f03b77fe640 1 -- 172.21.15.47:0/3876675647 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c0034a10 con 0x558d183568a0 2024-04-17T06:01:09.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.785+0000 7f03d6e15740 1 -- 172.21.15.47:0/3876675647 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558d183568a0 msgr2=0x558d18351800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:09.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.785+0000 7f03d6e15740 1 --2- 172.21.15.47:0/3876675647 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558d183568a0 0x558d18351800 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f03c00096c0 tx=0x7f03c0000bc0 comp rx=0 tx=0).stop 2024-04-17T06:01:09.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.785+0000 7f03d6e15740 1 -- 172.21.15.47:0/3876675647 shutdown_connections 2024-04-17T06:01:09.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.785+0000 7f03d6e15740 1 --2- 172.21.15.47:0/3876675647 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d1835ba10 0x558d18357e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:09.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.785+0000 7f03d6e15740 1 --2- 172.21.15.47:0/3876675647 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558d183568a0 0x558d18351800 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:09.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.785+0000 7f03d6e15740 1 --2- 172.21.15.47:0/3876675647 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558d18355eb0 0x558d1835f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:09.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.785+0000 7f03d6e15740 1 -- 172.21.15.47:0/3876675647 >> 172.21.15.47:0/3876675647 conn(0x558d18351420 msgr2=0x558d1836bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:01:09.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.785+0000 7f03d6e15740 1 -- 172.21.15.47:0/3876675647 shutdown_connections 2024-04-17T06:01:09.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.785+0000 7f03d6e15740 1 -- 172.21.15.47:0/3876675647 wait complete. 2024-04-17T06:01:09.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.786+0000 7f03d6e15740 1 -- start start 2024-04-17T06:01:09.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.786+0000 7f03d6e15740 1 Processor -- start 2024-04-17T06:01:09.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.786+0000 7f03d6e15740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558d18355eb0 0x558d18359b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:09.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.786+0000 7f03d6e15740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d183568a0 0x558d1835a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:09.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.786+0000 7f03d6e15740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558d1835ba10 0x558d18365740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:09.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.786+0000 7f03d6e15740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x558d1836e0f0 con 0x558d1835ba10 2024-04-17T06:01:09.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.786+0000 7f03d6e15740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x558d1836da20 con 0x558d18355eb0 2024-04-17T06:01:09.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.786+0000 7f03d6e15740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558d1836e7c0 con 0x558d183568a0 2024-04-17T06:01:09.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.786+0000 7f03d6e15740 10 client.0 _ll_register_callbacks cb 0x7f03c0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:01:09.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.786+0000 7f03d27c7640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558d1835ba10 0x558d18365740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:09.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.786+0000 7f03d27c7640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558d1835ba10 0x558d18365740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40594/0 (socket says 172.21.15.47:40594) 2024-04-17T06:01:09.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.786+0000 7f03d27c7640 1 -- 172.21.15.47:0/589811985 learned_addr learned my addr 172.21.15.47:0/589811985 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:01:09.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.786+0000 7f03d1fc6640 1 --2- 172.21.15.47:0/589811985 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558d18355eb0 0x558d18359b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:09.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.787+0000 7f03d17c5640 1 --2- 172.21.15.47:0/589811985 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d183568a0 0x558d1835a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:09.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.787+0000 7f03d27c7640 1 -- 172.21.15.47:0/589811985 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d183568a0 msgr2=0x558d1835a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:01:09.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.787+0000 7f03d27c7640 1 --2- 172.21.15.47:0/589811985 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d183568a0 0x558d1835a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:09.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.787+0000 7f03d27c7640 1 -- 172.21.15.47:0/589811985 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558d18355eb0 msgr2=0x558d18359b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:01:09.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.787+0000 7f03d27c7640 1 --2- 172.21.15.47:0/589811985 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558d18355eb0 0x558d18359b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:09.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.787+0000 7f03d27c7640 1 -- 172.21.15.47:0/589811985 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d18365d30 con 0x558d1835ba10 2024-04-17T06:01:09.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.787+0000 7f03d1fc6640 1 --2- 172.21.15.47:0/589811985 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558d18355eb0 0x558d18359b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:01:09.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.787+0000 7f03d27c7640 1 --2- 172.21.15.47:0/589811985 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558d1835ba10 0x558d18365740 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f03c400ead0 tx=0x7f03c400eed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec60add4cce9c186 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:01:09.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.787+0000 7f03b57fa640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-17T06:01:09.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.787+0000 7f03b57fa640 1 -- 172.21.15.47:0/589811985 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c4023030 con 0x558d1835ba10 2024-04-17T06:01:09.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.787+0000 7f03b57fa640 1 -- 172.21.15.47:0/589811985 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f03c4011b50 con 0x558d1835ba10 2024-04-17T06:01:09.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.787+0000 7f03d6e15740 1 -- 172.21.15.47:0/589811985 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03c00035f0 con 0x558d1835ba10 2024-04-17T06:01:09.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.787+0000 7f03b57fa640 1 -- 172.21.15.47:0/589811985 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f03c402f050 con 0x558d1835ba10 2024-04-17T06:01:09.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.787+0000 7f03d6e15740 10 client.15276 fetch_fsmap learned FSMap version 21 2024-04-17T06:01:09.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.787+0000 7f03d6e15740 1 -- 172.21.15.47:0/589811985 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x558d183d51b0 con 0x558d1835ba10 2024-04-17T06:01:09.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.787+0000 7f03b57fa640 1 -- 172.21.15.47:0/589811985 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03c00035f0 con 0x558d1835ba10 2024-04-17T06:01:09.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.788+0000 7f03b57fa640 1 -- 172.21.15.47:0/589811985 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 21) ==== 8339+0+0 (secure 0 0 0) 0x7f03c401c050 con 0x558d1835ba10 2024-04-17T06:01:09.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.788+0000 7f03d6e15740 10 client.15276 fetch_fsmap finished waiting for FSMap version 21 2024-04-17T06:01:09.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.788+0000 7f03d6e15740 10 client.15276 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-17T06:01:09.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.788+0000 7f03d6e15740 20 client.15276 populate_metadata read hostname 'smithi047' 2024-04-17T06:01:09.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.788+0000 7f03d6e15740 1 --2- 172.21.15.47:0/589811985 >> [v2:172.21.15.98:6836/1117440719,v1:172.21.15.98:6837/1117440719] conn(0x558d183d9dd0 0x558d183d8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:09.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.788+0000 7f03d6e15740 4 client.15276 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-17T06:01:09.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.788+0000 7f03d6e15740 1 -- 172.21.15.47:0/589811985 --> [v2:172.21.15.98:6836/1117440719,v1:172.21.15.98:6837/1117440719] -- command(tid 0: {"prefix": "status"}) -- 0x558d182c74e0 con 0x558d183d9dd0 2024-04-17T06:01:09.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.788+0000 7f03d1fc6640 1 --2- 172.21.15.47:0/589811985 >> [v2:172.21.15.98:6836/1117440719,v1:172.21.15.98:6837/1117440719] conn(0x558d183d9dd0 0x558d183d8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:09.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.788+0000 7f03b57fa640 10 client.15276 ms_handle_connect on v2:172.21.15.98:6836/1117440719 2024-04-17T06:01:09.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.788+0000 7f03d1fc6640 1 --2- 172.21.15.47:0/589811985 >> [v2:172.21.15.98:6836/1117440719,v1:172.21.15.98:6837/1117440719] conn(0x558d183d9dd0 0x558d183d8ae0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=497e0bf83e90afab server_cookie=dec83bb2dc714fc0 in_seq=0 out_seq=0 2024-04-17T06:01:09.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.788+0000 7f03b57fa640 1 -- 172.21.15.47:0/589811985 <== mds.1 v2:172.21.15.98:6836/1117440719 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x558d182c74e0 con 0x558d183d9dd0 2024-04-17T06:01:09.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.788+0000 7f03b57fa640 10 client.15276 handle_command_reply: tid=0 2024-04-17T06:01:09.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.788+0000 7f03d6e15740 1 client.15276 shutdown 2024-04-17T06:01:09.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.789+0000 7f03d6e15740 1 -- 172.21.15.47:0/589811985 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558d1835ba10 msgr2=0x558d18365740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:09.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.789+0000 7f03d6e15740 1 --2- 172.21.15.47:0/589811985 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558d1835ba10 0x558d18365740 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f03c400ead0 tx=0x7f03c400eed0 comp rx=0 tx=0).stop 2024-04-17T06:01:09.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.789+0000 7f03d6e15740 1 -- 172.21.15.47:0/589811985 shutdown_connections 2024-04-17T06:01:09.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.789+0000 7f03d6e15740 1 --2- 172.21.15.47:0/589811985 >> [v2:172.21.15.98:6836/1117440719,v1:172.21.15.98:6837/1117440719] conn(0x558d183d9dd0 0x558d183d8ae0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:09.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.789+0000 7f03d6e15740 1 --2- 172.21.15.47:0/589811985 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558d1835ba10 0x558d18365740 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:09.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.789+0000 7f03d6e15740 1 --2- 172.21.15.47:0/589811985 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d183568a0 0x558d1835a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:09.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.789+0000 7f03d6e15740 1 --2- 172.21.15.47:0/589811985 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558d18355eb0 0x558d18359b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:09.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.789+0000 7f03b57fa640 0 client.15276 ms_handle_reset on v2:172.21.15.98:6836/1117440719 2024-04-17T06:01:09.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.789+0000 7f03d6e15740 1 -- 172.21.15.47:0/589811985 >> 172.21.15.47:0/589811985 conn(0x558d18351420 msgr2=0x558d18351800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:01:09.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.789+0000 7f03d6e15740 1 -- 172.21.15.47:0/589811985 shutdown_connections 2024-04-17T06:01:09.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.789+0000 7f03d6e15740 1 -- 172.21.15.47:0/589811985 wait complete. 2024-04-17T06:01:09.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.789+0000 7f03d6e15740 20 client.15276 trim_cache size 0 max 16384 2024-04-17T06:01:09.795 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-17T06:01:09.797 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_fsid": "565db2d8-fc7e-11ee-bc91-c7b262605968", 2024-04-17T06:01:09.797 INFO:teuthology.orchestra.run.smithi047.stdout: "whoami": 1, 2024-04-17T06:01:09.798 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 24479, 2024-04-17T06:01:09.798 INFO:teuthology.orchestra.run.smithi047.stdout: "want_state": "up:active", 2024-04-17T06:01:09.798 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "up:active", 2024-04-17T06:01:09.798 INFO:teuthology.orchestra.run.smithi047.stdout: "fs_name": "cephfs", 2024-04-17T06:01:09.798 INFO:teuthology.orchestra.run.smithi047.stdout: "rank_uptime": 112.172783495, 2024-04-17T06:01:09.798 INFO:teuthology.orchestra.run.smithi047.stdout: "mdsmap_epoch": 21, 2024-04-17T06:01:09.798 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch": 88, 2024-04-17T06:01:09.798 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch_barrier": 86, 2024-04-17T06:01:09.798 INFO:teuthology.orchestra.run.smithi047.stdout: "uptime": 132.62977312999999 2024-04-17T06:01:09.798 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-17T06:01:09.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.789+0000 7f03d422a640 1 -- 172.21.15.47:0/4164383573 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f03b80a3bb0 msgr2=0x7f03b80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:09.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.789+0000 7f03d422a640 1 --2- 172.21.15.47:0/4164383573 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f03b80a3bb0 0x7f03b80a6070 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x558d18357100 tx=0x7f03c4007390 comp rx=0 tx=0).stop 2024-04-17T06:01:09.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.789+0000 7f03d422a640 1 -- 172.21.15.47:0/4164383573 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03cc100ea0 msgr2=0x7f03cc1971a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:09.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.789+0000 7f03d422a640 1 --2- 172.21.15.47:0/4164383573 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03cc100ea0 0x7f03cc1971a0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f03c800d950 tx=0x7f03c800de10 comp rx=0 tx=0).stop 2024-04-17T06:01:09.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.789+0000 7f03d422a640 1 -- 172.21.15.47:0/4164383573 shutdown_connections 2024-04-17T06:01:09.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.789+0000 7f03d422a640 1 --2- 172.21.15.47:0/4164383573 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f03b80a3bb0 0x7f03b80a6070 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:09.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.789+0000 7f03d422a640 1 --2- 172.21.15.47:0/4164383573 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f03cc1029b0 0x7f03cc19e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:09.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.789+0000 7f03d422a640 1 --2- 172.21.15.47:0/4164383573 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03cc101ff0 0x7f03cc1976e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:09.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.789+0000 7f03d422a640 1 --2- 172.21.15.47:0/4164383573 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03cc100ea0 0x7f03cc1971a0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:09.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.789+0000 7f03d422a640 1 -- 172.21.15.47:0/4164383573 >> 172.21.15.47:0/4164383573 conn(0x7f03cc0fc410 msgr2=0x7f03cc105380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:01:09.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.789+0000 7f03d422a640 1 -- 172.21.15.47:0/4164383573 shutdown_connections 2024-04-17T06:01:09.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.789+0000 7f03d422a640 1 -- 172.21.15.47:0/4164383573 wait complete. 2024-04-17T06:01:09.801 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-17T06:01:09.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.909+0000 7f5d0ff96640 1 -- 172.21.15.47:0/3618783908 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5d08101fc0 msgr2=0x7f5d081023e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:09.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.909+0000 7f5d0ff96640 1 --2- 172.21.15.47:0/3618783908 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5d08101fc0 0x7f5d081023e0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f5d04009900 tx=0x7f5d0402bc30 comp rx=0 tx=0).stop 2024-04-17T06:01:09.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.909+0000 7f5d0ff96640 1 -- 172.21.15.47:0/3618783908 shutdown_connections 2024-04-17T06:01:09.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.909+0000 7f5d0ff96640 1 --2- 172.21.15.47:0/3618783908 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5d08103b70 0x7f5d0810a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:09.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.909+0000 7f5d0ff96640 1 --2- 172.21.15.47:0/3618783908 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5d081031b0 0x7f5d08103630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:09.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.909+0000 7f5d0ff96640 1 --2- 172.21.15.47:0/3618783908 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5d08101fc0 0x7f5d081023e0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:09.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.909+0000 7f5d0ff96640 1 -- 172.21.15.47:0/3618783908 >> 172.21.15.47:0/3618783908 conn(0x7f5d080fd530 msgr2=0x7f5d080ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:01:09.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.909+0000 7f5d0ff96640 1 -- 172.21.15.47:0/3618783908 shutdown_connections 2024-04-17T06:01:09.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.909+0000 7f5d0ff96640 1 -- 172.21.15.47:0/3618783908 wait complete. 2024-04-17T06:01:09.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.910+0000 7f5d0ff96640 1 Processor -- start 2024-04-17T06:01:09.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.910+0000 7f5d0ff96640 1 -- start start 2024-04-17T06:01:09.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.910+0000 7f5d0ff96640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5d08101fc0 0x7f5d08198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:09.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.910+0000 7f5d0ff96640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5d081031b0 0x7f5d081987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:09.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.910+0000 7f5d0ff96640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5d08103b70 0x7f5d0819f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:09.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.910+0000 7f5d0ff96640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f5d0810dca0 con 0x7f5d08103b70 2024-04-17T06:01:09.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.910+0000 7f5d0ff96640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5d0810db20 con 0x7f5d08101fc0 2024-04-17T06:01:09.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.910+0000 7f5d0ff96640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5d0810de20 con 0x7f5d081031b0 2024-04-17T06:01:09.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.910+0000 7f5d0e533640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5d08103b70 0x7f5d0819f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:09.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.910+0000 7f5d0e533640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5d08103b70 0x7f5d0819f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40604/0 (socket says 172.21.15.47:40604) 2024-04-17T06:01:09.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.910+0000 7f5d0e533640 1 -- 172.21.15.47:0/2534174526 learned_addr learned my addr 172.21.15.47:0/2534174526 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:01:09.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.910+0000 7f5d0d531640 1 --2- 172.21.15.47:0/2534174526 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5d081031b0 0x7f5d081987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:09.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.910+0000 7f5d0dd32640 1 --2- 172.21.15.47:0/2534174526 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5d08101fc0 0x7f5d08198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:09.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.910+0000 7f5d0e533640 1 -- 172.21.15.47:0/2534174526 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5d081031b0 msgr2=0x7f5d081987a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:09.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.910+0000 7f5d0e533640 1 --2- 172.21.15.47:0/2534174526 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5d081031b0 0x7f5d081987a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:09.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.910+0000 7f5d0e533640 1 -- 172.21.15.47:0/2534174526 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5d08101fc0 msgr2=0x7f5d08198260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:09.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.910+0000 7f5d0e533640 1 --2- 172.21.15.47:0/2534174526 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5d08101fc0 0x7f5d08198260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:09.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.910+0000 7f5d0e533640 1 -- 172.21.15.47:0/2534174526 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d0819fe80 con 0x7f5d08103b70 2024-04-17T06:01:09.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.911+0000 7f5d0e533640 1 --2- 172.21.15.47:0/2534174526 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5d08103b70 0x7f5d0819f810 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f5d0000cce0 tx=0x7f5d0000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:01:09.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.911+0000 7f5cfeffd640 1 -- 172.21.15.47:0/2534174526 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d0000f450 con 0x7f5d08103b70 2024-04-17T06:01:09.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.911+0000 7f5d0ff96640 1 -- 172.21.15.47:0/2534174526 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d081a0170 con 0x7f5d08103b70 2024-04-17T06:01:09.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.911+0000 7f5cfeffd640 1 -- 172.21.15.47:0/2534174526 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d0000fad0 con 0x7f5d08103b70 2024-04-17T06:01:09.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.911+0000 7f5cfeffd640 1 -- 172.21.15.47:0/2534174526 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d00018c10 con 0x7f5d08103b70 2024-04-17T06:01:09.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.911+0000 7f5d0ff96640 1 -- 172.21.15.47:0/2534174526 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d081a0670 con 0x7f5d08103b70 2024-04-17T06:01:09.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.914+0000 7f5cfeffd640 1 -- 172.21.15.47:0/2534174526 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f5d0000f5f0 con 0x7f5d08103b70 2024-04-17T06:01:09.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.914+0000 7f5d12b7e740 1 Processor -- start 2024-04-17T06:01:09.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.914+0000 7f5d12b7e740 1 -- start start 2024-04-17T06:01:09.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.914+0000 7f5d12b7e740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562f2f9be0b0 0x562f2f9be4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:09.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.914+0000 7f5d12b7e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562f2f9beaa0 0x562f2f9c35a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:09.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.914+0000 7f5d12b7e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562f2f9c3c10 0x562f2f9c60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:09.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.914+0000 7f5d12b7e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x562f2f8cd240 con 0x562f2f9beaa0 2024-04-17T06:01:09.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.914+0000 7f5d12b7e740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x562f2f92a9f0 con 0x562f2f9be0b0 2024-04-17T06:01:09.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.914+0000 7f5d12b7e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x562f2f8e59e0 con 0x562f2f9c3c10 2024-04-17T06:01:09.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.914+0000 7f5d0dd32640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562f2f9beaa0 0x562f2f9c35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:09.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.914+0000 7f5d0dd32640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562f2f9beaa0 0x562f2f9c35a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40612/0 (socket says 172.21.15.47:40612) 2024-04-17T06:01:09.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.914+0000 7f5d0dd32640 1 -- 172.21.15.47:0/3065437516 learned_addr learned my addr 172.21.15.47:0/3065437516 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:01:09.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.914+0000 7f5d0d531640 1 --2- 172.21.15.47:0/3065437516 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562f2f9be0b0 0x562f2f9be4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:09.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.915+0000 7f5d0d531640 1 --2- 172.21.15.47:0/3065437516 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562f2f9c3c10 0x562f2f9c60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:09.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.915+0000 7f5cfeffd640 1 --2- 172.21.15.47:0/2534174526 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f5cf40a3bb0 0x7f5cf40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:09.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.915+0000 7f5d0dd32640 1 -- 172.21.15.47:0/3065437516 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562f2f9c3c10 msgr2=0x562f2f9c60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:01:09.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.915+0000 7f5d0dd32640 1 --2- 172.21.15.47:0/3065437516 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562f2f9c3c10 0x562f2f9c60a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:09.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.915+0000 7f5d0dd32640 1 -- 172.21.15.47:0/3065437516 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562f2f9be0b0 msgr2=0x562f2f9be4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:01:09.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.915+0000 7f5d0dd32640 1 --2- 172.21.15.47:0/3065437516 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562f2f9be0b0 0x562f2f9be4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:09.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.915+0000 7f5d0dd32640 1 -- 172.21.15.47:0/3065437516 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f2f8d8bc0 con 0x562f2f9beaa0 2024-04-17T06:01:09.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.915+0000 7f5d0e533640 1 --2- 172.21.15.47:0/2534174526 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f5cf40a3bb0 0x7f5cf40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:09.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.915+0000 7f5d0dd32640 1 --2- 172.21.15.47:0/3065437516 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562f2f9beaa0 0x562f2f9c35a0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f5d04006010 tx=0x7f5d04005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=258672d825a2643a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:01:09.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.915+0000 7f5cfeffd640 1 -- 172.21.15.47:0/2534174526 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f5d000d0a90 con 0x7f5d08103b70 2024-04-17T06:01:09.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.915+0000 7f5d0e533640 1 --2- 172.21.15.47:0/2534174526 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f5cf40a3bb0 0x7f5cf40a6070 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5d000991e0 tx=0x7f5d0006f8c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:01:09.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.916+0000 7f5cf3fff640 1 -- 172.21.15.47:0/3065437516 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d04044020 con 0x562f2f9beaa0 2024-04-17T06:01:09.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.916+0000 7f5cf3fff640 1 -- 172.21.15.47:0/3065437516 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d0402cb80 con 0x562f2f9beaa0 2024-04-17T06:01:09.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.916+0000 7f5cf3fff640 1 -- 172.21.15.47:0/3065437516 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d04004920 con 0x562f2f9beaa0 2024-04-17T06:01:09.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.917+0000 7f5d12b7e740 1 -- 172.21.15.47:0/3065437516 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562f2f9beaa0 msgr2=0x562f2f9c35a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:09.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.917+0000 7f5d12b7e740 1 --2- 172.21.15.47:0/3065437516 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562f2f9beaa0 0x562f2f9c35a0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f5d04006010 tx=0x7f5d04005d50 comp rx=0 tx=0).stop 2024-04-17T06:01:09.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.917+0000 7f5d12b7e740 1 -- 172.21.15.47:0/3065437516 shutdown_connections 2024-04-17T06:01:09.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.917+0000 7f5d12b7e740 1 --2- 172.21.15.47:0/3065437516 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562f2f9c3c10 0x562f2f9c60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:09.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.917+0000 7f5d12b7e740 1 --2- 172.21.15.47:0/3065437516 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562f2f9beaa0 0x562f2f9c35a0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:09.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.917+0000 7f5d12b7e740 1 --2- 172.21.15.47:0/3065437516 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562f2f9be0b0 0x562f2f9be4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:09.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.917+0000 7f5d12b7e740 1 -- 172.21.15.47:0/3065437516 >> 172.21.15.47:0/3065437516 conn(0x562f2f9b5210 msgr2=0x562f2f9b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:01:09.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.917+0000 7f5d12b7e740 1 -- 172.21.15.47:0/3065437516 shutdown_connections 2024-04-17T06:01:09.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.917+0000 7f5d12b7e740 1 -- 172.21.15.47:0/3065437516 wait complete. 2024-04-17T06:01:09.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.918+0000 7f5d12b7e740 1 -- start start 2024-04-17T06:01:09.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.918+0000 7f5d12b7e740 1 Processor -- start 2024-04-17T06:01:09.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.918+0000 7f5d12b7e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562f2f9be0b0 0x562f2f9c8270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:09.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.918+0000 7f5d12b7e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562f2f9beaa0 0x562f2f9c87b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:09.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.918+0000 7f5d12b7e740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562f2f9c3c10 0x562f2f9cf7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:09.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.918+0000 7f5d12b7e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x562f2f9240f0 con 0x562f2f9be0b0 2024-04-17T06:01:09.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.918+0000 7f5d12b7e740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x562f2f91edd0 con 0x562f2f9c3c10 2024-04-17T06:01:09.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.918+0000 7f5d12b7e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x562f2f92ccd0 con 0x562f2f9beaa0 2024-04-17T06:01:09.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.918+0000 7f5d12b7e740 10 client.0 _ll_register_callbacks cb 0x562f2f915570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:01:09.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.918+0000 7f5d0d531640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562f2f9be0b0 0x562f2f9c8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:09.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.918+0000 7f5d0d531640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562f2f9be0b0 0x562f2f9c8270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40616/0 (socket says 172.21.15.47:40616) 2024-04-17T06:01:09.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.918+0000 7f5d0d531640 1 -- 172.21.15.47:0/1458369810 learned_addr learned my addr 172.21.15.47:0/1458369810 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:01:09.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.918+0000 7f5d0dd32640 1 --2- 172.21.15.47:0/1458369810 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562f2f9beaa0 0x562f2f9c87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:09.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.918+0000 7f5d0d531640 1 --2- 172.21.15.47:0/1458369810 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562f2f9c3c10 0x562f2f9cf7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:09.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.918+0000 7f5d0d531640 1 -- 172.21.15.47:0/1458369810 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562f2f9beaa0 msgr2=0x562f2f9c87b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:01:09.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.918+0000 7f5d0d531640 1 --2- 172.21.15.47:0/1458369810 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562f2f9beaa0 0x562f2f9c87b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:09.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.918+0000 7f5d0d531640 1 -- 172.21.15.47:0/1458369810 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562f2f9c3c10 msgr2=0x562f2f9cf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:01:09.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.918+0000 7f5d0d531640 1 --2- 172.21.15.47:0/1458369810 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562f2f9c3c10 0x562f2f9cf7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:09.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.918+0000 7f5d0d531640 1 -- 172.21.15.47:0/1458369810 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f2f6322d0 con 0x562f2f9be0b0 2024-04-17T06:01:09.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.919+0000 7f5d0d531640 1 --2- 172.21.15.47:0/1458369810 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562f2f9be0b0 0x562f2f9c8270 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f5cf8004610 tx=0x7f5cf8014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce684bd088f4fffb server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:01:09.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.919+0000 7f5cf17fa640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-17T06:01:09.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.919+0000 7f5cf17fa640 1 -- 172.21.15.47:0/1458369810 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cf80128d0 con 0x562f2f9be0b0 2024-04-17T06:01:09.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.919+0000 7f5cf17fa640 1 -- 172.21.15.47:0/1458369810 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f5cf8012a70 con 0x562f2f9be0b0 2024-04-17T06:01:09.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.919+0000 7f5d12b7e740 1 -- 172.21.15.47:0/1458369810 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d0402cb80 con 0x562f2f9be0b0 2024-04-17T06:01:09.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.919+0000 7f5cf17fa640 1 -- 172.21.15.47:0/1458369810 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f5cf8012c70 con 0x562f2f9be0b0 2024-04-17T06:01:09.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.919+0000 7f5d12b7e740 10 client.15288 fetch_fsmap learned FSMap version 21 2024-04-17T06:01:09.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.919+0000 7f5d12b7e740 1 -- 172.21.15.47:0/1458369810 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x562f2f6fc6b0 con 0x562f2f9be0b0 2024-04-17T06:01:09.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.919+0000 7f5cf17fa640 1 -- 172.21.15.47:0/1458369810 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d0402cb80 con 0x562f2f9be0b0 2024-04-17T06:01:09.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.919+0000 7f5cf17fa640 1 -- 172.21.15.47:0/1458369810 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 21) ==== 8339+0+0 (secure 0 0 0) 0x7f5cf802c3f0 con 0x562f2f9be0b0 2024-04-17T06:01:09.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.919+0000 7f5d12b7e740 10 client.15288 fetch_fsmap finished waiting for FSMap version 21 2024-04-17T06:01:09.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.919+0000 7f5d12b7e740 10 client.15288 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-17T06:01:09.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.919+0000 7f5d12b7e740 20 client.15288 populate_metadata read hostname 'smithi047' 2024-04-17T06:01:09.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.919+0000 7f5d12b7e740 1 --2- 172.21.15.47:0/1458369810 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x562f2f9d4f10 0x562f2f9d6380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:09.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.919+0000 7f5d12b7e740 4 client.15288 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T06:01:09.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.919+0000 7f5d12b7e740 1 -- 172.21.15.47:0/1458369810 --> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562f2f8a64c0 con 0x562f2f9d4f10 2024-04-17T06:01:09.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.920+0000 7f5d0dd32640 1 --2- 172.21.15.47:0/1458369810 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x562f2f9d4f10 0x562f2f9d6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:09.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.920+0000 7f5d0dd32640 1 --2- 172.21.15.47:0/1458369810 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x562f2f9d4f10 0x562f2f9d6380 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8423ef1415bcaade server_cookie=efbc6e1db8d4b512 in_seq=0 out_seq=0 2024-04-17T06:01:09.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.920+0000 7f5cf17fa640 10 client.15288 ms_handle_connect on v2:172.21.15.98:6838/2841287917 2024-04-17T06:01:09.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.921+0000 7f5cf17fa640 1 -- 172.21.15.47:0/1458369810 <== mds.0 v2:172.21.15.98:6838/2841287917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562f2f8a64c0 con 0x562f2f9d4f10 2024-04-17T06:01:09.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.921+0000 7f5cf17fa640 10 client.15288 handle_command_reply: tid=0 2024-04-17T06:01:09.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.921+0000 7f5d12b7e740 1 client.15288 shutdown 2024-04-17T06:01:09.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.921+0000 7f5d12b7e740 1 -- 172.21.15.47:0/1458369810 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562f2f9be0b0 msgr2=0x562f2f9c8270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:09.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.921+0000 7f5d12b7e740 1 --2- 172.21.15.47:0/1458369810 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562f2f9be0b0 0x562f2f9c8270 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f5cf8004610 tx=0x7f5cf8014040 comp rx=0 tx=0).stop 2024-04-17T06:01:09.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.921+0000 7f5d12b7e740 1 -- 172.21.15.47:0/1458369810 shutdown_connections 2024-04-17T06:01:09.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.921+0000 7f5d12b7e740 1 --2- 172.21.15.47:0/1458369810 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x562f2f9d4f10 0x562f2f9d6380 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:09.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.921+0000 7f5d12b7e740 1 --2- 172.21.15.47:0/1458369810 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562f2f9c3c10 0x562f2f9cf7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:09.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.921+0000 7f5cf17fa640 0 client.15288 ms_handle_reset on v2:172.21.15.98:6838/2841287917 2024-04-17T06:01:09.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.921+0000 7f5d12b7e740 1 --2- 172.21.15.47:0/1458369810 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562f2f9beaa0 0x562f2f9c87b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:09.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.921+0000 7f5d12b7e740 1 --2- 172.21.15.47:0/1458369810 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562f2f9be0b0 0x562f2f9c8270 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:09.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.921+0000 7f5d12b7e740 1 -- 172.21.15.47:0/1458369810 >> 172.21.15.47:0/1458369810 conn(0x562f2f9b5210 msgr2=0x562f2f9b4350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:01:09.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.921+0000 7f5d12b7e740 1 -- 172.21.15.47:0/1458369810 shutdown_connections 2024-04-17T06:01:09.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.921+0000 7f5d12b7e740 1 -- 172.21.15.47:0/1458369810 wait complete. 2024-04-17T06:01:09.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.921+0000 7f5d12b7e740 20 client.15288 trim_cache size 0 max 16384 2024-04-17T06:01:09.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.939+0000 7f5d12b7e740 1 Processor -- start 2024-04-17T06:01:09.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.939+0000 7f5d12b7e740 1 -- start start 2024-04-17T06:01:09.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.939+0000 7f5d12b7e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562f2f9be0b0 0x562f2f9c7e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:09.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.939+0000 7f5d12b7e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562f2f9beaa0 0x562f2f9c83c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:09.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.939+0000 7f5d12b7e740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562f2f9c3c10 0x562f2f9b3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:09.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.939+0000 7f5d12b7e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x562f2f9d46c0 con 0x562f2f9beaa0 2024-04-17T06:01:09.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.939+0000 7f5d12b7e740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x562f2f9d4d90 con 0x562f2f9c3c10 2024-04-17T06:01:09.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.939+0000 7f5d12b7e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x562f2f9d5460 con 0x562f2f9be0b0 2024-04-17T06:01:09.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.939+0000 7f5d0dd32640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562f2f9beaa0 0x562f2f9c83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:09.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.940+0000 7f5d0dd32640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562f2f9beaa0 0x562f2f9c83c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40628/0 (socket says 172.21.15.47:40628) 2024-04-17T06:01:09.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.940+0000 7f5d0dd32640 1 -- 172.21.15.47:0/3406091345 learned_addr learned my addr 172.21.15.47:0/3406091345 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:01:09.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.940+0000 7f5d0d531640 1 --2- 172.21.15.47:0/3406091345 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562f2f9be0b0 0x562f2f9c7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:09.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.940+0000 7f5d0dd32640 1 -- 172.21.15.47:0/3406091345 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562f2f9be0b0 msgr2=0x562f2f9c7e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:01:09.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.940+0000 7f5d0dd32640 1 --2- 172.21.15.47:0/3406091345 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562f2f9be0b0 0x562f2f9c7e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:09.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.940+0000 7f5d0d531640 1 --2- 172.21.15.47:0/3406091345 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562f2f9c3c10 0x562f2f9b3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:09.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.940+0000 7f5d0dd32640 1 -- 172.21.15.47:0/3406091345 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562f2f9c3c10 msgr2=0x562f2f9b3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:01:09.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.940+0000 7f5d0dd32640 1 --2- 172.21.15.47:0/3406091345 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562f2f9c3c10 0x562f2f9b3ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:09.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.940+0000 7f5d0dd32640 1 -- 172.21.15.47:0/3406091345 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f2f9b5f80 con 0x562f2f9beaa0 2024-04-17T06:01:09.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.940+0000 7f5d0d531640 1 --2- 172.21.15.47:0/3406091345 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562f2f9c3c10 0x562f2f9b3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:01:09.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.940+0000 7f5d0dd32640 1 --2- 172.21.15.47:0/3406091345 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562f2f9beaa0 0x562f2f9c83c0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f5d0402c7b0 tx=0x7f5d0402df90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f28bd315a986d04 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:01:09.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.940+0000 7f5cf37fe640 1 -- 172.21.15.47:0/3406091345 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d04044040 con 0x562f2f9beaa0 2024-04-17T06:01:09.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.940+0000 7f5cf37fe640 1 -- 172.21.15.47:0/3406091345 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d0402cd00 con 0x562f2f9beaa0 2024-04-17T06:01:09.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.940+0000 7f5cf37fe640 1 -- 172.21.15.47:0/3406091345 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d040406d0 con 0x562f2f9beaa0 2024-04-17T06:01:09.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.941+0000 7f5d12b7e740 1 -- 172.21.15.47:0/3406091345 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562f2f9beaa0 msgr2=0x562f2f9c83c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:09.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.941+0000 7f5d12b7e740 1 --2- 172.21.15.47:0/3406091345 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562f2f9beaa0 0x562f2f9c83c0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f5d0402c7b0 tx=0x7f5d0402df90 comp rx=0 tx=0).stop 2024-04-17T06:01:09.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.941+0000 7f5d12b7e740 1 -- 172.21.15.47:0/3406091345 shutdown_connections 2024-04-17T06:01:09.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.941+0000 7f5d12b7e740 1 --2- 172.21.15.47:0/3406091345 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562f2f9c3c10 0x562f2f9b3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:09.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.941+0000 7f5d12b7e740 1 --2- 172.21.15.47:0/3406091345 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562f2f9beaa0 0x562f2f9c83c0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:09.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.941+0000 7f5d12b7e740 1 --2- 172.21.15.47:0/3406091345 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562f2f9be0b0 0x562f2f9c7e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:09.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.941+0000 7f5d12b7e740 1 -- 172.21.15.47:0/3406091345 >> 172.21.15.47:0/3406091345 conn(0x562f2f9b5210 msgr2=0x562f2f9b55f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:01:09.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.941+0000 7f5d12b7e740 1 -- 172.21.15.47:0/3406091345 shutdown_connections 2024-04-17T06:01:09.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.941+0000 7f5d12b7e740 1 -- 172.21.15.47:0/3406091345 wait complete. 2024-04-17T06:01:09.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.941+0000 7f5d12b7e740 1 -- start start 2024-04-17T06:01:09.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.941+0000 7f5d12b7e740 1 Processor -- start 2024-04-17T06:01:09.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.941+0000 7f5d12b7e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562f2f9be0b0 0x562f2f9c1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:09.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.941+0000 7f5d12b7e740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562f2f9beaa0 0x562f2f9c21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:09.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.941+0000 7f5d12b7e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562f2f9c3c10 0x562f2f9cda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:09.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.941+0000 7f5d12b7e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x562f2f9d6200 con 0x562f2f9be0b0 2024-04-17T06:01:09.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.941+0000 7f5d12b7e740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x562f2f9d5b30 con 0x562f2f9beaa0 2024-04-17T06:01:09.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.941+0000 7f5d12b7e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x562f2f9d68d0 con 0x562f2f9c3c10 2024-04-17T06:01:09.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.941+0000 7f5d12b7e740 10 client.0 _ll_register_callbacks cb 0x7f5cf8004250 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:01:09.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.941+0000 7f5d0d531640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562f2f9be0b0 0x562f2f9c1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:09.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.942+0000 7f5d0d531640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562f2f9be0b0 0x562f2f9c1ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40642/0 (socket says 172.21.15.47:40642) 2024-04-17T06:01:09.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.942+0000 7f5d0d531640 1 -- 172.21.15.47:0/1927630961 learned_addr learned my addr 172.21.15.47:0/1927630961 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:01:09.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.942+0000 7f5d0dd32640 1 --2- 172.21.15.47:0/1927630961 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562f2f9beaa0 0x562f2f9c21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:09.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.942+0000 7f5d0d531640 1 -- 172.21.15.47:0/1927630961 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562f2f9c3c10 msgr2=0x562f2f9cda10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:01:09.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.942+0000 7f5d0d531640 1 --2- 172.21.15.47:0/1927630961 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562f2f9c3c10 0x562f2f9cda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:09.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.942+0000 7f5d0d531640 1 -- 172.21.15.47:0/1927630961 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562f2f9beaa0 msgr2=0x562f2f9c21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:01:09.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.942+0000 7f5d0d531640 1 --2- 172.21.15.47:0/1927630961 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562f2f9beaa0 0x562f2f9c21e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:09.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.942+0000 7f5d0d531640 1 -- 172.21.15.47:0/1927630961 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f2f9ce050 con 0x562f2f9be0b0 2024-04-17T06:01:09.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.942+0000 7f5d0dd32640 1 --2- 172.21.15.47:0/1927630961 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562f2f9beaa0 0x562f2f9c21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:01:09.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.942+0000 7f5d0d531640 1 --2- 172.21.15.47:0/1927630961 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562f2f9be0b0 0x562f2f9c1ca0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f5cf8004610 tx=0x7f5cf8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2041ec118edbd5f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:01:09.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.942+0000 7f5cf17fa640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-17T06:01:09.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.942+0000 7f5cf17fa640 1 -- 172.21.15.47:0/1927630961 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cf802ca00 con 0x562f2f9be0b0 2024-04-17T06:01:09.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.942+0000 7f5cf17fa640 1 -- 172.21.15.47:0/1927630961 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f5cf802cba0 con 0x562f2f9be0b0 2024-04-17T06:01:09.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.942+0000 7f5d12b7e740 1 -- 172.21.15.47:0/1927630961 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d0402cd00 con 0x562f2f9be0b0 2024-04-17T06:01:09.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.942+0000 7f5cf17fa640 1 -- 172.21.15.47:0/1927630961 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f5cf803d050 con 0x562f2f9be0b0 2024-04-17T06:01:09.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.942+0000 7f5d12b7e740 10 client.15294 fetch_fsmap learned FSMap version 21 2024-04-17T06:01:09.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.942+0000 7f5d12b7e740 1 -- 172.21.15.47:0/1927630961 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x562f2fa3af50 con 0x562f2f9be0b0 2024-04-17T06:01:09.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.942+0000 7f5cf17fa640 1 -- 172.21.15.47:0/1927630961 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d0402cd00 con 0x562f2f9be0b0 2024-04-17T06:01:09.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.943+0000 7f5cf17fa640 1 -- 172.21.15.47:0/1927630961 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 21) ==== 8339+0+0 (secure 0 0 0) 0x7f5cf8021b10 con 0x562f2f9be0b0 2024-04-17T06:01:09.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.943+0000 7f5d12b7e740 10 client.15294 fetch_fsmap finished waiting for FSMap version 21 2024-04-17T06:01:09.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.943+0000 7f5d12b7e740 10 client.15294 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-17T06:01:09.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.943+0000 7f5d12b7e740 20 client.15294 populate_metadata read hostname 'smithi047' 2024-04-17T06:01:09.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.943+0000 7f5d12b7e740 1 --2- 172.21.15.47:0/1927630961 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x562f2f9d4f10 0x562f2fa3c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:09.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.943+0000 7f5d12b7e740 4 client.15294 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-17T06:01:09.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.943+0000 7f5d12b7e740 1 -- 172.21.15.47:0/1927630961 --> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] -- command(tid 0: {"prefix": "status"}) -- 0x562f2f92d4e0 con 0x562f2f9d4f10 2024-04-17T06:01:09.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.943+0000 7f5d0dd32640 1 --2- 172.21.15.47:0/1927630961 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x562f2f9d4f10 0x562f2fa3c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:09.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.943+0000 7f5cf17fa640 10 client.15294 ms_handle_connect on v2:172.21.15.98:6838/2841287917 2024-04-17T06:01:09.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.943+0000 7f5d0dd32640 1 --2- 172.21.15.47:0/1927630961 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x562f2f9d4f10 0x562f2fa3c010 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=897dff2bb8015da6 server_cookie=20e33566c6018829 in_seq=0 out_seq=0 2024-04-17T06:01:09.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.943+0000 7f5cf17fa640 1 -- 172.21.15.47:0/1927630961 <== mds.0 v2:172.21.15.98:6838/2841287917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x562f2f92d4e0 con 0x562f2f9d4f10 2024-04-17T06:01:09.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.943+0000 7f5cf17fa640 10 client.15294 handle_command_reply: tid=0 2024-04-17T06:01:09.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.943+0000 7f5d12b7e740 1 client.15294 shutdown 2024-04-17T06:01:09.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.944+0000 7f5d12b7e740 1 -- 172.21.15.47:0/1927630961 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562f2f9be0b0 msgr2=0x562f2f9c1ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:09.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.944+0000 7f5d12b7e740 1 --2- 172.21.15.47:0/1927630961 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562f2f9be0b0 0x562f2f9c1ca0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f5cf8004610 tx=0x7f5cf8000bc0 comp rx=0 tx=0).stop 2024-04-17T06:01:09.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.944+0000 7f5d12b7e740 1 -- 172.21.15.47:0/1927630961 shutdown_connections 2024-04-17T06:01:09.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.944+0000 7f5d12b7e740 1 --2- 172.21.15.47:0/1927630961 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x562f2f9d4f10 0x562f2fa3c010 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:09.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.944+0000 7f5d12b7e740 1 --2- 172.21.15.47:0/1927630961 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562f2f9c3c10 0x562f2f9cda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:09.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.944+0000 7f5d12b7e740 1 --2- 172.21.15.47:0/1927630961 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562f2f9beaa0 0x562f2f9c21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:09.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.944+0000 7f5d12b7e740 1 --2- 172.21.15.47:0/1927630961 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562f2f9be0b0 0x562f2f9c1ca0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:09.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.944+0000 7f5cf17fa640 0 client.15294 ms_handle_reset on v2:172.21.15.98:6838/2841287917 2024-04-17T06:01:09.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.944+0000 7f5d12b7e740 1 -- 172.21.15.47:0/1927630961 >> 172.21.15.47:0/1927630961 conn(0x562f2f9b5210 msgr2=0x562f2f9c7e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:01:09.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.944+0000 7f5d12b7e740 1 -- 172.21.15.47:0/1927630961 shutdown_connections 2024-04-17T06:01:09.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.944+0000 7f5d12b7e740 1 -- 172.21.15.47:0/1927630961 wait complete. 2024-04-17T06:01:09.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.944+0000 7f5d12b7e740 20 client.15294 trim_cache size 0 max 16384 2024-04-17T06:01:09.951 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-17T06:01:09.952 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_fsid": "565db2d8-fc7e-11ee-bc91-c7b262605968", 2024-04-17T06:01:09.952 INFO:teuthology.orchestra.run.smithi047.stdout: "whoami": 0, 2024-04-17T06:01:09.952 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 24518, 2024-04-17T06:01:09.952 INFO:teuthology.orchestra.run.smithi047.stdout: "want_state": "up:active", 2024-04-17T06:01:09.952 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "up:active", 2024-04-17T06:01:09.952 INFO:teuthology.orchestra.run.smithi047.stdout: "fs_name": "cephfs", 2024-04-17T06:01:09.952 INFO:teuthology.orchestra.run.smithi047.stdout: "rank_uptime": 115.379192608, 2024-04-17T06:01:09.952 INFO:teuthology.orchestra.run.smithi047.stdout: "mdsmap_epoch": 21, 2024-04-17T06:01:09.952 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch": 88, 2024-04-17T06:01:09.952 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch_barrier": 86, 2024-04-17T06:01:09.952 INFO:teuthology.orchestra.run.smithi047.stdout: "uptime": 121.426850715 2024-04-17T06:01:09.952 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-17T06:01:09.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.944+0000 7f5d0ff96640 1 -- 172.21.15.47:0/2534174526 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f5cf40a3bb0 msgr2=0x7f5cf40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:09.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.944+0000 7f5d0ff96640 1 --2- 172.21.15.47:0/2534174526 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f5cf40a3bb0 0x7f5cf40a6070 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5d000991e0 tx=0x7f5d0006f8c0 comp rx=0 tx=0).stop 2024-04-17T06:01:09.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.944+0000 7f5d0ff96640 1 -- 172.21.15.47:0/2534174526 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5d08103b70 msgr2=0x7f5d0819f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:09.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.944+0000 7f5d0ff96640 1 --2- 172.21.15.47:0/2534174526 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5d08103b70 0x7f5d0819f810 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f5d0000cce0 tx=0x7f5d0000a620 comp rx=0 tx=0).stop 2024-04-17T06:01:09.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.944+0000 7f5d0ff96640 1 -- 172.21.15.47:0/2534174526 shutdown_connections 2024-04-17T06:01:09.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.944+0000 7f5d0ff96640 1 --2- 172.21.15.47:0/2534174526 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f5cf40a3bb0 0x7f5cf40a6070 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:09.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.944+0000 7f5d0ff96640 1 --2- 172.21.15.47:0/2534174526 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5d08103b70 0x7f5d0819f810 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:09.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.944+0000 7f5d0ff96640 1 --2- 172.21.15.47:0/2534174526 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5d081031b0 0x7f5d081987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:09.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.944+0000 7f5d0ff96640 1 --2- 172.21.15.47:0/2534174526 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5d08101fc0 0x7f5d08198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:09.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.944+0000 7f5d0ff96640 1 -- 172.21.15.47:0/2534174526 >> 172.21.15.47:0/2534174526 conn(0x7f5d080fd530 msgr2=0x7f5d08106450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:01:09.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.944+0000 7f5d0ff96640 1 -- 172.21.15.47:0/2534174526 shutdown_connections 2024-04-17T06:01:09.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:09.944+0000 7f5d0ff96640 1 -- 172.21.15.47:0/2534174526 wait complete. 2024-04-17T06:01:09.955 INFO:teuthology.run_tasks:Running task sequential... 2024-04-17T06:01:09.967 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-17T06:01:09.967 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-17T06:01:09.968 INFO:teuthology.task.print:Enabling mgr modules 2024-04-17T06:01:09.968 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-04-17T06:01:09.969 INFO:teuthology.task.exec:Executing custom commands... 2024-04-17T06:01:09.969 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi047.front.sepia.ceph.com 2024-04-17T06:01:09.969 DEBUG:teuthology.orchestra.run.smithi047:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-04-17T06:01:10.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:10.076+0000 7f84e144a640 1 -- 172.21.15.47:0/3930941227 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f84dc100e70 msgr2=0x7f84dc101290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:10.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:10.076+0000 7f84e144a640 1 --2- 172.21.15.47:0/3930941227 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f84dc100e70 0x7f84dc101290 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f84cc009900 tx=0x7f84cc02bc30 comp rx=0 tx=0).stop 2024-04-17T06:01:10.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:10.076+0000 7f84e144a640 1 -- 172.21.15.47:0/3930941227 shutdown_connections 2024-04-17T06:01:10.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:10.076+0000 7f84e144a640 1 --2- 172.21.15.47:0/3930941227 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f84dc102a10 0x7f84dc109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:10.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:10.076+0000 7f84e144a640 1 --2- 172.21.15.47:0/3930941227 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f84dc102050 0x7f84dc1024d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:10.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:10.076+0000 7f84e144a640 1 --2- 172.21.15.47:0/3930941227 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f84dc100e70 0x7f84dc101290 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:10.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:10.076+0000 7f84e144a640 1 -- 172.21.15.47:0/3930941227 >> 172.21.15.47:0/3930941227 conn(0x7f84dc0fc450 msgr2=0x7f84dc0fe870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:01:10.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:10.076+0000 7f84e144a640 1 -- 172.21.15.47:0/3930941227 shutdown_connections 2024-04-17T06:01:10.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:10.076+0000 7f84e144a640 1 -- 172.21.15.47:0/3930941227 wait complete. 2024-04-17T06:01:10.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:10.077+0000 7f84e144a640 1 Processor -- start 2024-04-17T06:01:10.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:10.077+0000 7f84e144a640 1 -- start start 2024-04-17T06:01:10.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:10.077+0000 7f84e144a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f84dc100e70 0x7f84dc198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:10.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:10.077+0000 7f84e144a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f84dc102050 0x7f84dc198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:10.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:10.077+0000 7f84e144a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f84dc102a10 0x7f84dc19f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:10.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:10.077+0000 7f84e144a640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f84dc10cb30 con 0x7f84dc100e70 2024-04-17T06:01:10.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:10.077+0000 7f84e144a640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f84dc10c9b0 con 0x7f84dc102050 2024-04-17T06:01:10.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:10.077+0000 7f84e144a640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f84dc10ccb0 con 0x7f84dc102a10 2024-04-17T06:01:10.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:10.077+0000 7f84daffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f84dc100e70 0x7f84dc198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:10.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:10.077+0000 7f84daffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f84dc100e70 0x7f84dc198230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40668/0 (socket says 172.21.15.47:40668) 2024-04-17T06:01:10.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:10.077+0000 7f84daffd640 1 -- 172.21.15.47:0/49380131 learned_addr learned my addr 172.21.15.47:0/49380131 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:01:10.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:10.077+0000 7f84da7fc640 1 --2- 172.21.15.47:0/49380131 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f84dc102050 0x7f84dc198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:10.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:10.077+0000 7f84daffd640 1 -- 172.21.15.47:0/49380131 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f84dc102a10 msgr2=0x7f84dc19f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:10.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:10.077+0000 7f84db7fe640 1 --2- 172.21.15.47:0/49380131 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f84dc102a10 0x7f84dc19f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:10.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:10.078+0000 7f84daffd640 1 --2- 172.21.15.47:0/49380131 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f84dc102a10 0x7f84dc19f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:10.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:10.078+0000 7f84daffd640 1 -- 172.21.15.47:0/49380131 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f84dc102050 msgr2=0x7f84dc198770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:10.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:10.078+0000 7f84daffd640 1 --2- 172.21.15.47:0/49380131 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f84dc102050 0x7f84dc198770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:10.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:10.078+0000 7f84daffd640 1 -- 172.21.15.47:0/49380131 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84dc19fe50 con 0x7f84dc100e70 2024-04-17T06:01:10.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:10.078+0000 7f84da7fc640 1 --2- 172.21.15.47:0/49380131 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f84dc102050 0x7f84dc198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:01:10.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:10.078+0000 7f84daffd640 1 --2- 172.21.15.47:0/49380131 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f84dc100e70 0x7f84dc198230 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f84cc000c00 tx=0x7f84cc004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:01:10.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:10.078+0000 7f84c3fff640 1 -- 172.21.15.47:0/49380131 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84cc041070 con 0x7f84dc100e70 2024-04-17T06:01:10.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:10.078+0000 7f84c3fff640 1 -- 172.21.15.47:0/49380131 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84cc0054e0 con 0x7f84dc100e70 2024-04-17T06:01:10.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:10.078+0000 7f84e144a640 1 -- 172.21.15.47:0/49380131 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84dc1a00e0 con 0x7f84dc100e70 2024-04-17T06:01:10.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:10.078+0000 7f84c3fff640 1 -- 172.21.15.47:0/49380131 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84cc0346a0 con 0x7f84dc100e70 2024-04-17T06:01:10.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:10.078+0000 7f84e144a640 1 -- 172.21.15.47:0/49380131 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84dc1a05d0 con 0x7f84dc100e70 2024-04-17T06:01:10.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:10.079+0000 7f84e144a640 1 -- 172.21.15.47:0/49380131 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84dc1a08f0 con 0x7f84dc100e70 2024-04-17T06:01:10.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:10.080+0000 7f84c3fff640 1 -- 172.21.15.47:0/49380131 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f84cc044070 con 0x7f84dc100e70 2024-04-17T06:01:10.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:10.081+0000 7f84c3fff640 1 --2- 172.21.15.47:0/49380131 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f84c40a3c50 0x7f84c40a6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:10.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:10.081+0000 7f84da7fc640 1 --2- 172.21.15.47:0/49380131 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f84c40a3c50 0x7f84c40a6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:10.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:10.081+0000 7f84c3fff640 1 -- 172.21.15.47:0/49380131 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f84cc0f2440 con 0x7f84dc100e70 2024-04-17T06:01:10.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:10.081+0000 7f84da7fc640 1 --2- 172.21.15.47:0/49380131 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f84c40a3c50 0x7f84c40a6110 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f84dc199740 tx=0x7f84d0005d30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:01:10.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:10.084+0000 7f84c3fff640 1 -- 172.21.15.47:0/49380131 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f84cc0bedd0 con 0x7f84dc100e70 2024-04-17T06:01:10.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:09 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/2794484033' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T06:01:10.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:09 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/246711209' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T06:01:10.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:09 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/2794484033' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T06:01:10.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:09 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/246711209' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T06:01:10.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:10.324+0000 7f84e144a640 1 -- 172.21.15.47:0/49380131 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7f84dc1a0ac0 con 0x7f84dc100e70 2024-04-17T06:01:10.382 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:09 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/2794484033' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T06:01:10.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:09 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/246711209' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T06:01:10.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:10.920+0000 7f84c3fff640 1 -- 172.21.15.47:0/49380131 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7f84cc003e20 con 0x7f84dc100e70 2024-04-17T06:01:10.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:10.923+0000 7f84e144a640 1 -- 172.21.15.47:0/49380131 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f84c40a3c50 msgr2=0x7f84c40a6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:10.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:10.923+0000 7f84e144a640 1 --2- 172.21.15.47:0/49380131 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f84c40a3c50 0x7f84c40a6110 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f84dc199740 tx=0x7f84d0005d30 comp rx=0 tx=0).stop 2024-04-17T06:01:10.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:10.923+0000 7f84e144a640 1 -- 172.21.15.47:0/49380131 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f84dc100e70 msgr2=0x7f84dc198230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:10.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:10.923+0000 7f84e144a640 1 --2- 172.21.15.47:0/49380131 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f84dc100e70 0x7f84dc198230 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f84cc000c00 tx=0x7f84cc004040 comp rx=0 tx=0).stop 2024-04-17T06:01:10.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:10.924+0000 7f84e144a640 1 -- 172.21.15.47:0/49380131 shutdown_connections 2024-04-17T06:01:10.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:10.924+0000 7f84e144a640 1 --2- 172.21.15.47:0/49380131 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f84c40a3c50 0x7f84c40a6110 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:10.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:10.924+0000 7f84e144a640 1 --2- 172.21.15.47:0/49380131 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f84dc102a10 0x7f84dc19f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:10.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:10.924+0000 7f84e144a640 1 --2- 172.21.15.47:0/49380131 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f84dc102050 0x7f84dc198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:10.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:10.924+0000 7f84e144a640 1 --2- 172.21.15.47:0/49380131 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f84dc100e70 0x7f84dc198230 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:10.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:10.924+0000 7f84e144a640 1 -- 172.21.15.47:0/49380131 >> 172.21.15.47:0/49380131 conn(0x7f84dc0fc450 msgr2=0x7f84dc0fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:01:10.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:10.924+0000 7f84e144a640 1 -- 172.21.15.47:0/49380131 shutdown_connections 2024-04-17T06:01:10.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:10.924+0000 7f84e144a640 1 -- 172.21.15.47:0/49380131 wait complete. 2024-04-17T06:01:10.940 DEBUG:teuthology.orchestra.run.smithi047:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-04-17T06:01:11.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:11.055+0000 7f9851111640 1 -- 172.21.15.47:0/3724726061 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f984c102050 msgr2=0x7f984c1024d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:11.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:11.055+0000 7f9851111640 1 --2- 172.21.15.47:0/3724726061 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f984c102050 0x7f984c1024d0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f9834009900 tx=0x7f983402bc30 comp rx=0 tx=0).stop 2024-04-17T06:01:11.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:11.055+0000 7f9851111640 1 -- 172.21.15.47:0/3724726061 shutdown_connections 2024-04-17T06:01:11.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:11.055+0000 7f9851111640 1 --2- 172.21.15.47:0/3724726061 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f984c102a10 0x7f984c109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:11.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:11.055+0000 7f9851111640 1 --2- 172.21.15.47:0/3724726061 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f984c102050 0x7f984c1024d0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:11.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:11.055+0000 7f9851111640 1 --2- 172.21.15.47:0/3724726061 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f984c100e70 0x7f984c101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:11.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:11.055+0000 7f9851111640 1 -- 172.21.15.47:0/3724726061 >> 172.21.15.47:0/3724726061 conn(0x7f984c0fc430 msgr2=0x7f984c0fe870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:01:11.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:11.055+0000 7f9851111640 1 -- 172.21.15.47:0/3724726061 shutdown_connections 2024-04-17T06:01:11.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:11.055+0000 7f9851111640 1 -- 172.21.15.47:0/3724726061 wait complete. 2024-04-17T06:01:11.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:11.055+0000 7f9851111640 1 Processor -- start 2024-04-17T06:01:11.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:11.056+0000 7f9851111640 1 -- start start 2024-04-17T06:01:11.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:11.056+0000 7f9851111640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f984c100e70 0x7f984c06a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:11.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:11.056+0000 7f9851111640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f984c102050 0x7f984c19d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:11.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:11.056+0000 7f9851111640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f984c102a10 0x7f984c19f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:11.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:11.056+0000 7f9851111640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f984c10cb50 con 0x7f984c100e70 2024-04-17T06:01:11.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:11.056+0000 7f9851111640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f984c10c9d0 con 0x7f984c102050 2024-04-17T06:01:11.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:11.056+0000 7f9851111640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f984c10ccd0 con 0x7f984c102a10 2024-04-17T06:01:11.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:11.056+0000 7f984ad9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f984c100e70 0x7f984c06a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:11.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:11.056+0000 7f984ad9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f984c100e70 0x7f984c06a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40696/0 (socket says 172.21.15.47:40696) 2024-04-17T06:01:11.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:11.056+0000 7f984ad9d640 1 -- 172.21.15.47:0/163342240 learned_addr learned my addr 172.21.15.47:0/163342240 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:01:11.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:11.056+0000 7f984a59c640 1 --2- 172.21.15.47:0/163342240 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f984c102050 0x7f984c19d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:11.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:11.056+0000 7f984b59e640 1 --2- 172.21.15.47:0/163342240 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f984c102a10 0x7f984c19f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:11.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:11.056+0000 7f984ad9d640 1 -- 172.21.15.47:0/163342240 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f984c102a10 msgr2=0x7f984c19f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:11.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:11.056+0000 7f984ad9d640 1 --2- 172.21.15.47:0/163342240 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f984c102a10 0x7f984c19f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:11.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:11.056+0000 7f984ad9d640 1 -- 172.21.15.47:0/163342240 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f984c102050 msgr2=0x7f984c19d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:11.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:11.056+0000 7f984ad9d640 1 --2- 172.21.15.47:0/163342240 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f984c102050 0x7f984c19d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:11.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:11.056+0000 7f984ad9d640 1 -- 172.21.15.47:0/163342240 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f984c19ff30 con 0x7f984c100e70 2024-04-17T06:01:11.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:11.057+0000 7f984ad9d640 1 --2- 172.21.15.47:0/163342240 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f984c100e70 0x7f984c06a180 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f983c00e9c0 tx=0x7f983c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:01:11.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:11.057+0000 7f9843fff640 1 -- 172.21.15.47:0/163342240 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f983c00cd80 con 0x7f984c100e70 2024-04-17T06:01:11.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:11.057+0000 7f9851111640 1 -- 172.21.15.47:0/163342240 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f984c1a01c0 con 0x7f984c100e70 2024-04-17T06:01:11.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:11.057+0000 7f9843fff640 1 -- 172.21.15.47:0/163342240 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f983c0035f0 con 0x7f984c100e70 2024-04-17T06:01:11.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:11.057+0000 7f9843fff640 1 -- 172.21.15.47:0/163342240 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f983c010690 con 0x7f984c100e70 2024-04-17T06:01:11.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:11.057+0000 7f9851111640 1 -- 172.21.15.47:0/163342240 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f984c1a0730 con 0x7f984c100e70 2024-04-17T06:01:11.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:11.058+0000 7f9851111640 1 -- 172.21.15.47:0/163342240 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f984c1081b0 con 0x7f984c100e70 2024-04-17T06:01:11.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:11.059+0000 7f9843fff640 1 -- 172.21.15.47:0/163342240 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f983c0037a0 con 0x7f984c100e70 2024-04-17T06:01:11.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:11.060+0000 7f9843fff640 1 --2- 172.21.15.47:0/163342240 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f98380a3ca0 0x7f98380a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:11.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:11.060+0000 7f984a59c640 1 -- 172.21.15.47:0/163342240 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f98380a3ca0 msgr2=0x7f98380a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/3157902806 2024-04-17T06:01:11.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:11.060+0000 7f9843fff640 1 -- 172.21.15.47:0/163342240 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f983c0d0920 con 0x7f984c100e70 2024-04-17T06:01:11.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:11.060+0000 7f984a59c640 1 --2- 172.21.15.47:0/163342240 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f98380a3ca0 0x7f98380a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-17T06:01:11.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:11.064+0000 7f9843fff640 1 -- 172.21.15.47:0/163342240 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f983c09d340 con 0x7f984c100e70 2024-04-17T06:01:11.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:11.252+0000 7f9851111640 1 -- 172.21.15.47:0/163342240 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f984c02d5d0 con 0x7f984c100e70 2024-04-17T06:01:11.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:11.257+0000 7f9843fff640 1 -- 172.21.15.47:0/163342240 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v28)=0 v28) ==== 138+0+0 (secure 0 0 0) 0x7f983c09cc40 con 0x7f984c100e70 2024-04-17T06:01:11.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:11.259+0000 7f9851111640 1 -- 172.21.15.47:0/163342240 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f98380a3ca0 msgr2=0x7f98380a6160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-17T06:01:11.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:11.259+0000 7f9851111640 1 --2- 172.21.15.47:0/163342240 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f98380a3ca0 0x7f98380a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:11.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:11.259+0000 7f9851111640 1 -- 172.21.15.47:0/163342240 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f984c100e70 msgr2=0x7f984c06a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:11.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:11.259+0000 7f9851111640 1 --2- 172.21.15.47:0/163342240 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f984c100e70 0x7f984c06a180 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f983c00e9c0 tx=0x7f983c00ee80 comp rx=0 tx=0).stop 2024-04-17T06:01:11.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:11.259+0000 7f9851111640 1 -- 172.21.15.47:0/163342240 shutdown_connections 2024-04-17T06:01:11.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:11.259+0000 7f9851111640 1 --2- 172.21.15.47:0/163342240 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f98380a3ca0 0x7f98380a6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:11.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:11.259+0000 7f9851111640 1 --2- 172.21.15.47:0/163342240 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f984c102a10 0x7f984c19f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:11.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:11.259+0000 7f9851111640 1 --2- 172.21.15.47:0/163342240 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f984c102050 0x7f984c19d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:11.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:11.259+0000 7f9851111640 1 --2- 172.21.15.47:0/163342240 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f984c100e70 0x7f984c06a180 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:11.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:11.259+0000 7f9851111640 1 -- 172.21.15.47:0/163342240 >> 172.21.15.47:0/163342240 conn(0x7f984c0fc430 msgr2=0x7f984c105380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:01:11.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:11.259+0000 7f9851111640 1 -- 172.21.15.47:0/163342240 shutdown_connections 2024-04-17T06:01:11.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:11.259+0000 7f9851111640 1 -- 172.21.15.47:0/163342240 wait complete. 2024-04-17T06:01:11.268 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:10 smithi047 ceph-mon[32223]: pgmap v346: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-17T06:01:11.268 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:10 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/49380131' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-17T06:01:11.268 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 06:01:10 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: ignoring --setuser ceph since I am not root 2024-04-17T06:01:11.268 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 06:01:10 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: ignoring --setgroup ceph since I am not root 2024-04-17T06:01:11.269 DEBUG:teuthology.orchestra.run.smithi047:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-04-17T06:01:11.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:10 smithi103 ceph-mon[36243]: pgmap v346: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-17T06:01:11.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:10 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/49380131' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-17T06:01:11.284 INFO:journalctl@ceph.mgr.z.smithi103.stdout:Apr 17 06:01:10 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-z[38091]: ignoring --setuser ceph since I am not root 2024-04-17T06:01:11.284 INFO:journalctl@ceph.mgr.z.smithi103.stdout:Apr 17 06:01:10 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-z[38091]: ignoring --setgroup ceph since I am not root 2024-04-17T06:01:11.284 INFO:journalctl@ceph.mgr.z.smithi103.stdout:Apr 17 06:01:10 smithi103 ceph-mgr[38095]: -- 172.21.15.103:0/743267670 <== mon.1 v2:172.21.15.103:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5629e9ddc5a0 con 0x5629e9db0800 2024-04-17T06:01:11.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:11.379+0000 7f4d8880d640 1 -- 172.21.15.47:0/1650507476 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d84102050 msgr2=0x7f4d841024d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:11.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:11.379+0000 7f4d8880d640 1 --2- 172.21.15.47:0/1650507476 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d84102050 0x7f4d841024d0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c009900 tx=0x7f4d6c02bc30 comp rx=0 tx=0).stop 2024-04-17T06:01:11.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:11.379+0000 7f4d8880d640 1 -- 172.21.15.47:0/1650507476 shutdown_connections 2024-04-17T06:01:11.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:11.379+0000 7f4d8880d640 1 --2- 172.21.15.47:0/1650507476 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4d84102a10 0x7f4d84109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:11.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:11.379+0000 7f4d8880d640 1 --2- 172.21.15.47:0/1650507476 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d84102050 0x7f4d841024d0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:11.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:11.379+0000 7f4d8880d640 1 --2- 172.21.15.47:0/1650507476 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d84100e70 0x7f4d84101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:11.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:11.379+0000 7f4d8880d640 1 -- 172.21.15.47:0/1650507476 >> 172.21.15.47:0/1650507476 conn(0x7f4d840fc430 msgr2=0x7f4d840fe870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:01:11.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:11.379+0000 7f4d8880d640 1 -- 172.21.15.47:0/1650507476 shutdown_connections 2024-04-17T06:01:11.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:11.379+0000 7f4d8880d640 1 -- 172.21.15.47:0/1650507476 wait complete. 2024-04-17T06:01:11.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:11.380+0000 7f4d8880d640 1 Processor -- start 2024-04-17T06:01:11.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:11.380+0000 7f4d8880d640 1 -- start start 2024-04-17T06:01:11.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:11.380+0000 7f4d8880d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d84100e70 0x7f4d8406a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:11.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:11.380+0000 7f4d8880d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d84102050 0x7f4d8419cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:11.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:11.380+0000 7f4d8880d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4d84102a10 0x7f4d8419f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:11.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:11.380+0000 7f4d8880d640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f4d8410cb70 con 0x7f4d84100e70 2024-04-17T06:01:11.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:11.380+0000 7f4d8880d640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f4d8410c9f0 con 0x7f4d84102a10 2024-04-17T06:01:11.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:11.380+0000 7f4d8880d640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4d8410ccf0 con 0x7f4d84102050 2024-04-17T06:01:11.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:11.380+0000 7f4d8259c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d84100e70 0x7f4d8406a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:11.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:11.380+0000 7f4d8259c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d84100e70 0x7f4d8406a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40720/0 (socket says 172.21.15.47:40720) 2024-04-17T06:01:11.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:11.380+0000 7f4d8259c640 1 -- 172.21.15.47:0/4030037675 learned_addr learned my addr 172.21.15.47:0/4030037675 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:01:11.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:11.380+0000 7f4d82d9d640 1 --2- 172.21.15.47:0/4030037675 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4d84102a10 0x7f4d8419f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:11.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:11.380+0000 7f4d7bfff640 1 --2- 172.21.15.47:0/4030037675 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d84102050 0x7f4d8419cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:11.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:11.381+0000 7f4d8259c640 1 -- 172.21.15.47:0/4030037675 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d84102050 msgr2=0x7f4d8419cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:11.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:11.381+0000 7f4d8259c640 1 --2- 172.21.15.47:0/4030037675 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d84102050 0x7f4d8419cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:11.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:11.381+0000 7f4d8259c640 1 -- 172.21.15.47:0/4030037675 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4d84102a10 msgr2=0x7f4d8419f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:11.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:11.381+0000 7f4d8259c640 1 --2- 172.21.15.47:0/4030037675 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4d84102a10 0x7f4d8419f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:11.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:11.381+0000 7f4d8259c640 1 -- 172.21.15.47:0/4030037675 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d8419feb0 con 0x7f4d84100e70 2024-04-17T06:01:11.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:11.381+0000 7f4d7bfff640 1 --2- 172.21.15.47:0/4030037675 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d84102050 0x7f4d8419cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T06:01:11.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:11.381+0000 7f4d82d9d640 1 --2- 172.21.15.47:0/4030037675 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4d84102a10 0x7f4d8419f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T06:01:11.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:11.381+0000 7f4d8259c640 1 --2- 172.21.15.47:0/4030037675 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d84100e70 0x7f4d8406a180 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f4d7400e9c0 tx=0x7f4d7400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:01:11.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:11.382+0000 7f4d7b7fe640 1 -- 172.21.15.47:0/4030037675 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d7400cd80 con 0x7f4d84100e70 2024-04-17T06:01:11.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:11.382+0000 7f4d7b7fe640 1 -- 172.21.15.47:0/4030037675 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d740035f0 con 0x7f4d84100e70 2024-04-17T06:01:11.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:11.382+0000 7f4d7b7fe640 1 -- 172.21.15.47:0/4030037675 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d74010690 con 0x7f4d84100e70 2024-04-17T06:01:11.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:11.382+0000 7f4d8880d640 1 -- 172.21.15.47:0/4030037675 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d841a0140 con 0x7f4d84100e70 2024-04-17T06:01:11.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:11.382+0000 7f4d8880d640 1 -- 172.21.15.47:0/4030037675 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d841a0610 con 0x7f4d84100e70 2024-04-17T06:01:11.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:11.383+0000 7f4d8880d640 1 -- 172.21.15.47:0/4030037675 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d841081b0 con 0x7f4d84100e70 2024-04-17T06:01:11.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:10 smithi098 ceph-mon[36516]: pgmap v346: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-17T06:01:11.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:10 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/49380131' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-17T06:01:11.384 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 17 06:01:10 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-y[37889]: ignoring --setuser ceph since I am not root 2024-04-17T06:01:11.384 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 17 06:01:10 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-y[37889]: ignoring --setgroup ceph since I am not root 2024-04-17T06:01:11.384 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 17 06:01:10 smithi098 ceph-mgr[37900]: -- 172.21.15.98:0/24801268 <== mon.2 v2:172.21.15.98:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5564685725a0 con 0x556468546800 2024-04-17T06:01:11.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:11.385+0000 7f4d7b7fe640 1 -- 172.21.15.47:0/4030037675 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f4d740037a0 con 0x7f4d84100e70 2024-04-17T06:01:11.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:11.385+0000 7f4d7b7fe640 1 --2- 172.21.15.47:0/4030037675 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f4d700a3d20 0x7f4d700a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:11.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:11.385+0000 7f4d7bfff640 1 -- 172.21.15.47:0/4030037675 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f4d700a3d20 msgr2=0x7f4d700a61e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/3157902806 2024-04-17T06:01:11.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:11.386+0000 7f4d7b7fe640 1 -- 172.21.15.47:0/4030037675 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f4d740d0730 con 0x7f4d84100e70 2024-04-17T06:01:11.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:11.386+0000 7f4d7bfff640 1 --2- 172.21.15.47:0/4030037675 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f4d700a3d20 0x7f4d700a61e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-17T06:01:11.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:11.390+0000 7f4d7b7fe640 1 -- 172.21.15.47:0/4030037675 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f4d7409d120 con 0x7f4d84100e70 2024-04-17T06:01:11.527 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 06:01:11 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T06:01:11.529+0000 7effca8c7200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-17T06:01:11.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:11.580+0000 7f4d8880d640 1 -- 172.21.15.47:0/4030037675 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7f4d8402d5d0 con 0x7f4d84100e70 2024-04-17T06:01:11.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:11.583+0000 7f4d7b7fe640 1 -- 172.21.15.47:0/4030037675 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v29)=0 v29) ==== 133+0+0 (secure 0 0 0) 0x7f4d7409ca20 con 0x7f4d84100e70 2024-04-17T06:01:11.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:11.585+0000 7f4d8880d640 1 -- 172.21.15.47:0/4030037675 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f4d700a3d20 msgr2=0x7f4d700a61e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-17T06:01:11.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:11.585+0000 7f4d8880d640 1 --2- 172.21.15.47:0/4030037675 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f4d700a3d20 0x7f4d700a61e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:11.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:11.585+0000 7f4d8880d640 1 -- 172.21.15.47:0/4030037675 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d84100e70 msgr2=0x7f4d8406a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:11.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:11.585+0000 7f4d8880d640 1 --2- 172.21.15.47:0/4030037675 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d84100e70 0x7f4d8406a180 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f4d7400e9c0 tx=0x7f4d7400ee80 comp rx=0 tx=0).stop 2024-04-17T06:01:11.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:11.585+0000 7f4d8880d640 1 -- 172.21.15.47:0/4030037675 shutdown_connections 2024-04-17T06:01:11.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:11.585+0000 7f4d8880d640 1 --2- 172.21.15.47:0/4030037675 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f4d700a3d20 0x7f4d700a61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:11.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:11.585+0000 7f4d8880d640 1 --2- 172.21.15.47:0/4030037675 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4d84102a10 0x7f4d8419f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:11.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:11.585+0000 7f4d8880d640 1 --2- 172.21.15.47:0/4030037675 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d84102050 0x7f4d8419cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:11.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:11.585+0000 7f4d8880d640 1 --2- 172.21.15.47:0/4030037675 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d84100e70 0x7f4d8406a180 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:11.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:11.585+0000 7f4d8880d640 1 -- 172.21.15.47:0/4030037675 >> 172.21.15.47:0/4030037675 conn(0x7f4d840fc430 msgr2=0x7f4d84105380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:01:11.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:11.585+0000 7f4d8880d640 1 -- 172.21.15.47:0/4030037675 shutdown_connections 2024-04-17T06:01:11.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:11.586+0000 7f4d8880d640 1 -- 172.21.15.47:0/4030037675 wait complete. 2024-04-17T06:01:11.595 INFO:teuthology.run_tasks:Running task kclient... 2024-04-17T06:01:11.621 INFO:tasks.kclient:Mounting kernel clients... 2024-04-17T06:01:11.621 INFO:tasks.kclient:config is {'client.0': {}} 2024-04-17T06:01:11.621 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T06:01:11.622 DEBUG:teuthology.orchestra.run.smithi047:> ip netns list 2024-04-17T06:01:11.639 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T06:01:11.639 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link delete ceph-brx 2024-04-17T06:01:11.709 INFO:teuthology.orchestra.run.smithi047.stderr:Cannot find device "ceph-brx" 2024-04-17T06:01:11.710 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T06:01:11.711 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'wsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2024-04-17T06:01:11.711 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-17T06:01:11.711 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-17T06:01:11.711 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 565db2d8-fc7e-11ee-bc91-c7b262605968 -- ceph fs ls 2024-04-17T06:01:11.778 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 06:01:11 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T06:01:11.628+0000 7effca8c7200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-17T06:01:11.783 INFO:journalctl@ceph.mgr.z.smithi103.stdout:Apr 17 06:01:11 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-z[38091]: 2024-04-17T06:01:11.494+0000 7f2e3a595200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-17T06:01:11.783 INFO:journalctl@ceph.mgr.z.smithi103.stdout:Apr 17 06:01:11 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-z[38091]: 2024-04-17T06:01:11.588+0000 7f2e3a595200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-17T06:01:11.883 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 17 06:01:11 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-y[37889]: 2024-04-17T06:01:11.487+0000 7f716b579200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-17T06:01:11.883 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 17 06:01:11 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-y[37889]: 2024-04-17T06:01:11.580+0000 7f716b579200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-17T06:01:12.004 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T06:01:12.127 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:11 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/49380131' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-17T06:01:12.127 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:11 smithi047 ceph-mon[32223]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-04-17T06:01:12.127 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:11 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/163342240' entity='client.admin' 2024-04-17T06:01:12.127 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:11 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/4030037675' entity='client.admin' 2024-04-17T06:01:12.216 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:11 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/49380131' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-17T06:01:12.216 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:11 smithi098 ceph-mon[36516]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-04-17T06:01:12.216 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:11 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/163342240' entity='client.admin' 2024-04-17T06:01:12.217 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:11 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/4030037675' entity='client.admin' 2024-04-17T06:01:12.226 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:11 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/49380131' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-17T06:01:12.227 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:11 smithi103 ceph-mon[36243]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-04-17T06:01:12.227 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:11 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/163342240' entity='client.admin' 2024-04-17T06:01:12.227 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:11 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/4030037675' entity='client.admin' 2024-04-17T06:01:12.389 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 06:01:12 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T06:01:12.316+0000 7effca8c7200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-17T06:01:12.533 INFO:journalctl@ceph.mgr.z.smithi103.stdout:Apr 17 06:01:12 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-z[38091]: 2024-04-17T06:01:12.228+0000 7f2e3a595200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-17T06:01:12.534 INFO:journalctl@ceph.mgr.z.smithi103.stdout:Apr 17 06:01:12 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-z[38091]: 2024-04-17T06:01:12.321+0000 7f2e3a595200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-17T06:01:12.534 INFO:journalctl@ceph.mgr.z.smithi103.stdout:Apr 17 06:01:12 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-z[38091]: 2024-04-17T06:01:12.463+0000 7f2e3a595200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-17T06:01:12.546 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 17 06:01:12 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-y[37889]: 2024-04-17T06:01:12.217+0000 7f716b579200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-17T06:01:12.547 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 17 06:01:12 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-y[37889]: 2024-04-17T06:01:12.309+0000 7f716b579200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-17T06:01:12.547 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 17 06:01:12 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-y[37889]: 2024-04-17T06:01:12.451+0000 7f716b579200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-17T06:01:12.547 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 17 06:01:12 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-y[37889]: 2024-04-17T06:01:12.547+0000 7f716b579200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-17T06:01:12.748 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 06:01:12 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T06:01:12.415+0000 7effca8c7200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-17T06:01:12.748 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 06:01:12 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T06:01:12.562+0000 7effca8c7200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-17T06:01:12.748 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 06:01:12 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T06:01:12.661+0000 7effca8c7200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-17T06:01:12.748 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 06:01:12 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T06:01:12.749+0000 7effca8c7200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-17T06:01:12.882 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 17 06:01:12 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-y[37889]: 2024-04-17T06:01:12.629+0000 7f716b579200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-17T06:01:12.908 INFO:journalctl@ceph.mgr.z.smithi103.stdout:Apr 17 06:01:12 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-z[38091]: 2024-04-17T06:01:12.560+0000 7f2e3a595200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-17T06:01:12.908 INFO:journalctl@ceph.mgr.z.smithi103.stdout:Apr 17 06:01:12 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-z[38091]: 2024-04-17T06:01:12.644+0000 7f2e3a595200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-17T06:01:13.177 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 17 06:01:12 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-y[37889]: 2024-04-17T06:01:12.890+0000 7f716b579200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-17T06:01:13.177 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 17 06:01:13 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-y[37889]: 2024-04-17T06:01:12.999+0000 7f716b579200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-17T06:01:13.177 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 17 06:01:13 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-y[37889]: 2024-04-17T06:01:13.084+0000 7f716b579200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-17T06:01:13.177 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 17 06:01:13 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-y[37889]: 2024-04-17T06:01:13.178+0000 7f716b579200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-17T06:01:13.199 INFO:journalctl@ceph.mgr.z.smithi103.stdout:Apr 17 06:01:12 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-z[38091]: 2024-04-17T06:01:12.910+0000 7f2e3a595200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-17T06:01:13.199 INFO:journalctl@ceph.mgr.z.smithi103.stdout:Apr 17 06:01:13 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-z[38091]: 2024-04-17T06:01:13.019+0000 7f2e3a595200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-17T06:01:13.199 INFO:journalctl@ceph.mgr.z.smithi103.stdout:Apr 17 06:01:13 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-z[38091]: 2024-04-17T06:01:13.105+0000 7f2e3a595200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-17T06:01:13.199 INFO:journalctl@ceph.mgr.z.smithi103.stdout:Apr 17 06:01:13 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-z[38091]: 2024-04-17T06:01:13.200+0000 7f2e3a595200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-17T06:01:13.337 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 06:01:13 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T06:01:13.025+0000 7effca8c7200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-17T06:01:13.337 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 06:01:13 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T06:01:13.139+0000 7effca8c7200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-17T06:01:13.337 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 06:01:13 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T06:01:13.229+0000 7effca8c7200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-17T06:01:13.337 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 06:01:13 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T06:01:13.339+0000 7effca8c7200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-17T06:01:13.533 INFO:journalctl@ceph.mgr.z.smithi103.stdout:Apr 17 06:01:13 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-z[38091]: 2024-04-17T06:01:13.376+0000 7f2e3a595200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-17T06:01:13.590 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 06:01:13 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T06:01:13.522+0000 7effca8c7200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-17T06:01:13.632 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 17 06:01:13 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-y[37889]: 2024-04-17T06:01:13.353+0000 7f716b579200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-17T06:01:13.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:13.870+0000 7f68f6caa700 1 -- 172.21.15.47:0/2745625564 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68f0107930 msgr2=0x7f68f0107d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:13.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:13.870+0000 7f68f6caa700 1 --2- 172.21.15.47:0/2745625564 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68f0107930 0x7f68f0107d10 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f68e4009990 tx=0x7f68e4009ca0 comp rx=0 tx=0).stop 2024-04-17T06:01:13.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:13.870+0000 7f68f6caa700 1 -- 172.21.15.47:0/2745625564 shutdown_connections 2024-04-17T06:01:13.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:13.870+0000 7f68f6caa700 1 --2- 172.21.15.47:0/2745625564 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f68f0102340 0x7f68f010e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:13.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:13.870+0000 7f68f6caa700 1 --2- 172.21.15.47:0/2745625564 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68f0107930 0x7f68f0107d10 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:13.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:13.870+0000 7f68f6caa700 1 --2- 172.21.15.47:0/2745625564 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f68f0101980 0x7f68f0101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:13.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:13.870+0000 7f68f6caa700 1 -- 172.21.15.47:0/2745625564 >> 172.21.15.47:0/2745625564 conn(0x7f68f00779c0 msgr2=0x7f68f0077dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:01:13.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:13.870+0000 7f68f6caa700 1 -- 172.21.15.47:0/2745625564 shutdown_connections 2024-04-17T06:01:13.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:13.870+0000 7f68f6caa700 1 -- 172.21.15.47:0/2745625564 wait complete. 2024-04-17T06:01:13.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:13.871+0000 7f68f6caa700 1 Processor -- start 2024-04-17T06:01:13.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:13.871+0000 7f68f6caa700 1 -- start start 2024-04-17T06:01:13.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:13.871+0000 7f68f6caa700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f68f0101980 0x7f68f019b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:13.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:13.871+0000 7f68f6caa700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68f0102340 0x7f68f019b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:13.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:13.871+0000 7f68f6caa700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f68f0107930 0x7f68f019fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:13.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:13.871+0000 7f68f6caa700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f68f0111c60 con 0x7f68f0102340 2024-04-17T06:01:13.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:13.871+0000 7f68f6caa700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f68f0111ae0 con 0x7f68f0101980 2024-04-17T06:01:13.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:13.871+0000 7f68f6caa700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f68f0111de0 con 0x7f68f0107930 2024-04-17T06:01:13.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:13.871+0000 7f68effff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68f0102340 0x7f68f019b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:13.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:13.871+0000 7f68effff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68f0102340 0x7f68f019b920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40742/0 (socket says 172.21.15.47:40742) 2024-04-17T06:01:13.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:13.871+0000 7f68effff700 1 -- 172.21.15.47:0/1339820569 learned_addr learned my addr 172.21.15.47:0/1339820569 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:01:13.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:13.871+0000 7f68f4a46700 1 --2- 172.21.15.47:0/1339820569 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f68f0101980 0x7f68f019b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:13.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:13.871+0000 7f68f5247700 1 --2- 172.21.15.47:0/1339820569 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f68f0107930 0x7f68f019fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:13.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:13.871+0000 7f68effff700 1 -- 172.21.15.47:0/1339820569 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f68f0107930 msgr2=0x7f68f019fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:13.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:13.871+0000 7f68effff700 1 --2- 172.21.15.47:0/1339820569 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f68f0107930 0x7f68f019fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:13.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:13.871+0000 7f68effff700 1 -- 172.21.15.47:0/1339820569 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f68f0101980 msgr2=0x7f68f019b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:13.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:13.871+0000 7f68effff700 1 --2- 172.21.15.47:0/1339820569 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f68f0101980 0x7f68f019b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:13.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:13.871+0000 7f68effff700 1 -- 172.21.15.47:0/1339820569 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68f01a03a0 con 0x7f68f0102340 2024-04-17T06:01:13.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:13.872+0000 7f68effff700 1 --2- 172.21.15.47:0/1339820569 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68f0102340 0x7f68f019b920 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f68dc00d980 tx=0x7f68dc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:01:13.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:13.872+0000 7f68edffb700 1 -- 172.21.15.47:0/1339820569 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68dc016070 con 0x7f68f0102340 2024-04-17T06:01:13.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:13.872+0000 7f68f6caa700 1 -- 172.21.15.47:0/1339820569 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68f01a0690 con 0x7f68f0102340 2024-04-17T06:01:13.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:13.872+0000 7f68edffb700 1 -- 172.21.15.47:0/1339820569 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68dc004d10 con 0x7f68f0102340 2024-04-17T06:01:13.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:13.872+0000 7f68edffb700 1 -- 172.21.15.47:0/1339820569 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68dc005670 con 0x7f68f0102340 2024-04-17T06:01:13.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:13.872+0000 7f68f6caa700 1 -- 172.21.15.47:0/1339820569 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68f01a7f10 con 0x7f68f0102340 2024-04-17T06:01:13.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:13.873+0000 7f68f6caa700 1 -- 172.21.15.47:0/1339820569 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68f0103070 con 0x7f68f0102340 2024-04-17T06:01:13.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:13.875+0000 7f68edffb700 1 -- 172.21.15.47:0/1339820569 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f68dc004750 con 0x7f68f0102340 2024-04-17T06:01:13.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:13.875+0000 7f68edffb700 1 --2- 172.21.15.47:0/1339820569 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f68e00a3c90 0x7f68e00a6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:13.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:13.876+0000 7f68f4a46700 1 -- 172.21.15.47:0/1339820569 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f68e00a3c90 msgr2=0x7f68e00a6150 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/3157902806 2024-04-17T06:01:13.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:13.876+0000 7f68edffb700 1 -- 172.21.15.47:0/1339820569 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f68dc0d0790 con 0x7f68f0102340 2024-04-17T06:01:13.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:13.876+0000 7f68f4a46700 1 --2- 172.21.15.47:0/1339820569 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f68e00a3c90 0x7f68e00a6150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-17T06:01:13.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:13.878+0000 7f68edffb700 1 -- 172.21.15.47:0/1339820569 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f68dc01f020 con 0x7f68f0102340 2024-04-17T06:01:14.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:14.077+0000 7f68f4a46700 1 -- 172.21.15.47:0/1339820569 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f68e00a3c90 msgr2=0x7f68e00a6150 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/3157902806 2024-04-17T06:01:14.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:14.077+0000 7f68f4a46700 1 --2- 172.21.15.47:0/1339820569 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f68e00a3c90 0x7f68e00a6150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-17T06:01:14.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:14.183+0000 7f68f6caa700 1 -- 172.21.15.47:0/1339820569 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f68f0069740 con 0x7f68f0102340 2024-04-17T06:01:14.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:14.183+0000 7f68edffb700 1 -- 172.21.15.47:0/1339820569 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v21) ==== 53+0+73 (secure 0 0 0) 0x7f68f0069740 con 0x7f68f0102340 2024-04-17T06:01:14.181 INFO:teuthology.orchestra.run.smithi047.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-17T06:01:14.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:14.185+0000 7f68f6caa700 1 -- 172.21.15.47:0/1339820569 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f68e00a3c90 msgr2=0x7f68e00a6150 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-17T06:01:14.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:14.185+0000 7f68f6caa700 1 --2- 172.21.15.47:0/1339820569 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f68e00a3c90 0x7f68e00a6150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:14.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:14.185+0000 7f68f6caa700 1 -- 172.21.15.47:0/1339820569 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68f0102340 msgr2=0x7f68f019b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:14.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:14.185+0000 7f68f6caa700 1 --2- 172.21.15.47:0/1339820569 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68f0102340 0x7f68f019b920 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f68dc00d980 tx=0x7f68dc00dc90 comp rx=0 tx=0).stop 2024-04-17T06:01:14.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:14.185+0000 7f68f6caa700 1 -- 172.21.15.47:0/1339820569 shutdown_connections 2024-04-17T06:01:14.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:14.185+0000 7f68f6caa700 1 --2- 172.21.15.47:0/1339820569 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f68e00a3c90 0x7f68e00a6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:14.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:14.185+0000 7f68f6caa700 1 --2- 172.21.15.47:0/1339820569 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f68f0107930 0x7f68f019fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:14.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:14.185+0000 7f68f6caa700 1 --2- 172.21.15.47:0/1339820569 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f68f0101980 0x7f68f019b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:14.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:14.185+0000 7f68f6caa700 1 --2- 172.21.15.47:0/1339820569 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68f0102340 0x7f68f019b920 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:14.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:14.185+0000 7f68f6caa700 1 -- 172.21.15.47:0/1339820569 >> 172.21.15.47:0/1339820569 conn(0x7f68f00779c0 msgr2=0x7f68f006b300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:01:14.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:14.185+0000 7f68f6caa700 1 -- 172.21.15.47:0/1339820569 shutdown_connections 2024-04-17T06:01:14.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:14.185+0000 7f68f6caa700 1 -- 172.21.15.47:0/1339820569 wait complete. 2024-04-17T06:01:14.484 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:14 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/1339820569' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-17T06:01:14.533 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:14 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/1339820569' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-17T06:01:14.534 INFO:journalctl@ceph.mgr.z.smithi103.stdout:Apr 17 06:01:14 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-z[38091]: 2024-04-17T06:01:14.349+0000 7f2e3a595200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-17T06:01:14.534 INFO:journalctl@ceph.mgr.z.smithi103.stdout:Apr 17 06:01:14 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-z[38091]: 2024-04-17T06:01:14.481+0000 7f2e3a595200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-17T06:01:14.632 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:14 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/1339820569' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-17T06:01:14.633 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 17 06:01:14 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-y[37889]: 2024-04-17T06:01:14.306+0000 7f716b579200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-17T06:01:14.633 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 17 06:01:14 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-y[37889]: 2024-04-17T06:01:14.434+0000 7f716b579200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-17T06:01:14.889 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 06:01:14 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T06:01:14.553+0000 7effca8c7200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-17T06:01:14.889 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 06:01:14 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T06:01:14.698+0000 7effca8c7200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-17T06:01:15.009 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-17T06:01:15.009 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-17T06:01:15.009 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi047.front.sepia.ceph.com 2024-04-17T06:01:15.009 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-17T06:01:15.009 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-17T06:01:15.010 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-17T06:01:15.010 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-17T06:01:15.010 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-17T06:01:15.010 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-17T06:01:15.010 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T06:01:15.010 DEBUG:teuthology.orchestra.run.smithi047:> ip addr 2024-04-17T06:01:15.054 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 17 06:01:14 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-y[37889]: 2024-04-17T06:01:14.769+0000 7f716b579200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-17T06:01:15.054 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 17 06:01:14 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-y[37889]: 2024-04-17T06:01:14.859+0000 7f716b579200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-17T06:01:15.065 INFO:teuthology.orchestra.run.smithi047.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-17T06:01:15.066 INFO:teuthology.orchestra.run.smithi047.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-17T06:01:15.066 INFO:teuthology.orchestra.run.smithi047.stdout: inet 127.0.0.1/8 scope host lo 2024-04-17T06:01:15.066 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-04-17T06:01:15.066 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 ::1/128 scope host 2024-04-17T06:01:15.066 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-04-17T06:01:15.066 INFO:teuthology.orchestra.run.smithi047.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-17T06:01:15.066 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:78 brd ff:ff:ff:ff:ff:ff 2024-04-17T06:01:15.066 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp7s0f0 2024-04-17T06:01:15.066 INFO:teuthology.orchestra.run.smithi047.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-17T06:01:15.066 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:79 brd ff:ff:ff:ff:ff:ff 2024-04-17T06:01:15.066 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp7s0f1 2024-04-17T06:01:15.066 INFO:teuthology.orchestra.run.smithi047.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-17T06:01:15.066 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8e brd ff:ff:ff:ff:ff:ff 2024-04-17T06:01:15.066 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp1s0f0 2024-04-17T06:01:15.067 INFO:teuthology.orchestra.run.smithi047.stdout: inet 172.21.15.47/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-17T06:01:15.067 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft 42096sec preferred_lft 42096sec 2024-04-17T06:01:15.067 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::ec4:7aff:febd:148e/64 scope link noprefixroute 2024-04-17T06:01:15.067 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-04-17T06:01:15.067 INFO:teuthology.orchestra.run.smithi047.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-17T06:01:15.067 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8f brd ff:ff:ff:ff:ff:ff 2024-04-17T06:01:15.067 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp1s0f1 2024-04-17T06:01:15.067 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-17T06:01:15.068 DEBUG:teuthology.orchestra.run.smithi047:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-17T06:01:15.068 DEBUG:teuthology.orchestra.run.smithi047:> set -e 2024-04-17T06:01:15.068 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link add name ceph-brx type bridge 2024-04-17T06:01:15.068 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip addr flush dev ceph-brx 2024-04-17T06:01:15.068 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link set ceph-brx up 2024-04-17T06:01:15.068 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-17T06:01:15.068 DEBUG:teuthology.orchestra.run.smithi047:> ') 2024-04-17T06:01:15.104 INFO:journalctl@ceph.mgr.z.smithi103.stdout:Apr 17 06:01:14 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-z[38091]: 2024-04-17T06:01:14.818+0000 7f2e3a595200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-17T06:01:15.104 INFO:journalctl@ceph.mgr.z.smithi103.stdout:Apr 17 06:01:14 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-z[38091]: 2024-04-17T06:01:14.908+0000 7f2e3a595200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-17T06:01:15.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:15 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-17T06:01:15.237 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 06:01:15 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T06:01:15.053+0000 7effca8c7200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-17T06:01:15.237 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 06:01:15 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T06:01:15.149+0000 7effca8c7200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-17T06:01:15.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T06:01:15.279 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T06:01:15.279 DEBUG:teuthology.orchestra.run.smithi047:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-17T06:01:15.311 INFO:teuthology.orchestra.run.smithi047.stdout:1 2024-04-17T06:01:15.312 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T06:01:15.312 DEBUG:teuthology.orchestra.run.smithi047:> route 2024-04-17T06:01:15.382 INFO:teuthology.orchestra.run.smithi047.stdout:Kernel IP routing table 2024-04-17T06:01:15.382 INFO:teuthology.orchestra.run.smithi047.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-17T06:01:15.382 INFO:teuthology.orchestra.run.smithi047.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-17T06:01:15.382 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-17T06:01:15.382 INFO:teuthology.orchestra.run.smithi047.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-17T06:01:15.383 DEBUG:teuthology.orchestra.run.smithi047:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-17T06:01:15.383 DEBUG:teuthology.orchestra.run.smithi047:> set -e 2024-04-17T06:01:15.383 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-17T06:01:15.383 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-17T06:01:15.383 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-17T06:01:15.383 DEBUG:teuthology.orchestra.run.smithi047:> ') 2024-04-17T06:01:15.384 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 17 06:01:15 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-y[37889]: 2024-04-17T06:01:15.055+0000 7f716b579200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-17T06:01:15.384 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 17 06:01:15 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-y[37889]: 2024-04-17T06:01:15.238+0000 7f716b579200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-17T06:01:15.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:15 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-17T06:01:15.487 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 06:01:15 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T06:01:15.375+0000 7effca8c7200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-17T06:01:15.533 INFO:journalctl@ceph.mgr.z.smithi103.stdout:Apr 17 06:01:15 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-z[38091]: 2024-04-17T06:01:15.106+0000 7f2e3a595200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-17T06:01:15.534 INFO:journalctl@ceph.mgr.z.smithi103.stdout:Apr 17 06:01:15 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-z[38091]: 2024-04-17T06:01:15.293+0000 7f2e3a595200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-17T06:01:15.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T06:01:15.698 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T06:01:15.699 DEBUG:teuthology.orchestra.run.smithi047:> ip netns list 2024-04-17T06:01:15.718 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T06:01:15.718 DEBUG:teuthology.orchestra.run.smithi047:> ip netns list-id 2024-04-17T06:01:15.771 DEBUG:teuthology.orchestra.run.smithi047:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-17T06:01:15.772 DEBUG:teuthology.orchestra.run.smithi047:> set -e 2024-04-17T06:01:15.772 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T06:01:15.772 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-17T06:01:15.772 DEBUG:teuthology.orchestra.run.smithi047:> ') 2024-04-17T06:01:15.779 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 17 06:01:15 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-y[37889]: 2024-04-17T06:01:15.520+0000 7f716b579200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-17T06:01:15.839 INFO:journalctl@ceph.mgr.z.smithi103.stdout:Apr 17 06:01:15 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-z[38091]: 2024-04-17T06:01:15.581+0000 7f2e3a595200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-17T06:01:15.839 INFO:journalctl@ceph.mgr.z.smithi103.stdout:Apr 17 06:01:15 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-z[38091]: 2024-04-17T06:01:15.840+0000 7f2e3a595200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-17T06:01:15.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:15 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-17T06:01:15.874 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 06:01:15 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T06:01:15.577+0000 7effca8c7200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-17T06:01:15.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T06:01:15.900 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-17T06:01:15.900 DEBUG:teuthology.orchestra.run.smithi047:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-17T06:01:15.900 DEBUG:teuthology.orchestra.run.smithi047:> set -e 2024-04-17T06:01:15.900 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-17T06:01:15.900 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-17T06:01:15.900 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-17T06:01:15.900 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-17T06:01:15.901 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-17T06:01:15.901 DEBUG:teuthology.orchestra.run.smithi047:> ') 2024-04-17T06:01:15.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:15 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-17T06:01:16.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T06:01:16.056 DEBUG:teuthology.orchestra.run.smithi047:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-17T06:01:16.056 DEBUG:teuthology.orchestra.run.smithi047:> set -e 2024-04-17T06:01:16.056 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link set brx.0 up 2024-04-17T06:01:16.056 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link set dev brx.0 master ceph-brx 2024-04-17T06:01:16.056 DEBUG:teuthology.orchestra.run.smithi047:> ') 2024-04-17T06:01:16.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-17T06:01:16.132 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 17 06:01:15 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-y[37889]: 2024-04-17T06:01:15.780+0000 7f716b579200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-17T06:01:16.132 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 17 06:01:16 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-y[37889]: 2024-04-17T06:01:16.015+0000 7f716b579200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-17T06:01:16.133 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 06:01:15 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T06:01:15.894+0000 7effca8c7200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-17T06:01:16.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T06:01:16.145 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-17T06:01:16.146 DEBUG:teuthology.orchestra.run.smithi047:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-17T06:01:16.161 INFO:teuthology.orchestra.run.smithi047.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-17T06:01:16.161 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-17T06:01:16.161 DEBUG:teuthology.orchestra.run.smithi047:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-17T06:01:16.216 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-04-17T06:01:16.217 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-04-17T06:01:16.217 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/8477d852-ccf4-4ec5-8586-1c752352b391 2024-04-17T06:01:16.217 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T06:01:16.217 DEBUG:teuthology.orchestra.run.smithi047:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/8477d852-ccf4-4ec5-8586-1c752352b391 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,wsync,nofallback 2024-04-17T06:01:16.283 INFO:journalctl@ceph.mgr.z.smithi103.stdout:Apr 17 06:01:16 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-z[38091]: 2024-04-17T06:01:16.077+0000 7f2e3a595200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-17T06:01:16.303 INFO:teuthology.orchestra.run.smithi047.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,wsync,nofallback 2024-04-17T06:01:16.303 INFO:teuthology.orchestra.run.smithi047.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,wsync". 2024-04-17T06:01:16.388 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 06:01:16 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T06:01:16.180+0000 7effca8c7200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-17T06:01:16.431 INFO:teuthology.orchestra.run.smithi047.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,wsync,nofallback 2024-04-17T06:01:16.431 INFO:teuthology.orchestra.run.smithi047.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,wsync". 2024-04-17T06:01:16.431 INFO:teuthology.orchestra.run.smithi047.stdout:mount.ceph: resolved to: "172.21.15.47:3300,172.21.15.98:3300,172.21.15.103:3300" 2024-04-17T06:01:16.431 INFO:teuthology.orchestra.run.smithi047.stdout:mount.ceph: trying mount with new device syntax: 0@565db2d8-fc7e-11ee-bc91-c7b262605968.cephfs=/volumes/qa/sv_1/8477d852-ccf4-4ec5-8586-1c752352b391 2024-04-17T06:01:16.432 INFO:teuthology.orchestra.run.smithi047.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,wsync,name=0,key=0,mon_addr=172.21.15.47:3300/172.21.15.98:3300/172.21.15.103:3300" will pass to kernel 2024-04-17T06:01:16.432 INFO:teuthology.orchestra.run.smithi047.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-17T06:01:16.432 INFO:teuthology.orchestra.run.smithi047.stdout: You just mounted a file system that supports labels which does not 2024-04-17T06:01:16.432 INFO:teuthology.orchestra.run.smithi047.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-17T06:01:16.432 INFO:teuthology.orchestra.run.smithi047.stdout: applications will generate AVC messages and not be allowed access to 2024-04-17T06:01:16.432 INFO:teuthology.orchestra.run.smithi047.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-17T06:01:16.434 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-17T06:01:16.434 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T06:01:16.434 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-17T06:01:16.464 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-17T06:01:16.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:16 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-17T06:01:16.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T06:01:16.563 INFO:teuthology.orchestra.run.smithi047.stdout:client24944 2024-04-17T06:01:16.563 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-17T06:01:16.563 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:01:16.580 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-17T06:01:16.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:16 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-17T06:01:16.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T06:01:16.677 INFO:teuthology.orchestra.run.smithi047.stdout:client24944 2024-04-17T06:01:16.678 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-17T06:01:16.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:16 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-17T06:01:16.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T06:01:16.739 INFO:teuthology.orchestra.run.smithi047.stdout:client24944 2024-04-17T06:01:16.739 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968.client24944/status') 2024-04-17T06:01:16.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-17T06:01:16.800 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 06:01:16 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T06:01:16.436+0000 7effca8c7200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-17T06:01:16.802 INFO:teuthology.orchestra.run.smithi047.stdout:instance: client.24944 (3)172.21.15.47:0/1375268301 2024-04-17T06:01:16.802 INFO:teuthology.orchestra.run.smithi047.stdout:blocklisted: false 2024-04-17T06:01:16.803 INFO:teuthology.orchestra.run.smithi047.stderr:0+1 records in 2024-04-17T06:01:16.803 INFO:teuthology.orchestra.run.smithi047.stderr:0+1 records out 2024-04-17T06:01:16.803 INFO:teuthology.orchestra.run.smithi047.stderr:71 bytes copied, 2.6922e-05 s, 2.6 MB/s 2024-04-17T06:01:16.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T06:01:16.811 INFO:teuthology.run_tasks:Running task exec... 2024-04-17T06:01:16.820 INFO:teuthology.task.exec:Executing custom commands... 2024-04-17T06:01:16.820 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi047.front.sepia.ceph.com 2024-04-17T06:01:16.821 DEBUG:teuthology.orchestra.run.smithi047:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-04-17T06:01:16.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:16.930+0000 7f70ff0c8640 1 -- 172.21.15.47:0/352057530 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f70f8107ca0 msgr2=0x7f70f810a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:16.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:16.930+0000 7f70ff0c8640 1 --2- 172.21.15.47:0/352057530 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f70f8107ca0 0x7f70f810a0f0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f70f0009ca0 tx=0x7f70f002c0f0 comp rx=0 tx=0).stop 2024-04-17T06:01:16.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:16.931+0000 7f70ff0c8640 1 -- 172.21.15.47:0/352057530 shutdown_connections 2024-04-17T06:01:16.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:16.931+0000 7f70ff0c8640 1 --2- 172.21.15.47:0/352057530 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f70f8107ca0 0x7f70f810a0f0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:16.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:16.931+0000 7f70ff0c8640 1 --2- 172.21.15.47:0/352057530 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f70f8105370 0x7f70f8107760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:16.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:16.931+0000 7f70ff0c8640 1 --2- 172.21.15.47:0/352057530 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f70f8102a40 0x7f70f8104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:16.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:16.931+0000 7f70ff0c8640 1 -- 172.21.15.47:0/352057530 >> 172.21.15.47:0/352057530 conn(0x7f70f80fc430 msgr2=0x7f70f80fe870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:01:16.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:16.931+0000 7f70ff0c8640 1 -- 172.21.15.47:0/352057530 shutdown_connections 2024-04-17T06:01:16.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:16.931+0000 7f70ff0c8640 1 -- 172.21.15.47:0/352057530 wait complete. 2024-04-17T06:01:16.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:16.931+0000 7f70ff0c8640 1 Processor -- start 2024-04-17T06:01:16.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:16.931+0000 7f70ff0c8640 1 -- start start 2024-04-17T06:01:16.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:16.931+0000 7f70ff0c8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f70f8102a40 0x7f70f8198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:16.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:16.931+0000 7f70ff0c8640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f70f8105370 0x7f70f8198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:16.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:16.931+0000 7f70ff0c8640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f70f8107ca0 0x7f70f819f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:16.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:16.931+0000 7f70ff0c8640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f70f810dc50 con 0x7f70f8107ca0 2024-04-17T06:01:16.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:16.931+0000 7f70ff0c8640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f70f810dad0 con 0x7f70f8105370 2024-04-17T06:01:16.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:16.931+0000 7f70ff0c8640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f70f810ddd0 con 0x7f70f8102a40 2024-04-17T06:01:16.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:16.931+0000 7f70fd665640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f70f8107ca0 0x7f70f819f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:16.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:16.932+0000 7f70fce64640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f70f8102a40 0x7f70f8198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:16.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:16.932+0000 7f70fce64640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f70f8102a40 0x7f70f8198230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.47:52552/0 (socket says 172.21.15.47:52552) 2024-04-17T06:01:16.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:16.932+0000 7f70fd665640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f70f8107ca0 0x7f70f819f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33484/0 (socket says 172.21.15.47:33484) 2024-04-17T06:01:16.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:16.932+0000 7f70fce64640 1 -- 172.21.15.47:0/1880394528 learned_addr learned my addr 172.21.15.47:0/1880394528 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:01:16.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:16.932+0000 7f70f7fff640 1 --2- 172.21.15.47:0/1880394528 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f70f8105370 0x7f70f8198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:16.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:16.932+0000 7f70fd665640 1 -- 172.21.15.47:0/1880394528 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f70f8102a40 msgr2=0x7f70f8198230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:16.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:16.932+0000 7f70fd665640 1 --2- 172.21.15.47:0/1880394528 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f70f8102a40 0x7f70f8198230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:16.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:16.932+0000 7f70fd665640 1 -- 172.21.15.47:0/1880394528 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f70f8105370 msgr2=0x7f70f8198770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:16.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:16.932+0000 7f70fd665640 1 --2- 172.21.15.47:0/1880394528 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f70f8105370 0x7f70f8198770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:16.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:16.932+0000 7f70fd665640 1 -- 172.21.15.47:0/1880394528 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70f819fe50 con 0x7f70f8107ca0 2024-04-17T06:01:16.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:16.932+0000 7f70fd665640 1 --2- 172.21.15.47:0/1880394528 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f70f8107ca0 0x7f70f819f7e0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f70f0037040 tx=0x7f70f0032ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:01:16.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:16.932+0000 7f70f5ffb640 1 -- 172.21.15.47:0/1880394528 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f0041020 con 0x7f70f8107ca0 2024-04-17T06:01:16.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:16.932+0000 7f70ff0c8640 1 -- 172.21.15.47:0/1880394528 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70f81a00e0 con 0x7f70f8107ca0 2024-04-17T06:01:16.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:16.932+0000 7f70f5ffb640 1 -- 172.21.15.47:0/1880394528 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70f0034cd0 con 0x7f70f8107ca0 2024-04-17T06:01:16.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:16.932+0000 7f70f5ffb640 1 -- 172.21.15.47:0/1880394528 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f0039040 con 0x7f70f8107ca0 2024-04-17T06:01:16.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:16.933+0000 7f70ff0c8640 1 -- 172.21.15.47:0/1880394528 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70f81a05f0 con 0x7f70f8107ca0 2024-04-17T06:01:16.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:16.933+0000 7f70ff0c8640 1 -- 172.21.15.47:0/1880394528 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70f80fdf20 con 0x7f70f8107ca0 2024-04-17T06:01:16.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:16.936+0000 7f70f5ffb640 1 -- 172.21.15.47:0/1880394528 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f70f0044020 con 0x7f70f8107ca0 2024-04-17T06:01:16.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:16.936+0000 7f70f5ffb640 1 --2- 172.21.15.47:0/1880394528 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f70e80a3ca0 0x7f70e80a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:16.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:16.936+0000 7f70fce64640 1 -- 172.21.15.47:0/1880394528 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f70e80a3ca0 msgr2=0x7f70e80a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/3157902806 2024-04-17T06:01:16.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:16.936+0000 7f70f5ffb640 1 -- 172.21.15.47:0/1880394528 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f70f00f26b0 con 0x7f70f8107ca0 2024-04-17T06:01:16.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:16.936+0000 7f70fce64640 1 --2- 172.21.15.47:0/1880394528 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f70e80a3ca0 0x7f70e80a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-17T06:01:16.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:16.939+0000 7f70f5ffb640 1 -- 172.21.15.47:0/1880394528 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f70f004b020 con 0x7f70f8107ca0 2024-04-17T06:01:17.132 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 17 06:01:16 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-y[37889]: 2024-04-17T06:01:16.819+0000 7f716b579200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-17T06:01:17.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:17.137+0000 7f70fce64640 1 -- 172.21.15.47:0/1880394528 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f70e80a3ca0 msgr2=0x7f70e80a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/3157902806 2024-04-17T06:01:17.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:17.137+0000 7f70fce64640 1 --2- 172.21.15.47:0/1880394528 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f70e80a3ca0 0x7f70e80a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-17T06:01:17.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:17.170+0000 7f70ff0c8640 1 -- 172.21.15.47:0/1880394528 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7f70f8062b60 con 0x7f70f8107ca0 2024-04-17T06:01:17.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:17.222+0000 7f70f5ffb640 1 -- 172.21.15.47:0/1880394528 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v22) ==== 118+0+0 (secure 0 0 0) 0x7f70f00bf130 con 0x7f70f8107ca0 2024-04-17T06:01:17.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:17.225+0000 7f70ff0c8640 1 -- 172.21.15.47:0/1880394528 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f70e80a3ca0 msgr2=0x7f70e80a6160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-17T06:01:17.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:17.225+0000 7f70ff0c8640 1 --2- 172.21.15.47:0/1880394528 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f70e80a3ca0 0x7f70e80a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:17.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:17.225+0000 7f70ff0c8640 1 -- 172.21.15.47:0/1880394528 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f70f8107ca0 msgr2=0x7f70f819f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:17.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:17.225+0000 7f70ff0c8640 1 --2- 172.21.15.47:0/1880394528 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f70f8107ca0 0x7f70f819f7e0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f70f0037040 tx=0x7f70f0032ea0 comp rx=0 tx=0).stop 2024-04-17T06:01:17.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:17.225+0000 7f70ff0c8640 1 -- 172.21.15.47:0/1880394528 shutdown_connections 2024-04-17T06:01:17.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:17.225+0000 7f70ff0c8640 1 --2- 172.21.15.47:0/1880394528 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7f70e80a3ca0 0x7f70e80a6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:17.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:17.225+0000 7f70ff0c8640 1 --2- 172.21.15.47:0/1880394528 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f70f8107ca0 0x7f70f819f7e0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:17.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:17.225+0000 7f70ff0c8640 1 --2- 172.21.15.47:0/1880394528 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f70f8105370 0x7f70f8198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:17.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:17.225+0000 7f70ff0c8640 1 --2- 172.21.15.47:0/1880394528 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f70f8102a40 0x7f70f8198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:17.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:17.225+0000 7f70ff0c8640 1 -- 172.21.15.47:0/1880394528 >> 172.21.15.47:0/1880394528 conn(0x7f70f80fc430 msgr2=0x7f70f8103380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:01:17.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:17.225+0000 7f70ff0c8640 1 -- 172.21.15.47:0/1880394528 shutdown_connections 2024-04-17T06:01:17.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:17.225+0000 7f70ff0c8640 1 -- 172.21.15.47:0/1880394528 wait complete. 2024-04-17T06:01:17.224 INFO:journalctl@ceph.mgr.z.smithi103.stdout:Apr 17 06:01:16 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-z[38091]: 2024-04-17T06:01:16.884+0000 7f2e3a595200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-17T06:01:17.236 DEBUG:teuthology.orchestra.run.smithi047:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-04-17T06:01:17.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:17.348+0000 7fd80d972640 1 -- 172.21.15.47:0/1391661125 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd808102a10 msgr2=0x7fd808109330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:17.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:17.348+0000 7fd80d972640 1 --2- 172.21.15.47:0/1391661125 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd808102a10 0x7fd808109330 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fd800009ca0 tx=0x7fd80002c0f0 comp rx=0 tx=0).stop 2024-04-17T06:01:17.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:17.349+0000 7fd80d972640 1 -- 172.21.15.47:0/1391661125 shutdown_connections 2024-04-17T06:01:17.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:17.349+0000 7fd80d972640 1 --2- 172.21.15.47:0/1391661125 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd808102a10 0x7fd808109330 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:17.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:17.349+0000 7fd80d972640 1 --2- 172.21.15.47:0/1391661125 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd808102050 0x7fd8081024d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:17.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:17.349+0000 7fd80d972640 1 --2- 172.21.15.47:0/1391661125 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd808100e70 0x7fd808101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:17.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:17.349+0000 7fd80d972640 1 -- 172.21.15.47:0/1391661125 >> 172.21.15.47:0/1391661125 conn(0x7fd8080fc430 msgr2=0x7fd8080fe870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:01:17.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:17.349+0000 7fd80d972640 1 -- 172.21.15.47:0/1391661125 shutdown_connections 2024-04-17T06:01:17.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:17.349+0000 7fd80d972640 1 -- 172.21.15.47:0/1391661125 wait complete. 2024-04-17T06:01:17.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:17.349+0000 7fd80d972640 1 Processor -- start 2024-04-17T06:01:17.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:17.349+0000 7fd80d972640 1 -- start start 2024-04-17T06:01:17.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:17.349+0000 7fd80d972640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd808100e70 0x7fd80806a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:17.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:17.349+0000 7fd80d972640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd808102050 0x7fd80819cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:17.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:17.349+0000 7fd80d972640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd808102a10 0x7fd80819f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:17.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:17.349+0000 7fd80d972640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fd80810cb60 con 0x7fd808102050 2024-04-17T06:01:17.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:17.349+0000 7fd80d972640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd80810c9e0 con 0x7fd808100e70 2024-04-17T06:01:17.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:17.349+0000 7fd80d972640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd80810cce0 con 0x7fd808102a10 2024-04-17T06:01:17.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:17.350+0000 7fd7fffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd808102050 0x7fd80819cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:17.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:17.350+0000 7fd7fffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd808102050 0x7fd80819cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33506/0 (socket says 172.21.15.47:33506) 2024-04-17T06:01:17.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:17.350+0000 7fd806ffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd808100e70 0x7fd80806a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:17.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:17.350+0000 7fd7fffff640 1 -- 172.21.15.47:0/495460483 learned_addr learned my addr 172.21.15.47:0/495460483 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:01:17.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:17.350+0000 7fd8077fe640 1 --2- 172.21.15.47:0/495460483 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd808102a10 0x7fd80819f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:17.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:17.350+0000 7fd7fffff640 1 -- 172.21.15.47:0/495460483 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd808102a10 msgr2=0x7fd80819f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:17.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:17.350+0000 7fd7fffff640 1 --2- 172.21.15.47:0/495460483 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd808102a10 0x7fd80819f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:17.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:17.350+0000 7fd7fffff640 1 -- 172.21.15.47:0/495460483 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd808100e70 msgr2=0x7fd80806a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:17.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:17.350+0000 7fd7fffff640 1 --2- 172.21.15.47:0/495460483 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd808100e70 0x7fd80806a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:17.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:17.350+0000 7fd7fffff640 1 -- 172.21.15.47:0/495460483 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd80819fe40 con 0x7fd808102050 2024-04-17T06:01:17.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:17.350+0000 7fd806ffd640 1 --2- 172.21.15.47:0/495460483 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd808100e70 0x7fd80806a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T06:01:17.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:17.350+0000 7fd8077fe640 1 --2- 172.21.15.47:0/495460483 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd808102a10 0x7fd80819f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T06:01:17.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:17.350+0000 7fd7fffff640 1 --2- 172.21.15.47:0/495460483 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd808102050 0x7fd80819cf10 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fd7f000e9f0 tx=0x7fd7f000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:01:17.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:17.350+0000 7fd80c970640 1 -- 172.21.15.47:0/495460483 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7f000cd80 con 0x7fd808102050 2024-04-17T06:01:17.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:17.350+0000 7fd80d972640 1 -- 172.21.15.47:0/495460483 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8081a00d0 con 0x7fd808102050 2024-04-17T06:01:17.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:17.350+0000 7fd80c970640 1 -- 172.21.15.47:0/495460483 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7f0005500 con 0x7fd808102050 2024-04-17T06:01:17.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:17.350+0000 7fd80c970640 1 -- 172.21.15.47:0/495460483 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7f0010690 con 0x7fd808102050 2024-04-17T06:01:17.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:17.350+0000 7fd80d972640 1 -- 172.21.15.47:0/495460483 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8081a05d0 con 0x7fd808102050 2024-04-17T06:01:17.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:17.351+0000 7fd80d972640 1 -- 172.21.15.47:0/495460483 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8081081b0 con 0x7fd808102050 2024-04-17T06:01:17.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:17.353+0000 7fd80c970640 1 -- 172.21.15.47:0/495460483 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7fd7f00046c0 con 0x7fd808102050 2024-04-17T06:01:17.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:17.353+0000 7fd80c970640 1 --2- 172.21.15.47:0/495460483 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fd7f40a3ca0 0x7fd7f40a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:17.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:17.353+0000 7fd806ffd640 1 -- 172.21.15.47:0/495460483 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fd7f40a3ca0 msgr2=0x7fd7f40a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/3157902806 2024-04-17T06:01:17.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:17.353+0000 7fd80c970640 1 -- 172.21.15.47:0/495460483 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fd7f00d0a50 con 0x7fd808102050 2024-04-17T06:01:17.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:17.353+0000 7fd806ffd640 1 --2- 172.21.15.47:0/495460483 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fd7f40a3ca0 0x7fd7f40a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-17T06:01:17.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:17.357+0000 7fd80c970640 1 -- 172.21.15.47:0/495460483 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fd7f0053ff0 con 0x7fd808102050 2024-04-17T06:01:17.484 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:17 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/1880394528' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-17T06:01:17.485 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 17 06:01:17 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-y[37889]: 2024-04-17T06:01:17.390+0000 7f716b579200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-17T06:01:17.533 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:17 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/1880394528' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-17T06:01:17.533 INFO:journalctl@ceph.mgr.z.smithi103.stdout:Apr 17 06:01:17 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-z[38091]: 2024-04-17T06:01:17.456+0000 7f2e3a595200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-17T06:01:17.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:17.554+0000 7fd806ffd640 1 -- 172.21.15.47:0/495460483 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fd7f40a3ca0 msgr2=0x7fd7f40a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/3157902806 2024-04-17T06:01:17.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:17.554+0000 7fd806ffd640 1 --2- 172.21.15.47:0/495460483 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fd7f40a3ca0 0x7fd7f40a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-17T06:01:17.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:17.571+0000 7fd80d972640 1 -- 172.21.15.47:0/495460483 --> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7fd808062500 con 0x7fd7f40a3ca0 2024-04-17T06:01:17.638 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:17 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/1880394528' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-17T06:01:17.639 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 06:01:17 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T06:01:17.308+0000 7effca8c7200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-17T06:01:17.883 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 17 06:01:17 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-y[37889]: 2024-04-17T06:01:17.486+0000 7f716b579200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-17T06:01:17.883 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 17 06:01:17 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-y[37889]: 2024-04-17T06:01:17.702+0000 7f716b579200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-17T06:01:17.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:17.955+0000 7fd806ffd640 1 -- 172.21.15.47:0/495460483 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fd7f40a3ca0 msgr2=0x7fd7f40a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/3157902806 2024-04-17T06:01:17.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:17.955+0000 7fd806ffd640 1 --2- 172.21.15.47:0/495460483 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fd7f40a3ca0 0x7fd7f40a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-17T06:01:18.033 INFO:journalctl@ceph.mgr.z.smithi103.stdout:Apr 17 06:01:17 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-z[38091]: 2024-04-17T06:01:17.555+0000 7f2e3a595200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-17T06:01:18.034 INFO:journalctl@ceph.mgr.z.smithi103.stdout:Apr 17 06:01:17 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-z[38091]: 2024-04-17T06:01:17.773+0000 7f2e3a595200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-17T06:01:18.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:18.237+0000 7fd80c970640 1 -- 172.21.15.47:0/495460483 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mgrmap(e 20) ==== 138663+0+0 (secure 0 0 0) 0x7fd7f00993b0 con 0x7fd808102050 2024-04-17T06:01:18.236 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 06:01:17 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T06:01:17.904+0000 7effca8c7200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-17T06:01:18.236 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 06:01:18 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T06:01:18.009+0000 7effca8c7200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-17T06:01:18.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:18.256+0000 7fd80c970640 1 -- 172.21.15.47:0/495460483 <== mon.0 v2:172.21.15.47:3300/0 8 ==== mgrmap(e 21) ==== 138331+0+0 (secure 0 0 0) 0x7fd7f00035f0 con 0x7fd808102050 2024-04-17T06:01:18.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:18.256+0000 7fd80c970640 1 -- 172.21.15.47:0/495460483 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fd7f40a3ca0 msgr2=0x7fd7f40a6160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-17T06:01:18.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:18.256+0000 7fd80c970640 1 --2- 172.21.15.47:0/495460483 >> [v2:172.21.15.47:6800/3157902806,v1:172.21.15.47:6801/3157902806] conn(0x7fd7f40a3ca0 0x7fd7f40a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:18.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:18 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/1880394528' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-17T06:01:18.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:18 smithi098 ceph-mon[36516]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-17T06:01:18.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:18 smithi098 ceph-mon[36516]: Standby manager daemon y restarted 2024-04-17T06:01:18.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:18 smithi098 ceph-mon[36516]: Standby manager daemon y started 2024-04-17T06:01:18.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:18 smithi098 ceph-mon[36516]: from='mgr.? 172.21.15.98:0/237736639' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-17T06:01:18.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:18 smithi098 ceph-mon[36516]: from='mgr.? 172.21.15.98:0/237736639' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-17T06:01:18.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:18 smithi098 ceph-mon[36516]: from='mgr.? 172.21.15.98:0/237736639' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-17T06:01:18.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:18 smithi098 ceph-mon[36516]: from='mgr.? 172.21.15.98:0/237736639' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-17T06:01:18.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:18 smithi098 ceph-mon[36516]: Standby manager daemon z restarted 2024-04-17T06:01:18.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:18 smithi098 ceph-mon[36516]: Standby manager daemon z started 2024-04-17T06:01:18.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:18 smithi098 ceph-mon[36516]: from='mgr.? 172.21.15.103:0/4151991708' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-17T06:01:18.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:18 smithi098 ceph-mon[36516]: from='mgr.? 172.21.15.103:0/4151991708' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-17T06:01:18.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:18 smithi098 ceph-mon[36516]: from='mgr.? 172.21.15.103:0/4151991708' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-17T06:01:18.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:18 smithi098 ceph-mon[36516]: from='mgr.? 172.21.15.103:0/4151991708' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-17T06:01:18.516 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:18 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/1880394528' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-17T06:01:18.516 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:18 smithi103 ceph-mon[36243]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-17T06:01:18.516 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:18 smithi103 ceph-mon[36243]: Standby manager daemon y restarted 2024-04-17T06:01:18.516 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:18 smithi103 ceph-mon[36243]: Standby manager daemon y started 2024-04-17T06:01:18.516 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:18 smithi103 ceph-mon[36243]: from='mgr.? 172.21.15.98:0/237736639' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-17T06:01:18.516 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:18 smithi103 ceph-mon[36243]: from='mgr.? 172.21.15.98:0/237736639' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-17T06:01:18.516 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:18 smithi103 ceph-mon[36243]: from='mgr.? 172.21.15.98:0/237736639' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-17T06:01:18.516 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:18 smithi103 ceph-mon[36243]: from='mgr.? 172.21.15.98:0/237736639' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-17T06:01:18.517 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:18 smithi103 ceph-mon[36243]: Standby manager daemon z restarted 2024-04-17T06:01:18.517 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:18 smithi103 ceph-mon[36243]: Standby manager daemon z started 2024-04-17T06:01:18.517 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:18 smithi103 ceph-mon[36243]: from='mgr.? 172.21.15.103:0/4151991708' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-17T06:01:18.517 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:18 smithi103 ceph-mon[36243]: from='mgr.? 172.21.15.103:0/4151991708' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-17T06:01:18.517 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:18 smithi103 ceph-mon[36243]: from='mgr.? 172.21.15.103:0/4151991708' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-17T06:01:18.517 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:18 smithi103 ceph-mon[36243]: from='mgr.? 172.21.15.103:0/4151991708' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-17T06:01:18.531 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:18 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/1880394528' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-17T06:01:18.531 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:18 smithi047 ceph-mon[32223]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-17T06:01:18.531 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:18 smithi047 ceph-mon[32223]: Standby manager daemon y restarted 2024-04-17T06:01:18.531 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:18 smithi047 ceph-mon[32223]: Standby manager daemon y started 2024-04-17T06:01:18.532 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:18 smithi047 ceph-mon[32223]: from='mgr.? 172.21.15.98:0/237736639' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-17T06:01:18.532 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:18 smithi047 ceph-mon[32223]: from='mgr.? 172.21.15.98:0/237736639' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-17T06:01:18.532 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:18 smithi047 ceph-mon[32223]: from='mgr.? 172.21.15.98:0/237736639' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-17T06:01:18.532 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:18 smithi047 ceph-mon[32223]: from='mgr.? 172.21.15.98:0/237736639' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-17T06:01:18.532 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:18 smithi047 ceph-mon[32223]: Standby manager daemon z restarted 2024-04-17T06:01:18.532 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:18 smithi047 ceph-mon[32223]: Standby manager daemon z started 2024-04-17T06:01:18.532 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:18 smithi047 ceph-mon[32223]: from='mgr.? 172.21.15.103:0/4151991708' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-17T06:01:18.532 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:18 smithi047 ceph-mon[32223]: from='mgr.? 172.21.15.103:0/4151991708' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-17T06:01:18.532 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:18 smithi047 ceph-mon[32223]: from='mgr.? 172.21.15.103:0/4151991708' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-17T06:01:18.532 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:18 smithi047 ceph-mon[32223]: from='mgr.? 172.21.15.103:0/4151991708' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-17T06:01:18.533 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 06:01:18 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T06:01:18.244+0000 7effca8c7200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-17T06:01:18.533 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 06:01:18 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T06:01:18.487+0000 7eff2d2d7700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-17T06:01:18.533 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 06:01:18 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T06:01:18.488+0000 7eff2d2d7700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-17T06:01:18.533 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 06:01:18 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T06:01:18.488+0000 7eff2d2d7700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-17T06:01:18.533 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 06:01:18 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T06:01:18.488+0000 7eff2d2d7700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-17T06:01:18.533 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 06:01:18 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T06:01:18.488+0000 7eff2d2d7700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-17T06:01:18.533 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 06:01:18 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T06:01:18.488+0000 7eff29ad0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-17T06:01:18.533 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 06:01:18 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T06:01:18.488+0000 7eff29ad0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-17T06:01:18.533 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 06:01:18 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T06:01:18.488+0000 7eff29ad0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-17T06:01:18.533 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 06:01:18 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T06:01:18.488+0000 7eff29ad0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-17T06:01:18.533 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 06:01:18 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T06:01:18.488+0000 7eff29ad0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-17T06:01:19.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:19.264+0000 7fd80c970640 1 -- 172.21.15.47:0/495460483 <== mon.0 v2:172.21.15.47:3300/0 9 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7fd7f009aa70 con 0x7fd808102050 2024-04-17T06:01:19.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:19.265+0000 7fd80c970640 1 --2- 172.21.15.47:0/495460483 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fd7f4035600 0x7fd7f40359e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:19.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:19.265+0000 7fd80c970640 1 -- 172.21.15.47:0/495460483 --> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7fd7f009ba30 con 0x7fd7f4035600 2024-04-17T06:01:19.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:19.267+0000 7fd806ffd640 1 --2- 172.21.15.47:0/495460483 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fd7f4035600 0x7fd7f40359e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:19.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:19.270+0000 7fd806ffd640 1 --2- 172.21.15.47:0/495460483 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fd7f4035600 0x7fd7f40359e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fd7f800a010 tx=0x7fd7f8005f90 comp rx=0 tx=0).ready entity=mgr.24923 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:01:19.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:19.290+0000 7fd80c970640 1 -- 172.21.15.47:0/495460483 <== mgr.24923 v2:172.21.15.47:6800/255579447 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7fd7f009ba30 con 0x7fd7f4035600 2024-04-17T06:01:19.287 INFO:teuthology.orchestra.run.smithi047.stdout:{} 2024-04-17T06:01:19.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:19.292+0000 7fd80d972640 1 -- 172.21.15.47:0/495460483 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fd7f4035600 msgr2=0x7fd7f40359e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:19.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:19.292+0000 7fd80d972640 1 --2- 172.21.15.47:0/495460483 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fd7f4035600 0x7fd7f40359e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fd7f800a010 tx=0x7fd7f8005f90 comp rx=0 tx=0).stop 2024-04-17T06:01:19.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:19.292+0000 7fd80d972640 1 -- 172.21.15.47:0/495460483 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd808102050 msgr2=0x7fd80819cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:19.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:19.292+0000 7fd80d972640 1 --2- 172.21.15.47:0/495460483 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd808102050 0x7fd80819cf10 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fd7f000e9f0 tx=0x7fd7f000edf0 comp rx=0 tx=0).stop 2024-04-17T06:01:19.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:19.292+0000 7fd8077fe640 1 -- 172.21.15.47:0/495460483 reap_dead start 2024-04-17T06:01:19.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:19.292+0000 7fd80d972640 1 -- 172.21.15.47:0/495460483 shutdown_connections 2024-04-17T06:01:19.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:19.292+0000 7fd80d972640 1 -- 172.21.15.47:0/495460483 >> 172.21.15.47:0/495460483 conn(0x7fd8080fc430 msgr2=0x7fd808105380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:01:19.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:19.292+0000 7fd80d972640 1 -- 172.21.15.47:0/495460483 shutdown_connections 2024-04-17T06:01:19.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:19.292+0000 7fd80d972640 1 -- 172.21.15.47:0/495460483 wait complete. 2024-04-17T06:01:19.302 INFO:teuthology.run_tasks:Running task check-counter... 2024-04-17T06:01:19.314 INFO:teuthology.run_tasks:Running task exec... 2024-04-17T06:01:19.322 INFO:teuthology.task.exec:Executing custom commands... 2024-04-17T06:01:19.322 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi047.front.sepia.ceph.com 2024-04-17T06:01:19.322 DEBUG:teuthology.orchestra.run.smithi047:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-04-17T06:01:19.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:19.435+0000 7f5ab2e9a640 1 -- 172.21.15.47:0/1100343147 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5aac102aa0 msgr2=0x7f5aac104e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:19.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:19.435+0000 7f5ab2e9a640 1 --2- 172.21.15.47:0/1100343147 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5aac102aa0 0x7f5aac104e90 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f5aa0009900 tx=0x7f5aa002bc30 comp rx=0 tx=0).stop 2024-04-17T06:01:19.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:19.435+0000 7f5ab2e9a640 1 -- 172.21.15.47:0/1100343147 shutdown_connections 2024-04-17T06:01:19.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:19.435+0000 7f5ab2e9a640 1 --2- 172.21.15.47:0/1100343147 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5aac107d00 0x7f5aac10a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:19.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:19.435+0000 7f5ab2e9a640 1 --2- 172.21.15.47:0/1100343147 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5aac1053d0 0x7f5aac1077c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:19.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:19.435+0000 7f5ab2e9a640 1 --2- 172.21.15.47:0/1100343147 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5aac102aa0 0x7f5aac104e90 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:19.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:19.435+0000 7f5ab2e9a640 1 -- 172.21.15.47:0/1100343147 >> 172.21.15.47:0/1100343147 conn(0x7f5aac0fc430 msgr2=0x7f5aac0fe870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:01:19.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:19.435+0000 7f5ab2e9a640 1 -- 172.21.15.47:0/1100343147 shutdown_connections 2024-04-17T06:01:19.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:19.435+0000 7f5ab2e9a640 1 -- 172.21.15.47:0/1100343147 wait complete. 2024-04-17T06:01:19.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:19.435+0000 7f5ab2e9a640 1 Processor -- start 2024-04-17T06:01:19.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:19.435+0000 7f5ab2e9a640 1 -- start start 2024-04-17T06:01:19.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:19.436+0000 7f5ab2e9a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5aac102aa0 0x7f5aac198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:19.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:19.436+0000 7f5ab2e9a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5aac1053d0 0x7f5aac198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:19.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:19.436+0000 7f5ab2e9a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5aac107d00 0x7f5aac19f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:19.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:19.436+0000 7f5ab2e9a640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f5aac10dc50 con 0x7f5aac1053d0 2024-04-17T06:01:19.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:19.436+0000 7f5ab2e9a640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5aac10dad0 con 0x7f5aac107d00 2024-04-17T06:01:19.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:19.436+0000 7f5ab2e9a640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5aac10ddd0 con 0x7f5aac102aa0 2024-04-17T06:01:19.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:19.436+0000 7f5aabfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5aac1053d0 0x7f5aac198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:19.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:19.436+0000 7f5aabfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5aac1053d0 0x7f5aac198780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33662/0 (socket says 172.21.15.47:33662) 2024-04-17T06:01:19.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:19.436+0000 7f5aabfff640 1 -- 172.21.15.47:0/3393882901 learned_addr learned my addr 172.21.15.47:0/3393882901 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:01:19.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:19.436+0000 7f5ab0c36640 1 --2- 172.21.15.47:0/3393882901 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5aac102aa0 0x7f5aac198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:19.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:19.436+0000 7f5ab1437640 1 --2- 172.21.15.47:0/3393882901 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5aac107d00 0x7f5aac19f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:19.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:19.436+0000 7f5aabfff640 1 -- 172.21.15.47:0/3393882901 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5aac102aa0 msgr2=0x7f5aac198240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:19.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:19.436+0000 7f5aabfff640 1 --2- 172.21.15.47:0/3393882901 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5aac102aa0 0x7f5aac198240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:19.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:19.436+0000 7f5aabfff640 1 -- 172.21.15.47:0/3393882901 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5aac107d00 msgr2=0x7f5aac19f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:19.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:19.436+0000 7f5aabfff640 1 --2- 172.21.15.47:0/3393882901 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5aac107d00 0x7f5aac19f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:19.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:19.436+0000 7f5aabfff640 1 -- 172.21.15.47:0/3393882901 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5aac19fe60 con 0x7f5aac1053d0 2024-04-17T06:01:19.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:19.436+0000 7f5aabfff640 1 --2- 172.21.15.47:0/3393882901 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5aac1053d0 0x7f5aac198780 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f5a9800e9c0 tx=0x7f5a9800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:01:19.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:19.436+0000 7f5aa9ffb640 1 -- 172.21.15.47:0/3393882901 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a9800cd80 con 0x7f5aac1053d0 2024-04-17T06:01:19.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:19.436+0000 7f5aa9ffb640 1 -- 172.21.15.47:0/3393882901 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a98005500 con 0x7f5aac1053d0 2024-04-17T06:01:19.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:19.436+0000 7f5ab2e9a640 1 -- 172.21.15.47:0/3393882901 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5aac1a0150 con 0x7f5aac1053d0 2024-04-17T06:01:19.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:19.436+0000 7f5aa9ffb640 1 -- 172.21.15.47:0/3393882901 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a98010690 con 0x7f5aac1053d0 2024-04-17T06:01:19.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:19.437+0000 7f5ab2e9a640 1 -- 172.21.15.47:0/3393882901 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5aac1a06a0 con 0x7f5aac1053d0 2024-04-17T06:01:19.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:19.438+0000 7f5ab2e9a640 1 -- 172.21.15.47:0/3393882901 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5aac0fdf20 con 0x7f5aac1053d0 2024-04-17T06:01:19.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:19.439+0000 7f5aa9ffb640 1 -- 172.21.15.47:0/3393882901 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7f5a980046c0 con 0x7f5aac1053d0 2024-04-17T06:01:19.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:19.439+0000 7f5aa9ffb640 1 --2- 172.21.15.47:0/3393882901 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f5a9c0a3a60 0x7f5a9c0a5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:19.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:19.439+0000 7f5ab0c36640 1 --2- 172.21.15.47:0/3393882901 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f5a9c0a3a60 0x7f5a9c0a5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:19.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:19.439+0000 7f5aa9ffb640 1 -- 172.21.15.47:0/3393882901 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f5a980d0450 con 0x7f5aac1053d0 2024-04-17T06:01:19.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:19.440+0000 7f5ab0c36640 1 --2- 172.21.15.47:0/3393882901 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f5a9c0a3a60 0x7f5a9c0a5f20 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5aa002c7b0 tx=0x7f5aa0005fd0 comp rx=0 tx=0).ready entity=mgr.24923 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:01:19.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:19.444+0000 7f5aa9ffb640 1 -- 172.21.15.47:0/3393882901 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f5a9809d030 con 0x7f5aac1053d0 2024-04-17T06:01:19.505 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:19 smithi103 ceph-mon[36243]: mgrmap e20: x(active, since 9m), standbys: z, y 2024-04-17T06:01:19.505 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:19 smithi103 ceph-mon[36243]: Active manager daemon x restarted 2024-04-17T06:01:19.505 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:19 smithi103 ceph-mon[36243]: Activating manager daemon x 2024-04-17T06:01:19.505 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:19 smithi103 ceph-mon[36243]: osdmap e89: 12 total, 12 up, 12 in 2024-04-17T06:01:19.505 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:19 smithi103 ceph-mon[36243]: mgrmap e21: x(active, starting, since 0.0110546s), standbys: z, y 2024-04-17T06:01:19.505 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:19 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-17T06:01:19.505 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:19 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-17T06:01:19.505 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:19 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-17T06:01:19.505 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:19 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-17T06:01:19.505 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:19 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-17T06:01:19.505 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:19 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-17T06:01:19.505 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:19 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-17T06:01:19.505 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:19 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-17T06:01:19.506 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:19 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-17T06:01:19.506 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:19 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-17T06:01:19.506 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:19 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-17T06:01:19.506 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:19 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-17T06:01:19.506 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:19 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-17T06:01:19.506 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:19 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-17T06:01:19.506 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:19 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-17T06:01:19.506 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:19 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-17T06:01:19.506 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:19 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-17T06:01:19.506 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:19 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-17T06:01:19.506 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:19 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-17T06:01:19.506 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:19 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-17T06:01:19.506 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:19 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-17T06:01:19.506 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:19 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-17T06:01:19.507 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:19 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-17T06:01:19.507 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:19 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-17T06:01:19.507 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:19 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-17T06:01:19.507 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:19 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-17T06:01:19.507 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:19 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-17T06:01:19.507 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:19 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-17T06:01:19.507 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:19 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-17T06:01:19.507 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:19 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-17T06:01:19.507 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:19 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-17T06:01:19.507 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:19 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-17T06:01:19.507 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:19 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-17T06:01:19.507 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:19 smithi103 ceph-mon[36243]: Manager daemon x is now available 2024-04-17T06:01:19.507 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:19 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T06:01:19.507 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:19 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-17T06:01:19.507 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:19 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-17T06:01:19.529 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:19 smithi047 ceph-mon[32223]: mgrmap e20: x(active, since 9m), standbys: z, y 2024-04-17T06:01:19.529 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:19 smithi047 ceph-mon[32223]: Active manager daemon x restarted 2024-04-17T06:01:19.529 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:19 smithi047 ceph-mon[32223]: Activating manager daemon x 2024-04-17T06:01:19.530 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:19 smithi047 ceph-mon[32223]: osdmap e89: 12 total, 12 up, 12 in 2024-04-17T06:01:19.530 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:19 smithi047 ceph-mon[32223]: mgrmap e21: x(active, starting, since 0.0110546s), standbys: z, y 2024-04-17T06:01:19.530 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:19 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-17T06:01:19.530 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:19 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-17T06:01:19.530 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:19 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-17T06:01:19.530 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:19 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-17T06:01:19.530 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:19 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-17T06:01:19.530 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:19 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-17T06:01:19.530 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:19 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-17T06:01:19.530 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:19 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-17T06:01:19.530 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:19 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-17T06:01:19.530 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:19 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-17T06:01:19.530 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:19 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-17T06:01:19.531 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:19 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-17T06:01:19.531 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:19 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-17T06:01:19.531 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:19 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-17T06:01:19.531 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:19 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-17T06:01:19.531 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:19 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-17T06:01:19.531 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:19 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-17T06:01:19.531 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:19 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-17T06:01:19.531 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:19 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-17T06:01:19.531 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:19 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-17T06:01:19.531 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:19 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-17T06:01:19.531 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:19 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-17T06:01:19.531 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:19 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-17T06:01:19.532 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 06:01:19 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T06:01:19.278+0000 7eff272cb700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-17T06:01:19.532 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 06:01:19 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T06:01:19.278+0000 7eff272cb700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-17T06:01:19.532 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 06:01:19 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T06:01:19.278+0000 7eff272cb700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-17T06:01:19.532 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 06:01:19 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T06:01:19.278+0000 7eff272cb700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-17T06:01:19.532 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 06:01:19 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T06:01:19.278+0000 7eff272cb700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-17T06:01:19.532 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:19 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-17T06:01:19.532 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:19 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-17T06:01:19.532 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:19 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-17T06:01:19.532 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:19 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-17T06:01:19.532 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:19 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-17T06:01:19.533 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:19 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-17T06:01:19.533 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:19 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-17T06:01:19.533 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:19 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-17T06:01:19.533 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:19 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-17T06:01:19.533 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:19 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-17T06:01:19.533 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:19 smithi047 ceph-mon[32223]: Manager daemon x is now available 2024-04-17T06:01:19.533 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:19 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T06:01:19.533 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:19 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-17T06:01:19.533 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:19 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-17T06:01:19.629 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:19 smithi098 ceph-mon[36516]: mgrmap e20: x(active, since 9m), standbys: z, y 2024-04-17T06:01:19.629 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:19 smithi098 ceph-mon[36516]: Active manager daemon x restarted 2024-04-17T06:01:19.630 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:19 smithi098 ceph-mon[36516]: Activating manager daemon x 2024-04-17T06:01:19.630 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:19 smithi098 ceph-mon[36516]: osdmap e89: 12 total, 12 up, 12 in 2024-04-17T06:01:19.630 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:19 smithi098 ceph-mon[36516]: mgrmap e21: x(active, starting, since 0.0110546s), standbys: z, y 2024-04-17T06:01:19.630 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:19 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-17T06:01:19.630 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:19 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-17T06:01:19.630 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:19 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-17T06:01:19.630 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:19 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-17T06:01:19.630 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:19 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-17T06:01:19.630 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:19 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-17T06:01:19.630 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:19 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-17T06:01:19.630 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:19 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-17T06:01:19.630 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:19 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-17T06:01:19.630 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:19 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-17T06:01:19.630 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:19 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-17T06:01:19.631 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:19 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-17T06:01:19.631 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:19 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-17T06:01:19.631 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:19 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-17T06:01:19.631 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:19 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-17T06:01:19.631 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:19 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-17T06:01:19.631 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:19 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-17T06:01:19.631 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:19 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-17T06:01:19.631 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:19 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-17T06:01:19.631 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:19 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-17T06:01:19.631 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:19 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-17T06:01:19.631 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:19 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-17T06:01:19.631 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:19 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-17T06:01:19.631 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:19 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-17T06:01:19.631 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:19 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-17T06:01:19.631 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:19 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-17T06:01:19.632 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:19 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-17T06:01:19.632 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:19 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-17T06:01:19.632 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:19 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-17T06:01:19.632 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:19 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-17T06:01:19.632 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:19 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-17T06:01:19.632 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:19 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-17T06:01:19.632 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:19 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-17T06:01:19.632 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:19 smithi098 ceph-mon[36516]: Manager daemon x is now available 2024-04-17T06:01:19.632 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:19 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T06:01:19.632 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:19 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-17T06:01:19.632 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:19 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-17T06:01:19.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:19.656+0000 7f5ab2e9a640 1 -- 172.21.15.47:0/3393882901 --> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f5aac0623a0 con 0x7f5a9c0a3a60 2024-04-17T06:01:19.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:19.717+0000 7f5aa9ffb640 1 -- 172.21.15.47:0/3393882901 <== mgr.24923 v2:172.21.15.47:6800/255579447 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f5aac0623a0 con 0x7f5a9c0a3a60 2024-04-17T06:01:19.716 INFO:teuthology.orchestra.run.smithi047.stdout:Schedule set for path / 2024-04-17T06:01:19.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:19.720+0000 7f5ab2e9a640 1 -- 172.21.15.47:0/3393882901 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f5a9c0a3a60 msgr2=0x7f5a9c0a5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:19.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:19.720+0000 7f5ab2e9a640 1 --2- 172.21.15.47:0/3393882901 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f5a9c0a3a60 0x7f5a9c0a5f20 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5aa002c7b0 tx=0x7f5aa0005fd0 comp rx=0 tx=0).stop 2024-04-17T06:01:19.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:19.720+0000 7f5ab2e9a640 1 -- 172.21.15.47:0/3393882901 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5aac1053d0 msgr2=0x7f5aac198780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:19.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:19.720+0000 7f5ab2e9a640 1 --2- 172.21.15.47:0/3393882901 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5aac1053d0 0x7f5aac198780 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f5a9800e9c0 tx=0x7f5a9800ee80 comp rx=0 tx=0).stop 2024-04-17T06:01:19.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:19.720+0000 7f5ab2e9a640 1 -- 172.21.15.47:0/3393882901 shutdown_connections 2024-04-17T06:01:19.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:19.720+0000 7f5ab2e9a640 1 --2- 172.21.15.47:0/3393882901 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5aac107d00 0x7f5aac19f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:19.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:19.720+0000 7f5ab2e9a640 1 --2- 172.21.15.47:0/3393882901 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f5a9c0a3a60 0x7f5a9c0a5f20 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:19.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:19.720+0000 7f5ab2e9a640 1 --2- 172.21.15.47:0/3393882901 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5aac1053d0 0x7f5aac198780 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:19.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:19.720+0000 7f5ab2e9a640 1 --2- 172.21.15.47:0/3393882901 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5aac102aa0 0x7f5aac198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:19.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:19.720+0000 7f5ab2e9a640 1 -- 172.21.15.47:0/3393882901 >> 172.21.15.47:0/3393882901 conn(0x7f5aac0fc430 msgr2=0x7f5aac103360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:01:19.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:19.720+0000 7f5ab2e9a640 1 -- 172.21.15.47:0/3393882901 shutdown_connections 2024-04-17T06:01:19.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:19.720+0000 7f5ab2e9a640 1 -- 172.21.15.47:0/3393882901 wait complete. 2024-04-17T06:01:19.729 DEBUG:teuthology.orchestra.run.smithi047:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-04-17T06:01:19.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:19.842+0000 7efcb018f640 1 -- 172.21.15.47:0/3322015921 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efca8100f20 msgr2=0x7efca8101360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:19.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:19.842+0000 7efcb018f640 1 --2- 172.21.15.47:0/3322015921 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efca8100f20 0x7efca8101360 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7efca4009900 tx=0x7efca402bc30 comp rx=0 tx=0).stop 2024-04-17T06:01:19.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:19.842+0000 7efcb018f640 1 -- 172.21.15.47:0/3322015921 shutdown_connections 2024-04-17T06:01:19.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:19.842+0000 7efcb018f640 1 --2- 172.21.15.47:0/3322015921 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efca8102ab0 0x7efca81093d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:19.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:19.842+0000 7efcb018f640 1 --2- 172.21.15.47:0/3322015921 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efca81020f0 0x7efca8102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:19.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:19.842+0000 7efcb018f640 1 --2- 172.21.15.47:0/3322015921 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efca8100f20 0x7efca8101360 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:19.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:19.842+0000 7efcb018f640 1 -- 172.21.15.47:0/3322015921 >> 172.21.15.47:0/3322015921 conn(0x7efca80fc4e0 msgr2=0x7efca80fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:01:19.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:19.842+0000 7efcb018f640 1 -- 172.21.15.47:0/3322015921 shutdown_connections 2024-04-17T06:01:19.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:19.842+0000 7efcb018f640 1 -- 172.21.15.47:0/3322015921 wait complete. 2024-04-17T06:01:19.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:19.843+0000 7efcb018f640 1 Processor -- start 2024-04-17T06:01:19.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:19.843+0000 7efcb018f640 1 -- start start 2024-04-17T06:01:19.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:19.843+0000 7efcb018f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efca8100f20 0x7efca806b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:19.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:19.843+0000 7efcb018f640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efca81020f0 0x7efca819cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:19.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:19.843+0000 7efcb018f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efca8102ab0 0x7efca819f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:19.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:19.843+0000 7efcb018f640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7efca810cb70 con 0x7efca8100f20 2024-04-17T06:01:19.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:19.843+0000 7efcb018f640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7efca810c9f0 con 0x7efca81020f0 2024-04-17T06:01:19.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:19.843+0000 7efcb018f640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7efca810ccf0 con 0x7efca8102ab0 2024-04-17T06:01:19.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:19.843+0000 7efcadf2b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efca8100f20 0x7efca806b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:19.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:19.843+0000 7efcadf2b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efca8100f20 0x7efca806b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33678/0 (socket says 172.21.15.47:33678) 2024-04-17T06:01:19.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:19.843+0000 7efcadf2b640 1 -- 172.21.15.47:0/2222640840 learned_addr learned my addr 172.21.15.47:0/2222640840 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:01:19.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:19.843+0000 7efcad72a640 1 --2- 172.21.15.47:0/2222640840 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efca81020f0 0x7efca819cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:19.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:19.843+0000 7efcae72c640 1 --2- 172.21.15.47:0/2222640840 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efca8102ab0 0x7efca819f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:19.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:19.843+0000 7efcadf2b640 1 -- 172.21.15.47:0/2222640840 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efca8102ab0 msgr2=0x7efca819f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:19.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:19.843+0000 7efcadf2b640 1 --2- 172.21.15.47:0/2222640840 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efca8102ab0 0x7efca819f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:19.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:19.843+0000 7efcadf2b640 1 -- 172.21.15.47:0/2222640840 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efca81020f0 msgr2=0x7efca819cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:19.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:19.843+0000 7efcadf2b640 1 --2- 172.21.15.47:0/2222640840 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efca81020f0 0x7efca819cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:19.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:19.843+0000 7efcadf2b640 1 -- 172.21.15.47:0/2222640840 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efca819fea0 con 0x7efca8100f20 2024-04-17T06:01:19.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:19.843+0000 7efcad72a640 1 --2- 172.21.15.47:0/2222640840 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efca81020f0 0x7efca819cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:01:19.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:19.843+0000 7efcadf2b640 1 --2- 172.21.15.47:0/2222640840 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efca8100f20 0x7efca806b300 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7efca4000c00 tx=0x7efca4004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:01:19.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:19.844+0000 7efc9effd640 1 -- 172.21.15.47:0/2222640840 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca4041070 con 0x7efca8100f20 2024-04-17T06:01:19.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:19.844+0000 7efc9effd640 1 -- 172.21.15.47:0/2222640840 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efca40054e0 con 0x7efca8100f20 2024-04-17T06:01:19.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:19.844+0000 7efc9effd640 1 -- 172.21.15.47:0/2222640840 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca40346a0 con 0x7efca8100f20 2024-04-17T06:01:19.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:19.844+0000 7efcb018f640 1 -- 172.21.15.47:0/2222640840 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efca81a00d0 con 0x7efca8100f20 2024-04-17T06:01:19.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:19.844+0000 7efcb018f640 1 -- 172.21.15.47:0/2222640840 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efca81a0610 con 0x7efca8100f20 2024-04-17T06:01:19.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:19.845+0000 7efcb018f640 1 -- 172.21.15.47:0/2222640840 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efca81082b0 con 0x7efca8100f20 2024-04-17T06:01:19.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:19.846+0000 7efc9effd640 1 -- 172.21.15.47:0/2222640840 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7efca4044070 con 0x7efca8100f20 2024-04-17T06:01:19.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:19.846+0000 7efc9effd640 1 --2- 172.21.15.47:0/2222640840 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7efc940a3ae0 0x7efc940a5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:19.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:19.847+0000 7efcad72a640 1 --2- 172.21.15.47:0/2222640840 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7efc940a3ae0 0x7efc940a5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:19.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:19.847+0000 7efc9effd640 1 -- 172.21.15.47:0/2222640840 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7efca40f2060 con 0x7efca8100f20 2024-04-17T06:01:19.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:19.847+0000 7efcad72a640 1 --2- 172.21.15.47:0/2222640840 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7efc940a3ae0 0x7efc940a5fa0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7efca806b070 tx=0x7efc98005ec0 comp rx=0 tx=0).ready entity=mgr.24923 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:01:19.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:19.850+0000 7efc9effd640 1 -- 172.21.15.47:0/2222640840 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7efca40bec40 con 0x7efca8100f20 2024-04-17T06:01:20.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.063+0000 7efcb018f640 1 -- 172.21.15.47:0/2222640840 --> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7efca8062500 con 0x7efc940a3ae0 2024-04-17T06:01:20.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.124+0000 7efc9effd640 1 -- 172.21.15.47:0/2222640840 <== mgr.24923 v2:172.21.15.47:6800/255579447 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7efca8062500 con 0x7efc940a3ae0 2024-04-17T06:01:20.122 INFO:teuthology.orchestra.run.smithi047.stdout:Retention added to path / 2024-04-17T06:01:20.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.127+0000 7efcb018f640 1 -- 172.21.15.47:0/2222640840 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7efc940a3ae0 msgr2=0x7efc940a5fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:20.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.127+0000 7efcb018f640 1 --2- 172.21.15.47:0/2222640840 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7efc940a3ae0 0x7efc940a5fa0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7efca806b070 tx=0x7efc98005ec0 comp rx=0 tx=0).stop 2024-04-17T06:01:20.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.127+0000 7efcb018f640 1 -- 172.21.15.47:0/2222640840 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efca8100f20 msgr2=0x7efca806b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:20.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.127+0000 7efcb018f640 1 --2- 172.21.15.47:0/2222640840 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efca8100f20 0x7efca806b300 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7efca4000c00 tx=0x7efca4004040 comp rx=0 tx=0).stop 2024-04-17T06:01:20.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.127+0000 7efcb018f640 1 -- 172.21.15.47:0/2222640840 shutdown_connections 2024-04-17T06:01:20.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.127+0000 7efcb018f640 1 --2- 172.21.15.47:0/2222640840 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efca8102ab0 0x7efca819f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:20.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.127+0000 7efcb018f640 1 --2- 172.21.15.47:0/2222640840 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efca81020f0 0x7efca819cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:20.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.127+0000 7efcb018f640 1 --2- 172.21.15.47:0/2222640840 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7efc940a3ae0 0x7efc940a5fa0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:20.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.127+0000 7efcb018f640 1 --2- 172.21.15.47:0/2222640840 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efca8100f20 0x7efca806b300 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:20.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.127+0000 7efcb018f640 1 -- 172.21.15.47:0/2222640840 >> 172.21.15.47:0/2222640840 conn(0x7efca80fc4e0 msgr2=0x7efca8105480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:01:20.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.127+0000 7efcb018f640 1 -- 172.21.15.47:0/2222640840 shutdown_connections 2024-04-17T06:01:20.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.127+0000 7efcb018f640 1 -- 172.21.15.47:0/2222640840 wait complete. 2024-04-17T06:01:20.137 DEBUG:teuthology.orchestra.run.smithi047:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-17T06:01:20.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.253+0000 7f7244350640 1 -- 172.21.15.47:0/3766647906 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f723c1053d0 msgr2=0x7f723c105850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:20.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.253+0000 7f7244350640 1 --2- 172.21.15.47:0/3766647906 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f723c1053d0 0x7f723c105850 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f722c009900 tx=0x7f722c02bc30 comp rx=0 tx=0).stop 2024-04-17T06:01:20.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.253+0000 7f7244350640 1 -- 172.21.15.47:0/3766647906 shutdown_connections 2024-04-17T06:01:20.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.253+0000 7f7244350640 1 --2- 172.21.15.47:0/3766647906 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f723c105d90 0x7f723c10c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:20.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.253+0000 7f7244350640 1 --2- 172.21.15.47:0/3766647906 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f723c1053d0 0x7f723c105850 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:20.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.253+0000 7f7244350640 1 --2- 172.21.15.47:0/3766647906 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f723c1041e0 0x7f723c104600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:20.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.253+0000 7f7244350640 1 -- 172.21.15.47:0/3766647906 >> 172.21.15.47:0/3766647906 conn(0x7f723c0ff750 msgr2=0x7f723c101b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:01:20.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.253+0000 7f7244350640 1 -- 172.21.15.47:0/3766647906 shutdown_connections 2024-04-17T06:01:20.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.253+0000 7f7244350640 1 -- 172.21.15.47:0/3766647906 wait complete. 2024-04-17T06:01:20.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.254+0000 7f7244350640 1 Processor -- start 2024-04-17T06:01:20.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.254+0000 7f7244350640 1 -- start start 2024-04-17T06:01:20.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.254+0000 7f7244350640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f723c1041e0 0x7f723c19a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:20.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.254+0000 7f7244350640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f723c1053d0 0x7f723c19a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:20.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.254+0000 7f7244350640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f723c105d90 0x7f723c1a1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:20.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.254+0000 7f7244350640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f723c10fe70 con 0x7f723c105d90 2024-04-17T06:01:20.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.254+0000 7f7244350640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f723c10fcf0 con 0x7f723c1041e0 2024-04-17T06:01:20.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.255+0000 7f7244350640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f723c10fff0 con 0x7f723c1053d0 2024-04-17T06:01:20.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.255+0000 7f72418eb640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f723c1053d0 0x7f723c19a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:20.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.255+0000 7f72420ec640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f723c1041e0 0x7f723c19a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:20.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.255+0000 7f72428ed640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f723c105d90 0x7f723c1a1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:20.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.255+0000 7f72420ec640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f723c1041e0 0x7f723c19a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.47:41882/0 (socket says 172.21.15.47:41882) 2024-04-17T06:01:20.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.255+0000 7f72420ec640 1 -- 172.21.15.47:0/694737685 learned_addr learned my addr 172.21.15.47:0/694737685 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:01:20.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.255+0000 7f72428ed640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f723c105d90 0x7f723c1a1a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33696/0 (socket says 172.21.15.47:33696) 2024-04-17T06:01:20.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.255+0000 7f72420ec640 1 -- 172.21.15.47:0/694737685 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f723c1053d0 msgr2=0x7f723c19a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:20.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.255+0000 7f72420ec640 1 --2- 172.21.15.47:0/694737685 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f723c1053d0 0x7f723c19a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:20.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.255+0000 7f72420ec640 1 -- 172.21.15.47:0/694737685 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f723c105d90 msgr2=0x7f723c1a1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:20.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.255+0000 7f72420ec640 1 --2- 172.21.15.47:0/694737685 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f723c105d90 0x7f723c1a1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:20.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.255+0000 7f72420ec640 1 -- 172.21.15.47:0/694737685 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f723c1a20b0 con 0x7f723c1041e0 2024-04-17T06:01:20.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.255+0000 7f72418eb640 1 --2- 172.21.15.47:0/694737685 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f723c1053d0 0x7f723c19a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T06:01:20.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.255+0000 7f72428ed640 1 --2- 172.21.15.47:0/694737685 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f723c105d90 0x7f723c1a1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T06:01:20.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.255+0000 7f72420ec640 1 --2- 172.21.15.47:0/694737685 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f723c1041e0 0x7f723c19a490 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f723800e9c0 tx=0x7f723800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:01:20.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.256+0000 7f72337fe640 1 -- 172.21.15.47:0/694737685 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f723800cd80 con 0x7f723c1041e0 2024-04-17T06:01:20.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.256+0000 7f72337fe640 1 -- 172.21.15.47:0/694737685 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72380035f0 con 0x7f723c1041e0 2024-04-17T06:01:20.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.256+0000 7f7244350640 1 -- 172.21.15.47:0/694737685 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f723c1a23a0 con 0x7f723c1041e0 2024-04-17T06:01:20.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.256+0000 7f72337fe640 1 -- 172.21.15.47:0/694737685 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7238010690 con 0x7f723c1041e0 2024-04-17T06:01:20.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.257+0000 7f7244350640 1 -- 172.21.15.47:0/694737685 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f723c1a2910 con 0x7f723c1041e0 2024-04-17T06:01:20.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.258+0000 7f7244350640 1 -- 172.21.15.47:0/694737685 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f723c04fcf0 con 0x7f723c1041e0 2024-04-17T06:01:20.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.259+0000 7f72337fe640 1 -- 172.21.15.47:0/694737685 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7f7238010830 con 0x7f723c1041e0 2024-04-17T06:01:20.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.260+0000 7f72337fe640 1 --2- 172.21.15.47:0/694737685 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f72280a3ae0 0x7f72280a5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:20.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.260+0000 7f72418eb640 1 --2- 172.21.15.47:0/694737685 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f72280a3ae0 0x7f72280a5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:20.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.260+0000 7f72337fe640 1 -- 172.21.15.47:0/694737685 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f72380d0480 con 0x7f723c1041e0 2024-04-17T06:01:20.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.260+0000 7f72418eb640 1 --2- 172.21.15.47:0/694737685 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f72280a3ae0 0x7f72280a5fa0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f722c009230 tx=0x7f722c005fd0 comp rx=0 tx=0).ready entity=mgr.24923 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:01:20.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.267+0000 7f72337fe640 1 -- 172.21.15.47:0/694737685 <== mon.2 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f723809d020 con 0x7f723c1041e0 2024-04-17T06:01:20.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.276+0000 7f72337fe640 1 -- 172.21.15.47:0/694737685 <== mon.2 v2:172.21.15.98:3300/0 7 ==== mgrmap(e 23) ==== 138817+0+0 (secure 0 0 0) 0x7f7238098f90 con 0x7f723c1041e0 2024-04-17T06:01:20.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:20 smithi047 ceph-mon[32223]: mgrmap e22: x(active, since 1.01854s), standbys: z, y 2024-04-17T06:01:20.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:20 smithi047 ceph-mon[32223]: [17/Apr/2024:06:01:19] ENGINE Bus STARTING 2024-04-17T06:01:20.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:20 smithi047 ceph-mon[32223]: [17/Apr/2024:06:01:19] ENGINE Serving on http://172.21.15.47:8765 2024-04-17T06:01:20.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.526+0000 7f7244350640 1 -- 172.21.15.47:0/694737685 --> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f723c0623a0 con 0x7f72280a3ae0 2024-04-17T06:01:20.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.534+0000 7f72337fe640 1 -- 172.21.15.47:0/694737685 <== mgr.24923 v2:172.21.15.47:6800/255579447 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7f723c0623a0 con 0x7f72280a3ae0 2024-04-17T06:01:20.532 INFO:teuthology.orchestra.run.smithi047.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-17T00:00:00", "created": "2024-04-17T06:01:19", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-04-17T06:01:20.533 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:20 smithi103 ceph-mon[36243]: mgrmap e22: x(active, since 1.01854s), standbys: z, y 2024-04-17T06:01:20.533 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:20 smithi103 ceph-mon[36243]: [17/Apr/2024:06:01:19] ENGINE Bus STARTING 2024-04-17T06:01:20.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:20 smithi103 ceph-mon[36243]: [17/Apr/2024:06:01:19] ENGINE Serving on http://172.21.15.47:8765 2024-04-17T06:01:20.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.536+0000 7f7244350640 1 -- 172.21.15.47:0/694737685 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f72280a3ae0 msgr2=0x7f72280a5fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:20.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.536+0000 7f7244350640 1 --2- 172.21.15.47:0/694737685 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f72280a3ae0 0x7f72280a5fa0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f722c009230 tx=0x7f722c005fd0 comp rx=0 tx=0).stop 2024-04-17T06:01:20.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.536+0000 7f7244350640 1 -- 172.21.15.47:0/694737685 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f723c1041e0 msgr2=0x7f723c19a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:20.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.536+0000 7f7244350640 1 --2- 172.21.15.47:0/694737685 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f723c1041e0 0x7f723c19a490 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f723800e9c0 tx=0x7f723800ee80 comp rx=0 tx=0).stop 2024-04-17T06:01:20.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.536+0000 7f7244350640 1 -- 172.21.15.47:0/694737685 shutdown_connections 2024-04-17T06:01:20.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.536+0000 7f7244350640 1 --2- 172.21.15.47:0/694737685 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f72280a3ae0 0x7f72280a5fa0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:20.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.536+0000 7f7244350640 1 --2- 172.21.15.47:0/694737685 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f723c105d90 0x7f723c1a1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:20.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.536+0000 7f7244350640 1 --2- 172.21.15.47:0/694737685 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f723c1053d0 0x7f723c19a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:20.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.536+0000 7f7244350640 1 --2- 172.21.15.47:0/694737685 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f723c1041e0 0x7f723c19a490 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:20.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.536+0000 7f7244350640 1 -- 172.21.15.47:0/694737685 >> 172.21.15.47:0/694737685 conn(0x7f723c0ff750 msgr2=0x7f723c108670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:01:20.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.536+0000 7f7244350640 1 -- 172.21.15.47:0/694737685 shutdown_connections 2024-04-17T06:01:20.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.536+0000 7f7244350640 1 -- 172.21.15.47:0/694737685 wait complete. 2024-04-17T06:01:20.545 DEBUG:teuthology.orchestra.run.smithi047:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-17T06:01:20.632 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:20 smithi098 ceph-mon[36516]: mgrmap e22: x(active, since 1.01854s), standbys: z, y 2024-04-17T06:01:20.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:20 smithi098 ceph-mon[36516]: [17/Apr/2024:06:01:19] ENGINE Bus STARTING 2024-04-17T06:01:20.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:20 smithi098 ceph-mon[36516]: [17/Apr/2024:06:01:19] ENGINE Serving on http://172.21.15.47:8765 2024-04-17T06:01:20.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.657+0000 7f09ab4ba640 1 -- 172.21.15.47:0/4282960208 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f09a4102a60 msgr2=0x7f09a4104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:20.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.657+0000 7f09ab4ba640 1 --2- 172.21.15.47:0/4282960208 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f09a4102a60 0x7f09a4104e50 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f09a0009900 tx=0x7f09a002bc30 comp rx=0 tx=0).stop 2024-04-17T06:01:20.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.657+0000 7f09ab4ba640 1 -- 172.21.15.47:0/4282960208 shutdown_connections 2024-04-17T06:01:20.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.657+0000 7f09ab4ba640 1 --2- 172.21.15.47:0/4282960208 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f09a4107cc0 0x7f09a410a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:20.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.657+0000 7f09ab4ba640 1 --2- 172.21.15.47:0/4282960208 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f09a4105390 0x7f09a4107780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:20.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.657+0000 7f09ab4ba640 1 --2- 172.21.15.47:0/4282960208 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f09a4102a60 0x7f09a4104e50 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:20.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.657+0000 7f09ab4ba640 1 -- 172.21.15.47:0/4282960208 >> 172.21.15.47:0/4282960208 conn(0x7f09a40fc470 msgr2=0x7f09a40fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:01:20.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.657+0000 7f09ab4ba640 1 -- 172.21.15.47:0/4282960208 shutdown_connections 2024-04-17T06:01:20.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.657+0000 7f09ab4ba640 1 -- 172.21.15.47:0/4282960208 wait complete. 2024-04-17T06:01:20.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.658+0000 7f09ab4ba640 1 Processor -- start 2024-04-17T06:01:20.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.658+0000 7f09ab4ba640 1 -- start start 2024-04-17T06:01:20.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.658+0000 7f09ab4ba640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f09a4102a60 0x7f09a41023d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:20.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.658+0000 7f09ab4ba640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f09a4105390 0x7f09a4100a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:20.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.658+0000 7f09ab4ba640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f09a4107cc0 0x7f09a4100f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:20.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.658+0000 7f09ab4ba640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f09a410dc70 con 0x7f09a4105390 2024-04-17T06:01:20.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.658+0000 7f09ab4ba640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f09a410daf0 con 0x7f09a4102a60 2024-04-17T06:01:20.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.658+0000 7f09ab4ba640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f09a410ddf0 con 0x7f09a4107cc0 2024-04-17T06:01:20.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.658+0000 7f09a8a55640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f09a4105390 0x7f09a4100a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:20.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.658+0000 7f09a8a55640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f09a4105390 0x7f09a4100a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33728/0 (socket says 172.21.15.47:33728) 2024-04-17T06:01:20.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.658+0000 7f09a8a55640 1 -- 172.21.15.47:0/1390561297 learned_addr learned my addr 172.21.15.47:0/1390561297 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:01:20.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.658+0000 7f09a9256640 1 --2- 172.21.15.47:0/1390561297 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f09a4102a60 0x7f09a41023d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:20.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.658+0000 7f09a9a57640 1 --2- 172.21.15.47:0/1390561297 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f09a4107cc0 0x7f09a4100f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:20.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.658+0000 7f09a8a55640 1 -- 172.21.15.47:0/1390561297 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f09a4107cc0 msgr2=0x7f09a4100f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:20.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.658+0000 7f09a8a55640 1 --2- 172.21.15.47:0/1390561297 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f09a4107cc0 0x7f09a4100f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:20.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.658+0000 7f09a8a55640 1 -- 172.21.15.47:0/1390561297 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f09a4102a60 msgr2=0x7f09a41023d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:20.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.658+0000 7f09a8a55640 1 --2- 172.21.15.47:0/1390561297 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f09a4102a60 0x7f09a41023d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:20.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.658+0000 7f09a8a55640 1 -- 172.21.15.47:0/1390561297 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09a41017e0 con 0x7f09a4105390 2024-04-17T06:01:20.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.658+0000 7f09a9256640 1 --2- 172.21.15.47:0/1390561297 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f09a4102a60 0x7f09a41023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T06:01:20.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.658+0000 7f09a9a57640 1 --2- 172.21.15.47:0/1390561297 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f09a4107cc0 0x7f09a4100f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T06:01:20.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.659+0000 7f09a8a55640 1 --2- 172.21.15.47:0/1390561297 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f09a4105390 0x7f09a4100a10 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f099800e9f0 tx=0x7f099800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:01:20.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.659+0000 7f09967fc640 1 -- 172.21.15.47:0/1390561297 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f099800ccc0 con 0x7f09a4105390 2024-04-17T06:01:20.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.659+0000 7f09967fc640 1 -- 172.21.15.47:0/1390561297 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0998005500 con 0x7f09a4105390 2024-04-17T06:01:20.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.659+0000 7f09ab4ba640 1 -- 172.21.15.47:0/1390561297 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09a41a2380 con 0x7f09a4105390 2024-04-17T06:01:20.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.659+0000 7f09967fc640 1 -- 172.21.15.47:0/1390561297 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0998010690 con 0x7f09a4105390 2024-04-17T06:01:20.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.659+0000 7f09ab4ba640 1 -- 172.21.15.47:0/1390561297 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09a41a2880 con 0x7f09a4105390 2024-04-17T06:01:20.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.660+0000 7f09ab4ba640 1 -- 172.21.15.47:0/1390561297 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09a40fdf60 con 0x7f09a4105390 2024-04-17T06:01:20.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.662+0000 7f09967fc640 1 -- 172.21.15.47:0/1390561297 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 23) ==== 138817+0+0 (secure 0 0 0) 0x7f09980046c0 con 0x7f09a4105390 2024-04-17T06:01:20.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.663+0000 7f09967fc640 1 --2- 172.21.15.47:0/1390561297 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f09900a3e60 0x7f09900a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:20.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.663+0000 7f09a9256640 1 --2- 172.21.15.47:0/1390561297 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f09900a3e60 0x7f09900a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:20.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.663+0000 7f09967fc640 1 -- 172.21.15.47:0/1390561297 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f09980d0fd0 con 0x7f09a4105390 2024-04-17T06:01:20.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.663+0000 7f09a9256640 1 --2- 172.21.15.47:0/1390561297 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f09900a3e60 0x7f09900a6320 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f09a002c7b0 tx=0x7f09a0005fd0 comp rx=0 tx=0).ready entity=mgr.24923 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:01:20.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.666+0000 7f09967fc640 1 -- 172.21.15.47:0/1390561297 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f0998053ff0 con 0x7f09a4105390 2024-04-17T06:01:20.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.877+0000 7f09ab4ba640 1 -- 172.21.15.47:0/1390561297 --> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f09a40623a0 con 0x7f09900a3e60 2024-04-17T06:01:20.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.884+0000 7f09967fc640 1 -- 172.21.15.47:0/1390561297 <== mgr.24923 v2:172.21.15.47:6800/255579447 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f09a40623a0 con 0x7f09900a3e60 2024-04-17T06:01:20.882 INFO:teuthology.orchestra.run.smithi047.stdout:/ 1m 6m3h 2024-04-17T06:01:20.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.886+0000 7f09ab4ba640 1 -- 172.21.15.47:0/1390561297 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f09900a3e60 msgr2=0x7f09900a6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:20.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.886+0000 7f09ab4ba640 1 --2- 172.21.15.47:0/1390561297 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f09900a3e60 0x7f09900a6320 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f09a002c7b0 tx=0x7f09a0005fd0 comp rx=0 tx=0).stop 2024-04-17T06:01:20.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.886+0000 7f09ab4ba640 1 -- 172.21.15.47:0/1390561297 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f09a4105390 msgr2=0x7f09a4100a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:20.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.886+0000 7f09ab4ba640 1 --2- 172.21.15.47:0/1390561297 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f09a4105390 0x7f09a4100a10 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f099800e9f0 tx=0x7f099800eeb0 comp rx=0 tx=0).stop 2024-04-17T06:01:20.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.886+0000 7f09ab4ba640 1 -- 172.21.15.47:0/1390561297 shutdown_connections 2024-04-17T06:01:20.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.886+0000 7f09ab4ba640 1 --2- 172.21.15.47:0/1390561297 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f09a4107cc0 0x7f09a4100f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:20.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.886+0000 7f09ab4ba640 1 --2- 172.21.15.47:0/1390561297 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f09900a3e60 0x7f09900a6320 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:20.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.886+0000 7f09ab4ba640 1 --2- 172.21.15.47:0/1390561297 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f09a4105390 0x7f09a4100a10 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:20.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.886+0000 7f09ab4ba640 1 --2- 172.21.15.47:0/1390561297 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f09a4102a60 0x7f09a41023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:20.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.886+0000 7f09ab4ba640 1 -- 172.21.15.47:0/1390561297 >> 172.21.15.47:0/1390561297 conn(0x7f09a40fc470 msgr2=0x7f09a4103560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:01:20.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.886+0000 7f09ab4ba640 1 -- 172.21.15.47:0/1390561297 shutdown_connections 2024-04-17T06:01:20.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:01:20.886+0000 7f09ab4ba640 1 -- 172.21.15.47:0/1390561297 wait complete. 2024-04-17T06:01:20.895 DEBUG:teuthology.orchestra.run.smithi047:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-04-17T06:01:20.924 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-04-17T06:01:20.935 INFO:teuthology.run_tasks:Running task background_exec... 2024-04-17T06:01:20.944 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi047.front.sepia.ceph.com 2024-04-17T06:01:20.944 DEBUG:teuthology.orchestra.run.smithi047:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-04-17T06:01:20.967 INFO:teuthology.run_tasks:Running task workunit... 2024-04-17T06:01:20.979 INFO:tasks.workunit:Pulling workunits from ref a8c84f5997950fd7d008204839253291d52e6eda 2024-04-17T06:01:20.979 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-17T06:01:20.979 INFO:tasks.workunit:timeout=3h 2024-04-17T06:01:20.979 INFO:tasks.workunit:cleanup=True 2024-04-17T06:01:20.979 DEBUG:teuthology.orchestra.run.smithi047:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-17T06:01:20.996 INFO:teuthology.orchestra.run.smithi047.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-04-17T06:01:20.996 INFO:teuthology.orchestra.run.smithi047.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-04-17T06:01:20.996 INFO:teuthology.orchestra.run.smithi047.stdout:Device: 77h/119d Inode: 1099511628283 Links: 2 2024-04-17T06:01:20.996 INFO:teuthology.orchestra.run.smithi047.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-17T06:01:20.996 INFO:teuthology.orchestra.run.smithi047.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-04-17T06:01:20.996 INFO:teuthology.orchestra.run.smithi047.stdout:Access: 2024-04-17 05:59:21.068268303 +0000 2024-04-17T06:01:20.996 INFO:teuthology.orchestra.run.smithi047.stdout:Modify: 2024-04-17 05:59:21.068268303 +0000 2024-04-17T06:01:20.996 INFO:teuthology.orchestra.run.smithi047.stdout:Change: 2024-04-17 06:01:16.464207673 +0000 2024-04-17T06:01:20.996 INFO:teuthology.orchestra.run.smithi047.stdout: Birth: 2024-04-17 05:59:21.068268303 +0000 2024-04-17T06:01:20.996 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-04-17T06:01:20.997 DEBUG:teuthology.orchestra.run.smithi047:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-04-17T06:01:21.075 DEBUG:teuthology.orchestra.run.smithi047:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout a8c84f5997950fd7d008204839253291d52e6eda 2024-04-17T06:01:21.131 INFO:tasks.workunit.client.0.smithi047.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-17T06:01:21.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:21 smithi103 ceph-mon[36243]: [17/Apr/2024:06:01:20] ENGINE Serving on https://172.21.15.47:7150 2024-04-17T06:01:21.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:21 smithi103 ceph-mon[36243]: [17/Apr/2024:06:01:20] ENGINE Bus STARTED 2024-04-17T06:01:21.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:21 smithi103 ceph-mon[36243]: from='client.15396 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T06:01:21.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:21 smithi103 ceph-mon[36243]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T06:01:21.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:21 smithi103 ceph-mon[36243]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-04-17T06:01:21.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:21 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:01:21.534 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:21 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:01:21.535 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:21 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:01:21.535 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:21 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:01:21.535 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:21 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:01:21.535 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:21 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:01:21.535 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:21 smithi103 ceph-mon[36243]: from='client.25001 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T06:01:21.535 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:21 smithi103 ceph-mon[36243]: from='client.15414 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T06:01:21.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:21 smithi098 ceph-mon[36516]: [17/Apr/2024:06:01:20] ENGINE Serving on https://172.21.15.47:7150 2024-04-17T06:01:21.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:21 smithi098 ceph-mon[36516]: [17/Apr/2024:06:01:20] ENGINE Bus STARTED 2024-04-17T06:01:21.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:21 smithi098 ceph-mon[36516]: from='client.15396 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T06:01:21.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:21 smithi098 ceph-mon[36516]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T06:01:21.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:21 smithi098 ceph-mon[36516]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-04-17T06:01:21.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:21 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:01:21.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:21 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:01:21.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:21 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:01:21.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:21 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:01:21.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:21 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:01:21.634 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:21 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:01:21.634 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:21 smithi098 ceph-mon[36516]: from='client.25001 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T06:01:21.634 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:21 smithi098 ceph-mon[36516]: from='client.15414 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T06:01:21.638 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:21 smithi047 ceph-mon[32223]: [17/Apr/2024:06:01:20] ENGINE Serving on https://172.21.15.47:7150 2024-04-17T06:01:21.638 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:21 smithi047 ceph-mon[32223]: [17/Apr/2024:06:01:20] ENGINE Bus STARTED 2024-04-17T06:01:21.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:21 smithi047 ceph-mon[32223]: from='client.15396 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T06:01:21.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:21 smithi047 ceph-mon[32223]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T06:01:21.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:21 smithi047 ceph-mon[32223]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-04-17T06:01:21.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:21 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:01:21.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:21 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:01:21.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:21 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:01:21.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:21 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:01:21.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:21 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:01:21.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:21 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:01:21.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:21 smithi047 ceph-mon[32223]: from='client.25001 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T06:01:21.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:21 smithi047 ceph-mon[32223]: from='client.15414 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T06:01:22.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:22 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:01:22.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:22 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:01:22.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:22 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:01:22.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:22 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-17T06:01:22.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:22 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:01:22.785 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:22 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-17T06:01:22.785 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:22 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-17T06:01:22.785 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:22 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-17T06:01:22.785 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:22 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-17T06:01:22.785 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:22 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-17T06:01:22.785 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:22 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-17T06:01:22.785 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:22 smithi103 ceph-mon[36243]: Adjusting osd_memory_target on smithi103 to 174.1M 2024-04-17T06:01:22.785 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:22 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-17T06:01:22.785 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:22 smithi103 ceph-mon[36243]: Unable to set osd_memory_target on smithi103 to 182637056: error parsing value: Value '182637056' is below minimum 939524096 2024-04-17T06:01:22.785 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:22 smithi103 ceph-mon[36243]: Adjusting osd_memory_target on smithi098 to 174.1M 2024-04-17T06:01:22.785 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:22 smithi103 ceph-mon[36243]: Unable to set osd_memory_target on smithi098 to 182632755: error parsing value: Value '182632755' is below minimum 939524096 2024-04-17T06:01:22.785 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:22 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:01:22.785 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:22 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:01:22.785 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:22 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-17T06:01:22.786 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:22 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-17T06:01:22.786 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:22 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-17T06:01:22.786 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:22 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-17T06:01:22.786 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:22 smithi103 ceph-mon[36243]: Adjusting osd_memory_target on smithi047 to 177.2M 2024-04-17T06:01:22.786 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:22 smithi103 ceph-mon[36243]: Unable to set osd_memory_target on smithi047 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-04-17T06:01:22.786 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:22 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T06:01:22.786 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:22 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T06:01:22.786 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:22 smithi103 ceph-mon[36243]: Updating smithi047:/etc/ceph/ceph.conf 2024-04-17T06:01:22.786 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:22 smithi103 ceph-mon[36243]: Updating smithi098:/etc/ceph/ceph.conf 2024-04-17T06:01:22.786 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:22 smithi103 ceph-mon[36243]: Updating smithi103:/etc/ceph/ceph.conf 2024-04-17T06:01:22.786 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:22 smithi103 ceph-mon[36243]: Updating smithi103:/var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/config/ceph.conf 2024-04-17T06:01:22.786 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:22 smithi103 ceph-mon[36243]: Updating smithi098:/var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/config/ceph.conf 2024-04-17T06:01:22.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:22 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:01:22.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:22 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:01:22.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:22 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:01:22.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:22 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-17T06:01:22.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:22 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:01:22.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:22 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-17T06:01:22.884 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:22 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-17T06:01:22.884 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:22 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-17T06:01:22.884 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:22 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-17T06:01:22.884 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:22 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-17T06:01:22.884 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:22 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-17T06:01:22.884 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:22 smithi098 ceph-mon[36516]: Adjusting osd_memory_target on smithi103 to 174.1M 2024-04-17T06:01:22.884 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:22 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-17T06:01:22.884 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:22 smithi098 ceph-mon[36516]: Unable to set osd_memory_target on smithi103 to 182637056: error parsing value: Value '182637056' is below minimum 939524096 2024-04-17T06:01:22.884 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:22 smithi098 ceph-mon[36516]: Adjusting osd_memory_target on smithi098 to 174.1M 2024-04-17T06:01:22.884 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:22 smithi098 ceph-mon[36516]: Unable to set osd_memory_target on smithi098 to 182632755: error parsing value: Value '182632755' is below minimum 939524096 2024-04-17T06:01:22.884 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:22 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:01:22.884 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:22 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:01:22.884 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:22 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-17T06:01:22.884 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:22 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-17T06:01:22.885 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:22 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-17T06:01:22.885 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:22 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-17T06:01:22.885 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:22 smithi098 ceph-mon[36516]: Adjusting osd_memory_target on smithi047 to 177.2M 2024-04-17T06:01:22.885 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:22 smithi098 ceph-mon[36516]: Unable to set osd_memory_target on smithi047 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-04-17T06:01:22.885 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:22 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T06:01:22.885 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:22 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T06:01:22.885 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:22 smithi098 ceph-mon[36516]: Updating smithi047:/etc/ceph/ceph.conf 2024-04-17T06:01:22.885 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:22 smithi098 ceph-mon[36516]: Updating smithi098:/etc/ceph/ceph.conf 2024-04-17T06:01:22.885 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:22 smithi098 ceph-mon[36516]: Updating smithi103:/etc/ceph/ceph.conf 2024-04-17T06:01:22.885 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:22 smithi098 ceph-mon[36516]: Updating smithi103:/var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/config/ceph.conf 2024-04-17T06:01:22.885 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:22 smithi098 ceph-mon[36516]: Updating smithi098:/var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/config/ceph.conf 2024-04-17T06:01:22.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:22 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:01:22.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:22 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:01:22.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:22 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:01:22.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:22 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-17T06:01:22.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:22 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:01:22.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:22 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-17T06:01:22.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:22 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-17T06:01:22.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:22 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-17T06:01:22.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:22 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-17T06:01:22.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:22 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-17T06:01:22.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:22 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-17T06:01:22.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:22 smithi047 ceph-mon[32223]: Adjusting osd_memory_target on smithi103 to 174.1M 2024-04-17T06:01:22.890 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:22 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-17T06:01:22.890 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:22 smithi047 ceph-mon[32223]: Unable to set osd_memory_target on smithi103 to 182637056: error parsing value: Value '182637056' is below minimum 939524096 2024-04-17T06:01:22.890 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:22 smithi047 ceph-mon[32223]: Adjusting osd_memory_target on smithi098 to 174.1M 2024-04-17T06:01:22.890 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:22 smithi047 ceph-mon[32223]: Unable to set osd_memory_target on smithi098 to 182632755: error parsing value: Value '182632755' is below minimum 939524096 2024-04-17T06:01:22.890 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:22 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:01:22.890 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:22 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:01:22.890 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:22 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-17T06:01:22.890 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:22 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-17T06:01:22.890 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:22 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-17T06:01:22.890 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:22 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-17T06:01:22.890 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:22 smithi047 ceph-mon[32223]: Adjusting osd_memory_target on smithi047 to 177.2M 2024-04-17T06:01:22.890 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:22 smithi047 ceph-mon[32223]: Unable to set osd_memory_target on smithi047 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-04-17T06:01:22.890 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:22 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T06:01:22.891 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:22 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T06:01:22.891 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:22 smithi047 ceph-mon[32223]: Updating smithi047:/etc/ceph/ceph.conf 2024-04-17T06:01:22.891 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:22 smithi047 ceph-mon[32223]: Updating smithi098:/etc/ceph/ceph.conf 2024-04-17T06:01:22.891 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:22 smithi047 ceph-mon[32223]: Updating smithi103:/etc/ceph/ceph.conf 2024-04-17T06:01:22.891 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:22 smithi047 ceph-mon[32223]: Updating smithi103:/var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/config/ceph.conf 2024-04-17T06:01:22.891 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:22 smithi047 ceph-mon[32223]: Updating smithi098:/var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/config/ceph.conf 2024-04-17T06:01:24.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:23 smithi103 ceph-mon[36243]: Updating smithi047:/var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/config/ceph.conf 2024-04-17T06:01:24.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:23 smithi103 ceph-mon[36243]: Updating smithi103:/etc/ceph/ceph.client.admin.keyring 2024-04-17T06:01:24.035 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:23 smithi103 ceph-mon[36243]: Updating smithi098:/etc/ceph/ceph.client.admin.keyring 2024-04-17T06:01:24.035 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:23 smithi103 ceph-mon[36243]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T06:01:24.035 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:23 smithi103 ceph-mon[36243]: Updating smithi047:/etc/ceph/ceph.client.admin.keyring 2024-04-17T06:01:24.035 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:23 smithi103 ceph-mon[36243]: Updating smithi103:/var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/config/ceph.client.admin.keyring 2024-04-17T06:01:24.035 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:23 smithi103 ceph-mon[36243]: Updating smithi098:/var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/config/ceph.client.admin.keyring 2024-04-17T06:01:24.035 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:23 smithi103 ceph-mon[36243]: Updating smithi047:/var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/config/ceph.client.admin.keyring 2024-04-17T06:01:24.035 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:23 smithi103 ceph-mon[36243]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-04-17T06:01:24.035 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:23 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:01:24.035 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:23 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:01:24.035 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:23 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:01:24.035 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:23 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:01:24.035 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:23 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:01:24.035 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:23 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:01:24.035 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:23 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:01:24.036 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:23 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T06:01:24.036 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:23 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T06:01:24.036 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:23 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T06:01:24.036 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:23 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:01:24.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:23 smithi098 ceph-mon[36516]: Updating smithi047:/var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/config/ceph.conf 2024-04-17T06:01:24.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:23 smithi098 ceph-mon[36516]: Updating smithi103:/etc/ceph/ceph.client.admin.keyring 2024-04-17T06:01:24.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:23 smithi098 ceph-mon[36516]: Updating smithi098:/etc/ceph/ceph.client.admin.keyring 2024-04-17T06:01:24.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:23 smithi098 ceph-mon[36516]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T06:01:24.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:23 smithi098 ceph-mon[36516]: Updating smithi047:/etc/ceph/ceph.client.admin.keyring 2024-04-17T06:01:24.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:23 smithi098 ceph-mon[36516]: Updating smithi103:/var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/config/ceph.client.admin.keyring 2024-04-17T06:01:24.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:23 smithi098 ceph-mon[36516]: Updating smithi098:/var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/config/ceph.client.admin.keyring 2024-04-17T06:01:24.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:23 smithi098 ceph-mon[36516]: Updating smithi047:/var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/config/ceph.client.admin.keyring 2024-04-17T06:01:24.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:23 smithi098 ceph-mon[36516]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-04-17T06:01:24.134 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:23 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:01:24.134 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:23 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:01:24.134 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:23 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:01:24.134 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:23 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:01:24.134 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:23 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:01:24.134 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:23 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:01:24.134 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:23 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:01:24.134 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:23 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T06:01:24.134 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:23 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T06:01:24.134 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:23 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T06:01:24.134 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:23 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:01:24.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:23 smithi047 ceph-mon[32223]: Updating smithi047:/var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/config/ceph.conf 2024-04-17T06:01:24.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:23 smithi047 ceph-mon[32223]: Updating smithi103:/etc/ceph/ceph.client.admin.keyring 2024-04-17T06:01:24.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:23 smithi047 ceph-mon[32223]: Updating smithi098:/etc/ceph/ceph.client.admin.keyring 2024-04-17T06:01:24.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:23 smithi047 ceph-mon[32223]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T06:01:24.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:23 smithi047 ceph-mon[32223]: Updating smithi047:/etc/ceph/ceph.client.admin.keyring 2024-04-17T06:01:24.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:23 smithi047 ceph-mon[32223]: Updating smithi103:/var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/config/ceph.client.admin.keyring 2024-04-17T06:01:24.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:23 smithi047 ceph-mon[32223]: Updating smithi098:/var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/config/ceph.client.admin.keyring 2024-04-17T06:01:24.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:23 smithi047 ceph-mon[32223]: Updating smithi047:/var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/config/ceph.client.admin.keyring 2024-04-17T06:01:24.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:23 smithi047 ceph-mon[32223]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-04-17T06:01:24.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:23 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:01:24.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:23 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:01:24.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:23 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:01:24.140 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:23 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:01:24.140 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:23 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:01:24.140 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:23 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:01:24.140 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:23 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:01:24.140 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:23 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T06:01:24.140 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:23 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T06:01:24.140 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:23 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T06:01:24.140 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:23 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:01:26.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:25 smithi103 ceph-mon[36243]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T06:01:26.051 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:25 smithi047 ceph-mon[32223]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T06:01:26.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:25 smithi098 ceph-mon[36516]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T06:01:28.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:27 smithi103 ceph-mon[36243]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 133 KiB/s wr, 78 op/s 2024-04-17T06:01:28.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:27 smithi098 ceph-mon[36516]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 133 KiB/s wr, 78 op/s 2024-04-17T06:01:28.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:27 smithi047 ceph-mon[32223]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 133 KiB/s wr, 78 op/s 2024-04-17T06:01:30.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:29 smithi103 ceph-mon[36243]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 103 KiB/s wr, 60 op/s 2024-04-17T06:01:30.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:29 smithi098 ceph-mon[36516]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 103 KiB/s wr, 60 op/s 2024-04-17T06:01:30.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:29 smithi047 ceph-mon[32223]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 103 KiB/s wr, 60 op/s 2024-04-17T06:01:32.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:31 smithi103 ceph-mon[36243]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 85 KiB/s wr, 52 op/s 2024-04-17T06:01:32.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:31 smithi098 ceph-mon[36516]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 85 KiB/s wr, 52 op/s 2024-04-17T06:01:32.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:31 smithi047 ceph-mon[32223]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 85 KiB/s wr, 52 op/s 2024-04-17T06:01:34.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:33 smithi103 ceph-mon[36243]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-04-17T06:01:34.114 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.115+0000 7f7294b59640 1 -- 172.21.15.47:0/1107042112 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7290101b70 msgr2=0x7f729010dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:34.114 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.115+0000 7f7294b59640 1 --2- 172.21.15.47:0/1107042112 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7290101b70 0x7f729010dc80 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f7288009ca0 tx=0x7f728802c140 comp rx=0 tx=0).stop 2024-04-17T06:01:34.114 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.115+0000 7f7294b59640 1 -- 172.21.15.47:0/1107042112 shutdown_connections 2024-04-17T06:01:34.114 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.115+0000 7f7294b59640 1 --2- 172.21.15.47:0/1107042112 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7290101b70 0x7f729010dc80 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:34.114 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.115+0000 7f7294b59640 1 --2- 172.21.15.47:0/1107042112 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f72901011b0 0x7f7290101630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:34.115 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.115+0000 7f7294b59640 1 --2- 172.21.15.47:0/1107042112 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7290106fc0 0x7f72901073a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:34.115 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.115+0000 7f7294b59640 1 -- 172.21.15.47:0/1107042112 >> 172.21.15.47:0/1107042112 conn(0x7f7290076c60 msgr2=0x7f7290077070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:01:34.115 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.115+0000 7f7294b59640 1 -- 172.21.15.47:0/1107042112 shutdown_connections 2024-04-17T06:01:34.115 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.115+0000 7f7294b59640 1 -- 172.21.15.47:0/1107042112 wait complete. 2024-04-17T06:01:34.117 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.116+0000 7f7294b59640 1 Processor -- start 2024-04-17T06:01:34.117 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.116+0000 7f7294b59640 1 -- start start 2024-04-17T06:01:34.117 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.116+0000 7f7294b59640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f72901011b0 0x7f7290198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:34.117 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.116+0000 7f7294b59640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7290101b70 0x7f7290198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:34.117 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.116+0000 7f7294b59640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7290106fc0 0x7f729019d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:34.117 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.116+0000 7f7294b59640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f7290111260 con 0x7f7290101b70 2024-04-17T06:01:34.117 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.116+0000 7f7294b59640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f72901110e0 con 0x7f72901011b0 2024-04-17T06:01:34.117 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.116+0000 7f7294b59640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f72901113e0 con 0x7f7290106fc0 2024-04-17T06:01:34.117 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.116+0000 7f7285d9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7290101b70 0x7f7290198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:34.117 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.116+0000 7f7285d9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7290101b70 0x7f7290198c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51948/0 (socket says 172.21.15.47:51948) 2024-04-17T06:01:34.118 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.116+0000 7f7285d9b640 1 -- 172.21.15.47:0/354714572 learned_addr learned my addr 172.21.15.47:0/354714572 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:01:34.118 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.116+0000 7f728ed9d640 1 --2- 172.21.15.47:0/354714572 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7290106fc0 0x7f729019d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:34.118 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.116+0000 7f728e59c640 1 --2- 172.21.15.47:0/354714572 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f72901011b0 0x7f7290198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:34.118 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.117+0000 7f7285d9b640 1 -- 172.21.15.47:0/354714572 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7290106fc0 msgr2=0x7f729019d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:34.118 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.117+0000 7f7285d9b640 1 --2- 172.21.15.47:0/354714572 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7290106fc0 0x7f729019d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:34.118 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.117+0000 7f7285d9b640 1 -- 172.21.15.47:0/354714572 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f72901011b0 msgr2=0x7f7290198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:34.118 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.117+0000 7f7285d9b640 1 --2- 172.21.15.47:0/354714572 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f72901011b0 0x7f7290198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:34.118 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.117+0000 7f7285d9b640 1 -- 172.21.15.47:0/354714572 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f729019d740 con 0x7f7290101b70 2024-04-17T06:01:34.118 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.117+0000 7f7285d9b640 1 --2- 172.21.15.47:0/354714572 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7290101b70 0x7f7290198c90 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f727800b4a0 tx=0x7f727800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:01:34.118 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.117+0000 7f7287fff640 1 -- 172.21.15.47:0/354714572 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7278014070 con 0x7f7290101b70 2024-04-17T06:01:34.118 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.117+0000 7f7294b59640 1 -- 172.21.15.47:0/354714572 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f729019da30 con 0x7f7290101b70 2024-04-17T06:01:34.118 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.117+0000 7f7287fff640 1 -- 172.21.15.47:0/354714572 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7278010af0 con 0x7f7290101b70 2024-04-17T06:01:34.118 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.117+0000 7f7287fff640 1 -- 172.21.15.47:0/354714572 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7278017d10 con 0x7f7290101b70 2024-04-17T06:01:34.119 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.117+0000 7f7294b59640 1 -- 172.21.15.47:0/354714572 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72901a5340 con 0x7f7290101b70 2024-04-17T06:01:34.126 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.119+0000 7f7287fff640 1 -- 172.21.15.47:0/354714572 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f7278017070 con 0x7f7290101b70 2024-04-17T06:01:34.126 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.120+0000 7f7297829740 1 Processor -- start 2024-04-17T06:01:34.126 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.120+0000 7f7297829740 1 -- start start 2024-04-17T06:01:34.126 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.120+0000 7f7297829740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bf182634d0 0x55bf182638b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:34.126 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.120+0000 7f7297829740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bf18263df0 0x55bf1826c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:34.126 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.120+0000 7f7297829740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf1826cbc0 0x55bf1826efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:34.126 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.120+0000 7f7297829740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55bf181c47d0 con 0x55bf182634d0 2024-04-17T06:01:34.126 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.120+0000 7f7297829740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55bf18161f70 con 0x55bf18263df0 2024-04-17T06:01:34.127 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.120+0000 7f7297829740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55bf181c0790 con 0x55bf1826cbc0 2024-04-17T06:01:34.127 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.120+0000 7f728ed9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bf182634d0 0x55bf182638b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:34.127 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.120+0000 7f728ed9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bf182634d0 0x55bf182638b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51958/0 (socket says 172.21.15.47:51958) 2024-04-17T06:01:34.127 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.120+0000 7f728ed9d640 1 -- 172.21.15.47:0/3213855241 learned_addr learned my addr 172.21.15.47:0/3213855241 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:01:34.127 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.121+0000 7f728e59c640 1 --2- 172.21.15.47:0/3213855241 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bf18263df0 0x55bf1826c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:34.127 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.121+0000 7f7287fff640 1 --2- 172.21.15.47:0/354714572 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f727c0a3ee0 0x7f727c0a63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:34.127 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.121+0000 7f7285d9b640 1 --2- 172.21.15.47:0/3213855241 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf1826cbc0 0x55bf1826efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:34.127 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.121+0000 7f728ed9d640 1 -- 172.21.15.47:0/3213855241 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf1826cbc0 msgr2=0x55bf1826efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:01:34.127 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.121+0000 7f728ed9d640 1 --2- 172.21.15.47:0/3213855241 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf1826cbc0 0x55bf1826efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:34.127 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.121+0000 7f728ed9d640 1 -- 172.21.15.47:0/3213855241 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bf18263df0 msgr2=0x55bf1826c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:01:34.127 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.121+0000 7f728ed9d640 1 --2- 172.21.15.47:0/3213855241 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bf18263df0 0x55bf1826c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:34.127 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.121+0000 7f728ed9d640 1 -- 172.21.15.47:0/3213855241 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf17ed7210 con 0x55bf182634d0 2024-04-17T06:01:34.127 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.121+0000 7f728e59c640 1 --2- 172.21.15.47:0/3213855241 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bf18263df0 0x55bf1826c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:01:34.127 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.121+0000 7f728ed9d640 1 --2- 172.21.15.47:0/354714572 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f727c0a3ee0 0x7f727c0a63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:34.128 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.121+0000 7f7287fff640 1 -- 172.21.15.47:0/354714572 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f72780cfca0 con 0x7f7290101b70 2024-04-17T06:01:34.128 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.121+0000 7f728ed9d640 1 --2- 172.21.15.47:0/3213855241 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bf182634d0 0x55bf182638b0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f7288009ca0 tx=0x7f72880323e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6d19a3d7a86c695 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:01:34.128 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.121+0000 7f7284d99640 1 -- 172.21.15.47:0/3213855241 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7288039040 con 0x55bf182634d0 2024-04-17T06:01:34.128 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.121+0000 7f7284d99640 1 -- 172.21.15.47:0/3213855241 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7288003d30 con 0x55bf182634d0 2024-04-17T06:01:34.128 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.121+0000 7f7284d99640 1 -- 172.21.15.47:0/3213855241 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7288034840 con 0x55bf182634d0 2024-04-17T06:01:34.128 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.122+0000 7f7297829740 1 -- 172.21.15.47:0/3213855241 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bf182634d0 msgr2=0x55bf182638b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:34.128 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.122+0000 7f728ed9d640 1 --2- 172.21.15.47:0/354714572 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f727c0a3ee0 0x7f727c0a63a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f728804c000 tx=0x7f7288045000 comp rx=0 tx=0).ready entity=mgr.24923 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:01:34.128 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.122+0000 7f7297829740 1 --2- 172.21.15.47:0/3213855241 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bf182634d0 0x55bf182638b0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f7288009ca0 tx=0x7f72880323e0 comp rx=0 tx=0).stop 2024-04-17T06:01:34.128 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.122+0000 7f7297829740 1 -- 172.21.15.47:0/3213855241 shutdown_connections 2024-04-17T06:01:34.128 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.122+0000 7f7297829740 1 --2- 172.21.15.47:0/3213855241 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf1826cbc0 0x55bf1826efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:34.128 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.122+0000 7f7297829740 1 --2- 172.21.15.47:0/3213855241 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bf18263df0 0x55bf1826c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:34.128 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.122+0000 7f7297829740 1 --2- 172.21.15.47:0/3213855241 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bf182634d0 0x55bf182638b0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:34.128 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.122+0000 7f7297829740 1 -- 172.21.15.47:0/3213855241 >> 172.21.15.47:0/3213855241 conn(0x55bf1825ddf0 msgr2=0x55bf1825e1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:01:34.128 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.122+0000 7f7297829740 1 -- 172.21.15.47:0/3213855241 shutdown_connections 2024-04-17T06:01:34.129 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.122+0000 7f7297829740 1 -- 172.21.15.47:0/3213855241 wait complete. 2024-04-17T06:01:34.129 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.123+0000 7f7297829740 1 -- start start 2024-04-17T06:01:34.129 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.123+0000 7f7297829740 1 Processor -- start 2024-04-17T06:01:34.129 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.123+0000 7f7297829740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bf182634d0 0x55bf18269bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:34.129 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.123+0000 7f7297829740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bf18263df0 0x55bf1826a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:34.129 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.123+0000 7f7297829740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf1826cbc0 0x55bf1826a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:34.129 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.123+0000 7f7297829740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55bf1826f7f0 con 0x55bf18263df0 2024-04-17T06:01:34.129 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.123+0000 7f7297829740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55bf1814ce90 con 0x55bf182634d0 2024-04-17T06:01:34.129 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.123+0000 7f7297829740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55bf1826f970 con 0x55bf1826cbc0 2024-04-17T06:01:34.129 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.123+0000 7f7297829740 10 client.0 _ll_register_callbacks cb 0x55bf18162d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:01:34.129 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.123+0000 7f7285d9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bf18263df0 0x55bf1826a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:34.129 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.123+0000 7f7285d9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bf18263df0 0x55bf1826a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51972/0 (socket says 172.21.15.47:51972) 2024-04-17T06:01:34.129 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.123+0000 7f7285d9b640 1 -- 172.21.15.47:0/2856946177 learned_addr learned my addr 172.21.15.47:0/2856946177 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:01:34.129 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.123+0000 7f728e59c640 1 --2- 172.21.15.47:0/2856946177 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bf182634d0 0x55bf18269bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:34.130 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.123+0000 7f728ed9d640 1 --2- 172.21.15.47:0/2856946177 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf1826cbc0 0x55bf1826a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:34.130 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.123+0000 7f7285d9b640 1 -- 172.21.15.47:0/2856946177 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf1826cbc0 msgr2=0x55bf1826a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:01:34.130 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.123+0000 7f7285d9b640 1 --2- 172.21.15.47:0/2856946177 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf1826cbc0 0x55bf1826a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:34.130 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.123+0000 7f7285d9b640 1 -- 172.21.15.47:0/2856946177 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bf182634d0 msgr2=0x55bf18269bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:01:34.130 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.123+0000 7f7285d9b640 1 --2- 172.21.15.47:0/2856946177 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bf182634d0 0x55bf18269bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:34.130 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.123+0000 7f7285d9b640 1 -- 172.21.15.47:0/2856946177 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf17fa3620 con 0x55bf18263df0 2024-04-17T06:01:34.130 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.123+0000 7f728e59c640 1 --2- 172.21.15.47:0/2856946177 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bf182634d0 0x55bf18269bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:01:34.130 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.124+0000 7f7285d9b640 1 --2- 172.21.15.47:0/2856946177 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bf18263df0 0x55bf1826a100 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f727800be60 tx=0x7f727809ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bc97af8222d20ad server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:01:34.130 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.124+0000 7f7275ffb640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-17T06:01:34.130 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.124+0000 7f7275ffb640 1 -- 172.21.15.47:0/2856946177 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f727801e070 con 0x55bf18263df0 2024-04-17T06:01:34.131 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.124+0000 7f7275ffb640 1 -- 172.21.15.47:0/2856946177 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f72780981f0 con 0x55bf18263df0 2024-04-17T06:01:34.131 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.124+0000 7f7297829740 1 -- 172.21.15.47:0/2856946177 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7288003d30 con 0x55bf18263df0 2024-04-17T06:01:34.131 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.124+0000 7f7275ffb640 1 -- 172.21.15.47:0/2856946177 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f727809fe80 con 0x55bf18263df0 2024-04-17T06:01:34.131 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.124+0000 7f7297829740 10 client.15426 fetch_fsmap learned FSMap version 22 2024-04-17T06:01:34.131 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.124+0000 7f7297829740 1 -- 172.21.15.47:0/2856946177 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55bf1827ba60 con 0x55bf18263df0 2024-04-17T06:01:34.131 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.124+0000 7f7275ffb640 1 -- 172.21.15.47:0/2856946177 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7288003d30 con 0x55bf18263df0 2024-04-17T06:01:34.131 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.124+0000 7f7275ffb640 1 -- 172.21.15.47:0/2856946177 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 22) ==== 8339+0+0 (secure 0 0 0) 0x7f727809afd0 con 0x55bf18263df0 2024-04-17T06:01:34.131 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.124+0000 7f7297829740 10 client.15426 fetch_fsmap finished waiting for FSMap version 22 2024-04-17T06:01:34.131 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.124+0000 7f7297829740 10 client.15426 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-17T06:01:34.131 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.124+0000 7f7297829740 20 client.15426 populate_metadata read hostname 'smithi047' 2024-04-17T06:01:34.131 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.124+0000 7f7297829740 1 --2- 172.21.15.47:0/2856946177 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55bf1827cf20 0x55bf1827f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:34.131 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.124+0000 7f7297829740 4 client.15426 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T06:01:34.131 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.124+0000 7f7297829740 1 -- 172.21.15.47:0/2856946177 --> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bf181d21f0 con 0x55bf1827cf20 2024-04-17T06:01:34.131 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.125+0000 7f728e59c640 1 --2- 172.21.15.47:0/2856946177 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55bf1827cf20 0x55bf1827f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:34.131 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.125+0000 7f728e59c640 1 --2- 172.21.15.47:0/2856946177 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55bf1827cf20 0x55bf1827f310 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65ba98f364363fd5 server_cookie=aede89f26cedb829 in_seq=0 out_seq=0 2024-04-17T06:01:34.131 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.125+0000 7f7275ffb640 10 client.15426 ms_handle_connect on v2:172.21.15.98:6838/2841287917 2024-04-17T06:01:34.132 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.126+0000 7f7275ffb640 1 -- 172.21.15.47:0/2856946177 <== mds.0 v2:172.21.15.98:6838/2841287917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bf181d21f0 con 0x55bf1827cf20 2024-04-17T06:01:34.132 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.126+0000 7f7275ffb640 10 client.15426 handle_command_reply: tid=0 2024-04-17T06:01:34.132 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.126+0000 7f7297829740 1 client.15426 shutdown 2024-04-17T06:01:34.132 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.126+0000 7f7297829740 1 -- 172.21.15.47:0/2856946177 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bf18263df0 msgr2=0x55bf1826a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:34.132 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.126+0000 7f7297829740 1 --2- 172.21.15.47:0/2856946177 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bf18263df0 0x55bf1826a100 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f727800be60 tx=0x7f727809ce00 comp rx=0 tx=0).stop 2024-04-17T06:01:34.132 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.126+0000 7f7297829740 1 -- 172.21.15.47:0/2856946177 shutdown_connections 2024-04-17T06:01:34.132 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.126+0000 7f7297829740 1 --2- 172.21.15.47:0/2856946177 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55bf1827cf20 0x55bf1827f310 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:34.132 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.126+0000 7f7297829740 1 --2- 172.21.15.47:0/2856946177 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf1826cbc0 0x55bf1826a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:34.132 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.126+0000 7f7297829740 1 --2- 172.21.15.47:0/2856946177 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bf18263df0 0x55bf1826a100 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:34.132 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.126+0000 7f7275ffb640 0 client.15426 ms_handle_reset on v2:172.21.15.98:6838/2841287917 2024-04-17T06:01:34.132 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.126+0000 7f7297829740 1 --2- 172.21.15.47:0/2856946177 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bf182634d0 0x55bf18269bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:34.132 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.126+0000 7f7297829740 1 -- 172.21.15.47:0/2856946177 >> 172.21.15.47:0/2856946177 conn(0x55bf1825ddf0 msgr2=0x55bf1825fb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:01:34.132 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.126+0000 7f7297829740 1 -- 172.21.15.47:0/2856946177 shutdown_connections 2024-04-17T06:01:34.133 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.126+0000 7f7297829740 1 -- 172.21.15.47:0/2856946177 wait complete. 2024-04-17T06:01:34.133 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.126+0000 7f7297829740 20 client.15426 trim_cache size 0 max 16384 2024-04-17T06:01:34.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:33 smithi098 ceph-mon[36516]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-04-17T06:01:34.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:33 smithi047 ceph-mon[32223]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-04-17T06:01:34.144 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.141+0000 7f7297829740 1 Processor -- start 2024-04-17T06:01:34.144 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.141+0000 7f7297829740 1 -- start start 2024-04-17T06:01:34.144 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.141+0000 7f7297829740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bf182634d0 0x55bf1826c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:34.144 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.141+0000 7f7297829740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bf18263df0 0x55bf182786b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:34.144 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.141+0000 7f7297829740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf1826cbc0 0x55bf18284c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:34.144 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.141+0000 7f7297829740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f72901ae170 con 0x55bf18263df0 2024-04-17T06:01:34.144 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.141+0000 7f7297829740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f72901aba30 con 0x55bf182634d0 2024-04-17T06:01:34.145 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.141+0000 7f7297829740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f72901a9880 con 0x55bf1826cbc0 2024-04-17T06:01:34.145 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.141+0000 7f7285d9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bf18263df0 0x55bf182786b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:34.145 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.141+0000 7f7285d9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bf18263df0 0x55bf182786b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51988/0 (socket says 172.21.15.47:51988) 2024-04-17T06:01:34.145 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.141+0000 7f7285d9b640 1 -- 172.21.15.47:0/352272515 learned_addr learned my addr 172.21.15.47:0/352272515 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:01:34.145 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.141+0000 7f728ed9d640 1 --2- 172.21.15.47:0/352272515 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf1826cbc0 0x55bf18284c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:34.145 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.141+0000 7f728e59c640 1 --2- 172.21.15.47:0/352272515 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bf182634d0 0x55bf1826c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:34.145 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.142+0000 7f7285d9b640 1 -- 172.21.15.47:0/352272515 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf1826cbc0 msgr2=0x55bf18284c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:01:34.145 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.142+0000 7f7285d9b640 1 --2- 172.21.15.47:0/352272515 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf1826cbc0 0x55bf18284c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:34.145 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.142+0000 7f7285d9b640 1 -- 172.21.15.47:0/352272515 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bf182634d0 msgr2=0x55bf1826c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:01:34.145 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.142+0000 7f7285d9b640 1 --2- 172.21.15.47:0/352272515 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bf182634d0 0x55bf1826c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:34.145 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.142+0000 7f7285d9b640 1 -- 172.21.15.47:0/352272515 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf1827f970 con 0x55bf18263df0 2024-04-17T06:01:34.145 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.142+0000 7f7285d9b640 1 --2- 172.21.15.47:0/352272515 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bf18263df0 0x55bf182786b0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f727800b470 tx=0x7f7278097e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc271778a453264a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:01:34.145 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.142+0000 7f7277fff640 1 -- 172.21.15.47:0/352272515 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f727801e070 con 0x55bf18263df0 2024-04-17T06:01:34.146 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.142+0000 7f7277fff640 1 -- 172.21.15.47:0/352272515 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72780cd290 con 0x55bf18263df0 2024-04-17T06:01:34.146 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.142+0000 7f7277fff640 1 -- 172.21.15.47:0/352272515 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72780cd520 con 0x55bf18263df0 2024-04-17T06:01:34.146 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.143+0000 7f7297829740 1 -- 172.21.15.47:0/352272515 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bf18263df0 msgr2=0x55bf182786b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:34.146 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.143+0000 7f7297829740 1 --2- 172.21.15.47:0/352272515 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bf18263df0 0x55bf182786b0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f727800b470 tx=0x7f7278097e20 comp rx=0 tx=0).stop 2024-04-17T06:01:34.146 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.143+0000 7f7297829740 1 -- 172.21.15.47:0/352272515 shutdown_connections 2024-04-17T06:01:34.146 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.143+0000 7f7297829740 1 --2- 172.21.15.47:0/352272515 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf1826cbc0 0x55bf18284c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:34.146 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.143+0000 7f7297829740 1 --2- 172.21.15.47:0/352272515 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bf18263df0 0x55bf182786b0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:34.146 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.143+0000 7f7297829740 1 --2- 172.21.15.47:0/352272515 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bf182634d0 0x55bf1826c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:34.146 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.143+0000 7f7297829740 1 -- 172.21.15.47:0/352272515 >> 172.21.15.47:0/352272515 conn(0x55bf1825ddf0 msgr2=0x55bf18258b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:01:34.146 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.143+0000 7f7297829740 1 -- 172.21.15.47:0/352272515 shutdown_connections 2024-04-17T06:01:34.146 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.143+0000 7f7297829740 1 -- 172.21.15.47:0/352272515 wait complete. 2024-04-17T06:01:34.146 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.143+0000 7f7297829740 1 -- start start 2024-04-17T06:01:34.146 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.143+0000 7f7297829740 1 Processor -- start 2024-04-17T06:01:34.146 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.143+0000 7f7297829740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bf182634d0 0x55bf18274f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:34.147 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.143+0000 7f7297829740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bf18263df0 0x55bf18271b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:34.147 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.143+0000 7f7297829740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf1826cbc0 0x55bf182720d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:34.147 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.143+0000 7f7297829740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f72901a9ef0 con 0x55bf18263df0 2024-04-17T06:01:34.147 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.143+0000 7f7297829740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f72901ab360 con 0x55bf182634d0 2024-04-17T06:01:34.147 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.143+0000 7f7297829740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f72901aa5c0 con 0x55bf1826cbc0 2024-04-17T06:01:34.147 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.143+0000 7f7297829740 10 client.0 _ll_register_callbacks cb 0x7f7278097ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:01:34.147 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.143+0000 7f7285d9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bf18263df0 0x55bf18271b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:34.147 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.143+0000 7f7285d9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bf18263df0 0x55bf18271b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51998/0 (socket says 172.21.15.47:51998) 2024-04-17T06:01:34.147 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.143+0000 7f7285d9b640 1 -- 172.21.15.47:0/1517869061 learned_addr learned my addr 172.21.15.47:0/1517869061 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:01:34.147 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.143+0000 7f728ed9d640 1 --2- 172.21.15.47:0/1517869061 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf1826cbc0 0x55bf182720d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:34.147 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.143+0000 7f728e59c640 1 --2- 172.21.15.47:0/1517869061 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bf182634d0 0x55bf18274f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:34.147 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.144+0000 7f7285d9b640 1 -- 172.21.15.47:0/1517869061 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf1826cbc0 msgr2=0x55bf182720d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:01:34.147 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.144+0000 7f7285d9b640 1 --2- 172.21.15.47:0/1517869061 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf1826cbc0 0x55bf182720d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:34.147 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.144+0000 7f7285d9b640 1 -- 172.21.15.47:0/1517869061 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bf182634d0 msgr2=0x55bf18274f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:01:34.148 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.144+0000 7f7285d9b640 1 --2- 172.21.15.47:0/1517869061 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bf182634d0 0x55bf18274f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:34.148 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.144+0000 7f7285d9b640 1 -- 172.21.15.47:0/1517869061 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf1826efb0 con 0x55bf18263df0 2024-04-17T06:01:34.148 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.144+0000 7f728ed9d640 1 --2- 172.21.15.47:0/1517869061 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf1826cbc0 0x55bf182720d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:01:34.148 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.144+0000 7f7285d9b640 1 --2- 172.21.15.47:0/1517869061 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bf18263df0 0x55bf18271b90 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f7278098550 tx=0x7f727809ba80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbc6efe79447dede server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:01:34.148 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.144+0000 7f7275ffb640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-17T06:01:34.148 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.144+0000 7f7275ffb640 1 -- 172.21.15.47:0/1517869061 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f727801e040 con 0x55bf18263df0 2024-04-17T06:01:34.148 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.144+0000 7f7275ffb640 1 -- 172.21.15.47:0/1517869061 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f7278010810 con 0x55bf18263df0 2024-04-17T06:01:34.148 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.144+0000 7f7297829740 1 -- 172.21.15.47:0/1517869061 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72780cd290 con 0x55bf18263df0 2024-04-17T06:01:34.148 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.144+0000 7f7275ffb640 1 -- 172.21.15.47:0/1517869061 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f72780cd730 con 0x55bf18263df0 2024-04-17T06:01:34.148 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.144+0000 7f7297829740 10 client.15432 fetch_fsmap learned FSMap version 22 2024-04-17T06:01:34.148 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.144+0000 7f7297829740 1 -- 172.21.15.47:0/1517869061 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55bf1826efb0 con 0x55bf18263df0 2024-04-17T06:01:34.148 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.144+0000 7f7275ffb640 1 -- 172.21.15.47:0/1517869061 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72780cd290 con 0x55bf18263df0 2024-04-17T06:01:34.148 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.144+0000 7f7275ffb640 1 -- 172.21.15.47:0/1517869061 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 22) ==== 8339+0+0 (secure 0 0 0) 0x7f727809ab60 con 0x55bf18263df0 2024-04-17T06:01:34.148 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.145+0000 7f7297829740 10 client.15432 fetch_fsmap finished waiting for FSMap version 22 2024-04-17T06:01:34.148 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.145+0000 7f7297829740 10 client.15432 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-17T06:01:34.149 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.145+0000 7f7297829740 20 client.15432 populate_metadata read hostname 'smithi047' 2024-04-17T06:01:34.149 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.145+0000 7f7297829740 1 --2- 172.21.15.47:0/1517869061 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55bf182e4630 0x55bf182e6a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:34.149 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.145+0000 7f7297829740 4 client.15432 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-17T06:01:34.149 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.145+0000 7f7297829740 1 -- 172.21.15.47:0/1517869061 --> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bf181cc060 con 0x55bf182e4630 2024-04-17T06:01:34.149 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.145+0000 7f728e59c640 1 --2- 172.21.15.47:0/1517869061 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55bf182e4630 0x55bf182e6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:34.149 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.145+0000 7f7275ffb640 10 client.15432 ms_handle_connect on v2:172.21.15.98:6838/2841287917 2024-04-17T06:01:34.149 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.145+0000 7f728e59c640 1 --2- 172.21.15.47:0/1517869061 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55bf182e4630 0x55bf182e6a20 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d84e83beb432b75 server_cookie=92abe5b45c3e25ab in_seq=0 out_seq=0 2024-04-17T06:01:34.156 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.156+0000 7f7275ffb640 1 -- 172.21.15.47:0/1517869061 <== mds.0 v2:172.21.15.98:6838/2841287917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bf181cc060 con 0x55bf182e4630 2024-04-17T06:01:34.156 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.156+0000 7f7275ffb640 10 client.15432 handle_command_reply: tid=0 2024-04-17T06:01:34.156 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.156+0000 7f7297829740 1 client.15432 shutdown 2024-04-17T06:01:34.156 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.157+0000 7f7297829740 1 -- 172.21.15.47:0/1517869061 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bf18263df0 msgr2=0x55bf18271b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:34.156 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.157+0000 7f7297829740 1 --2- 172.21.15.47:0/1517869061 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bf18263df0 0x55bf18271b90 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f7278098550 tx=0x7f727809ba80 comp rx=0 tx=0).stop 2024-04-17T06:01:34.156 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.157+0000 7f7297829740 1 -- 172.21.15.47:0/1517869061 shutdown_connections 2024-04-17T06:01:34.156 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.157+0000 7f7297829740 1 --2- 172.21.15.47:0/1517869061 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55bf182e4630 0x55bf182e6a20 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:34.156 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.157+0000 7f7297829740 1 --2- 172.21.15.47:0/1517869061 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf1826cbc0 0x55bf182720d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:34.156 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.157+0000 7f7297829740 1 --2- 172.21.15.47:0/1517869061 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bf18263df0 0x55bf18271b90 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:34.157 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.157+0000 7f7275ffb640 0 client.15432 ms_handle_reset on v2:172.21.15.98:6838/2841287917 2024-04-17T06:01:34.157 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.157+0000 7f7297829740 1 --2- 172.21.15.47:0/1517869061 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bf182634d0 0x55bf18274f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:34.157 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.157+0000 7f7297829740 1 -- 172.21.15.47:0/1517869061 >> 172.21.15.47:0/1517869061 conn(0x55bf1825ddf0 msgr2=0x55bf18269b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:01:34.157 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.157+0000 7f7297829740 1 -- 172.21.15.47:0/1517869061 shutdown_connections 2024-04-17T06:01:34.157 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.157+0000 7f7297829740 1 -- 172.21.15.47:0/1517869061 wait complete. 2024-04-17T06:01:34.157 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.157+0000 7f7297829740 20 client.15432 trim_cache size 0 max 16384 2024-04-17T06:01:34.157 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-04-17T06:01:34.157 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-04-17T06:01:34.157 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-04-17T06:01:34.157 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-04-17T06:01:34.158 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.157+0000 7f7294b59640 1 -- 172.21.15.47:0/354714572 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f727c0a3ee0 msgr2=0x7f727c0a63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:34.158 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.157+0000 7f7294b59640 1 --2- 172.21.15.47:0/354714572 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f727c0a3ee0 0x7f727c0a63a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f728804c000 tx=0x7f7288045000 comp rx=0 tx=0).stop 2024-04-17T06:01:34.158 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.157+0000 7f7294b59640 1 -- 172.21.15.47:0/354714572 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7290101b70 msgr2=0x7f7290198c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:34.159 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.157+0000 7f7294b59640 1 --2- 172.21.15.47:0/354714572 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7290101b70 0x7f7290198c90 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f727800b4a0 tx=0x7f727800b960 comp rx=0 tx=0).stop 2024-04-17T06:01:34.159 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.157+0000 7f7294b59640 1 -- 172.21.15.47:0/354714572 shutdown_connections 2024-04-17T06:01:34.159 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.157+0000 7f7294b59640 1 --2- 172.21.15.47:0/354714572 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7290106fc0 0x7f729019d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:34.159 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.157+0000 7f7294b59640 1 --2- 172.21.15.47:0/354714572 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f727c0a3ee0 0x7f727c0a63a0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:34.159 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.157+0000 7f7294b59640 1 --2- 172.21.15.47:0/354714572 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7290101b70 0x7f7290198c90 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:34.159 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.157+0000 7f7294b59640 1 --2- 172.21.15.47:0/354714572 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f72901011b0 0x7f7290198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:34.159 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.157+0000 7f7294b59640 1 -- 172.21.15.47:0/354714572 >> 172.21.15.47:0/354714572 conn(0x7f7290076c60 msgr2=0x7f72901003e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:01:34.159 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.158+0000 7f7294b59640 1 -- 172.21.15.47:0/354714572 shutdown_connections 2024-04-17T06:01:34.159 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:34.158+0000 7f7294b59640 1 -- 172.21.15.47:0/354714572 wait complete. 2024-04-17T06:01:36.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:35 smithi103 ceph-mon[36243]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 78 KiB/s wr, 48 op/s 2024-04-17T06:01:36.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:35 smithi098 ceph-mon[36516]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 78 KiB/s wr, 48 op/s 2024-04-17T06:01:36.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:35 smithi047 ceph-mon[32223]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 78 KiB/s wr, 48 op/s 2024-04-17T06:01:38.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:37 smithi103 ceph-mon[36243]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 79 KiB/s wr, 54 op/s 2024-04-17T06:01:38.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:37 smithi098 ceph-mon[36516]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 79 KiB/s wr, 54 op/s 2024-04-17T06:01:38.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:37 smithi047 ceph-mon[32223]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 79 KiB/s wr, 54 op/s 2024-04-17T06:01:40.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:39 smithi103 ceph-mon[36243]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-04-17T06:01:40.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:39 smithi098 ceph-mon[36516]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-04-17T06:01:40.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:39 smithi047 ceph-mon[32223]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-04-17T06:01:42.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:41 smithi103 ceph-mon[36243]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2024-04-17T06:01:42.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:41 smithi098 ceph-mon[36516]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2024-04-17T06:01:42.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:41 smithi047 ceph-mon[32223]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2024-04-17T06:01:44.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:43 smithi103 ceph-mon[36243]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-04-17T06:01:44.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:43 smithi098 ceph-mon[36516]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-04-17T06:01:44.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:43 smithi047 ceph-mon[32223]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-04-17T06:01:46.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:45 smithi103 ceph-mon[36243]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-04-17T06:01:46.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:45 smithi098 ceph-mon[36516]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-04-17T06:01:46.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:45 smithi047 ceph-mon[32223]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-04-17T06:01:47.256 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.256+0000 7f369f5f2640 1 -- 172.21.15.47:0/492127959 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36981072b0 msgr2=0x7f369810ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:47.256 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.256+0000 7f369f5f2640 1 --2- 172.21.15.47:0/492127959 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36981072b0 0x7f369810ee80 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f3688009ca0 tx=0x7f368802c0f0 comp rx=0 tx=0).stop 2024-04-17T06:01:47.256 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.256+0000 7f369f5f2640 1 -- 172.21.15.47:0/492127959 shutdown_connections 2024-04-17T06:01:47.257 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.256+0000 7f369f5f2640 1 --2- 172.21.15.47:0/492127959 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36981072b0 0x7f369810ee80 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:47.257 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.256+0000 7f369f5f2640 1 --2- 172.21.15.47:0/492127959 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3698102b90 0x7f3698106bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:47.257 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.256+0000 7f369f5f2640 1 --2- 172.21.15.47:0/492127959 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f36981021e0 0x7f36981025c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:47.257 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.256+0000 7f369f5f2640 1 -- 172.21.15.47:0/492127959 >> 172.21.15.47:0/492127959 conn(0x7f36980793f0 msgr2=0x7f3698079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:01:47.257 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.256+0000 7f369f5f2640 1 -- 172.21.15.47:0/492127959 shutdown_connections 2024-04-17T06:01:47.257 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.256+0000 7f369f5f2640 1 -- 172.21.15.47:0/492127959 wait complete. 2024-04-17T06:01:47.257 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.256+0000 7f369f5f2640 1 Processor -- start 2024-04-17T06:01:47.257 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.257+0000 7f369f5f2640 1 -- start start 2024-04-17T06:01:47.257 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.257+0000 7f369f5f2640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36981021e0 0x7f3698198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:47.257 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.257+0000 7f369f5f2640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f36981072b0 0x7f3698198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:47.257 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.257+0000 7f369f5f2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f369819d010 0x7f369819d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:47.257 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.257+0000 7f369f5f2640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f3698112350 con 0x7f36981021e0 2024-04-17T06:01:47.257 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.257+0000 7f369f5f2640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f36981121d0 con 0x7f36981072b0 2024-04-17T06:01:47.257 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.257+0000 7f369f5f2640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f36981124d0 con 0x7f369819d010 2024-04-17T06:01:47.258 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.257+0000 7f369d38e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36981021e0 0x7f3698198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:47.258 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.257+0000 7f369d38e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36981021e0 0x7f3698198750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55616/0 (socket says 172.21.15.47:55616) 2024-04-17T06:01:47.258 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.257+0000 7f369d38e640 1 -- 172.21.15.47:0/3243062082 learned_addr learned my addr 172.21.15.47:0/3243062082 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:01:47.258 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.257+0000 7f369cb8d640 1 --2- 172.21.15.47:0/3243062082 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f36981072b0 0x7f3698198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:47.258 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.257+0000 7f369db8f640 1 --2- 172.21.15.47:0/3243062082 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f369819d010 0x7f369819d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:47.258 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.257+0000 7f369d38e640 1 -- 172.21.15.47:0/3243062082 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f369819d010 msgr2=0x7f369819d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:47.258 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.257+0000 7f369d38e640 1 --2- 172.21.15.47:0/3243062082 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f369819d010 0x7f369819d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:47.258 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.257+0000 7f369d38e640 1 -- 172.21.15.47:0/3243062082 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f36981072b0 msgr2=0x7f3698198c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:47.258 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.257+0000 7f369d38e640 1 --2- 172.21.15.47:0/3243062082 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f36981072b0 0x7f3698198c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:47.258 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.257+0000 7f369d38e640 1 -- 172.21.15.47:0/3243062082 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f369819db00 con 0x7f36981021e0 2024-04-17T06:01:47.258 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.257+0000 7f369d38e640 1 --2- 172.21.15.47:0/3243062082 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36981021e0 0x7f3698198750 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f369000c890 tx=0x7f369000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:01:47.258 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.257+0000 7f36967fc640 1 -- 172.21.15.47:0/3243062082 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3690004220 con 0x7f36981021e0 2024-04-17T06:01:47.258 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.257+0000 7f36967fc640 1 -- 172.21.15.47:0/3243062082 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36900043c0 con 0x7f36981021e0 2024-04-17T06:01:47.258 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.257+0000 7f369f5f2640 1 -- 172.21.15.47:0/3243062082 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f369819ddf0 con 0x7f36981021e0 2024-04-17T06:01:47.259 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.257+0000 7f36967fc640 1 -- 172.21.15.47:0/3243062082 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36900049e0 con 0x7f36981021e0 2024-04-17T06:01:47.259 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.257+0000 7f369f5f2640 1 -- 172.21.15.47:0/3243062082 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36981a55e0 con 0x7f36981021e0 2024-04-17T06:01:47.265 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.260+0000 7f36967fc640 1 -- 172.21.15.47:0/3243062082 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f36900079b0 con 0x7f36981021e0 2024-04-17T06:01:47.265 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.260+0000 7f36a21d7740 1 Processor -- start 2024-04-17T06:01:47.265 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.260+0000 7f36a21d7740 1 -- start start 2024-04-17T06:01:47.265 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.260+0000 7f36a21d7740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559c24d3ebe0 0x559c24d3efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:47.266 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.260+0000 7f36a21d7740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c24d3f500 0x559c24d4c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:47.266 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.260+0000 7f36a21d7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559c24d4cbc0 0x559c24d4efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:47.266 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.260+0000 7f36a21d7740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x559c24ca47d0 con 0x559c24d3ebe0 2024-04-17T06:01:47.266 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.260+0000 7f36a21d7740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x559c24c41f70 con 0x559c24d3f500 2024-04-17T06:01:47.266 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.260+0000 7f36a21d7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x559c24ca0790 con 0x559c24d4cbc0 2024-04-17T06:01:47.266 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.260+0000 7f36967fc640 1 --2- 172.21.15.47:0/3243062082 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f36800a3ee0 0x7f36800a63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:47.266 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.261+0000 7f369db8f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559c24d3ebe0 0x559c24d3efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:47.266 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.261+0000 7f369cb8d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559c24d4cbc0 0x559c24d4efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:47.266 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.261+0000 7f369db8f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559c24d3ebe0 0x559c24d3efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55628/0 (socket says 172.21.15.47:55628) 2024-04-17T06:01:47.266 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.261+0000 7f369db8f640 1 -- 172.21.15.47:0/1028056374 learned_addr learned my addr 172.21.15.47:0/1028056374 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:01:47.266 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.261+0000 7f369d38e640 1 --2- 172.21.15.47:0/1028056374 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c24d3f500 0x559c24d4c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:47.266 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.261+0000 7f369db8f640 1 --2- 172.21.15.47:0/3243062082 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f36800a3ee0 0x7f36800a63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:47.266 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.261+0000 7f36967fc640 1 -- 172.21.15.47:0/3243062082 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f36900d0bd0 con 0x7f36981021e0 2024-04-17T06:01:47.267 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.261+0000 7f369db8f640 1 -- 172.21.15.47:0/1028056374 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559c24d4cbc0 msgr2=0x559c24d4efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:01:47.267 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.261+0000 7f369db8f640 1 --2- 172.21.15.47:0/1028056374 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559c24d4cbc0 0x559c24d4efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:47.267 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.261+0000 7f369db8f640 1 -- 172.21.15.47:0/1028056374 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c24d3f500 msgr2=0x559c24d4c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:01:47.267 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.261+0000 7f369db8f640 1 --2- 172.21.15.47:0/1028056374 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c24d3f500 0x559c24d4c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:47.267 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.261+0000 7f369db8f640 1 -- 172.21.15.47:0/1028056374 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c249b7210 con 0x559c24d3ebe0 2024-04-17T06:01:47.267 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.261+0000 7f369cb8d640 1 --2- 172.21.15.47:0/1028056374 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559c24d4cbc0 0x559c24d4efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T06:01:47.267 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.261+0000 7f369db8f640 1 --2- 172.21.15.47:0/1028056374 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559c24d3ebe0 0x559c24d3efc0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f3688036000 tx=0x7f3688032690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d3e4eec9666d9a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:01:47.267 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.261+0000 7f369db8f640 1 --2- 172.21.15.47:0/3243062082 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f36800a3ee0 0x7f36800a63a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3688033000 tx=0x7f3688041000 comp rx=0 tx=0).ready entity=mgr.24923 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:01:47.267 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.261+0000 7f367f7fe640 1 -- 172.21.15.47:0/1028056374 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3688051070 con 0x559c24d3ebe0 2024-04-17T06:01:47.267 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.261+0000 7f367f7fe640 1 -- 172.21.15.47:0/1028056374 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3688048070 con 0x559c24d3ebe0 2024-04-17T06:01:47.267 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.261+0000 7f367f7fe640 1 -- 172.21.15.47:0/1028056374 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3688039040 con 0x559c24d3ebe0 2024-04-17T06:01:47.267 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.262+0000 7f36a21d7740 1 -- 172.21.15.47:0/1028056374 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559c24d3ebe0 msgr2=0x559c24d3efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:47.267 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.262+0000 7f36a21d7740 1 --2- 172.21.15.47:0/1028056374 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559c24d3ebe0 0x559c24d3efc0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f3688036000 tx=0x7f3688032690 comp rx=0 tx=0).stop 2024-04-17T06:01:47.268 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.262+0000 7f36a21d7740 1 -- 172.21.15.47:0/1028056374 shutdown_connections 2024-04-17T06:01:47.268 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.262+0000 7f36a21d7740 1 --2- 172.21.15.47:0/1028056374 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559c24d4cbc0 0x559c24d4efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:47.268 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.262+0000 7f36a21d7740 1 --2- 172.21.15.47:0/1028056374 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c24d3f500 0x559c24d4c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:47.268 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.262+0000 7f36a21d7740 1 --2- 172.21.15.47:0/1028056374 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559c24d3ebe0 0x559c24d3efc0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:47.268 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.262+0000 7f36a21d7740 1 -- 172.21.15.47:0/1028056374 >> 172.21.15.47:0/1028056374 conn(0x559c24d399e0 msgr2=0x559c24d39dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:01:47.268 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.262+0000 7f36a21d7740 1 -- 172.21.15.47:0/1028056374 shutdown_connections 2024-04-17T06:01:47.268 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.262+0000 7f36a21d7740 1 -- 172.21.15.47:0/1028056374 wait complete. 2024-04-17T06:01:47.268 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.262+0000 7f36a21d7740 1 -- start start 2024-04-17T06:01:47.268 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.263+0000 7f36a21d7740 1 Processor -- start 2024-04-17T06:01:47.268 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.263+0000 7f36a21d7740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559c24d3ebe0 0x559c24d49c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:47.268 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.263+0000 7f36a21d7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559c24d3f500 0x559c24d4a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:47.268 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.263+0000 7f36a21d7740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c24d4cbc0 0x559c24d4a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:47.268 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.263+0000 7f36a21d7740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x559c24d4f720 con 0x559c24d3ebe0 2024-04-17T06:01:47.268 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.263+0000 7f36a21d7740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x559c24c2ce90 con 0x559c24d4cbc0 2024-04-17T06:01:47.268 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.263+0000 7f36a21d7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x559c24d4f8a0 con 0x559c24d3f500 2024-04-17T06:01:47.269 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.263+0000 7f36a21d7740 10 client.0 _ll_register_callbacks cb 0x559c24c42d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:01:47.269 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.263+0000 7f369d38e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559c24d3ebe0 0x559c24d49c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:47.269 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.263+0000 7f369d38e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559c24d3ebe0 0x559c24d49c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55634/0 (socket says 172.21.15.47:55634) 2024-04-17T06:01:47.269 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.263+0000 7f369d38e640 1 -- 172.21.15.47:0/284440814 learned_addr learned my addr 172.21.15.47:0/284440814 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:01:47.269 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.263+0000 7f369cb8d640 1 --2- 172.21.15.47:0/284440814 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559c24d3f500 0x559c24d4a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:47.269 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.263+0000 7f369d38e640 1 -- 172.21.15.47:0/284440814 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559c24d3f500 msgr2=0x559c24d4a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:01:47.269 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.263+0000 7f369d38e640 1 --2- 172.21.15.47:0/284440814 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559c24d3f500 0x559c24d4a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:47.269 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.263+0000 7f369d38e640 1 -- 172.21.15.47:0/284440814 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c24d4cbc0 msgr2=0x559c24d4a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:01:47.269 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.263+0000 7f369d38e640 1 --2- 172.21.15.47:0/284440814 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c24d4cbc0 0x559c24d4a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:47.269 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.263+0000 7f369d38e640 1 -- 172.21.15.47:0/284440814 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c24a83620 con 0x559c24d3ebe0 2024-04-17T06:01:47.269 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.263+0000 7f369cb8d640 1 --2- 172.21.15.47:0/284440814 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559c24d3f500 0x559c24d4a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:01:47.269 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.264+0000 7f369d38e640 1 --2- 172.21.15.47:0/284440814 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559c24d3ebe0 0x559c24d49c40 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f369009c8e0 tx=0x7f36900148a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9736647f0b28705c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:01:47.269 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.264+0000 7f367cff9640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-17T06:01:47.269 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.264+0000 7f367cff9640 1 -- 172.21.15.47:0/284440814 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f369001d040 con 0x559c24d3ebe0 2024-04-17T06:01:47.270 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.264+0000 7f367cff9640 1 -- 172.21.15.47:0/284440814 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f36900a5130 con 0x559c24d3ebe0 2024-04-17T06:01:47.270 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.264+0000 7f36a21d7740 1 -- 172.21.15.47:0/284440814 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3688048070 con 0x559c24d3ebe0 2024-04-17T06:01:47.270 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.264+0000 7f367cff9640 1 -- 172.21.15.47:0/284440814 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f369009d050 con 0x559c24d3ebe0 2024-04-17T06:01:47.270 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.264+0000 7f36a21d7740 10 client.15444 fetch_fsmap learned FSMap version 22 2024-04-17T06:01:47.270 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.264+0000 7f36a21d7740 1 -- 172.21.15.47:0/284440814 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x559c24d5bb00 con 0x559c24d3ebe0 2024-04-17T06:01:47.270 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.264+0000 7f367cff9640 1 -- 172.21.15.47:0/284440814 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3688048070 con 0x559c24d3ebe0 2024-04-17T06:01:47.270 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.264+0000 7f367cff9640 1 -- 172.21.15.47:0/284440814 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 22) ==== 8339+0+0 (secure 0 0 0) 0x7f36900870b0 con 0x559c24d3ebe0 2024-04-17T06:01:47.270 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.264+0000 7f36a21d7740 10 client.15444 fetch_fsmap finished waiting for FSMap version 22 2024-04-17T06:01:47.270 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.264+0000 7f36a21d7740 10 client.15444 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-17T06:01:47.270 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.264+0000 7f36a21d7740 20 client.15444 populate_metadata read hostname 'smithi047' 2024-04-17T06:01:47.271 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.264+0000 7f36a21d7740 1 --2- 172.21.15.47:0/284440814 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x559c24d5cfc0 0x559c24d5f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:47.271 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.264+0000 7f36a21d7740 4 client.15444 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T06:01:47.271 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.264+0000 7f36a21d7740 1 -- 172.21.15.47:0/284440814 --> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559c24cb21f0 con 0x559c24d5cfc0 2024-04-17T06:01:47.271 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.265+0000 7f369cb8d640 1 --2- 172.21.15.47:0/284440814 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x559c24d5cfc0 0x559c24d5f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:47.271 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.265+0000 7f367cff9640 10 client.15444 ms_handle_connect on v2:172.21.15.98:6838/2841287917 2024-04-17T06:01:47.271 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.265+0000 7f369cb8d640 1 --2- 172.21.15.47:0/284440814 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x559c24d5cfc0 0x559c24d5f3b0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=575f330778a2f6e3 server_cookie=b768bd0f9d085e4d in_seq=0 out_seq=0 2024-04-17T06:01:47.271 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.266+0000 7f367cff9640 1 -- 172.21.15.47:0/284440814 <== mds.0 v2:172.21.15.98:6838/2841287917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559c24cb21f0 con 0x559c24d5cfc0 2024-04-17T06:01:47.271 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.266+0000 7f367cff9640 10 client.15444 handle_command_reply: tid=0 2024-04-17T06:01:47.271 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.266+0000 7f36a21d7740 1 client.15444 shutdown 2024-04-17T06:01:47.271 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.266+0000 7f36a21d7740 1 -- 172.21.15.47:0/284440814 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559c24d3ebe0 msgr2=0x559c24d49c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:47.271 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.266+0000 7f36a21d7740 1 --2- 172.21.15.47:0/284440814 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559c24d3ebe0 0x559c24d49c40 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f369009c8e0 tx=0x7f36900148a0 comp rx=0 tx=0).stop 2024-04-17T06:01:47.271 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.266+0000 7f36a21d7740 1 -- 172.21.15.47:0/284440814 shutdown_connections 2024-04-17T06:01:47.271 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.266+0000 7f36a21d7740 1 --2- 172.21.15.47:0/284440814 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x559c24d5cfc0 0x559c24d5f3b0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:47.271 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.266+0000 7f36a21d7740 1 --2- 172.21.15.47:0/284440814 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c24d4cbc0 0x559c24d4a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:47.272 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.266+0000 7f36a21d7740 1 --2- 172.21.15.47:0/284440814 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559c24d3f500 0x559c24d4a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:47.272 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.266+0000 7f367cff9640 0 client.15444 ms_handle_reset on v2:172.21.15.98:6838/2841287917 2024-04-17T06:01:47.272 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.266+0000 7f36a21d7740 1 --2- 172.21.15.47:0/284440814 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559c24d3ebe0 0x559c24d49c40 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:47.272 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.266+0000 7f36a21d7740 1 -- 172.21.15.47:0/284440814 >> 172.21.15.47:0/284440814 conn(0x559c24d399e0 msgr2=0x559c24d3b730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:01:47.272 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.266+0000 7f36a21d7740 1 -- 172.21.15.47:0/284440814 shutdown_connections 2024-04-17T06:01:47.272 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.266+0000 7f36a21d7740 1 -- 172.21.15.47:0/284440814 wait complete. 2024-04-17T06:01:47.272 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.266+0000 7f36a21d7740 20 client.15444 trim_cache size 0 max 16384 2024-04-17T06:01:47.284 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.281+0000 7f36a21d7740 1 Processor -- start 2024-04-17T06:01:47.284 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.281+0000 7f36a21d7740 1 -- start start 2024-04-17T06:01:47.284 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.281+0000 7f36a21d7740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559c24d3ebe0 0x559c24d4c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:47.284 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.281+0000 7f36a21d7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559c24d3f500 0x559c24d58620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:47.284 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.281+0000 7f36a21d7740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c24d4cbc0 0x559c24d3b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:47.284 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.281+0000 7f36a21d7740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f36981ae1b0 con 0x559c24d3ebe0 2024-04-17T06:01:47.285 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.281+0000 7f36a21d7740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f36981aba70 con 0x559c24d4cbc0 2024-04-17T06:01:47.285 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.281+0000 7f36a21d7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f36981a98c0 con 0x559c24d3f500 2024-04-17T06:01:47.285 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.281+0000 7f369d38e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559c24d3ebe0 0x559c24d4c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:47.285 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.281+0000 7f369d38e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559c24d3ebe0 0x559c24d4c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55640/0 (socket says 172.21.15.47:55640) 2024-04-17T06:01:47.285 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.281+0000 7f369d38e640 1 -- 172.21.15.47:0/182575474 learned_addr learned my addr 172.21.15.47:0/182575474 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:01:47.285 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.282+0000 7f369cb8d640 1 --2- 172.21.15.47:0/182575474 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559c24d3f500 0x559c24d58620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:47.285 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.282+0000 7f369d38e640 1 -- 172.21.15.47:0/182575474 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559c24d3f500 msgr2=0x559c24d58620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:01:47.285 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.282+0000 7f369d38e640 1 --2- 172.21.15.47:0/182575474 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559c24d3f500 0x559c24d58620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:47.285 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.282+0000 7f369db8f640 1 --2- 172.21.15.47:0/182575474 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c24d4cbc0 0x559c24d3b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:47.285 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.282+0000 7f369d38e640 1 -- 172.21.15.47:0/182575474 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c24d4cbc0 msgr2=0x559c24d3b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:01:47.285 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.282+0000 7f369d38e640 1 --2- 172.21.15.47:0/182575474 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c24d4cbc0 0x559c24d3b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:47.285 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.282+0000 7f369d38e640 1 -- 172.21.15.47:0/182575474 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c24d5fa10 con 0x559c24d3ebe0 2024-04-17T06:01:47.285 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.282+0000 7f369db8f640 1 --2- 172.21.15.47:0/182575474 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c24d4cbc0 0x559c24d3b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:01:47.285 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.282+0000 7f369d38e640 1 --2- 172.21.15.47:0/182575474 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559c24d3ebe0 0x559c24d4c370 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f3690098db0 tx=0x7f3690005300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b640a5ecc39b05a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:01:47.286 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.282+0000 7f367effd640 1 -- 172.21.15.47:0/182575474 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f369001d070 con 0x559c24d3ebe0 2024-04-17T06:01:47.286 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.282+0000 7f367effd640 1 -- 172.21.15.47:0/182575474 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3690091e50 con 0x559c24d3ebe0 2024-04-17T06:01:47.286 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.282+0000 7f367effd640 1 -- 172.21.15.47:0/182575474 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3690007ca0 con 0x559c24d3ebe0 2024-04-17T06:01:47.286 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.282+0000 7f36a21d7740 1 -- 172.21.15.47:0/182575474 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559c24d3ebe0 msgr2=0x559c24d4c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:47.286 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.282+0000 7f36a21d7740 1 --2- 172.21.15.47:0/182575474 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559c24d3ebe0 0x559c24d4c370 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f3690098db0 tx=0x7f3690005300 comp rx=0 tx=0).stop 2024-04-17T06:01:47.286 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.282+0000 7f36a21d7740 1 -- 172.21.15.47:0/182575474 shutdown_connections 2024-04-17T06:01:47.286 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.282+0000 7f36a21d7740 1 --2- 172.21.15.47:0/182575474 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c24d4cbc0 0x559c24d3b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:47.286 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.282+0000 7f36a21d7740 1 --2- 172.21.15.47:0/182575474 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559c24d3f500 0x559c24d58620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:47.286 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.283+0000 7f36a21d7740 1 --2- 172.21.15.47:0/182575474 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559c24d3ebe0 0x559c24d4c370 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:47.286 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.283+0000 7f36a21d7740 1 -- 172.21.15.47:0/182575474 >> 172.21.15.47:0/182575474 conn(0x559c24d399e0 msgr2=0x559c24d3ae30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:01:47.286 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.283+0000 7f36a21d7740 1 -- 172.21.15.47:0/182575474 shutdown_connections 2024-04-17T06:01:47.286 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.283+0000 7f36a21d7740 1 -- 172.21.15.47:0/182575474 wait complete. 2024-04-17T06:01:47.286 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.283+0000 7f36a21d7740 1 -- start start 2024-04-17T06:01:47.286 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.283+0000 7f36a21d7740 1 Processor -- start 2024-04-17T06:01:47.287 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.283+0000 7f36a21d7740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559c24d3ebe0 0x559c24d46bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:47.287 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.283+0000 7f36a21d7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559c24d3f500 0x559c24d47100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:47.287 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.283+0000 7f36a21d7740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c24d4cbc0 0x559c24d47660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:47.287 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.283+0000 7f36a21d7740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f36981a9f30 con 0x559c24d3ebe0 2024-04-17T06:01:47.287 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.283+0000 7f36a21d7740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f36981ab3a0 con 0x559c24d4cbc0 2024-04-17T06:01:47.287 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.283+0000 7f36a21d7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f36981aa600 con 0x559c24d3f500 2024-04-17T06:01:47.287 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.283+0000 7f36a21d7740 10 client.0 _ll_register_callbacks cb 0x7f36900058c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:01:47.287 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.283+0000 7f369d38e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559c24d3ebe0 0x559c24d46bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:47.287 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.283+0000 7f369d38e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559c24d3ebe0 0x559c24d46bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55644/0 (socket says 172.21.15.47:55644) 2024-04-17T06:01:47.287 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.283+0000 7f369d38e640 1 -- 172.21.15.47:0/741891760 learned_addr learned my addr 172.21.15.47:0/741891760 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:01:47.287 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.283+0000 7f369cb8d640 1 --2- 172.21.15.47:0/741891760 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559c24d3f500 0x559c24d47100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:47.287 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.284+0000 7f369db8f640 1 --2- 172.21.15.47:0/741891760 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c24d4cbc0 0x559c24d47660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:47.287 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.284+0000 7f369d38e640 1 -- 172.21.15.47:0/741891760 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559c24d3f500 msgr2=0x559c24d47100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:01:47.287 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.284+0000 7f369d38e640 1 --2- 172.21.15.47:0/741891760 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559c24d3f500 0x559c24d47100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:47.288 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.284+0000 7f369d38e640 1 -- 172.21.15.47:0/741891760 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c24d4cbc0 msgr2=0x559c24d47660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:01:47.288 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.284+0000 7f369d38e640 1 --2- 172.21.15.47:0/741891760 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c24d4cbc0 0x559c24d47660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:47.288 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.284+0000 7f369d38e640 1 -- 172.21.15.47:0/741891760 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c24d4efb0 con 0x559c24d3ebe0 2024-04-17T06:01:47.288 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.284+0000 7f369d38e640 1 --2- 172.21.15.47:0/741891760 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559c24d3ebe0 0x559c24d46bc0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f369009c650 tx=0x7f36900a54f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d75ec518c04431a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:01:47.288 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.284+0000 7f367cff9640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-17T06:01:47.288 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.284+0000 7f367cff9640 1 -- 172.21.15.47:0/741891760 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36900b8070 con 0x559c24d3ebe0 2024-04-17T06:01:47.288 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.284+0000 7f36a21d7740 1 -- 172.21.15.47:0/741891760 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3690091e50 con 0x559c24d3ebe0 2024-04-17T06:01:47.288 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.284+0000 7f367cff9640 1 -- 172.21.15.47:0/741891760 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f3690013cc0 con 0x559c24d3ebe0 2024-04-17T06:01:47.288 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.284+0000 7f367cff9640 1 -- 172.21.15.47:0/741891760 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f369001d040 con 0x559c24d3ebe0 2024-04-17T06:01:47.288 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.284+0000 7f36a21d7740 10 client.15450 fetch_fsmap learned FSMap version 22 2024-04-17T06:01:47.288 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.284+0000 7f36a21d7740 1 -- 172.21.15.47:0/741891760 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x559c24dc3890 con 0x559c24d3ebe0 2024-04-17T06:01:47.288 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.284+0000 7f367cff9640 1 -- 172.21.15.47:0/741891760 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3690091e50 con 0x559c24d3ebe0 2024-04-17T06:01:47.288 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.285+0000 7f367cff9640 1 -- 172.21.15.47:0/741891760 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 22) ==== 8339+0+0 (secure 0 0 0) 0x7f36900ce5b0 con 0x559c24d3ebe0 2024-04-17T06:01:47.288 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.285+0000 7f36a21d7740 10 client.15450 fetch_fsmap finished waiting for FSMap version 22 2024-04-17T06:01:47.288 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.285+0000 7f36a21d7740 10 client.15450 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-17T06:01:47.289 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.285+0000 7f36a21d7740 20 client.15450 populate_metadata read hostname 'smithi047' 2024-04-17T06:01:47.289 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.285+0000 7f36a21d7740 1 --2- 172.21.15.47:0/741891760 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x559c24dc3c70 0x559c24dc6060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:01:47.289 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.285+0000 7f36a21d7740 4 client.15450 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-17T06:01:47.289 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.285+0000 7f36a21d7740 1 -- 172.21.15.47:0/741891760 --> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559c24cac060 con 0x559c24dc3c70 2024-04-17T06:01:47.289 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.285+0000 7f369cb8d640 1 --2- 172.21.15.47:0/741891760 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x559c24dc3c70 0x559c24dc6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:01:47.289 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.285+0000 7f367cff9640 10 client.15450 ms_handle_connect on v2:172.21.15.98:6838/2841287917 2024-04-17T06:01:47.289 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.285+0000 7f369cb8d640 1 --2- 172.21.15.47:0/741891760 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x559c24dc3c70 0x559c24dc6060 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a330775df91a4fc1 server_cookie=1a579cc0728a2dd1 in_seq=0 out_seq=0 2024-04-17T06:01:47.290 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.289+0000 7f367cff9640 1 -- 172.21.15.47:0/741891760 <== mds.0 v2:172.21.15.98:6838/2841287917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559c24cac060 con 0x559c24dc3c70 2024-04-17T06:01:47.290 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.289+0000 7f367cff9640 10 client.15450 handle_command_reply: tid=0 2024-04-17T06:01:47.290 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.289+0000 7f36a21d7740 1 client.15450 shutdown 2024-04-17T06:01:47.290 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.289+0000 7f36a21d7740 1 -- 172.21.15.47:0/741891760 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559c24d3ebe0 msgr2=0x559c24d46bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:47.290 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.289+0000 7f36a21d7740 1 --2- 172.21.15.47:0/741891760 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559c24d3ebe0 0x559c24d46bc0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f369009c650 tx=0x7f36900a54f0 comp rx=0 tx=0).stop 2024-04-17T06:01:47.291 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.289+0000 7f36a21d7740 1 -- 172.21.15.47:0/741891760 shutdown_connections 2024-04-17T06:01:47.291 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.289+0000 7f36a21d7740 1 --2- 172.21.15.47:0/741891760 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x559c24dc3c70 0x559c24dc6060 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:47.291 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.289+0000 7f36a21d7740 1 --2- 172.21.15.47:0/741891760 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c24d4cbc0 0x559c24d47660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:47.291 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.289+0000 7f36a21d7740 1 --2- 172.21.15.47:0/741891760 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559c24d3f500 0x559c24d47100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:47.291 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.289+0000 7f367cff9640 0 client.15450 ms_handle_reset on v2:172.21.15.98:6838/2841287917 2024-04-17T06:01:47.291 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.289+0000 7f36a21d7740 1 --2- 172.21.15.47:0/741891760 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559c24d3ebe0 0x559c24d46bc0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:47.291 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.289+0000 7f36a21d7740 1 -- 172.21.15.47:0/741891760 >> 172.21.15.47:0/741891760 conn(0x559c24d399e0 msgr2=0x559c24d3b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:01:47.291 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.289+0000 7f36a21d7740 1 -- 172.21.15.47:0/741891760 shutdown_connections 2024-04-17T06:01:47.291 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.289+0000 7f36a21d7740 1 -- 172.21.15.47:0/741891760 wait complete. 2024-04-17T06:01:47.291 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.289+0000 7f36a21d7740 20 client.15450 trim_cache size 0 max 16384 2024-04-17T06:01:47.292 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-04-17T06:01:47.292 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-04-17T06:01:47.292 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-04-17T06:01:47.292 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-04-17T06:01:47.293 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.290+0000 7f369f5f2640 1 -- 172.21.15.47:0/3243062082 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f36800a3ee0 msgr2=0x7f36800a63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:47.293 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.290+0000 7f369f5f2640 1 --2- 172.21.15.47:0/3243062082 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f36800a3ee0 0x7f36800a63a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3688033000 tx=0x7f3688041000 comp rx=0 tx=0).stop 2024-04-17T06:01:47.293 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.290+0000 7f369f5f2640 1 -- 172.21.15.47:0/3243062082 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36981021e0 msgr2=0x7f3698198750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:01:47.293 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.290+0000 7f369f5f2640 1 --2- 172.21.15.47:0/3243062082 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36981021e0 0x7f3698198750 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f369000c890 tx=0x7f369000cd50 comp rx=0 tx=0).stop 2024-04-17T06:01:47.293 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.290+0000 7f369f5f2640 1 -- 172.21.15.47:0/3243062082 shutdown_connections 2024-04-17T06:01:47.293 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.290+0000 7f369f5f2640 1 --2- 172.21.15.47:0/3243062082 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f369819d010 0x7f369819d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:47.293 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.290+0000 7f369f5f2640 1 --2- 172.21.15.47:0/3243062082 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f36981072b0 0x7f3698198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:47.293 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.290+0000 7f369f5f2640 1 --2- 172.21.15.47:0/3243062082 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f36800a3ee0 0x7f36800a63a0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:47.293 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.290+0000 7f369f5f2640 1 --2- 172.21.15.47:0/3243062082 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36981021e0 0x7f3698198750 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:01:47.293 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.290+0000 7f369f5f2640 1 -- 172.21.15.47:0/3243062082 >> 172.21.15.47:0/3243062082 conn(0x7f36980793f0 msgr2=0x7f36980ffa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:01:47.293 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.290+0000 7f369f5f2640 1 -- 172.21.15.47:0/3243062082 shutdown_connections 2024-04-17T06:01:47.293 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:01:47.290+0000 7f369f5f2640 1 -- 172.21.15.47:0/3243062082 wait complete. 2024-04-17T06:01:48.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:47 smithi103 ceph-mon[36243]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-04-17T06:01:48.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:47 smithi098 ceph-mon[36516]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-04-17T06:01:48.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:47 smithi047 ceph-mon[32223]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-04-17T06:01:50.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:49 smithi103 ceph-mon[36243]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-04-17T06:01:50.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:49 smithi098 ceph-mon[36516]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-04-17T06:01:50.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:49 smithi047 ceph-mon[32223]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-04-17T06:01:52.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:51 smithi103 ceph-mon[36243]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-17T06:01:52.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:51 smithi098 ceph-mon[36516]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-17T06:01:52.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:51 smithi047 ceph-mon[32223]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-17T06:01:54.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:53 smithi103 ceph-mon[36243]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-17T06:01:54.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:53 smithi098 ceph-mon[36516]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-17T06:01:54.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:53 smithi047 ceph-mon[32223]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-17T06:01:56.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:55 smithi103 ceph-mon[36243]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-17T06:01:56.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:55 smithi098 ceph-mon[36516]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-17T06:01:56.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:55 smithi047 ceph-mon[32223]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-17T06:01:58.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:57 smithi103 ceph-mon[36243]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-17T06:01:58.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:57 smithi098 ceph-mon[36516]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-17T06:01:58.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:57 smithi047 ceph-mon[32223]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-17T06:02:00.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:01:59 smithi103 ceph-mon[36243]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-17T06:02:00.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:01:59 smithi098 ceph-mon[36516]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-17T06:02:00.134 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:01:59 smithi047 ceph-mon[32223]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-17T06:02:00.382 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.383+0000 7fa3863d4640 1 -- 172.21.15.47:0/2001269740 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa380102b70 msgr2=0x7fa380106bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:02:00.382 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.383+0000 7fa3863d4640 1 --2- 172.21.15.47:0/2001269740 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa380102b70 0x7fa380106bb0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fa374009900 tx=0x7fa37402bc30 comp rx=0 tx=0).stop 2024-04-17T06:02:00.382 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.383+0000 7fa3863d4640 1 -- 172.21.15.47:0/2001269740 shutdown_connections 2024-04-17T06:02:00.382 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.383+0000 7fa3863d4640 1 --2- 172.21.15.47:0/2001269740 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa380107290 0x7fa38010ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:00.382 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.383+0000 7fa3863d4640 1 --2- 172.21.15.47:0/2001269740 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa380102b70 0x7fa380106bb0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:00.382 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.383+0000 7fa3863d4640 1 --2- 172.21.15.47:0/2001269740 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa3801021c0 0x7fa3801025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:00.382 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.383+0000 7fa3863d4640 1 -- 172.21.15.47:0/2001269740 >> 172.21.15.47:0/2001269740 conn(0x7fa3800fda00 msgr2=0x7fa3800ffe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:02:00.382 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.383+0000 7fa3863d4640 1 -- 172.21.15.47:0/2001269740 shutdown_connections 2024-04-17T06:02:00.382 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.383+0000 7fa3863d4640 1 -- 172.21.15.47:0/2001269740 wait complete. 2024-04-17T06:02:00.383 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.384+0000 7fa3863d4640 1 Processor -- start 2024-04-17T06:02:00.383 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.384+0000 7fa3863d4640 1 -- start start 2024-04-17T06:02:00.384 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.384+0000 7fa3863d4640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa3801021c0 0x7fa38019e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:02:00.385 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.384+0000 7fa3863d4640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa380107290 0x7fa38019e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:02:00.385 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.384+0000 7fa3863d4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa380198600 0x7fa380198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:02:00.385 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.384+0000 7fa3863d4640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa380112300 con 0x7fa380107290 2024-04-17T06:02:00.385 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.384+0000 7fa3863d4640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa380112180 con 0x7fa3801021c0 2024-04-17T06:02:00.385 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.384+0000 7fa3863d4640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa380112480 con 0x7fa380198600 2024-04-17T06:02:00.385 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.384+0000 7fa37f7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa380107290 0x7fa38019e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:02:00.385 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.384+0000 7fa37f7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa380107290 0x7fa38019e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58282/0 (socket says 172.21.15.47:58282) 2024-04-17T06:02:00.385 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.385+0000 7fa37f7fe640 1 -- 172.21.15.47:0/816498693 learned_addr learned my addr 172.21.15.47:0/816498693 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:02:00.385 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.385+0000 7fa384971640 1 --2- 172.21.15.47:0/816498693 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa380198600 0x7fa380198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:02:00.385 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.385+0000 7fa37ffff640 1 --2- 172.21.15.47:0/816498693 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa3801021c0 0x7fa38019e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:02:00.385 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.385+0000 7fa37f7fe640 1 -- 172.21.15.47:0/816498693 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa380198600 msgr2=0x7fa380198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:02:00.385 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.385+0000 7fa37f7fe640 1 --2- 172.21.15.47:0/816498693 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa380198600 0x7fa380198ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:00.385 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.385+0000 7fa37f7fe640 1 -- 172.21.15.47:0/816498693 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa3801021c0 msgr2=0x7fa38019e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:02:00.386 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.385+0000 7fa37f7fe640 1 --2- 172.21.15.47:0/816498693 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa3801021c0 0x7fa38019e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:00.386 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.385+0000 7fa37f7fe640 1 -- 172.21.15.47:0/816498693 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa380199120 con 0x7fa380107290 2024-04-17T06:02:00.386 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.385+0000 7fa384971640 1 --2- 172.21.15.47:0/816498693 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa380198600 0x7fa380198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:02:00.386 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.385+0000 7fa37ffff640 1 --2- 172.21.15.47:0/816498693 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa3801021c0 0x7fa38019e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:02:00.386 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.385+0000 7fa37f7fe640 1 --2- 172.21.15.47:0/816498693 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa380107290 0x7fa38019e940 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fa374036040 tx=0x7fa37402ce20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:02:00.386 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.385+0000 7fa37d7fa640 1 -- 172.21.15.47:0/816498693 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa374041070 con 0x7fa380107290 2024-04-17T06:02:00.386 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.385+0000 7fa37d7fa640 1 -- 172.21.15.47:0/816498693 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3740054e0 con 0x7fa380107290 2024-04-17T06:02:00.386 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.385+0000 7fa37d7fa640 1 -- 172.21.15.47:0/816498693 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa374034440 con 0x7fa380107290 2024-04-17T06:02:00.386 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.385+0000 7fa3863d4640 1 -- 172.21.15.47:0/816498693 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3801993b0 con 0x7fa380107290 2024-04-17T06:02:00.386 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.385+0000 7fa3863d4640 1 -- 172.21.15.47:0/816498693 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3801a55b0 con 0x7fa380107290 2024-04-17T06:02:00.390 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.387+0000 7fa37d7fa640 1 -- 172.21.15.47:0/816498693 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7fa374044070 con 0x7fa380107290 2024-04-17T06:02:00.391 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.388+0000 7fa388fc4740 1 Processor -- start 2024-04-17T06:02:00.391 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.388+0000 7fa388fc4740 1 -- start start 2024-04-17T06:02:00.391 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.388+0000 7fa388fc4740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55778425cbe0 0x55778425cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:02:00.391 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.388+0000 7fa388fc4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55778425d500 0x55778426a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:02:00.391 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.388+0000 7fa388fc4740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55778426abc0 0x55778426cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:02:00.391 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.388+0000 7fa388fc4740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5577841c27d0 con 0x55778426abc0 2024-04-17T06:02:00.391 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.388+0000 7fa388fc4740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55778415ff70 con 0x55778425cbe0 2024-04-17T06:02:00.391 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.388+0000 7fa388fc4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5577841be790 con 0x55778425d500 2024-04-17T06:02:00.391 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.388+0000 7fa37f7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55778426abc0 0x55778426cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:02:00.391 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.388+0000 7fa37f7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55778426abc0 0x55778426cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58288/0 (socket says 172.21.15.47:58288) 2024-04-17T06:02:00.391 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.388+0000 7fa37f7fe640 1 -- 172.21.15.47:0/1008811716 learned_addr learned my addr 172.21.15.47:0/1008811716 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:02:00.391 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.388+0000 7fa384971640 1 --2- 172.21.15.47:0/1008811716 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55778425cbe0 0x55778425cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:02:00.391 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.388+0000 7fa37d7fa640 1 --2- 172.21.15.47:0/816498693 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fa3680a3f60 0x7fa3680a6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:02:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.388+0000 7fa37ffff640 1 --2- 172.21.15.47:0/1008811716 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55778425d500 0x55778426a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:02:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.388+0000 7fa384971640 1 --2- 172.21.15.47:0/816498693 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fa3680a3f60 0x7fa3680a6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:02:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.388+0000 7fa37d7fa640 1 -- 172.21.15.47:0/816498693 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fa3740f2620 con 0x7fa380107290 2024-04-17T06:02:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.389+0000 7fa37f7fe640 1 -- 172.21.15.47:0/1008811716 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55778425d500 msgr2=0x55778426a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:02:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.389+0000 7fa37f7fe640 1 --2- 172.21.15.47:0/1008811716 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55778425d500 0x55778426a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.389+0000 7fa37f7fe640 1 -- 172.21.15.47:0/1008811716 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55778425cbe0 msgr2=0x55778425cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:02:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.389+0000 7fa37f7fe640 1 --2- 172.21.15.47:0/1008811716 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55778425cbe0 0x55778425cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.389+0000 7fa37f7fe640 1 -- 172.21.15.47:0/1008811716 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557783ed5210 con 0x55778426abc0 2024-04-17T06:02:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.389+0000 7fa384971640 1 --2- 172.21.15.47:0/1008811716 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55778425cbe0 0x55778425cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T06:02:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.389+0000 7fa37f7fe640 1 --2- 172.21.15.47:0/1008811716 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55778426abc0 0x55778426cfb0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fa3740be1e0 tx=0x7fa37402bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=965db4b46a90ff75 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:02:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.389+0000 7fa384971640 1 --2- 172.21.15.47:0/816498693 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fa3680a3f60 0x7fa3680a6420 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa378012a20 tx=0x7fa378004ab0 comp rx=0 tx=0).ready entity=mgr.24923 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:02:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.389+0000 7fa3667fc640 1 -- 172.21.15.47:0/1008811716 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa37404b070 con 0x55778426abc0 2024-04-17T06:02:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.389+0000 7fa3667fc640 1 -- 172.21.15.47:0/1008811716 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa374005690 con 0x55778426abc0 2024-04-17T06:02:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.389+0000 7fa3667fc640 1 -- 172.21.15.47:0/1008811716 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3740be4e0 con 0x55778426abc0 2024-04-17T06:02:00.393 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.390+0000 7fa388fc4740 1 -- 172.21.15.47:0/1008811716 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55778426abc0 msgr2=0x55778426cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:02:00.393 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.390+0000 7fa388fc4740 1 --2- 172.21.15.47:0/1008811716 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55778426abc0 0x55778426cfb0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fa3740be1e0 tx=0x7fa37402bc30 comp rx=0 tx=0).stop 2024-04-17T06:02:00.393 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.390+0000 7fa388fc4740 1 -- 172.21.15.47:0/1008811716 shutdown_connections 2024-04-17T06:02:00.393 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.390+0000 7fa388fc4740 1 --2- 172.21.15.47:0/1008811716 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55778426abc0 0x55778426cfb0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:00.393 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.390+0000 7fa388fc4740 1 --2- 172.21.15.47:0/1008811716 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55778425d500 0x55778426a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:00.393 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.390+0000 7fa388fc4740 1 --2- 172.21.15.47:0/1008811716 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55778425cbe0 0x55778425cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:00.393 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.390+0000 7fa388fc4740 1 -- 172.21.15.47:0/1008811716 >> 172.21.15.47:0/1008811716 conn(0x5577842579e0 msgr2=0x557784257dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:02:00.393 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.390+0000 7fa388fc4740 1 -- 172.21.15.47:0/1008811716 shutdown_connections 2024-04-17T06:02:00.393 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.390+0000 7fa388fc4740 1 -- 172.21.15.47:0/1008811716 wait complete. 2024-04-17T06:02:00.393 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.390+0000 7fa388fc4740 1 -- start start 2024-04-17T06:02:00.393 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.390+0000 7fa388fc4740 1 Processor -- start 2024-04-17T06:02:00.393 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.391+0000 7fa388fc4740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55778425cbe0 0x557784267c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:02:00.393 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.391+0000 7fa388fc4740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55778425d500 0x557784268160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:02:00.393 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.391+0000 7fa388fc4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55778426abc0 0x5577842686a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:02:00.394 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.391+0000 7fa388fc4740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5577841c27d0 con 0x55778425d500 2024-04-17T06:02:00.394 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.391+0000 7fa388fc4740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55778414ae90 con 0x55778425cbe0 2024-04-17T06:02:00.394 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.391+0000 7fa388fc4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55778426d540 con 0x55778426abc0 2024-04-17T06:02:00.394 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.391+0000 7fa388fc4740 10 client.0 _ll_register_callbacks cb 0x557784160d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:02:00.394 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 06:02:00 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T06:02:00.135+0000 7eff0660a700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-17T06:02:00.394 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 06:02:00 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T06:02:00.135+0000 7eff0660a700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-17T06:02:00.394 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 06:02:00 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T06:02:00.135+0000 7eff0660a700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-17T06:02:00.394 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 06:02:00 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T06:02:00.135+0000 7eff0660a700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-17T06:02:00.394 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 06:02:00 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T06:02:00.135+0000 7eff0660a700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-17T06:02:00.397 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.391+0000 7fa37f7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55778425d500 0x557784268160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:02:00.397 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.391+0000 7fa37f7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55778425d500 0x557784268160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58302/0 (socket says 172.21.15.47:58302) 2024-04-17T06:02:00.397 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.391+0000 7fa37f7fe640 1 -- 172.21.15.47:0/1936721996 learned_addr learned my addr 172.21.15.47:0/1936721996 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:02:00.398 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.391+0000 7fa384971640 1 --2- 172.21.15.47:0/1936721996 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55778426abc0 0x5577842686a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:02:00.398 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.391+0000 7fa37ffff640 1 --2- 172.21.15.47:0/1936721996 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55778425cbe0 0x557784267c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:02:00.398 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.391+0000 7fa37f7fe640 1 -- 172.21.15.47:0/1936721996 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55778426abc0 msgr2=0x5577842686a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:02:00.398 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.391+0000 7fa37f7fe640 1 --2- 172.21.15.47:0/1936721996 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55778426abc0 0x5577842686a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:00.398 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.391+0000 7fa37f7fe640 1 -- 172.21.15.47:0/1936721996 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55778425cbe0 msgr2=0x557784267c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:02:00.398 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.391+0000 7fa37f7fe640 1 --2- 172.21.15.47:0/1936721996 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55778425cbe0 0x557784267c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:00.398 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.391+0000 7fa37f7fe640 1 -- 172.21.15.47:0/1936721996 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557783fa1620 con 0x55778425d500 2024-04-17T06:02:00.398 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.391+0000 7fa37f7fe640 1 --2- 172.21.15.47:0/1936721996 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55778425d500 0x557784268160 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fa3740be470 tx=0x7fa3740ba8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82566cab66993baa server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:02:00.398 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.391+0000 7fa35f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-17T06:02:00.398 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.392+0000 7fa35f7fe640 1 -- 172.21.15.47:0/1936721996 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa37404b040 con 0x55778425d500 2024-04-17T06:02:00.398 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.392+0000 7fa35f7fe640 1 -- 172.21.15.47:0/1936721996 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fa3740c6e40 con 0x55778425d500 2024-04-17T06:02:00.398 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.392+0000 7fa388fc4740 1 -- 172.21.15.47:0/1936721996 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5577841d01f0 con 0x55778425d500 2024-04-17T06:02:00.399 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.392+0000 7fa35f7fe640 1 -- 172.21.15.47:0/1936721996 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fa3740be2b0 con 0x55778425d500 2024-04-17T06:02:00.399 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.392+0000 7fa388fc4740 10 client.15468 fetch_fsmap learned FSMap version 22 2024-04-17T06:02:00.399 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.392+0000 7fa388fc4740 1 -- 172.21.15.47:0/1936721996 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5577842796d0 con 0x55778425d500 2024-04-17T06:02:00.399 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.392+0000 7fa35f7fe640 1 -- 172.21.15.47:0/1936721996 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5577841d01f0 con 0x55778425d500 2024-04-17T06:02:00.399 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.392+0000 7fa35f7fe640 1 -- 172.21.15.47:0/1936721996 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 22) ==== 8339+0+0 (secure 0 0 0) 0x7fa3740eff30 con 0x55778425d500 2024-04-17T06:02:00.399 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.392+0000 7fa388fc4740 10 client.15468 fetch_fsmap finished waiting for FSMap version 22 2024-04-17T06:02:00.399 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.392+0000 7fa388fc4740 10 client.15468 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-17T06:02:00.399 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.392+0000 7fa388fc4740 20 client.15468 populate_metadata read hostname 'smithi047' 2024-04-17T06:02:00.399 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.392+0000 7fa388fc4740 1 --2- 172.21.15.47:0/1936721996 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55778427aa20 0x55778427ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:02:00.399 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.392+0000 7fa388fc4740 4 client.15468 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T06:02:00.399 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.392+0000 7fa388fc4740 1 -- 172.21.15.47:0/1936721996 --> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5577841ca060 con 0x55778427aa20 2024-04-17T06:02:00.399 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.393+0000 7fa37ffff640 1 --2- 172.21.15.47:0/1936721996 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55778427aa20 0x55778427ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:02:00.399 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.393+0000 7fa37ffff640 1 --2- 172.21.15.47:0/1936721996 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55778427aa20 0x55778427ce80 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e22719cbe184516f server_cookie=c69c5260e75acf41 in_seq=0 out_seq=0 2024-04-17T06:02:00.399 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.393+0000 7fa35f7fe640 10 client.15468 ms_handle_connect on v2:172.21.15.98:6838/2841287917 2024-04-17T06:02:00.400 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.394+0000 7fa35f7fe640 1 -- 172.21.15.47:0/1936721996 <== mds.0 v2:172.21.15.98:6838/2841287917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5577841ca060 con 0x55778427aa20 2024-04-17T06:02:00.400 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.394+0000 7fa35f7fe640 10 client.15468 handle_command_reply: tid=0 2024-04-17T06:02:00.400 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.394+0000 7fa388fc4740 1 client.15468 shutdown 2024-04-17T06:02:00.400 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.394+0000 7fa388fc4740 1 -- 172.21.15.47:0/1936721996 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55778425d500 msgr2=0x557784268160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:02:00.400 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.394+0000 7fa388fc4740 1 --2- 172.21.15.47:0/1936721996 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55778425d500 0x557784268160 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fa3740be470 tx=0x7fa3740ba8b0 comp rx=0 tx=0).stop 2024-04-17T06:02:00.400 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.394+0000 7fa388fc4740 1 -- 172.21.15.47:0/1936721996 shutdown_connections 2024-04-17T06:02:00.400 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.394+0000 7fa388fc4740 1 --2- 172.21.15.47:0/1936721996 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55778427aa20 0x55778427ce80 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:00.400 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.394+0000 7fa388fc4740 1 --2- 172.21.15.47:0/1936721996 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55778426abc0 0x5577842686a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:00.400 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.394+0000 7fa388fc4740 1 --2- 172.21.15.47:0/1936721996 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55778425d500 0x557784268160 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:00.400 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.394+0000 7fa388fc4740 1 --2- 172.21.15.47:0/1936721996 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55778425cbe0 0x557784267c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:00.400 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.394+0000 7fa35f7fe640 0 client.15468 ms_handle_reset on v2:172.21.15.98:6838/2841287917 2024-04-17T06:02:00.400 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.394+0000 7fa388fc4740 1 -- 172.21.15.47:0/1936721996 >> 172.21.15.47:0/1936721996 conn(0x5577842579e0 msgr2=0x557784259730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:02:00.400 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.394+0000 7fa388fc4740 1 -- 172.21.15.47:0/1936721996 shutdown_connections 2024-04-17T06:02:00.400 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.394+0000 7fa388fc4740 1 -- 172.21.15.47:0/1936721996 wait complete. 2024-04-17T06:02:00.400 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.394+0000 7fa388fc4740 20 client.15468 trim_cache size 0 max 16384 2024-04-17T06:02:00.411 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.409+0000 7fa388fc4740 1 Processor -- start 2024-04-17T06:02:00.411 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.409+0000 7fa388fc4740 1 -- start start 2024-04-17T06:02:00.412 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.409+0000 7fa388fc4740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55778425cbe0 0x55778426a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:02:00.412 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.409+0000 7fa388fc4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55778425d500 0x5577842762c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:02:00.412 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.409+0000 7fa388fc4740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55778426abc0 0x5577842682f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:02:00.412 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.409+0000 7fa388fc4740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa3801ae2e0 con 0x55778425cbe0 2024-04-17T06:02:00.412 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.409+0000 7fa388fc4740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa3801abba0 con 0x55778426abc0 2024-04-17T06:02:00.412 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.409+0000 7fa388fc4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa3801a9a10 con 0x55778425d500 2024-04-17T06:02:00.412 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.409+0000 7fa37ffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55778425cbe0 0x55778426a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:02:00.412 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.409+0000 7fa37ffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55778425cbe0 0x55778426a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58310/0 (socket says 172.21.15.47:58310) 2024-04-17T06:02:00.412 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.409+0000 7fa37ffff640 1 -- 172.21.15.47:0/1123073583 learned_addr learned my addr 172.21.15.47:0/1123073583 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:02:00.412 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.410+0000 7fa37f7fe640 1 --2- 172.21.15.47:0/1123073583 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55778425d500 0x5577842762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:02:00.412 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.410+0000 7fa37ffff640 1 -- 172.21.15.47:0/1123073583 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55778425d500 msgr2=0x5577842762c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:02:00.412 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.410+0000 7fa37ffff640 1 --2- 172.21.15.47:0/1123073583 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55778425d500 0x5577842762c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:00.412 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.410+0000 7fa37ffff640 1 -- 172.21.15.47:0/1123073583 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55778426abc0 msgr2=0x5577842682f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:02:00.413 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.410+0000 7fa37ffff640 1 --2- 172.21.15.47:0/1123073583 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55778426abc0 0x5577842682f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:00.413 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.410+0000 7fa37ffff640 1 -- 172.21.15.47:0/1123073583 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55778427d500 con 0x55778425cbe0 2024-04-17T06:02:00.413 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.410+0000 7fa37ffff640 1 --2- 172.21.15.47:0/1123073583 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55778425cbe0 0x55778426a350 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x55778427d730 tx=0x7fa3700094a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea2fe74c35388252 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:02:00.413 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.410+0000 7fa365ffb640 1 -- 172.21.15.47:0/1123073583 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3700140d0 con 0x55778425cbe0 2024-04-17T06:02:00.413 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.410+0000 7fa365ffb640 1 -- 172.21.15.47:0/1123073583 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa370014270 con 0x55778425cbe0 2024-04-17T06:02:00.413 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.410+0000 7fa365ffb640 1 -- 172.21.15.47:0/1123073583 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa370003850 con 0x55778425cbe0 2024-04-17T06:02:00.413 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.411+0000 7fa388fc4740 1 -- 172.21.15.47:0/1123073583 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55778425cbe0 msgr2=0x55778426a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:02:00.413 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.411+0000 7fa388fc4740 1 --2- 172.21.15.47:0/1123073583 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55778425cbe0 0x55778426a350 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x55778427d730 tx=0x7fa3700094a0 comp rx=0 tx=0).stop 2024-04-17T06:02:00.413 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.411+0000 7fa388fc4740 1 -- 172.21.15.47:0/1123073583 shutdown_connections 2024-04-17T06:02:00.413 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.411+0000 7fa388fc4740 1 --2- 172.21.15.47:0/1123073583 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55778426abc0 0x5577842682f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:00.413 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.411+0000 7fa388fc4740 1 --2- 172.21.15.47:0/1123073583 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55778425d500 0x5577842762c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:00.413 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.411+0000 7fa388fc4740 1 --2- 172.21.15.47:0/1123073583 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55778425cbe0 0x55778426a350 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:00.413 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.411+0000 7fa388fc4740 1 -- 172.21.15.47:0/1123073583 >> 172.21.15.47:0/1123073583 conn(0x5577842579e0 msgr2=0x557784267be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:02:00.413 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.411+0000 7fa388fc4740 1 -- 172.21.15.47:0/1123073583 shutdown_connections 2024-04-17T06:02:00.414 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.411+0000 7fa388fc4740 1 -- 172.21.15.47:0/1123073583 wait complete. 2024-04-17T06:02:00.414 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.411+0000 7fa388fc4740 1 -- start start 2024-04-17T06:02:00.414 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.411+0000 7fa388fc4740 1 Processor -- start 2024-04-17T06:02:00.414 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.411+0000 7fa388fc4740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55778425cbe0 0x5577842647f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:02:00.414 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.411+0000 7fa388fc4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55778425d500 0x557784264d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:02:00.414 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.411+0000 7fa388fc4740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55778426abc0 0x557784265270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:02:00.414 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.411+0000 7fa388fc4740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa3801aa060 con 0x55778426abc0 2024-04-17T06:02:00.414 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.411+0000 7fa388fc4740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa3801ab4d0 con 0x55778425cbe0 2024-04-17T06:02:00.414 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.411+0000 7fa388fc4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa3801aa730 con 0x55778425d500 2024-04-17T06:02:00.414 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.411+0000 7fa388fc4740 10 client.0 _ll_register_callbacks cb 0x7fa37800a610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:02:00.414 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.411+0000 7fa384971640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55778426abc0 0x557784265270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:02:00.414 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.412+0000 7fa384971640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55778426abc0 0x557784265270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58324/0 (socket says 172.21.15.47:58324) 2024-04-17T06:02:00.414 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.412+0000 7fa384971640 1 -- 172.21.15.47:0/3717288821 learned_addr learned my addr 172.21.15.47:0/3717288821 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:02:00.415 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.412+0000 7fa37f7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55778425d500 0x557784264d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:02:00.415 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.412+0000 7fa37ffff640 1 --2- 172.21.15.47:0/3717288821 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55778425cbe0 0x5577842647f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:02:00.415 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.412+0000 7fa384971640 1 -- 172.21.15.47:0/3717288821 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55778425d500 msgr2=0x557784264d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:02:00.415 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.412+0000 7fa384971640 1 --2- 172.21.15.47:0/3717288821 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55778425d500 0x557784264d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:00.415 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.412+0000 7fa384971640 1 -- 172.21.15.47:0/3717288821 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55778425cbe0 msgr2=0x5577842647f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:02:00.415 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.412+0000 7fa384971640 1 --2- 172.21.15.47:0/3717288821 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55778425cbe0 0x5577842647f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:00.415 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.412+0000 7fa384971640 1 -- 172.21.15.47:0/3717288821 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55778425c820 con 0x55778426abc0 2024-04-17T06:02:00.415 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.412+0000 7fa384971640 1 --2- 172.21.15.47:0/3717288821 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55778426abc0 0x557784265270 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fa37800f6f0 tx=0x7fa37800acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=603446488f151562 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:02:00.415 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.412+0000 7fa35f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-17T06:02:00.415 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.412+0000 7fa35f7fe640 1 -- 172.21.15.47:0/3717288821 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa378018070 con 0x55778426abc0 2024-04-17T06:02:00.415 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.412+0000 7fa388fc4740 1 -- 172.21.15.47:0/3717288821 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa370014270 con 0x55778426abc0 2024-04-17T06:02:00.415 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.412+0000 7fa35f7fe640 1 -- 172.21.15.47:0/3717288821 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fa378004d20 con 0x55778426abc0 2024-04-17T06:02:00.415 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.412+0000 7fa35f7fe640 1 -- 172.21.15.47:0/3717288821 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fa37800faa0 con 0x55778426abc0 2024-04-17T06:02:00.416 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.412+0000 7fa388fc4740 10 client.15474 fetch_fsmap learned FSMap version 22 2024-04-17T06:02:00.416 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.412+0000 7fa388fc4740 1 -- 172.21.15.47:0/3717288821 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5577842e10f0 con 0x55778426abc0 2024-04-17T06:02:00.416 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.412+0000 7fa35f7fe640 1 -- 172.21.15.47:0/3717288821 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa370014270 con 0x55778426abc0 2024-04-17T06:02:00.416 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.413+0000 7fa35f7fe640 1 -- 172.21.15.47:0/3717288821 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 22) ==== 8339+0+0 (secure 0 0 0) 0x7fa3780042f0 con 0x55778426abc0 2024-04-17T06:02:00.416 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.413+0000 7fa388fc4740 10 client.15474 fetch_fsmap finished waiting for FSMap version 22 2024-04-17T06:02:00.416 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.413+0000 7fa388fc4740 10 client.15474 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-17T06:02:00.416 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.413+0000 7fa388fc4740 20 client.15474 populate_metadata read hostname 'smithi047' 2024-04-17T06:02:00.416 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.413+0000 7fa388fc4740 1 --2- 172.21.15.47:0/3717288821 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x5577842e1880 0x5577842e3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:02:00.416 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.413+0000 7fa388fc4740 4 client.15474 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-17T06:02:00.416 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.413+0000 7fa388fc4740 1 -- 172.21.15.47:0/3717288821 --> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557784146670 con 0x5577842e1880 2024-04-17T06:02:00.416 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.413+0000 7fa37ffff640 1 --2- 172.21.15.47:0/3717288821 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x5577842e1880 0x5577842e3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:02:00.416 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.413+0000 7fa35f7fe640 10 client.15474 ms_handle_connect on v2:172.21.15.98:6838/2841287917 2024-04-17T06:02:00.416 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.413+0000 7fa37ffff640 1 --2- 172.21.15.47:0/3717288821 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x5577842e1880 0x5577842e3ca0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce3c50477a872f34 server_cookie=a19ef2228ddc5872 in_seq=0 out_seq=0 2024-04-17T06:02:00.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:00.785+0000 7fa37d7fa640 1 -- 172.21.15.47:0/816498693 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7fa3740bdaf0 con 0x7fa380107290 2024-04-17T06:02:02.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:02:01 smithi103 ceph-mon[36243]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-17T06:02:02.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:02:01 smithi103 ceph-mon[36243]: mgrmap e25: x(active, since 42s), standbys: z, y 2024-04-17T06:02:02.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:02:01 smithi098 ceph-mon[36516]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-17T06:02:02.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:02:01 smithi098 ceph-mon[36516]: mgrmap e25: x(active, since 42s), standbys: z, y 2024-04-17T06:02:02.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:02:01 smithi047 ceph-mon[32223]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-17T06:02:02.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:02:01 smithi047 ceph-mon[32223]: mgrmap e25: x(active, since 42s), standbys: z, y 2024-04-17T06:02:03.525 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:03.527+0000 7fa35f7fe640 1 -- 172.21.15.47:0/3717288821 <== mds.0 v2:172.21.15.98:6838/2841287917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557784146670 con 0x5577842e1880 2024-04-17T06:02:03.525 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:03.527+0000 7fa35f7fe640 10 client.15474 handle_command_reply: tid=0 2024-04-17T06:02:03.525 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:03.527+0000 7fa388fc4740 1 client.15474 shutdown 2024-04-17T06:02:03.526 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:03.527+0000 7fa388fc4740 1 -- 172.21.15.47:0/3717288821 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55778426abc0 msgr2=0x557784265270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:02:03.526 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:03.527+0000 7fa388fc4740 1 --2- 172.21.15.47:0/3717288821 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55778426abc0 0x557784265270 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fa37800f6f0 tx=0x7fa37800acc0 comp rx=0 tx=0).stop 2024-04-17T06:02:03.526 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:03.527+0000 7fa388fc4740 1 -- 172.21.15.47:0/3717288821 shutdown_connections 2024-04-17T06:02:03.526 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:03.527+0000 7fa388fc4740 1 --2- 172.21.15.47:0/3717288821 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x5577842e1880 0x5577842e3ca0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:03.526 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:03.527+0000 7fa388fc4740 1 --2- 172.21.15.47:0/3717288821 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55778426abc0 0x557784265270 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:03.526 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:03.527+0000 7fa388fc4740 1 --2- 172.21.15.47:0/3717288821 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55778425d500 0x557784264d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:03.526 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:03.527+0000 7fa35f7fe640 0 client.15474 ms_handle_reset on v2:172.21.15.98:6838/2841287917 2024-04-17T06:02:03.526 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:03.527+0000 7fa388fc4740 1 --2- 172.21.15.47:0/3717288821 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55778425cbe0 0x5577842647f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:03.526 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:03.527+0000 7fa388fc4740 1 -- 172.21.15.47:0/3717288821 >> 172.21.15.47:0/3717288821 conn(0x5577842579e0 msgr2=0x557784256b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:02:03.526 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:03.527+0000 7fa388fc4740 1 -- 172.21.15.47:0/3717288821 shutdown_connections 2024-04-17T06:02:03.526 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:03.527+0000 7fa388fc4740 1 -- 172.21.15.47:0/3717288821 wait complete. 2024-04-17T06:02:03.526 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:03.527+0000 7fa388fc4740 20 client.15474 trim_cache size 0 max 16384 2024-04-17T06:02:03.527 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-04-17T06:02:03.527 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-04-17T06:02:03.527 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-04-17T06:02:03.527 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-04-17T06:02:03.528 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:03.528+0000 7fa3863d4640 1 -- 172.21.15.47:0/816498693 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fa3680a3f60 msgr2=0x7fa3680a6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:02:03.528 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:03.528+0000 7fa3863d4640 1 --2- 172.21.15.47:0/816498693 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fa3680a3f60 0x7fa3680a6420 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa378012a20 tx=0x7fa378004ab0 comp rx=0 tx=0).stop 2024-04-17T06:02:03.528 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:03.528+0000 7fa3863d4640 1 -- 172.21.15.47:0/816498693 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa380107290 msgr2=0x7fa38019e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:02:03.528 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:03.528+0000 7fa3863d4640 1 --2- 172.21.15.47:0/816498693 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa380107290 0x7fa38019e940 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fa374036040 tx=0x7fa37402ce20 comp rx=0 tx=0).stop 2024-04-17T06:02:03.528 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:03.528+0000 7fa3863d4640 1 -- 172.21.15.47:0/816498693 shutdown_connections 2024-04-17T06:02:03.528 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:03.528+0000 7fa3863d4640 1 --2- 172.21.15.47:0/816498693 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa380198600 0x7fa380198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:03.528 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:03.528+0000 7fa3863d4640 1 --2- 172.21.15.47:0/816498693 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fa3680a3f60 0x7fa3680a6420 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:03.528 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:03.528+0000 7fa3863d4640 1 --2- 172.21.15.47:0/816498693 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa380107290 0x7fa38019e940 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:03.528 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:03.528+0000 7fa3863d4640 1 --2- 172.21.15.47:0/816498693 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa3801021c0 0x7fa38019e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:03.528 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:03.528+0000 7fa3863d4640 1 -- 172.21.15.47:0/816498693 >> 172.21.15.47:0/816498693 conn(0x7fa3800fda00 msgr2=0x7fa3800ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:02:03.528 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:03.528+0000 7fa3863d4640 1 -- 172.21.15.47:0/816498693 shutdown_connections 2024-04-17T06:02:03.528 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:03.528+0000 7fa3863d4640 1 -- 172.21.15.47:0/816498693 wait complete. 2024-04-17T06:02:04.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:02:03 smithi103 ceph-mon[36243]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 938 B/s wr, 5 op/s 2024-04-17T06:02:04.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:02:03 smithi098 ceph-mon[36516]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 938 B/s wr, 5 op/s 2024-04-17T06:02:04.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:02:03 smithi047 ceph-mon[32223]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 938 B/s wr, 5 op/s 2024-04-17T06:02:04.612 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:02:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[66741]: *** Caught signal (Segmentation fault) ** 2024-04-17T06:02:04.612 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:02:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[66741]: in thread 7fa9d69df700 thread_name:md_log_replay 2024-04-17T06:02:04.612 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:02:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[66741]: ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev) 2024-04-17T06:02:04.612 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:02:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[66741]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fa9e7da0d20] 2024-04-17T06:02:04.612 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:02:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[66741]: 2: /usr/bin/ceph-mds(+0x549612) [0x564aaa58e612] 2024-04-17T06:02:04.612 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:02:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[66741]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x564aaa591dc3] 2024-04-17T06:02:04.612 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:02:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[66741]: 4: (MDLog::_replay_thread()+0x617) [0x564aaa5ec8e7] 2024-04-17T06:02:04.612 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:02:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[66741]: 5: (MDLog::ReplayThread::entry()+0x11) [0x564aaa5f5fd1] 2024-04-17T06:02:04.612 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:02:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[66741]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7fa9e7d961ca] 2024-04-17T06:02:04.612 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:02:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[66741]: 7: clone() 2024-04-17T06:02:04.613 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:02:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[66741]: 2024-04-17T06:02:04.163+0000 7fa9d69df700 -1 *** Caught signal (Segmentation fault) ** 2024-04-17T06:02:04.613 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:02:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[66741]: in thread 7fa9d69df700 thread_name:md_log_replay 2024-04-17T06:02:04.613 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:02:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[66741]: 2024-04-17T06:02:04.613 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:02:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[66741]: ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev) 2024-04-17T06:02:04.613 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:02:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[66741]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fa9e7da0d20] 2024-04-17T06:02:04.613 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:02:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[66741]: 2: /usr/bin/ceph-mds(+0x549612) [0x564aaa58e612] 2024-04-17T06:02:04.613 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:02:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[66741]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x564aaa591dc3] 2024-04-17T06:02:04.613 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:02:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[66741]: 4: (MDLog::_replay_thread()+0x617) [0x564aaa5ec8e7] 2024-04-17T06:02:04.613 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:02:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[66741]: 5: (MDLog::ReplayThread::entry()+0x11) [0x564aaa5f5fd1] 2024-04-17T06:02:04.613 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:02:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[66741]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7fa9e7d961ca] 2024-04-17T06:02:04.613 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:02:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[66741]: 7: clone() 2024-04-17T06:02:04.613 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:02:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[66741]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-17T06:02:04.613 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:02:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[66741]: 2024-04-17T06:02:04.613 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:02:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[66741]: 0> 2024-04-17T06:02:04.163+0000 7fa9d69df700 -1 *** Caught signal (Segmentation fault) ** 2024-04-17T06:02:04.613 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:02:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[66741]: in thread 7fa9d69df700 thread_name:md_log_replay 2024-04-17T06:02:04.614 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:02:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[66741]: 2024-04-17T06:02:04.614 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:02:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[66741]: ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev) 2024-04-17T06:02:04.614 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:02:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[66741]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fa9e7da0d20] 2024-04-17T06:02:04.614 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:02:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[66741]: 2: /usr/bin/ceph-mds(+0x549612) [0x564aaa58e612] 2024-04-17T06:02:04.614 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:02:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[66741]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x564aaa591dc3] 2024-04-17T06:02:04.614 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:02:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[66741]: 4: (MDLog::_replay_thread()+0x617) [0x564aaa5ec8e7] 2024-04-17T06:02:04.614 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:02:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[66741]: 5: (MDLog::ReplayThread::entry()+0x11) [0x564aaa5f5fd1] 2024-04-17T06:02:04.614 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:02:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[66741]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7fa9e7d961ca] 2024-04-17T06:02:04.614 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:02:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[66741]: 7: clone() 2024-04-17T06:02:04.614 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:02:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[66741]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-17T06:02:04.614 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:02:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[66741]: 2024-04-17T06:02:04.614 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:02:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[66741]: 0> 2024-04-17T06:02:04.163+0000 7fa9d69df700 -1 *** Caught signal (Segmentation fault) ** 2024-04-17T06:02:04.614 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:02:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[66741]: in thread 7fa9d69df700 thread_name:md_log_replay 2024-04-17T06:02:04.614 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:02:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[66741]: 2024-04-17T06:02:04.614 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:02:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[66741]: ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev) 2024-04-17T06:02:04.615 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:02:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[66741]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fa9e7da0d20] 2024-04-17T06:02:04.615 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:02:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[66741]: 2: /usr/bin/ceph-mds(+0x549612) [0x564aaa58e612] 2024-04-17T06:02:04.615 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:02:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[66741]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x564aaa591dc3] 2024-04-17T06:02:04.615 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:02:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[66741]: 4: (MDLog::_replay_thread()+0x617) [0x564aaa5ec8e7] 2024-04-17T06:02:04.615 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:02:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[66741]: 5: (MDLog::ReplayThread::entry()+0x11) [0x564aaa5f5fd1] 2024-04-17T06:02:04.615 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:02:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[66741]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7fa9e7d961ca] 2024-04-17T06:02:04.615 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:02:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[66741]: 7: clone() 2024-04-17T06:02:04.615 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:02:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[66741]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-17T06:02:04.615 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:02:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[66741]: 2024-04-17T06:02:04.615 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:02:04 smithi098 podman[70275]: 2024-04-17 06:02:04.224086483 +0000 UTC m=+0.022243332 container died aac8438892f4a29d84b5c3f241b76586040ce0739446f42cd5e409ca8696b9e1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True) 2024-04-17T06:02:05.382 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:02:05 smithi098 podman[70275]: 2024-04-17 06:02:05.005570259 +0000 UTC m=+0.803727149 container remove aac8438892f4a29d84b5c3f241b76586040ce0739446f42cd5e409ca8696b9e1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, ceph=True, CEPH_POINT_RELEASE=) 2024-04-17T06:02:05.383 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:02:05 smithi098 systemd[1]: ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@mds.e.service: Main process exited, code=exited, status=139/n/a 2024-04-17T06:02:05.383 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:02:05 smithi098 systemd[1]: ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@mds.e.service: Failed with result 'exit-code'. 2024-04-17T06:02:06.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:02:05 smithi103 ceph-mon[36243]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 21 op/s 2024-04-17T06:02:06.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:02:05 smithi098 ceph-mon[36516]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 21 op/s 2024-04-17T06:02:06.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:02:05 smithi047 ceph-mon[32223]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 21 op/s 2024-04-17T06:02:08.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:02:07 smithi098 ceph-mon[36516]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 25 op/s 2024-04-17T06:02:08.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:02:07 smithi047 ceph-mon[32223]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 25 op/s 2024-04-17T06:02:08.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:02:07 smithi103 ceph-mon[36243]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 25 op/s 2024-04-17T06:02:10.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:02:09 smithi098 ceph-mon[36516]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 22 op/s 2024-04-17T06:02:10.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:02:09 smithi047 ceph-mon[32223]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 22 op/s 2024-04-17T06:02:10.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:02:09 smithi103 ceph-mon[36243]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 22 op/s 2024-04-17T06:02:12.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:02:11 smithi098 ceph-mon[36516]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-04-17T06:02:12.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:02:11 smithi047 ceph-mon[32223]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-04-17T06:02:12.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:02:11 smithi103 ceph-mon[36243]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-04-17T06:02:14.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:02:13 smithi098 ceph-mon[36516]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-04-17T06:02:14.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:02:13 smithi047 ceph-mon[32223]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-04-17T06:02:14.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:02:13 smithi103 ceph-mon[36243]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-04-17T06:02:15.637 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:02:15 smithi098 systemd[1]: ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@mds.e.service: Scheduled restart job, restart counter is at 1. 2024-04-17T06:02:15.638 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:02:15 smithi098 systemd[1]: Stopped Ceph mds.e for 565db2d8-fc7e-11ee-bc91-c7b262605968. 2024-04-17T06:02:15.638 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:02:15 smithi098 systemd[1]: Starting Ceph mds.e for 565db2d8-fc7e-11ee-bc91-c7b262605968... 2024-04-17T06:02:15.638 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:02:15 smithi098 podman[70371]: 2024-04-17 06:02:15.526886656 +0000 UTC m=+0.017917854 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-17T06:02:15.638 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:02:15 smithi098 podman[70371]: 2024-04-17 06:02:15.640236687 +0000 UTC m=+0.131267931 container create 65048e80ea08b24da1ad9d393b8516b29f3df5e0d540ee31d12615b0b7386ef5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-04-17T06:02:15.907 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:02:15 smithi098 ceph-mon[36516]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-04-17T06:02:15.907 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:02:15 smithi098 podman[70371]: 2024-04-17 06:02:15.884940589 +0000 UTC m=+0.375971771 container init 65048e80ea08b24da1ad9d393b8516b29f3df5e0d540ee31d12615b0b7386ef5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416) 2024-04-17T06:02:15.907 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:02:15 smithi098 podman[70371]: 2024-04-17 06:02:15.887749353 +0000 UTC m=+0.378780535 container start 65048e80ea08b24da1ad9d393b8516b29f3df5e0d540ee31d12615b0b7386ef5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5) 2024-04-17T06:02:16.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:02:15 smithi047 ceph-mon[32223]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-04-17T06:02:16.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:02:15 smithi103 ceph-mon[36243]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-04-17T06:02:16.382 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:02:15 smithi098 ceph-mds[70391]: -- 172.21.15.98:0/723559150 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55a4de55a4e0 con 0x55a4de530c00 2024-04-17T06:02:16.383 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:02:15 smithi098 ceph-mds[70391]: -- 172.21.15.98:0/723559150 <== mon.2 v2:172.21.15.98:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a4de55da40 con 0x55a4de530c00 2024-04-17T06:02:16.383 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:02:15 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[70387]: starting mds.e at 2024-04-17T06:02:16.383 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:02:15 smithi098 bash[70371]: 65048e80ea08b24da1ad9d393b8516b29f3df5e0d540ee31d12615b0b7386ef5 2024-04-17T06:02:16.383 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:02:15 smithi098 systemd[1]: Started Ceph mds.e for 565db2d8-fc7e-11ee-bc91-c7b262605968. 2024-04-17T06:02:16.620 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.621+0000 7f714d11c640 1 -- 172.21.15.47:0/640953445 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7148101100 msgr2=0x7f71481014e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:02:16.620 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.621+0000 7f714d11c640 1 --2- 172.21.15.47:0/640953445 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7148101100 0x7f71481014e0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f7138009900 tx=0x7f713802bc30 comp rx=0 tx=0).stop 2024-04-17T06:02:16.621 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.621+0000 7f714d11c640 1 -- 172.21.15.47:0/640953445 shutdown_connections 2024-04-17T06:02:16.621 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.621+0000 7f714d11c640 1 --2- 172.21.15.47:0/640953445 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7148106180 0x7f714810dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:16.621 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.621+0000 7f714d11c640 1 --2- 172.21.15.47:0/640953445 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7148101ab0 0x7f7148105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:16.621 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.621+0000 7f714d11c640 1 --2- 172.21.15.47:0/640953445 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7148101100 0x7f71481014e0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:16.621 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.621+0000 7f714d11c640 1 -- 172.21.15.47:0/640953445 >> 172.21.15.47:0/640953445 conn(0x7f7148077fe0 msgr2=0x7f71480783f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:02:16.621 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.621+0000 7f714d11c640 1 -- 172.21.15.47:0/640953445 shutdown_connections 2024-04-17T06:02:16.621 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.621+0000 7f714d11c640 1 -- 172.21.15.47:0/640953445 wait complete. 2024-04-17T06:02:16.621 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.622+0000 7f714d11c640 1 Processor -- start 2024-04-17T06:02:16.623 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.622+0000 7f714d11c640 1 -- start start 2024-04-17T06:02:16.623 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.622+0000 7f714d11c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7148101100 0x7f7148102b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:02:16.623 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.622+0000 7f714d11c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7148106180 0x7f7148103080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:02:16.623 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.622+0000 7f714d11c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f71481035c0 0x7f71481a6940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:02:16.624 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.622+0000 7f714d11c640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f7148111210 con 0x7f7148106180 2024-04-17T06:02:16.624 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.622+0000 7f714d11c640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f7148111090 con 0x7f71481035c0 2024-04-17T06:02:16.624 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.622+0000 7f714d11c640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7148111390 con 0x7f7148101100 2024-04-17T06:02:16.624 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.622+0000 7f714659c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7148106180 0x7f7148103080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:02:16.624 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.622+0000 7f714659c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7148106180 0x7f7148103080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46862/0 (socket says 172.21.15.47:46862) 2024-04-17T06:02:16.624 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.622+0000 7f714659c640 1 -- 172.21.15.47:0/116877972 learned_addr learned my addr 172.21.15.47:0/116877972 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:02:16.624 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.623+0000 7f7146d9d640 1 --2- 172.21.15.47:0/116877972 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7148101100 0x7f7148102b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:02:16.624 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.623+0000 7f714759e640 1 --2- 172.21.15.47:0/116877972 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f71481035c0 0x7f71481a6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:02:16.624 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.623+0000 7f714659c640 1 -- 172.21.15.47:0/116877972 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7148101100 msgr2=0x7f7148102b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:02:16.624 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.623+0000 7f714659c640 1 --2- 172.21.15.47:0/116877972 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7148101100 0x7f7148102b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:16.624 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.623+0000 7f714659c640 1 -- 172.21.15.47:0/116877972 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f71481035c0 msgr2=0x7f71481a6940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:02:16.624 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.623+0000 7f714659c640 1 --2- 172.21.15.47:0/116877972 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f71481035c0 0x7f71481a6940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:16.624 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.623+0000 7f714659c640 1 -- 172.21.15.47:0/116877972 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71481a7040 con 0x7f7148106180 2024-04-17T06:02:16.624 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.623+0000 7f714759e640 1 --2- 172.21.15.47:0/116877972 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f71481035c0 0x7f71481a6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:02:16.625 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.623+0000 7f714659c640 1 --2- 172.21.15.47:0/116877972 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7148106180 0x7f7148103080 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f713c00c970 tx=0x7f713c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:02:16.625 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.623+0000 7f712ffff640 1 -- 172.21.15.47:0/116877972 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f713c007bb0 con 0x7f7148106180 2024-04-17T06:02:16.625 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.623+0000 7f712ffff640 1 -- 172.21.15.47:0/116877972 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f713c007d50 con 0x7f7148106180 2024-04-17T06:02:16.625 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.623+0000 7f714d11c640 1 -- 172.21.15.47:0/116877972 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71481a7330 con 0x7f7148106180 2024-04-17T06:02:16.625 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.623+0000 7f712ffff640 1 -- 172.21.15.47:0/116877972 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f713c00f450 con 0x7f7148106180 2024-04-17T06:02:16.625 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.623+0000 7f714d11c640 1 -- 172.21.15.47:0/116877972 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71481a7880 con 0x7f7148106180 2024-04-17T06:02:16.631 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.626+0000 7f712ffff640 1 -- 172.21.15.47:0/116877972 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f713c0046c0 con 0x7f7148106180 2024-04-17T06:02:16.631 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.626+0000 7f714fde4740 1 Processor -- start 2024-04-17T06:02:16.631 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.626+0000 7f714fde4740 1 -- start start 2024-04-17T06:02:16.631 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.626+0000 7f714fde4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b2d50fe0c0 0x55b2d50fe4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:02:16.632 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.626+0000 7f714fde4740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b2d50fe9e0 0x55b2d5107270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:02:16.632 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.626+0000 7f714fde4740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b2d51077b0 0x55b2d5109ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:02:16.632 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.626+0000 7f714fde4740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55b2d50637d0 con 0x55b2d51077b0 2024-04-17T06:02:16.632 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.626+0000 7f714fde4740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55b2d5000f70 con 0x55b2d50fe9e0 2024-04-17T06:02:16.632 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.626+0000 7f714fde4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55b2d505f790 con 0x55b2d50fe0c0 2024-04-17T06:02:16.632 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.627+0000 7f714659c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b2d51077b0 0x55b2d5109ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:02:16.632 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.627+0000 7f712ffff640 1 --2- 172.21.15.47:0/116877972 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f71300a3fb0 0x7f71300a6470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:02:16.632 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.627+0000 7f714659c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b2d51077b0 0x55b2d5109ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46872/0 (socket says 172.21.15.47:46872) 2024-04-17T06:02:16.632 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.627+0000 7f714659c640 1 -- 172.21.15.47:0/2790318265 learned_addr learned my addr 172.21.15.47:0/2790318265 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:02:16.632 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.627+0000 7f714759e640 1 --2- 172.21.15.47:0/2790318265 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b2d50fe0c0 0x55b2d50fe4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:02:16.632 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.627+0000 7f7146d9d640 1 --2- 172.21.15.47:0/2790318265 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b2d50fe9e0 0x55b2d5107270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:02:16.632 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.627+0000 7f714759e640 1 --2- 172.21.15.47:0/116877972 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f71300a3fb0 0x7f71300a6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:02:16.632 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.627+0000 7f712ffff640 1 -- 172.21.15.47:0/116877972 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f713c0d0750 con 0x7f7148106180 2024-04-17T06:02:16.633 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.627+0000 7f714659c640 1 -- 172.21.15.47:0/2790318265 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b2d50fe0c0 msgr2=0x55b2d50fe4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:02:16.633 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.627+0000 7f714659c640 1 --2- 172.21.15.47:0/2790318265 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b2d50fe0c0 0x55b2d50fe4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:16.633 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.627+0000 7f714659c640 1 -- 172.21.15.47:0/2790318265 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b2d50fe9e0 msgr2=0x55b2d5107270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:02:16.633 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.627+0000 7f714659c640 1 --2- 172.21.15.47:0/2790318265 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b2d50fe9e0 0x55b2d5107270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:16.633 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.627+0000 7f714659c640 1 -- 172.21.15.47:0/2790318265 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2d4d76210 con 0x55b2d51077b0 2024-04-17T06:02:16.633 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.627+0000 7f714759e640 1 --2- 172.21.15.47:0/2790318265 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b2d50fe0c0 0x55b2d50fe4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:02:16.633 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.627+0000 7f714659c640 1 --2- 172.21.15.47:0/2790318265 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b2d51077b0 0x55b2d5109ba0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f713c0a4c20 tx=0x7f713c087010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dfc5dcccef46b43 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:02:16.633 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.627+0000 7f714759e640 1 --2- 172.21.15.47:0/116877972 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f71300a3fb0 0x7f71300a6470 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7140005c40 tx=0x7f7140009200 comp rx=0 tx=0).ready entity=mgr.24923 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:02:16.633 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.627+0000 7f712d7fa640 1 -- 172.21.15.47:0/2790318265 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f713c01d070 con 0x55b2d51077b0 2024-04-17T06:02:16.633 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.627+0000 7f712d7fa640 1 -- 172.21.15.47:0/2790318265 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f713c09cbe0 con 0x55b2d51077b0 2024-04-17T06:02:16.633 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.627+0000 7f712d7fa640 1 -- 172.21.15.47:0/2790318265 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f713c09b930 con 0x55b2d51077b0 2024-04-17T06:02:16.633 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.628+0000 7f714fde4740 1 -- 172.21.15.47:0/2790318265 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b2d51077b0 msgr2=0x55b2d5109ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:02:16.633 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.628+0000 7f714fde4740 1 --2- 172.21.15.47:0/2790318265 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b2d51077b0 0x55b2d5109ba0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f713c0a4c20 tx=0x7f713c087010 comp rx=0 tx=0).stop 2024-04-17T06:02:16.633 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.628+0000 7f714fde4740 1 -- 172.21.15.47:0/2790318265 shutdown_connections 2024-04-17T06:02:16.634 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.628+0000 7f714fde4740 1 --2- 172.21.15.47:0/2790318265 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b2d51077b0 0x55b2d5109ba0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:16.634 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.628+0000 7f714fde4740 1 --2- 172.21.15.47:0/2790318265 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b2d50fe9e0 0x55b2d5107270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:16.634 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.628+0000 7f714fde4740 1 --2- 172.21.15.47:0/2790318265 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b2d50fe0c0 0x55b2d50fe4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:16.634 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.628+0000 7f714fde4740 1 -- 172.21.15.47:0/2790318265 >> 172.21.15.47:0/2790318265 conn(0x55b2d50f89e0 msgr2=0x55b2d50f8dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:02:16.634 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.628+0000 7f714fde4740 1 -- 172.21.15.47:0/2790318265 shutdown_connections 2024-04-17T06:02:16.634 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.628+0000 7f714fde4740 1 -- 172.21.15.47:0/2790318265 wait complete. 2024-04-17T06:02:16.634 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.628+0000 7f714fde4740 1 -- start start 2024-04-17T06:02:16.634 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.629+0000 7f714fde4740 1 Processor -- start 2024-04-17T06:02:16.634 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.629+0000 7f714fde4740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b2d50fe0c0 0x55b2d5104880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:02:16.634 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.629+0000 7f714fde4740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b2d50fe9e0 0x55b2d5104dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:02:16.634 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.629+0000 7f714fde4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b2d51077b0 0x55b2d5105300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:02:16.634 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.629+0000 7f714fde4740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55b2d510a320 con 0x55b2d50fe9e0 2024-04-17T06:02:16.635 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.629+0000 7f714fde4740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55b2d4febe90 con 0x55b2d50fe0c0 2024-04-17T06:02:16.635 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.629+0000 7f714fde4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55b2d510a4a0 con 0x55b2d51077b0 2024-04-17T06:02:16.635 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.629+0000 7f714fde4740 10 client.0 _ll_register_callbacks cb 0x55b2d5001d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:02:16.635 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.629+0000 7f714659c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b2d50fe9e0 0x55b2d5104dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:02:16.635 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.629+0000 7f714659c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b2d50fe9e0 0x55b2d5104dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46888/0 (socket says 172.21.15.47:46888) 2024-04-17T06:02:16.635 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.629+0000 7f714659c640 1 -- 172.21.15.47:0/2305402298 learned_addr learned my addr 172.21.15.47:0/2305402298 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:02:16.635 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.629+0000 7f7146d9d640 1 --2- 172.21.15.47:0/2305402298 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b2d50fe0c0 0x55b2d5104880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:02:16.635 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.629+0000 7f714759e640 1 --2- 172.21.15.47:0/2305402298 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b2d51077b0 0x55b2d5105300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:02:16.635 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.629+0000 7f714659c640 1 -- 172.21.15.47:0/2305402298 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b2d51077b0 msgr2=0x55b2d5105300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:02:16.635 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.629+0000 7f714659c640 1 --2- 172.21.15.47:0/2305402298 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b2d51077b0 0x55b2d5105300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:16.635 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.629+0000 7f714659c640 1 -- 172.21.15.47:0/2305402298 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b2d50fe0c0 msgr2=0x55b2d5104880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:02:16.635 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.629+0000 7f714659c640 1 --2- 172.21.15.47:0/2305402298 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b2d50fe0c0 0x55b2d5104880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:16.635 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.629+0000 7f714659c640 1 -- 172.21.15.47:0/2305402298 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2d4e42620 con 0x55b2d50fe9e0 2024-04-17T06:02:16.635 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.629+0000 7f714759e640 1 --2- 172.21.15.47:0/2305402298 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b2d51077b0 0x55b2d5105300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:02:16.635 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.629+0000 7f714659c640 1 --2- 172.21.15.47:0/2305402298 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b2d50fe9e0 0x55b2d5104dc0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f713c09c470 tx=0x7f713c098d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79ccb676f690bf server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:02:16.636 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.629+0000 7f712aff5640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-17T06:02:16.636 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.630+0000 7f712aff5640 1 -- 172.21.15.47:0/2305402298 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f713c0b5060 con 0x55b2d50fe9e0 2024-04-17T06:02:16.636 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.630+0000 7f712aff5640 1 -- 172.21.15.47:0/2305402298 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f713c098440 con 0x55b2d50fe9e0 2024-04-17T06:02:16.636 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.630+0000 7f714fde4740 1 -- 172.21.15.47:0/2305402298 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f713c09cbe0 con 0x55b2d50fe9e0 2024-04-17T06:02:16.636 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.630+0000 7f712aff5640 1 -- 172.21.15.47:0/2305402298 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f713c01d070 con 0x55b2d50fe9e0 2024-04-17T06:02:16.636 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.630+0000 7f714fde4740 10 client.15492 fetch_fsmap learned FSMap version 22 2024-04-17T06:02:16.636 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.630+0000 7f714fde4740 1 -- 172.21.15.47:0/2305402298 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55b2d5116610 con 0x55b2d50fe9e0 2024-04-17T06:02:16.636 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.630+0000 7f712aff5640 1 -- 172.21.15.47:0/2305402298 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f713c09cbe0 con 0x55b2d50fe9e0 2024-04-17T06:02:16.636 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.630+0000 7f712aff5640 1 -- 172.21.15.47:0/2305402298 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 22) ==== 8339+0+0 (secure 0 0 0) 0x7f713c0ce160 con 0x55b2d50fe9e0 2024-04-17T06:02:16.636 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.630+0000 7f714fde4740 10 client.15492 fetch_fsmap finished waiting for FSMap version 22 2024-04-17T06:02:16.636 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.630+0000 7f714fde4740 10 client.15492 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-17T06:02:16.636 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.630+0000 7f714fde4740 20 client.15492 populate_metadata read hostname 'smithi047' 2024-04-17T06:02:16.636 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.630+0000 7f714fde4740 1 --2- 172.21.15.47:0/2305402298 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55b2d5117960 0x55b2d5119d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:02:16.637 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.630+0000 7f714fde4740 4 client.15492 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T06:02:16.637 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.630+0000 7f714fde4740 1 -- 172.21.15.47:0/2305402298 --> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b2d50711f0 con 0x55b2d5117960 2024-04-17T06:02:16.637 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.631+0000 7f7146d9d640 1 --2- 172.21.15.47:0/2305402298 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55b2d5117960 0x55b2d5119d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:02:16.637 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.631+0000 7f712aff5640 10 client.15492 ms_handle_connect on v2:172.21.15.98:6838/2841287917 2024-04-17T06:02:16.637 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.631+0000 7f7146d9d640 1 --2- 172.21.15.47:0/2305402298 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55b2d5117960 0x55b2d5119d50 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ccf1e5d10689c420 server_cookie=4e11c1bba7464960 in_seq=0 out_seq=0 2024-04-17T06:02:16.637 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.632+0000 7f712aff5640 1 -- 172.21.15.47:0/2305402298 <== mds.0 v2:172.21.15.98:6838/2841287917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b2d50711f0 con 0x55b2d5117960 2024-04-17T06:02:16.637 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.632+0000 7f712aff5640 10 client.15492 handle_command_reply: tid=0 2024-04-17T06:02:16.637 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.632+0000 7f714fde4740 1 client.15492 shutdown 2024-04-17T06:02:16.637 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.632+0000 7f714fde4740 1 -- 172.21.15.47:0/2305402298 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b2d50fe9e0 msgr2=0x55b2d5104dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:02:16.637 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.632+0000 7f714fde4740 1 --2- 172.21.15.47:0/2305402298 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b2d50fe9e0 0x55b2d5104dc0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f713c09c470 tx=0x7f713c098d80 comp rx=0 tx=0).stop 2024-04-17T06:02:16.637 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.632+0000 7f714fde4740 1 -- 172.21.15.47:0/2305402298 shutdown_connections 2024-04-17T06:02:16.637 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.632+0000 7f714fde4740 1 --2- 172.21.15.47:0/2305402298 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55b2d5117960 0x55b2d5119d50 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:16.637 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.632+0000 7f714fde4740 1 --2- 172.21.15.47:0/2305402298 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b2d51077b0 0x55b2d5105300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:16.638 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.632+0000 7f712aff5640 0 client.15492 ms_handle_reset on v2:172.21.15.98:6838/2841287917 2024-04-17T06:02:16.638 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.632+0000 7f714fde4740 1 --2- 172.21.15.47:0/2305402298 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b2d50fe9e0 0x55b2d5104dc0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:16.638 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.632+0000 7f714fde4740 1 --2- 172.21.15.47:0/2305402298 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b2d50fe0c0 0x55b2d5104880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:16.638 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.632+0000 7f714fde4740 1 -- 172.21.15.47:0/2305402298 >> 172.21.15.47:0/2305402298 conn(0x55b2d50f89e0 msgr2=0x55b2d50fa730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:02:16.638 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.632+0000 7f714fde4740 1 -- 172.21.15.47:0/2305402298 shutdown_connections 2024-04-17T06:02:16.638 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.632+0000 7f714fde4740 1 -- 172.21.15.47:0/2305402298 wait complete. 2024-04-17T06:02:16.638 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.632+0000 7f714fde4740 20 client.15492 trim_cache size 0 max 16384 2024-04-17T06:02:16.650 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.647+0000 7f714fde4740 1 Processor -- start 2024-04-17T06:02:16.650 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.647+0000 7f714fde4740 1 -- start start 2024-04-17T06:02:16.650 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.647+0000 7f714fde4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b2d50fe0c0 0x55b2d51131e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:02:16.650 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.647+0000 7f714fde4740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b2d50fe9e0 0x55b2d5106f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:02:16.650 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.647+0000 7f714fde4740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b2d51077b0 0x55b2d511f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:02:16.650 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.647+0000 7f714fde4740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f71481b2720 con 0x55b2d50fe9e0 2024-04-17T06:02:16.650 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.647+0000 7f714fde4740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f71481affe0 con 0x55b2d51077b0 2024-04-17T06:02:16.650 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.647+0000 7f714fde4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f71481ade30 con 0x55b2d50fe0c0 2024-04-17T06:02:16.650 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.647+0000 7f714659c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b2d50fe9e0 0x55b2d5106f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:02:16.651 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.647+0000 7f714659c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b2d50fe9e0 0x55b2d5106f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46904/0 (socket says 172.21.15.47:46904) 2024-04-17T06:02:16.651 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.647+0000 7f714659c640 1 -- 172.21.15.47:0/1793965267 learned_addr learned my addr 172.21.15.47:0/1793965267 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:02:16.651 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.647+0000 7f7146d9d640 1 --2- 172.21.15.47:0/1793965267 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b2d50fe0c0 0x55b2d51131e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:02:16.651 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.647+0000 7f714759e640 1 --2- 172.21.15.47:0/1793965267 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b2d51077b0 0x55b2d511f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:02:16.651 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.647+0000 7f714659c640 1 -- 172.21.15.47:0/1793965267 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b2d50fe0c0 msgr2=0x55b2d51131e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:02:16.651 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.647+0000 7f714659c640 1 --2- 172.21.15.47:0/1793965267 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b2d50fe0c0 0x55b2d51131e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:16.651 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.647+0000 7f714659c640 1 -- 172.21.15.47:0/1793965267 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b2d51077b0 msgr2=0x55b2d511f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:02:16.651 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.647+0000 7f714659c640 1 --2- 172.21.15.47:0/1793965267 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b2d51077b0 0x55b2d511f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:16.651 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.647+0000 7f714659c640 1 -- 172.21.15.47:0/1793965267 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2d50f9c10 con 0x55b2d50fe9e0 2024-04-17T06:02:16.651 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.647+0000 7f714659c640 1 --2- 172.21.15.47:0/1793965267 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b2d50fe9e0 0x55b2d5106f40 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f713c098940 tx=0x7f713c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86b1454425b2275c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:02:16.651 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.648+0000 7f712cff9640 1 -- 172.21.15.47:0/1793965267 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f713c0b5070 con 0x55b2d50fe9e0 2024-04-17T06:02:16.651 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.648+0000 7f712cff9640 1 -- 172.21.15.47:0/1793965267 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f713c09cbe0 con 0x55b2d50fe9e0 2024-04-17T06:02:16.651 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.648+0000 7f712cff9640 1 -- 172.21.15.47:0/1793965267 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f713c0d0b30 con 0x55b2d50fe9e0 2024-04-17T06:02:16.652 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.648+0000 7f714fde4740 1 -- 172.21.15.47:0/1793965267 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b2d50fe9e0 msgr2=0x55b2d5106f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:02:16.652 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.648+0000 7f714fde4740 1 --2- 172.21.15.47:0/1793965267 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b2d50fe9e0 0x55b2d5106f40 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f713c098940 tx=0x7f713c000bc0 comp rx=0 tx=0).stop 2024-04-17T06:02:16.652 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.648+0000 7f714fde4740 1 -- 172.21.15.47:0/1793965267 shutdown_connections 2024-04-17T06:02:16.652 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.648+0000 7f714fde4740 1 --2- 172.21.15.47:0/1793965267 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b2d51077b0 0x55b2d511f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:16.652 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.648+0000 7f714fde4740 1 --2- 172.21.15.47:0/1793965267 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b2d50fe9e0 0x55b2d5106f40 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:16.652 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.648+0000 7f714fde4740 1 --2- 172.21.15.47:0/1793965267 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b2d50fe0c0 0x55b2d51131e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:16.652 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.648+0000 7f714fde4740 1 -- 172.21.15.47:0/1793965267 >> 172.21.15.47:0/1793965267 conn(0x55b2d50f89e0 msgr2=0x55b2d5113d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:02:16.652 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.648+0000 7f714fde4740 1 -- 172.21.15.47:0/1793965267 shutdown_connections 2024-04-17T06:02:16.652 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.648+0000 7f714fde4740 1 -- 172.21.15.47:0/1793965267 wait complete. 2024-04-17T06:02:16.652 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.649+0000 7f714fde4740 1 -- start start 2024-04-17T06:02:16.652 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.649+0000 7f714fde4740 1 Processor -- start 2024-04-17T06:02:16.652 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.649+0000 7f714fde4740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b2d50fe0c0 0x55b2d510f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:02:16.652 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.649+0000 7f714fde4740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b2d50fe9e0 0x55b2d510ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:02:16.652 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.649+0000 7f714fde4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b2d51077b0 0x55b2d510b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:02:16.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.649+0000 7f714fde4740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f71481ae4a0 con 0x55b2d50fe0c0 2024-04-17T06:02:16.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.649+0000 7f714fde4740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f71481af910 con 0x55b2d50fe9e0 2024-04-17T06:02:16.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.649+0000 7f714fde4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f71481aeb70 con 0x55b2d51077b0 2024-04-17T06:02:16.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.649+0000 7f714fde4740 10 client.0 _ll_register_callbacks cb 0x7f713c007f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:02:16.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.649+0000 7f7146d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b2d50fe0c0 0x55b2d510f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:02:16.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.649+0000 7f7146d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b2d50fe0c0 0x55b2d510f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46912/0 (socket says 172.21.15.47:46912) 2024-04-17T06:02:16.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.649+0000 7f7146d9d640 1 -- 172.21.15.47:0/2243265450 learned_addr learned my addr 172.21.15.47:0/2243265450 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:02:16.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.649+0000 7f714759e640 1 --2- 172.21.15.47:0/2243265450 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b2d51077b0 0x55b2d510b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:02:16.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.649+0000 7f714659c640 1 --2- 172.21.15.47:0/2243265450 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b2d50fe9e0 0x55b2d510ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:02:16.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.649+0000 7f7146d9d640 1 -- 172.21.15.47:0/2243265450 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b2d51077b0 msgr2=0x55b2d510b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:02:16.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.649+0000 7f7146d9d640 1 --2- 172.21.15.47:0/2243265450 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b2d51077b0 0x55b2d510b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:16.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.649+0000 7f7146d9d640 1 -- 172.21.15.47:0/2243265450 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b2d50fe9e0 msgr2=0x55b2d510ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:02:16.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.649+0000 7f7146d9d640 1 --2- 172.21.15.47:0/2243265450 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b2d50fe9e0 0x55b2d510ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:16.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.649+0000 7f7146d9d640 1 -- 172.21.15.47:0/2243265450 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2d510b6f0 con 0x55b2d50fe0c0 2024-04-17T06:02:16.654 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.650+0000 7f714759e640 1 --2- 172.21.15.47:0/2243265450 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b2d51077b0 0x55b2d510b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:02:16.654 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.650+0000 7f7146d9d640 1 --2- 172.21.15.47:0/2243265450 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b2d50fe0c0 0x55b2d510f790 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f71380096c0 tx=0x7f713802cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5db56502e949a71 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:02:16.654 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.650+0000 7f712aff5640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-17T06:02:16.654 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.650+0000 7f712aff5640 1 -- 172.21.15.47:0/2243265450 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7138004230 con 0x55b2d50fe0c0 2024-04-17T06:02:16.654 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.650+0000 7f712aff5640 1 -- 172.21.15.47:0/2243265450 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f71380043d0 con 0x55b2d50fe0c0 2024-04-17T06:02:16.654 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.650+0000 7f714fde4740 1 -- 172.21.15.47:0/2243265450 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f713c09cbe0 con 0x55b2d50fe0c0 2024-04-17T06:02:16.654 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.650+0000 7f712aff5640 1 -- 172.21.15.47:0/2243265450 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f713805a050 con 0x55b2d50fe0c0 2024-04-17T06:02:16.654 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.650+0000 7f714fde4740 10 client.15498 fetch_fsmap learned FSMap version 22 2024-04-17T06:02:16.654 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.650+0000 7f714fde4740 1 -- 172.21.15.47:0/2243265450 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55b2d517d8e0 con 0x55b2d50fe0c0 2024-04-17T06:02:16.654 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.650+0000 7f712aff5640 1 -- 172.21.15.47:0/2243265450 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f713c09cbe0 con 0x55b2d50fe0c0 2024-04-17T06:02:16.654 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.650+0000 7f712aff5640 1 -- 172.21.15.47:0/2243265450 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 22) ==== 8339+0+0 (secure 0 0 0) 0x7f7138004ec0 con 0x55b2d50fe0c0 2024-04-17T06:02:16.654 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.650+0000 7f714fde4740 10 client.15498 fetch_fsmap finished waiting for FSMap version 22 2024-04-17T06:02:16.654 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.650+0000 7f714fde4740 10 client.15498 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-17T06:02:16.655 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.650+0000 7f714fde4740 20 client.15498 populate_metadata read hostname 'smithi047' 2024-04-17T06:02:16.655 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.651+0000 7f714fde4740 1 --2- 172.21.15.47:0/2243265450 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55b2d517e060 0x55b2d5180480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:02:16.655 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.651+0000 7f714fde4740 4 client.15498 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-17T06:02:16.655 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.651+0000 7f714fde4740 1 -- 172.21.15.47:0/2243265450 --> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b2d506b060 con 0x55b2d517e060 2024-04-17T06:02:16.655 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.651+0000 7f714659c640 1 --2- 172.21.15.47:0/2243265450 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55b2d517e060 0x55b2d5180480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:02:16.655 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.651+0000 7f712aff5640 10 client.15498 ms_handle_connect on v2:172.21.15.98:6838/2841287917 2024-04-17T06:02:16.655 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.651+0000 7f714659c640 1 --2- 172.21.15.47:0/2243265450 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55b2d517e060 0x55b2d5180480 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2b27edf1e8821a9 server_cookie=a4ac63ba50dfb7de in_seq=0 out_seq=0 2024-04-17T06:02:16.656 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.654+0000 7f712aff5640 1 -- 172.21.15.47:0/2243265450 <== mds.0 v2:172.21.15.98:6838/2841287917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b2d506b060 con 0x55b2d517e060 2024-04-17T06:02:16.656 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.654+0000 7f712aff5640 10 client.15498 handle_command_reply: tid=0 2024-04-17T06:02:16.656 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.654+0000 7f714fde4740 1 client.15498 shutdown 2024-04-17T06:02:16.656 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.654+0000 7f714fde4740 1 -- 172.21.15.47:0/2243265450 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b2d50fe0c0 msgr2=0x55b2d510f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:02:16.656 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.654+0000 7f714fde4740 1 --2- 172.21.15.47:0/2243265450 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b2d50fe0c0 0x55b2d510f790 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f71380096c0 tx=0x7f713802cae0 comp rx=0 tx=0).stop 2024-04-17T06:02:16.656 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.655+0000 7f714fde4740 1 -- 172.21.15.47:0/2243265450 shutdown_connections 2024-04-17T06:02:16.656 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.655+0000 7f714fde4740 1 --2- 172.21.15.47:0/2243265450 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55b2d517e060 0x55b2d5180480 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:16.656 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.655+0000 7f714fde4740 1 --2- 172.21.15.47:0/2243265450 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b2d51077b0 0x55b2d510b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:16.657 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.655+0000 7f714fde4740 1 --2- 172.21.15.47:0/2243265450 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b2d50fe9e0 0x55b2d510ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:16.657 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.655+0000 7f712aff5640 0 client.15498 ms_handle_reset on v2:172.21.15.98:6838/2841287917 2024-04-17T06:02:16.657 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.655+0000 7f714fde4740 1 --2- 172.21.15.47:0/2243265450 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b2d50fe0c0 0x55b2d510f790 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:16.657 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.655+0000 7f714fde4740 1 -- 172.21.15.47:0/2243265450 >> 172.21.15.47:0/2243265450 conn(0x55b2d50f89e0 msgr2=0x55b2d51047d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:02:16.657 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.655+0000 7f714fde4740 1 -- 172.21.15.47:0/2243265450 shutdown_connections 2024-04-17T06:02:16.657 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.655+0000 7f714fde4740 1 -- 172.21.15.47:0/2243265450 wait complete. 2024-04-17T06:02:16.657 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.655+0000 7f714fde4740 20 client.15498 trim_cache size 0 max 16384 2024-04-17T06:02:16.657 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-04-17T06:02:16.657 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-04-17T06:02:16.657 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-04-17T06:02:16.658 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-04-17T06:02:16.658 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.655+0000 7f714d11c640 1 -- 172.21.15.47:0/116877972 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f71300a3fb0 msgr2=0x7f71300a6470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:02:16.658 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.655+0000 7f714d11c640 1 --2- 172.21.15.47:0/116877972 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f71300a3fb0 0x7f71300a6470 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7140005c40 tx=0x7f7140009200 comp rx=0 tx=0).stop 2024-04-17T06:02:16.659 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.655+0000 7f714d11c640 1 -- 172.21.15.47:0/116877972 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7148106180 msgr2=0x7f7148103080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:02:16.659 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.655+0000 7f714d11c640 1 --2- 172.21.15.47:0/116877972 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7148106180 0x7f7148103080 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f713c00c970 tx=0x7f713c00ce30 comp rx=0 tx=0).stop 2024-04-17T06:02:16.659 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.656+0000 7f714d11c640 1 -- 172.21.15.47:0/116877972 shutdown_connections 2024-04-17T06:02:16.659 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.656+0000 7f714d11c640 1 --2- 172.21.15.47:0/116877972 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f71481035c0 0x7f71481a6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:16.659 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.656+0000 7f714d11c640 1 --2- 172.21.15.47:0/116877972 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f71300a3fb0 0x7f71300a6470 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:16.659 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.656+0000 7f714d11c640 1 --2- 172.21.15.47:0/116877972 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7148106180 0x7f7148103080 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:16.659 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.656+0000 7f714d11c640 1 --2- 172.21.15.47:0/116877972 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7148101100 0x7f7148102b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:16.659 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.656+0000 7f714d11c640 1 -- 172.21.15.47:0/116877972 >> 172.21.15.47:0/116877972 conn(0x7f7148077fe0 msgr2=0x7f7148069920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:02:16.659 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.656+0000 7f714d11c640 1 -- 172.21.15.47:0/116877972 shutdown_connections 2024-04-17T06:02:16.659 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:16.656+0000 7f714d11c640 1 -- 172.21.15.47:0/116877972 wait complete. 2024-04-17T06:02:17.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:02:16 smithi098 ceph-mon[36516]: daemon mds.e restarted 2024-04-17T06:02:17.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:02:16 smithi047 ceph-mon[32223]: daemon mds.e restarted 2024-04-17T06:02:17.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:02:16 smithi103 ceph-mon[36243]: daemon mds.e restarted 2024-04-17T06:02:18.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:02:17 smithi098 ceph-mon[36516]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-04-17T06:02:18.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:02:17 smithi098 ceph-mon[36516]: osdmap e90: 12 total, 12 up, 12 in 2024-04-17T06:02:18.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:02:17 smithi098 ceph-mon[36516]: mds.? [v2:172.21.15.98:6834/2376966242,v1:172.21.15.98:6835/2376966242] up:boot 2024-04-17T06:02:18.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:02:17 smithi098 ceph-mon[36516]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-04-17T06:02:18.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:02:17 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-17T06:02:18.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:02:17 smithi098 ceph-mon[36516]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-17T06:02:18.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:02:17 smithi047 ceph-mon[32223]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-04-17T06:02:18.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:02:17 smithi047 ceph-mon[32223]: osdmap e90: 12 total, 12 up, 12 in 2024-04-17T06:02:18.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:02:17 smithi047 ceph-mon[32223]: mds.? [v2:172.21.15.98:6834/2376966242,v1:172.21.15.98:6835/2376966242] up:boot 2024-04-17T06:02:18.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:02:17 smithi047 ceph-mon[32223]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-04-17T06:02:18.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:02:17 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-17T06:02:18.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:02:17 smithi047 ceph-mon[32223]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-17T06:02:18.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:02:17 smithi103 ceph-mon[36243]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-04-17T06:02:18.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:02:17 smithi103 ceph-mon[36243]: osdmap e90: 12 total, 12 up, 12 in 2024-04-17T06:02:18.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:02:17 smithi103 ceph-mon[36243]: mds.? [v2:172.21.15.98:6834/2376966242,v1:172.21.15.98:6835/2376966242] up:boot 2024-04-17T06:02:18.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:02:17 smithi103 ceph-mon[36243]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-04-17T06:02:18.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:02:17 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-17T06:02:18.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:02:17 smithi103 ceph-mon[36243]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-17T06:02:20.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:02:19 smithi098 ceph-mon[36516]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 3 op/s 2024-04-17T06:02:20.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:02:19 smithi047 ceph-mon[32223]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 3 op/s 2024-04-17T06:02:20.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:02:19 smithi103 ceph-mon[36243]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 3 op/s 2024-04-17T06:02:22.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:02:21 smithi098 ceph-mon[36516]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 307 B/s wr, 5 op/s 2024-04-17T06:02:22.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:02:21 smithi047 ceph-mon[32223]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 307 B/s wr, 5 op/s 2024-04-17T06:02:22.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:02:21 smithi103 ceph-mon[36243]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 307 B/s wr, 5 op/s 2024-04-17T06:02:24.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:02:23 smithi098 ceph-mon[36516]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 307 B/s wr, 5 op/s 2024-04-17T06:02:24.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:02:23 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T06:02:24.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:02:23 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:02:24.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:02:23 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:02:24.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:02:23 smithi047 ceph-mon[32223]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 307 B/s wr, 5 op/s 2024-04-17T06:02:24.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:02:23 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T06:02:24.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:02:23 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:02:24.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:02:23 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:02:24.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:02:23 smithi103 ceph-mon[36243]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 307 B/s wr, 5 op/s 2024-04-17T06:02:24.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:02:23 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T06:02:24.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:02:23 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:02:24.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:02:23 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:02:25.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:02:24 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T06:02:25.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:02:24 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T06:02:25.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:02:24 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:02:25.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:02:24 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T06:02:25.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:02:24 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T06:02:25.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:02:24 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:02:25.185 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:02:24 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T06:02:25.185 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:02:24 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T06:02:25.185 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:02:24 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:02:26.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:02:25 smithi098 ceph-mon[36516]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 307 B/s wr, 6 op/s 2024-04-17T06:02:26.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:02:25 smithi047 ceph-mon[32223]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 307 B/s wr, 6 op/s 2024-04-17T06:02:26.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:02:25 smithi103 ceph-mon[36243]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 307 B/s wr, 6 op/s 2024-04-17T06:02:28.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:02:27 smithi098 ceph-mon[36516]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 307 B/s wr, 6 op/s 2024-04-17T06:02:28.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:02:27 smithi047 ceph-mon[32223]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 307 B/s wr, 6 op/s 2024-04-17T06:02:28.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:02:27 smithi103 ceph-mon[36243]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 307 B/s wr, 6 op/s 2024-04-17T06:02:29.795 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.797+0000 7fe0871a8640 1 -- 172.21.15.47:0/2607209369 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe080101100 msgr2=0x7fe0801014e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:02:29.796 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.797+0000 7fe0871a8640 1 --2- 172.21.15.47:0/2607209369 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe080101100 0x7fe0801014e0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fe074009900 tx=0x7fe07402bc30 comp rx=0 tx=0).stop 2024-04-17T06:02:29.796 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.797+0000 7fe0871a8640 1 -- 172.21.15.47:0/2607209369 shutdown_connections 2024-04-17T06:02:29.796 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.797+0000 7fe0871a8640 1 --2- 172.21.15.47:0/2607209369 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe080106180 0x7fe08010dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:29.796 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.797+0000 7fe0871a8640 1 --2- 172.21.15.47:0/2607209369 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe080101ab0 0x7fe080105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:29.796 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.797+0000 7fe0871a8640 1 --2- 172.21.15.47:0/2607209369 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe080101100 0x7fe0801014e0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:29.796 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.797+0000 7fe0871a8640 1 -- 172.21.15.47:0/2607209369 >> 172.21.15.47:0/2607209369 conn(0x7fe080077fe0 msgr2=0x7fe0800783f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:02:29.796 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.797+0000 7fe0871a8640 1 -- 172.21.15.47:0/2607209369 shutdown_connections 2024-04-17T06:02:29.796 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.797+0000 7fe0871a8640 1 -- 172.21.15.47:0/2607209369 wait complete. 2024-04-17T06:02:29.798 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.798+0000 7fe0871a8640 1 Processor -- start 2024-04-17T06:02:29.798 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.798+0000 7fe0871a8640 1 -- start start 2024-04-17T06:02:29.798 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.799+0000 7fe0871a8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe080101100 0x7fe08019e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:02:29.798 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.799+0000 7fe0871a8640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe080106180 0x7fe08019e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:02:29.798 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.799+0000 7fe0871a8640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe0801985d0 0x7fe080198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:02:29.798 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.799+0000 7fe0871a8640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fe080112350 con 0x7fe0801985d0 2024-04-17T06:02:29.798 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.799+0000 7fe0871a8640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe0801121d0 con 0x7fe080106180 2024-04-17T06:02:29.798 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.799+0000 7fe0871a8640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe0801124d0 con 0x7fe080101100 2024-04-17T06:02:29.799 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.799+0000 7fe084f44640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe080101100 0x7fe08019e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:02:29.799 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.799+0000 7fe085745640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe0801985d0 0x7fe080198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:02:29.799 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.799+0000 7fe084f44640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe080101100 0x7fe08019e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.47:34186/0 (socket says 172.21.15.47:34186) 2024-04-17T06:02:29.799 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.799+0000 7fe084f44640 1 -- 172.21.15.47:0/3778545468 learned_addr learned my addr 172.21.15.47:0/3778545468 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:02:29.799 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.799+0000 7fe07ffff640 1 --2- 172.21.15.47:0/3778545468 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe080106180 0x7fe08019e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:02:29.799 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.799+0000 7fe085745640 1 -- 172.21.15.47:0/3778545468 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe080101100 msgr2=0x7fe08019e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:02:29.799 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.799+0000 7fe085745640 1 --2- 172.21.15.47:0/3778545468 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe080101100 0x7fe08019e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:29.799 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.799+0000 7fe085745640 1 -- 172.21.15.47:0/3778545468 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe080106180 msgr2=0x7fe08019e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:02:29.799 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.799+0000 7fe085745640 1 --2- 172.21.15.47:0/3778545468 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe080106180 0x7fe08019e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:29.799 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.800+0000 7fe085745640 1 -- 172.21.15.47:0/3778545468 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0801990f0 con 0x7fe0801985d0 2024-04-17T06:02:29.799 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.800+0000 7fe084f44640 1 --2- 172.21.15.47:0/3778545468 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe080101100 0x7fe08019e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T06:02:29.799 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.800+0000 7fe07ffff640 1 --2- 172.21.15.47:0/3778545468 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe080106180 0x7fe08019e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:02:29.799 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.800+0000 7fe085745640 1 --2- 172.21.15.47:0/3778545468 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe0801985d0 0x7fe080198a80 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fe07800e940 tx=0x7fe07800ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:02:29.799 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.800+0000 7fe07dffb640 1 -- 172.21.15.47:0/3778545468 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe078004d20 con 0x7fe0801985d0 2024-04-17T06:02:29.800 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.800+0000 7fe07dffb640 1 -- 172.21.15.47:0/3778545468 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe078005020 con 0x7fe0801985d0 2024-04-17T06:02:29.800 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.800+0000 7fe0871a8640 1 -- 172.21.15.47:0/3778545468 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0801993e0 con 0x7fe0801985d0 2024-04-17T06:02:29.800 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.801+0000 7fe07dffb640 1 -- 172.21.15.47:0/3778545468 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0780116b0 con 0x7fe0801985d0 2024-04-17T06:02:29.800 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.801+0000 7fe0871a8640 1 -- 172.21.15.47:0/3778545468 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0801a5630 con 0x7fe0801985d0 2024-04-17T06:02:29.802 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.804+0000 7fe07dffb640 1 -- 172.21.15.47:0/3778545468 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7fe0780054d0 con 0x7fe0801985d0 2024-04-17T06:02:29.803 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.805+0000 7fe089d92740 1 Processor -- start 2024-04-17T06:02:29.803 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.805+0000 7fe089d92740 1 -- start start 2024-04-17T06:02:29.803 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.805+0000 7fe089d92740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a9b125f0c0 0x55a9b125f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:02:29.803 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.805+0000 7fe089d92740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a9b125f9e0 0x55a9b1268270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:02:29.803 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.805+0000 7fe089d92740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a9b12687b0 0x55a9b126aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:02:29.803 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.805+0000 7fe089d92740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55a9b11c47d0 con 0x55a9b125f0c0 2024-04-17T06:02:29.803 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.805+0000 7fe089d92740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55a9b1161f70 con 0x55a9b12687b0 2024-04-17T06:02:29.803 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.805+0000 7fe089d92740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55a9b11c0790 con 0x55a9b125f9e0 2024-04-17T06:02:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.805+0000 7fe07ffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a9b125f0c0 0x55a9b125f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:02:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.805+0000 7fe07ffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a9b125f0c0 0x55a9b125f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55282/0 (socket says 172.21.15.47:55282) 2024-04-17T06:02:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.805+0000 7fe07ffff640 1 -- 172.21.15.47:0/2810829645 learned_addr learned my addr 172.21.15.47:0/2810829645 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:02:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.805+0000 7fe07dffb640 1 --2- 172.21.15.47:0/3778545468 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fe0700a8480 0x7fe0700aa940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:02:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.805+0000 7fe084f44640 1 --2- 172.21.15.47:0/2810829645 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a9b125f9e0 0x55a9b1268270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:02:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.805+0000 7fe07ffff640 1 --2- 172.21.15.47:0/2810829645 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a9b12687b0 0x55a9b126aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:02:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.806+0000 7fe07ffff640 1 -- 172.21.15.47:0/2810829645 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a9b125f9e0 msgr2=0x55a9b1268270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:02:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.806+0000 7fe07ffff640 1 --2- 172.21.15.47:0/2810829645 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a9b125f9e0 0x55a9b1268270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.806+0000 7fe07ffff640 1 -- 172.21.15.47:0/2810829645 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a9b12687b0 msgr2=0x55a9b126aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:02:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.806+0000 7fe07ffff640 1 --2- 172.21.15.47:0/2810829645 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a9b12687b0 0x55a9b126aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.806+0000 7fe07ffff640 1 -- 172.21.15.47:0/2810829645 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9b0ed7210 con 0x55a9b125f0c0 2024-04-17T06:02:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.806+0000 7fe07ffff640 1 --2- 172.21.15.47:0/2810829645 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a9b125f0c0 0x55a9b125f4a0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fe06c00bfa0 tx=0x7fe06c00b650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f01ba9db8ffe1aa server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:02:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.806+0000 7fe085745640 1 --2- 172.21.15.47:0/3778545468 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fe0700a8480 0x7fe0700aa940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:02:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.806+0000 7fe06affd640 1 -- 172.21.15.47:0/2810829645 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe06c019070 con 0x55a9b125f0c0 2024-04-17T06:02:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.806+0000 7fe06affd640 1 -- 172.21.15.47:0/2810829645 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe06c005500 con 0x55a9b125f0c0 2024-04-17T06:02:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.806+0000 7fe06affd640 1 -- 172.21.15.47:0/2810829645 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe06c004e00 con 0x55a9b125f0c0 2024-04-17T06:02:29.807 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.807+0000 7fe07dffb640 1 -- 172.21.15.47:0/3778545468 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8469+0+0 (secure 0 0 0) 0x7fe0780d0b10 con 0x7fe0801985d0 2024-04-17T06:02:29.807 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.807+0000 7fe089d92740 1 -- 172.21.15.47:0/2810829645 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a9b125f0c0 msgr2=0x55a9b125f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:02:29.807 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.807+0000 7fe085745640 1 --2- 172.21.15.47:0/3778545468 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fe0700a8480 0x7fe0700aa940 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe0780ce0b0 tx=0x7fe078098bc0 comp rx=0 tx=0).ready entity=mgr.24923 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:02:29.807 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.807+0000 7fe089d92740 1 --2- 172.21.15.47:0/2810829645 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a9b125f0c0 0x55a9b125f4a0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fe06c00bfa0 tx=0x7fe06c00b650 comp rx=0 tx=0).stop 2024-04-17T06:02:29.807 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.807+0000 7fe089d92740 1 -- 172.21.15.47:0/2810829645 shutdown_connections 2024-04-17T06:02:29.807 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.807+0000 7fe089d92740 1 --2- 172.21.15.47:0/2810829645 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a9b12687b0 0x55a9b126aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:29.807 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.807+0000 7fe089d92740 1 --2- 172.21.15.47:0/2810829645 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a9b125f9e0 0x55a9b1268270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:29.807 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.807+0000 7fe089d92740 1 --2- 172.21.15.47:0/2810829645 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a9b125f0c0 0x55a9b125f4a0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:29.807 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.807+0000 7fe089d92740 1 -- 172.21.15.47:0/2810829645 >> 172.21.15.47:0/2810829645 conn(0x55a9b12599e0 msgr2=0x55a9b1259dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:02:29.807 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.807+0000 7fe089d92740 1 -- 172.21.15.47:0/2810829645 shutdown_connections 2024-04-17T06:02:29.807 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.808+0000 7fe089d92740 1 -- 172.21.15.47:0/2810829645 wait complete. 2024-04-17T06:02:29.807 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.808+0000 7fe089d92740 1 -- start start 2024-04-17T06:02:29.807 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.808+0000 7fe089d92740 1 Processor -- start 2024-04-17T06:02:29.810 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.809+0000 7fe089d92740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a9b125f0c0 0x55a9b1265830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:02:29.810 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.809+0000 7fe089d92740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a9b125f9e0 0x55a9b1265d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:02:29.811 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.809+0000 7fe089d92740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a9b12687b0 0x55a9b12662b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:02:29.811 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.809+0000 7fe084f44640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a9b125f9e0 0x55a9b1265d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:02:29.811 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.809+0000 7fe084f44640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a9b125f9e0 0x55a9b1265d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55294/0 (socket says 172.21.15.47:55294) 2024-04-17T06:02:29.811 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.809+0000 7fe084f44640 1 -- 172.21.15.47:0/2697966626 learned_addr learned my addr 172.21.15.47:0/2697966626 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:02:29.811 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.809+0000 7fe07ffff640 1 --2- 172.21.15.47:0/2697966626 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a9b125f0c0 0x55a9b1265830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:02:29.811 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.809+0000 7fe089d92740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55a9b126b3e0 con 0x55a9b125f9e0 2024-04-17T06:02:29.811 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.810+0000 7fe089d92740 1 -- 172.21.15.47:0/2697966626 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55a9b114ce90 con 0x55a9b125f0c0 2024-04-17T06:02:29.811 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.810+0000 7fe089d92740 1 -- 172.21.15.47:0/2697966626 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55a9b126b560 con 0x55a9b12687b0 2024-04-17T06:02:29.811 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.810+0000 7fe07ffff640 1 --2- 172.21.15.47:0/2697966626 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a9b12687b0 0x55a9b12662b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:02:29.811 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.810+0000 7fe089d92740 10 client.0 _ll_register_callbacks cb 0x55a9b1162d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:02:29.811 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.810+0000 7fe084f44640 1 -- 172.21.15.47:0/2697966626 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a9b12687b0 msgr2=0x55a9b12662b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:02:29.811 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.810+0000 7fe084f44640 1 --2- 172.21.15.47:0/2697966626 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a9b12687b0 0x55a9b12662b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:29.811 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.810+0000 7fe084f44640 1 -- 172.21.15.47:0/2697966626 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a9b125f0c0 msgr2=0x55a9b1265830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:02:29.811 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.810+0000 7fe084f44640 1 --2- 172.21.15.47:0/2697966626 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a9b125f0c0 0x55a9b1265830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:29.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.810+0000 7fe084f44640 1 -- 172.21.15.47:0/2697966626 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9b0fa3620 con 0x55a9b125f9e0 2024-04-17T06:02:29.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.810+0000 7fe07ffff640 1 --2- 172.21.15.47:0/2697966626 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a9b12687b0 0x55a9b12662b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:02:29.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.810+0000 7fe089d92740 1 -- 172.21.15.47:0/2697966626 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe06c005500 con 0x55a9b125f9e0 2024-04-17T06:02:29.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.810+0000 7fe084f44640 1 --2- 172.21.15.47:0/2697966626 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a9b125f9e0 0x55a9b1265d70 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fe074006010 tx=0x7fe07402cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27359d13c4e2d94 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:02:29.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.810+0000 7fe05ffff640 10 client.15510 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-17T06:02:29.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.811+0000 7fe05ffff640 1 -- 172.21.15.47:0/2697966626 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe074042070 con 0x55a9b125f9e0 2024-04-17T06:02:29.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.811+0000 7fe05ffff640 1 -- 172.21.15.47:0/2697966626 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8469+0+0 (secure 0 0 0) 0x7fe074004240 con 0x55a9b125f9e0 2024-04-17T06:02:29.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.811+0000 7fe05ffff640 1 -- 172.21.15.47:0/2697966626 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fe074004440 con 0x55a9b125f9e0 2024-04-17T06:02:29.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.811+0000 7fe089d92740 10 client.15510 fetch_fsmap learned FSMap version 24 2024-04-17T06:02:29.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.811+0000 7fe089d92740 1 -- 172.21.15.47:0/2697966626 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({fsmap=24}) -- 0x55a9b12775a0 con 0x55a9b125f9e0 2024-04-17T06:02:29.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.811+0000 7fe05ffff640 1 -- 172.21.15.47:0/2697966626 <== mon.0 v2:172.21.15.47:3300/0 4 ==== fsmap(e 24) ==== 8339+0+0 (secure 0 0 0) 0x7fe074004f00 con 0x55a9b125f9e0 2024-04-17T06:02:29.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.811+0000 7fe089d92740 10 client.15510 fetch_fsmap finished waiting for FSMap version 24 2024-04-17T06:02:29.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.811+0000 7fe089d92740 10 client.15510 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-17T06:02:29.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.811+0000 7fe089d92740 20 client.15510 populate_metadata read hostname 'smithi047' 2024-04-17T06:02:29.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.811+0000 7fe089d92740 1 --2- 172.21.15.47:0/2697966626 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55a9b1278a00 0x55a9b127adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:02:29.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.811+0000 7fe089d92740 4 client.15510 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T06:02:29.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.811+0000 7fe089d92740 1 -- 172.21.15.47:0/2697966626 --> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a9b11d21f0 con 0x55a9b1278a00 2024-04-17T06:02:29.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.812+0000 7fe07ffff640 1 --2- 172.21.15.47:0/2697966626 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55a9b1278a00 0x55a9b127adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:02:29.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.812+0000 7fe07ffff640 1 --2- 172.21.15.47:0/2697966626 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55a9b1278a00 0x55a9b127adf0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=954f3b893460f6f2 server_cookie=f263cbaba2156cd7 in_seq=0 out_seq=0 2024-04-17T06:02:29.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.812+0000 7fe05ffff640 10 client.15510 ms_handle_connect on v2:172.21.15.98:6838/2841287917 2024-04-17T06:02:29.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.813+0000 7fe05ffff640 1 -- 172.21.15.47:0/2697966626 <== mds.0 v2:172.21.15.98:6838/2841287917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a9b11d21f0 con 0x55a9b1278a00 2024-04-17T06:02:29.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.813+0000 7fe05ffff640 10 client.15510 handle_command_reply: tid=0 2024-04-17T06:02:29.815 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.813+0000 7fe089d92740 1 client.15510 shutdown 2024-04-17T06:02:29.815 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.813+0000 7fe089d92740 1 -- 172.21.15.47:0/2697966626 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a9b125f9e0 msgr2=0x55a9b1265d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:02:29.815 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.813+0000 7fe089d92740 1 --2- 172.21.15.47:0/2697966626 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a9b125f9e0 0x55a9b1265d70 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fe074006010 tx=0x7fe07402cab0 comp rx=0 tx=0).stop 2024-04-17T06:02:29.815 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.813+0000 7fe089d92740 1 -- 172.21.15.47:0/2697966626 shutdown_connections 2024-04-17T06:02:29.815 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.814+0000 7fe089d92740 1 --2- 172.21.15.47:0/2697966626 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55a9b1278a00 0x55a9b127adf0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:29.815 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.814+0000 7fe089d92740 1 --2- 172.21.15.47:0/2697966626 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a9b12687b0 0x55a9b12662b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:29.815 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.814+0000 7fe05ffff640 0 client.15510 ms_handle_reset on v2:172.21.15.98:6838/2841287917 2024-04-17T06:02:29.815 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.814+0000 7fe089d92740 1 --2- 172.21.15.47:0/2697966626 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a9b125f9e0 0x55a9b1265d70 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:29.815 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.814+0000 7fe089d92740 1 --2- 172.21.15.47:0/2697966626 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a9b125f0c0 0x55a9b1265830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:29.815 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.814+0000 7fe089d92740 1 -- 172.21.15.47:0/2697966626 >> 172.21.15.47:0/2697966626 conn(0x55a9b12599e0 msgr2=0x55a9b125b730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:02:29.815 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.814+0000 7fe089d92740 1 -- 172.21.15.47:0/2697966626 shutdown_connections 2024-04-17T06:02:29.815 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.814+0000 7fe089d92740 1 -- 172.21.15.47:0/2697966626 wait complete. 2024-04-17T06:02:29.815 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.814+0000 7fe089d92740 20 client.15510 trim_cache size 0 max 16384 2024-04-17T06:02:29.843 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.844+0000 7fe089d92740 1 Processor -- start 2024-04-17T06:02:29.843 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.844+0000 7fe089d92740 1 -- start start 2024-04-17T06:02:29.843 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.845+0000 7fe089d92740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a9b125f0c0 0x55a9b1267f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:02:29.843 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.845+0000 7fe089d92740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a9b125f9e0 0x55a9b12742a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:02:29.843 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.845+0000 7fe089d92740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a9b12687b0 0x55a9b1258980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:02:29.843 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.845+0000 7fe089d92740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fe0801ae270 con 0x55a9b12687b0 2024-04-17T06:02:29.843 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.845+0000 7fe089d92740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe0801abb30 con 0x55a9b125f9e0 2024-04-17T06:02:29.844 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.845+0000 7fe089d92740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe0801a99a0 con 0x55a9b125f0c0 2024-04-17T06:02:29.844 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.845+0000 7fe07ffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a9b12687b0 0x55a9b1258980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:02:29.844 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.845+0000 7fe07ffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a9b12687b0 0x55a9b1258980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55304/0 (socket says 172.21.15.47:55304) 2024-04-17T06:02:29.844 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.845+0000 7fe07ffff640 1 -- 172.21.15.47:0/952109960 learned_addr learned my addr 172.21.15.47:0/952109960 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:02:29.844 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.845+0000 7fe07ffff640 1 --2- 172.21.15.47:0/952109960 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a9b125f0c0 0x55a9b1267f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:02:29.844 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.845+0000 7fe084f44640 1 --2- 172.21.15.47:0/952109960 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a9b125f9e0 0x55a9b12742a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:02:29.844 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.845+0000 7fe07ffff640 1 -- 172.21.15.47:0/952109960 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a9b125f0c0 msgr2=0x55a9b1267f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:02:29.844 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.845+0000 7fe07ffff640 1 --2- 172.21.15.47:0/952109960 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a9b125f0c0 0x55a9b1267f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:29.844 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.845+0000 7fe07ffff640 1 -- 172.21.15.47:0/952109960 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a9b125f9e0 msgr2=0x55a9b12742a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:02:29.844 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.845+0000 7fe07ffff640 1 --2- 172.21.15.47:0/952109960 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a9b125f9e0 0x55a9b12742a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:29.844 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.845+0000 7fe07ffff640 1 -- 172.21.15.47:0/952109960 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9b127b160 con 0x55a9b12687b0 2024-04-17T06:02:29.844 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.845+0000 7fe084f44640 1 --2- 172.21.15.47:0/952109960 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a9b125f9e0 0x55a9b12742a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:02:29.845 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.846+0000 7fe07ffff640 1 --2- 172.21.15.47:0/952109960 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a9b12687b0 0x55a9b1258980 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fe06c00c010 tx=0x7fe06c00b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=909bec1596e39e97 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:02:29.848 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.846+0000 7fe06a7fc640 1 -- 172.21.15.47:0/952109960 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe06c035070 con 0x55a9b12687b0 2024-04-17T06:02:29.848 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.846+0000 7fe06a7fc640 1 -- 172.21.15.47:0/952109960 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe06c02d8b0 con 0x55a9b12687b0 2024-04-17T06:02:29.848 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.846+0000 7fe06a7fc640 1 -- 172.21.15.47:0/952109960 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe06c019070 con 0x55a9b12687b0 2024-04-17T06:02:29.848 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.846+0000 7fe089d92740 1 -- 172.21.15.47:0/952109960 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a9b12687b0 msgr2=0x55a9b1258980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:02:29.848 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.846+0000 7fe089d92740 1 --2- 172.21.15.47:0/952109960 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a9b12687b0 0x55a9b1258980 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fe06c00c010 tx=0x7fe06c00b5b0 comp rx=0 tx=0).stop 2024-04-17T06:02:29.848 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.847+0000 7fe089d92740 1 -- 172.21.15.47:0/952109960 shutdown_connections 2024-04-17T06:02:29.848 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.847+0000 7fe089d92740 1 --2- 172.21.15.47:0/952109960 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a9b12687b0 0x55a9b1258980 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:29.848 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.847+0000 7fe089d92740 1 --2- 172.21.15.47:0/952109960 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a9b125f9e0 0x55a9b12742a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:29.848 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.847+0000 7fe089d92740 1 --2- 172.21.15.47:0/952109960 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a9b125f0c0 0x55a9b1267f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:29.848 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.847+0000 7fe089d92740 1 -- 172.21.15.47:0/952109960 >> 172.21.15.47:0/952109960 conn(0x55a9b12599e0 msgr2=0x55a9b1258270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:02:29.848 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.847+0000 7fe089d92740 1 -- 172.21.15.47:0/952109960 shutdown_connections 2024-04-17T06:02:29.849 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.847+0000 7fe089d92740 1 -- 172.21.15.47:0/952109960 wait complete. 2024-04-17T06:02:29.849 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.847+0000 7fe089d92740 1 -- start start 2024-04-17T06:02:29.849 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.847+0000 7fe089d92740 1 Processor -- start 2024-04-17T06:02:29.849 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.847+0000 7fe089d92740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a9b125f0c0 0x55a9b12707b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:02:29.849 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.848+0000 7fe089d92740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a9b125f9e0 0x55a9b1270cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:02:29.849 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.848+0000 7fe089d92740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a9b12687b0 0x55a9b126d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:02:29.849 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.848+0000 7fe089d92740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fe0801a9ff0 con 0x55a9b125f0c0 2024-04-17T06:02:29.849 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.848+0000 7fe089d92740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe0801ab460 con 0x55a9b125f9e0 2024-04-17T06:02:29.849 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.848+0000 7fe089d92740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe0801aa6c0 con 0x55a9b12687b0 2024-04-17T06:02:29.849 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.848+0000 7fe089d92740 10 client.0 _ll_register_callbacks cb 0x7fe074005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:02:29.849 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.848+0000 7fe07ffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a9b125f0c0 0x55a9b12707b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:02:29.849 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.848+0000 7fe07ffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a9b125f0c0 0x55a9b12707b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55312/0 (socket says 172.21.15.47:55312) 2024-04-17T06:02:29.849 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.848+0000 7fe07ffff640 1 -- 172.21.15.47:0/2746826807 learned_addr learned my addr 172.21.15.47:0/2746826807 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:02:29.849 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.848+0000 7fe084f44640 1 --2- 172.21.15.47:0/2746826807 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a9b125f9e0 0x55a9b1270cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:02:29.850 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.848+0000 7fe07ffff640 1 -- 172.21.15.47:0/2746826807 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a9b12687b0 msgr2=0x55a9b126d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:02:29.850 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.848+0000 7fe07ffff640 1 --2- 172.21.15.47:0/2746826807 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a9b12687b0 0x55a9b126d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:29.850 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.848+0000 7fe07ffff640 1 -- 172.21.15.47:0/2746826807 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a9b125f9e0 msgr2=0x55a9b1270cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:02:29.850 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.848+0000 7fe07ffff640 1 --2- 172.21.15.47:0/2746826807 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a9b125f9e0 0x55a9b1270cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:29.850 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.848+0000 7fe07ffff640 1 -- 172.21.15.47:0/2746826807 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9b126dfb0 con 0x55a9b125f0c0 2024-04-17T06:02:29.850 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.848+0000 7fe07ffff640 1 --2- 172.21.15.47:0/2746826807 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a9b125f0c0 0x55a9b12707b0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fe06c0057c0 tx=0x7fe06c013f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94c4a9a323a9d642 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:02:29.851 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.848+0000 7fe05ffff640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-17T06:02:29.851 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.848+0000 7fe05ffff640 1 -- 172.21.15.47:0/2746826807 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe06c035070 con 0x55a9b125f0c0 2024-04-17T06:02:29.851 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.848+0000 7fe05ffff640 1 -- 172.21.15.47:0/2746826807 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8469+0+0 (secure 0 0 0) 0x7fe06c005250 con 0x55a9b125f0c0 2024-04-17T06:02:29.851 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.848+0000 7fe089d92740 1 -- 172.21.15.47:0/2746826807 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe06c02d8b0 con 0x55a9b125f0c0 2024-04-17T06:02:29.851 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.849+0000 7fe05ffff640 1 -- 172.21.15.47:0/2746826807 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fe06c03f050 con 0x55a9b125f0c0 2024-04-17T06:02:29.851 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.849+0000 7fe089d92740 10 client.15516 fetch_fsmap learned FSMap version 24 2024-04-17T06:02:29.851 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.849+0000 7fe089d92740 1 -- 172.21.15.47:0/2746826807 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55a9b12e81b0 con 0x55a9b125f0c0 2024-04-17T06:02:29.851 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.849+0000 7fe05ffff640 1 -- 172.21.15.47:0/2746826807 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe06c004960 con 0x55a9b125f0c0 2024-04-17T06:02:29.851 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.849+0000 7fe05ffff640 1 -- 172.21.15.47:0/2746826807 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 24) ==== 8339+0+0 (secure 0 0 0) 0x7fe06c01d940 con 0x55a9b125f0c0 2024-04-17T06:02:29.851 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.849+0000 7fe089d92740 10 client.15516 fetch_fsmap finished waiting for FSMap version 24 2024-04-17T06:02:29.851 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.849+0000 7fe089d92740 10 client.15516 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-17T06:02:29.851 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.849+0000 7fe089d92740 20 client.15516 populate_metadata read hostname 'smithi047' 2024-04-17T06:02:29.851 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.849+0000 7fe089d92740 1 --2- 172.21.15.47:0/2746826807 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55a9b1278a00 0x55a9b12ea590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:02:29.852 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.849+0000 7fe089d92740 4 client.15516 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-17T06:02:29.852 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.849+0000 7fe089d92740 1 -- 172.21.15.47:0/2746826807 --> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fe06c02d8b0 con 0x55a9b1278a00 2024-04-17T06:02:29.852 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.849+0000 7fe084f44640 1 --2- 172.21.15.47:0/2746826807 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55a9b1278a00 0x55a9b12ea590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:02:29.852 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.850+0000 7fe084f44640 1 --2- 172.21.15.47:0/2746826807 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55a9b1278a00 0x55a9b12ea590 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a0cefe9468242c8 server_cookie=36203fcc25f88fc9 in_seq=0 out_seq=0 2024-04-17T06:02:29.852 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.850+0000 7fe05ffff640 10 client.15516 ms_handle_connect on v2:172.21.15.98:6838/2841287917 2024-04-17T06:02:29.853 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.853+0000 7fe05ffff640 1 -- 172.21.15.47:0/2746826807 <== mds.0 v2:172.21.15.98:6838/2841287917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7fe06c02d8b0 con 0x55a9b1278a00 2024-04-17T06:02:29.854 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.853+0000 7fe05ffff640 10 client.15516 handle_command_reply: tid=0 2024-04-17T06:02:29.854 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.853+0000 7fe089d92740 1 client.15516 shutdown 2024-04-17T06:02:29.854 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.854+0000 7fe089d92740 1 -- 172.21.15.47:0/2746826807 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a9b125f0c0 msgr2=0x55a9b12707b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:02:29.854 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.854+0000 7fe089d92740 1 --2- 172.21.15.47:0/2746826807 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a9b125f0c0 0x55a9b12707b0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fe06c0057c0 tx=0x7fe06c013f50 comp rx=0 tx=0).stop 2024-04-17T06:02:29.854 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.854+0000 7fe089d92740 1 -- 172.21.15.47:0/2746826807 shutdown_connections 2024-04-17T06:02:29.854 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.854+0000 7fe089d92740 1 --2- 172.21.15.47:0/2746826807 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55a9b1278a00 0x55a9b12ea590 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:29.854 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.854+0000 7fe05ffff640 0 client.15516 ms_handle_reset on v2:172.21.15.98:6838/2841287917 2024-04-17T06:02:29.854 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.854+0000 7fe089d92740 1 --2- 172.21.15.47:0/2746826807 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a9b12687b0 0x55a9b126d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:29.854 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.854+0000 7fe089d92740 1 --2- 172.21.15.47:0/2746826807 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a9b125f9e0 0x55a9b1270cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:29.854 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.854+0000 7fe089d92740 1 --2- 172.21.15.47:0/2746826807 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a9b125f0c0 0x55a9b12707b0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:29.854 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.854+0000 7fe089d92740 1 -- 172.21.15.47:0/2746826807 >> 172.21.15.47:0/2746826807 conn(0x55a9b12599e0 msgr2=0x55a9b12657f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:02:29.854 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.854+0000 7fe089d92740 1 -- 172.21.15.47:0/2746826807 shutdown_connections 2024-04-17T06:02:29.854 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.854+0000 7fe089d92740 1 -- 172.21.15.47:0/2746826807 wait complete. 2024-04-17T06:02:29.854 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.854+0000 7fe089d92740 20 client.15516 trim_cache size 0 max 16384 2024-04-17T06:02:29.855 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-04-17T06:02:29.855 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-04-17T06:02:29.855 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-04-17T06:02:29.855 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-04-17T06:02:29.856 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.855+0000 7fe0871a8640 1 -- 172.21.15.47:0/3778545468 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fe0700a8480 msgr2=0x7fe0700aa940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:02:29.856 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.855+0000 7fe0871a8640 1 --2- 172.21.15.47:0/3778545468 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fe0700a8480 0x7fe0700aa940 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe0780ce0b0 tx=0x7fe078098bc0 comp rx=0 tx=0).stop 2024-04-17T06:02:29.856 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.855+0000 7fe0871a8640 1 -- 172.21.15.47:0/3778545468 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe0801985d0 msgr2=0x7fe080198a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:02:29.856 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.855+0000 7fe0871a8640 1 --2- 172.21.15.47:0/3778545468 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe0801985d0 0x7fe080198a80 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fe07800e940 tx=0x7fe07800ee00 comp rx=0 tx=0).stop 2024-04-17T06:02:29.856 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.855+0000 7fe0871a8640 1 -- 172.21.15.47:0/3778545468 shutdown_connections 2024-04-17T06:02:29.856 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.855+0000 7fe0871a8640 1 --2- 172.21.15.47:0/3778545468 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fe0700a8480 0x7fe0700aa940 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:29.856 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.855+0000 7fe0871a8640 1 --2- 172.21.15.47:0/3778545468 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe0801985d0 0x7fe080198a80 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:29.856 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.855+0000 7fe0871a8640 1 --2- 172.21.15.47:0/3778545468 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe080106180 0x7fe08019e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:29.856 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.855+0000 7fe0871a8640 1 --2- 172.21.15.47:0/3778545468 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe080101100 0x7fe08019e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:29.856 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.855+0000 7fe0871a8640 1 -- 172.21.15.47:0/3778545468 >> 172.21.15.47:0/3778545468 conn(0x7fe080077fe0 msgr2=0x7fe0800fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:02:29.856 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.855+0000 7fe0871a8640 1 -- 172.21.15.47:0/3778545468 shutdown_connections 2024-04-17T06:02:29.857 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:29.855+0000 7fe0871a8640 1 -- 172.21.15.47:0/3778545468 wait complete. 2024-04-17T06:02:30.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:02:29 smithi098 ceph-mon[36516]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 267 B/s wr, 5 op/s 2024-04-17T06:02:30.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:02:29 smithi047 ceph-mon[32223]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 267 B/s wr, 5 op/s 2024-04-17T06:02:30.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:02:29 smithi103 ceph-mon[36243]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 267 B/s wr, 5 op/s 2024-04-17T06:02:32.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:02:31 smithi098 ceph-mon[36516]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-17T06:02:32.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:02:31 smithi047 ceph-mon[32223]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-17T06:02:32.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:02:31 smithi103 ceph-mon[36243]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-17T06:02:34.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:02:33 smithi098 ceph-mon[36516]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-17T06:02:34.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:02:33 smithi047 ceph-mon[32223]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-17T06:02:34.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:02:33 smithi103 ceph-mon[36243]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-17T06:02:36.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:02:35 smithi098 ceph-mon[36516]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-17T06:02:36.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:02:35 smithi047 ceph-mon[32223]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-17T06:02:36.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:02:35 smithi103 ceph-mon[36243]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-17T06:02:38.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:02:37 smithi098 ceph-mon[36516]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-17T06:02:38.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:02:37 smithi047 ceph-mon[32223]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-17T06:02:38.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:02:37 smithi103 ceph-mon[36243]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-17T06:02:40.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:02:39 smithi098 ceph-mon[36516]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-17T06:02:40.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:02:39 smithi047 ceph-mon[32223]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-17T06:02:40.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:02:39 smithi103 ceph-mon[36243]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-17T06:02:42.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:02:41 smithi098 ceph-mon[36516]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-17T06:02:42.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:02:41 smithi047 ceph-mon[32223]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-17T06:02:42.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:02:41 smithi103 ceph-mon[36243]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-17T06:02:42.944 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.946+0000 7f771a5e8640 1 -- 172.21.15.47:0/2029682817 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7714101100 msgr2=0x7f77141014e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:02:42.944 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.946+0000 7f771a5e8640 1 --2- 172.21.15.47:0/2029682817 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7714101100 0x7f77141014e0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f7704009900 tx=0x7f770402bc30 comp rx=0 tx=0).stop 2024-04-17T06:02:42.944 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.946+0000 7f771a5e8640 1 -- 172.21.15.47:0/2029682817 shutdown_connections 2024-04-17T06:02:42.944 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.946+0000 7f771a5e8640 1 --2- 172.21.15.47:0/2029682817 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7714106180 0x7f771410dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:42.944 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.946+0000 7f771a5e8640 1 --2- 172.21.15.47:0/2029682817 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7714101ab0 0x7f7714105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:42.944 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.946+0000 7f771a5e8640 1 --2- 172.21.15.47:0/2029682817 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7714101100 0x7f77141014e0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:42.944 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.946+0000 7f771a5e8640 1 -- 172.21.15.47:0/2029682817 >> 172.21.15.47:0/2029682817 conn(0x7f7714077fe0 msgr2=0x7f77140783f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:02:42.945 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.946+0000 7f771a5e8640 1 -- 172.21.15.47:0/2029682817 shutdown_connections 2024-04-17T06:02:42.945 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.946+0000 7f771a5e8640 1 -- 172.21.15.47:0/2029682817 wait complete. 2024-04-17T06:02:42.946 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.947+0000 7f771a5e8640 1 Processor -- start 2024-04-17T06:02:42.946 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.947+0000 7f771a5e8640 1 -- start start 2024-04-17T06:02:42.947 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.947+0000 7f771a5e8640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7714101100 0x7f7714197740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:02:42.947 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.947+0000 7f771a5e8640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7714106180 0x7f7714197c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:02:42.947 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.947+0000 7f771a5e8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f771419c000 0x7f771419c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:02:42.947 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.947+0000 7f771a5e8640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f7714111240 con 0x7f7714101100 2024-04-17T06:02:42.947 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.947+0000 7f771a5e8640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f77141110c0 con 0x7f7714106180 2024-04-17T06:02:42.947 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.947+0000 7f771a5e8640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f77141113c0 con 0x7f771419c000 2024-04-17T06:02:42.947 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.947+0000 7f7713fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7714101100 0x7f7714197740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:02:42.947 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.947+0000 7f7713fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7714101100 0x7f7714197740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37394/0 (socket says 172.21.15.47:37394) 2024-04-17T06:02:42.947 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.947+0000 7f7713fff640 1 -- 172.21.15.47:0/2295258110 learned_addr learned my addr 172.21.15.47:0/2295258110 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:02:42.947 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.948+0000 7f7718b85640 1 --2- 172.21.15.47:0/2295258110 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f771419c000 0x7f771419c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:02:42.947 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.948+0000 7f77137fe640 1 --2- 172.21.15.47:0/2295258110 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7714106180 0x7f7714197c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:02:42.947 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.948+0000 7f7713fff640 1 -- 172.21.15.47:0/2295258110 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f771419c000 msgr2=0x7f771419c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:02:42.947 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.948+0000 7f7713fff640 1 --2- 172.21.15.47:0/2295258110 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f771419c000 0x7f771419c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:42.947 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.948+0000 7f7713fff640 1 -- 172.21.15.47:0/2295258110 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7714106180 msgr2=0x7f7714197c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:02:42.948 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.948+0000 7f7713fff640 1 --2- 172.21.15.47:0/2295258110 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7714106180 0x7f7714197c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:42.948 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.948+0000 7f7713fff640 1 -- 172.21.15.47:0/2295258110 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f771419caf0 con 0x7f7714101100 2024-04-17T06:02:42.948 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.948+0000 7f77137fe640 1 --2- 172.21.15.47:0/2295258110 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7714106180 0x7f7714197c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:02:42.948 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.948+0000 7f7713fff640 1 --2- 172.21.15.47:0/2295258110 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7714101100 0x7f7714197740 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f7704006010 tx=0x7f770402cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:02:42.948 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.948+0000 7f77117fa640 1 -- 172.21.15.47:0/2295258110 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7704041070 con 0x7f7714101100 2024-04-17T06:02:42.948 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.948+0000 7f771a5e8640 1 -- 172.21.15.47:0/2295258110 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f771419cd80 con 0x7f7714101100 2024-04-17T06:02:42.948 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.948+0000 7f77117fa640 1 -- 172.21.15.47:0/2295258110 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7704004450 con 0x7f7714101100 2024-04-17T06:02:42.948 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.948+0000 7f77117fa640 1 -- 172.21.15.47:0/2295258110 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7704034440 con 0x7f7714101100 2024-04-17T06:02:42.948 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.948+0000 7f771a5e8640 1 -- 172.21.15.47:0/2295258110 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7714069900 con 0x7f7714101100 2024-04-17T06:02:42.951 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.951+0000 7f771d1cb740 1 Processor -- start 2024-04-17T06:02:42.951 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.951+0000 7f771d1cb740 1 -- start start 2024-04-17T06:02:42.951 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.952+0000 7f771d1cb740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555f13f1f0c0 0x555f13f1f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:02:42.951 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.952+0000 7f771d1cb740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f13f1f9e0 0x555f13f28270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:02:42.951 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.952+0000 7f771d1cb740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555f13f287b0 0x555f13f2aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:02:42.951 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.952+0000 7f771d1cb740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x555f13e847d0 con 0x555f13f1f9e0 2024-04-17T06:02:42.951 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.952+0000 7f771d1cb740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x555f13e21f70 con 0x555f13f287b0 2024-04-17T06:02:42.951 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.952+0000 7f771d1cb740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x555f13e80790 con 0x555f13f1f0c0 2024-04-17T06:02:42.951 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.952+0000 7f7718b85640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555f13f1f0c0 0x555f13f1f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:02:42.951 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.952+0000 7f7718b85640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555f13f1f0c0 0x555f13f1f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.47:34958/0 (socket says 172.21.15.47:34958) 2024-04-17T06:02:42.951 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.952+0000 7f7718b85640 1 -- 172.21.15.47:0/1992722200 learned_addr learned my addr 172.21.15.47:0/1992722200 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:02:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.952+0000 7f77137fe640 1 --2- 172.21.15.47:0/1992722200 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555f13f287b0 0x555f13f2aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:02:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.952+0000 7f77117fa640 1 -- 172.21.15.47:0/2295258110 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f7704044070 con 0x7f7714101100 2024-04-17T06:02:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.952+0000 7f7718b85640 1 -- 172.21.15.47:0/1992722200 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555f13f287b0 msgr2=0x555f13f2aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:02:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.952+0000 7f7718b85640 1 --2- 172.21.15.47:0/1992722200 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555f13f287b0 0x555f13f2aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.952+0000 7f7718b85640 1 -- 172.21.15.47:0/1992722200 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f13f1f9e0 msgr2=0x555f13f28270 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-17T06:02:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.952+0000 7f7713fff640 1 --2- 172.21.15.47:0/1992722200 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f13f1f9e0 0x555f13f28270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:02:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.952+0000 7f7718b85640 1 --2- 172.21.15.47:0/1992722200 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f13f1f9e0 0x555f13f28270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.952+0000 7f7718b85640 1 -- 172.21.15.47:0/1992722200 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f13b97210 con 0x555f13f1f0c0 2024-04-17T06:02:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.953+0000 7f7713fff640 1 --2- 172.21.15.47:0/1992722200 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f13f1f9e0 0x555f13f28270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:02:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.953+0000 7f7718b85640 1 --2- 172.21.15.47:0/1992722200 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555f13f1f0c0 0x555f13f1f4a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f770c009f70 tx=0x7f770c010800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aff02778bfa0d3fd server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:02:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.953+0000 7f77117fa640 1 --2- 172.21.15.47:0/2295258110 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f76fc0a3ff0 0x7f76fc0a64b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:02:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.953+0000 7f76fa7fc640 1 -- 172.21.15.47:0/1992722200 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f770c016070 con 0x555f13f1f0c0 2024-04-17T06:02:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.953+0000 7f76fa7fc640 1 -- 172.21.15.47:0/1992722200 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f770c004d20 con 0x555f13f1f0c0 2024-04-17T06:02:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.953+0000 7f7713fff640 1 --2- 172.21.15.47:0/2295258110 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f76fc0a3ff0 0x7f76fc0a64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:02:42.956 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.953+0000 7f77117fa640 1 -- 172.21.15.47:0/2295258110 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8469+0+0 (secure 0 0 0) 0x7f77040f2560 con 0x7f7714101100 2024-04-17T06:02:42.956 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.953+0000 7f76fa7fc640 1 -- 172.21.15.47:0/1992722200 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f770c012950 con 0x555f13f1f0c0 2024-04-17T06:02:42.956 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.954+0000 7f771d1cb740 1 -- 172.21.15.47:0/1992722200 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555f13f1f0c0 msgr2=0x555f13f1f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:02:42.956 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.954+0000 7f771d1cb740 1 --2- 172.21.15.47:0/1992722200 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555f13f1f0c0 0x555f13f1f4a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f770c009f70 tx=0x7f770c010800 comp rx=0 tx=0).stop 2024-04-17T06:02:42.956 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.954+0000 7f7713fff640 1 --2- 172.21.15.47:0/2295258110 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f76fc0a3ff0 0x7f76fc0a64b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f77040efc10 tx=0x7f770402bc30 comp rx=0 tx=0).ready entity=mgr.24923 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:02:42.956 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.954+0000 7f771d1cb740 1 -- 172.21.15.47:0/1992722200 shutdown_connections 2024-04-17T06:02:42.956 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.954+0000 7f771d1cb740 1 --2- 172.21.15.47:0/1992722200 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555f13f287b0 0x555f13f2aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:42.957 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.954+0000 7f771d1cb740 1 --2- 172.21.15.47:0/1992722200 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f13f1f9e0 0x555f13f28270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:42.957 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.954+0000 7f771d1cb740 1 --2- 172.21.15.47:0/1992722200 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555f13f1f0c0 0x555f13f1f4a0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:42.957 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.954+0000 7f771d1cb740 1 -- 172.21.15.47:0/1992722200 >> 172.21.15.47:0/1992722200 conn(0x555f13f199e0 msgr2=0x555f13f19dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:02:42.957 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.954+0000 7f771d1cb740 1 -- 172.21.15.47:0/1992722200 shutdown_connections 2024-04-17T06:02:42.957 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.954+0000 7f771d1cb740 1 -- 172.21.15.47:0/1992722200 wait complete. 2024-04-17T06:02:42.957 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.954+0000 7f771d1cb740 1 -- start start 2024-04-17T06:02:42.957 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.955+0000 7f771d1cb740 1 Processor -- start 2024-04-17T06:02:42.957 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.955+0000 7f771d1cb740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f13f1f0c0 0x555f13f257d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:02:42.957 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.955+0000 7f771d1cb740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555f13f1f9e0 0x555f13f25d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:02:42.957 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.955+0000 7f771d1cb740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555f13f287b0 0x555f13f26250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:02:42.957 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.955+0000 7f771d1cb740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x555f13f2f7f0 con 0x555f13f1f0c0 2024-04-17T06:02:42.957 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.955+0000 7f771d1cb740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x555f13e0ce90 con 0x555f13f1f9e0 2024-04-17T06:02:42.957 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.955+0000 7f771d1cb740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x555f13f2f970 con 0x555f13f287b0 2024-04-17T06:02:42.957 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.955+0000 7f771d1cb740 10 client.0 _ll_register_callbacks cb 0x555f13e22d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:02:42.958 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.955+0000 7f7718b85640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f13f1f0c0 0x555f13f257d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:02:42.958 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.955+0000 7f7718b85640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f13f1f0c0 0x555f13f257d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37408/0 (socket says 172.21.15.47:37408) 2024-04-17T06:02:42.958 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.955+0000 7f7718b85640 1 -- 172.21.15.47:0/3049732203 learned_addr learned my addr 172.21.15.47:0/3049732203 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:02:42.958 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.955+0000 7f77137fe640 1 --2- 172.21.15.47:0/3049732203 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555f13f1f9e0 0x555f13f25d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:02:42.958 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.956+0000 7f7718b85640 1 -- 172.21.15.47:0/3049732203 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555f13f287b0 msgr2=0x555f13f26250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:02:42.958 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.956+0000 7f7718b85640 1 --2- 172.21.15.47:0/3049732203 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555f13f287b0 0x555f13f26250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:42.958 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.956+0000 7f7718b85640 1 -- 172.21.15.47:0/3049732203 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555f13f1f9e0 msgr2=0x555f13f25d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:02:42.958 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.956+0000 7f7718b85640 1 --2- 172.21.15.47:0/3049732203 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555f13f1f9e0 0x555f13f25d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:42.958 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.956+0000 7f7718b85640 1 -- 172.21.15.47:0/3049732203 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f13c63620 con 0x555f13f1f0c0 2024-04-17T06:02:42.958 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.956+0000 7f7718b85640 1 --2- 172.21.15.47:0/3049732203 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f13f1f0c0 0x555f13f257d0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f770c006010 tx=0x7f770c004c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26afd2e027bc3fbb server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:02:42.958 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.956+0000 7f76f37fe640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-17T06:02:42.958 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.956+0000 7f76f37fe640 1 -- 172.21.15.47:0/3049732203 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f770c029060 con 0x555f13f1f0c0 2024-04-17T06:02:42.958 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.956+0000 7f771d1cb740 1 -- 172.21.15.47:0/3049732203 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f770c004d20 con 0x555f13f1f0c0 2024-04-17T06:02:42.959 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.956+0000 7f76f37fe640 1 -- 172.21.15.47:0/3049732203 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8469+0+0 (secure 0 0 0) 0x7f770c0051e0 con 0x555f13f1f0c0 2024-04-17T06:02:42.959 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.956+0000 7f76f37fe640 1 -- 172.21.15.47:0/3049732203 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f770c016070 con 0x555f13f1f0c0 2024-04-17T06:02:42.959 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.956+0000 7f771d1cb740 10 client.15525 fetch_fsmap learned FSMap version 24 2024-04-17T06:02:42.959 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.956+0000 7f771d1cb740 1 -- 172.21.15.47:0/3049732203 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x555f13f3ba60 con 0x555f13f1f0c0 2024-04-17T06:02:42.959 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.957+0000 7f76f37fe640 1 -- 172.21.15.47:0/3049732203 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f770c004d20 con 0x555f13f1f0c0 2024-04-17T06:02:42.959 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.957+0000 7f76f37fe640 1 -- 172.21.15.47:0/3049732203 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 24) ==== 8339+0+0 (secure 0 0 0) 0x7f770c00f040 con 0x555f13f1f0c0 2024-04-17T06:02:42.959 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.957+0000 7f771d1cb740 10 client.15525 fetch_fsmap finished waiting for FSMap version 24 2024-04-17T06:02:42.959 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.957+0000 7f771d1cb740 10 client.15525 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-17T06:02:42.959 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.957+0000 7f771d1cb740 20 client.15525 populate_metadata read hostname 'smithi047' 2024-04-17T06:02:42.959 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.957+0000 7f771d1cb740 1 --2- 172.21.15.47:0/3049732203 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x555f13f3ce70 0x555f13f3f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:02:42.959 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.957+0000 7f771d1cb740 4 client.15525 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T06:02:42.959 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.957+0000 7f771d1cb740 1 -- 172.21.15.47:0/3049732203 --> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555f13e921f0 con 0x555f13f3ce70 2024-04-17T06:02:42.959 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.958+0000 7f77137fe640 1 --2- 172.21.15.47:0/3049732203 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x555f13f3ce70 0x555f13f3f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:02:42.959 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.958+0000 7f76f37fe640 10 client.15525 ms_handle_connect on v2:172.21.15.98:6838/2841287917 2024-04-17T06:02:42.960 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.958+0000 7f77137fe640 1 --2- 172.21.15.47:0/3049732203 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x555f13f3ce70 0x555f13f3f260 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c12be8884eb225f2 server_cookie=f0a4abc9d385cf5e in_seq=0 out_seq=0 2024-04-17T06:02:42.961 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.959+0000 7f76f37fe640 1 -- 172.21.15.47:0/3049732203 <== mds.0 v2:172.21.15.98:6838/2841287917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555f13e921f0 con 0x555f13f3ce70 2024-04-17T06:02:42.961 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.959+0000 7f76f37fe640 10 client.15525 handle_command_reply: tid=0 2024-04-17T06:02:42.961 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.959+0000 7f771d1cb740 1 client.15525 shutdown 2024-04-17T06:02:42.961 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.959+0000 7f771d1cb740 1 -- 172.21.15.47:0/3049732203 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f13f1f0c0 msgr2=0x555f13f257d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:02:42.961 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.959+0000 7f771d1cb740 1 --2- 172.21.15.47:0/3049732203 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f13f1f0c0 0x555f13f257d0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f770c006010 tx=0x7f770c004c50 comp rx=0 tx=0).stop 2024-04-17T06:02:42.961 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.959+0000 7f771d1cb740 1 -- 172.21.15.47:0/3049732203 shutdown_connections 2024-04-17T06:02:42.961 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.959+0000 7f771d1cb740 1 --2- 172.21.15.47:0/3049732203 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x555f13f3ce70 0x555f13f3f260 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:42.961 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.959+0000 7f771d1cb740 1 --2- 172.21.15.47:0/3049732203 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555f13f287b0 0x555f13f26250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:42.961 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.959+0000 7f76f37fe640 0 client.15525 ms_handle_reset on v2:172.21.15.98:6838/2841287917 2024-04-17T06:02:42.961 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.959+0000 7f771d1cb740 1 --2- 172.21.15.47:0/3049732203 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555f13f1f9e0 0x555f13f25d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:42.962 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.959+0000 7f771d1cb740 1 --2- 172.21.15.47:0/3049732203 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f13f1f0c0 0x555f13f257d0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:42.962 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.959+0000 7f771d1cb740 1 -- 172.21.15.47:0/3049732203 >> 172.21.15.47:0/3049732203 conn(0x555f13f199e0 msgr2=0x555f13f1b730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:02:42.962 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.959+0000 7f771d1cb740 1 -- 172.21.15.47:0/3049732203 shutdown_connections 2024-04-17T06:02:42.962 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.959+0000 7f771d1cb740 1 -- 172.21.15.47:0/3049732203 wait complete. 2024-04-17T06:02:42.962 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.960+0000 7f771d1cb740 20 client.15525 trim_cache size 0 max 16384 2024-04-17T06:02:42.988 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.989+0000 7f771d1cb740 1 Processor -- start 2024-04-17T06:02:42.988 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.989+0000 7f771d1cb740 1 -- start start 2024-04-17T06:02:42.988 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.989+0000 7f771d1cb740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555f13f1f0c0 0x555f13f27f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:02:42.988 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.989+0000 7f771d1cb740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f13f1f9e0 0x555f13f386e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:02:42.988 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.989+0000 7f771d1cb740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555f13f287b0 0x555f13f1b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:02:42.988 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.989+0000 7f771d1cb740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f77141ae1e0 con 0x555f13f1f9e0 2024-04-17T06:02:42.988 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.989+0000 7f771d1cb740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f77141abaa0 con 0x555f13f287b0 2024-04-17T06:02:42.989 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.989+0000 7f771d1cb740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f77141a98f0 con 0x555f13f1f0c0 2024-04-17T06:02:42.989 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.989+0000 7f77137fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f13f1f9e0 0x555f13f386e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:02:42.989 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.989+0000 7f7718b85640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555f13f1f0c0 0x555f13f27f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:02:42.989 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.989+0000 7f77137fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f13f1f9e0 0x555f13f386e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37420/0 (socket says 172.21.15.47:37420) 2024-04-17T06:02:42.989 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.989+0000 7f77137fe640 1 -- 172.21.15.47:0/644954323 learned_addr learned my addr 172.21.15.47:0/644954323 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:02:42.989 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.989+0000 7f7718b85640 1 --2- 172.21.15.47:0/644954323 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555f13f287b0 0x555f13f1b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:02:42.989 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.990+0000 7f7718b85640 1 -- 172.21.15.47:0/644954323 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555f13f287b0 msgr2=0x555f13f1b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:02:42.989 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.990+0000 7f7718b85640 1 --2- 172.21.15.47:0/644954323 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555f13f287b0 0x555f13f1b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:42.989 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.990+0000 7f7718b85640 1 -- 172.21.15.47:0/644954323 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f13f1f9e0 msgr2=0x555f13f386e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:02:42.989 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.990+0000 7f7718b85640 1 --2- 172.21.15.47:0/644954323 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f13f1f9e0 0x555f13f386e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:42.989 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.990+0000 7f7718b85640 1 -- 172.21.15.47:0/644954323 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f13f2aba0 con 0x555f13f1f0c0 2024-04-17T06:02:42.989 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.990+0000 7f7718b85640 1 --2- 172.21.15.47:0/644954323 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555f13f1f0c0 0x555f13f27f00 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f770c01a410 tx=0x7f770c030040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8aee2468057c670 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:02:42.989 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.990+0000 7f76f9ffb640 1 -- 172.21.15.47:0/644954323 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f770c029040 con 0x555f13f1f0c0 2024-04-17T06:02:42.989 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.990+0000 7f76f9ffb640 1 -- 172.21.15.47:0/644954323 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f770c004cc0 con 0x555f13f1f0c0 2024-04-17T06:02:42.989 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.990+0000 7f76f9ffb640 1 -- 172.21.15.47:0/644954323 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f770c003cd0 con 0x555f13f1f0c0 2024-04-17T06:02:42.993 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.991+0000 7f771d1cb740 1 -- 172.21.15.47:0/644954323 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555f13f1f0c0 msgr2=0x555f13f27f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:02:42.993 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.991+0000 7f771d1cb740 1 --2- 172.21.15.47:0/644954323 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555f13f1f0c0 0x555f13f27f00 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f770c01a410 tx=0x7f770c030040 comp rx=0 tx=0).stop 2024-04-17T06:02:42.993 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.991+0000 7f771d1cb740 1 -- 172.21.15.47:0/644954323 shutdown_connections 2024-04-17T06:02:42.993 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.991+0000 7f771d1cb740 1 --2- 172.21.15.47:0/644954323 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555f13f287b0 0x555f13f1b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:42.993 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.991+0000 7f771d1cb740 1 --2- 172.21.15.47:0/644954323 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f13f1f9e0 0x555f13f386e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:42.993 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.991+0000 7f771d1cb740 1 --2- 172.21.15.47:0/644954323 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555f13f1f0c0 0x555f13f27f00 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:42.993 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.991+0000 7f771d1cb740 1 -- 172.21.15.47:0/644954323 >> 172.21.15.47:0/644954323 conn(0x555f13f199e0 msgr2=0x555f13f1ae30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:02:42.993 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.991+0000 7f771d1cb740 1 -- 172.21.15.47:0/644954323 shutdown_connections 2024-04-17T06:02:42.994 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.991+0000 7f771d1cb740 1 -- 172.21.15.47:0/644954323 wait complete. 2024-04-17T06:02:42.994 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.991+0000 7f771d1cb740 1 -- start start 2024-04-17T06:02:42.994 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.992+0000 7f771d1cb740 1 Processor -- start 2024-04-17T06:02:42.994 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.992+0000 7f771d1cb740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f13f1f0c0 0x555f13f35090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:02:42.994 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.992+0000 7f771d1cb740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555f13f1f9e0 0x555f13f30400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:02:42.994 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.992+0000 7f771d1cb740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555f13f287b0 0x555f13f30a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:02:42.994 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.992+0000 7f771d1cb740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f77141a9f60 con 0x555f13f1f0c0 2024-04-17T06:02:42.994 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.992+0000 7f771d1cb740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f77141ab3d0 con 0x555f13f1f9e0 2024-04-17T06:02:42.994 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.992+0000 7f771d1cb740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f77141aa630 con 0x555f13f287b0 2024-04-17T06:02:42.994 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.992+0000 7f771d1cb740 10 client.0 _ll_register_callbacks cb 0x7f7708009340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:02:42.994 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.992+0000 7f7718b85640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f13f1f0c0 0x555f13f35090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:02:42.994 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.992+0000 7f7718b85640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f13f1f0c0 0x555f13f35090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37428/0 (socket says 172.21.15.47:37428) 2024-04-17T06:02:42.994 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.992+0000 7f7718b85640 1 -- 172.21.15.47:0/3216635433 learned_addr learned my addr 172.21.15.47:0/3216635433 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:02:42.995 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.992+0000 7f77137fe640 1 --2- 172.21.15.47:0/3216635433 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555f13f1f9e0 0x555f13f30400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:02:42.995 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.993+0000 7f7718b85640 1 --2- 172.21.15.47:0/3216635433 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555f13f287b0 0x555f13f30a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:02:42.995 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.993+0000 7f7718b85640 1 -- 172.21.15.47:0/3216635433 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555f13f287b0 msgr2=0x555f13f30a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:02:42.995 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.993+0000 7f7718b85640 1 --2- 172.21.15.47:0/3216635433 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555f13f287b0 0x555f13f30a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:42.995 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.993+0000 7f7718b85640 1 -- 172.21.15.47:0/3216635433 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555f13f1f9e0 msgr2=0x555f13f30400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:02:42.995 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.993+0000 7f7718b85640 1 --2- 172.21.15.47:0/3216635433 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555f13f1f9e0 0x555f13f30400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:42.995 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.993+0000 7f7718b85640 1 -- 172.21.15.47:0/3216635433 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f13f311e0 con 0x555f13f1f0c0 2024-04-17T06:02:42.995 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.993+0000 7f77137fe640 1 --2- 172.21.15.47:0/3216635433 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555f13f1f9e0 0x555f13f30400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:02:42.995 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.993+0000 7f7718b85640 1 --2- 172.21.15.47:0/3216635433 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f13f1f0c0 0x555f13f35090 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f770c003a50 tx=0x7f770c01a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88e1f5fd3233ea15 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:02:42.995 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.993+0000 7f76f37fe640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-17T06:02:42.995 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.993+0000 7f76f37fe640 1 -- 172.21.15.47:0/3216635433 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f770c01e040 con 0x555f13f1f0c0 2024-04-17T06:02:42.995 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.993+0000 7f76f37fe640 1 -- 172.21.15.47:0/3216635433 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8469+0+0 (secure 0 0 0) 0x7f770c002030 con 0x555f13f1f0c0 2024-04-17T06:02:42.995 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.993+0000 7f771d1cb740 1 -- 172.21.15.47:0/3216635433 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f770c004cc0 con 0x555f13f1f0c0 2024-04-17T06:02:42.996 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.993+0000 7f76f37fe640 1 -- 172.21.15.47:0/3216635433 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f770c029040 con 0x555f13f1f0c0 2024-04-17T06:02:42.996 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.993+0000 7f771d1cb740 10 client.15531 fetch_fsmap learned FSMap version 24 2024-04-17T06:02:42.996 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.993+0000 7f771d1cb740 1 -- 172.21.15.47:0/3216635433 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x555f13fa3180 con 0x555f13f1f0c0 2024-04-17T06:02:42.996 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.994+0000 7f76f37fe640 1 -- 172.21.15.47:0/3216635433 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f770c004cc0 con 0x555f13f1f0c0 2024-04-17T06:02:42.996 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.994+0000 7f76f37fe640 1 -- 172.21.15.47:0/3216635433 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 24) ==== 8339+0+0 (secure 0 0 0) 0x7f770c02eb20 con 0x555f13f1f0c0 2024-04-17T06:02:42.996 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.994+0000 7f771d1cb740 10 client.15531 fetch_fsmap finished waiting for FSMap version 24 2024-04-17T06:02:42.996 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.994+0000 7f771d1cb740 10 client.15531 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-17T06:02:42.996 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.994+0000 7f771d1cb740 20 client.15531 populate_metadata read hostname 'smithi047' 2024-04-17T06:02:42.996 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.994+0000 7f771d1cb740 1 --2- 172.21.15.47:0/3216635433 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x555f13fa3760 0x555f13fa5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:02:42.996 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.994+0000 7f771d1cb740 4 client.15531 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-17T06:02:42.996 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.994+0000 7f771d1cb740 1 -- 172.21.15.47:0/3216635433 --> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555f13e8c060 con 0x555f13fa3760 2024-04-17T06:02:42.996 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.994+0000 7f77137fe640 1 --2- 172.21.15.47:0/3216635433 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x555f13fa3760 0x555f13fa5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:02:42.996 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.995+0000 7f77137fe640 1 --2- 172.21.15.47:0/3216635433 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x555f13fa3760 0x555f13fa5b80 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57efb3401404f14c server_cookie=762e5e1fe39356b8 in_seq=0 out_seq=0 2024-04-17T06:02:42.996 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.995+0000 7f76f37fe640 10 client.15531 ms_handle_connect on v2:172.21.15.98:6838/2841287917 2024-04-17T06:02:42.998 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.998+0000 7f76f37fe640 1 -- 172.21.15.47:0/3216635433 <== mds.0 v2:172.21.15.98:6838/2841287917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555f13e8c060 con 0x555f13fa3760 2024-04-17T06:02:42.998 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.998+0000 7f76f37fe640 10 client.15531 handle_command_reply: tid=0 2024-04-17T06:02:42.998 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.998+0000 7f771d1cb740 1 client.15531 shutdown 2024-04-17T06:02:42.998 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.998+0000 7f771d1cb740 1 -- 172.21.15.47:0/3216635433 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f13f1f0c0 msgr2=0x555f13f35090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:02:42.998 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.998+0000 7f771d1cb740 1 --2- 172.21.15.47:0/3216635433 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f13f1f0c0 0x555f13f35090 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f770c003a50 tx=0x7f770c01a6a0 comp rx=0 tx=0).stop 2024-04-17T06:02:42.998 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.998+0000 7f771d1cb740 1 -- 172.21.15.47:0/3216635433 shutdown_connections 2024-04-17T06:02:42.998 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.998+0000 7f771d1cb740 1 --2- 172.21.15.47:0/3216635433 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x555f13fa3760 0x555f13fa5b80 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:42.998 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.998+0000 7f771d1cb740 1 --2- 172.21.15.47:0/3216635433 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555f13f287b0 0x555f13f30a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:42.998 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.998+0000 7f771d1cb740 1 --2- 172.21.15.47:0/3216635433 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555f13f1f9e0 0x555f13f30400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:42.998 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.998+0000 7f771d1cb740 1 --2- 172.21.15.47:0/3216635433 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f13f1f0c0 0x555f13f35090 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:42.998 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.998+0000 7f771d1cb740 1 -- 172.21.15.47:0/3216635433 >> 172.21.15.47:0/3216635433 conn(0x555f13f199e0 msgr2=0x555f13f25790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:02:42.999 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.998+0000 7f76f37fe640 0 client.15531 ms_handle_reset on v2:172.21.15.98:6838/2841287917 2024-04-17T06:02:42.999 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.999+0000 7f771d1cb740 1 -- 172.21.15.47:0/3216635433 shutdown_connections 2024-04-17T06:02:42.999 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.999+0000 7f771d1cb740 1 -- 172.21.15.47:0/3216635433 wait complete. 2024-04-17T06:02:42.999 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.999+0000 7f771d1cb740 20 client.15531 trim_cache size 0 max 16384 2024-04-17T06:02:42.999 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-04-17T06:02:42.999 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-04-17T06:02:42.999 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-04-17T06:02:42.999 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-04-17T06:02:43.000 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.999+0000 7f771a5e8640 1 -- 172.21.15.47:0/2295258110 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f76fc0a3ff0 msgr2=0x7f76fc0a64b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:02:43.000 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.999+0000 7f771a5e8640 1 --2- 172.21.15.47:0/2295258110 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f76fc0a3ff0 0x7f76fc0a64b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f77040efc10 tx=0x7f770402bc30 comp rx=0 tx=0).stop 2024-04-17T06:02:43.000 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.999+0000 7f771a5e8640 1 -- 172.21.15.47:0/2295258110 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7714101100 msgr2=0x7f7714197740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:02:43.000 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.999+0000 7f771a5e8640 1 --2- 172.21.15.47:0/2295258110 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7714101100 0x7f7714197740 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f7704006010 tx=0x7f770402cc70 comp rx=0 tx=0).stop 2024-04-17T06:02:43.000 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.999+0000 7f771a5e8640 1 -- 172.21.15.47:0/2295258110 shutdown_connections 2024-04-17T06:02:43.000 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.999+0000 7f771a5e8640 1 --2- 172.21.15.47:0/2295258110 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f771419c000 0x7f771419c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:43.001 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.999+0000 7f771a5e8640 1 --2- 172.21.15.47:0/2295258110 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7714106180 0x7f7714197c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:43.001 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.999+0000 7f771a5e8640 1 --2- 172.21.15.47:0/2295258110 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f76fc0a3ff0 0x7f76fc0a64b0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:43.001 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.999+0000 7f771a5e8640 1 --2- 172.21.15.47:0/2295258110 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7714101100 0x7f7714197740 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:43.001 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:42.999+0000 7f771a5e8640 1 -- 172.21.15.47:0/2295258110 >> 172.21.15.47:0/2295258110 conn(0x7f7714077fe0 msgr2=0x7f77140fefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:02:43.001 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:43.000+0000 7f771a5e8640 1 -- 172.21.15.47:0/2295258110 shutdown_connections 2024-04-17T06:02:43.001 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:43.000+0000 7f771a5e8640 1 -- 172.21.15.47:0/2295258110 wait complete. 2024-04-17T06:02:44.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:02:43 smithi098 ceph-mon[36516]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-17T06:02:44.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:02:43 smithi047 ceph-mon[32223]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-17T06:02:44.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:02:43 smithi103 ceph-mon[36243]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-17T06:02:46.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:02:45 smithi098 ceph-mon[36516]: pgmap v47: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 511 B/s wr, 7 op/s 2024-04-17T06:02:46.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:02:45 smithi047 ceph-mon[32223]: pgmap v47: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 511 B/s wr, 7 op/s 2024-04-17T06:02:46.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:02:45 smithi103 ceph-mon[36243]: pgmap v47: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 511 B/s wr, 7 op/s 2024-04-17T06:02:48.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:02:47 smithi098 ceph-mon[36516]: pgmap v48: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-17T06:02:48.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:02:47 smithi047 ceph-mon[32223]: pgmap v48: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-17T06:02:48.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:02:47 smithi103 ceph-mon[36243]: pgmap v48: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-17T06:02:50.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:02:49 smithi098 ceph-mon[36516]: pgmap v49: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-17T06:02:50.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:02:49 smithi047 ceph-mon[32223]: pgmap v49: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-17T06:02:50.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:02:49 smithi103 ceph-mon[36243]: pgmap v49: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-17T06:02:50.369 INFO:tasks.workunit.client.0.smithi047.stderr:Updating files: 95% (11694/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-04-17T06:02:51.001 INFO:tasks.workunit.client.0.smithi047.stderr:Note: switching to 'a8c84f5997950fd7d008204839253291d52e6eda'. 2024-04-17T06:02:51.001 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-04-17T06:02:51.001 INFO:tasks.workunit.client.0.smithi047.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-17T06:02:51.001 INFO:tasks.workunit.client.0.smithi047.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-17T06:02:51.001 INFO:tasks.workunit.client.0.smithi047.stderr:state without impacting any branches by switching back to a branch. 2024-04-17T06:02:51.001 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-04-17T06:02:51.001 INFO:tasks.workunit.client.0.smithi047.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-17T06:02:51.001 INFO:tasks.workunit.client.0.smithi047.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-17T06:02:51.001 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-04-17T06:02:51.001 INFO:tasks.workunit.client.0.smithi047.stderr: git switch -c 2024-04-17T06:02:51.001 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-04-17T06:02:51.002 INFO:tasks.workunit.client.0.smithi047.stderr:Or undo this operation with: 2024-04-17T06:02:51.002 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-04-17T06:02:51.002 INFO:tasks.workunit.client.0.smithi047.stderr: git switch - 2024-04-17T06:02:51.002 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-04-17T06:02:51.002 INFO:tasks.workunit.client.0.smithi047.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-17T06:02:51.002 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-04-17T06:02:51.002 INFO:tasks.workunit.client.0.smithi047.stderr:HEAD is now at a8c84f59979 Merge PR #51659 into wip-rishabh-testing-20240416.193735 2024-04-17T06:02:51.007 DEBUG:teuthology.orchestra.run.smithi047:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-17T06:02:51.074 INFO:tasks.workunit.client.0.smithi047.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-17T06:02:51.077 INFO:tasks.workunit.client.0.smithi047.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-17T06:02:51.077 INFO:tasks.workunit.client.0.smithi047.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-17T06:02:51.884 INFO:tasks.workunit.client.0.smithi047.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-17T06:02:51.975 INFO:tasks.workunit.client.0.smithi047.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-17T06:02:52.017 INFO:tasks.workunit.client.0.smithi047.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-17T06:02:52.019 INFO:tasks.workunit.client.0.smithi047.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-17T06:02:52.019 INFO:tasks.workunit.client.0.smithi047.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-17T06:02:52.059 INFO:tasks.workunit.client.0.smithi047.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-17T06:02:52.062 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-17T06:02:52.062 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-17T06:02:52.119 INFO:tasks.workunit:Running workunits matching fs/test_o_trunc.sh on client.0... 2024-04-17T06:02:52.120 INFO:tasks.workunit:Running workunit fs/test_o_trunc.sh... 2024-04-17T06:02:52.120 DEBUG:teuthology.orchestra.run.smithi047:workunit test fs/test_o_trunc.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a8c84f5997950fd7d008204839253291d52e6eda TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/test_o_trunc.sh 2024-04-17T06:02:52.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:02:51 smithi098 ceph-mon[36516]: pgmap v50: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 938 B/s wr, 9 op/s 2024-04-17T06:02:52.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:02:51 smithi098 ceph-mon[36516]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-04-17T06:02:52.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:02:51 smithi047 ceph-mon[32223]: pgmap v50: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 938 B/s wr, 9 op/s 2024-04-17T06:02:52.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:02:51 smithi047 ceph-mon[32223]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-04-17T06:02:52.159 INFO:tasks.workunit.client.0.smithi047.stderr:++ dirname /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/test_o_trunc.sh 2024-04-17T06:02:52.160 INFO:tasks.workunit.client.0.smithi047.stderr:+ mydir=/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs 2024-04-17T06:02:52.160 INFO:tasks.workunit.client.0.smithi047.stderr:+ /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/test_o_trunc trunc.foo 600 2024-04-17T06:02:52.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:02:51 smithi103 ceph-mon[36243]: pgmap v50: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 938 B/s wr, 9 op/s 2024-04-17T06:02:52.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:02:51 smithi103 ceph-mon[36243]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-04-17T06:02:54.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:02:53 smithi098 ceph-mon[36516]: pgmap v51: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 938 B/s wr, 6 op/s 2024-04-17T06:02:54.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:02:53 smithi047 ceph-mon[32223]: pgmap v51: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 938 B/s wr, 6 op/s 2024-04-17T06:02:54.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:02:53 smithi103 ceph-mon[36243]: pgmap v51: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 938 B/s wr, 6 op/s 2024-04-17T06:02:55.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:02:54 smithi098 ceph-mon[36516]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-17T06:02:55.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:02:54 smithi047 ceph-mon[32223]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-17T06:02:55.185 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:02:54 smithi103 ceph-mon[36243]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-17T06:02:56.108 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.099+0000 7fbc7f972640 1 -- 172.21.15.47:0/2388653177 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbc781021e0 msgr2=0x7fbc781025c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:02:56.108 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.099+0000 7fbc7f972640 1 --2- 172.21.15.47:0/2388653177 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbc781021e0 0x7fbc781025c0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fbc74009900 tx=0x7fbc7402bc30 comp rx=0 tx=0).stop 2024-04-17T06:02:56.109 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.099+0000 7fbc7f972640 1 -- 172.21.15.47:0/2388653177 shutdown_connections 2024-04-17T06:02:56.109 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.099+0000 7fbc7f972640 1 --2- 172.21.15.47:0/2388653177 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbc781072b0 0x7fbc7810ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:56.109 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.099+0000 7fbc7f972640 1 --2- 172.21.15.47:0/2388653177 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbc78102b90 0x7fbc78106bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:56.109 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.099+0000 7fbc7f972640 1 --2- 172.21.15.47:0/2388653177 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbc781021e0 0x7fbc781025c0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:56.109 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.099+0000 7fbc7f972640 1 -- 172.21.15.47:0/2388653177 >> 172.21.15.47:0/2388653177 conn(0x7fbc780793f0 msgr2=0x7fbc78079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:02:56.109 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.099+0000 7fbc7f972640 1 -- 172.21.15.47:0/2388653177 shutdown_connections 2024-04-17T06:02:56.109 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.099+0000 7fbc7f972640 1 -- 172.21.15.47:0/2388653177 wait complete. 2024-04-17T06:02:56.109 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.100+0000 7fbc7f972640 1 Processor -- start 2024-04-17T06:02:56.109 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.100+0000 7fbc7f972640 1 -- start start 2024-04-17T06:02:56.109 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.100+0000 7fbc7f972640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbc781021e0 0x7fbc7819e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:02:56.109 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.100+0000 7fbc7f972640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbc781072b0 0x7fbc7819e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:02:56.109 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.100+0000 7fbc7f972640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbc78198590 0x7fbc78198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:02:56.109 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.100+0000 7fbc7f972640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fbc78112370 con 0x7fbc781072b0 2024-04-17T06:02:56.110 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.100+0000 7fbc7f972640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fbc781121f0 con 0x7fbc78198590 2024-04-17T06:02:56.110 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.100+0000 7fbc7f972640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fbc781124f0 con 0x7fbc781021e0 2024-04-17T06:02:56.110 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.100+0000 7fbc7cf0d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbc781072b0 0x7fbc7819e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:02:56.110 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.100+0000 7fbc7cf0d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbc781072b0 0x7fbc7819e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60210/0 (socket says 172.21.15.47:60210) 2024-04-17T06:02:56.110 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.100+0000 7fbc7cf0d640 1 -- 172.21.15.47:0/3953088376 learned_addr learned my addr 172.21.15.47:0/3953088376 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:02:56.110 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.100+0000 7fbc7d70e640 1 --2- 172.21.15.47:0/3953088376 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbc781021e0 0x7fbc7819e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:02:56.110 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.100+0000 7fbc7df0f640 1 --2- 172.21.15.47:0/3953088376 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbc78198590 0x7fbc78198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:02:56.110 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.101+0000 7fbc7cf0d640 1 -- 172.21.15.47:0/3953088376 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbc781021e0 msgr2=0x7fbc7819e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:02:56.110 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.101+0000 7fbc7cf0d640 1 --2- 172.21.15.47:0/3953088376 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbc781021e0 0x7fbc7819e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:56.110 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.101+0000 7fbc7cf0d640 1 -- 172.21.15.47:0/3953088376 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbc78198590 msgr2=0x7fbc78198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:02:56.110 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.101+0000 7fbc7cf0d640 1 --2- 172.21.15.47:0/3953088376 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbc78198590 0x7fbc78198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:56.111 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.101+0000 7fbc7cf0d640 1 -- 172.21.15.47:0/3953088376 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc781990b0 con 0x7fbc781072b0 2024-04-17T06:02:56.111 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.101+0000 7fbc7df0f640 1 --2- 172.21.15.47:0/3953088376 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbc78198590 0x7fbc78198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:02:56.111 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.101+0000 7fbc7cf0d640 1 --2- 172.21.15.47:0/3953088376 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbc781072b0 0x7fbc7819e920 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fbc6800d770 tx=0x7fbc6800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:02:56.111 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.101+0000 7fbc6e7fc640 1 -- 172.21.15.47:0/3953088376 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc68017070 con 0x7fbc781072b0 2024-04-17T06:02:56.111 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.101+0000 7fbc6e7fc640 1 -- 172.21.15.47:0/3953088376 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc68009c30 con 0x7fbc781072b0 2024-04-17T06:02:56.111 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.101+0000 7fbc7f972640 1 -- 172.21.15.47:0/3953088376 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc781993a0 con 0x7fbc781072b0 2024-04-17T06:02:56.111 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.101+0000 7fbc6e7fc640 1 -- 172.21.15.47:0/3953088376 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc6800f620 con 0x7fbc781072b0 2024-04-17T06:02:56.111 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.101+0000 7fbc7f972640 1 -- 172.21.15.47:0/3953088376 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc781a5640 con 0x7fbc781072b0 2024-04-17T06:02:56.111 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.103+0000 7fbc6e7fc640 1 -- 172.21.15.47:0/3953088376 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fbc6801a070 con 0x7fbc781072b0 2024-04-17T06:02:56.111 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.104+0000 7fbc82637740 1 Processor -- start 2024-04-17T06:02:56.111 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.104+0000 7fbc82637740 1 -- start start 2024-04-17T06:02:56.111 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.104+0000 7fbc82637740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a6345030c0 0x55a6345034a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:02:56.111 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.104+0000 7fbc82637740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a6345039e0 0x55a63450c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:02:56.111 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.104+0000 7fbc82637740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a63450c7b0 0x55a63450eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:02:56.111 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.104+0000 7fbc82637740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55a6344687d0 con 0x55a63450c7b0 2024-04-17T06:02:56.112 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.104+0000 7fbc82637740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55a634405f70 con 0x55a6345030c0 2024-04-17T06:02:56.112 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.104+0000 7fbc82637740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55a634464790 con 0x55a6345039e0 2024-04-17T06:02:56.112 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.104+0000 7fbc6e7fc640 1 --2- 172.21.15.47:0/3953088376 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fbc640a3e40 0x7fbc640a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:02:56.112 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.105+0000 7fbc7cf0d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a63450c7b0 0x55a63450eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:02:56.112 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.105+0000 7fbc7df0f640 1 --2- 172.21.15.47:0/3953088376 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fbc640a3e40 0x7fbc640a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:02:56.112 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.105+0000 7fbc7cf0d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a63450c7b0 0x55a63450eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60224/0 (socket says 172.21.15.47:60224) 2024-04-17T06:02:56.112 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.105+0000 7fbc7cf0d640 1 -- 172.21.15.47:0/1348858207 learned_addr learned my addr 172.21.15.47:0/1348858207 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:02:56.112 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.105+0000 7fbc7d70e640 1 --2- 172.21.15.47:0/1348858207 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a6345039e0 0x55a63450c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:02:56.112 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.105+0000 7fbc7df0f640 1 --2- 172.21.15.47:0/1348858207 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a6345030c0 0x55a6345034a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:02:56.112 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.105+0000 7fbc6e7fc640 1 -- 172.21.15.47:0/3953088376 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8469+0+0 (secure 0 0 0) 0x7fbc680d0270 con 0x7fbc781072b0 2024-04-17T06:02:56.112 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.105+0000 7fbc7cf0d640 1 -- 172.21.15.47:0/1348858207 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a6345039e0 msgr2=0x55a63450c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:02:56.112 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.105+0000 7fbc7cf0d640 1 --2- 172.21.15.47:0/1348858207 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a6345039e0 0x55a63450c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:56.112 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.105+0000 7fbc7cf0d640 1 -- 172.21.15.47:0/1348858207 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a6345030c0 msgr2=0x55a6345034a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:02:56.112 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.105+0000 7fbc7cf0d640 1 --2- 172.21.15.47:0/1348858207 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a6345030c0 0x55a6345034a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:56.113 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.105+0000 7fbc7cf0d640 1 -- 172.21.15.47:0/1348858207 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a63417b210 con 0x55a63450c7b0 2024-04-17T06:02:56.113 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.105+0000 7fbc7df0f640 1 --2- 172.21.15.47:0/3953088376 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fbc640a3e40 0x7fbc640a6300 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fbc70005fd0 tx=0x7fbc70009df0 comp rx=0 tx=0).ready entity=mgr.24923 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:02:56.113 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.105+0000 7fbc7cf0d640 1 --2- 172.21.15.47:0/1348858207 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a63450c7b0 0x55a63450eba0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fbc6809c1d0 tx=0x7fbc68010cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2736188984321fc4 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:02:56.113 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.105+0000 7fbc637fe640 1 -- 172.21.15.47:0/1348858207 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc68021070 con 0x55a63450c7b0 2024-04-17T06:02:56.113 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.105+0000 7fbc637fe640 1 -- 172.21.15.47:0/1348858207 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc68009de0 con 0x55a63450c7b0 2024-04-17T06:02:56.113 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.105+0000 7fbc637fe640 1 -- 172.21.15.47:0/1348858207 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc680988b0 con 0x55a63450c7b0 2024-04-17T06:02:56.113 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.106+0000 7fbc82637740 1 -- 172.21.15.47:0/1348858207 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a63450c7b0 msgr2=0x55a63450eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:02:56.113 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.106+0000 7fbc82637740 1 --2- 172.21.15.47:0/1348858207 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a63450c7b0 0x55a63450eba0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fbc6809c1d0 tx=0x7fbc68010cd0 comp rx=0 tx=0).stop 2024-04-17T06:02:56.113 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.106+0000 7fbc82637740 1 -- 172.21.15.47:0/1348858207 shutdown_connections 2024-04-17T06:02:56.113 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.106+0000 7fbc82637740 1 --2- 172.21.15.47:0/1348858207 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a63450c7b0 0x55a63450eba0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:56.113 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.106+0000 7fbc82637740 1 --2- 172.21.15.47:0/1348858207 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a6345039e0 0x55a63450c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:56.113 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.106+0000 7fbc82637740 1 --2- 172.21.15.47:0/1348858207 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a6345030c0 0x55a6345034a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:56.113 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.106+0000 7fbc82637740 1 -- 172.21.15.47:0/1348858207 >> 172.21.15.47:0/1348858207 conn(0x55a6344fd9e0 msgr2=0x55a6344fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:02:56.113 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.106+0000 7fbc82637740 1 -- 172.21.15.47:0/1348858207 shutdown_connections 2024-04-17T06:02:56.114 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.106+0000 7fbc82637740 1 -- 172.21.15.47:0/1348858207 wait complete. 2024-04-17T06:02:56.114 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.107+0000 7fbc82637740 1 -- start start 2024-04-17T06:02:56.114 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.107+0000 7fbc82637740 1 Processor -- start 2024-04-17T06:02:56.114 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.107+0000 7fbc82637740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a6345030c0 0x55a634509880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:02:56.114 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.107+0000 7fbc82637740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a6345039e0 0x55a634509dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:02:56.114 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.107+0000 7fbc82637740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a63450c7b0 0x55a63450a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:02:56.114 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.107+0000 7fbc82637740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55a63450f380 con 0x55a63450c7b0 2024-04-17T06:02:56.114 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.107+0000 7fbc82637740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55a6343f0e90 con 0x55a6345030c0 2024-04-17T06:02:56.114 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.107+0000 7fbc82637740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55a63450f500 con 0x55a6345039e0 2024-04-17T06:02:56.114 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.107+0000 7fbc82637740 10 client.0 _ll_register_callbacks cb 0x55a634406d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:02:56.114 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.107+0000 7fbc7df0f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a63450c7b0 0x55a63450a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:02:56.114 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.107+0000 7fbc7df0f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a63450c7b0 0x55a63450a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60240/0 (socket says 172.21.15.47:60240) 2024-04-17T06:02:56.114 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.107+0000 7fbc7df0f640 1 -- 172.21.15.47:0/3927390814 learned_addr learned my addr 172.21.15.47:0/3927390814 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:02:56.114 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.107+0000 7fbc7d70e640 1 --2- 172.21.15.47:0/3927390814 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a6345030c0 0x55a634509880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:02:56.115 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.108+0000 7fbc7cf0d640 1 --2- 172.21.15.47:0/3927390814 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a6345039e0 0x55a634509dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:02:56.115 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.108+0000 7fbc7df0f640 1 -- 172.21.15.47:0/3927390814 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a6345039e0 msgr2=0x55a634509dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:02:56.115 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.108+0000 7fbc7df0f640 1 --2- 172.21.15.47:0/3927390814 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a6345039e0 0x55a634509dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:56.115 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.108+0000 7fbc7df0f640 1 -- 172.21.15.47:0/3927390814 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a6345030c0 msgr2=0x55a634509880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:02:56.115 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.108+0000 7fbc7df0f640 1 --2- 172.21.15.47:0/3927390814 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a6345030c0 0x55a634509880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:56.115 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.108+0000 7fbc7df0f640 1 -- 172.21.15.47:0/3927390814 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a634247620 con 0x55a63450c7b0 2024-04-17T06:02:56.115 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.108+0000 7fbc7cf0d640 1 --2- 172.21.15.47:0/3927390814 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a6345039e0 0x55a634509dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:02:56.115 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.108+0000 7fbc7df0f640 1 --2- 172.21.15.47:0/3927390814 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a63450c7b0 0x55a63450a300 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fbc70014310 tx=0x7fbc7001b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c414f497e73b93e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:02:56.115 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.108+0000 7fbc60ff9640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-17T06:02:56.115 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.108+0000 7fbc60ff9640 1 -- 172.21.15.47:0/3927390814 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc70025030 con 0x55a63450c7b0 2024-04-17T06:02:56.115 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.108+0000 7fbc60ff9640 1 -- 172.21.15.47:0/3927390814 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8469+0+0 (secure 0 0 0) 0x7fbc70012b80 con 0x55a63450c7b0 2024-04-17T06:02:56.115 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.108+0000 7fbc82637740 1 -- 172.21.15.47:0/3927390814 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc68009de0 con 0x55a63450c7b0 2024-04-17T06:02:56.115 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.108+0000 7fbc60ff9640 1 -- 172.21.15.47:0/3927390814 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbc70014340 con 0x55a63450c7b0 2024-04-17T06:02:56.115 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.108+0000 7fbc82637740 10 client.15543 fetch_fsmap learned FSMap version 25 2024-04-17T06:02:56.116 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.108+0000 7fbc82637740 1 -- 172.21.15.47:0/3927390814 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a63451b670 con 0x55a63450c7b0 2024-04-17T06:02:56.116 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.109+0000 7fbc60ff9640 1 -- 172.21.15.47:0/3927390814 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc68009de0 con 0x55a63450c7b0 2024-04-17T06:02:56.116 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.109+0000 7fbc60ff9640 1 -- 172.21.15.47:0/3927390814 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 25) ==== 8343+0+0 (secure 0 0 0) 0x7fbc7000f050 con 0x55a63450c7b0 2024-04-17T06:02:56.116 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.109+0000 7fbc82637740 10 client.15543 fetch_fsmap finished waiting for FSMap version 25 2024-04-17T06:02:56.116 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.109+0000 7fbc82637740 10 client.15543 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-17T06:02:56.116 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.109+0000 7fbc82637740 20 client.15543 populate_metadata read hostname 'smithi047' 2024-04-17T06:02:56.116 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.109+0000 7fbc82637740 1 --2- 172.21.15.47:0/3927390814 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55a63451cad0 0x55a63451eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:02:56.116 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.109+0000 7fbc82637740 4 client.15543 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T06:02:56.116 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.109+0000 7fbc82637740 1 -- 172.21.15.47:0/3927390814 --> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a6344761f0 con 0x55a63451cad0 2024-04-17T06:02:56.116 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.109+0000 7fbc7d70e640 1 --2- 172.21.15.47:0/3927390814 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55a63451cad0 0x55a63451eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:02:56.116 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.109+0000 7fbc7d70e640 1 --2- 172.21.15.47:0/3927390814 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55a63451cad0 0x55a63451eec0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3efb6d5b5da432da server_cookie=d3a4556ac539db83 in_seq=0 out_seq=0 2024-04-17T06:02:56.116 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.109+0000 7fbc60ff9640 10 client.15543 ms_handle_connect on v2:172.21.15.98:6838/2841287917 2024-04-17T06:02:56.116 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.110+0000 7fbc60ff9640 1 -- 172.21.15.47:0/3927390814 <== mds.0 v2:172.21.15.98:6838/2841287917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a6344761f0 con 0x55a63451cad0 2024-04-17T06:02:56.116 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.110+0000 7fbc60ff9640 10 client.15543 handle_command_reply: tid=0 2024-04-17T06:02:56.116 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.110+0000 7fbc82637740 1 client.15543 shutdown 2024-04-17T06:02:56.117 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.110+0000 7fbc82637740 1 -- 172.21.15.47:0/3927390814 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a63450c7b0 msgr2=0x55a63450a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:02:56.117 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.110+0000 7fbc82637740 1 --2- 172.21.15.47:0/3927390814 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a63450c7b0 0x55a63450a300 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fbc70014310 tx=0x7fbc7001b000 comp rx=0 tx=0).stop 2024-04-17T06:02:56.117 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.110+0000 7fbc82637740 1 -- 172.21.15.47:0/3927390814 shutdown_connections 2024-04-17T06:02:56.117 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.110+0000 7fbc82637740 1 --2- 172.21.15.47:0/3927390814 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55a63451cad0 0x55a63451eec0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:56.117 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.110+0000 7fbc82637740 1 --2- 172.21.15.47:0/3927390814 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a63450c7b0 0x55a63450a300 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:56.117 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.110+0000 7fbc82637740 1 --2- 172.21.15.47:0/3927390814 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a6345039e0 0x55a634509dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:56.117 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.110+0000 7fbc82637740 1 --2- 172.21.15.47:0/3927390814 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a6345030c0 0x55a634509880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:56.117 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.110+0000 7fbc60ff9640 0 client.15543 ms_handle_reset on v2:172.21.15.98:6838/2841287917 2024-04-17T06:02:56.117 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.110+0000 7fbc82637740 1 -- 172.21.15.47:0/3927390814 >> 172.21.15.47:0/3927390814 conn(0x55a6344fd9e0 msgr2=0x55a6344ff730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:02:56.117 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.111+0000 7fbc82637740 1 -- 172.21.15.47:0/3927390814 shutdown_connections 2024-04-17T06:02:56.117 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.111+0000 7fbc82637740 1 -- 172.21.15.47:0/3927390814 wait complete. 2024-04-17T06:02:56.117 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.111+0000 7fbc82637740 20 client.15543 trim_cache size 0 max 16384 2024-04-17T06:02:56.125 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.126+0000 7fbc82637740 1 Processor -- start 2024-04-17T06:02:56.125 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.126+0000 7fbc82637740 1 -- start start 2024-04-17T06:02:56.125 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.126+0000 7fbc82637740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6344fd9e0 0x55a63450bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:02:56.126 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.126+0000 7fbc82637740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a6345030c0 0x55a634518240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:02:56.126 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.126+0000 7fbc82637740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a6345039e0 0x55a6344ff4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:02:56.126 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.126+0000 7fbc82637740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fbc781b04d0 con 0x55a6344fd9e0 2024-04-17T06:02:56.126 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.126+0000 7fbc82637740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fbc781add60 con 0x55a6345030c0 2024-04-17T06:02:56.126 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.126+0000 7fbc82637740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fbc781abbb0 con 0x55a6345039e0 2024-04-17T06:02:56.126 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.126+0000 7fbc7d70e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6344fd9e0 0x55a63450bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:02:56.126 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.126+0000 7fbc7d70e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6344fd9e0 0x55a63450bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60252/0 (socket says 172.21.15.47:60252) 2024-04-17T06:02:56.126 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.126+0000 7fbc7d70e640 1 -- 172.21.15.47:0/3908010720 learned_addr learned my addr 172.21.15.47:0/3908010720 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:02:56.126 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.126+0000 7fbc7cf0d640 1 --2- 172.21.15.47:0/3908010720 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a6345030c0 0x55a634518240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:02:56.126 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.126+0000 7fbc7d70e640 1 -- 172.21.15.47:0/3908010720 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a6345039e0 msgr2=0x55a6344ff4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:02:56.126 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.126+0000 7fbc7d70e640 1 --2- 172.21.15.47:0/3908010720 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a6345039e0 0x55a6344ff4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:56.126 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.126+0000 7fbc7d70e640 1 -- 172.21.15.47:0/3908010720 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a6345030c0 msgr2=0x55a634518240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:02:56.126 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.126+0000 7fbc7d70e640 1 --2- 172.21.15.47:0/3908010720 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a6345030c0 0x55a634518240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:56.126 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.126+0000 7fbc7d70e640 1 -- 172.21.15.47:0/3908010720 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a634502860 con 0x55a6344fd9e0 2024-04-17T06:02:56.127 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.126+0000 7fbc7cf0d640 1 --2- 172.21.15.47:0/3908010720 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a6345030c0 0x55a634518240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:02:56.127 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.127+0000 7fbc7d70e640 1 --2- 172.21.15.47:0/3908010720 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6344fd9e0 0x55a63450bfb0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fbc74006010 tx=0x7fbc7402e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5cee276cbb3d79d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:02:56.127 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.127+0000 7fbc62ffd640 1 -- 172.21.15.47:0/3908010720 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc7404d070 con 0x55a6344fd9e0 2024-04-17T06:02:56.127 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.127+0000 7fbc62ffd640 1 -- 172.21.15.47:0/3908010720 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc7402cc60 con 0x55a6344fd9e0 2024-04-17T06:02:56.127 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.127+0000 7fbc62ffd640 1 -- 172.21.15.47:0/3908010720 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc74004920 con 0x55a6344fd9e0 2024-04-17T06:02:56.127 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.127+0000 7fbc82637740 1 -- 172.21.15.47:0/3908010720 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6344fd9e0 msgr2=0x55a63450bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:02:56.127 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.127+0000 7fbc82637740 1 --2- 172.21.15.47:0/3908010720 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6344fd9e0 0x55a63450bfb0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fbc74006010 tx=0x7fbc7402e040 comp rx=0 tx=0).stop 2024-04-17T06:02:56.127 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.127+0000 7fbc82637740 1 -- 172.21.15.47:0/3908010720 shutdown_connections 2024-04-17T06:02:56.127 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.127+0000 7fbc82637740 1 --2- 172.21.15.47:0/3908010720 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a6345039e0 0x55a6344ff4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:56.127 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.127+0000 7fbc82637740 1 --2- 172.21.15.47:0/3908010720 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a6345030c0 0x55a634518240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:56.127 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.127+0000 7fbc82637740 1 --2- 172.21.15.47:0/3908010720 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6344fd9e0 0x55a63450bfb0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:56.127 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.127+0000 7fbc82637740 1 -- 172.21.15.47:0/3908010720 >> 172.21.15.47:0/3908010720 conn(0x55a634505250 msgr2=0x55a6344fede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:02:56.127 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.127+0000 7fbc82637740 1 -- 172.21.15.47:0/3908010720 shutdown_connections 2024-04-17T06:02:56.127 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.127+0000 7fbc82637740 1 -- 172.21.15.47:0/3908010720 wait complete. 2024-04-17T06:02:56.127 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.128+0000 7fbc82637740 1 -- start start 2024-04-17T06:02:56.128 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.128+0000 7fbc82637740 1 Processor -- start 2024-04-17T06:02:56.130 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.128+0000 7fbc82637740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6344fd9e0 0x55a634510170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:02:56.130 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.128+0000 7fbc82637740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a6345030c0 0x55a6345106b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:02:56.130 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.128+0000 7fbc82637740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a6345039e0 0x55a634510bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:02:56.130 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.128+0000 7fbc82637740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fbc781ac220 con 0x55a6344fd9e0 2024-04-17T06:02:56.131 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.128+0000 7fbc82637740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fbc781ad690 con 0x55a6345030c0 2024-04-17T06:02:56.131 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.128+0000 7fbc82637740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fbc781ac8f0 con 0x55a6345039e0 2024-04-17T06:02:56.131 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.128+0000 7fbc82637740 10 client.0 _ll_register_callbacks cb 0x7fbc70014550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:02:56.131 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.128+0000 7fbc7d70e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6344fd9e0 0x55a634510170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:02:56.131 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.128+0000 7fbc7d70e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6344fd9e0 0x55a634510170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60264/0 (socket says 172.21.15.47:60264) 2024-04-17T06:02:56.131 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.128+0000 7fbc7d70e640 1 -- 172.21.15.47:0/3163760233 learned_addr learned my addr 172.21.15.47:0/3163760233 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:02:56.131 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.128+0000 7fbc7cf0d640 1 --2- 172.21.15.47:0/3163760233 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a6345030c0 0x55a6345106b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:02:56.131 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.128+0000 7fbc7df0f640 1 --2- 172.21.15.47:0/3163760233 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a6345039e0 0x55a634510bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:02:56.131 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.129+0000 7fbc7d70e640 1 -- 172.21.15.47:0/3163760233 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a6345039e0 msgr2=0x55a634510bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:02:56.131 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.129+0000 7fbc7d70e640 1 --2- 172.21.15.47:0/3163760233 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a6345039e0 0x55a634510bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:56.131 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.129+0000 7fbc7d70e640 1 -- 172.21.15.47:0/3163760233 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a6345030c0 msgr2=0x55a6345106b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:02:56.131 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.129+0000 7fbc7d70e640 1 --2- 172.21.15.47:0/3163760233 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a6345030c0 0x55a6345106b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:56.131 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.129+0000 7fbc7d70e640 1 -- 172.21.15.47:0/3163760233 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a634582040 con 0x55a6344fd9e0 2024-04-17T06:02:56.131 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.129+0000 7fbc7d70e640 1 --2- 172.21.15.47:0/3163760233 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6344fd9e0 0x55a634510170 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fbc74009230 tx=0x7fbc74005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87b818e4eff6b7f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:02:56.131 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.129+0000 7fbc60ff9640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-17T06:02:56.132 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.129+0000 7fbc60ff9640 1 -- 172.21.15.47:0/3163760233 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc74056070 con 0x55a6344fd9e0 2024-04-17T06:02:56.132 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.129+0000 7fbc60ff9640 1 -- 172.21.15.47:0/3163760233 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8469+0+0 (secure 0 0 0) 0x7fbc74004240 con 0x55a6344fd9e0 2024-04-17T06:02:56.132 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.129+0000 7fbc82637740 1 -- 172.21.15.47:0/3163760233 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc7402cc60 con 0x55a6344fd9e0 2024-04-17T06:02:56.132 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.129+0000 7fbc60ff9640 1 -- 172.21.15.47:0/3163760233 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbc74062050 con 0x55a6344fd9e0 2024-04-17T06:02:56.132 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.129+0000 7fbc82637740 10 client.15549 fetch_fsmap learned FSMap version 25 2024-04-17T06:02:56.132 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.129+0000 7fbc82637740 1 -- 172.21.15.47:0/3163760233 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a6345832a0 con 0x55a6344fd9e0 2024-04-17T06:02:56.132 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.129+0000 7fbc60ff9640 1 -- 172.21.15.47:0/3163760233 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc7402cc60 con 0x55a6344fd9e0 2024-04-17T06:02:56.132 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.130+0000 7fbc60ff9640 1 -- 172.21.15.47:0/3163760233 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 25) ==== 8343+0+0 (secure 0 0 0) 0x7fbc740052a0 con 0x55a6344fd9e0 2024-04-17T06:02:56.132 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.130+0000 7fbc82637740 10 client.15549 fetch_fsmap finished waiting for FSMap version 25 2024-04-17T06:02:56.132 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.130+0000 7fbc82637740 10 client.15549 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-17T06:02:56.132 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.130+0000 7fbc82637740 20 client.15549 populate_metadata read hostname 'smithi047' 2024-04-17T06:02:56.132 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.130+0000 7fbc82637740 1 --2- 172.21.15.47:0/3163760233 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55a63450c7b0 0x55a634585a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:02:56.132 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.130+0000 7fbc82637740 4 client.15549 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-17T06:02:56.132 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.130+0000 7fbc82637740 1 -- 172.21.15.47:0/3163760233 --> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a634470060 con 0x55a63450c7b0 2024-04-17T06:02:56.132 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.130+0000 7fbc7cf0d640 1 --2- 172.21.15.47:0/3163760233 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55a63450c7b0 0x55a634585a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:02:56.133 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.130+0000 7fbc7cf0d640 1 --2- 172.21.15.47:0/3163760233 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55a63450c7b0 0x55a634585a00 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4fe0c1951a4594f server_cookie=2ae9ec2a1b7b020 in_seq=0 out_seq=0 2024-04-17T06:02:56.133 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.130+0000 7fbc60ff9640 10 client.15549 ms_handle_connect on v2:172.21.15.98:6838/2841287917 2024-04-17T06:02:56.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:02:55 smithi098 ceph-mon[36516]: pgmap v52: 97 pgs: 97 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.4 KiB/s wr, 10 op/s 2024-04-17T06:02:56.138 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.140+0000 7fbc60ff9640 1 -- 172.21.15.47:0/3163760233 <== mds.0 v2:172.21.15.98:6838/2841287917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a634470060 con 0x55a63450c7b0 2024-04-17T06:02:56.138 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.140+0000 7fbc60ff9640 10 client.15549 handle_command_reply: tid=0 2024-04-17T06:02:56.138 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.140+0000 7fbc82637740 1 client.15549 shutdown 2024-04-17T06:02:56.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:02:55 smithi047 ceph-mon[32223]: pgmap v52: 97 pgs: 97 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.4 KiB/s wr, 10 op/s 2024-04-17T06:02:56.140 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.141+0000 7fbc82637740 1 -- 172.21.15.47:0/3163760233 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6344fd9e0 msgr2=0x55a634510170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:02:56.140 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.141+0000 7fbc82637740 1 --2- 172.21.15.47:0/3163760233 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6344fd9e0 0x55a634510170 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fbc74009230 tx=0x7fbc74005c00 comp rx=0 tx=0).stop 2024-04-17T06:02:56.140 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.141+0000 7fbc82637740 1 -- 172.21.15.47:0/3163760233 shutdown_connections 2024-04-17T06:02:56.140 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.141+0000 7fbc82637740 1 --2- 172.21.15.47:0/3163760233 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55a63450c7b0 0x55a634585a00 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:56.140 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.141+0000 7fbc82637740 1 --2- 172.21.15.47:0/3163760233 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a6345039e0 0x55a634510bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:56.140 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.141+0000 7fbc82637740 1 --2- 172.21.15.47:0/3163760233 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a6345030c0 0x55a6345106b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:56.140 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.141+0000 7fbc60ff9640 0 client.15549 ms_handle_reset on v2:172.21.15.98:6838/2841287917 2024-04-17T06:02:56.140 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.141+0000 7fbc82637740 1 --2- 172.21.15.47:0/3163760233 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6344fd9e0 0x55a634510170 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:56.140 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.141+0000 7fbc82637740 1 -- 172.21.15.47:0/3163760233 >> 172.21.15.47:0/3163760233 conn(0x55a634505250 msgr2=0x55a6344fc270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:02:56.140 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.141+0000 7fbc82637740 1 -- 172.21.15.47:0/3163760233 shutdown_connections 2024-04-17T06:02:56.140 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.141+0000 7fbc82637740 1 -- 172.21.15.47:0/3163760233 wait complete. 2024-04-17T06:02:56.140 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.141+0000 7fbc82637740 20 client.15549 trim_cache size 0 max 16384 2024-04-17T06:02:56.141 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-04-17T06:02:56.141 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-04-17T06:02:56.141 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-04-17T06:02:56.141 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-04-17T06:02:56.142 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.142+0000 7fbc7f972640 1 -- 172.21.15.47:0/3953088376 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fbc640a3e40 msgr2=0x7fbc640a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:02:56.142 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.142+0000 7fbc7f972640 1 --2- 172.21.15.47:0/3953088376 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fbc640a3e40 0x7fbc640a6300 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fbc70005fd0 tx=0x7fbc70009df0 comp rx=0 tx=0).stop 2024-04-17T06:02:56.142 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.142+0000 7fbc7f972640 1 -- 172.21.15.47:0/3953088376 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbc781072b0 msgr2=0x7fbc7819e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:02:56.142 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.142+0000 7fbc7f972640 1 --2- 172.21.15.47:0/3953088376 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbc781072b0 0x7fbc7819e920 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fbc6800d770 tx=0x7fbc6800dc30 comp rx=0 tx=0).stop 2024-04-17T06:02:56.142 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.142+0000 7fbc7f972640 1 -- 172.21.15.47:0/3953088376 shutdown_connections 2024-04-17T06:02:56.142 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.142+0000 7fbc7f972640 1 --2- 172.21.15.47:0/3953088376 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbc78198590 0x7fbc78198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:56.142 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.142+0000 7fbc7f972640 1 --2- 172.21.15.47:0/3953088376 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fbc640a3e40 0x7fbc640a6300 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:56.142 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.142+0000 7fbc7f972640 1 --2- 172.21.15.47:0/3953088376 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbc781072b0 0x7fbc7819e920 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:56.142 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.142+0000 7fbc7f972640 1 --2- 172.21.15.47:0/3953088376 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbc781021e0 0x7fbc7819e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:02:56.142 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.142+0000 7fbc7f972640 1 -- 172.21.15.47:0/3953088376 >> 172.21.15.47:0/3953088376 conn(0x7fbc780793f0 msgr2=0x7fbc780ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:02:56.142 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.142+0000 7fbc7f972640 1 -- 172.21.15.47:0/3953088376 shutdown_connections 2024-04-17T06:02:56.142 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:02:56.142+0000 7fbc7f972640 1 -- 172.21.15.47:0/3953088376 wait complete. 2024-04-17T06:02:56.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:02:55 smithi103 ceph-mon[36243]: pgmap v52: 97 pgs: 97 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.4 KiB/s wr, 10 op/s 2024-04-17T06:02:58.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:02:57 smithi098 ceph-mon[36516]: pgmap v53: 97 pgs: 97 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 5.2 KiB/s wr, 8 op/s 2024-04-17T06:02:58.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:02:57 smithi047 ceph-mon[32223]: pgmap v53: 97 pgs: 97 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 5.2 KiB/s wr, 8 op/s 2024-04-17T06:02:58.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:02:57 smithi103 ceph-mon[36243]: pgmap v53: 97 pgs: 97 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 5.2 KiB/s wr, 8 op/s 2024-04-17T06:03:00.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:02:59 smithi098 ceph-mon[36516]: pgmap v54: 97 pgs: 97 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 5.2 KiB/s wr, 7 op/s 2024-04-17T06:03:00.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:02:59 smithi047 ceph-mon[32223]: pgmap v54: 97 pgs: 97 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 5.2 KiB/s wr, 7 op/s 2024-04-17T06:03:00.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:02:59 smithi103 ceph-mon[36243]: pgmap v54: 97 pgs: 97 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 5.2 KiB/s wr, 7 op/s 2024-04-17T06:03:02.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:03:01 smithi098 ceph-mon[36516]: pgmap v55: 97 pgs: 97 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-04-17T06:03:02.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:03:01 smithi047 ceph-mon[32223]: pgmap v55: 97 pgs: 97 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-04-17T06:03:02.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:03:01 smithi103 ceph-mon[36243]: pgmap v55: 97 pgs: 97 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-04-17T06:03:04.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:03:03 smithi098 ceph-mon[36516]: pgmap v56: 97 pgs: 97 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 10 KiB/s wr, 11 op/s 2024-04-17T06:03:04.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:03:03 smithi047 ceph-mon[32223]: pgmap v56: 97 pgs: 97 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 10 KiB/s wr, 11 op/s 2024-04-17T06:03:04.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:03:03 smithi103 ceph-mon[36243]: pgmap v56: 97 pgs: 97 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 10 KiB/s wr, 11 op/s 2024-04-17T06:03:06.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:03:05 smithi047 ceph-mon[32223]: pgmap v57: 97 pgs: 97 active+clean; 1.5 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 36 KiB/s wr, 31 op/s 2024-04-17T06:03:06.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:03:05 smithi103 ceph-mon[36243]: pgmap v57: 97 pgs: 97 active+clean; 1.5 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 36 KiB/s wr, 31 op/s 2024-04-17T06:03:06.382 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:03:05 smithi098 ceph-mon[36516]: pgmap v57: 97 pgs: 97 active+clean; 1.5 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 36 KiB/s wr, 31 op/s 2024-04-17T06:03:08.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:03:07 smithi047 ceph-mon[32223]: pgmap v58: 97 pgs: 97 active+clean; 1.5 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 31 KiB/s wr, 27 op/s 2024-04-17T06:03:08.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:03:07 smithi103 ceph-mon[36243]: pgmap v58: 97 pgs: 97 active+clean; 1.5 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 31 KiB/s wr, 27 op/s 2024-04-17T06:03:08.382 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:03:07 smithi098 ceph-mon[36516]: pgmap v58: 97 pgs: 97 active+clean; 1.5 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 31 KiB/s wr, 27 op/s 2024-04-17T06:03:09.240 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.240+0000 7f771be77640 1 -- 172.21.15.47:0/383535575 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7714101b70 msgr2=0x7f771410dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:03:09.240 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.240+0000 7f771be77640 1 --2- 172.21.15.47:0/383535575 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7714101b70 0x7f771410dc80 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f770c009ca0 tx=0x7f770c02c140 comp rx=0 tx=0).stop 2024-04-17T06:03:09.240 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.240+0000 7f771be77640 1 -- 172.21.15.47:0/383535575 shutdown_connections 2024-04-17T06:03:09.240 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.240+0000 7f771be77640 1 --2- 172.21.15.47:0/383535575 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7714101b70 0x7f771410dc80 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:09.240 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.240+0000 7f771be77640 1 --2- 172.21.15.47:0/383535575 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f77141011b0 0x7f7714101630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:09.240 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.240+0000 7f771be77640 1 --2- 172.21.15.47:0/383535575 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7714106fc0 0x7f77141073a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:09.240 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.240+0000 7f771be77640 1 -- 172.21.15.47:0/383535575 >> 172.21.15.47:0/383535575 conn(0x7f7714076c60 msgr2=0x7f7714077070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:03:09.240 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.241+0000 7f771be77640 1 -- 172.21.15.47:0/383535575 shutdown_connections 2024-04-17T06:03:09.240 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.241+0000 7f771be77640 1 -- 172.21.15.47:0/383535575 wait complete. 2024-04-17T06:03:09.242 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.241+0000 7f771be77640 1 Processor -- start 2024-04-17T06:03:09.242 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.241+0000 7f771be77640 1 -- start start 2024-04-17T06:03:09.243 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.242+0000 7f771be77640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f77141011b0 0x7f77141975a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:03:09.243 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.242+0000 7f771be77640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7714101b70 0x7f7714197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:03:09.243 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.242+0000 7f771be77640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7714106fc0 0x7f771419be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:03:09.243 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.242+0000 7f771be77640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f77141111e0 con 0x7f7714101b70 2024-04-17T06:03:09.243 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.242+0000 7f771be77640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f7714111060 con 0x7f77141011b0 2024-04-17T06:03:09.243 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.242+0000 7f771be77640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7714111360 con 0x7f7714106fc0 2024-04-17T06:03:09.243 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.242+0000 7f7719412640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7714101b70 0x7f7714197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:03:09.243 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.242+0000 7f7719412640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7714101b70 0x7f7714197ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55390/0 (socket says 172.21.15.47:55390) 2024-04-17T06:03:09.243 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.242+0000 7f7719412640 1 -- 172.21.15.47:0/3318795138 learned_addr learned my addr 172.21.15.47:0/3318795138 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:03:09.243 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.242+0000 7f771a414640 1 --2- 172.21.15.47:0/3318795138 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7714106fc0 0x7f771419be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:03:09.243 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.242+0000 7f7719c13640 1 --2- 172.21.15.47:0/3318795138 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f77141011b0 0x7f77141975a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:03:09.243 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.242+0000 7f7719412640 1 -- 172.21.15.47:0/3318795138 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7714106fc0 msgr2=0x7f771419be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:03:09.243 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.242+0000 7f7719412640 1 --2- 172.21.15.47:0/3318795138 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7714106fc0 0x7f771419be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:09.243 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.242+0000 7f7719412640 1 -- 172.21.15.47:0/3318795138 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f77141011b0 msgr2=0x7f77141975a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:03:09.244 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.242+0000 7f7719412640 1 --2- 172.21.15.47:0/3318795138 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f77141011b0 0x7f77141975a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:09.244 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.242+0000 7f7719412640 1 -- 172.21.15.47:0/3318795138 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f771419c590 con 0x7f7714101b70 2024-04-17T06:03:09.244 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.242+0000 7f7719412640 1 --2- 172.21.15.47:0/3318795138 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7714101b70 0x7f7714197ae0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f770400e9f0 tx=0x7f770400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:03:09.244 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.242+0000 7f770affd640 1 -- 172.21.15.47:0/3318795138 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f770400cd80 con 0x7f7714101b70 2024-04-17T06:03:09.244 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.242+0000 7f770affd640 1 -- 172.21.15.47:0/3318795138 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7704010910 con 0x7f7714101b70 2024-04-17T06:03:09.244 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.242+0000 7f771be77640 1 -- 172.21.15.47:0/3318795138 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f771419c880 con 0x7f7714101b70 2024-04-17T06:03:09.244 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.242+0000 7f770affd640 1 -- 172.21.15.47:0/3318795138 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7704018990 con 0x7f7714101b70 2024-04-17T06:03:09.244 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.243+0000 7f771be77640 1 -- 172.21.15.47:0/3318795138 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77141a4110 con 0x7f7714101b70 2024-04-17T06:03:09.246 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.245+0000 7f770affd640 1 -- 172.21.15.47:0/3318795138 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7704010ac0 con 0x7f7714101b70 2024-04-17T06:03:09.246 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.245+0000 7f771ea5c740 1 Processor -- start 2024-04-17T06:03:09.246 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.246+0000 7f771ea5c740 1 -- start start 2024-04-17T06:03:09.247 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.246+0000 7f771ea5c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55659663c0c0 0x55659663c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:03:09.247 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.246+0000 7f771ea5c740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55659663c9e0 0x556596645270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:03:09.247 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.246+0000 7f771ea5c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565966457b0 0x556596647ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:03:09.247 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.246+0000 7f771ea5c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5565965a17d0 con 0x55659663c0c0 2024-04-17T06:03:09.247 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.246+0000 7f771ea5c740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55659653ef70 con 0x55659663c9e0 2024-04-17T06:03:09.247 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.246+0000 7f771ea5c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55659659d790 con 0x5565966457b0 2024-04-17T06:03:09.247 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.246+0000 7f770affd640 1 --2- 172.21.15.47:0/3318795138 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f77000a3e40 0x7f77000a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:03:09.247 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.246+0000 7f771a414640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55659663c0c0 0x55659663c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:03:09.247 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.246+0000 7f771a414640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55659663c0c0 0x55659663c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55406/0 (socket says 172.21.15.47:55406) 2024-04-17T06:03:09.247 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.246+0000 7f771a414640 1 -- 172.21.15.47:0/1453970119 learned_addr learned my addr 172.21.15.47:0/1453970119 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:03:09.247 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.246+0000 7f771a414640 1 --2- 172.21.15.47:0/3318795138 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f77000a3e40 0x7f77000a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:03:09.247 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.247+0000 7f770affd640 1 -- 172.21.15.47:0/3318795138 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8469+0+0 (secure 0 0 0) 0x7f77040d0b00 con 0x7f7714101b70 2024-04-17T06:03:09.247 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.247+0000 7f7719412640 1 --2- 172.21.15.47:0/1453970119 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565966457b0 0x556596647ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:03:09.248 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.247+0000 7f7719c13640 1 --2- 172.21.15.47:0/1453970119 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55659663c9e0 0x556596645270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:03:09.248 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.247+0000 7f771a414640 1 -- 172.21.15.47:0/1453970119 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565966457b0 msgr2=0x556596647ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:03:09.248 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.247+0000 7f771a414640 1 --2- 172.21.15.47:0/1453970119 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565966457b0 0x556596647ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:09.248 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.247+0000 7f771a414640 1 -- 172.21.15.47:0/1453970119 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55659663c9e0 msgr2=0x556596645270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:03:09.248 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.247+0000 7f771a414640 1 --2- 172.21.15.47:0/1453970119 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55659663c9e0 0x556596645270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:09.248 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.247+0000 7f771a414640 1 -- 172.21.15.47:0/1453970119 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565962b4210 con 0x55659663c0c0 2024-04-17T06:03:09.248 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.247+0000 7f771a414640 1 --2- 172.21.15.47:0/1453970119 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55659663c0c0 0x55659663c4a0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f770c009a60 tx=0x7f770c0327a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59d160121ab09f94 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:03:09.248 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.247+0000 7f771a414640 1 --2- 172.21.15.47:0/3318795138 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f77000a3e40 0x7f77000a6300 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f770c037040 tx=0x7f770c044000 comp rx=0 tx=0).ready entity=mgr.24923 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:03:09.248 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.247+0000 7f76fffff640 1 -- 172.21.15.47:0/1453970119 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f770c039040 con 0x55659663c0c0 2024-04-17T06:03:09.248 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.247+0000 7f76fffff640 1 -- 172.21.15.47:0/1453970119 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f770c04f070 con 0x55659663c0c0 2024-04-17T06:03:09.248 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.247+0000 7f76fffff640 1 -- 172.21.15.47:0/1453970119 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f770c004740 con 0x55659663c0c0 2024-04-17T06:03:09.253 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.248+0000 7f771ea5c740 1 -- 172.21.15.47:0/1453970119 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55659663c0c0 msgr2=0x55659663c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:03:09.253 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.248+0000 7f771ea5c740 1 --2- 172.21.15.47:0/1453970119 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55659663c0c0 0x55659663c4a0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f770c009a60 tx=0x7f770c0327a0 comp rx=0 tx=0).stop 2024-04-17T06:03:09.253 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.248+0000 7f771ea5c740 1 -- 172.21.15.47:0/1453970119 shutdown_connections 2024-04-17T06:03:09.253 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.248+0000 7f771ea5c740 1 --2- 172.21.15.47:0/1453970119 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565966457b0 0x556596647ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:09.253 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.248+0000 7f771ea5c740 1 --2- 172.21.15.47:0/1453970119 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55659663c9e0 0x556596645270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:09.253 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.248+0000 7f771ea5c740 1 --2- 172.21.15.47:0/1453970119 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55659663c0c0 0x55659663c4a0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:09.253 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.248+0000 7f771ea5c740 1 -- 172.21.15.47:0/1453970119 >> 172.21.15.47:0/1453970119 conn(0x5565966369e0 msgr2=0x556596636dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:03:09.253 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.248+0000 7f771ea5c740 1 -- 172.21.15.47:0/1453970119 shutdown_connections 2024-04-17T06:03:09.253 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.248+0000 7f771ea5c740 1 -- 172.21.15.47:0/1453970119 wait complete. 2024-04-17T06:03:09.253 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.249+0000 7f771ea5c740 1 -- start start 2024-04-17T06:03:09.253 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.249+0000 7f771ea5c740 1 Processor -- start 2024-04-17T06:03:09.254 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.249+0000 7f771ea5c740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55659663c0c0 0x556596642880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:03:09.254 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.249+0000 7f771ea5c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55659663c9e0 0x556596642dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:03:09.254 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.249+0000 7f771ea5c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5565966457b0 0x556596643300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:03:09.254 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.249+0000 7f771ea5c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x556596648380 con 0x5565966457b0 2024-04-17T06:03:09.254 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.249+0000 7f771ea5c740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x556596529e90 con 0x55659663c0c0 2024-04-17T06:03:09.254 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.249+0000 7f771ea5c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556596648500 con 0x55659663c9e0 2024-04-17T06:03:09.254 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.249+0000 7f771ea5c740 10 client.0 _ll_register_callbacks cb 0x55659653fd40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:03:09.254 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.249+0000 7f771a414640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5565966457b0 0x556596643300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:03:09.255 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.249+0000 7f771a414640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5565966457b0 0x556596643300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55422/0 (socket says 172.21.15.47:55422) 2024-04-17T06:03:09.255 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.249+0000 7f771a414640 1 -- 172.21.15.47:0/1445025827 learned_addr learned my addr 172.21.15.47:0/1445025827 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:03:09.255 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.250+0000 7f7719c13640 1 --2- 172.21.15.47:0/1445025827 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55659663c0c0 0x556596642880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:03:09.255 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.250+0000 7f7719412640 1 --2- 172.21.15.47:0/1445025827 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55659663c9e0 0x556596642dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:03:09.255 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.250+0000 7f771a414640 1 -- 172.21.15.47:0/1445025827 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55659663c9e0 msgr2=0x556596642dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:03:09.255 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.250+0000 7f771a414640 1 --2- 172.21.15.47:0/1445025827 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55659663c9e0 0x556596642dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:09.255 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.250+0000 7f771a414640 1 -- 172.21.15.47:0/1445025827 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55659663c0c0 msgr2=0x556596642880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:03:09.255 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.250+0000 7f771a414640 1 --2- 172.21.15.47:0/1445025827 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55659663c0c0 0x556596642880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:09.255 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.250+0000 7f771a414640 1 -- 172.21.15.47:0/1445025827 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556596380620 con 0x5565966457b0 2024-04-17T06:03:09.255 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.250+0000 7f7719c13640 1 --2- 172.21.15.47:0/1445025827 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55659663c0c0 0x556596642880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:03:09.255 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.250+0000 7f771a414640 1 --2- 172.21.15.47:0/1445025827 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5565966457b0 0x556596643300 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f770c02c180 tx=0x7f770c004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=498a215b7100c8fb server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:03:09.255 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.250+0000 7f76fd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-17T06:03:09.255 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.250+0000 7f76fd7fa640 1 -- 172.21.15.47:0/1445025827 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f770c058060 con 0x5565966457b0 2024-04-17T06:03:09.255 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.250+0000 7f76fd7fa640 1 -- 172.21.15.47:0/1445025827 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8469+0+0 (secure 0 0 0) 0x7f770c004720 con 0x5565966457b0 2024-04-17T06:03:09.255 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.250+0000 7f771ea5c740 1 -- 172.21.15.47:0/1445025827 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f770c04f070 con 0x5565966457b0 2024-04-17T06:03:09.256 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.250+0000 7f76fd7fa640 1 -- 172.21.15.47:0/1445025827 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f770c039040 con 0x5565966457b0 2024-04-17T06:03:09.256 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.250+0000 7f771ea5c740 10 client.15561 fetch_fsmap learned FSMap version 25 2024-04-17T06:03:09.256 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.250+0000 7f771ea5c740 1 -- 172.21.15.47:0/1445025827 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556596648380 con 0x5565966457b0 2024-04-17T06:03:09.256 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.251+0000 7f76fd7fa640 1 -- 172.21.15.47:0/1445025827 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f770c04f070 con 0x5565966457b0 2024-04-17T06:03:09.256 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.251+0000 7f76fd7fa640 1 -- 172.21.15.47:0/1445025827 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 25) ==== 8343+0+0 (secure 0 0 0) 0x7f770c04a400 con 0x5565966457b0 2024-04-17T06:03:09.256 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.251+0000 7f771ea5c740 10 client.15561 fetch_fsmap finished waiting for FSMap version 25 2024-04-17T06:03:09.256 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.251+0000 7f771ea5c740 10 client.15561 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-17T06:03:09.256 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.251+0000 7f771ea5c740 20 client.15561 populate_metadata read hostname 'smithi047' 2024-04-17T06:03:09.256 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.251+0000 7f771ea5c740 1 --2- 172.21.15.47:0/1445025827 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x5565966557f0 0x556596657be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:03:09.256 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.251+0000 7f771ea5c740 4 client.15561 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T06:03:09.256 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.251+0000 7f771ea5c740 1 -- 172.21.15.47:0/1445025827 --> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5565965af1f0 con 0x5565966557f0 2024-04-17T06:03:09.256 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.251+0000 7f7719c13640 1 --2- 172.21.15.47:0/1445025827 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x5565966557f0 0x556596657be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:03:09.256 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.252+0000 7f7719c13640 1 --2- 172.21.15.47:0/1445025827 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x5565966557f0 0x556596657be0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f28ebfdaa9740f99 server_cookie=60ead30bade18f7b in_seq=0 out_seq=0 2024-04-17T06:03:09.256 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.252+0000 7f76fd7fa640 10 client.15561 ms_handle_connect on v2:172.21.15.98:6838/2841287917 2024-04-17T06:03:09.257 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.252+0000 7f76fd7fa640 1 -- 172.21.15.47:0/1445025827 <== mds.0 v2:172.21.15.98:6838/2841287917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5565965af1f0 con 0x5565966557f0 2024-04-17T06:03:09.257 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.252+0000 7f76fd7fa640 10 client.15561 handle_command_reply: tid=0 2024-04-17T06:03:09.257 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.252+0000 7f771ea5c740 1 client.15561 shutdown 2024-04-17T06:03:09.257 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.253+0000 7f771ea5c740 1 -- 172.21.15.47:0/1445025827 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5565966457b0 msgr2=0x556596643300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:03:09.257 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.253+0000 7f771ea5c740 1 --2- 172.21.15.47:0/1445025827 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5565966457b0 0x556596643300 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f770c02c180 tx=0x7f770c004530 comp rx=0 tx=0).stop 2024-04-17T06:03:09.257 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.253+0000 7f771ea5c740 1 -- 172.21.15.47:0/1445025827 shutdown_connections 2024-04-17T06:03:09.257 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.253+0000 7f771ea5c740 1 --2- 172.21.15.47:0/1445025827 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x5565966557f0 0x556596657be0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:09.257 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.253+0000 7f771ea5c740 1 --2- 172.21.15.47:0/1445025827 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5565966457b0 0x556596643300 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:09.257 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.253+0000 7f771ea5c740 1 --2- 172.21.15.47:0/1445025827 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55659663c9e0 0x556596642dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:09.257 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.253+0000 7f76fd7fa640 0 client.15561 ms_handle_reset on v2:172.21.15.98:6838/2841287917 2024-04-17T06:03:09.257 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.253+0000 7f771ea5c740 1 --2- 172.21.15.47:0/1445025827 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55659663c0c0 0x556596642880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:09.257 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.253+0000 7f771ea5c740 1 -- 172.21.15.47:0/1445025827 >> 172.21.15.47:0/1445025827 conn(0x5565966369e0 msgr2=0x556596638730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:03:09.257 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.253+0000 7f771ea5c740 1 -- 172.21.15.47:0/1445025827 shutdown_connections 2024-04-17T06:03:09.257 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.253+0000 7f771ea5c740 1 -- 172.21.15.47:0/1445025827 wait complete. 2024-04-17T06:03:09.258 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.253+0000 7f771ea5c740 20 client.15561 trim_cache size 0 max 16384 2024-04-17T06:03:09.269 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.268+0000 7f771ea5c740 1 Processor -- start 2024-04-17T06:03:09.269 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.268+0000 7f771ea5c740 1 -- start start 2024-04-17T06:03:09.269 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.268+0000 7f771ea5c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55659663c0c0 0x556596651240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:03:09.269 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.268+0000 7f771ea5c740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55659663c9e0 0x556596635980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:03:09.269 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.268+0000 7f771ea5c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565966457b0 0x556596635ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:03:09.269 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.268+0000 7f771ea5c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f77141ae140 con 0x55659663c0c0 2024-04-17T06:03:09.269 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.268+0000 7f771ea5c740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f77141ab9d0 con 0x55659663c9e0 2024-04-17T06:03:09.269 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.268+0000 7f771ea5c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f77141a9820 con 0x5565966457b0 2024-04-17T06:03:09.269 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.268+0000 7f7719c13640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55659663c0c0 0x556596651240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:03:09.269 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.269+0000 7f7719c13640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55659663c0c0 0x556596651240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55438/0 (socket says 172.21.15.47:55438) 2024-04-17T06:03:09.270 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.269+0000 7f7719c13640 1 -- 172.21.15.47:0/25780193 learned_addr learned my addr 172.21.15.47:0/25780193 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:03:09.270 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.269+0000 7f771a414640 1 --2- 172.21.15.47:0/25780193 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565966457b0 0x556596635ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:03:09.270 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.269+0000 7f7719c13640 1 -- 172.21.15.47:0/25780193 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565966457b0 msgr2=0x556596635ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:03:09.270 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.269+0000 7f7719c13640 1 --2- 172.21.15.47:0/25780193 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565966457b0 0x556596635ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:09.270 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.269+0000 7f7719412640 1 --2- 172.21.15.47:0/25780193 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55659663c9e0 0x556596635980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:03:09.270 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.269+0000 7f7719c13640 1 -- 172.21.15.47:0/25780193 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55659663c9e0 msgr2=0x556596635980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:03:09.270 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.269+0000 7f7719c13640 1 --2- 172.21.15.47:0/25780193 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55659663c9e0 0x556596635980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:09.270 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.269+0000 7f7719c13640 1 -- 172.21.15.47:0/25780193 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55659663b540 con 0x55659663c0c0 2024-04-17T06:03:09.270 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.269+0000 7f7719c13640 1 --2- 172.21.15.47:0/25780193 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55659663c0c0 0x556596651240 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f7710009fd0 tx=0x7f7710005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b66b2f7914496c48 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:03:09.270 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.269+0000 7f76ff7fe640 1 -- 172.21.15.47:0/25780193 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7710004050 con 0x55659663c0c0 2024-04-17T06:03:09.270 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.269+0000 7f76ff7fe640 1 -- 172.21.15.47:0/25780193 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7710005500 con 0x55659663c0c0 2024-04-17T06:03:09.270 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.269+0000 7f76ff7fe640 1 -- 172.21.15.47:0/25780193 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7710004dd0 con 0x55659663c0c0 2024-04-17T06:03:09.270 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.270+0000 7f771ea5c740 1 -- 172.21.15.47:0/25780193 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55659663c0c0 msgr2=0x556596651240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:03:09.271 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.270+0000 7f771ea5c740 1 --2- 172.21.15.47:0/25780193 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55659663c0c0 0x556596651240 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f7710009fd0 tx=0x7f7710005f90 comp rx=0 tx=0).stop 2024-04-17T06:03:09.271 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.270+0000 7f771ea5c740 1 -- 172.21.15.47:0/25780193 shutdown_connections 2024-04-17T06:03:09.271 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.270+0000 7f771ea5c740 1 --2- 172.21.15.47:0/25780193 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565966457b0 0x556596635ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:09.271 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.270+0000 7f771ea5c740 1 --2- 172.21.15.47:0/25780193 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55659663c9e0 0x556596635980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:09.271 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.270+0000 7f771ea5c740 1 --2- 172.21.15.47:0/25780193 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55659663c0c0 0x556596651240 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:09.271 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.270+0000 7f771ea5c740 1 -- 172.21.15.47:0/25780193 >> 172.21.15.47:0/25780193 conn(0x5565966369e0 msgr2=0x556596635270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:03:09.271 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.270+0000 7f771ea5c740 1 -- 172.21.15.47:0/25780193 shutdown_connections 2024-04-17T06:03:09.271 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.270+0000 7f771ea5c740 1 -- 172.21.15.47:0/25780193 wait complete. 2024-04-17T06:03:09.271 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.270+0000 7f771ea5c740 1 -- start start 2024-04-17T06:03:09.271 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.270+0000 7f771ea5c740 1 Processor -- start 2024-04-17T06:03:09.274 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.271+0000 7f771ea5c740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55659663c0c0 0x556596649500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:03:09.274 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.271+0000 7f771ea5c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55659663c9e0 0x556596649a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:03:09.274 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.271+0000 7f771ea5c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5565966457b0 0x5565966bac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:03:09.274 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.271+0000 7f771ea5c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f77141a9e90 con 0x5565966457b0 2024-04-17T06:03:09.274 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.271+0000 7f771ea5c740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f77141ab300 con 0x55659663c0c0 2024-04-17T06:03:09.274 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.271+0000 7f771ea5c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f77141aa560 con 0x55659663c9e0 2024-04-17T06:03:09.274 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.271+0000 7f771ea5c740 10 client.0 _ll_register_callbacks cb 0x7f7710009590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:03:09.274 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.271+0000 7f771a414640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5565966457b0 0x5565966bac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:03:09.274 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.271+0000 7f771a414640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5565966457b0 0x5565966bac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55446/0 (socket says 172.21.15.47:55446) 2024-04-17T06:03:09.274 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.271+0000 7f771a414640 1 -- 172.21.15.47:0/510171770 learned_addr learned my addr 172.21.15.47:0/510171770 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:03:09.275 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.271+0000 7f7719c13640 1 --2- 172.21.15.47:0/510171770 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55659663c0c0 0x556596649500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:03:09.275 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.271+0000 7f7719412640 1 --2- 172.21.15.47:0/510171770 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55659663c9e0 0x556596649a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:03:09.275 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.271+0000 7f771a414640 1 -- 172.21.15.47:0/510171770 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55659663c9e0 msgr2=0x556596649a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:03:09.275 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.271+0000 7f771a414640 1 --2- 172.21.15.47:0/510171770 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55659663c9e0 0x556596649a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:09.275 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.271+0000 7f771a414640 1 -- 172.21.15.47:0/510171770 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55659663c0c0 msgr2=0x556596649500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:03:09.275 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.271+0000 7f771a414640 1 --2- 172.21.15.47:0/510171770 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55659663c0c0 0x556596649500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:09.275 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.271+0000 7f771a414640 1 -- 172.21.15.47:0/510171770 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565966bb1a0 con 0x5565966457b0 2024-04-17T06:03:09.275 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.271+0000 7f771a414640 1 --2- 172.21.15.47:0/510171770 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5565966457b0 0x5565966bac60 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f770c002230 tx=0x7f770c004850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d902faeb41670b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:03:09.275 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.271+0000 7f76fd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-17T06:03:09.275 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.272+0000 7f76fd7fa640 1 -- 172.21.15.47:0/510171770 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f770c058070 con 0x5565966457b0 2024-04-17T06:03:09.275 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.272+0000 7f76fd7fa640 1 -- 172.21.15.47:0/510171770 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8469+0+0 (secure 0 0 0) 0x7f770c003e00 con 0x5565966457b0 2024-04-17T06:03:09.275 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.272+0000 7f771ea5c740 1 -- 172.21.15.47:0/510171770 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7710005500 con 0x5565966457b0 2024-04-17T06:03:09.275 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.272+0000 7f76fd7fa640 1 -- 172.21.15.47:0/510171770 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f770c0098e0 con 0x5565966457b0 2024-04-17T06:03:09.275 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.272+0000 7f771ea5c740 10 client.15567 fetch_fsmap learned FSMap version 25 2024-04-17T06:03:09.275 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.272+0000 7f771ea5c740 1 -- 172.21.15.47:0/510171770 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5565966bc2e0 con 0x5565966457b0 2024-04-17T06:03:09.276 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.272+0000 7f76fd7fa640 1 -- 172.21.15.47:0/510171770 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7710005500 con 0x5565966457b0 2024-04-17T06:03:09.276 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.272+0000 7f76fd7fa640 1 -- 172.21.15.47:0/510171770 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 25) ==== 8343+0+0 (secure 0 0 0) 0x7f770c04ac50 con 0x5565966457b0 2024-04-17T06:03:09.276 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.272+0000 7f771ea5c740 10 client.15567 fetch_fsmap finished waiting for FSMap version 25 2024-04-17T06:03:09.276 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.272+0000 7f771ea5c740 10 client.15567 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-17T06:03:09.276 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.272+0000 7f771ea5c740 20 client.15567 populate_metadata read hostname 'smithi047' 2024-04-17T06:03:09.276 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.272+0000 7f771ea5c740 1 --2- 172.21.15.47:0/510171770 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x5565966bc940 0x5565966bed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:03:09.276 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.272+0000 7f771ea5c740 4 client.15567 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-17T06:03:09.276 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.272+0000 7f771ea5c740 1 -- 172.21.15.47:0/510171770 --> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5565965a9060 con 0x5565966bc940 2024-04-17T06:03:09.276 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.273+0000 7f7719c13640 1 --2- 172.21.15.47:0/510171770 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x5565966bc940 0x5565966bed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:03:09.276 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.273+0000 7f7719c13640 1 --2- 172.21.15.47:0/510171770 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x5565966bc940 0x5565966bed60 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb0cc2d59967a783 server_cookie=15177fc21f2b8b5d in_seq=0 out_seq=0 2024-04-17T06:03:09.276 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.273+0000 7f76fd7fa640 10 client.15567 ms_handle_connect on v2:172.21.15.98:6838/2841287917 2024-04-17T06:03:09.282 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.282+0000 7f76fd7fa640 1 -- 172.21.15.47:0/510171770 <== mds.0 v2:172.21.15.98:6838/2841287917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5565965a9060 con 0x5565966bc940 2024-04-17T06:03:09.282 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.282+0000 7f76fd7fa640 10 client.15567 handle_command_reply: tid=0 2024-04-17T06:03:09.282 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.282+0000 7f771ea5c740 1 client.15567 shutdown 2024-04-17T06:03:09.282 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.282+0000 7f771ea5c740 1 -- 172.21.15.47:0/510171770 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5565966457b0 msgr2=0x5565966bac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:03:09.282 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.282+0000 7f771ea5c740 1 --2- 172.21.15.47:0/510171770 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5565966457b0 0x5565966bac60 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f770c002230 tx=0x7f770c004850 comp rx=0 tx=0).stop 2024-04-17T06:03:09.282 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.283+0000 7f771ea5c740 1 -- 172.21.15.47:0/510171770 shutdown_connections 2024-04-17T06:03:09.282 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.283+0000 7f771ea5c740 1 --2- 172.21.15.47:0/510171770 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x5565966bc940 0x5565966bed60 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:09.282 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.283+0000 7f771ea5c740 1 --2- 172.21.15.47:0/510171770 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5565966457b0 0x5565966bac60 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:09.282 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.283+0000 7f771ea5c740 1 --2- 172.21.15.47:0/510171770 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55659663c9e0 0x556596649a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:09.282 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.283+0000 7f771ea5c740 1 --2- 172.21.15.47:0/510171770 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55659663c0c0 0x556596649500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:09.282 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.283+0000 7f76fd7fa640 0 client.15567 ms_handle_reset on v2:172.21.15.98:6838/2841287917 2024-04-17T06:03:09.283 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.283+0000 7f771ea5c740 1 -- 172.21.15.47:0/510171770 >> 172.21.15.47:0/510171770 conn(0x5565966369e0 msgr2=0x556596635b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:03:09.283 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.283+0000 7f771ea5c740 1 -- 172.21.15.47:0/510171770 shutdown_connections 2024-04-17T06:03:09.283 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.283+0000 7f771ea5c740 1 -- 172.21.15.47:0/510171770 wait complete. 2024-04-17T06:03:09.283 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.283+0000 7f771ea5c740 20 client.15567 trim_cache size 0 max 16384 2024-04-17T06:03:09.283 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-04-17T06:03:09.283 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-04-17T06:03:09.283 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-04-17T06:03:09.283 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-04-17T06:03:09.284 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.283+0000 7f771be77640 1 -- 172.21.15.47:0/3318795138 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f77000a3e40 msgr2=0x7f77000a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:03:09.284 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.283+0000 7f771be77640 1 --2- 172.21.15.47:0/3318795138 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f77000a3e40 0x7f77000a6300 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f770c037040 tx=0x7f770c044000 comp rx=0 tx=0).stop 2024-04-17T06:03:09.284 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.283+0000 7f771be77640 1 -- 172.21.15.47:0/3318795138 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7714101b70 msgr2=0x7f7714197ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:03:09.284 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.283+0000 7f771be77640 1 --2- 172.21.15.47:0/3318795138 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7714101b70 0x7f7714197ae0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f770400e9f0 tx=0x7f770400eeb0 comp rx=0 tx=0).stop 2024-04-17T06:03:09.284 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.283+0000 7f771be77640 1 -- 172.21.15.47:0/3318795138 shutdown_connections 2024-04-17T06:03:09.285 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.283+0000 7f771be77640 1 --2- 172.21.15.47:0/3318795138 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7714106fc0 0x7f771419be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:09.285 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.283+0000 7f771be77640 1 --2- 172.21.15.47:0/3318795138 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f77000a3e40 0x7f77000a6300 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:09.285 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.283+0000 7f771be77640 1 --2- 172.21.15.47:0/3318795138 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7714101b70 0x7f7714197ae0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:09.285 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.283+0000 7f771be77640 1 --2- 172.21.15.47:0/3318795138 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f77141011b0 0x7f77141975a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:09.285 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.283+0000 7f771be77640 1 -- 172.21.15.47:0/3318795138 >> 172.21.15.47:0/3318795138 conn(0x7f7714076c60 msgr2=0x7f77140ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:03:09.285 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.284+0000 7f771be77640 1 -- 172.21.15.47:0/3318795138 shutdown_connections 2024-04-17T06:03:09.285 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:09.284+0000 7f771be77640 1 -- 172.21.15.47:0/3318795138 wait complete. 2024-04-17T06:03:10.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:03:09 smithi103 ceph-mon[36243]: pgmap v59: 97 pgs: 97 active+clean; 1.5 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 31 KiB/s wr, 27 op/s 2024-04-17T06:03:10.382 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:03:09 smithi098 ceph-mon[36516]: pgmap v59: 97 pgs: 97 active+clean; 1.5 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 31 KiB/s wr, 27 op/s 2024-04-17T06:03:10.388 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:03:09 smithi047 ceph-mon[32223]: pgmap v59: 97 pgs: 97 active+clean; 1.5 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 31 KiB/s wr, 27 op/s 2024-04-17T06:03:12.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:03:11 smithi103 ceph-mon[36243]: pgmap v60: 97 pgs: 97 active+clean; 1.5 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 35 KiB/s wr, 33 op/s 2024-04-17T06:03:12.382 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:03:11 smithi098 ceph-mon[36516]: pgmap v60: 97 pgs: 97 active+clean; 1.5 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 35 KiB/s wr, 33 op/s 2024-04-17T06:03:12.388 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:03:11 smithi047 ceph-mon[32223]: pgmap v60: 97 pgs: 97 active+clean; 1.5 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 35 KiB/s wr, 33 op/s 2024-04-17T06:03:14.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:03:13 smithi103 ceph-mon[36243]: pgmap v61: 97 pgs: 97 active+clean; 1.5 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 30 KiB/s wr, 25 op/s 2024-04-17T06:03:14.740 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:03:13 smithi047 ceph-mon[32223]: pgmap v61: 97 pgs: 97 active+clean; 1.5 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 30 KiB/s wr, 25 op/s 2024-04-17T06:03:14.740 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:03:13 smithi098 ceph-mon[36516]: pgmap v61: 97 pgs: 97 active+clean; 1.5 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 30 KiB/s wr, 25 op/s 2024-04-17T06:03:16.191 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:03:15 smithi047 ceph-mon[32223]: pgmap v62: 97 pgs: 97 active+clean; 1.5 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 34 KiB/s wr, 31 op/s 2024-04-17T06:03:16.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:03:15 smithi103 ceph-mon[36243]: pgmap v62: 97 pgs: 97 active+clean; 1.5 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 34 KiB/s wr, 31 op/s 2024-04-17T06:03:16.382 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:03:15 smithi098 ceph-mon[36516]: pgmap v62: 97 pgs: 97 active+clean; 1.5 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 34 KiB/s wr, 31 op/s 2024-04-17T06:03:18.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:03:17 smithi103 ceph-mon[36243]: pgmap v63: 97 pgs: 97 active+clean; 1.5 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 7.9 KiB/s wr, 11 op/s 2024-04-17T06:03:18.382 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:03:17 smithi098 ceph-mon[36516]: pgmap v63: 97 pgs: 97 active+clean; 1.5 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 7.9 KiB/s wr, 11 op/s 2024-04-17T06:03:18.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:03:17 smithi047 ceph-mon[32223]: pgmap v63: 97 pgs: 97 active+clean; 1.5 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 7.9 KiB/s wr, 11 op/s 2024-04-17T06:03:20.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:03:19 smithi103 ceph-mon[36243]: pgmap v64: 97 pgs: 97 active+clean; 1.5 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 7.9 KiB/s wr, 11 op/s 2024-04-17T06:03:20.382 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:03:19 smithi098 ceph-mon[36516]: pgmap v64: 97 pgs: 97 active+clean; 1.5 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 7.9 KiB/s wr, 11 op/s 2024-04-17T06:03:20.388 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:03:19 smithi047 ceph-mon[32223]: pgmap v64: 97 pgs: 97 active+clean; 1.5 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 7.9 KiB/s wr, 11 op/s 2024-04-17T06:03:22.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:03:21 smithi103 ceph-mon[36243]: pgmap v65: 97 pgs: 97 active+clean; 1.5 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-04-17T06:03:22.384 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.384+0000 7fc91ed7c640 1 -- 172.21.15.47:0/269514144 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc918106fc0 msgr2=0x7fc9181073a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:03:22.385 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.384+0000 7fc91ed7c640 1 --2- 172.21.15.47:0/269514144 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc918106fc0 0x7fc9181073a0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fc900009900 tx=0x7fc90002bc80 comp rx=0 tx=0).stop 2024-04-17T06:03:22.385 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.384+0000 7fc91ed7c640 1 -- 172.21.15.47:0/269514144 shutdown_connections 2024-04-17T06:03:22.385 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.384+0000 7fc91ed7c640 1 --2- 172.21.15.47:0/269514144 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc918101b70 0x7fc91810dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:22.385 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.384+0000 7fc91ed7c640 1 --2- 172.21.15.47:0/269514144 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc9181011b0 0x7fc918101630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:22.385 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.384+0000 7fc91ed7c640 1 --2- 172.21.15.47:0/269514144 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc918106fc0 0x7fc9181073a0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:22.385 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.385+0000 7fc91ed7c640 1 -- 172.21.15.47:0/269514144 >> 172.21.15.47:0/269514144 conn(0x7fc918076c60 msgr2=0x7fc918077070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:03:22.385 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.385+0000 7fc91ed7c640 1 -- 172.21.15.47:0/269514144 shutdown_connections 2024-04-17T06:03:22.385 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.385+0000 7fc91ed7c640 1 -- 172.21.15.47:0/269514144 wait complete. 2024-04-17T06:03:22.385 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.385+0000 7fc91ed7c640 1 Processor -- start 2024-04-17T06:03:22.386 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.385+0000 7fc91ed7c640 1 -- start start 2024-04-17T06:03:22.386 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.385+0000 7fc91ed7c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc9181011b0 0x7fc918199510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:03:22.386 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.385+0000 7fc91ed7c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc918101b70 0x7fc918199a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:03:22.386 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.386+0000 7fc91ed7c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc918106fc0 0x7fc9181a4590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:03:22.386 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.386+0000 7fc91ed7c640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fc9181111f0 con 0x7fc918101b70 2024-04-17T06:03:22.386 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.386+0000 7fc91ed7c640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc918111070 con 0x7fc918106fc0 2024-04-17T06:03:22.386 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.386+0000 7fc91ed7c640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc918111370 con 0x7fc9181011b0 2024-04-17T06:03:22.386 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.386+0000 7fc917fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc918101b70 0x7fc918199a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:03:22.386 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.386+0000 7fc917fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc918101b70 0x7fc918199a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46410/0 (socket says 172.21.15.47:46410) 2024-04-17T06:03:22.386 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.386+0000 7fc917fff640 1 -- 172.21.15.47:0/400694794 learned_addr learned my addr 172.21.15.47:0/400694794 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:03:22.387 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.386+0000 7fc91cb18640 1 --2- 172.21.15.47:0/400694794 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc9181011b0 0x7fc918199510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:03:22.387 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.386+0000 7fc91d319640 1 --2- 172.21.15.47:0/400694794 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc918106fc0 0x7fc9181a4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:03:22.387 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.386+0000 7fc917fff640 1 -- 172.21.15.47:0/400694794 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc9181011b0 msgr2=0x7fc918199510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:03:22.387 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.386+0000 7fc917fff640 1 --2- 172.21.15.47:0/400694794 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc9181011b0 0x7fc918199510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:22.387 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.386+0000 7fc917fff640 1 -- 172.21.15.47:0/400694794 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc918106fc0 msgr2=0x7fc9181a4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:03:22.387 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.386+0000 7fc917fff640 1 --2- 172.21.15.47:0/400694794 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc918106fc0 0x7fc9181a4590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:22.387 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.386+0000 7fc917fff640 1 -- 172.21.15.47:0/400694794 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9181a4ad0 con 0x7fc918101b70 2024-04-17T06:03:22.387 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.386+0000 7fc917fff640 1 --2- 172.21.15.47:0/400694794 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc918101b70 0x7fc918199a50 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fc90800e9f0 tx=0x7fc90800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:03:22.387 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.386+0000 7fc915ffb640 1 -- 172.21.15.47:0/400694794 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc90800cd80 con 0x7fc918101b70 2024-04-17T06:03:22.387 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.386+0000 7fc915ffb640 1 -- 172.21.15.47:0/400694794 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc908010910 con 0x7fc918101b70 2024-04-17T06:03:22.387 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.386+0000 7fc91ed7c640 1 -- 172.21.15.47:0/400694794 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9181a4d00 con 0x7fc918101b70 2024-04-17T06:03:22.387 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.386+0000 7fc915ffb640 1 -- 172.21.15.47:0/400694794 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc908018990 con 0x7fc918101b70 2024-04-17T06:03:22.387 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.387+0000 7fc91ed7c640 1 -- 172.21.15.47:0/400694794 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9181a51d0 con 0x7fc918101b70 2024-04-17T06:03:22.388 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:03:21 smithi098 ceph-mon[36516]: pgmap v65: 97 pgs: 97 active+clean; 1.5 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-04-17T06:03:22.390 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.389+0000 7fc915ffb640 1 -- 172.21.15.47:0/400694794 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc908010ac0 con 0x7fc918101b70 2024-04-17T06:03:22.390 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.389+0000 7fc9215fd740 1 Processor -- start 2024-04-17T06:03:22.390 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.389+0000 7fc9215fd740 1 -- start start 2024-04-17T06:03:22.390 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.390+0000 7fc9215fd740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555c2a9b50c0 0x555c2a9b54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:03:22.390 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.390+0000 7fc9215fd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c2a9b59e0 0x555c2a9be270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:03:22.391 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.390+0000 7fc9215fd740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c2a9be7b0 0x555c2a9c0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:03:22.391 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.390+0000 7fc9215fd740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x555c2a91a7d0 con 0x555c2a9b50c0 2024-04-17T06:03:22.391 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.390+0000 7fc9215fd740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x555c2a8b7f70 con 0x555c2a9be7b0 2024-04-17T06:03:22.391 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.390+0000 7fc9215fd740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x555c2a916790 con 0x555c2a9b59e0 2024-04-17T06:03:22.391 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.390+0000 7fc91d319640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555c2a9b50c0 0x555c2a9b54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:03:22.391 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.390+0000 7fc915ffb640 1 --2- 172.21.15.47:0/400694794 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fc9040a3dc0 0x7fc9040a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:03:22.391 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.390+0000 7fc917fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c2a9be7b0 0x555c2a9c0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:03:22.391 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.390+0000 7fc91d319640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555c2a9b50c0 0x555c2a9b54a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46416/0 (socket says 172.21.15.47:46416) 2024-04-17T06:03:22.391 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.390+0000 7fc91d319640 1 -- 172.21.15.47:0/3232941083 learned_addr learned my addr 172.21.15.47:0/3232941083 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:03:22.391 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.390+0000 7fc91cb18640 1 --2- 172.21.15.47:0/3232941083 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c2a9b59e0 0x555c2a9be270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:03:22.391 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.390+0000 7fc91d319640 1 --2- 172.21.15.47:0/400694794 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fc9040a3dc0 0x7fc9040a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:03:22.391 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.390+0000 7fc915ffb640 1 -- 172.21.15.47:0/400694794 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8469+0+0 (secure 0 0 0) 0x7fc9080d0b50 con 0x7fc918101b70 2024-04-17T06:03:22.391 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.390+0000 7fc91d319640 1 -- 172.21.15.47:0/3232941083 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c2a9b59e0 msgr2=0x555c2a9be270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:03:22.391 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.390+0000 7fc91d319640 1 --2- 172.21.15.47:0/3232941083 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c2a9b59e0 0x555c2a9be270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:22.392 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.390+0000 7fc91d319640 1 -- 172.21.15.47:0/3232941083 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c2a9be7b0 msgr2=0x555c2a9c0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:03:22.392 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.390+0000 7fc91d319640 1 --2- 172.21.15.47:0/3232941083 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c2a9be7b0 0x555c2a9c0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:22.392 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.390+0000 7fc91d319640 1 -- 172.21.15.47:0/3232941083 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c2a62d210 con 0x555c2a9b50c0 2024-04-17T06:03:22.392 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.390+0000 7fc917fff640 1 --2- 172.21.15.47:0/3232941083 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c2a9be7b0 0x555c2a9c0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T06:03:22.392 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.391+0000 7fc91d319640 1 --2- 172.21.15.47:0/3232941083 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555c2a9b50c0 0x555c2a9b54a0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fc910013d90 tx=0x7fc9100117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28da43ecffda96b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:03:22.392 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:03:21 smithi047 ceph-mon[32223]: pgmap v65: 97 pgs: 97 active+clean; 1.5 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-04-17T06:03:22.397 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.391+0000 7fc91d319640 1 --2- 172.21.15.47:0/400694794 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fc9040a3dc0 0x7fc9040a6280 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc910011cb0 tx=0x7fc910018000 comp rx=0 tx=0).ready entity=mgr.24923 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:03:22.397 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.391+0000 7fc90effd640 1 -- 172.21.15.47:0/3232941083 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc910009e40 con 0x555c2a9b50c0 2024-04-17T06:03:22.398 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.391+0000 7fc90effd640 1 -- 172.21.15.47:0/3232941083 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc910003e00 con 0x555c2a9b50c0 2024-04-17T06:03:22.398 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.391+0000 7fc90effd640 1 -- 172.21.15.47:0/3232941083 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc910005280 con 0x555c2a9b50c0 2024-04-17T06:03:22.398 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.391+0000 7fc9215fd740 1 -- 172.21.15.47:0/3232941083 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555c2a9b50c0 msgr2=0x555c2a9b54a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:03:22.398 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.391+0000 7fc9215fd740 1 --2- 172.21.15.47:0/3232941083 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555c2a9b50c0 0x555c2a9b54a0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fc910013d90 tx=0x7fc9100117b0 comp rx=0 tx=0).stop 2024-04-17T06:03:22.398 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.392+0000 7fc9215fd740 1 -- 172.21.15.47:0/3232941083 shutdown_connections 2024-04-17T06:03:22.398 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.392+0000 7fc9215fd740 1 --2- 172.21.15.47:0/3232941083 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c2a9be7b0 0x555c2a9c0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:22.398 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.392+0000 7fc9215fd740 1 --2- 172.21.15.47:0/3232941083 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c2a9b59e0 0x555c2a9be270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:22.398 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.392+0000 7fc9215fd740 1 --2- 172.21.15.47:0/3232941083 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555c2a9b50c0 0x555c2a9b54a0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:22.398 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.392+0000 7fc9215fd740 1 -- 172.21.15.47:0/3232941083 >> 172.21.15.47:0/3232941083 conn(0x555c2a9af9e0 msgr2=0x555c2a9afdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:03:22.398 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.392+0000 7fc9215fd740 1 -- 172.21.15.47:0/3232941083 shutdown_connections 2024-04-17T06:03:22.398 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.392+0000 7fc9215fd740 1 -- 172.21.15.47:0/3232941083 wait complete. 2024-04-17T06:03:22.398 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.392+0000 7fc9215fd740 1 -- start start 2024-04-17T06:03:22.399 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.392+0000 7fc9215fd740 1 Processor -- start 2024-04-17T06:03:22.399 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.393+0000 7fc9215fd740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555c2a9b50c0 0x555c2a9bb870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:03:22.399 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.393+0000 7fc9215fd740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c2a9b59e0 0x555c2a9bbdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:03:22.399 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.393+0000 7fc9215fd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c2a9be7b0 0x555c2a9bc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:03:22.399 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.393+0000 7fc9215fd740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x555c2a9c1380 con 0x555c2a9b50c0 2024-04-17T06:03:22.399 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.393+0000 7fc9215fd740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x555c2a8a2e90 con 0x555c2a9b59e0 2024-04-17T06:03:22.399 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.393+0000 7fc9215fd740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x555c2a9c1500 con 0x555c2a9be7b0 2024-04-17T06:03:22.399 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.393+0000 7fc9215fd740 10 client.0 _ll_register_callbacks cb 0x555c2a8b8d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:03:22.399 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.393+0000 7fc91cb18640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555c2a9b50c0 0x555c2a9bb870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:03:22.399 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.393+0000 7fc91cb18640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555c2a9b50c0 0x555c2a9bb870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46422/0 (socket says 172.21.15.47:46422) 2024-04-17T06:03:22.399 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.393+0000 7fc91cb18640 1 -- 172.21.15.47:0/3534278916 learned_addr learned my addr 172.21.15.47:0/3534278916 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:03:22.399 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.393+0000 7fc917fff640 1 --2- 172.21.15.47:0/3534278916 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c2a9b59e0 0x555c2a9bbdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:03:22.399 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.393+0000 7fc91d319640 1 --2- 172.21.15.47:0/3534278916 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c2a9be7b0 0x555c2a9bc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:03:22.399 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.393+0000 7fc91cb18640 1 -- 172.21.15.47:0/3534278916 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c2a9be7b0 msgr2=0x555c2a9bc2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:03:22.400 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.393+0000 7fc91cb18640 1 --2- 172.21.15.47:0/3534278916 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c2a9be7b0 0x555c2a9bc2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:22.400 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.393+0000 7fc91cb18640 1 -- 172.21.15.47:0/3534278916 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c2a9b59e0 msgr2=0x555c2a9bbdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:03:22.400 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.393+0000 7fc91cb18640 1 --2- 172.21.15.47:0/3534278916 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c2a9b59e0 0x555c2a9bbdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:22.400 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.393+0000 7fc91cb18640 1 -- 172.21.15.47:0/3534278916 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c2a6f9620 con 0x555c2a9b50c0 2024-04-17T06:03:22.400 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.393+0000 7fc91d319640 1 --2- 172.21.15.47:0/3534278916 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c2a9be7b0 0x555c2a9bc2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:03:22.400 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.393+0000 7fc917fff640 1 --2- 172.21.15.47:0/3534278916 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c2a9b59e0 0x555c2a9bbdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T06:03:22.400 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.393+0000 7fc91cb18640 1 --2- 172.21.15.47:0/3534278916 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555c2a9b50c0 0x555c2a9bb870 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fc900009900 tx=0x7fc90002ce20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bf73b0298aafccd server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:03:22.400 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.393+0000 7fc8f7fff640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-17T06:03:22.400 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.393+0000 7fc8f7fff640 1 -- 172.21.15.47:0/3534278916 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc900004240 con 0x555c2a9b50c0 2024-04-17T06:03:22.400 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.393+0000 7fc8f7fff640 1 -- 172.21.15.47:0/3534278916 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8469+0+0 (secure 0 0 0) 0x7fc9000043e0 con 0x555c2a9b50c0 2024-04-17T06:03:22.400 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.394+0000 7fc9215fd740 1 -- 172.21.15.47:0/3534278916 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc910003e00 con 0x555c2a9b50c0 2024-04-17T06:03:22.400 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.394+0000 7fc8f7fff640 1 -- 172.21.15.47:0/3534278916 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc9000046c0 con 0x555c2a9b50c0 2024-04-17T06:03:22.400 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.394+0000 7fc9215fd740 10 client.15579 fetch_fsmap learned FSMap version 25 2024-04-17T06:03:22.400 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.394+0000 7fc9215fd740 1 -- 172.21.15.47:0/3534278916 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555c2a9cd6c0 con 0x555c2a9b50c0 2024-04-17T06:03:22.400 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.394+0000 7fc8f7fff640 1 -- 172.21.15.47:0/3534278916 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc910003e00 con 0x555c2a9b50c0 2024-04-17T06:03:22.401 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.394+0000 7fc8f7fff640 1 -- 172.21.15.47:0/3534278916 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 25) ==== 8343+0+0 (secure 0 0 0) 0x7fc9000035f0 con 0x555c2a9b50c0 2024-04-17T06:03:22.401 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.394+0000 7fc9215fd740 10 client.15579 fetch_fsmap finished waiting for FSMap version 25 2024-04-17T06:03:22.401 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.394+0000 7fc9215fd740 10 client.15579 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-17T06:03:22.401 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.394+0000 7fc9215fd740 20 client.15579 populate_metadata read hostname 'smithi047' 2024-04-17T06:03:22.401 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.394+0000 7fc9215fd740 1 --2- 172.21.15.47:0/3534278916 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x555c2a9cebd0 0x555c2a9d1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:03:22.401 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.394+0000 7fc9215fd740 4 client.15579 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T06:03:22.401 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.394+0000 7fc9215fd740 1 -- 172.21.15.47:0/3534278916 --> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555c2a9281f0 con 0x555c2a9cebd0 2024-04-17T06:03:22.401 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.394+0000 7fc917fff640 1 --2- 172.21.15.47:0/3534278916 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x555c2a9cebd0 0x555c2a9d1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:03:22.401 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.395+0000 7fc917fff640 1 --2- 172.21.15.47:0/3534278916 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x555c2a9cebd0 0x555c2a9d1060 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8cd11d43b695d9e server_cookie=bebb5dbf0324a70b in_seq=0 out_seq=0 2024-04-17T06:03:22.401 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.395+0000 7fc8f7fff640 10 client.15579 ms_handle_connect on v2:172.21.15.98:6838/2841287917 2024-04-17T06:03:22.401 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.396+0000 7fc8f7fff640 1 -- 172.21.15.47:0/3534278916 <== mds.0 v2:172.21.15.98:6838/2841287917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555c2a9281f0 con 0x555c2a9cebd0 2024-04-17T06:03:22.401 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.396+0000 7fc8f7fff640 10 client.15579 handle_command_reply: tid=0 2024-04-17T06:03:22.401 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.396+0000 7fc9215fd740 1 client.15579 shutdown 2024-04-17T06:03:22.401 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.396+0000 7fc9215fd740 1 -- 172.21.15.47:0/3534278916 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555c2a9b50c0 msgr2=0x555c2a9bb870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:03:22.402 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.396+0000 7fc9215fd740 1 --2- 172.21.15.47:0/3534278916 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555c2a9b50c0 0x555c2a9bb870 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fc900009900 tx=0x7fc90002ce20 comp rx=0 tx=0).stop 2024-04-17T06:03:22.402 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.396+0000 7fc9215fd740 1 -- 172.21.15.47:0/3534278916 shutdown_connections 2024-04-17T06:03:22.402 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.396+0000 7fc9215fd740 1 --2- 172.21.15.47:0/3534278916 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x555c2a9cebd0 0x555c2a9d1060 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:22.402 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.396+0000 7fc9215fd740 1 --2- 172.21.15.47:0/3534278916 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c2a9be7b0 0x555c2a9bc2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:22.402 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.396+0000 7fc9215fd740 1 --2- 172.21.15.47:0/3534278916 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c2a9b59e0 0x555c2a9bbdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:22.402 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.396+0000 7fc9215fd740 1 --2- 172.21.15.47:0/3534278916 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555c2a9b50c0 0x555c2a9bb870 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:22.402 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.396+0000 7fc9215fd740 1 -- 172.21.15.47:0/3534278916 >> 172.21.15.47:0/3534278916 conn(0x555c2a9af9e0 msgr2=0x555c2a9b1730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:03:22.402 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.396+0000 7fc8f7fff640 0 client.15579 ms_handle_reset on v2:172.21.15.98:6838/2841287917 2024-04-17T06:03:22.402 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.396+0000 7fc9215fd740 1 -- 172.21.15.47:0/3534278916 shutdown_connections 2024-04-17T06:03:22.402 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.396+0000 7fc9215fd740 1 -- 172.21.15.47:0/3534278916 wait complete. 2024-04-17T06:03:22.402 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.396+0000 7fc9215fd740 20 client.15579 trim_cache size 0 max 16384 2024-04-17T06:03:22.414 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.411+0000 7fc9215fd740 1 Processor -- start 2024-04-17T06:03:22.414 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.411+0000 7fc9215fd740 1 -- start start 2024-04-17T06:03:22.414 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.411+0000 7fc9215fd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c2a9b50c0 0x555c2a9bdfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:03:22.414 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.411+0000 7fc9215fd740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c2a9b59e0 0x555c2a9ca240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:03:22.414 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.411+0000 7fc9215fd740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555c2a9be7b0 0x555c2a9d6970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:03:22.414 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.411+0000 7fc9215fd740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fc9181b0430 con 0x555c2a9be7b0 2024-04-17T06:03:22.414 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.411+0000 7fc9215fd740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc9181adcc0 con 0x555c2a9b59e0 2024-04-17T06:03:22.414 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.411+0000 7fc9215fd740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc9181abb10 con 0x555c2a9b50c0 2024-04-17T06:03:22.415 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.411+0000 7fc91d319640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555c2a9be7b0 0x555c2a9d6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:03:22.415 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.411+0000 7fc91d319640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555c2a9be7b0 0x555c2a9d6970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46430/0 (socket says 172.21.15.47:46430) 2024-04-17T06:03:22.415 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.411+0000 7fc91d319640 1 -- 172.21.15.47:0/4281489059 learned_addr learned my addr 172.21.15.47:0/4281489059 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:03:22.415 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.412+0000 7fc917fff640 1 --2- 172.21.15.47:0/4281489059 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c2a9b59e0 0x555c2a9ca240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:03:22.415 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.412+0000 7fc91d319640 1 -- 172.21.15.47:0/4281489059 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c2a9b50c0 msgr2=0x555c2a9bdfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:03:22.415 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.412+0000 7fc91d319640 1 --2- 172.21.15.47:0/4281489059 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c2a9b50c0 0x555c2a9bdfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:22.415 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.412+0000 7fc91d319640 1 -- 172.21.15.47:0/4281489059 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c2a9b59e0 msgr2=0x555c2a9ca240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:03:22.415 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.412+0000 7fc91d319640 1 --2- 172.21.15.47:0/4281489059 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c2a9b59e0 0x555c2a9ca240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:22.415 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.412+0000 7fc91d319640 1 -- 172.21.15.47:0/4281489059 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c2a9b5810 con 0x555c2a9be7b0 2024-04-17T06:03:22.415 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.412+0000 7fc91d319640 1 --2- 172.21.15.47:0/4281489059 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555c2a9be7b0 0x555c2a9d6970 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fc910013680 tx=0x7fc9100136b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70e3acd7a343f8d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:03:22.415 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.412+0000 7fc90e7fc640 1 -- 172.21.15.47:0/4281489059 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc910035070 con 0x555c2a9be7b0 2024-04-17T06:03:22.415 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.412+0000 7fc90e7fc640 1 -- 172.21.15.47:0/4281489059 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9100138a0 con 0x555c2a9be7b0 2024-04-17T06:03:22.415 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.412+0000 7fc90e7fc640 1 -- 172.21.15.47:0/4281489059 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9100306e0 con 0x555c2a9be7b0 2024-04-17T06:03:22.416 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.413+0000 7fc9215fd740 1 -- 172.21.15.47:0/4281489059 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555c2a9be7b0 msgr2=0x555c2a9d6970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:03:22.416 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.413+0000 7fc9215fd740 1 --2- 172.21.15.47:0/4281489059 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555c2a9be7b0 0x555c2a9d6970 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fc910013680 tx=0x7fc9100136b0 comp rx=0 tx=0).stop 2024-04-17T06:03:22.416 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.413+0000 7fc9215fd740 1 -- 172.21.15.47:0/4281489059 shutdown_connections 2024-04-17T06:03:22.416 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.413+0000 7fc9215fd740 1 --2- 172.21.15.47:0/4281489059 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555c2a9be7b0 0x555c2a9d6970 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:22.416 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.413+0000 7fc9215fd740 1 --2- 172.21.15.47:0/4281489059 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c2a9b59e0 0x555c2a9ca240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:22.416 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.413+0000 7fc9215fd740 1 --2- 172.21.15.47:0/4281489059 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c2a9b50c0 0x555c2a9bdfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:22.416 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.413+0000 7fc9215fd740 1 -- 172.21.15.47:0/4281489059 >> 172.21.15.47:0/4281489059 conn(0x555c2a9af9e0 msgr2=0x555c2a9b6b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:03:22.416 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.413+0000 7fc9215fd740 1 -- 172.21.15.47:0/4281489059 shutdown_connections 2024-04-17T06:03:22.416 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.413+0000 7fc9215fd740 1 -- 172.21.15.47:0/4281489059 wait complete. 2024-04-17T06:03:22.416 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.413+0000 7fc9215fd740 1 -- start start 2024-04-17T06:03:22.416 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.413+0000 7fc9215fd740 1 Processor -- start 2024-04-17T06:03:22.416 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.413+0000 7fc9215fd740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555c2a9b50c0 0x555c2a9b7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:03:22.416 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.413+0000 7fc9215fd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c2a9b59e0 0x555c2a9b8210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:03:22.416 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.413+0000 7fc9215fd740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c2a9be7b0 0x555c2a9b8840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:03:22.416 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.413+0000 7fc9215fd740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fc9181ac180 con 0x555c2a9b50c0 2024-04-17T06:03:22.417 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.413+0000 7fc9215fd740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc9181ad5f0 con 0x555c2a9be7b0 2024-04-17T06:03:22.417 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.413+0000 7fc9215fd740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc9181ac850 con 0x555c2a9b59e0 2024-04-17T06:03:22.417 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.414+0000 7fc9215fd740 10 client.0 _ll_register_callbacks cb 0x7fc900009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:03:22.417 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.414+0000 7fc91cb18640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555c2a9b50c0 0x555c2a9b7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:03:22.417 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.414+0000 7fc91cb18640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555c2a9b50c0 0x555c2a9b7cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46444/0 (socket says 172.21.15.47:46444) 2024-04-17T06:03:22.417 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.414+0000 7fc91cb18640 1 -- 172.21.15.47:0/1836876013 learned_addr learned my addr 172.21.15.47:0/1836876013 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:03:22.417 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.414+0000 7fc917fff640 1 --2- 172.21.15.47:0/1836876013 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c2a9b59e0 0x555c2a9b8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:03:22.417 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.414+0000 7fc91d319640 1 --2- 172.21.15.47:0/1836876013 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c2a9be7b0 0x555c2a9b8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:03:22.417 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.414+0000 7fc91cb18640 1 -- 172.21.15.47:0/1836876013 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c2a9b59e0 msgr2=0x555c2a9b8210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:03:22.417 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.414+0000 7fc91cb18640 1 --2- 172.21.15.47:0/1836876013 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c2a9b59e0 0x555c2a9b8210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:22.417 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.414+0000 7fc91cb18640 1 -- 172.21.15.47:0/1836876013 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c2a9be7b0 msgr2=0x555c2a9b8840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:03:22.417 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.414+0000 7fc91cb18640 1 --2- 172.21.15.47:0/1836876013 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c2a9be7b0 0x555c2a9b8840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:22.417 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.414+0000 7fc91cb18640 1 -- 172.21.15.47:0/1836876013 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c2a9b9040 con 0x555c2a9b50c0 2024-04-17T06:03:22.417 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.414+0000 7fc91cb18640 1 --2- 172.21.15.47:0/1836876013 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555c2a9b50c0 0x555c2a9b7cd0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fc900003a90 tx=0x7fc900005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2847671a2e66c821 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:03:22.418 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.414+0000 7fc8f7fff640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-17T06:03:22.418 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.414+0000 7fc8f7fff640 1 -- 172.21.15.47:0/1836876013 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc900055040 con 0x555c2a9b50c0 2024-04-17T06:03:22.418 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.414+0000 7fc8f7fff640 1 -- 172.21.15.47:0/1836876013 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8469+0+0 (secure 0 0 0) 0x7fc9000035f0 con 0x555c2a9b50c0 2024-04-17T06:03:22.418 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.414+0000 7fc9215fd740 1 -- 172.21.15.47:0/1836876013 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9100138a0 con 0x555c2a9b50c0 2024-04-17T06:03:22.418 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.414+0000 7fc8f7fff640 1 -- 172.21.15.47:0/1836876013 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc9000037f0 con 0x555c2a9b50c0 2024-04-17T06:03:22.418 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.414+0000 7fc9215fd740 10 client.15585 fetch_fsmap learned FSMap version 25 2024-04-17T06:03:22.418 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.414+0000 7fc9215fd740 1 -- 172.21.15.47:0/1836876013 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555c2aa39a50 con 0x555c2a9b50c0 2024-04-17T06:03:22.418 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.415+0000 7fc8f7fff640 1 -- 172.21.15.47:0/1836876013 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9100138a0 con 0x555c2a9b50c0 2024-04-17T06:03:22.418 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.415+0000 7fc8f7fff640 1 -- 172.21.15.47:0/1836876013 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 25) ==== 8343+0+0 (secure 0 0 0) 0x7fc900003c80 con 0x555c2a9b50c0 2024-04-17T06:03:22.418 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.415+0000 7fc9215fd740 10 client.15585 fetch_fsmap finished waiting for FSMap version 25 2024-04-17T06:03:22.418 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.415+0000 7fc9215fd740 10 client.15585 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-17T06:03:22.418 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.415+0000 7fc9215fd740 20 client.15585 populate_metadata read hostname 'smithi047' 2024-04-17T06:03:22.418 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.415+0000 7fc9215fd740 1 --2- 172.21.15.47:0/1836876013 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x555c2aa3a1d0 0x555c2aa3c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:03:22.418 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.415+0000 7fc9215fd740 4 client.15585 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-17T06:03:22.419 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.415+0000 7fc9215fd740 1 -- 172.21.15.47:0/1836876013 --> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555c2a922060 con 0x555c2aa3a1d0 2024-04-17T06:03:22.419 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.415+0000 7fc917fff640 1 --2- 172.21.15.47:0/1836876013 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x555c2aa3a1d0 0x555c2aa3c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:03:22.419 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.415+0000 7fc917fff640 1 --2- 172.21.15.47:0/1836876013 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x555c2aa3a1d0 0x555c2aa3c5f0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9e82057e5a91ce9 server_cookie=c654b6072da58ef3 in_seq=0 out_seq=0 2024-04-17T06:03:22.419 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.415+0000 7fc8f7fff640 10 client.15585 ms_handle_connect on v2:172.21.15.98:6838/2841287917 2024-04-17T06:03:22.421 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.423+0000 7fc8f7fff640 1 -- 172.21.15.47:0/1836876013 <== mds.0 v2:172.21.15.98:6838/2841287917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555c2a922060 con 0x555c2aa3a1d0 2024-04-17T06:03:22.421 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.423+0000 7fc8f7fff640 10 client.15585 handle_command_reply: tid=0 2024-04-17T06:03:22.421 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.423+0000 7fc9215fd740 1 client.15585 shutdown 2024-04-17T06:03:22.421 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.423+0000 7fc9215fd740 1 -- 172.21.15.47:0/1836876013 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555c2a9b50c0 msgr2=0x555c2a9b7cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:03:22.421 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.423+0000 7fc9215fd740 1 --2- 172.21.15.47:0/1836876013 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555c2a9b50c0 0x555c2a9b7cd0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fc900003a90 tx=0x7fc900005fd0 comp rx=0 tx=0).stop 2024-04-17T06:03:22.421 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.423+0000 7fc9215fd740 1 -- 172.21.15.47:0/1836876013 shutdown_connections 2024-04-17T06:03:22.421 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.423+0000 7fc9215fd740 1 --2- 172.21.15.47:0/1836876013 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x555c2aa3a1d0 0x555c2aa3c5f0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:22.422 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.423+0000 7fc9215fd740 1 --2- 172.21.15.47:0/1836876013 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c2a9be7b0 0x555c2a9b8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:22.422 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.423+0000 7fc9215fd740 1 --2- 172.21.15.47:0/1836876013 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c2a9b59e0 0x555c2a9b8210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:22.422 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.423+0000 7fc8f7fff640 0 client.15585 ms_handle_reset on v2:172.21.15.98:6838/2841287917 2024-04-17T06:03:22.422 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.423+0000 7fc9215fd740 1 --2- 172.21.15.47:0/1836876013 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555c2a9b50c0 0x555c2a9b7cd0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:22.422 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.423+0000 7fc9215fd740 1 -- 172.21.15.47:0/1836876013 >> 172.21.15.47:0/1836876013 conn(0x555c2a9af9e0 msgr2=0x555c2a9bb810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:03:22.422 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.423+0000 7fc9215fd740 1 -- 172.21.15.47:0/1836876013 shutdown_connections 2024-04-17T06:03:22.422 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.423+0000 7fc9215fd740 1 -- 172.21.15.47:0/1836876013 wait complete. 2024-04-17T06:03:22.422 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.423+0000 7fc9215fd740 20 client.15585 trim_cache size 0 max 16384 2024-04-17T06:03:22.422 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-04-17T06:03:22.422 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-04-17T06:03:22.423 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-04-17T06:03:22.423 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-04-17T06:03:22.423 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.424+0000 7fc91ed7c640 1 -- 172.21.15.47:0/400694794 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fc9040a3dc0 msgr2=0x7fc9040a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:03:22.424 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.424+0000 7fc91ed7c640 1 --2- 172.21.15.47:0/400694794 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fc9040a3dc0 0x7fc9040a6280 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc910011cb0 tx=0x7fc910018000 comp rx=0 tx=0).stop 2024-04-17T06:03:22.424 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.424+0000 7fc91ed7c640 1 -- 172.21.15.47:0/400694794 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc918101b70 msgr2=0x7fc918199a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:03:22.424 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.424+0000 7fc91ed7c640 1 --2- 172.21.15.47:0/400694794 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc918101b70 0x7fc918199a50 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fc90800e9f0 tx=0x7fc90800eeb0 comp rx=0 tx=0).stop 2024-04-17T06:03:22.424 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.424+0000 7fc91ed7c640 1 -- 172.21.15.47:0/400694794 shutdown_connections 2024-04-17T06:03:22.424 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.424+0000 7fc91ed7c640 1 --2- 172.21.15.47:0/400694794 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc918106fc0 0x7fc9181a4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:22.424 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.424+0000 7fc91ed7c640 1 --2- 172.21.15.47:0/400694794 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fc9040a3dc0 0x7fc9040a6280 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:22.424 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.424+0000 7fc91ed7c640 1 --2- 172.21.15.47:0/400694794 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc918101b70 0x7fc918199a50 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:22.424 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.424+0000 7fc91ed7c640 1 --2- 172.21.15.47:0/400694794 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc9181011b0 0x7fc918199510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:22.424 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.424+0000 7fc91ed7c640 1 -- 172.21.15.47:0/400694794 >> 172.21.15.47:0/400694794 conn(0x7fc918076c60 msgr2=0x7fc9180ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:03:22.424 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.424+0000 7fc91ed7c640 1 -- 172.21.15.47:0/400694794 shutdown_connections 2024-04-17T06:03:22.424 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:22.424+0000 7fc91ed7c640 1 -- 172.21.15.47:0/400694794 wait complete. 2024-04-17T06:03:24.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:03:23 smithi103 ceph-mon[36243]: pgmap v66: 97 pgs: 97 active+clean; 1.5 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 6.8 KiB/s wr, 10 op/s 2024-04-17T06:03:24.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:03:23 smithi098 ceph-mon[36516]: pgmap v66: 97 pgs: 97 active+clean; 1.5 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 6.8 KiB/s wr, 10 op/s 2024-04-17T06:03:24.388 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:03:23 smithi047 ceph-mon[32223]: pgmap v66: 97 pgs: 97 active+clean; 1.5 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 6.8 KiB/s wr, 10 op/s 2024-04-17T06:03:26.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:03:25 smithi103 ceph-mon[36243]: pgmap v67: 97 pgs: 97 active+clean; 1.6 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-17T06:03:26.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:03:25 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T06:03:26.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:03:25 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:03:26.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:03:25 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:03:26.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:03:25 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:03:26.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:03:25 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:03:26.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:03:25 smithi098 ceph-mon[36516]: pgmap v67: 97 pgs: 97 active+clean; 1.6 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-17T06:03:26.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:03:25 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T06:03:26.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:03:25 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:03:26.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:03:25 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:03:26.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:03:25 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:03:26.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:03:25 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:03:26.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:03:25 smithi047 ceph-mon[32223]: pgmap v67: 97 pgs: 97 active+clean; 1.6 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-17T06:03:26.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:03:25 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T06:03:26.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:03:25 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:03:26.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:03:25 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:03:26.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:03:25 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:03:26.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:03:25 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:03:27.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:03:26 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T06:03:27.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:03:26 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T06:03:27.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:03:26 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:03:27.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:03:26 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T06:03:27.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:03:26 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T06:03:27.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:03:26 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:03:27.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:03:26 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T06:03:27.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:03:26 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T06:03:27.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:03:26 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:03:28.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:03:27 smithi103 ceph-mon[36243]: pgmap v68: 97 pgs: 97 active+clean; 1.6 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 9.0 KiB/s wr, 11 op/s 2024-04-17T06:03:28.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:03:27 smithi098 ceph-mon[36516]: pgmap v68: 97 pgs: 97 active+clean; 1.6 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 9.0 KiB/s wr, 11 op/s 2024-04-17T06:03:28.388 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:03:27 smithi047 ceph-mon[32223]: pgmap v68: 97 pgs: 97 active+clean; 1.6 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 9.0 KiB/s wr, 11 op/s 2024-04-17T06:03:30.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:03:29 smithi103 ceph-mon[36243]: pgmap v69: 97 pgs: 97 active+clean; 1.6 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 9.0 KiB/s wr, 11 op/s 2024-04-17T06:03:30.382 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:03:29 smithi098 ceph-mon[36516]: pgmap v69: 97 pgs: 97 active+clean; 1.6 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 9.0 KiB/s wr, 11 op/s 2024-04-17T06:03:30.388 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:03:29 smithi047 ceph-mon[32223]: pgmap v69: 97 pgs: 97 active+clean; 1.6 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 9.0 KiB/s wr, 11 op/s 2024-04-17T06:03:32.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:03:31 smithi103 ceph-mon[36243]: pgmap v70: 97 pgs: 97 active+clean; 1.6 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-17T06:03:32.382 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:03:31 smithi098 ceph-mon[36516]: pgmap v70: 97 pgs: 97 active+clean; 1.6 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-17T06:03:32.388 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:03:31 smithi047 ceph-mon[32223]: pgmap v70: 97 pgs: 97 active+clean; 1.6 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-17T06:03:34.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:03:34 smithi103 ceph-mon[36243]: pgmap v71: 97 pgs: 97 active+clean; 1.6 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 9.0 KiB/s wr, 11 op/s 2024-04-17T06:03:34.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:03:34 smithi098 ceph-mon[36516]: pgmap v71: 97 pgs: 97 active+clean; 1.6 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 9.0 KiB/s wr, 11 op/s 2024-04-17T06:03:34.388 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:03:34 smithi047 ceph-mon[32223]: pgmap v71: 97 pgs: 97 active+clean; 1.6 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 9.0 KiB/s wr, 11 op/s 2024-04-17T06:03:35.522 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.524+0000 7f05bd79d640 1 -- 172.21.15.47:0/3034759562 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f05b81072b0 msgr2=0x7f05b810ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:03:35.522 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.524+0000 7f05bd79d640 1 --2- 172.21.15.47:0/3034759562 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f05b81072b0 0x7f05b810ee80 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f05b0009900 tx=0x7f05b002bc30 comp rx=0 tx=0).stop 2024-04-17T06:03:35.522 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.524+0000 7f05bd79d640 1 -- 172.21.15.47:0/3034759562 shutdown_connections 2024-04-17T06:03:35.522 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.524+0000 7f05bd79d640 1 --2- 172.21.15.47:0/3034759562 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f05b81072b0 0x7f05b810ee80 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:35.522 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.524+0000 7f05bd79d640 1 --2- 172.21.15.47:0/3034759562 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f05b8102b90 0x7f05b8106bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:35.522 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.524+0000 7f05bd79d640 1 --2- 172.21.15.47:0/3034759562 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f05b81021e0 0x7f05b81025c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:35.523 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.524+0000 7f05bd79d640 1 -- 172.21.15.47:0/3034759562 >> 172.21.15.47:0/3034759562 conn(0x7f05b80793f0 msgr2=0x7f05b8079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:03:35.523 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.524+0000 7f05bd79d640 1 -- 172.21.15.47:0/3034759562 shutdown_connections 2024-04-17T06:03:35.523 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.524+0000 7f05bd79d640 1 -- 172.21.15.47:0/3034759562 wait complete. 2024-04-17T06:03:35.523 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.524+0000 7f05bd79d640 1 Processor -- start 2024-04-17T06:03:35.525 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.525+0000 7f05bd79d640 1 -- start start 2024-04-17T06:03:35.525 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.525+0000 7f05bd79d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f05b81021e0 0x7f05b8198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:03:35.525 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.525+0000 7f05bd79d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f05b81072b0 0x7f05b8198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:03:35.525 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.525+0000 7f05bd79d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f05b819d010 0x7f05b819d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:03:35.525 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.525+0000 7f05bd79d640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f05b8112350 con 0x7f05b819d010 2024-04-17T06:03:35.525 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.525+0000 7f05bd79d640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f05b81121d0 con 0x7f05b81072b0 2024-04-17T06:03:35.525 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.525+0000 7f05bd79d640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f05b81124d0 con 0x7f05b81021e0 2024-04-17T06:03:35.525 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.525+0000 7f05b77fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f05b819d010 0x7f05b819d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:03:35.525 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.525+0000 7f05b77fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f05b819d010 0x7f05b819d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41564/0 (socket says 172.21.15.47:41564) 2024-04-17T06:03:35.525 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.525+0000 7f05b77fe640 1 -- 172.21.15.47:0/1255473359 learned_addr learned my addr 172.21.15.47:0/1255473359 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:03:35.525 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.525+0000 7f05b67fc640 1 --2- 172.21.15.47:0/1255473359 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f05b81072b0 0x7f05b8198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:03:35.525 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.525+0000 7f05b6ffd640 1 --2- 172.21.15.47:0/1255473359 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f05b81021e0 0x7f05b8198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:03:35.525 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.525+0000 7f05b77fe640 1 -- 172.21.15.47:0/1255473359 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f05b81021e0 msgr2=0x7f05b8198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:03:35.526 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.525+0000 7f05b77fe640 1 --2- 172.21.15.47:0/1255473359 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f05b81021e0 0x7f05b8198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:35.526 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.525+0000 7f05b77fe640 1 -- 172.21.15.47:0/1255473359 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f05b81072b0 msgr2=0x7f05b8198c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:03:35.526 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.525+0000 7f05b77fe640 1 --2- 172.21.15.47:0/1255473359 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f05b81072b0 0x7f05b8198c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:35.526 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.525+0000 7f05b77fe640 1 -- 172.21.15.47:0/1255473359 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05b819db00 con 0x7f05b819d010 2024-04-17T06:03:35.526 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.526+0000 7f05b77fe640 1 --2- 172.21.15.47:0/1255473359 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f05b819d010 0x7f05b819d4c0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f05b0009230 tx=0x7f05b002cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:03:35.526 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.526+0000 7f05affff640 1 -- 172.21.15.47:0/1255473359 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b0041070 con 0x7f05b819d010 2024-04-17T06:03:35.526 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.526+0000 7f05affff640 1 -- 172.21.15.47:0/1255473359 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05b0004450 con 0x7f05b819d010 2024-04-17T06:03:35.526 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.526+0000 7f05bd79d640 1 -- 172.21.15.47:0/1255473359 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05b819dd90 con 0x7f05b819d010 2024-04-17T06:03:35.526 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.526+0000 7f05affff640 1 -- 172.21.15.47:0/1255473359 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b0003d30 con 0x7f05b819d010 2024-04-17T06:03:35.526 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.526+0000 7f05bd79d640 1 -- 172.21.15.47:0/1255473359 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05b81a5550 con 0x7f05b819d010 2024-04-17T06:03:35.531 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.528+0000 7f05affff640 1 -- 172.21.15.47:0/1255473359 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f05b0044070 con 0x7f05b819d010 2024-04-17T06:03:35.531 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.528+0000 7f05c0385740 1 Processor -- start 2024-04-17T06:03:35.531 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.528+0000 7f05c0385740 1 -- start start 2024-04-17T06:03:35.531 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.529+0000 7f05c0385740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d7d07650c0 0x55d7d07654a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:03:35.531 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.529+0000 7f05c0385740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7d07659e0 0x55d7d076e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:03:35.531 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.529+0000 7f05c0385740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d7d076e7b0 0x55d7d0770ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:03:35.532 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.529+0000 7f05c0385740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55d7d06ca7d0 con 0x55d7d07650c0 2024-04-17T06:03:35.532 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.529+0000 7f05c0385740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55d7d0667f70 con 0x55d7d076e7b0 2024-04-17T06:03:35.532 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.529+0000 7f05c0385740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d7d06c6790 con 0x55d7d07659e0 2024-04-17T06:03:35.532 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.529+0000 7f05b67fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d7d07650c0 0x55d7d07654a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:03:35.532 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.529+0000 7f05b67fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d7d07650c0 0x55d7d07654a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41574/0 (socket says 172.21.15.47:41574) 2024-04-17T06:03:35.532 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.529+0000 7f05b67fc640 1 -- 172.21.15.47:0/201137346 learned_addr learned my addr 172.21.15.47:0/201137346 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:03:35.532 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.529+0000 7f05affff640 1 --2- 172.21.15.47:0/1255473359 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f05a40a3dc0 0x7f05a40a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:03:35.532 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.529+0000 7f05b67fc640 1 --2- 172.21.15.47:0/201137346 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d7d076e7b0 0x55d7d0770ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:03:35.532 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.529+0000 7f05b6ffd640 1 --2- 172.21.15.47:0/201137346 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7d07659e0 0x55d7d076e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:03:35.532 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.529+0000 7f05b67fc640 1 -- 172.21.15.47:0/201137346 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7d07659e0 msgr2=0x55d7d076e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:03:35.532 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.529+0000 7f05b67fc640 1 --2- 172.21.15.47:0/201137346 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7d07659e0 0x55d7d076e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:35.532 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.529+0000 7f05b77fe640 1 --2- 172.21.15.47:0/1255473359 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f05a40a3dc0 0x7f05a40a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:03:35.532 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.529+0000 7f05b67fc640 1 -- 172.21.15.47:0/201137346 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d7d076e7b0 msgr2=0x55d7d0770ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:03:35.532 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.529+0000 7f05b67fc640 1 --2- 172.21.15.47:0/201137346 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d7d076e7b0 0x55d7d0770ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:35.533 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.529+0000 7f05b67fc640 1 -- 172.21.15.47:0/201137346 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7d03dd210 con 0x55d7d07650c0 2024-04-17T06:03:35.533 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.529+0000 7f05affff640 1 -- 172.21.15.47:0/1255473359 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8469+0+0 (secure 0 0 0) 0x7f05b00f20d0 con 0x7f05b819d010 2024-04-17T06:03:35.533 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.530+0000 7f05b67fc640 1 --2- 172.21.15.47:0/201137346 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d7d07650c0 0x55d7d07654a0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f05a000b6c0 tx=0x7f05a000b650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d72ef542bb179b95 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:03:35.533 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.530+0000 7f05ad7fa640 1 -- 172.21.15.47:0/201137346 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05a0004020 con 0x55d7d07650c0 2024-04-17T06:03:35.533 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.530+0000 7f05ad7fa640 1 -- 172.21.15.47:0/201137346 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05a00041c0 con 0x55d7d07650c0 2024-04-17T06:03:35.533 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.530+0000 7f05b77fe640 1 --2- 172.21.15.47:0/1255473359 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f05a40a3dc0 0x7f05a40a6280 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f05b00efab0 tx=0x7f05b0088e50 comp rx=0 tx=0).ready entity=mgr.24923 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:03:35.533 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.530+0000 7f05ad7fa640 1 -- 172.21.15.47:0/201137346 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05a000f6a0 con 0x55d7d07650c0 2024-04-17T06:03:35.533 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.531+0000 7f05c0385740 1 -- 172.21.15.47:0/201137346 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d7d07650c0 msgr2=0x55d7d07654a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:03:35.533 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.531+0000 7f05c0385740 1 --2- 172.21.15.47:0/201137346 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d7d07650c0 0x55d7d07654a0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f05a000b6c0 tx=0x7f05a000b650 comp rx=0 tx=0).stop 2024-04-17T06:03:35.533 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.531+0000 7f05c0385740 1 -- 172.21.15.47:0/201137346 shutdown_connections 2024-04-17T06:03:35.533 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.531+0000 7f05c0385740 1 --2- 172.21.15.47:0/201137346 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d7d076e7b0 0x55d7d0770ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:35.533 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.531+0000 7f05c0385740 1 --2- 172.21.15.47:0/201137346 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7d07659e0 0x55d7d076e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:35.533 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.531+0000 7f05c0385740 1 --2- 172.21.15.47:0/201137346 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d7d07650c0 0x55d7d07654a0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:35.533 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.531+0000 7f05c0385740 1 -- 172.21.15.47:0/201137346 >> 172.21.15.47:0/201137346 conn(0x55d7d075f9e0 msgr2=0x55d7d075fdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:03:35.534 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.531+0000 7f05c0385740 1 -- 172.21.15.47:0/201137346 shutdown_connections 2024-04-17T06:03:35.534 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.531+0000 7f05c0385740 1 -- 172.21.15.47:0/201137346 wait complete. 2024-04-17T06:03:35.534 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.531+0000 7f05c0385740 1 -- start start 2024-04-17T06:03:35.534 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.531+0000 7f05c0385740 1 Processor -- start 2024-04-17T06:03:35.534 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.531+0000 7f05c0385740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d7d07650c0 0x55d7d076b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:03:35.534 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.531+0000 7f05c0385740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d7d07659e0 0x55d7d076bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:03:35.534 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.531+0000 7f05c0385740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7d076e7b0 0x55d7d076c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:03:35.534 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.531+0000 7f05c0385740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55d7d0771380 con 0x55d7d07650c0 2024-04-17T06:03:35.534 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.531+0000 7f05c0385740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55d7d0652e90 con 0x55d7d07659e0 2024-04-17T06:03:35.534 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.531+0000 7f05c0385740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d7d0771500 con 0x55d7d076e7b0 2024-04-17T06:03:35.534 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.531+0000 7f05c0385740 10 client.0 _ll_register_callbacks cb 0x55d7d0668d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:03:35.534 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.532+0000 7f05b67fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d7d07650c0 0x55d7d076b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:03:35.534 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.532+0000 7f05b67fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d7d07650c0 0x55d7d076b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41584/0 (socket says 172.21.15.47:41584) 2024-04-17T06:03:35.535 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.532+0000 7f05b67fc640 1 -- 172.21.15.47:0/1084478044 learned_addr learned my addr 172.21.15.47:0/1084478044 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:03:35.535 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.532+0000 7f05b6ffd640 1 --2- 172.21.15.47:0/1084478044 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d7d07659e0 0x55d7d076bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:03:35.535 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.532+0000 7f05b67fc640 1 -- 172.21.15.47:0/1084478044 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7d076e7b0 msgr2=0x55d7d076c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:03:35.535 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.532+0000 7f05b67fc640 1 --2- 172.21.15.47:0/1084478044 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7d076e7b0 0x55d7d076c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:35.535 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.532+0000 7f05b67fc640 1 -- 172.21.15.47:0/1084478044 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d7d07659e0 msgr2=0x55d7d076bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:03:35.535 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.532+0000 7f05b67fc640 1 --2- 172.21.15.47:0/1084478044 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d7d07659e0 0x55d7d076bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:35.535 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.532+0000 7f05b67fc640 1 -- 172.21.15.47:0/1084478044 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7d04a9620 con 0x55d7d07650c0 2024-04-17T06:03:35.535 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.532+0000 7f05b67fc640 1 --2- 172.21.15.47:0/1084478044 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d7d07650c0 0x55d7d076b750 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f05a0003e60 tx=0x7f05a0004140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a85cbbe29ef5f0dc server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:03:35.535 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.532+0000 7f059e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-17T06:03:35.535 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.532+0000 7f059e7fc640 1 -- 172.21.15.47:0/1084478044 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05a00037f0 con 0x55d7d07650c0 2024-04-17T06:03:35.535 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.532+0000 7f059e7fc640 1 -- 172.21.15.47:0/1084478044 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8469+0+0 (secure 0 0 0) 0x7f05a0003990 con 0x55d7d07650c0 2024-04-17T06:03:35.535 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.532+0000 7f05c0385740 1 -- 172.21.15.47:0/1084478044 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05a00041c0 con 0x55d7d07650c0 2024-04-17T06:03:35.535 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.532+0000 7f059e7fc640 1 -- 172.21.15.47:0/1084478044 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f05a0005070 con 0x55d7d07650c0 2024-04-17T06:03:35.536 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.532+0000 7f05c0385740 10 client.15597 fetch_fsmap learned FSMap version 25 2024-04-17T06:03:35.536 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.532+0000 7f05c0385740 1 -- 172.21.15.47:0/1084478044 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d7d077d5b0 con 0x55d7d07650c0 2024-04-17T06:03:35.536 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.533+0000 7f059e7fc640 1 -- 172.21.15.47:0/1084478044 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05a00041c0 con 0x55d7d07650c0 2024-04-17T06:03:35.536 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.533+0000 7f059e7fc640 1 -- 172.21.15.47:0/1084478044 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 25) ==== 8343+0+0 (secure 0 0 0) 0x7f05a0026020 con 0x55d7d07650c0 2024-04-17T06:03:35.536 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.533+0000 7f05c0385740 10 client.15597 fetch_fsmap finished waiting for FSMap version 25 2024-04-17T06:03:35.536 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.533+0000 7f05c0385740 10 client.15597 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-17T06:03:35.536 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.533+0000 7f05c0385740 20 client.15597 populate_metadata read hostname 'smithi047' 2024-04-17T06:03:35.536 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.533+0000 7f05c0385740 1 --2- 172.21.15.47:0/1084478044 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55d7d077e950 0x55d7d0780d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:03:35.536 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.533+0000 7f05c0385740 4 client.15597 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T06:03:35.536 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.533+0000 7f05c0385740 1 -- 172.21.15.47:0/1084478044 --> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d7d06d81f0 con 0x55d7d077e950 2024-04-17T06:03:35.536 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.533+0000 7f05b6ffd640 1 --2- 172.21.15.47:0/1084478044 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55d7d077e950 0x55d7d0780d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:03:35.536 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.534+0000 7f05b6ffd640 1 --2- 172.21.15.47:0/1084478044 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55d7d077e950 0x55d7d0780d40 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=566b0ebec37f5f74 server_cookie=c08e819e1588a7ad in_seq=0 out_seq=0 2024-04-17T06:03:35.537 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.534+0000 7f059e7fc640 10 client.15597 ms_handle_connect on v2:172.21.15.98:6838/2841287917 2024-04-17T06:03:35.538 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.534+0000 7f059e7fc640 1 -- 172.21.15.47:0/1084478044 <== mds.0 v2:172.21.15.98:6838/2841287917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d7d06d81f0 con 0x55d7d077e950 2024-04-17T06:03:35.538 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.534+0000 7f059e7fc640 10 client.15597 handle_command_reply: tid=0 2024-04-17T06:03:35.538 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.534+0000 7f05c0385740 1 client.15597 shutdown 2024-04-17T06:03:35.538 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.534+0000 7f05c0385740 1 -- 172.21.15.47:0/1084478044 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d7d07650c0 msgr2=0x55d7d076b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:03:35.538 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.534+0000 7f05c0385740 1 --2- 172.21.15.47:0/1084478044 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d7d07650c0 0x55d7d076b750 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f05a0003e60 tx=0x7f05a0004140 comp rx=0 tx=0).stop 2024-04-17T06:03:35.538 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.534+0000 7f05c0385740 1 -- 172.21.15.47:0/1084478044 shutdown_connections 2024-04-17T06:03:35.538 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.534+0000 7f05c0385740 1 --2- 172.21.15.47:0/1084478044 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55d7d077e950 0x55d7d0780d40 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:35.538 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.534+0000 7f05c0385740 1 --2- 172.21.15.47:0/1084478044 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7d076e7b0 0x55d7d076c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:35.538 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.534+0000 7f05c0385740 1 --2- 172.21.15.47:0/1084478044 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d7d07659e0 0x55d7d076bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:35.538 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.534+0000 7f05c0385740 1 --2- 172.21.15.47:0/1084478044 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d7d07650c0 0x55d7d076b750 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:35.539 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.534+0000 7f059e7fc640 0 client.15597 ms_handle_reset on v2:172.21.15.98:6838/2841287917 2024-04-17T06:03:35.539 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.534+0000 7f05c0385740 1 -- 172.21.15.47:0/1084478044 >> 172.21.15.47:0/1084478044 conn(0x55d7d075f9e0 msgr2=0x55d7d0761730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:03:35.539 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.535+0000 7f05c0385740 1 -- 172.21.15.47:0/1084478044 shutdown_connections 2024-04-17T06:03:35.539 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.535+0000 7f05c0385740 1 -- 172.21.15.47:0/1084478044 wait complete. 2024-04-17T06:03:35.539 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.535+0000 7f05c0385740 20 client.15597 trim_cache size 0 max 16384 2024-04-17T06:03:35.547 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.550+0000 7f05c0385740 1 Processor -- start 2024-04-17T06:03:35.547 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.550+0000 7f05c0385740 1 -- start start 2024-04-17T06:03:35.550 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.550+0000 7f05c0385740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d7d07650c0 0x55d7d076de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:03:35.550 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.550+0000 7f05c0385740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d7d07659e0 0x55d7d077a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:03:35.550 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.550+0000 7f05c0385740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7d076e7b0 0x55d7d075e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:03:35.550 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.550+0000 7f05c0385740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f05b81b03f0 con 0x55d7d07650c0 2024-04-17T06:03:35.550 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.550+0000 7f05c0385740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f05b81adc80 con 0x55d7d07659e0 2024-04-17T06:03:35.550 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.550+0000 7f05c0385740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f05b81abad0 con 0x55d7d076e7b0 2024-04-17T06:03:35.550 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.550+0000 7f05b67fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d7d07650c0 0x55d7d076de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:03:35.550 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.550+0000 7f05b67fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d7d07650c0 0x55d7d076de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41600/0 (socket says 172.21.15.47:41600) 2024-04-17T06:03:35.550 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.550+0000 7f05b67fc640 1 -- 172.21.15.47:0/106539931 learned_addr learned my addr 172.21.15.47:0/106539931 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:03:35.550 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.550+0000 7f05b6ffd640 1 --2- 172.21.15.47:0/106539931 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d7d07659e0 0x55d7d077a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:03:35.551 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.551+0000 7f05b67fc640 1 -- 172.21.15.47:0/106539931 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7d076e7b0 msgr2=0x55d7d075e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:03:35.551 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.551+0000 7f05b67fc640 1 --2- 172.21.15.47:0/106539931 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7d076e7b0 0x55d7d075e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:35.551 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.551+0000 7f05b67fc640 1 -- 172.21.15.47:0/106539931 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d7d07659e0 msgr2=0x55d7d077a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:03:35.551 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.551+0000 7f05b67fc640 1 --2- 172.21.15.47:0/106539931 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d7d07659e0 0x55d7d077a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:35.551 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.551+0000 7f05b67fc640 1 -- 172.21.15.47:0/106539931 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7d075f810 con 0x55d7d07650c0 2024-04-17T06:03:35.551 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.551+0000 7f05b67fc640 1 --2- 172.21.15.47:0/106539931 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d7d07650c0 0x55d7d076de80 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f05a000b5a0 tx=0x7f05a0003dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42b9624a26e3926e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:03:35.551 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.551+0000 7f05acff9640 1 -- 172.21.15.47:0/106539931 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05a000b710 con 0x55d7d07650c0 2024-04-17T06:03:35.551 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.551+0000 7f05acff9640 1 -- 172.21.15.47:0/106539931 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05a00041c0 con 0x55d7d07650c0 2024-04-17T06:03:35.551 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.551+0000 7f05acff9640 1 -- 172.21.15.47:0/106539931 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05a0004b90 con 0x55d7d07650c0 2024-04-17T06:03:35.551 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.552+0000 7f05c0385740 1 -- 172.21.15.47:0/106539931 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d7d07650c0 msgr2=0x55d7d076de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:03:35.551 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.552+0000 7f05c0385740 1 --2- 172.21.15.47:0/106539931 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d7d07650c0 0x55d7d076de80 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f05a000b5a0 tx=0x7f05a0003dd0 comp rx=0 tx=0).stop 2024-04-17T06:03:35.551 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.552+0000 7f05c0385740 1 -- 172.21.15.47:0/106539931 shutdown_connections 2024-04-17T06:03:35.551 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.552+0000 7f05c0385740 1 --2- 172.21.15.47:0/106539931 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7d076e7b0 0x55d7d075e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:35.552 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.552+0000 7f05c0385740 1 --2- 172.21.15.47:0/106539931 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d7d07659e0 0x55d7d077a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:35.552 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.552+0000 7f05c0385740 1 --2- 172.21.15.47:0/106539931 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d7d07650c0 0x55d7d076de80 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:35.552 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.552+0000 7f05c0385740 1 -- 172.21.15.47:0/106539931 >> 172.21.15.47:0/106539931 conn(0x55d7d075f9e0 msgr2=0x55d7d075e270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:03:35.552 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.552+0000 7f05c0385740 1 -- 172.21.15.47:0/106539931 shutdown_connections 2024-04-17T06:03:35.552 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.552+0000 7f05c0385740 1 -- 172.21.15.47:0/106539931 wait complete. 2024-04-17T06:03:35.552 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.552+0000 7f05c0385740 1 -- start start 2024-04-17T06:03:35.552 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.552+0000 7f05c0385740 1 Processor -- start 2024-04-17T06:03:35.555 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.552+0000 7f05c0385740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d7d07650c0 0x55d7d0768420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:03:35.555 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.552+0000 7f05c0385740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7d07659e0 0x55d7d0768960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:03:35.555 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.552+0000 7f05c0385740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d7d076e7b0 0x55d7d0768ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:03:35.555 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.552+0000 7f05c0385740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f05b81ac140 con 0x55d7d076e7b0 2024-04-17T06:03:35.555 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.552+0000 7f05c0385740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f05b81ad5b0 con 0x55d7d07650c0 2024-04-17T06:03:35.556 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.552+0000 7f05c0385740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f05b81ac810 con 0x55d7d07659e0 2024-04-17T06:03:35.556 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.552+0000 7f05c0385740 10 client.0 _ll_register_callbacks cb 0x7f05a00123f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:03:35.556 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.553+0000 7f05b67fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d7d076e7b0 0x55d7d0768ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:03:35.556 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.553+0000 7f05b67fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d7d076e7b0 0x55d7d0768ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41612/0 (socket says 172.21.15.47:41612) 2024-04-17T06:03:35.556 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.553+0000 7f05b67fc640 1 -- 172.21.15.47:0/2435861262 learned_addr learned my addr 172.21.15.47:0/2435861262 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:03:35.556 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.553+0000 7f05b67fc640 1 --2- 172.21.15.47:0/2435861262 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d7d07650c0 0x55d7d0768420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:03:35.556 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.553+0000 7f05b6ffd640 1 --2- 172.21.15.47:0/2435861262 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7d07659e0 0x55d7d0768960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:03:35.556 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.553+0000 7f05b67fc640 1 -- 172.21.15.47:0/2435861262 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7d07659e0 msgr2=0x55d7d0768960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:03:35.556 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.553+0000 7f05b67fc640 1 --2- 172.21.15.47:0/2435861262 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7d07659e0 0x55d7d0768960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:35.556 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.553+0000 7f05b67fc640 1 -- 172.21.15.47:0/2435861262 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d7d07650c0 msgr2=0x55d7d0768420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:03:35.556 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.553+0000 7f05b67fc640 1 --2- 172.21.15.47:0/2435861262 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d7d07650c0 0x55d7d0768420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:35.556 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.553+0000 7f05b67fc640 1 -- 172.21.15.47:0/2435861262 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7d07e3e60 con 0x55d7d076e7b0 2024-04-17T06:03:35.557 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.553+0000 7f05b67fc640 1 --2- 172.21.15.47:0/2435861262 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d7d076e7b0 0x55d7d0768ea0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x55d7d07699d0 tx=0x7f05a0004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8308007e78933204 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:03:35.557 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.553+0000 7f059e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-17T06:03:35.557 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.553+0000 7f059e7fc640 1 -- 172.21.15.47:0/2435861262 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05a0004810 con 0x55d7d076e7b0 2024-04-17T06:03:35.557 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.553+0000 7f059e7fc640 1 -- 172.21.15.47:0/2435861262 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8469+0+0 (secure 0 0 0) 0x7f05a00049b0 con 0x55d7d076e7b0 2024-04-17T06:03:35.557 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.553+0000 7f05c0385740 1 -- 172.21.15.47:0/2435861262 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05a00041c0 con 0x55d7d076e7b0 2024-04-17T06:03:35.557 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.553+0000 7f059e7fc640 1 -- 172.21.15.47:0/2435861262 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f05a0004bb0 con 0x55d7d076e7b0 2024-04-17T06:03:35.557 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.553+0000 7f05c0385740 10 client.15603 fetch_fsmap learned FSMap version 25 2024-04-17T06:03:35.557 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.553+0000 7f05c0385740 1 -- 172.21.15.47:0/2435861262 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d7d07e50f0 con 0x55d7d076e7b0 2024-04-17T06:03:35.557 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.554+0000 7f059e7fc640 1 -- 172.21.15.47:0/2435861262 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05a00041c0 con 0x55d7d076e7b0 2024-04-17T06:03:35.557 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.554+0000 7f059e7fc640 1 -- 172.21.15.47:0/2435861262 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 25) ==== 8343+0+0 (secure 0 0 0) 0x7f05a003c040 con 0x55d7d076e7b0 2024-04-17T06:03:35.557 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.554+0000 7f05c0385740 10 client.15603 fetch_fsmap finished waiting for FSMap version 25 2024-04-17T06:03:35.557 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.554+0000 7f05c0385740 10 client.15603 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-17T06:03:35.557 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.554+0000 7f05c0385740 20 client.15603 populate_metadata read hostname 'smithi047' 2024-04-17T06:03:35.557 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.554+0000 7f05c0385740 1 --2- 172.21.15.47:0/2435861262 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55d7d07e5710 0x55d7d07e7b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:03:35.558 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.554+0000 7f05c0385740 4 client.15603 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-17T06:03:35.558 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.554+0000 7f05c0385740 1 -- 172.21.15.47:0/2435861262 --> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d7d06d2060 con 0x55d7d07e5710 2024-04-17T06:03:35.558 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.554+0000 7f05b6ffd640 1 --2- 172.21.15.47:0/2435861262 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55d7d07e5710 0x55d7d07e7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:03:35.558 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.554+0000 7f059e7fc640 10 client.15603 ms_handle_connect on v2:172.21.15.98:6838/2841287917 2024-04-17T06:03:35.558 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.554+0000 7f05b6ffd640 1 --2- 172.21.15.47:0/2435861262 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55d7d07e5710 0x55d7d07e7b30 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5bec7a689500823e server_cookie=6763d60c7965f769 in_seq=0 out_seq=0 2024-04-17T06:03:35.560 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.562+0000 7f059e7fc640 1 -- 172.21.15.47:0/2435861262 <== mds.0 v2:172.21.15.98:6838/2841287917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d7d06d2060 con 0x55d7d07e5710 2024-04-17T06:03:35.560 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.562+0000 7f059e7fc640 10 client.15603 handle_command_reply: tid=0 2024-04-17T06:03:35.560 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.562+0000 7f05c0385740 1 client.15603 shutdown 2024-04-17T06:03:35.561 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.562+0000 7f05c0385740 1 -- 172.21.15.47:0/2435861262 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d7d076e7b0 msgr2=0x55d7d0768ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:03:35.561 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.562+0000 7f05c0385740 1 --2- 172.21.15.47:0/2435861262 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d7d076e7b0 0x55d7d0768ea0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x55d7d07699d0 tx=0x7f05a0004770 comp rx=0 tx=0).stop 2024-04-17T06:03:35.561 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.562+0000 7f05c0385740 1 -- 172.21.15.47:0/2435861262 shutdown_connections 2024-04-17T06:03:35.561 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.562+0000 7f05c0385740 1 --2- 172.21.15.47:0/2435861262 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55d7d07e5710 0x55d7d07e7b30 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:35.561 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.562+0000 7f05c0385740 1 --2- 172.21.15.47:0/2435861262 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d7d076e7b0 0x55d7d0768ea0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:35.561 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.562+0000 7f05c0385740 1 --2- 172.21.15.47:0/2435861262 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7d07659e0 0x55d7d0768960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:35.561 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.562+0000 7f05c0385740 1 --2- 172.21.15.47:0/2435861262 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d7d07650c0 0x55d7d0768420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:35.561 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.562+0000 7f059e7fc640 0 client.15603 ms_handle_reset on v2:172.21.15.98:6838/2841287917 2024-04-17T06:03:35.561 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.562+0000 7f05c0385740 1 -- 172.21.15.47:0/2435861262 >> 172.21.15.47:0/2435861262 conn(0x55d7d075f9e0 msgr2=0x55d7d076bfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:03:35.561 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.562+0000 7f05c0385740 1 -- 172.21.15.47:0/2435861262 shutdown_connections 2024-04-17T06:03:35.561 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.562+0000 7f05c0385740 1 -- 172.21.15.47:0/2435861262 wait complete. 2024-04-17T06:03:35.561 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.562+0000 7f05c0385740 20 client.15603 trim_cache size 0 max 16384 2024-04-17T06:03:35.562 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-04-17T06:03:35.562 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-04-17T06:03:35.562 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-04-17T06:03:35.562 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-04-17T06:03:35.563 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.563+0000 7f05bd79d640 1 -- 172.21.15.47:0/1255473359 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f05a40a3dc0 msgr2=0x7f05a40a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:03:35.563 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.563+0000 7f05bd79d640 1 --2- 172.21.15.47:0/1255473359 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f05a40a3dc0 0x7f05a40a6280 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f05b00efab0 tx=0x7f05b0088e50 comp rx=0 tx=0).stop 2024-04-17T06:03:35.563 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.563+0000 7f05bd79d640 1 -- 172.21.15.47:0/1255473359 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f05b819d010 msgr2=0x7f05b819d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:03:35.563 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.563+0000 7f05bd79d640 1 --2- 172.21.15.47:0/1255473359 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f05b819d010 0x7f05b819d4c0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f05b0009230 tx=0x7f05b002cb30 comp rx=0 tx=0).stop 2024-04-17T06:03:35.563 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.563+0000 7f05bd79d640 1 -- 172.21.15.47:0/1255473359 shutdown_connections 2024-04-17T06:03:35.563 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.563+0000 7f05bd79d640 1 --2- 172.21.15.47:0/1255473359 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f05a40a3dc0 0x7f05a40a6280 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:35.563 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.563+0000 7f05bd79d640 1 --2- 172.21.15.47:0/1255473359 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f05b819d010 0x7f05b819d4c0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:35.563 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.563+0000 7f05bd79d640 1 --2- 172.21.15.47:0/1255473359 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f05b81072b0 0x7f05b8198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:35.563 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.563+0000 7f05bd79d640 1 --2- 172.21.15.47:0/1255473359 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f05b81021e0 0x7f05b8198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:35.563 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.563+0000 7f05bd79d640 1 -- 172.21.15.47:0/1255473359 >> 172.21.15.47:0/1255473359 conn(0x7f05b80793f0 msgr2=0x7f05b80ffb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:03:35.564 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.563+0000 7f05bd79d640 1 -- 172.21.15.47:0/1255473359 shutdown_connections 2024-04-17T06:03:35.564 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:35.563+0000 7f05bd79d640 1 -- 172.21.15.47:0/1255473359 wait complete. 2024-04-17T06:03:36.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:03:36 smithi103 ceph-mon[36243]: pgmap v72: 97 pgs: 97 active+clean; 1.6 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-04-17T06:03:36.382 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:03:36 smithi098 ceph-mon[36516]: pgmap v72: 97 pgs: 97 active+clean; 1.6 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-04-17T06:03:36.388 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:03:36 smithi047 ceph-mon[32223]: pgmap v72: 97 pgs: 97 active+clean; 1.6 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-04-17T06:03:38.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:03:38 smithi103 ceph-mon[36243]: pgmap v73: 97 pgs: 97 active+clean; 1.6 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.6 KiB/s wr, 9 op/s 2024-04-17T06:03:38.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:03:38 smithi098 ceph-mon[36516]: pgmap v73: 97 pgs: 97 active+clean; 1.6 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.6 KiB/s wr, 9 op/s 2024-04-17T06:03:38.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:03:38 smithi047 ceph-mon[32223]: pgmap v73: 97 pgs: 97 active+clean; 1.6 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.6 KiB/s wr, 9 op/s 2024-04-17T06:03:40.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:03:40 smithi103 ceph-mon[36243]: pgmap v74: 97 pgs: 97 active+clean; 1.6 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.6 KiB/s wr, 9 op/s 2024-04-17T06:03:40.382 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:03:40 smithi098 ceph-mon[36516]: pgmap v74: 97 pgs: 97 active+clean; 1.6 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.6 KiB/s wr, 9 op/s 2024-04-17T06:03:40.388 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:03:40 smithi047 ceph-mon[32223]: pgmap v74: 97 pgs: 97 active+clean; 1.6 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.6 KiB/s wr, 9 op/s 2024-04-17T06:03:41.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:03:41 smithi103 ceph-mon[36243]: pgmap v75: 97 pgs: 97 active+clean; 1.6 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-04-17T06:03:41.382 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:03:41 smithi098 ceph-mon[36516]: pgmap v75: 97 pgs: 97 active+clean; 1.6 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-04-17T06:03:41.388 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:03:41 smithi047 ceph-mon[32223]: pgmap v75: 97 pgs: 97 active+clean; 1.6 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-04-17T06:03:43.632 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:03:43 smithi098 ceph-mon[36516]: pgmap v76: 97 pgs: 97 active+clean; 1.6 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 8.7 KiB/s wr, 11 op/s 2024-04-17T06:03:43.638 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:03:43 smithi047 ceph-mon[32223]: pgmap v76: 97 pgs: 97 active+clean; 1.6 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 8.7 KiB/s wr, 11 op/s 2024-04-17T06:03:43.783 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:03:43 smithi103 ceph-mon[36243]: pgmap v76: 97 pgs: 97 active+clean; 1.6 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 8.7 KiB/s wr, 11 op/s 2024-04-17T06:03:45.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:03:45 smithi098 ceph-mon[36516]: pgmap v77: 97 pgs: 97 active+clean; 1.7 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-04-17T06:03:45.638 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:03:45 smithi047 ceph-mon[32223]: pgmap v77: 97 pgs: 97 active+clean; 1.7 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-04-17T06:03:45.783 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:03:45 smithi103 ceph-mon[36243]: pgmap v77: 97 pgs: 97 active+clean; 1.7 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-04-17T06:03:47.632 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:03:47 smithi098 ceph-mon[36516]: pgmap v78: 97 pgs: 97 active+clean; 1.7 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 8.7 KiB/s wr, 11 op/s 2024-04-17T06:03:47.638 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:03:47 smithi047 ceph-mon[32223]: pgmap v78: 97 pgs: 97 active+clean; 1.7 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 8.7 KiB/s wr, 11 op/s 2024-04-17T06:03:47.783 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:03:47 smithi103 ceph-mon[36243]: pgmap v78: 97 pgs: 97 active+clean; 1.7 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 8.7 KiB/s wr, 11 op/s 2024-04-17T06:03:48.663 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.662+0000 7f65b2cd9640 1 -- 172.21.15.47:0/1392673917 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65ac106190 msgr2=0x7f65ac10dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:03:48.663 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.662+0000 7f65b2cd9640 1 --2- 172.21.15.47:0/1392673917 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65ac106190 0x7f65ac10dd60 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f65a4009ca0 tx=0x7f65a402c0f0 comp rx=0 tx=0).stop 2024-04-17T06:03:48.663 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.662+0000 7f65b2cd9640 1 -- 172.21.15.47:0/1392673917 shutdown_connections 2024-04-17T06:03:48.663 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.662+0000 7f65b2cd9640 1 --2- 172.21.15.47:0/1392673917 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65ac106190 0x7f65ac10dd60 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:48.663 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.662+0000 7f65b2cd9640 1 --2- 172.21.15.47:0/1392673917 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f65ac101a70 0x7f65ac105ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:48.663 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.662+0000 7f65b2cd9640 1 --2- 172.21.15.47:0/1392673917 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f65ac1010c0 0x7f65ac1014a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:48.663 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.662+0000 7f65b2cd9640 1 -- 172.21.15.47:0/1392673917 >> 172.21.15.47:0/1392673917 conn(0x7f65ac078010 msgr2=0x7f65ac078420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:03:48.663 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.662+0000 7f65b2cd9640 1 -- 172.21.15.47:0/1392673917 shutdown_connections 2024-04-17T06:03:48.663 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.662+0000 7f65b2cd9640 1 -- 172.21.15.47:0/1392673917 wait complete. 2024-04-17T06:03:48.663 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.662+0000 7f65b2cd9640 1 Processor -- start 2024-04-17T06:03:48.664 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.662+0000 7f65b2cd9640 1 -- start start 2024-04-17T06:03:48.664 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.663+0000 7f65b2cd9640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f65ac1010c0 0x7f65ac197630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:03:48.664 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.663+0000 7f65b2cd9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f65ac106190 0x7f65ac197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:03:48.664 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.663+0000 7f65b2cd9640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65ac19bef0 0x7f65ac19c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:03:48.664 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.663+0000 7f65b2cd9640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f65ac111250 con 0x7f65ac19bef0 2024-04-17T06:03:48.664 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.663+0000 7f65b2cd9640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f65ac1110d0 con 0x7f65ac1010c0 2024-04-17T06:03:48.664 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.663+0000 7f65b2cd9640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f65ac1113d0 con 0x7f65ac106190 2024-04-17T06:03:48.664 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.663+0000 7f65b1276640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65ac19bef0 0x7f65ac19c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:03:48.664 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.663+0000 7f65b1276640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65ac19bef0 0x7f65ac19c3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40314/0 (socket says 172.21.15.47:40314) 2024-04-17T06:03:48.664 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.663+0000 7f65b1276640 1 -- 172.21.15.47:0/1075004541 learned_addr learned my addr 172.21.15.47:0/1075004541 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:03:48.664 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.663+0000 7f65b0a75640 1 --2- 172.21.15.47:0/1075004541 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f65ac1010c0 0x7f65ac197630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:03:48.664 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.663+0000 7f65abfff640 1 --2- 172.21.15.47:0/1075004541 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f65ac106190 0x7f65ac197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:03:48.664 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.663+0000 7f65b1276640 1 -- 172.21.15.47:0/1075004541 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f65ac106190 msgr2=0x7f65ac197b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:03:48.664 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.663+0000 7f65b1276640 1 --2- 172.21.15.47:0/1075004541 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f65ac106190 0x7f65ac197b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:48.664 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.663+0000 7f65b1276640 1 -- 172.21.15.47:0/1075004541 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f65ac1010c0 msgr2=0x7f65ac197630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:03:48.664 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.663+0000 7f65b1276640 1 --2- 172.21.15.47:0/1075004541 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f65ac1010c0 0x7f65ac197630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:48.665 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.663+0000 7f65b1276640 1 -- 172.21.15.47:0/1075004541 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65ac19c9e0 con 0x7f65ac19bef0 2024-04-17T06:03:48.665 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.663+0000 7f65b1276640 1 --2- 172.21.15.47:0/1075004541 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65ac19bef0 0x7f65ac19c3a0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f65a4005e30 tx=0x7f65a4032fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:03:48.665 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.664+0000 7f65a9ffb640 1 -- 172.21.15.47:0/1075004541 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a4034ad0 con 0x7f65ac19bef0 2024-04-17T06:03:48.665 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.664+0000 7f65b2cd9640 1 -- 172.21.15.47:0/1075004541 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65ac19cc70 con 0x7f65ac19bef0 2024-04-17T06:03:48.665 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.664+0000 7f65a9ffb640 1 -- 172.21.15.47:0/1075004541 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65a4034c70 con 0x7f65ac19bef0 2024-04-17T06:03:48.665 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.664+0000 7f65a9ffb640 1 -- 172.21.15.47:0/1075004541 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a40042a0 con 0x7f65ac19bef0 2024-04-17T06:03:48.665 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.664+0000 7f65b2cd9640 1 -- 172.21.15.47:0/1075004541 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65ac06aa80 con 0x7f65ac19bef0 2024-04-17T06:03:48.668 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.666+0000 7f65a9ffb640 1 -- 172.21.15.47:0/1075004541 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f65a4004440 con 0x7f65ac19bef0 2024-04-17T06:03:48.668 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.666+0000 7f65b59a3740 1 Processor -- start 2024-04-17T06:03:48.668 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.667+0000 7f65b59a3740 1 -- start start 2024-04-17T06:03:48.668 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.667+0000 7f65b59a3740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55878e683be0 0x55878e683fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:03:48.668 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.667+0000 7f65b59a3740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55878e684500 0x55878e691680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:03:48.668 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.667+0000 7f65b59a3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55878e691bc0 0x55878e693fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:03:48.668 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.667+0000 7f65b59a3740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55878e5e97d0 con 0x55878e683be0 2024-04-17T06:03:48.668 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.667+0000 7f65b59a3740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55878e586f70 con 0x55878e684500 2024-04-17T06:03:48.668 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.667+0000 7f65b59a3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55878e5e5790 con 0x55878e691bc0 2024-04-17T06:03:48.668 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.667+0000 7f65abfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55878e683be0 0x55878e683fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:03:48.668 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.667+0000 7f65abfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55878e683be0 0x55878e683fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40322/0 (socket says 172.21.15.47:40322) 2024-04-17T06:03:48.668 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.667+0000 7f65abfff640 1 -- 172.21.15.47:0/1138925883 learned_addr learned my addr 172.21.15.47:0/1138925883 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:03:48.669 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.667+0000 7f65a9ffb640 1 --2- 172.21.15.47:0/1075004541 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f65940a3dc0 0x7f65940a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:03:48.669 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.667+0000 7f65b0a75640 1 --2- 172.21.15.47:0/1138925883 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55878e684500 0x55878e691680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:03:48.669 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.667+0000 7f65abfff640 1 -- 172.21.15.47:0/1138925883 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55878e691bc0 msgr2=0x55878e693fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:03:48.669 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.667+0000 7f65abfff640 1 --2- 172.21.15.47:0/1138925883 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55878e691bc0 0x55878e693fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:48.669 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.667+0000 7f65abfff640 1 -- 172.21.15.47:0/1138925883 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55878e684500 msgr2=0x55878e691680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:03:48.669 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.667+0000 7f65abfff640 1 --2- 172.21.15.47:0/1138925883 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55878e684500 0x55878e691680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:48.669 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.667+0000 7f65abfff640 1 -- 172.21.15.47:0/1138925883 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55878e2fc210 con 0x55878e683be0 2024-04-17T06:03:48.669 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.667+0000 7f65a9ffb640 1 -- 172.21.15.47:0/1075004541 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8469+0+0 (secure 0 0 0) 0x7f65a40f4280 con 0x7f65ac19bef0 2024-04-17T06:03:48.669 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.667+0000 7f65b1276640 1 --2- 172.21.15.47:0/1075004541 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f65940a3dc0 0x7f65940a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:03:48.669 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.667+0000 7f65abfff640 1 --2- 172.21.15.47:0/1138925883 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55878e683be0 0x55878e683fc0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f65a000dc80 tx=0x7f65a0011750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dac2b42c5e7e4b3f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:03:48.669 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.668+0000 7f6592ffd640 1 -- 172.21.15.47:0/1138925883 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a001a400 con 0x55878e683be0 2024-04-17T06:03:48.669 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.668+0000 7f6592ffd640 1 -- 172.21.15.47:0/1138925883 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65a00128d0 con 0x55878e683be0 2024-04-17T06:03:48.669 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.668+0000 7f6592ffd640 1 -- 172.21.15.47:0/1138925883 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a000f8a0 con 0x55878e683be0 2024-04-17T06:03:48.670 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.668+0000 7f65b59a3740 1 -- 172.21.15.47:0/1138925883 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55878e683be0 msgr2=0x55878e683fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:03:48.670 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.668+0000 7f65b1276640 1 --2- 172.21.15.47:0/1075004541 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f65940a3dc0 0x7f65940a6280 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f65a40f1b70 tx=0x7f65a4091da0 comp rx=0 tx=0).ready entity=mgr.24923 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:03:48.670 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.668+0000 7f65b59a3740 1 --2- 172.21.15.47:0/1138925883 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55878e683be0 0x55878e683fc0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f65a000dc80 tx=0x7f65a0011750 comp rx=0 tx=0).stop 2024-04-17T06:03:48.670 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.669+0000 7f65b59a3740 1 -- 172.21.15.47:0/1138925883 shutdown_connections 2024-04-17T06:03:48.670 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.669+0000 7f65b59a3740 1 --2- 172.21.15.47:0/1138925883 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55878e691bc0 0x55878e693fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:48.670 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.669+0000 7f65b59a3740 1 --2- 172.21.15.47:0/1138925883 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55878e684500 0x55878e691680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:48.670 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.669+0000 7f65b59a3740 1 --2- 172.21.15.47:0/1138925883 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55878e683be0 0x55878e683fc0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:48.670 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.669+0000 7f65b59a3740 1 -- 172.21.15.47:0/1138925883 >> 172.21.15.47:0/1138925883 conn(0x55878e67e9e0 msgr2=0x55878e67edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:03:48.670 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.669+0000 7f65b59a3740 1 -- 172.21.15.47:0/1138925883 shutdown_connections 2024-04-17T06:03:48.670 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.669+0000 7f65b59a3740 1 -- 172.21.15.47:0/1138925883 wait complete. 2024-04-17T06:03:48.670 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.669+0000 7f65b59a3740 1 -- start start 2024-04-17T06:03:48.670 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.669+0000 7f65b59a3740 1 Processor -- start 2024-04-17T06:03:48.675 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.669+0000 7f65b59a3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55878e683be0 0x55878e68eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:03:48.675 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.669+0000 7f65b59a3740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55878e684500 0x55878e68f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:03:48.675 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.669+0000 7f65b59a3740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55878e691bc0 0x55878e68f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:03:48.675 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.669+0000 7f65b59a3740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55878e694780 con 0x55878e684500 2024-04-17T06:03:48.675 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.669+0000 7f65b59a3740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55878e571e90 con 0x55878e691bc0 2024-04-17T06:03:48.675 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.669+0000 7f65b59a3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55878e694900 con 0x55878e683be0 2024-04-17T06:03:48.675 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.670+0000 7f65b59a3740 10 client.0 _ll_register_callbacks cb 0x55878e587d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:03:48.675 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.670+0000 7f65b0a75640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55878e684500 0x55878e68f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:03:48.675 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.670+0000 7f65b0a75640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55878e684500 0x55878e68f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40326/0 (socket says 172.21.15.47:40326) 2024-04-17T06:03:48.675 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.670+0000 7f65b0a75640 1 -- 172.21.15.47:0/510222775 learned_addr learned my addr 172.21.15.47:0/510222775 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:03:48.675 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.670+0000 7f65abfff640 1 --2- 172.21.15.47:0/510222775 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55878e683be0 0x55878e68eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:03:48.676 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.670+0000 7f65abfff640 1 --2- 172.21.15.47:0/510222775 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55878e691bc0 0x55878e68f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:03:48.676 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.670+0000 7f65b0a75640 1 -- 172.21.15.47:0/510222775 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55878e683be0 msgr2=0x55878e68eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:03:48.676 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.670+0000 7f65b0a75640 1 --2- 172.21.15.47:0/510222775 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55878e683be0 0x55878e68eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:48.676 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.670+0000 7f65b0a75640 1 -- 172.21.15.47:0/510222775 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55878e691bc0 msgr2=0x55878e68f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:03:48.676 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.670+0000 7f65b0a75640 1 --2- 172.21.15.47:0/510222775 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55878e691bc0 0x55878e68f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:48.676 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.670+0000 7f65b0a75640 1 -- 172.21.15.47:0/510222775 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55878e3c8620 con 0x55878e684500 2024-04-17T06:03:48.676 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.670+0000 7f65abfff640 1 --2- 172.21.15.47:0/510222775 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55878e683be0 0x55878e68eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:03:48.676 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.670+0000 7f65b0a75640 1 --2- 172.21.15.47:0/510222775 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55878e684500 0x55878e68f1e0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f659c00b940 tx=0x7f659c00be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a43fa0c589ac3f42 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:03:48.676 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.670+0000 7f658bfff640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-17T06:03:48.676 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.670+0000 7f658bfff640 1 -- 172.21.15.47:0/510222775 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f659c005020 con 0x55878e684500 2024-04-17T06:03:48.676 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.670+0000 7f658bfff640 1 -- 172.21.15.47:0/510222775 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8469+0+0 (secure 0 0 0) 0x7f659c0051c0 con 0x55878e684500 2024-04-17T06:03:48.676 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.670+0000 7f65b59a3740 1 -- 172.21.15.47:0/510222775 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65a00128d0 con 0x55878e684500 2024-04-17T06:03:48.677 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.670+0000 7f658bfff640 1 -- 172.21.15.47:0/510222775 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f659c0053c0 con 0x55878e684500 2024-04-17T06:03:48.677 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.670+0000 7f65b59a3740 10 client.15615 fetch_fsmap learned FSMap version 25 2024-04-17T06:03:48.677 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.670+0000 7f65b59a3740 1 -- 172.21.15.47:0/510222775 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55878e6a0bb0 con 0x55878e684500 2024-04-17T06:03:48.677 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.671+0000 7f658bfff640 1 -- 172.21.15.47:0/510222775 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65a00128d0 con 0x55878e684500 2024-04-17T06:03:48.677 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.671+0000 7f658bfff640 1 -- 172.21.15.47:0/510222775 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 25) ==== 8343+0+0 (secure 0 0 0) 0x7f659c0237c0 con 0x55878e684500 2024-04-17T06:03:48.677 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.671+0000 7f65b59a3740 10 client.15615 fetch_fsmap finished waiting for FSMap version 25 2024-04-17T06:03:48.677 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.671+0000 7f65b59a3740 10 client.15615 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-17T06:03:48.677 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.671+0000 7f65b59a3740 20 client.15615 populate_metadata read hostname 'smithi047' 2024-04-17T06:03:48.677 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.671+0000 7f65b59a3740 1 --2- 172.21.15.47:0/510222775 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55878e6a20c0 0x55878e6a44d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:03:48.677 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.671+0000 7f65b59a3740 4 client.15615 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T06:03:48.677 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.671+0000 7f65b59a3740 1 -- 172.21.15.47:0/510222775 --> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55878e5f71f0 con 0x55878e6a20c0 2024-04-17T06:03:48.677 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.671+0000 7f65abfff640 1 --2- 172.21.15.47:0/510222775 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55878e6a20c0 0x55878e6a44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:03:48.677 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.672+0000 7f658bfff640 10 client.15615 ms_handle_connect on v2:172.21.15.98:6838/2841287917 2024-04-17T06:03:48.678 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.672+0000 7f65abfff640 1 --2- 172.21.15.47:0/510222775 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55878e6a20c0 0x55878e6a44d0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84ca70dd7a484c48 server_cookie=aa4ac396fbb58a84 in_seq=0 out_seq=0 2024-04-17T06:03:48.678 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.672+0000 7f658bfff640 1 -- 172.21.15.47:0/510222775 <== mds.0 v2:172.21.15.98:6838/2841287917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55878e5f71f0 con 0x55878e6a20c0 2024-04-17T06:03:48.678 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.672+0000 7f658bfff640 10 client.15615 handle_command_reply: tid=0 2024-04-17T06:03:48.678 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.672+0000 7f65b59a3740 1 client.15615 shutdown 2024-04-17T06:03:48.678 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.672+0000 7f65b59a3740 1 -- 172.21.15.47:0/510222775 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55878e684500 msgr2=0x55878e68f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:03:48.678 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.672+0000 7f65b59a3740 1 --2- 172.21.15.47:0/510222775 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55878e684500 0x55878e68f1e0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f659c00b940 tx=0x7f659c00be00 comp rx=0 tx=0).stop 2024-04-17T06:03:48.678 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.673+0000 7f65b59a3740 1 -- 172.21.15.47:0/510222775 shutdown_connections 2024-04-17T06:03:48.678 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.673+0000 7f65b59a3740 1 --2- 172.21.15.47:0/510222775 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55878e6a20c0 0x55878e6a44d0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:48.678 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.673+0000 7f65b59a3740 1 --2- 172.21.15.47:0/510222775 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55878e691bc0 0x55878e68f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:48.678 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.673+0000 7f65b59a3740 1 --2- 172.21.15.47:0/510222775 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55878e684500 0x55878e68f1e0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:48.678 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.673+0000 7f658bfff640 0 client.15615 ms_handle_reset on v2:172.21.15.98:6838/2841287917 2024-04-17T06:03:48.678 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.673+0000 7f65b59a3740 1 --2- 172.21.15.47:0/510222775 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55878e683be0 0x55878e68eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:48.679 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.673+0000 7f65b59a3740 1 -- 172.21.15.47:0/510222775 >> 172.21.15.47:0/510222775 conn(0x55878e67e9e0 msgr2=0x55878e680730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:03:48.679 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.673+0000 7f65b59a3740 1 -- 172.21.15.47:0/510222775 shutdown_connections 2024-04-17T06:03:48.679 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.673+0000 7f65b59a3740 1 -- 172.21.15.47:0/510222775 wait complete. 2024-04-17T06:03:48.679 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.673+0000 7f65b59a3740 20 client.15615 trim_cache size 0 max 16384 2024-04-17T06:03:48.691 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.688+0000 7f65b59a3740 1 Processor -- start 2024-04-17T06:03:48.691 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.688+0000 7f65b59a3740 1 -- start start 2024-04-17T06:03:48.691 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.688+0000 7f65b59a3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55878e683be0 0x55878e6913d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:03:48.691 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.688+0000 7f65b59a3740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55878e684500 0x55878e69d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:03:48.691 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.688+0000 7f65b59a3740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55878e691bc0 0x55878e6a9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:03:48.691 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.688+0000 7f65b59a3740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f65ac1ae1d0 con 0x55878e684500 2024-04-17T06:03:48.691 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.688+0000 7f65b59a3740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f65ac1aba60 con 0x55878e691bc0 2024-04-17T06:03:48.691 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.688+0000 7f65b59a3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f65ac1a98b0 con 0x55878e683be0 2024-04-17T06:03:48.691 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.688+0000 7f65b0a75640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55878e684500 0x55878e69d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:03:48.691 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.688+0000 7f65b0a75640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55878e684500 0x55878e69d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40336/0 (socket says 172.21.15.47:40336) 2024-04-17T06:03:48.692 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.688+0000 7f65b0a75640 1 -- 172.21.15.47:0/2498208425 learned_addr learned my addr 172.21.15.47:0/2498208425 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:03:48.692 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.688+0000 7f65abfff640 1 --2- 172.21.15.47:0/2498208425 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55878e683be0 0x55878e6913d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:03:48.692 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.688+0000 7f65abfff640 1 --2- 172.21.15.47:0/2498208425 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55878e691bc0 0x55878e6a9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:03:48.692 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.688+0000 7f65b0a75640 1 -- 172.21.15.47:0/2498208425 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55878e683be0 msgr2=0x55878e6913d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:03:48.692 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.688+0000 7f65b0a75640 1 --2- 172.21.15.47:0/2498208425 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55878e683be0 0x55878e6913d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:48.692 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.688+0000 7f65b0a75640 1 -- 172.21.15.47:0/2498208425 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55878e691bc0 msgr2=0x55878e6a9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:03:48.692 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.688+0000 7f65b0a75640 1 --2- 172.21.15.47:0/2498208425 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55878e691bc0 0x55878e6a9da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:48.692 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.688+0000 7f65b0a75640 1 -- 172.21.15.47:0/2498208425 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55878e67e510 con 0x55878e684500 2024-04-17T06:03:48.692 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.689+0000 7f65b0a75640 1 --2- 172.21.15.47:0/2498208425 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55878e684500 0x55878e69d6b0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f659c023b90 tx=0x7f659c0059b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3350eb4f3dce6b6b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:03:48.692 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.689+0000 7f65927fc640 1 -- 172.21.15.47:0/2498208425 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f659c029040 con 0x55878e684500 2024-04-17T06:03:48.692 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.689+0000 7f65927fc640 1 -- 172.21.15.47:0/2498208425 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f659c016070 con 0x55878e684500 2024-04-17T06:03:48.692 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.689+0000 7f65927fc640 1 -- 172.21.15.47:0/2498208425 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f659c012c40 con 0x55878e684500 2024-04-17T06:03:48.692 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.689+0000 7f65b59a3740 1 -- 172.21.15.47:0/2498208425 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55878e684500 msgr2=0x55878e69d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:03:48.692 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.689+0000 7f65b59a3740 1 --2- 172.21.15.47:0/2498208425 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55878e684500 0x55878e69d6b0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f659c023b90 tx=0x7f659c0059b0 comp rx=0 tx=0).stop 2024-04-17T06:03:48.693 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.689+0000 7f65b59a3740 1 -- 172.21.15.47:0/2498208425 shutdown_connections 2024-04-17T06:03:48.693 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.689+0000 7f65b59a3740 1 --2- 172.21.15.47:0/2498208425 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55878e691bc0 0x55878e6a9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:48.693 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.689+0000 7f65b59a3740 1 --2- 172.21.15.47:0/2498208425 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55878e684500 0x55878e69d6b0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:48.693 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.689+0000 7f65b59a3740 1 --2- 172.21.15.47:0/2498208425 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55878e683be0 0x55878e6913d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:48.693 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.689+0000 7f65b59a3740 1 -- 172.21.15.47:0/2498208425 >> 172.21.15.47:0/2498208425 conn(0x55878e67e9e0 msgr2=0x55878e689fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:03:48.693 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.689+0000 7f65b59a3740 1 -- 172.21.15.47:0/2498208425 shutdown_connections 2024-04-17T06:03:48.693 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.689+0000 7f65b59a3740 1 -- 172.21.15.47:0/2498208425 wait complete. 2024-04-17T06:03:48.693 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.690+0000 7f65b59a3740 1 -- start start 2024-04-17T06:03:48.693 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.690+0000 7f65b59a3740 1 Processor -- start 2024-04-17T06:03:48.693 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.690+0000 7f65b59a3740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55878e683be0 0x55878e699e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:03:48.693 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.690+0000 7f65b59a3740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55878e684500 0x55878e695200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:03:48.693 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.690+0000 7f65b59a3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55878e691bc0 0x55878e695740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:03:48.693 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.690+0000 7f65b59a3740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f65ac1a9f20 con 0x55878e683be0 2024-04-17T06:03:48.693 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.690+0000 7f65b59a3740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f65ac1ab390 con 0x55878e684500 2024-04-17T06:03:48.694 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.690+0000 7f65b59a3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f65ac1aa5f0 con 0x55878e691bc0 2024-04-17T06:03:48.694 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.690+0000 7f65b59a3740 10 client.0 _ll_register_callbacks cb 0x7f659c0040c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:03:48.694 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.690+0000 7f65abfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55878e683be0 0x55878e699e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:03:48.694 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.690+0000 7f65abfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55878e683be0 0x55878e699e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40338/0 (socket says 172.21.15.47:40338) 2024-04-17T06:03:48.694 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.690+0000 7f65abfff640 1 -- 172.21.15.47:0/1702495717 learned_addr learned my addr 172.21.15.47:0/1702495717 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:03:48.694 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.691+0000 7f65b0a75640 1 --2- 172.21.15.47:0/1702495717 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55878e684500 0x55878e695200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:03:48.694 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.691+0000 7f65abfff640 1 -- 172.21.15.47:0/1702495717 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55878e691bc0 msgr2=0x55878e695740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:03:48.694 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.691+0000 7f65abfff640 1 --2- 172.21.15.47:0/1702495717 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55878e691bc0 0x55878e695740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:48.694 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.691+0000 7f65abfff640 1 -- 172.21.15.47:0/1702495717 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55878e684500 msgr2=0x55878e695200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:03:48.694 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.691+0000 7f65abfff640 1 --2- 172.21.15.47:0/1702495717 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55878e684500 0x55878e695200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:48.694 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.691+0000 7f65abfff640 1 -- 172.21.15.47:0/1702495717 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55878e695e30 con 0x55878e683be0 2024-04-17T06:03:48.694 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.691+0000 7f65abfff640 1 --2- 172.21.15.47:0/1702495717 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55878e683be0 0x55878e699e90 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f65a000f430 tx=0x7f65a000f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a12eeea99bdebd server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:03:48.694 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.691+0000 7f658bfff640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-17T06:03:48.695 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.691+0000 7f658bfff640 1 -- 172.21.15.47:0/1702495717 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a002d030 con 0x55878e683be0 2024-04-17T06:03:48.695 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.691+0000 7f658bfff640 1 -- 172.21.15.47:0/1702495717 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8469+0+0 (secure 0 0 0) 0x7f65a0029620 con 0x55878e683be0 2024-04-17T06:03:48.695 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.691+0000 7f65b59a3740 1 -- 172.21.15.47:0/1702495717 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f659c016070 con 0x55878e683be0 2024-04-17T06:03:48.695 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.691+0000 7f658bfff640 1 -- 172.21.15.47:0/1702495717 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f65a0037050 con 0x55878e683be0 2024-04-17T06:03:48.695 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.691+0000 7f65b59a3740 10 client.15621 fetch_fsmap learned FSMap version 25 2024-04-17T06:03:48.695 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.691+0000 7f65b59a3740 1 -- 172.21.15.47:0/1702495717 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55878e708520 con 0x55878e683be0 2024-04-17T06:03:48.695 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.691+0000 7f658bfff640 1 -- 172.21.15.47:0/1702495717 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f659c016070 con 0x55878e683be0 2024-04-17T06:03:48.695 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.692+0000 7f658bfff640 1 -- 172.21.15.47:0/1702495717 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 25) ==== 8343+0+0 (secure 0 0 0) 0x7f65a0004810 con 0x55878e683be0 2024-04-17T06:03:48.695 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.692+0000 7f65b59a3740 10 client.15621 fetch_fsmap finished waiting for FSMap version 25 2024-04-17T06:03:48.695 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.692+0000 7f65b59a3740 10 client.15621 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-17T06:03:48.695 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.692+0000 7f65b59a3740 20 client.15621 populate_metadata read hostname 'smithi047' 2024-04-17T06:03:48.695 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.692+0000 7f65b59a3740 1 --2- 172.21.15.47:0/1702495717 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55878e708ad0 0x55878e70aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:03:48.695 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.692+0000 7f65b59a3740 4 client.15621 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-17T06:03:48.695 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.692+0000 7f65b59a3740 1 -- 172.21.15.47:0/1702495717 --> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55878e5f1060 con 0x55878e708ad0 2024-04-17T06:03:48.696 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.692+0000 7f65b0a75640 1 --2- 172.21.15.47:0/1702495717 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55878e708ad0 0x55878e70aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:03:48.696 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.692+0000 7f658bfff640 10 client.15621 ms_handle_connect on v2:172.21.15.98:6838/2841287917 2024-04-17T06:03:48.696 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.692+0000 7f65b0a75640 1 --2- 172.21.15.47:0/1702495717 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55878e708ad0 0x55878e70aef0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb0360ae2976bf7f server_cookie=94561f0af2cac2e in_seq=0 out_seq=0 2024-04-17T06:03:48.699 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.700+0000 7f658bfff640 1 -- 172.21.15.47:0/1702495717 <== mds.0 v2:172.21.15.98:6838/2841287917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55878e5f1060 con 0x55878e708ad0 2024-04-17T06:03:48.699 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.700+0000 7f658bfff640 10 client.15621 handle_command_reply: tid=0 2024-04-17T06:03:48.699 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.700+0000 7f65b59a3740 1 client.15621 shutdown 2024-04-17T06:03:48.700 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.700+0000 7f65b59a3740 1 -- 172.21.15.47:0/1702495717 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55878e683be0 msgr2=0x55878e699e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:03:48.700 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.700+0000 7f65b59a3740 1 --2- 172.21.15.47:0/1702495717 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55878e683be0 0x55878e699e90 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f65a000f430 tx=0x7f65a000f3c0 comp rx=0 tx=0).stop 2024-04-17T06:03:48.700 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.700+0000 7f65b59a3740 1 -- 172.21.15.47:0/1702495717 shutdown_connections 2024-04-17T06:03:48.700 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.700+0000 7f65b59a3740 1 --2- 172.21.15.47:0/1702495717 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55878e708ad0 0x55878e70aef0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:48.700 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.700+0000 7f65b59a3740 1 --2- 172.21.15.47:0/1702495717 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55878e691bc0 0x55878e695740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:48.700 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.700+0000 7f65b59a3740 1 --2- 172.21.15.47:0/1702495717 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55878e684500 0x55878e695200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:48.700 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.700+0000 7f658bfff640 0 client.15621 ms_handle_reset on v2:172.21.15.98:6838/2841287917 2024-04-17T06:03:48.700 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.700+0000 7f65b59a3740 1 --2- 172.21.15.47:0/1702495717 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55878e683be0 0x55878e699e90 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:48.700 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.700+0000 7f65b59a3740 1 -- 172.21.15.47:0/1702495717 >> 172.21.15.47:0/1702495717 conn(0x55878e67e9e0 msgr2=0x55878e67d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:03:48.700 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.700+0000 7f65b59a3740 1 -- 172.21.15.47:0/1702495717 shutdown_connections 2024-04-17T06:03:48.700 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.700+0000 7f65b59a3740 1 -- 172.21.15.47:0/1702495717 wait complete. 2024-04-17T06:03:48.700 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.700+0000 7f65b59a3740 20 client.15621 trim_cache size 0 max 16384 2024-04-17T06:03:48.701 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-04-17T06:03:48.701 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-04-17T06:03:48.701 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-04-17T06:03:48.701 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-04-17T06:03:48.702 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.701+0000 7f65b2cd9640 1 -- 172.21.15.47:0/1075004541 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f65940a3dc0 msgr2=0x7f65940a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:03:48.702 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.701+0000 7f65b2cd9640 1 --2- 172.21.15.47:0/1075004541 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f65940a3dc0 0x7f65940a6280 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f65a40f1b70 tx=0x7f65a4091da0 comp rx=0 tx=0).stop 2024-04-17T06:03:48.702 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.701+0000 7f65b2cd9640 1 -- 172.21.15.47:0/1075004541 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65ac19bef0 msgr2=0x7f65ac19c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:03:48.702 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.701+0000 7f65b2cd9640 1 --2- 172.21.15.47:0/1075004541 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65ac19bef0 0x7f65ac19c3a0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f65a4005e30 tx=0x7f65a4032fb0 comp rx=0 tx=0).stop 2024-04-17T06:03:48.702 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.701+0000 7f65b2cd9640 1 -- 172.21.15.47:0/1075004541 shutdown_connections 2024-04-17T06:03:48.702 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.701+0000 7f65b2cd9640 1 --2- 172.21.15.47:0/1075004541 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f65940a3dc0 0x7f65940a6280 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:48.702 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.701+0000 7f65b2cd9640 1 --2- 172.21.15.47:0/1075004541 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65ac19bef0 0x7f65ac19c3a0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:48.702 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.701+0000 7f65b2cd9640 1 --2- 172.21.15.47:0/1075004541 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f65ac106190 0x7f65ac197b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:48.702 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.701+0000 7f65b2cd9640 1 --2- 172.21.15.47:0/1075004541 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f65ac1010c0 0x7f65ac197630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:03:48.702 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.701+0000 7f65b2cd9640 1 -- 172.21.15.47:0/1075004541 >> 172.21.15.47:0/1075004541 conn(0x7f65ac078010 msgr2=0x7f65ac0fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:03:48.702 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.701+0000 7f65b2cd9640 1 -- 172.21.15.47:0/1075004541 shutdown_connections 2024-04-17T06:03:48.703 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:03:48.701+0000 7f65b2cd9640 1 -- 172.21.15.47:0/1075004541 wait complete. 2024-04-17T06:03:49.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:03:49 smithi098 ceph-mon[36516]: pgmap v79: 97 pgs: 97 active+clean; 1.7 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 8.7 KiB/s wr, 11 op/s 2024-04-17T06:03:49.638 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:03:49 smithi047 ceph-mon[32223]: pgmap v79: 97 pgs: 97 active+clean; 1.7 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 8.7 KiB/s wr, 11 op/s 2024-04-17T06:03:49.783 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:03:49 smithi103 ceph-mon[36243]: pgmap v79: 97 pgs: 97 active+clean; 1.7 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 8.7 KiB/s wr, 11 op/s 2024-04-17T06:03:51.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:03:51 smithi098 ceph-mon[36516]: pgmap v80: 97 pgs: 97 active+clean; 1.7 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-04-17T06:03:51.638 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:03:51 smithi047 ceph-mon[32223]: pgmap v80: 97 pgs: 97 active+clean; 1.7 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-04-17T06:03:51.783 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:03:51 smithi103 ceph-mon[36243]: pgmap v80: 97 pgs: 97 active+clean; 1.7 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-04-17T06:03:53.632 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:03:53 smithi098 ceph-mon[36516]: pgmap v81: 97 pgs: 97 active+clean; 1.7 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 8.9 KiB/s wr, 11 op/s 2024-04-17T06:03:53.638 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:03:53 smithi047 ceph-mon[32223]: pgmap v81: 97 pgs: 97 active+clean; 1.7 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 8.9 KiB/s wr, 11 op/s 2024-04-17T06:03:53.783 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:03:53 smithi103 ceph-mon[36243]: pgmap v81: 97 pgs: 97 active+clean; 1.7 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 8.9 KiB/s wr, 11 op/s 2024-04-17T06:03:55.559 INFO:tasks.workunit.client.0.smithi047.stdout:1/600: open fd = 3 2024-04-17T06:03:55.560 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.560 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.560 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:03:55.560 INFO:tasks.workunit.client.0.smithi047.stdout:2/600: open fd = 3 2024-04-17T06:03:55.560 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.560 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.560 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:03:55.560 INFO:tasks.workunit.client.0.smithi047.stdout:3/600: open fd = 3 2024-04-17T06:03:55.560 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.560 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.560 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:03:55.560 INFO:tasks.workunit.client.0.smithi047.stdout:4/600: open fd = 3 2024-04-17T06:03:55.560 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.561 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.561 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:03:55.561 INFO:tasks.workunit.client.0.smithi047.stdout:5/600: open fd = 3 2024-04-17T06:03:55.561 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.561 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.561 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:03:55.561 INFO:tasks.workunit.client.0.smithi047.stdout:6/600: open fd = 3 2024-04-17T06:03:55.561 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.561 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.561 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:03:55.561 INFO:tasks.workunit.client.0.smithi047.stdout:7/600: open fd = 3 2024-04-17T06:03:55.561 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.561 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.561 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:03:55.561 INFO:tasks.workunit.client.0.smithi047.stdout:8/600: open fd = 3 2024-04-17T06:03:55.562 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.562 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.562 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:03:55.562 INFO:tasks.workunit.client.0.smithi047.stdout:9/600: open fd = 3 2024-04-17T06:03:55.562 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.562 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.562 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:03:55.562 INFO:tasks.workunit.client.0.smithi047.stdout:10/600: open fd = 3 2024-04-17T06:03:55.562 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.562 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.562 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:03:55.562 INFO:tasks.workunit.client.0.smithi047.stdout:11/600: open fd = 3 2024-04-17T06:03:55.562 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.562 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.563 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:03:55.563 INFO:tasks.workunit.client.0.smithi047.stdout:12/600: open fd = 3 2024-04-17T06:03:55.563 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.563 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.563 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:03:55.563 INFO:tasks.workunit.client.0.smithi047.stdout:13/600: open fd = 3 2024-04-17T06:03:55.563 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.563 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.563 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:03:55.563 INFO:tasks.workunit.client.0.smithi047.stdout:14/600: open fd = 3 2024-04-17T06:03:55.563 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.563 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.563 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:03:55.563 INFO:tasks.workunit.client.0.smithi047.stdout:15/600: open fd = 3 2024-04-17T06:03:55.564 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.564 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.564 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:03:55.564 INFO:tasks.workunit.client.0.smithi047.stdout:16/600: open fd = 3 2024-04-17T06:03:55.564 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.564 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.564 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:03:55.564 INFO:tasks.workunit.client.0.smithi047.stdout:17/600: open fd = 3 2024-04-17T06:03:55.564 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.564 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.564 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:03:55.564 INFO:tasks.workunit.client.0.smithi047.stdout:18/600: open fd = 3 2024-04-17T06:03:55.564 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.564 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.565 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:03:55.565 INFO:tasks.workunit.client.0.smithi047.stdout:19/600: open fd = 3 2024-04-17T06:03:55.565 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.565 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.565 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:03:55.565 INFO:tasks.workunit.client.0.smithi047.stdout:20/600: open fd = 3 2024-04-17T06:03:55.565 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.565 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.565 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:03:55.565 INFO:tasks.workunit.client.0.smithi047.stdout:21/600: open fd = 3 2024-04-17T06:03:55.565 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.565 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.565 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:03:55.565 INFO:tasks.workunit.client.0.smithi047.stdout:22/600: open fd = 3 2024-04-17T06:03:55.566 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.566 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.566 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:03:55.566 INFO:tasks.workunit.client.0.smithi047.stdout:23/600: open fd = 3 2024-04-17T06:03:55.566 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.566 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.566 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:03:55.566 INFO:tasks.workunit.client.0.smithi047.stdout:24/600: open fd = 3 2024-04-17T06:03:55.566 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.566 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.566 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:03:55.566 INFO:tasks.workunit.client.0.smithi047.stdout:25/600: open fd = 3 2024-04-17T06:03:55.566 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.566 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.567 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:03:55.567 INFO:tasks.workunit.client.0.smithi047.stdout:26/600: open fd = 3 2024-04-17T06:03:55.567 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.567 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.567 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:03:55.567 INFO:tasks.workunit.client.0.smithi047.stdout:27/600: open fd = 3 2024-04-17T06:03:55.567 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.567 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.567 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:03:55.567 INFO:tasks.workunit.client.0.smithi047.stdout:28/600: open fd = 3 2024-04-17T06:03:55.567 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.567 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.567 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:03:55.567 INFO:tasks.workunit.client.0.smithi047.stdout:29/600: open fd = 3 2024-04-17T06:03:55.567 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.568 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.568 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:03:55.568 INFO:tasks.workunit.client.0.smithi047.stdout:30/600: open fd = 3 2024-04-17T06:03:55.568 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.568 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.568 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:03:55.568 INFO:tasks.workunit.client.0.smithi047.stdout:31/600: open fd = 3 2024-04-17T06:03:55.568 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.568 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.568 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:03:55.568 INFO:tasks.workunit.client.0.smithi047.stdout:32/600: open fd = 3 2024-04-17T06:03:55.568 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.568 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.568 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:03:55.568 INFO:tasks.workunit.client.0.smithi047.stdout:33/600: open fd = 3 2024-04-17T06:03:55.569 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.569 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.569 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:03:55.569 INFO:tasks.workunit.client.0.smithi047.stdout:34/600: open fd = 3 2024-04-17T06:03:55.569 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.569 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.569 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:03:55.569 INFO:tasks.workunit.client.0.smithi047.stdout:35/600: open fd = 3 2024-04-17T06:03:55.569 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.569 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.569 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:03:55.569 INFO:tasks.workunit.client.0.smithi047.stdout:36/600: open fd = 3 2024-04-17T06:03:55.569 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.569 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.569 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:03:55.570 INFO:tasks.workunit.client.0.smithi047.stdout:37/600: open fd = 3 2024-04-17T06:03:55.570 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.570 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.570 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:03:55.570 INFO:tasks.workunit.client.0.smithi047.stdout:38/600: open fd = 3 2024-04-17T06:03:55.570 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.570 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.570 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:03:55.570 INFO:tasks.workunit.client.0.smithi047.stdout:39/600: open fd = 3 2024-04-17T06:03:55.570 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.570 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.570 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:03:55.570 INFO:tasks.workunit.client.0.smithi047.stdout:40/600: open fd = 3 2024-04-17T06:03:55.570 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.571 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.571 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:03:55.571 INFO:tasks.workunit.client.0.smithi047.stdout:41/600: open fd = 3 2024-04-17T06:03:55.571 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.571 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.571 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:03:55.571 INFO:tasks.workunit.client.0.smithi047.stdout:42/600: open fd = 3 2024-04-17T06:03:55.571 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.571 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.571 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:03:55.571 INFO:tasks.workunit.client.0.smithi047.stdout:43/600: open fd = 3 2024-04-17T06:03:55.571 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.571 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.571 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:03:55.572 INFO:tasks.workunit.client.0.smithi047.stdout:44/600: open fd = 3 2024-04-17T06:03:55.572 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.572 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.572 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:03:55.572 INFO:tasks.workunit.client.0.smithi047.stdout:45/600: open fd = 3 2024-04-17T06:03:55.572 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.572 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.572 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:03:55.572 INFO:tasks.workunit.client.0.smithi047.stdout:46/600: open fd = 3 2024-04-17T06:03:55.572 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.572 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.572 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:03:55.572 INFO:tasks.workunit.client.0.smithi047.stdout:47/600: open fd = 3 2024-04-17T06:03:55.572 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.572 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.573 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:03:55.573 INFO:tasks.workunit.client.0.smithi047.stdout:48/600: open fd = 3 2024-04-17T06:03:55.573 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.573 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.573 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:03:55.573 INFO:tasks.workunit.client.0.smithi047.stdout:49/600: open fd = 3 2024-04-17T06:03:55.573 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.573 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.573 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:03:55.573 INFO:tasks.workunit.client.0.smithi047.stdout:50/600: open fd = 3 2024-04-17T06:03:55.573 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.573 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.573 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:03:55.574 INFO:tasks.workunit.client.0.smithi047.stdout:51/600: open fd = 3 2024-04-17T06:03:55.574 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.574 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.574 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:03:55.574 INFO:tasks.workunit.client.0.smithi047.stdout:52/600: open fd = 3 2024-04-17T06:03:55.574 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.574 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.574 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:03:55.574 INFO:tasks.workunit.client.0.smithi047.stdout:53/600: open fd = 3 2024-04-17T06:03:55.574 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.574 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.574 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:03:55.574 INFO:tasks.workunit.client.0.smithi047.stdout:54/600: open fd = 3 2024-04-17T06:03:55.574 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.575 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.575 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:03:55.575 INFO:tasks.workunit.client.0.smithi047.stdout:55/600: open fd = 3 2024-04-17T06:03:55.575 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.575 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.575 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:03:55.575 INFO:tasks.workunit.client.0.smithi047.stdout:56/600: open fd = 3 2024-04-17T06:03:55.575 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.575 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.575 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:03:55.575 INFO:tasks.workunit.client.0.smithi047.stdout:57/600: open fd = 3 2024-04-17T06:03:55.575 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.575 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.575 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:03:55.575 INFO:tasks.workunit.client.0.smithi047.stdout:58/600: open fd = 3 2024-04-17T06:03:55.576 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.576 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.576 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:03:55.576 INFO:tasks.workunit.client.0.smithi047.stdout:59/600: open fd = 3 2024-04-17T06:03:55.576 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.576 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.576 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:03:55.576 INFO:tasks.workunit.client.0.smithi047.stdout:60/600: open fd = 3 2024-04-17T06:03:55.576 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.576 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.576 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:03:55.576 INFO:tasks.workunit.client.0.smithi047.stdout:61/600: open fd = 3 2024-04-17T06:03:55.576 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.576 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.576 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:03:55.577 INFO:tasks.workunit.client.0.smithi047.stdout:62/600: open fd = 3 2024-04-17T06:03:55.577 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.577 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.577 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:03:55.577 INFO:tasks.workunit.client.0.smithi047.stdout:63/600: open fd = 3 2024-04-17T06:03:55.577 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.577 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:03:55.577 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:03:55.632 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:03:55 smithi098 ceph-mon[36516]: pgmap v82: 97 pgs: 97 active+clean; 1.7 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-04-17T06:03:55.638 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:03:55 smithi047 ceph-mon[32223]: pgmap v82: 97 pgs: 97 active+clean; 1.7 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-04-17T06:03:55.783 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:03:55 smithi103 ceph-mon[36243]: pgmap v82: 97 pgs: 97 active+clean; 1.7 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-04-17T06:03:57.632 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:03:57 smithi098 ceph-mon[36516]: pgmap v83: 97 pgs: 97 active+clean; 1.7 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 9.0 KiB/s wr, 11 op/s 2024-04-17T06:03:57.638 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:03:57 smithi047 ceph-mon[32223]: pgmap v83: 97 pgs: 97 active+clean; 1.7 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 9.0 KiB/s wr, 11 op/s 2024-04-17T06:03:57.783 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:03:57 smithi103 ceph-mon[36243]: pgmap v83: 97 pgs: 97 active+clean; 1.7 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 9.0 KiB/s wr, 11 op/s 2024-04-17T06:03:59.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:03:59 smithi098 ceph-mon[36516]: pgmap v84: 97 pgs: 97 active+clean; 1.7 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 9.0 KiB/s wr, 11 op/s 2024-04-17T06:03:59.638 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:03:59 smithi047 ceph-mon[32223]: pgmap v84: 97 pgs: 97 active+clean; 1.7 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 9.0 KiB/s wr, 11 op/s 2024-04-17T06:03:59.783 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:03:59 smithi103 ceph-mon[36243]: pgmap v84: 97 pgs: 97 active+clean; 1.7 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 9.0 KiB/s wr, 11 op/s 2024-04-17T06:04:01.632 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:04:01 smithi098 ceph-mon[36516]: pgmap v85: 97 pgs: 97 active+clean; 1.8 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-17T06:04:01.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:04:01 smithi047 ceph-mon[32223]: pgmap v85: 97 pgs: 97 active+clean; 1.8 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-17T06:04:01.783 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:04:01 smithi103 ceph-mon[36243]: pgmap v85: 97 pgs: 97 active+clean; 1.8 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-17T06:04:01.799 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.800+0000 7f7bfc9cd640 1 -- 172.21.15.47:0/1236232830 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7bf8103b10 msgr2=0x7f7bf810bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:04:01.800 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.800+0000 7f7bfc9cd640 1 --2- 172.21.15.47:0/1236232830 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7bf8103b10 0x7f7bf810bff0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f7be0009900 tx=0x7f7be002bd60 comp rx=0 tx=0).stop 2024-04-17T06:04:01.800 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.800+0000 7f7bfc9cd640 1 -- 172.21.15.47:0/1236232830 shutdown_connections 2024-04-17T06:04:01.800 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.800+0000 7f7bfc9cd640 1 --2- 172.21.15.47:0/1236232830 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7bf810c530 0x7f7bf810e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:01.800 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.800+0000 7f7bfc9cd640 1 --2- 172.21.15.47:0/1236232830 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7bf8103b10 0x7f7bf810bff0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:01.800 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.800+0000 7f7bfc9cd640 1 --2- 172.21.15.47:0/1236232830 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7bf81031f0 0x7f7bf81035d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:01.800 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.800+0000 7f7bfc9cd640 1 -- 172.21.15.47:0/1236232830 >> 172.21.15.47:0/1236232830 conn(0x7f7bf8077fe0 msgr2=0x7f7bf80783f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:04:01.800 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.801+0000 7f7bfc9cd640 1 -- 172.21.15.47:0/1236232830 shutdown_connections 2024-04-17T06:04:01.800 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.801+0000 7f7bfc9cd640 1 -- 172.21.15.47:0/1236232830 wait complete. 2024-04-17T06:04:01.802 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.801+0000 7f7bfc9cd640 1 Processor -- start 2024-04-17T06:04:01.802 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.802+0000 7f7bfc9cd640 1 -- start start 2024-04-17T06:04:01.803 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.802+0000 7f7bfc9cd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7bf81031f0 0x7f7bf819e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:04:01.803 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.802+0000 7f7bfc9cd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7bf8103b10 0x7f7bf819eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:04:01.803 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.802+0000 7f7bfc9cd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7bf810c530 0x7f7bf8198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:04:01.803 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.802+0000 7f7bfc9cd640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf8112340 con 0x7f7bf8103b10 2024-04-17T06:04:01.803 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.802+0000 7f7bfc9cd640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf81121c0 con 0x7f7bf81031f0 2024-04-17T06:04:01.803 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.802+0000 7f7bfc9cd640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf81124c0 con 0x7f7bf810c530 2024-04-17T06:04:01.803 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.802+0000 7f7bedd9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7bf8103b10 0x7f7bf819eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:04:01.803 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.802+0000 7f7bedd9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7bf8103b10 0x7f7bf819eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50318/0 (socket says 172.21.15.47:50318) 2024-04-17T06:04:01.803 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.802+0000 7f7bedd9b640 1 -- 172.21.15.47:0/3287777041 learned_addr learned my addr 172.21.15.47:0/3287777041 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:04:01.803 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.802+0000 7f7bf659c640 1 --2- 172.21.15.47:0/3287777041 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7bf81031f0 0x7f7bf819e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:04:01.803 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.802+0000 7f7bf6d9d640 1 --2- 172.21.15.47:0/3287777041 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7bf810c530 0x7f7bf8198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:04:01.803 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.802+0000 7f7bedd9b640 1 -- 172.21.15.47:0/3287777041 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7bf810c530 msgr2=0x7f7bf8198640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:04:01.803 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.802+0000 7f7bedd9b640 1 --2- 172.21.15.47:0/3287777041 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7bf810c530 0x7f7bf8198640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:01.804 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.802+0000 7f7bedd9b640 1 -- 172.21.15.47:0/3287777041 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7bf81031f0 msgr2=0x7f7bf819e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:04:01.804 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.802+0000 7f7bedd9b640 1 --2- 172.21.15.47:0/3287777041 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7bf81031f0 0x7f7bf819e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:01.804 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.802+0000 7f7bedd9b640 1 -- 172.21.15.47:0/3287777041 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bf8198f00 con 0x7f7bf8103b10 2024-04-17T06:04:01.804 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.803+0000 7f7bedd9b640 1 --2- 172.21.15.47:0/3287777041 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7bf8103b10 0x7f7bf819eac0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f7be0004f50 tx=0x7f7be002c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:04:01.804 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.803+0000 7f7beffff640 1 -- 172.21.15.47:0/3287777041 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be0043020 con 0x7f7bf8103b10 2024-04-17T06:04:01.804 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.803+0000 7f7beffff640 1 -- 172.21.15.47:0/3287777041 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7be00054e0 con 0x7f7bf8103b10 2024-04-17T06:04:01.804 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.803+0000 7f7bfc9cd640 1 -- 172.21.15.47:0/3287777041 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bf8199190 con 0x7f7bf8103b10 2024-04-17T06:04:01.804 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.803+0000 7f7beffff640 1 -- 172.21.15.47:0/3287777041 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be0003cd0 con 0x7f7bf8103b10 2024-04-17T06:04:01.804 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.803+0000 7f7bfc9cd640 1 -- 172.21.15.47:0/3287777041 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bf81a5500 con 0x7f7bf8103b10 2024-04-17T06:04:01.810 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.805+0000 7f7beffff640 1 -- 172.21.15.47:0/3287777041 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7be0046020 con 0x7f7bf8103b10 2024-04-17T06:04:01.810 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.806+0000 7f7bff5bb740 1 Processor -- start 2024-04-17T06:04:01.810 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.806+0000 7f7bff5bb740 1 -- start start 2024-04-17T06:04:01.810 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.806+0000 7f7bff5bb740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558cf4fdc0c0 0x558cf4fdc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:04:01.810 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.806+0000 7f7bff5bb740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558cf4fdc9e0 0x558cf4fe5270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:04:01.810 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.806+0000 7f7bff5bb740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558cf4fe57b0 0x558cf4fe7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:04:01.810 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.806+0000 7f7bff5bb740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x558cf4f417d0 con 0x558cf4fdc9e0 2024-04-17T06:04:01.810 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.806+0000 7f7bff5bb740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x558cf4edef70 con 0x558cf4fe57b0 2024-04-17T06:04:01.810 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.806+0000 7f7bff5bb740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558cf4f3d790 con 0x558cf4fdc0c0 2024-04-17T06:04:01.810 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.806+0000 7f7beffff640 1 --2- 172.21.15.47:0/3287777041 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f7be40a3dc0 0x7f7be40a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:04:01.811 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.806+0000 7f7bf659c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558cf4fdc9e0 0x558cf4fe5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:04:01.811 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.806+0000 7f7bf659c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558cf4fdc9e0 0x558cf4fe5270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50322/0 (socket says 172.21.15.47:50322) 2024-04-17T06:04:01.811 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.806+0000 7f7bf659c640 1 -- 172.21.15.47:0/3646422936 learned_addr learned my addr 172.21.15.47:0/3646422936 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:04:01.811 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.806+0000 7f7bf6d9d640 1 --2- 172.21.15.47:0/3287777041 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f7be40a3dc0 0x7f7be40a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:04:01.811 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.806+0000 7f7bedd9b640 1 --2- 172.21.15.47:0/3646422936 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558cf4fe57b0 0x558cf4fe7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:04:01.811 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.807+0000 7f7bf6d9d640 1 --2- 172.21.15.47:0/3646422936 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558cf4fdc0c0 0x558cf4fdc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:04:01.811 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.807+0000 7f7beffff640 1 -- 172.21.15.47:0/3287777041 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8469+0+0 (secure 0 0 0) 0x7f7be00f2920 con 0x7f7bf8103b10 2024-04-17T06:04:01.811 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.807+0000 7f7bf659c640 1 -- 172.21.15.47:0/3646422936 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558cf4fdc0c0 msgr2=0x558cf4fdc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:04:01.811 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.807+0000 7f7bf659c640 1 --2- 172.21.15.47:0/3646422936 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558cf4fdc0c0 0x558cf4fdc4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:01.811 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.807+0000 7f7bf659c640 1 -- 172.21.15.47:0/3646422936 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558cf4fe57b0 msgr2=0x558cf4fe7ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:04:01.811 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.807+0000 7f7bf659c640 1 --2- 172.21.15.47:0/3646422936 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558cf4fe57b0 0x558cf4fe7ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:01.811 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.807+0000 7f7bf659c640 1 -- 172.21.15.47:0/3646422936 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558cf4c54210 con 0x558cf4fdc9e0 2024-04-17T06:04:01.811 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.807+0000 7f7bedd9b640 1 --2- 172.21.15.47:0/3646422936 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558cf4fe57b0 0x558cf4fe7ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:04:01.811 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.807+0000 7f7bf659c640 1 --2- 172.21.15.47:0/3646422936 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558cf4fdc9e0 0x558cf4fe5270 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f7be800ad00 tx=0x7f7be800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ebc8589fad4eb21 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:04:01.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.807+0000 7f7bf6d9d640 1 --2- 172.21.15.47:0/3287777041 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f7be40a3dc0 0x7f7be40a6280 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7bf0005cc0 tx=0x7f7bf000a340 comp rx=0 tx=0).ready entity=mgr.24923 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:04:01.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.807+0000 7f7becd99640 1 -- 172.21.15.47:0/3646422936 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be801d070 con 0x558cf4fdc9e0 2024-04-17T06:04:01.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.807+0000 7f7becd99640 1 -- 172.21.15.47:0/3646422936 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7be8018b20 con 0x558cf4fdc9e0 2024-04-17T06:04:01.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.807+0000 7f7becd99640 1 -- 172.21.15.47:0/3646422936 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be8010bc0 con 0x558cf4fdc9e0 2024-04-17T06:04:01.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.808+0000 7f7bff5bb740 1 -- 172.21.15.47:0/3646422936 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558cf4fdc9e0 msgr2=0x558cf4fe5270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:04:01.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.808+0000 7f7bff5bb740 1 --2- 172.21.15.47:0/3646422936 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558cf4fdc9e0 0x558cf4fe5270 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f7be800ad00 tx=0x7f7be800f790 comp rx=0 tx=0).stop 2024-04-17T06:04:01.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.808+0000 7f7bff5bb740 1 -- 172.21.15.47:0/3646422936 shutdown_connections 2024-04-17T06:04:01.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.808+0000 7f7bff5bb740 1 --2- 172.21.15.47:0/3646422936 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558cf4fe57b0 0x558cf4fe7ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:01.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.808+0000 7f7bff5bb740 1 --2- 172.21.15.47:0/3646422936 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558cf4fdc9e0 0x558cf4fe5270 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:01.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.808+0000 7f7bff5bb740 1 --2- 172.21.15.47:0/3646422936 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558cf4fdc0c0 0x558cf4fdc4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:01.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.808+0000 7f7bff5bb740 1 -- 172.21.15.47:0/3646422936 >> 172.21.15.47:0/3646422936 conn(0x558cf4fd69e0 msgr2=0x558cf4fd6dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:04:01.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.808+0000 7f7bff5bb740 1 -- 172.21.15.47:0/3646422936 shutdown_connections 2024-04-17T06:04:01.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.808+0000 7f7bff5bb740 1 -- 172.21.15.47:0/3646422936 wait complete. 2024-04-17T06:04:01.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.808+0000 7f7bff5bb740 1 -- start start 2024-04-17T06:04:01.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.808+0000 7f7bff5bb740 1 Processor -- start 2024-04-17T06:04:01.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.809+0000 7f7bff5bb740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558cf4fdc0c0 0x558cf4fe2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:04:01.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.809+0000 7f7bff5bb740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558cf4fdc9e0 0x558cf4fe2c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:04:01.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.809+0000 7f7bff5bb740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558cf4fe57b0 0x558cf4fe31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:04:01.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.809+0000 7f7bff5bb740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x558cf4fe83e0 con 0x558cf4fdc9e0 2024-04-17T06:04:01.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.809+0000 7f7bff5bb740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x558cf4ec9e90 con 0x558cf4fdc0c0 2024-04-17T06:04:01.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.809+0000 7f7bff5bb740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558cf4fe8560 con 0x558cf4fe57b0 2024-04-17T06:04:01.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.809+0000 7f7bff5bb740 10 client.0 _ll_register_callbacks cb 0x558cf4edfd40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:04:01.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.809+0000 7f7bedd9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558cf4fdc9e0 0x558cf4fe2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:04:01.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.809+0000 7f7bedd9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558cf4fdc9e0 0x558cf4fe2c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50330/0 (socket says 172.21.15.47:50330) 2024-04-17T06:04:01.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.809+0000 7f7bedd9b640 1 -- 172.21.15.47:0/223296383 learned_addr learned my addr 172.21.15.47:0/223296383 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:04:01.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.809+0000 7f7bf659c640 1 --2- 172.21.15.47:0/223296383 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558cf4fdc0c0 0x558cf4fe2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:04:01.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.809+0000 7f7bf6d9d640 1 --2- 172.21.15.47:0/223296383 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558cf4fe57b0 0x558cf4fe31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:04:01.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.809+0000 7f7bedd9b640 1 -- 172.21.15.47:0/223296383 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558cf4fe57b0 msgr2=0x558cf4fe31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:04:01.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.809+0000 7f7bedd9b640 1 --2- 172.21.15.47:0/223296383 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558cf4fe57b0 0x558cf4fe31a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:01.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.809+0000 7f7bedd9b640 1 -- 172.21.15.47:0/223296383 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558cf4fdc0c0 msgr2=0x558cf4fe2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:04:01.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.809+0000 7f7bedd9b640 1 --2- 172.21.15.47:0/223296383 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558cf4fdc0c0 0x558cf4fe2720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:01.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.809+0000 7f7bedd9b640 1 -- 172.21.15.47:0/223296383 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558cf4d20620 con 0x558cf4fdc9e0 2024-04-17T06:04:01.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.809+0000 7f7bedd9b640 1 --2- 172.21.15.47:0/223296383 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558cf4fdc9e0 0x558cf4fe2c60 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f7be00f0210 tx=0x7f7be008d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b2adc51a922b7c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:04:01.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.809+0000 7f7bddffb640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-17T06:04:01.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.810+0000 7f7bddffb640 1 -- 172.21.15.47:0/223296383 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be00be7d0 con 0x558cf4fdc9e0 2024-04-17T06:04:01.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.810+0000 7f7bddffb640 1 -- 172.21.15.47:0/223296383 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8469+0+0 (secure 0 0 0) 0x7f7be003d3b0 con 0x558cf4fdc9e0 2024-04-17T06:04:01.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.810+0000 7f7bff5bb740 1 -- 172.21.15.47:0/223296383 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7be8018b20 con 0x558cf4fdc9e0 2024-04-17T06:04:01.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.810+0000 7f7bddffb640 1 -- 172.21.15.47:0/223296383 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7be00be360 con 0x558cf4fdc9e0 2024-04-17T06:04:01.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.810+0000 7f7bff5bb740 10 client.15633 fetch_fsmap learned FSMap version 25 2024-04-17T06:04:01.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.810+0000 7f7bff5bb740 1 -- 172.21.15.47:0/223296383 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558cf4ff4570 con 0x558cf4fdc9e0 2024-04-17T06:04:01.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.810+0000 7f7bddffb640 1 -- 172.21.15.47:0/223296383 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7be8018b20 con 0x558cf4fdc9e0 2024-04-17T06:04:01.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.810+0000 7f7bddffb640 1 -- 172.21.15.47:0/223296383 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 25) ==== 8343+0+0 (secure 0 0 0) 0x7f7be00c71e0 con 0x558cf4fdc9e0 2024-04-17T06:04:01.815 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.810+0000 7f7bff5bb740 10 client.15633 fetch_fsmap finished waiting for FSMap version 25 2024-04-17T06:04:01.815 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.810+0000 7f7bff5bb740 10 client.15633 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-17T06:04:01.815 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.810+0000 7f7bff5bb740 20 client.15633 populate_metadata read hostname 'smithi047' 2024-04-17T06:04:01.815 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.810+0000 7f7bff5bb740 1 --2- 172.21.15.47:0/223296383 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x558cf4ff59d0 0x558cf4ff7e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:04:01.815 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.810+0000 7f7bff5bb740 4 client.15633 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T06:04:01.815 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.810+0000 7f7bff5bb740 1 -- 172.21.15.47:0/223296383 --> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558cf4f4f1f0 con 0x558cf4ff59d0 2024-04-17T06:04:01.815 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.810+0000 7f7bf659c640 1 --2- 172.21.15.47:0/223296383 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x558cf4ff59d0 0x558cf4ff7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:04:01.815 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.811+0000 7f7bf659c640 1 --2- 172.21.15.47:0/223296383 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x558cf4ff59d0 0x558cf4ff7e60 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c7e70839021880b server_cookie=a3e6486fb7a4b40a in_seq=0 out_seq=0 2024-04-17T06:04:01.815 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.811+0000 7f7bddffb640 10 client.15633 ms_handle_connect on v2:172.21.15.98:6838/2841287917 2024-04-17T06:04:01.816 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.812+0000 7f7bddffb640 1 -- 172.21.15.47:0/223296383 <== mds.0 v2:172.21.15.98:6838/2841287917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558cf4f4f1f0 con 0x558cf4ff59d0 2024-04-17T06:04:01.816 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.812+0000 7f7bddffb640 10 client.15633 handle_command_reply: tid=0 2024-04-17T06:04:01.816 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.812+0000 7f7bff5bb740 1 client.15633 shutdown 2024-04-17T06:04:01.816 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.812+0000 7f7bff5bb740 1 -- 172.21.15.47:0/223296383 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558cf4fdc9e0 msgr2=0x558cf4fe2c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:04:01.816 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.812+0000 7f7bff5bb740 1 --2- 172.21.15.47:0/223296383 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558cf4fdc9e0 0x558cf4fe2c60 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f7be00f0210 tx=0x7f7be008d3b0 comp rx=0 tx=0).stop 2024-04-17T06:04:01.816 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.812+0000 7f7bff5bb740 1 -- 172.21.15.47:0/223296383 shutdown_connections 2024-04-17T06:04:01.817 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.812+0000 7f7bff5bb740 1 --2- 172.21.15.47:0/223296383 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x558cf4ff59d0 0x558cf4ff7e60 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:01.817 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.812+0000 7f7bff5bb740 1 --2- 172.21.15.47:0/223296383 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558cf4fe57b0 0x558cf4fe31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:01.817 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.812+0000 7f7bff5bb740 1 --2- 172.21.15.47:0/223296383 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558cf4fdc9e0 0x558cf4fe2c60 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:01.817 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.812+0000 7f7bff5bb740 1 --2- 172.21.15.47:0/223296383 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558cf4fdc0c0 0x558cf4fe2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:01.817 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.812+0000 7f7bddffb640 0 client.15633 ms_handle_reset on v2:172.21.15.98:6838/2841287917 2024-04-17T06:04:01.817 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.812+0000 7f7bff5bb740 1 -- 172.21.15.47:0/223296383 >> 172.21.15.47:0/223296383 conn(0x558cf4fd69e0 msgr2=0x558cf4fd8730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:04:01.817 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.812+0000 7f7bff5bb740 1 -- 172.21.15.47:0/223296383 shutdown_connections 2024-04-17T06:04:01.817 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.812+0000 7f7bff5bb740 1 -- 172.21.15.47:0/223296383 wait complete. 2024-04-17T06:04:01.817 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.812+0000 7f7bff5bb740 20 client.15633 trim_cache size 0 max 16384 2024-04-17T06:04:01.831 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.827+0000 7f7bff5bb740 1 Processor -- start 2024-04-17T06:04:01.831 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.827+0000 7f7bff5bb740 1 -- start start 2024-04-17T06:04:01.831 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.827+0000 7f7bff5bb740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558cf4fdc0c0 0x558cf4fe4e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:04:01.831 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.827+0000 7f7bff5bb740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558cf4fdc9e0 0x558cf4ff12a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:04:01.831 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.827+0000 7f7bff5bb740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558cf4fe57b0 0x558cf4ffd770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:04:01.831 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.828+0000 7f7bff5bb740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf81b03e0 con 0x558cf4fdc0c0 2024-04-17T06:04:01.831 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.828+0000 7f7bff5bb740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf81adc70 con 0x558cf4fdc9e0 2024-04-17T06:04:01.831 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.828+0000 7f7bff5bb740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf81abac0 con 0x558cf4fe57b0 2024-04-17T06:04:01.831 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.828+0000 7f7bf659c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558cf4fdc0c0 0x558cf4fe4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:04:01.831 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.828+0000 7f7bf659c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558cf4fdc0c0 0x558cf4fe4e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50342/0 (socket says 172.21.15.47:50342) 2024-04-17T06:04:01.831 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.828+0000 7f7bf659c640 1 -- 172.21.15.47:0/3577912243 learned_addr learned my addr 172.21.15.47:0/3577912243 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:04:01.832 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.828+0000 7f7bf6d9d640 1 --2- 172.21.15.47:0/3577912243 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558cf4fe57b0 0x558cf4ffd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:04:01.832 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.828+0000 7f7bf659c640 1 -- 172.21.15.47:0/3577912243 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558cf4fe57b0 msgr2=0x558cf4ffd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:04:01.832 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.828+0000 7f7bf659c640 1 --2- 172.21.15.47:0/3577912243 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558cf4fe57b0 0x558cf4ffd770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:01.832 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.828+0000 7f7bf659c640 1 -- 172.21.15.47:0/3577912243 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558cf4fdc9e0 msgr2=0x558cf4ff12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:04:01.832 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.828+0000 7f7bedd9b640 1 --2- 172.21.15.47:0/3577912243 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558cf4fdc9e0 0x558cf4ff12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:04:01.832 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.828+0000 7f7bf659c640 1 --2- 172.21.15.47:0/3577912243 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558cf4fdc9e0 0x558cf4ff12a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:01.832 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.828+0000 7f7bf659c640 1 -- 172.21.15.47:0/3577912243 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558cf4fd6810 con 0x558cf4fdc0c0 2024-04-17T06:04:01.832 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.828+0000 7f7bf6d9d640 1 --2- 172.21.15.47:0/3577912243 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558cf4fe57b0 0x558cf4ffd770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:04:01.832 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.828+0000 7f7bf659c640 1 --2- 172.21.15.47:0/3577912243 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558cf4fdc0c0 0x558cf4fe4e50 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f7be8010810 tx=0x7f7be8010700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=517f7ca8cba5de0c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:04:01.832 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.828+0000 7f7bdffff640 1 -- 172.21.15.47:0/3577912243 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be8031070 con 0x558cf4fdc0c0 2024-04-17T06:04:01.832 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.828+0000 7f7bdffff640 1 -- 172.21.15.47:0/3577912243 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7be800a630 con 0x558cf4fdc0c0 2024-04-17T06:04:01.832 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.828+0000 7f7bdffff640 1 -- 172.21.15.47:0/3577912243 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be801d040 con 0x558cf4fdc0c0 2024-04-17T06:04:01.832 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.829+0000 7f7bff5bb740 1 -- 172.21.15.47:0/3577912243 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558cf4fdc0c0 msgr2=0x558cf4fe4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:04:01.833 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.829+0000 7f7bff5bb740 1 --2- 172.21.15.47:0/3577912243 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558cf4fdc0c0 0x558cf4fe4e50 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f7be8010810 tx=0x7f7be8010700 comp rx=0 tx=0).stop 2024-04-17T06:04:01.833 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.829+0000 7f7bff5bb740 1 -- 172.21.15.47:0/3577912243 shutdown_connections 2024-04-17T06:04:01.833 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.829+0000 7f7bff5bb740 1 --2- 172.21.15.47:0/3577912243 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558cf4fe57b0 0x558cf4ffd770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:01.833 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.829+0000 7f7bff5bb740 1 --2- 172.21.15.47:0/3577912243 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558cf4fdc9e0 0x558cf4ff12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:01.833 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.829+0000 7f7bff5bb740 1 --2- 172.21.15.47:0/3577912243 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558cf4fdc0c0 0x558cf4fe4e50 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:01.833 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.829+0000 7f7bff5bb740 1 -- 172.21.15.47:0/3577912243 >> 172.21.15.47:0/3577912243 conn(0x558cf4fd69e0 msgr2=0x558cf4fdd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:04:01.833 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.829+0000 7f7bff5bb740 1 -- 172.21.15.47:0/3577912243 shutdown_connections 2024-04-17T06:04:01.833 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.829+0000 7f7bff5bb740 1 -- 172.21.15.47:0/3577912243 wait complete. 2024-04-17T06:04:01.833 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.829+0000 7f7bff5bb740 1 -- start start 2024-04-17T06:04:01.833 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.829+0000 7f7bff5bb740 1 Processor -- start 2024-04-17T06:04:01.833 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.830+0000 7f7bff5bb740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558cf4fdc0c0 0x558cf4fedcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:04:01.833 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.830+0000 7f7bff5bb740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558cf4fdc9e0 0x558cf4fdf220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:04:01.833 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.830+0000 7f7bff5bb740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558cf4fe57b0 0x558cf4fdf910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:04:01.833 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.830+0000 7f7bff5bb740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf81ac130 con 0x558cf4fdc9e0 2024-04-17T06:04:01.834 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.830+0000 7f7bff5bb740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf81ad5a0 con 0x558cf4fdc0c0 2024-04-17T06:04:01.834 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.830+0000 7f7bff5bb740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf81ac800 con 0x558cf4fe57b0 2024-04-17T06:04:01.834 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.830+0000 7f7bff5bb740 10 client.0 _ll_register_callbacks cb 0x7f7be00df0b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:04:01.834 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.830+0000 7f7bedd9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558cf4fdc9e0 0x558cf4fdf220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:04:01.834 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.830+0000 7f7bedd9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558cf4fdc9e0 0x558cf4fdf220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50352/0 (socket says 172.21.15.47:50352) 2024-04-17T06:04:01.834 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.830+0000 7f7bedd9b640 1 -- 172.21.15.47:0/242078191 learned_addr learned my addr 172.21.15.47:0/242078191 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:04:01.834 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.830+0000 7f7bf659c640 1 --2- 172.21.15.47:0/242078191 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558cf4fdc0c0 0x558cf4fedcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:04:01.834 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.830+0000 7f7bf6d9d640 1 --2- 172.21.15.47:0/242078191 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558cf4fe57b0 0x558cf4fdf910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:04:01.834 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.830+0000 7f7bedd9b640 1 -- 172.21.15.47:0/242078191 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558cf4fe57b0 msgr2=0x558cf4fdf910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:04:01.834 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.830+0000 7f7bedd9b640 1 --2- 172.21.15.47:0/242078191 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558cf4fe57b0 0x558cf4fdf910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:01.834 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.830+0000 7f7bedd9b640 1 -- 172.21.15.47:0/242078191 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558cf4fdc0c0 msgr2=0x558cf4fedcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:04:01.834 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.830+0000 7f7bedd9b640 1 --2- 172.21.15.47:0/242078191 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558cf4fdc0c0 0x558cf4fedcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:01.834 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.830+0000 7f7bedd9b640 1 -- 172.21.15.47:0/242078191 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558cf4fdb9a0 con 0x558cf4fdc9e0 2024-04-17T06:04:01.835 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.830+0000 7f7bedd9b640 1 --2- 172.21.15.47:0/242078191 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558cf4fdc9e0 0x558cf4fdf220 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f7be00f0260 tx=0x7f7be0005be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5585a2e89aff6f07 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:04:01.835 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.830+0000 7f7bddffb640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-17T06:04:01.835 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.830+0000 7f7bddffb640 1 -- 172.21.15.47:0/242078191 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be00be540 con 0x558cf4fdc9e0 2024-04-17T06:04:01.835 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.830+0000 7f7bddffb640 1 -- 172.21.15.47:0/242078191 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8469+0+0 (secure 0 0 0) 0x7f7be00bde50 con 0x558cf4fdc9e0 2024-04-17T06:04:01.835 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.830+0000 7f7bff5bb740 1 -- 172.21.15.47:0/242078191 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7be800a630 con 0x558cf4fdc9e0 2024-04-17T06:04:01.835 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.831+0000 7f7bddffb640 1 -- 172.21.15.47:0/242078191 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7be00be7d0 con 0x558cf4fdc9e0 2024-04-17T06:04:01.835 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.831+0000 7f7bff5bb740 10 client.15639 fetch_fsmap learned FSMap version 25 2024-04-17T06:04:01.835 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.831+0000 7f7bff5bb740 1 -- 172.21.15.47:0/242078191 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558cf505bea0 con 0x558cf4fdc9e0 2024-04-17T06:04:01.835 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.831+0000 7f7bddffb640 1 -- 172.21.15.47:0/242078191 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7be800a630 con 0x558cf4fdc9e0 2024-04-17T06:04:01.835 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.831+0000 7f7bddffb640 1 -- 172.21.15.47:0/242078191 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 25) ==== 8343+0+0 (secure 0 0 0) 0x7f7be00892e0 con 0x558cf4fdc9e0 2024-04-17T06:04:01.835 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.831+0000 7f7bff5bb740 10 client.15639 fetch_fsmap finished waiting for FSMap version 25 2024-04-17T06:04:01.835 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.831+0000 7f7bff5bb740 10 client.15639 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-17T06:04:01.835 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.831+0000 7f7bff5bb740 20 client.15639 populate_metadata read hostname 'smithi047' 2024-04-17T06:04:01.836 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.831+0000 7f7bff5bb740 1 --2- 172.21.15.47:0/242078191 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x558cf505c430 0x558cf505e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:04:01.836 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.831+0000 7f7bff5bb740 4 client.15639 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-17T06:04:01.836 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.831+0000 7f7bff5bb740 1 -- 172.21.15.47:0/242078191 --> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558cf4f49060 con 0x558cf505c430 2024-04-17T06:04:01.836 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.831+0000 7f7bf659c640 1 --2- 172.21.15.47:0/242078191 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x558cf505c430 0x558cf505e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:04:01.836 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.832+0000 7f7bddffb640 10 client.15639 ms_handle_connect on v2:172.21.15.98:6838/2841287917 2024-04-17T06:04:01.836 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:01.832+0000 7f7bf659c640 1 --2- 172.21.15.47:0/242078191 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x558cf505c430 0x558cf505e820 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c62a30d8df3443d6 server_cookie=7d496aca48e046cb in_seq=0 out_seq=0 2024-04-17T06:04:03.527 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:03.529+0000 7f7bddffb640 1 -- 172.21.15.47:0/242078191 <== mds.0 v2:172.21.15.98:6838/2841287917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558cf4f49060 con 0x558cf505c430 2024-04-17T06:04:03.528 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:03.529+0000 7f7bddffb640 10 client.15639 handle_command_reply: tid=0 2024-04-17T06:04:03.528 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:03.529+0000 7f7bff5bb740 1 client.15639 shutdown 2024-04-17T06:04:03.528 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:03.529+0000 7f7bff5bb740 1 -- 172.21.15.47:0/242078191 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558cf4fdc9e0 msgr2=0x558cf4fdf220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:04:03.528 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:03.529+0000 7f7bff5bb740 1 --2- 172.21.15.47:0/242078191 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558cf4fdc9e0 0x558cf4fdf220 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f7be00f0260 tx=0x7f7be0005be0 comp rx=0 tx=0).stop 2024-04-17T06:04:03.528 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:03.529+0000 7f7bff5bb740 1 -- 172.21.15.47:0/242078191 shutdown_connections 2024-04-17T06:04:03.528 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:03.529+0000 7f7bff5bb740 1 --2- 172.21.15.47:0/242078191 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x558cf505c430 0x558cf505e820 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:03.528 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:03.529+0000 7f7bff5bb740 1 --2- 172.21.15.47:0/242078191 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558cf4fe57b0 0x558cf4fdf910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:03.528 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:03.529+0000 7f7bff5bb740 1 --2- 172.21.15.47:0/242078191 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558cf4fdc9e0 0x558cf4fdf220 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:03.528 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:03.529+0000 7f7bddffb640 0 client.15639 ms_handle_reset on v2:172.21.15.98:6838/2841287917 2024-04-17T06:04:03.528 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:03.529+0000 7f7bff5bb740 1 --2- 172.21.15.47:0/242078191 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558cf4fdc0c0 0x558cf4fedcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:03.528 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:03.529+0000 7f7bff5bb740 1 -- 172.21.15.47:0/242078191 >> 172.21.15.47:0/242078191 conn(0x558cf4fd69e0 msgr2=0x558cf4fd5270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:04:03.528 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:03.529+0000 7f7bff5bb740 1 -- 172.21.15.47:0/242078191 shutdown_connections 2024-04-17T06:04:03.528 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:03.529+0000 7f7bff5bb740 1 -- 172.21.15.47:0/242078191 wait complete. 2024-04-17T06:04:03.528 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:03.529+0000 7f7bff5bb740 20 client.15639 trim_cache size 0 max 16384 2024-04-17T06:04:03.529 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-04-17T06:04:03.529 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-04-17T06:04:03.529 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-04-17T06:04:03.529 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-04-17T06:04:03.530 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:03.530+0000 7f7bfc9cd640 1 -- 172.21.15.47:0/3287777041 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f7be40a3dc0 msgr2=0x7f7be40a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:04:03.530 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:03.530+0000 7f7bfc9cd640 1 --2- 172.21.15.47:0/3287777041 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f7be40a3dc0 0x7f7be40a6280 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7bf0005cc0 tx=0x7f7bf000a340 comp rx=0 tx=0).stop 2024-04-17T06:04:03.530 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:03.530+0000 7f7bfc9cd640 1 -- 172.21.15.47:0/3287777041 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7bf8103b10 msgr2=0x7f7bf819eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:04:03.530 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:03.530+0000 7f7bfc9cd640 1 --2- 172.21.15.47:0/3287777041 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7bf8103b10 0x7f7bf819eac0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f7be0004f50 tx=0x7f7be002c860 comp rx=0 tx=0).stop 2024-04-17T06:04:03.530 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:03.530+0000 7f7bfc9cd640 1 -- 172.21.15.47:0/3287777041 shutdown_connections 2024-04-17T06:04:03.530 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:03.530+0000 7f7bfc9cd640 1 --2- 172.21.15.47:0/3287777041 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7bf810c530 0x7f7bf8198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:03.530 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:03.530+0000 7f7bfc9cd640 1 --2- 172.21.15.47:0/3287777041 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f7be40a3dc0 0x7f7be40a6280 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:03.530 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:03.530+0000 7f7bfc9cd640 1 --2- 172.21.15.47:0/3287777041 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7bf8103b10 0x7f7bf819eac0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:03.530 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:03.530+0000 7f7bfc9cd640 1 --2- 172.21.15.47:0/3287777041 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7bf81031f0 0x7f7bf819e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:03.530 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:03.530+0000 7f7bfc9cd640 1 -- 172.21.15.47:0/3287777041 >> 172.21.15.47:0/3287777041 conn(0x7f7bf8077fe0 msgr2=0x7f7bf80feff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:04:03.530 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:03.530+0000 7f7bfc9cd640 1 -- 172.21.15.47:0/3287777041 shutdown_connections 2024-04-17T06:04:03.531 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:03.530+0000 7f7bfc9cd640 1 -- 172.21.15.47:0/3287777041 wait complete. 2024-04-17T06:04:03.632 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:04:03 smithi098 ceph-mon[36516]: pgmap v86: 97 pgs: 97 active+clean; 1.8 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.7 KiB/s wr, 9 op/s 2024-04-17T06:04:03.638 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:04:03 smithi047 ceph-mon[32223]: pgmap v86: 97 pgs: 97 active+clean; 1.8 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.7 KiB/s wr, 9 op/s 2024-04-17T06:04:03.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:04:03 smithi103 ceph-mon[36243]: pgmap v86: 97 pgs: 97 active+clean; 1.8 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.7 KiB/s wr, 9 op/s 2024-04-17T06:04:04.361 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:04:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[70387]: *** Caught signal (Segmentation fault) ** 2024-04-17T06:04:04.361 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:04:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[70387]: in thread 7f8277f12700 thread_name:md_log_replay 2024-04-17T06:04:04.361 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:04:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[70387]: ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev) 2024-04-17T06:04:04.362 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:04:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[70387]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f82892d3d20] 2024-04-17T06:04:04.362 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:04:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[70387]: 2: /usr/bin/ceph-mds(+0x54962f) [0x55a4dd32b62f] 2024-04-17T06:04:04.362 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:04:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[70387]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55a4dd32edc3] 2024-04-17T06:04:04.362 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:04:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[70387]: 4: (MDLog::_replay_thread()+0x617) [0x55a4dd3898e7] 2024-04-17T06:04:04.362 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:04:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[70387]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55a4dd392fd1] 2024-04-17T06:04:04.362 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:04:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[70387]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f82892c91ca] 2024-04-17T06:04:04.362 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:04:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[70387]: 7: clone() 2024-04-17T06:04:04.362 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:04:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[70387]: 2024-04-17T06:04:04.002+0000 7f8277f12700 -1 *** Caught signal (Segmentation fault) ** 2024-04-17T06:04:04.362 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:04:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[70387]: in thread 7f8277f12700 thread_name:md_log_replay 2024-04-17T06:04:04.362 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:04:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[70387]: 2024-04-17T06:04:04.362 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:04:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[70387]: ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev) 2024-04-17T06:04:04.362 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:04:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[70387]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f82892d3d20] 2024-04-17T06:04:04.363 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:04:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[70387]: 2: /usr/bin/ceph-mds(+0x54962f) [0x55a4dd32b62f] 2024-04-17T06:04:04.363 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:04:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[70387]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55a4dd32edc3] 2024-04-17T06:04:04.363 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:04:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[70387]: 4: (MDLog::_replay_thread()+0x617) [0x55a4dd3898e7] 2024-04-17T06:04:04.363 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:04:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[70387]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55a4dd392fd1] 2024-04-17T06:04:04.363 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:04:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[70387]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f82892c91ca] 2024-04-17T06:04:04.363 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:04:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[70387]: 7: clone() 2024-04-17T06:04:04.363 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:04:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[70387]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-17T06:04:04.363 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:04:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[70387]: 2024-04-17T06:04:04.363 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:04:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[70387]: 0> 2024-04-17T06:04:04.002+0000 7f8277f12700 -1 *** Caught signal (Segmentation fault) ** 2024-04-17T06:04:04.363 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:04:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[70387]: in thread 7f8277f12700 thread_name:md_log_replay 2024-04-17T06:04:04.363 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:04:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[70387]: 2024-04-17T06:04:04.363 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:04:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[70387]: ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev) 2024-04-17T06:04:04.363 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:04:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[70387]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f82892d3d20] 2024-04-17T06:04:04.363 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:04:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[70387]: 2: /usr/bin/ceph-mds(+0x54962f) [0x55a4dd32b62f] 2024-04-17T06:04:04.364 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:04:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[70387]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55a4dd32edc3] 2024-04-17T06:04:04.364 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:04:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[70387]: 4: (MDLog::_replay_thread()+0x617) [0x55a4dd3898e7] 2024-04-17T06:04:04.364 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:04:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[70387]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55a4dd392fd1] 2024-04-17T06:04:04.364 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:04:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[70387]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f82892c91ca] 2024-04-17T06:04:04.364 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:04:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[70387]: 7: clone() 2024-04-17T06:04:04.364 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:04:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[70387]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-17T06:04:04.364 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:04:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[70387]: 2024-04-17T06:04:04.364 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:04:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[70387]: 0> 2024-04-17T06:04:04.002+0000 7f8277f12700 -1 *** Caught signal (Segmentation fault) ** 2024-04-17T06:04:04.364 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:04:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[70387]: in thread 7f8277f12700 thread_name:md_log_replay 2024-04-17T06:04:04.364 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:04:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[70387]: 2024-04-17T06:04:04.364 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:04:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[70387]: ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev) 2024-04-17T06:04:04.364 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:04:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[70387]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f82892d3d20] 2024-04-17T06:04:04.364 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:04:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[70387]: 2: /usr/bin/ceph-mds(+0x54962f) [0x55a4dd32b62f] 2024-04-17T06:04:04.364 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:04:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[70387]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55a4dd32edc3] 2024-04-17T06:04:04.364 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:04:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[70387]: 4: (MDLog::_replay_thread()+0x617) [0x55a4dd3898e7] 2024-04-17T06:04:04.365 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:04:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[70387]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55a4dd392fd1] 2024-04-17T06:04:04.365 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:04:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[70387]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f82892c91ca] 2024-04-17T06:04:04.365 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:04:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[70387]: 7: clone() 2024-04-17T06:04:04.365 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:04:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[70387]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-17T06:04:04.365 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:04:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[70387]: 2024-04-17T06:04:04.365 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:04:04 smithi098 podman[70803]: 2024-04-17 06:04:04.052323462 +0000 UTC m=+0.021208123 container died 65048e80ea08b24da1ad9d393b8516b29f3df5e0d540ee31d12615b0b7386ef5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-17T06:04:05.132 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:04:04 smithi098 podman[70803]: 2024-04-17 06:04:04.748990883 +0000 UTC m=+0.717875533 container remove 65048e80ea08b24da1ad9d393b8516b29f3df5e0d540ee31d12615b0b7386ef5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-17T06:04:05.133 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:04:04 smithi098 systemd[1]: ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@mds.e.service: Main process exited, code=exited, status=139/n/a 2024-04-17T06:04:05.133 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:04:04 smithi098 systemd[1]: ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@mds.e.service: Failed with result 'exit-code'. 2024-04-17T06:04:05.640 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:04:05 smithi047 ceph-mon[32223]: pgmap v87: 97 pgs: 97 active+clean; 1.8 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 35 KiB/s wr, 31 op/s 2024-04-17T06:04:05.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:04:05 smithi103 ceph-mon[36243]: pgmap v87: 97 pgs: 97 active+clean; 1.8 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 35 KiB/s wr, 31 op/s 2024-04-17T06:04:05.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:04:05 smithi098 ceph-mon[36516]: pgmap v87: 97 pgs: 97 active+clean; 1.8 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 35 KiB/s wr, 31 op/s 2024-04-17T06:04:07.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:04:07 smithi047 ceph-mon[32223]: pgmap v88: 97 pgs: 97 active+clean; 1.8 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 31 KiB/s wr, 26 op/s 2024-04-17T06:04:07.783 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:04:07 smithi103 ceph-mon[36243]: pgmap v88: 97 pgs: 97 active+clean; 1.8 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 31 KiB/s wr, 26 op/s 2024-04-17T06:04:07.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:04:07 smithi098 ceph-mon[36516]: pgmap v88: 97 pgs: 97 active+clean; 1.8 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 31 KiB/s wr, 26 op/s 2024-04-17T06:04:09.783 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:04:09 smithi103 ceph-mon[36243]: pgmap v89: 97 pgs: 97 active+clean; 1.8 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 31 KiB/s wr, 26 op/s 2024-04-17T06:04:09.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:04:09 smithi098 ceph-mon[36516]: pgmap v89: 97 pgs: 97 active+clean; 1.8 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 31 KiB/s wr, 26 op/s 2024-04-17T06:04:09.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:04:09 smithi047 ceph-mon[32223]: pgmap v89: 97 pgs: 97 active+clean; 1.8 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 31 KiB/s wr, 26 op/s 2024-04-17T06:04:11.783 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:04:11 smithi103 ceph-mon[36243]: pgmap v90: 97 pgs: 97 active+clean; 1.8 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 34 KiB/s wr, 30 op/s 2024-04-17T06:04:11.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:04:11 smithi098 ceph-mon[36516]: pgmap v90: 97 pgs: 97 active+clean; 1.8 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 34 KiB/s wr, 30 op/s 2024-04-17T06:04:11.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:04:11 smithi047 ceph-mon[32223]: pgmap v90: 97 pgs: 97 active+clean; 1.8 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 34 KiB/s wr, 30 op/s 2024-04-17T06:04:13.783 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:04:13 smithi103 ceph-mon[36243]: pgmap v91: 97 pgs: 97 active+clean; 1.8 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 31 KiB/s wr, 25 op/s 2024-04-17T06:04:13.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:04:13 smithi098 ceph-mon[36516]: pgmap v91: 97 pgs: 97 active+clean; 1.8 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 31 KiB/s wr, 25 op/s 2024-04-17T06:04:13.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:04:13 smithi047 ceph-mon[32223]: pgmap v91: 97 pgs: 97 active+clean; 1.8 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 31 KiB/s wr, 25 op/s 2024-04-17T06:04:15.397 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:04:15 smithi098 systemd[1]: ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@mds.e.service: Scheduled restart job, restart counter is at 2. 2024-04-17T06:04:15.397 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:04:15 smithi098 systemd[1]: Stopped Ceph mds.e for 565db2d8-fc7e-11ee-bc91-c7b262605968. 2024-04-17T06:04:15.397 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:04:15 smithi098 systemd[1]: Starting Ceph mds.e for 565db2d8-fc7e-11ee-bc91-c7b262605968... 2024-04-17T06:04:15.397 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:04:15 smithi098 podman[70898]: 2024-04-17 06:04:15.275935913 +0000 UTC m=+0.018463921 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-17T06:04:15.397 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:04:15 smithi098 podman[70898]: 2024-04-17 06:04:15.399791359 +0000 UTC m=+0.142319351 container create 7f9c4de4da23047896a6ce319c90b24489fd46448e31d3c2037eb0332d37acae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux ) 2024-04-17T06:04:15.650 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:04:15 smithi098 ceph-mon[36516]: pgmap v92: 97 pgs: 97 active+clean; 1.8 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 35 KiB/s wr, 29 op/s 2024-04-17T06:04:15.650 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:04:15 smithi098 podman[70898]: 2024-04-17 06:04:15.628185686 +0000 UTC m=+0.370713678 container init 7f9c4de4da23047896a6ce319c90b24489fd46448e31d3c2037eb0332d37acae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_BRANCH=HEAD) 2024-04-17T06:04:15.650 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:04:15 smithi098 podman[70898]: 2024-04-17 06:04:15.630849735 +0000 UTC m=+0.373377723 container start 7f9c4de4da23047896a6ce319c90b24489fd46448e31d3c2037eb0332d37acae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-04-17T06:04:15.783 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:04:15 smithi103 ceph-mon[36243]: pgmap v92: 97 pgs: 97 active+clean; 1.8 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 35 KiB/s wr, 29 op/s 2024-04-17T06:04:15.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:04:15 smithi047 ceph-mon[32223]: pgmap v92: 97 pgs: 97 active+clean; 1.8 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 35 KiB/s wr, 29 op/s 2024-04-17T06:04:16.132 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:04:15 smithi098 ceph-mds[70917]: -- 172.21.15.98:0/2732916000 <== mon.2 v2:172.21.15.98:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56362319fa40 con 0x563623172c00 2024-04-17T06:04:16.133 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:04:15 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[70913]: starting mds.e at 2024-04-17T06:04:16.133 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:04:15 smithi098 bash[70898]: 7f9c4de4da23047896a6ce319c90b24489fd46448e31d3c2037eb0332d37acae 2024-04-17T06:04:16.133 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:04:15 smithi098 systemd[1]: Started Ceph mds.e for 565db2d8-fc7e-11ee-bc91-c7b262605968. 2024-04-17T06:04:16.635 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.634+0000 7f6d25750640 1 -- 172.21.15.47:0/869256136 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d20103210 msgr2=0x7f6d201035f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:04:16.635 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.634+0000 7f6d25750640 1 --2- 172.21.15.47:0/869256136 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d20103210 0x7f6d201035f0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f6d10009900 tx=0x7f6d1002bcd0 comp rx=0 tx=0).stop 2024-04-17T06:04:16.635 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.634+0000 7f6d25750640 1 -- 172.21.15.47:0/869256136 shutdown_connections 2024-04-17T06:04:16.635 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.634+0000 7f6d25750640 1 --2- 172.21.15.47:0/869256136 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6d2010c550 0x7f6d2010e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:16.635 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.634+0000 7f6d25750640 1 --2- 172.21.15.47:0/869256136 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6d20103b30 0x7f6d2010c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:16.636 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.634+0000 7f6d25750640 1 --2- 172.21.15.47:0/869256136 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d20103210 0x7f6d201035f0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:16.636 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.634+0000 7f6d25750640 1 -- 172.21.15.47:0/869256136 >> 172.21.15.47:0/869256136 conn(0x7f6d20077fe0 msgr2=0x7f6d200783f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:04:16.636 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.634+0000 7f6d25750640 1 -- 172.21.15.47:0/869256136 shutdown_connections 2024-04-17T06:04:16.636 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.634+0000 7f6d25750640 1 -- 172.21.15.47:0/869256136 wait complete. 2024-04-17T06:04:16.636 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.634+0000 7f6d25750640 1 Processor -- start 2024-04-17T06:04:16.636 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.635+0000 7f6d25750640 1 -- start start 2024-04-17T06:04:16.636 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.635+0000 7f6d25750640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6d20103210 0x7f6d2019e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:04:16.636 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.635+0000 7f6d25750640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6d20103b30 0x7f6d2019e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:04:16.636 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.635+0000 7f6d25750640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d2010c550 0x7f6d20198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:04:16.636 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.635+0000 7f6d25750640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f6d20112300 con 0x7f6d2010c550 2024-04-17T06:04:16.637 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.635+0000 7f6d25750640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6d20112180 con 0x7f6d20103b30 2024-04-17T06:04:16.637 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.635+0000 7f6d25750640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6d20112480 con 0x7f6d20103210 2024-04-17T06:04:16.637 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.635+0000 7f6d1f7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d2010c550 0x7f6d20198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:04:16.637 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.635+0000 7f6d1f7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d2010c550 0x7f6d20198620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35434/0 (socket says 172.21.15.47:35434) 2024-04-17T06:04:16.637 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.635+0000 7f6d1f7fe640 1 -- 172.21.15.47:0/2482998775 learned_addr learned my addr 172.21.15.47:0/2482998775 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:04:16.637 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.635+0000 7f6d1e7fc640 1 --2- 172.21.15.47:0/2482998775 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6d20103b30 0x7f6d2019e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:04:16.637 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.635+0000 7f6d1effd640 1 --2- 172.21.15.47:0/2482998775 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6d20103210 0x7f6d2019e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:04:16.637 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.635+0000 7f6d1e7fc640 1 -- 172.21.15.47:0/2482998775 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6d20103210 msgr2=0x7f6d2019e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:04:16.637 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.635+0000 7f6d1e7fc640 1 --2- 172.21.15.47:0/2482998775 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6d20103210 0x7f6d2019e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:16.637 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.635+0000 7f6d1e7fc640 1 -- 172.21.15.47:0/2482998775 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d2010c550 msgr2=0x7f6d20198620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:04:16.637 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.635+0000 7f6d1e7fc640 1 --2- 172.21.15.47:0/2482998775 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d2010c550 0x7f6d20198620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:16.637 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.635+0000 7f6d1e7fc640 1 -- 172.21.15.47:0/2482998775 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d20198da0 con 0x7f6d20103b30 2024-04-17T06:04:16.638 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.636+0000 7f6d1e7fc640 1 --2- 172.21.15.47:0/2482998775 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6d20103b30 0x7f6d2019e960 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6d1400e9f0 tx=0x7f6d1400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:04:16.638 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.636+0000 7f6d07fff640 1 -- 172.21.15.47:0/2482998775 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d1400ccc0 con 0x7f6d20103b30 2024-04-17T06:04:16.638 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.636+0000 7f6d25750640 1 -- 172.21.15.47:0/2482998775 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d20199090 con 0x7f6d20103b30 2024-04-17T06:04:16.638 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.636+0000 7f6d07fff640 1 -- 172.21.15.47:0/2482998775 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d14010910 con 0x7f6d20103b30 2024-04-17T06:04:16.638 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.636+0000 7f6d07fff640 1 -- 172.21.15.47:0/2482998775 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d14018960 con 0x7f6d20103b30 2024-04-17T06:04:16.638 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.636+0000 7f6d25750640 1 -- 172.21.15.47:0/2482998775 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d201a52e0 con 0x7f6d20103b30 2024-04-17T06:04:16.645 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.638+0000 7f6d07fff640 1 -- 172.21.15.47:0/2482998775 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6d14010ac0 con 0x7f6d20103b30 2024-04-17T06:04:16.646 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.639+0000 7f6d28414740 1 Processor -- start 2024-04-17T06:04:16.646 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.639+0000 7f6d28414740 1 -- start start 2024-04-17T06:04:16.646 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.639+0000 7f6d07fff640 1 --2- 172.21.15.47:0/2482998775 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f6d080a3dc0 0x7f6d080a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:04:16.646 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.639+0000 7f6d28414740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559a0e02f0c0 0x559a0e02f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:04:16.646 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.639+0000 7f6d28414740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559a0e02f9e0 0x559a0e038270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:04:16.646 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.639+0000 7f6d28414740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559a0e0387b0 0x559a0e03aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:04:16.646 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.639+0000 7f6d28414740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x559a0df947d0 con 0x559a0e0387b0 2024-04-17T06:04:16.646 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.639+0000 7f6d28414740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x559a0df31f70 con 0x559a0e02f0c0 2024-04-17T06:04:16.646 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.639+0000 7f6d28414740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x559a0df90790 con 0x559a0e02f9e0 2024-04-17T06:04:16.646 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.639+0000 7f6d1effd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559a0e02f0c0 0x559a0e02f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:04:16.646 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.639+0000 7f6d1effd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559a0e02f0c0 0x559a0e02f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.47:51714/0 (socket says 172.21.15.47:51714) 2024-04-17T06:04:16.646 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.639+0000 7f6d1f7fe640 1 --2- 172.21.15.47:0/2482998775 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f6d080a3dc0 0x7f6d080a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:04:16.646 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.639+0000 7f6d1effd640 1 -- 172.21.15.47:0/799397903 learned_addr learned my addr 172.21.15.47:0/799397903 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:04:16.647 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.639+0000 7f6d07fff640 1 -- 172.21.15.47:0/2482998775 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f6d140d0b60 con 0x7f6d20103b30 2024-04-17T06:04:16.647 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.639+0000 7f6d1e7fc640 1 --2- 172.21.15.47:0/799397903 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559a0e02f9e0 0x559a0e038270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:04:16.647 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.639+0000 7f6d1f7fe640 1 --2- 172.21.15.47:0/799397903 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559a0e0387b0 0x559a0e03aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:04:16.647 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.640+0000 7f6d1effd640 1 -- 172.21.15.47:0/799397903 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559a0e02f9e0 msgr2=0x559a0e038270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:04:16.647 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.640+0000 7f6d1effd640 1 --2- 172.21.15.47:0/799397903 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559a0e02f9e0 0x559a0e038270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:16.647 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.640+0000 7f6d1effd640 1 -- 172.21.15.47:0/799397903 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559a0e0387b0 msgr2=0x559a0e03aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:04:16.647 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.640+0000 7f6d1effd640 1 --2- 172.21.15.47:0/799397903 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559a0e0387b0 0x559a0e03aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:16.647 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.640+0000 7f6d1effd640 1 -- 172.21.15.47:0/799397903 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a0dca7210 con 0x559a0e02f0c0 2024-04-17T06:04:16.647 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.640+0000 7f6d1f7fe640 1 --2- 172.21.15.47:0/799397903 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559a0e0387b0 0x559a0e03aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:04:16.647 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.640+0000 7f6d1f7fe640 1 --2- 172.21.15.47:0/2482998775 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f6d080a3dc0 0x7f6d080a6280 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x559a0e02ae00 tx=0x7f6d18009b60 comp rx=0 tx=0).ready entity=mgr.24923 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:04:16.647 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.640+0000 7f6d1effd640 1 --2- 172.21.15.47:0/799397903 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559a0e02f0c0 0x559a0e02f4a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6d10009230 tx=0x7f6d10005bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12ae55c89c4aba45 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:04:16.647 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.640+0000 7f6d057fa640 1 -- 172.21.15.47:0/799397903 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d10042070 con 0x559a0e02f0c0 2024-04-17T06:04:16.647 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.640+0000 7f6d057fa640 1 -- 172.21.15.47:0/799397903 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d1002c730 con 0x559a0e02f0c0 2024-04-17T06:04:16.647 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.640+0000 7f6d057fa640 1 -- 172.21.15.47:0/799397903 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d10004f10 con 0x559a0e02f0c0 2024-04-17T06:04:16.647 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.641+0000 7f6d28414740 1 -- 172.21.15.47:0/799397903 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559a0e02f0c0 msgr2=0x559a0e02f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:04:16.648 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.641+0000 7f6d28414740 1 --2- 172.21.15.47:0/799397903 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559a0e02f0c0 0x559a0e02f4a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6d10009230 tx=0x7f6d10005bf0 comp rx=0 tx=0).stop 2024-04-17T06:04:16.648 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.641+0000 7f6d28414740 1 -- 172.21.15.47:0/799397903 shutdown_connections 2024-04-17T06:04:16.648 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.641+0000 7f6d28414740 1 --2- 172.21.15.47:0/799397903 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559a0e0387b0 0x559a0e03aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:16.648 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.641+0000 7f6d28414740 1 --2- 172.21.15.47:0/799397903 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559a0e02f9e0 0x559a0e038270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:16.648 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.641+0000 7f6d28414740 1 --2- 172.21.15.47:0/799397903 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559a0e02f0c0 0x559a0e02f4a0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:16.648 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.641+0000 7f6d28414740 1 -- 172.21.15.47:0/799397903 >> 172.21.15.47:0/799397903 conn(0x559a0e0299e0 msgr2=0x559a0e029dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:04:16.648 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.641+0000 7f6d28414740 1 -- 172.21.15.47:0/799397903 shutdown_connections 2024-04-17T06:04:16.648 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.641+0000 7f6d28414740 1 -- 172.21.15.47:0/799397903 wait complete. 2024-04-17T06:04:16.648 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.641+0000 7f6d28414740 1 -- start start 2024-04-17T06:04:16.648 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.641+0000 7f6d28414740 1 Processor -- start 2024-04-17T06:04:16.648 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.642+0000 7f6d28414740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559a0e02f0c0 0x559a0e035790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:04:16.648 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.642+0000 7f6d28414740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559a0e02f9e0 0x559a0e035cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:04:16.648 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.642+0000 7f6d28414740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559a0e0387b0 0x559a0e036210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:04:16.648 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.642+0000 7f6d28414740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x559a0e03b380 con 0x559a0e0387b0 2024-04-17T06:04:16.649 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.642+0000 7f6d28414740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x559a0df1ce90 con 0x559a0e02f0c0 2024-04-17T06:04:16.649 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.642+0000 7f6d28414740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x559a0e03b500 con 0x559a0e02f9e0 2024-04-17T06:04:16.649 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.642+0000 7f6d28414740 10 client.0 _ll_register_callbacks cb 0x559a0df32d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:04:16.649 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.642+0000 7f6d1effd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559a0e02f0c0 0x559a0e035790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:04:16.649 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.642+0000 7f6d1f7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559a0e0387b0 0x559a0e036210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:04:16.649 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.642+0000 7f6d1effd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559a0e02f0c0 0x559a0e035790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.47:51726/0 (socket says 172.21.15.47:51726) 2024-04-17T06:04:16.649 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.642+0000 7f6d1effd640 1 -- 172.21.15.47:0/2247811618 learned_addr learned my addr 172.21.15.47:0/2247811618 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:04:16.649 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.642+0000 7f6d1e7fc640 1 --2- 172.21.15.47:0/2247811618 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559a0e02f9e0 0x559a0e035cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:04:16.649 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.642+0000 7f6d1f7fe640 1 -- 172.21.15.47:0/2247811618 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559a0e02f9e0 msgr2=0x559a0e035cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:04:16.649 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.642+0000 7f6d1f7fe640 1 --2- 172.21.15.47:0/2247811618 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559a0e02f9e0 0x559a0e035cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:16.649 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.642+0000 7f6d1f7fe640 1 -- 172.21.15.47:0/2247811618 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559a0e02f0c0 msgr2=0x559a0e035790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:04:16.649 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.642+0000 7f6d1f7fe640 1 --2- 172.21.15.47:0/2247811618 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559a0e02f0c0 0x559a0e035790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:16.650 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.642+0000 7f6d1f7fe640 1 -- 172.21.15.47:0/2247811618 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a0dd73620 con 0x559a0e0387b0 2024-04-17T06:04:16.650 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.642+0000 7f6d1e7fc640 1 --2- 172.21.15.47:0/2247811618 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559a0e02f9e0 0x559a0e035cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:04:16.650 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.642+0000 7f6d1effd640 1 --2- 172.21.15.47:0/2247811618 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559a0e02f0c0 0x559a0e035790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T06:04:16.650 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.642+0000 7f6d1f7fe640 1 --2- 172.21.15.47:0/2247811618 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559a0e0387b0 0x559a0e036210 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f6d18011af0 tx=0x7f6d18004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7d0550d3dc14824 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:04:16.650 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.642+0000 7f6d02ff5640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-17T06:04:16.650 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.643+0000 7f6d02ff5640 1 -- 172.21.15.47:0/2247811618 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d18026070 con 0x559a0e0387b0 2024-04-17T06:04:16.650 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.643+0000 7f6d28414740 1 -- 172.21.15.47:0/2247811618 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559a0dfa21f0 con 0x559a0e0387b0 2024-04-17T06:04:16.650 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.643+0000 7f6d02ff5640 1 -- 172.21.15.47:0/2247811618 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f6d18013020 con 0x559a0e0387b0 2024-04-17T06:04:16.650 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.643+0000 7f6d02ff5640 1 -- 172.21.15.47:0/2247811618 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6d18013220 con 0x559a0e0387b0 2024-04-17T06:04:16.650 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.643+0000 7f6d28414740 10 client.15651 fetch_fsmap learned FSMap version 27 2024-04-17T06:04:16.650 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.643+0000 7f6d28414740 1 -- 172.21.15.47:0/2247811618 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559a0e047680 con 0x559a0e0387b0 2024-04-17T06:04:16.650 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.643+0000 7f6d02ff5640 1 -- 172.21.15.47:0/2247811618 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x559a0dfa21f0 con 0x559a0e0387b0 2024-04-17T06:04:16.651 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.643+0000 7f6d02ff5640 1 -- 172.21.15.47:0/2247811618 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 27) ==== 8343+0+0 (secure 0 0 0) 0x7f6d1801d040 con 0x559a0e0387b0 2024-04-17T06:04:16.651 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.643+0000 7f6d28414740 10 client.15651 fetch_fsmap finished waiting for FSMap version 27 2024-04-17T06:04:16.651 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.643+0000 7f6d28414740 10 client.15651 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-17T06:04:16.651 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.643+0000 7f6d28414740 20 client.15651 populate_metadata read hostname 'smithi047' 2024-04-17T06:04:16.651 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.643+0000 7f6d28414740 1 --2- 172.21.15.47:0/2247811618 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x559a0e048c00 0x559a0e04b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:04:16.651 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.643+0000 7f6d28414740 4 client.15651 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T06:04:16.651 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.643+0000 7f6d28414740 1 -- 172.21.15.47:0/2247811618 --> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559a0df9c060 con 0x559a0e048c00 2024-04-17T06:04:16.651 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.643+0000 7f6d1effd640 1 --2- 172.21.15.47:0/2247811618 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x559a0e048c00 0x559a0e04b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:04:16.651 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.644+0000 7f6d02ff5640 10 client.15651 ms_handle_connect on v2:172.21.15.98:6838/2841287917 2024-04-17T06:04:16.651 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.644+0000 7f6d1effd640 1 --2- 172.21.15.47:0/2247811618 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x559a0e048c00 0x559a0e04b010 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b978bd6665d4fe91 server_cookie=c29d718a529fc801 in_seq=0 out_seq=0 2024-04-17T06:04:16.651 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.645+0000 7f6d02ff5640 1 -- 172.21.15.47:0/2247811618 <== mds.0 v2:172.21.15.98:6838/2841287917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559a0df9c060 con 0x559a0e048c00 2024-04-17T06:04:16.651 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.645+0000 7f6d02ff5640 10 client.15651 handle_command_reply: tid=0 2024-04-17T06:04:16.652 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.645+0000 7f6d28414740 1 client.15651 shutdown 2024-04-17T06:04:16.652 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.645+0000 7f6d28414740 1 -- 172.21.15.47:0/2247811618 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559a0e0387b0 msgr2=0x559a0e036210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:04:16.652 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.645+0000 7f6d28414740 1 --2- 172.21.15.47:0/2247811618 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559a0e0387b0 0x559a0e036210 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f6d18011af0 tx=0x7f6d18004ab0 comp rx=0 tx=0).stop 2024-04-17T06:04:16.652 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.645+0000 7f6d28414740 1 -- 172.21.15.47:0/2247811618 shutdown_connections 2024-04-17T06:04:16.652 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.645+0000 7f6d28414740 1 --2- 172.21.15.47:0/2247811618 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x559a0e048c00 0x559a0e04b010 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:16.652 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.645+0000 7f6d28414740 1 --2- 172.21.15.47:0/2247811618 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559a0e0387b0 0x559a0e036210 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:16.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.645+0000 7f6d28414740 1 --2- 172.21.15.47:0/2247811618 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559a0e02f9e0 0x559a0e035cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:16.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.645+0000 7f6d02ff5640 0 client.15651 ms_handle_reset on v2:172.21.15.98:6838/2841287917 2024-04-17T06:04:16.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.645+0000 7f6d28414740 1 --2- 172.21.15.47:0/2247811618 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559a0e02f0c0 0x559a0e035790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:16.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.645+0000 7f6d28414740 1 -- 172.21.15.47:0/2247811618 >> 172.21.15.47:0/2247811618 conn(0x559a0e0299e0 msgr2=0x559a0e02b730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:04:16.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.645+0000 7f6d28414740 1 -- 172.21.15.47:0/2247811618 shutdown_connections 2024-04-17T06:04:16.654 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.645+0000 7f6d28414740 1 -- 172.21.15.47:0/2247811618 wait complete. 2024-04-17T06:04:16.654 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.645+0000 7f6d28414740 20 client.15651 trim_cache size 0 max 16384 2024-04-17T06:04:16.668 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.660+0000 7f6d28414740 1 Processor -- start 2024-04-17T06:04:16.668 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.660+0000 7f6d28414740 1 -- start start 2024-04-17T06:04:16.668 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.660+0000 7f6d28414740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559a0e02f0c0 0x559a0e037ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:04:16.668 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.660+0000 7f6d28414740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559a0e02f9e0 0x559a0e044240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:04:16.668 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.660+0000 7f6d28414740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559a0e0387b0 0x559a0e028980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:04:16.668 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.660+0000 7f6d28414740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f6d201b03b0 con 0x559a0e0387b0 2024-04-17T06:04:16.668 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.660+0000 7f6d28414740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6d201adc40 con 0x559a0e02f9e0 2024-04-17T06:04:16.668 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.660+0000 7f6d28414740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6d201aba90 con 0x559a0e02f0c0 2024-04-17T06:04:16.668 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.660+0000 7f6d1f7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559a0e0387b0 0x559a0e028980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:04:16.668 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.660+0000 7f6d1e7fc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559a0e02f9e0 0x559a0e044240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:04:16.668 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.660+0000 7f6d1f7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559a0e0387b0 0x559a0e028980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35466/0 (socket says 172.21.15.47:35466) 2024-04-17T06:04:16.669 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.660+0000 7f6d1f7fe640 1 -- 172.21.15.47:0/636891527 learned_addr learned my addr 172.21.15.47:0/636891527 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:04:16.669 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.661+0000 7f6d1effd640 1 --2- 172.21.15.47:0/636891527 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559a0e02f0c0 0x559a0e037ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:04:16.669 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.661+0000 7f6d1f7fe640 1 -- 172.21.15.47:0/636891527 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559a0e02f0c0 msgr2=0x559a0e037ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:04:16.669 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.661+0000 7f6d1f7fe640 1 --2- 172.21.15.47:0/636891527 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559a0e02f0c0 0x559a0e037ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:16.669 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.661+0000 7f6d1f7fe640 1 -- 172.21.15.47:0/636891527 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559a0e02f9e0 msgr2=0x559a0e044240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:04:16.669 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.661+0000 7f6d1f7fe640 1 --2- 172.21.15.47:0/636891527 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559a0e02f9e0 0x559a0e044240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:16.669 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.661+0000 7f6d1f7fe640 1 -- 172.21.15.47:0/636891527 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a0e04b690 con 0x559a0e0387b0 2024-04-17T06:04:16.669 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.661+0000 7f6d1effd640 1 --2- 172.21.15.47:0/636891527 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559a0e02f0c0 0x559a0e037ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:04:16.669 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.661+0000 7f6d1e7fc640 1 --2- 172.21.15.47:0/636891527 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559a0e02f9e0 0x559a0e044240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T06:04:16.669 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.661+0000 7f6d1f7fe640 1 --2- 172.21.15.47:0/636891527 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559a0e0387b0 0x559a0e028980 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f6d18011840 tx=0x7f6d18005bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53dbc737347a5929 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:04:16.669 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.661+0000 7f6d04ff9640 1 -- 172.21.15.47:0/636891527 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d18037030 con 0x559a0e0387b0 2024-04-17T06:04:16.669 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.661+0000 7f6d04ff9640 1 -- 172.21.15.47:0/636891527 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d18009cb0 con 0x559a0e0387b0 2024-04-17T06:04:16.669 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.661+0000 7f6d04ff9640 1 -- 172.21.15.47:0/636891527 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d18026070 con 0x559a0e0387b0 2024-04-17T06:04:16.670 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.662+0000 7f6d28414740 1 -- 172.21.15.47:0/636891527 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559a0e0387b0 msgr2=0x559a0e028980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:04:16.670 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.662+0000 7f6d28414740 1 --2- 172.21.15.47:0/636891527 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559a0e0387b0 0x559a0e028980 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f6d18011840 tx=0x7f6d18005bf0 comp rx=0 tx=0).stop 2024-04-17T06:04:16.670 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.662+0000 7f6d28414740 1 -- 172.21.15.47:0/636891527 shutdown_connections 2024-04-17T06:04:16.670 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.662+0000 7f6d28414740 1 --2- 172.21.15.47:0/636891527 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559a0e0387b0 0x559a0e028980 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:16.670 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.662+0000 7f6d28414740 1 --2- 172.21.15.47:0/636891527 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559a0e02f9e0 0x559a0e044240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:16.670 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.662+0000 7f6d28414740 1 --2- 172.21.15.47:0/636891527 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559a0e02f0c0 0x559a0e037ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:16.670 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.662+0000 7f6d28414740 1 -- 172.21.15.47:0/636891527 >> 172.21.15.47:0/636891527 conn(0x559a0e0299e0 msgr2=0x559a0e028270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:04:16.670 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.662+0000 7f6d28414740 1 -- 172.21.15.47:0/636891527 shutdown_connections 2024-04-17T06:04:16.670 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.662+0000 7f6d28414740 1 -- 172.21.15.47:0/636891527 wait complete. 2024-04-17T06:04:16.670 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.662+0000 7f6d28414740 1 -- start start 2024-04-17T06:04:16.670 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.662+0000 7f6d28414740 1 Processor -- start 2024-04-17T06:04:16.670 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.662+0000 7f6d28414740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559a0e02f0c0 0x559a0e032400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:04:16.670 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.662+0000 7f6d28414740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559a0e02f9e0 0x559a0e032940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:04:16.671 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.662+0000 7f6d28414740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559a0e0387b0 0x559a0e032e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:04:16.671 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.662+0000 7f6d28414740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f6d201ac100 con 0x559a0e0387b0 2024-04-17T06:04:16.671 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.662+0000 7f6d28414740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6d201ad570 con 0x559a0e02f9e0 2024-04-17T06:04:16.671 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.662+0000 7f6d28414740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6d201ac7d0 con 0x559a0e02f0c0 2024-04-17T06:04:16.671 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.663+0000 7f6d28414740 10 client.0 _ll_register_callbacks cb 0x7f6d18013540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:04:16.671 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.663+0000 7f6d1f7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559a0e0387b0 0x559a0e032e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:04:16.671 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.663+0000 7f6d1f7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559a0e0387b0 0x559a0e032e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35472/0 (socket says 172.21.15.47:35472) 2024-04-17T06:04:16.671 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.663+0000 7f6d1f7fe640 1 -- 172.21.15.47:0/3664716027 learned_addr learned my addr 172.21.15.47:0/3664716027 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:04:16.671 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.663+0000 7f6d1e7fc640 1 --2- 172.21.15.47:0/3664716027 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559a0e02f9e0 0x559a0e032940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:04:16.671 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.663+0000 7f6d1effd640 1 --2- 172.21.15.47:0/3664716027 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559a0e02f0c0 0x559a0e032400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:04:16.671 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.663+0000 7f6d1f7fe640 1 -- 172.21.15.47:0/3664716027 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559a0e02f0c0 msgr2=0x559a0e032400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:04:16.671 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.663+0000 7f6d1f7fe640 1 --2- 172.21.15.47:0/3664716027 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559a0e02f0c0 0x559a0e032400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:16.671 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.663+0000 7f6d1f7fe640 1 -- 172.21.15.47:0/3664716027 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559a0e02f9e0 msgr2=0x559a0e032940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:04:16.671 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.663+0000 7f6d1f7fe640 1 --2- 172.21.15.47:0/3664716027 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559a0e02f9e0 0x559a0e032940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:16.672 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.663+0000 7f6d1f7fe640 1 -- 172.21.15.47:0/3664716027 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a0e03aba0 con 0x559a0e0387b0 2024-04-17T06:04:16.672 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.663+0000 7f6d1f7fe640 1 --2- 172.21.15.47:0/3664716027 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559a0e0387b0 0x559a0e032e80 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f6d180096e0 tx=0x7f6d1801ffa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33102d2209c62981 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:04:16.672 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.663+0000 7f6d02ff5640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-17T06:04:16.672 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.663+0000 7f6d02ff5640 1 -- 172.21.15.47:0/3664716027 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d18026040 con 0x559a0e0387b0 2024-04-17T06:04:16.672 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.663+0000 7f6d28414740 1 -- 172.21.15.47:0/3664716027 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d18009cb0 con 0x559a0e0387b0 2024-04-17T06:04:16.672 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.663+0000 7f6d02ff5640 1 -- 172.21.15.47:0/3664716027 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f6d1802a650 con 0x559a0e0387b0 2024-04-17T06:04:16.672 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.663+0000 7f6d02ff5640 1 -- 172.21.15.47:0/3664716027 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6d1804a050 con 0x559a0e0387b0 2024-04-17T06:04:16.672 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.663+0000 7f6d28414740 10 client.15657 fetch_fsmap learned FSMap version 27 2024-04-17T06:04:16.672 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.664+0000 7f6d28414740 1 -- 172.21.15.47:0/3664716027 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559a0e0af180 con 0x559a0e0387b0 2024-04-17T06:04:16.672 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.664+0000 7f6d02ff5640 1 -- 172.21.15.47:0/3664716027 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d18009cb0 con 0x559a0e0387b0 2024-04-17T06:04:16.672 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.664+0000 7f6d02ff5640 1 -- 172.21.15.47:0/3664716027 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 27) ==== 8343+0+0 (secure 0 0 0) 0x7f6d1801d030 con 0x559a0e0387b0 2024-04-17T06:04:16.672 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.664+0000 7f6d28414740 10 client.15657 fetch_fsmap finished waiting for FSMap version 27 2024-04-17T06:04:16.672 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.664+0000 7f6d28414740 10 client.15657 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-17T06:04:16.673 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.664+0000 7f6d28414740 20 client.15657 populate_metadata read hostname 'smithi047' 2024-04-17T06:04:16.673 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.664+0000 7f6d28414740 1 --2- 172.21.15.47:0/3664716027 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x559a0e0af7b0 0x559a0e0b1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:04:16.673 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.664+0000 7f6d28414740 4 client.15657 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-17T06:04:16.673 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.664+0000 7f6d28414740 1 -- 172.21.15.47:0/3664716027 --> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559a0df18670 con 0x559a0e0af7b0 2024-04-17T06:04:16.673 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.664+0000 7f6d1effd640 1 --2- 172.21.15.47:0/3664716027 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x559a0e0af7b0 0x559a0e0b1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:04:16.673 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.664+0000 7f6d1effd640 1 --2- 172.21.15.47:0/3664716027 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x559a0e0af7b0 0x559a0e0b1bd0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6bc0448888e25ae server_cookie=6007af4630eb26fb in_seq=0 out_seq=0 2024-04-17T06:04:16.673 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.664+0000 7f6d02ff5640 10 client.15657 ms_handle_connect on v2:172.21.15.98:6838/2841287917 2024-04-17T06:04:16.675 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.671+0000 7f6d02ff5640 1 -- 172.21.15.47:0/3664716027 <== mds.0 v2:172.21.15.98:6838/2841287917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559a0df18670 con 0x559a0e0af7b0 2024-04-17T06:04:16.675 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.671+0000 7f6d02ff5640 10 client.15657 handle_command_reply: tid=0 2024-04-17T06:04:16.675 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.671+0000 7f6d28414740 1 client.15657 shutdown 2024-04-17T06:04:16.675 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.671+0000 7f6d28414740 1 -- 172.21.15.47:0/3664716027 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559a0e0387b0 msgr2=0x559a0e032e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:04:16.675 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.671+0000 7f6d28414740 1 --2- 172.21.15.47:0/3664716027 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559a0e0387b0 0x559a0e032e80 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f6d180096e0 tx=0x7f6d1801ffa0 comp rx=0 tx=0).stop 2024-04-17T06:04:16.675 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.671+0000 7f6d28414740 1 -- 172.21.15.47:0/3664716027 shutdown_connections 2024-04-17T06:04:16.675 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.671+0000 7f6d28414740 1 --2- 172.21.15.47:0/3664716027 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x559a0e0af7b0 0x559a0e0b1bd0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:16.675 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.671+0000 7f6d28414740 1 --2- 172.21.15.47:0/3664716027 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559a0e0387b0 0x559a0e032e80 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:16.675 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.671+0000 7f6d28414740 1 --2- 172.21.15.47:0/3664716027 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559a0e02f9e0 0x559a0e032940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:16.676 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.671+0000 7f6d28414740 1 --2- 172.21.15.47:0/3664716027 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559a0e02f0c0 0x559a0e032400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:16.676 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.671+0000 7f6d02ff5640 0 client.15657 ms_handle_reset on v2:172.21.15.98:6838/2841287917 2024-04-17T06:04:16.676 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.671+0000 7f6d28414740 1 -- 172.21.15.47:0/3664716027 >> 172.21.15.47:0/3664716027 conn(0x559a0e0299e0 msgr2=0x559a0e02b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:04:16.676 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.672+0000 7f6d28414740 1 -- 172.21.15.47:0/3664716027 shutdown_connections 2024-04-17T06:04:16.676 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.672+0000 7f6d28414740 1 -- 172.21.15.47:0/3664716027 wait complete. 2024-04-17T06:04:16.676 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.672+0000 7f6d28414740 20 client.15657 trim_cache size 0 max 16384 2024-04-17T06:04:16.676 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-04-17T06:04:16.676 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-04-17T06:04:16.676 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-04-17T06:04:16.676 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-04-17T06:04:16.677 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.672+0000 7f6d25750640 1 -- 172.21.15.47:0/2482998775 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f6d080a3dc0 msgr2=0x7f6d080a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:04:16.677 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.672+0000 7f6d25750640 1 --2- 172.21.15.47:0/2482998775 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f6d080a3dc0 0x7f6d080a6280 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x559a0e02ae00 tx=0x7f6d18009b60 comp rx=0 tx=0).stop 2024-04-17T06:04:16.678 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.672+0000 7f6d25750640 1 -- 172.21.15.47:0/2482998775 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6d20103b30 msgr2=0x7f6d2019e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:04:16.678 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.672+0000 7f6d25750640 1 --2- 172.21.15.47:0/2482998775 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6d20103b30 0x7f6d2019e960 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6d1400e9f0 tx=0x7f6d1400eeb0 comp rx=0 tx=0).stop 2024-04-17T06:04:16.678 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.672+0000 7f6d25750640 1 -- 172.21.15.47:0/2482998775 shutdown_connections 2024-04-17T06:04:16.678 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.672+0000 7f6d25750640 1 --2- 172.21.15.47:0/2482998775 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f6d080a3dc0 0x7f6d080a6280 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:16.678 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.672+0000 7f6d25750640 1 --2- 172.21.15.47:0/2482998775 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d2010c550 0x7f6d20198620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:16.678 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.672+0000 7f6d25750640 1 --2- 172.21.15.47:0/2482998775 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6d20103b30 0x7f6d2019e960 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:16.678 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.672+0000 7f6d25750640 1 --2- 172.21.15.47:0/2482998775 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6d20103210 0x7f6d2019e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:16.678 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.672+0000 7f6d25750640 1 -- 172.21.15.47:0/2482998775 >> 172.21.15.47:0/2482998775 conn(0x7f6d20077fe0 msgr2=0x7f6d200fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:04:16.678 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.672+0000 7f6d25750640 1 -- 172.21.15.47:0/2482998775 shutdown_connections 2024-04-17T06:04:16.678 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:16.672+0000 7f6d25750640 1 -- 172.21.15.47:0/2482998775 wait complete. 2024-04-17T06:04:16.783 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:04:16 smithi103 ceph-mon[36243]: daemon mds.e restarted 2024-04-17T06:04:16.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:04:16 smithi098 ceph-mon[36516]: daemon mds.e restarted 2024-04-17T06:04:16.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:04:16 smithi047 ceph-mon[32223]: daemon mds.e restarted 2024-04-17T06:04:17.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:04:17 smithi103 ceph-mon[36243]: pgmap v93: 97 pgs: 97 active+clean; 1.8 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 6.7 KiB/s wr, 7 op/s 2024-04-17T06:04:17.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:04:17 smithi103 ceph-mon[36243]: osdmap e91: 12 total, 12 up, 12 in 2024-04-17T06:04:17.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:04:17 smithi103 ceph-mon[36243]: mds.? [v2:172.21.15.98:6834/3055276908,v1:172.21.15.98:6835/3055276908] up:boot 2024-04-17T06:04:17.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:04:17 smithi103 ceph-mon[36243]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-04-17T06:04:17.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:04:17 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-17T06:04:17.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:04:17 smithi103 ceph-mon[36243]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-17T06:04:17.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:04:17 smithi047 ceph-mon[32223]: pgmap v93: 97 pgs: 97 active+clean; 1.8 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 6.7 KiB/s wr, 7 op/s 2024-04-17T06:04:17.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:04:17 smithi047 ceph-mon[32223]: osdmap e91: 12 total, 12 up, 12 in 2024-04-17T06:04:17.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:04:17 smithi047 ceph-mon[32223]: mds.? [v2:172.21.15.98:6834/3055276908,v1:172.21.15.98:6835/3055276908] up:boot 2024-04-17T06:04:17.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:04:17 smithi047 ceph-mon[32223]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-04-17T06:04:17.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:04:17 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-17T06:04:17.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:04:17 smithi047 ceph-mon[32223]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-17T06:04:17.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:04:17 smithi098 ceph-mon[36516]: pgmap v93: 97 pgs: 97 active+clean; 1.8 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 6.7 KiB/s wr, 7 op/s 2024-04-17T06:04:17.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:04:17 smithi098 ceph-mon[36516]: osdmap e91: 12 total, 12 up, 12 in 2024-04-17T06:04:17.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:04:17 smithi098 ceph-mon[36516]: mds.? [v2:172.21.15.98:6834/3055276908,v1:172.21.15.98:6835/3055276908] up:boot 2024-04-17T06:04:17.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:04:17 smithi098 ceph-mon[36516]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-04-17T06:04:17.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:04:17 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-17T06:04:17.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:04:17 smithi098 ceph-mon[36516]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-17T06:04:19.783 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:04:19 smithi103 ceph-mon[36243]: pgmap v95: 97 pgs: 97 active+clean; 1.8 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 8.0 KiB/s wr, 9 op/s 2024-04-17T06:04:19.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:04:19 smithi098 ceph-mon[36516]: pgmap v95: 97 pgs: 97 active+clean; 1.8 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 8.0 KiB/s wr, 9 op/s 2024-04-17T06:04:19.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:04:19 smithi047 ceph-mon[32223]: pgmap v95: 97 pgs: 97 active+clean; 1.8 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 8.0 KiB/s wr, 9 op/s 2024-04-17T06:04:21.783 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:04:21 smithi103 ceph-mon[36243]: pgmap v96: 97 pgs: 97 active+clean; 1.9 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-04-17T06:04:21.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:04:21 smithi098 ceph-mon[36516]: pgmap v96: 97 pgs: 97 active+clean; 1.9 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-04-17T06:04:21.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:04:21 smithi047 ceph-mon[32223]: pgmap v96: 97 pgs: 97 active+clean; 1.9 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-04-17T06:04:23.786 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:04:23 smithi103 ceph-mon[36243]: pgmap v97: 97 pgs: 97 active+clean; 1.9 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-04-17T06:04:23.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:04:23 smithi098 ceph-mon[36516]: pgmap v97: 97 pgs: 97 active+clean; 1.9 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-04-17T06:04:23.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:04:23 smithi047 ceph-mon[32223]: pgmap v97: 97 pgs: 97 active+clean; 1.9 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-04-17T06:04:25.783 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:04:25 smithi103 ceph-mon[36243]: pgmap v98: 97 pgs: 97 active+clean; 1.9 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-04-17T06:04:25.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:04:25 smithi098 ceph-mon[36516]: pgmap v98: 97 pgs: 97 active+clean; 1.9 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-04-17T06:04:25.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:04:25 smithi047 ceph-mon[32223]: pgmap v98: 97 pgs: 97 active+clean; 1.9 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-04-17T06:04:26.783 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:04:26 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T06:04:26.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:04:26 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T06:04:26.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:04:26 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T06:04:27.685 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:04:27 smithi103 ceph-mon[36243]: pgmap v99: 97 pgs: 97 active+clean; 1.9 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-04-17T06:04:27.685 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:04:27 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T06:04:27.685 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:04:27 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T06:04:27.685 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:04:27 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:04:27.784 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:04:27 smithi098 ceph-mon[36516]: pgmap v99: 97 pgs: 97 active+clean; 1.9 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-04-17T06:04:27.784 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:04:27 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T06:04:27.784 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:04:27 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T06:04:27.784 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:04:27 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:04:27.890 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:04:27 smithi047 ceph-mon[32223]: pgmap v99: 97 pgs: 97 active+clean; 1.9 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-04-17T06:04:27.890 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:04:27 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T06:04:27.890 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:04:27 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T06:04:27.890 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:04:27 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:04:29.771 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.772+0000 7f3e06915640 1 -- 172.21.15.47:0/2864264710 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3e001021e0 msgr2=0x7f3e001025c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:04:29.771 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.772+0000 7f3e06915640 1 --2- 172.21.15.47:0/2864264710 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3e001021e0 0x7f3e001025c0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f3df0009900 tx=0x7f3df002bc30 comp rx=0 tx=0).stop 2024-04-17T06:04:29.771 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.772+0000 7f3e06915640 1 -- 172.21.15.47:0/2864264710 shutdown_connections 2024-04-17T06:04:29.771 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.772+0000 7f3e06915640 1 --2- 172.21.15.47:0/2864264710 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3e001072b0 0x7f3e0010ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:29.771 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.772+0000 7f3e06915640 1 --2- 172.21.15.47:0/2864264710 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3e00102b90 0x7f3e00106bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:29.771 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.772+0000 7f3e06915640 1 --2- 172.21.15.47:0/2864264710 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3e001021e0 0x7f3e001025c0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:29.771 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.772+0000 7f3e06915640 1 -- 172.21.15.47:0/2864264710 >> 172.21.15.47:0/2864264710 conn(0x7f3e000793f0 msgr2=0x7f3e00079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:04:29.771 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.772+0000 7f3e06915640 1 -- 172.21.15.47:0/2864264710 shutdown_connections 2024-04-17T06:04:29.771 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.772+0000 7f3e06915640 1 -- 172.21.15.47:0/2864264710 wait complete. 2024-04-17T06:04:29.771 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.773+0000 7f3e06915640 1 Processor -- start 2024-04-17T06:04:29.772 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.773+0000 7f3e06915640 1 -- start start 2024-04-17T06:04:29.773 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.774+0000 7f3e06915640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3e001021e0 0x7f3e0019e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:04:29.774 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.774+0000 7f3e06915640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3e001072b0 0x7f3e0019e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:04:29.774 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.774+0000 7f3e06915640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3e001985c0 0x7f3e00198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:04:29.774 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.774+0000 7f3e06915640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f3e00112360 con 0x7f3e001021e0 2024-04-17T06:04:29.774 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.774+0000 7f3e06915640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3e001121e0 con 0x7f3e001985c0 2024-04-17T06:04:29.774 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.774+0000 7f3e06915640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3e001124e0 con 0x7f3e001072b0 2024-04-17T06:04:29.774 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.774+0000 7f3dfffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3e001021e0 0x7f3e0019e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:04:29.774 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.774+0000 7f3dfffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3e001021e0 0x7f3e0019e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40712/0 (socket says 172.21.15.47:40712) 2024-04-17T06:04:29.774 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.774+0000 7f3dfffff640 1 -- 172.21.15.47:0/3307560263 learned_addr learned my addr 172.21.15.47:0/3307560263 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:04:29.774 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.774+0000 7f3e04eb2640 1 --2- 172.21.15.47:0/3307560263 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3e001985c0 0x7f3e00198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:04:29.774 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.774+0000 7f3dff7fe640 1 --2- 172.21.15.47:0/3307560263 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3e001072b0 0x7f3e0019e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:04:29.774 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.774+0000 7f3dfffff640 1 -- 172.21.15.47:0/3307560263 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3e001072b0 msgr2=0x7f3e0019e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:04:29.774 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.774+0000 7f3dfffff640 1 --2- 172.21.15.47:0/3307560263 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3e001072b0 0x7f3e0019e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:29.774 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.774+0000 7f3dfffff640 1 -- 172.21.15.47:0/3307560263 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3e001985c0 msgr2=0x7f3e00198a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:04:29.775 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.774+0000 7f3dfffff640 1 --2- 172.21.15.47:0/3307560263 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3e001985c0 0x7f3e00198a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:29.775 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.774+0000 7f3dfffff640 1 -- 172.21.15.47:0/3307560263 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e001990e0 con 0x7f3e001021e0 2024-04-17T06:04:29.775 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.774+0000 7f3dfffff640 1 --2- 172.21.15.47:0/3307560263 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3e001021e0 0x7f3e0019e410 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f3df0006010 tx=0x7f3df002cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:04:29.775 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.774+0000 7f3dfd7fa640 1 -- 172.21.15.47:0/3307560263 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df0041070 con 0x7f3e001021e0 2024-04-17T06:04:29.775 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.775+0000 7f3dfd7fa640 1 -- 172.21.15.47:0/3307560263 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3df0004450 con 0x7f3e001021e0 2024-04-17T06:04:29.775 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.775+0000 7f3dfd7fa640 1 -- 172.21.15.47:0/3307560263 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df0034440 con 0x7f3e001021e0 2024-04-17T06:04:29.775 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.775+0000 7f3e06915640 1 -- 172.21.15.47:0/3307560263 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e00199370 con 0x7f3e001021e0 2024-04-17T06:04:29.775 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.775+0000 7f3e06915640 1 -- 172.21.15.47:0/3307560263 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e001a55c0 con 0x7f3e001021e0 2024-04-17T06:04:29.782 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.777+0000 7f3dfd7fa640 1 -- 172.21.15.47:0/3307560263 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3df0044070 con 0x7f3e001021e0 2024-04-17T06:04:29.782 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.777+0000 7f3e095dd740 1 Processor -- start 2024-04-17T06:04:29.782 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.778+0000 7f3e095dd740 1 -- start start 2024-04-17T06:04:29.782 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.778+0000 7f3e095dd740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563e5d0590c0 0x563e5d0594a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:04:29.782 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.778+0000 7f3e095dd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e5d0599e0 0x563e5d062270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:04:29.782 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.778+0000 7f3e095dd740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e5d0627b0 0x563e5d064ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:04:29.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.778+0000 7f3e095dd740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x563e5cfbe7d0 con 0x563e5d0627b0 2024-04-17T06:04:29.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.778+0000 7f3e095dd740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x563e5cf5bf70 con 0x563e5d0590c0 2024-04-17T06:04:29.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.778+0000 7f3e095dd740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x563e5cfba790 con 0x563e5d0599e0 2024-04-17T06:04:29.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.778+0000 7f3dfd7fa640 1 --2- 172.21.15.47:0/3307560263 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f3de80a3e40 0x7f3de80a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:04:29.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.778+0000 7f3dff7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e5d0627b0 0x563e5d064ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:04:29.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.778+0000 7f3dfffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e5d0599e0 0x563e5d062270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:04:29.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.778+0000 7f3dff7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e5d0627b0 0x563e5d064ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40722/0 (socket says 172.21.15.47:40722) 2024-04-17T06:04:29.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.778+0000 7f3e04eb2640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563e5d0590c0 0x563e5d0594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:04:29.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.778+0000 7f3dff7fe640 1 -- 172.21.15.47:0/3993494172 learned_addr learned my addr 172.21.15.47:0/3993494172 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:04:29.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.778+0000 7f3e04eb2640 1 --2- 172.21.15.47:0/3307560263 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f3de80a3e40 0x7f3de80a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:04:29.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.778+0000 7f3dfd7fa640 1 -- 172.21.15.47:0/3307560263 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f3df00f2230 con 0x7f3e001021e0 2024-04-17T06:04:29.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.778+0000 7f3dfffff640 1 -- 172.21.15.47:0/3993494172 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563e5d0590c0 msgr2=0x563e5d0594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:04:29.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.778+0000 7f3dfffff640 1 --2- 172.21.15.47:0/3993494172 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563e5d0590c0 0x563e5d0594a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:29.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.778+0000 7f3dfffff640 1 -- 172.21.15.47:0/3993494172 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e5d0627b0 msgr2=0x563e5d064ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:04:29.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.778+0000 7f3dfffff640 1 --2- 172.21.15.47:0/3993494172 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e5d0627b0 0x563e5d064ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:29.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.778+0000 7f3dfffff640 1 -- 172.21.15.47:0/3993494172 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e5ccd1210 con 0x563e5d0599e0 2024-04-17T06:04:29.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.778+0000 7f3dff7fe640 1 --2- 172.21.15.47:0/3993494172 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e5d0627b0 0x563e5d064ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T06:04:29.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.779+0000 7f3dfffff640 1 --2- 172.21.15.47:0/3993494172 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e5d0599e0 0x563e5d062270 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3df00efa60 tx=0x7f3df00be260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10d2fca9d4485a0d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:04:29.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.779+0000 7f3e04eb2640 1 --2- 172.21.15.47:0/3307560263 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f3de80a3e40 0x7f3de80a6300 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3df8009fd0 tx=0x7f3df8004ab0 comp rx=0 tx=0).ready entity=mgr.24923 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:04:29.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.779+0000 7f3de67fc640 1 -- 172.21.15.47:0/3993494172 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df004b070 con 0x563e5d0599e0 2024-04-17T06:04:29.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.779+0000 7f3de67fc640 1 -- 172.21.15.47:0/3993494172 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3df00c2660 con 0x563e5d0599e0 2024-04-17T06:04:29.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.779+0000 7f3de67fc640 1 -- 172.21.15.47:0/3993494172 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df00efbd0 con 0x563e5d0599e0 2024-04-17T06:04:29.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.780+0000 7f3e095dd740 1 -- 172.21.15.47:0/3993494172 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e5d0599e0 msgr2=0x563e5d062270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:04:29.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.780+0000 7f3e095dd740 1 --2- 172.21.15.47:0/3993494172 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e5d0599e0 0x563e5d062270 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3df00efa60 tx=0x7f3df00be260 comp rx=0 tx=0).stop 2024-04-17T06:04:29.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.780+0000 7f3e095dd740 1 -- 172.21.15.47:0/3993494172 shutdown_connections 2024-04-17T06:04:29.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.780+0000 7f3e095dd740 1 --2- 172.21.15.47:0/3993494172 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e5d0627b0 0x563e5d064ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:29.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.780+0000 7f3e095dd740 1 --2- 172.21.15.47:0/3993494172 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e5d0599e0 0x563e5d062270 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:29.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.780+0000 7f3e095dd740 1 --2- 172.21.15.47:0/3993494172 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563e5d0590c0 0x563e5d0594a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:29.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.780+0000 7f3e095dd740 1 -- 172.21.15.47:0/3993494172 >> 172.21.15.47:0/3993494172 conn(0x563e5d0539e0 msgr2=0x563e5d053dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:04:29.785 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.780+0000 7f3e095dd740 1 -- 172.21.15.47:0/3993494172 shutdown_connections 2024-04-17T06:04:29.785 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.780+0000 7f3e095dd740 1 -- 172.21.15.47:0/3993494172 wait complete. 2024-04-17T06:04:29.785 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.780+0000 7f3e095dd740 1 -- start start 2024-04-17T06:04:29.785 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.780+0000 7f3e095dd740 1 Processor -- start 2024-04-17T06:04:29.785 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.781+0000 7f3e095dd740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e5d0590c0 0x563e5d05f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:04:29.785 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.781+0000 7f3e095dd740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563e5d0599e0 0x563e5d05fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:04:29.785 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.781+0000 7f3e095dd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e5d0627b0 0x563e5d060300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:04:29.785 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.781+0000 7f3e095dd740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x563e5cfba790 con 0x563e5d0590c0 2024-04-17T06:04:29.785 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.781+0000 7f3e095dd740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x563e5cf46e90 con 0x563e5d0599e0 2024-04-17T06:04:29.785 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.781+0000 7f3e095dd740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x563e5d0651a0 con 0x563e5d0627b0 2024-04-17T06:04:29.785 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.781+0000 7f3e095dd740 10 client.0 _ll_register_callbacks cb 0x563e5cf5cd40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:04:29.785 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.781+0000 7f3dfffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e5d0590c0 0x563e5d05f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:04:29.785 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.781+0000 7f3dfffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e5d0590c0 0x563e5d05f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40724/0 (socket says 172.21.15.47:40724) 2024-04-17T06:04:29.785 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.781+0000 7f3dfffff640 1 -- 172.21.15.47:0/919901004 learned_addr learned my addr 172.21.15.47:0/919901004 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:04:29.786 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.781+0000 7f3dff7fe640 1 --2- 172.21.15.47:0/919901004 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563e5d0599e0 0x563e5d05fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:04:29.786 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.781+0000 7f3e04eb2640 1 --2- 172.21.15.47:0/919901004 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e5d0627b0 0x563e5d060300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:04:29.786 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.781+0000 7f3dfffff640 1 -- 172.21.15.47:0/919901004 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e5d0627b0 msgr2=0x563e5d060300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:04:29.786 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.781+0000 7f3dfffff640 1 --2- 172.21.15.47:0/919901004 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e5d0627b0 0x563e5d060300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:29.786 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.781+0000 7f3dfffff640 1 -- 172.21.15.47:0/919901004 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563e5d0599e0 msgr2=0x563e5d05fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:04:29.786 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.781+0000 7f3dfffff640 1 --2- 172.21.15.47:0/919901004 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563e5d0599e0 0x563e5d05fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:29.786 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.781+0000 7f3dfffff640 1 -- 172.21.15.47:0/919901004 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e5cd9d620 con 0x563e5d0590c0 2024-04-17T06:04:29.786 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.781+0000 7f3dfffff640 1 --2- 172.21.15.47:0/919901004 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e5d0590c0 0x563e5d05f880 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f3df00c69f0 tx=0x7f3df002ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65aca41bbbe332fc server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:04:29.786 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.781+0000 7f3ddf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-17T06:04:29.786 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.782+0000 7f3ddf7fe640 1 -- 172.21.15.47:0/919901004 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df004b040 con 0x563e5d0590c0 2024-04-17T06:04:29.786 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.782+0000 7f3ddf7fe640 1 -- 172.21.15.47:0/919901004 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f3df00c6a90 con 0x563e5d0590c0 2024-04-17T06:04:29.786 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.782+0000 7f3e095dd740 1 -- 172.21.15.47:0/919901004 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3df00c2660 con 0x563e5d0590c0 2024-04-17T06:04:29.786 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.782+0000 7f3ddf7fe640 1 -- 172.21.15.47:0/919901004 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3df00be360 con 0x563e5d0590c0 2024-04-17T06:04:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.782+0000 7f3e095dd740 10 client.15669 fetch_fsmap learned FSMap version 27 2024-04-17T06:04:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.782+0000 7f3e095dd740 1 -- 172.21.15.47:0/919901004 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563e5d071310 con 0x563e5d0590c0 2024-04-17T06:04:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.782+0000 7f3ddf7fe640 1 -- 172.21.15.47:0/919901004 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3df00c2660 con 0x563e5d0590c0 2024-04-17T06:04:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.782+0000 7f3ddf7fe640 1 -- 172.21.15.47:0/919901004 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 27) ==== 8343+0+0 (secure 0 0 0) 0x7f3df00a1080 con 0x563e5d0590c0 2024-04-17T06:04:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.782+0000 7f3e095dd740 10 client.15669 fetch_fsmap finished waiting for FSMap version 27 2024-04-17T06:04:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.782+0000 7f3e095dd740 10 client.15669 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-17T06:04:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.782+0000 7f3e095dd740 20 client.15669 populate_metadata read hostname 'smithi047' 2024-04-17T06:04:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.782+0000 7f3e095dd740 1 --2- 172.21.15.47:0/919901004 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x563e5d072660 0x563e5d074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:04:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.782+0000 7f3e095dd740 4 client.15669 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T06:04:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.782+0000 7f3e095dd740 1 -- 172.21.15.47:0/919901004 --> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563e5cfcc1f0 con 0x563e5d072660 2024-04-17T06:04:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.783+0000 7f3dff7fe640 1 --2- 172.21.15.47:0/919901004 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x563e5d072660 0x563e5d074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:04:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.783+0000 7f3dff7fe640 1 --2- 172.21.15.47:0/919901004 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x563e5d072660 0x563e5d074ad0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80cd5768cf180d5e server_cookie=15ab5a9472e2844e in_seq=0 out_seq=0 2024-04-17T06:04:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.783+0000 7f3ddf7fe640 10 client.15669 ms_handle_connect on v2:172.21.15.98:6838/2841287917 2024-04-17T06:04:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.784+0000 7f3ddf7fe640 1 -- 172.21.15.47:0/919901004 <== mds.0 v2:172.21.15.98:6838/2841287917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563e5cfcc1f0 con 0x563e5d072660 2024-04-17T06:04:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.784+0000 7f3ddf7fe640 10 client.15669 handle_command_reply: tid=0 2024-04-17T06:04:29.788 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.784+0000 7f3e095dd740 1 client.15669 shutdown 2024-04-17T06:04:29.788 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.784+0000 7f3e095dd740 1 -- 172.21.15.47:0/919901004 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e5d0590c0 msgr2=0x563e5d05f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:04:29.788 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.784+0000 7f3e095dd740 1 --2- 172.21.15.47:0/919901004 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e5d0590c0 0x563e5d05f880 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f3df00c69f0 tx=0x7f3df002ce60 comp rx=0 tx=0).stop 2024-04-17T06:04:29.788 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.784+0000 7f3e095dd740 1 -- 172.21.15.47:0/919901004 shutdown_connections 2024-04-17T06:04:29.788 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.784+0000 7f3e095dd740 1 --2- 172.21.15.47:0/919901004 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x563e5d072660 0x563e5d074ad0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:29.788 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.784+0000 7f3e095dd740 1 --2- 172.21.15.47:0/919901004 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e5d0627b0 0x563e5d060300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:29.788 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.784+0000 7f3e095dd740 1 --2- 172.21.15.47:0/919901004 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563e5d0599e0 0x563e5d05fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:29.788 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.784+0000 7f3ddf7fe640 0 client.15669 ms_handle_reset on v2:172.21.15.98:6838/2841287917 2024-04-17T06:04:29.788 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.784+0000 7f3e095dd740 1 --2- 172.21.15.47:0/919901004 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e5d0590c0 0x563e5d05f880 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:29.788 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.784+0000 7f3e095dd740 1 -- 172.21.15.47:0/919901004 >> 172.21.15.47:0/919901004 conn(0x563e5d0539e0 msgr2=0x563e5d055730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:04:29.788 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.784+0000 7f3e095dd740 1 -- 172.21.15.47:0/919901004 shutdown_connections 2024-04-17T06:04:29.788 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.784+0000 7f3e095dd740 1 -- 172.21.15.47:0/919901004 wait complete. 2024-04-17T06:04:29.788 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.784+0000 7f3e095dd740 20 client.15669 trim_cache size 0 max 16384 2024-04-17T06:04:29.789 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:04:29 smithi103 ceph-mon[36243]: pgmap v100: 97 pgs: 97 active+clean; 1.9 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.7 KiB/s wr, 12 op/s 2024-04-17T06:04:29.802 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.799+0000 7f3e095dd740 1 Processor -- start 2024-04-17T06:04:29.802 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.799+0000 7f3e095dd740 1 -- start start 2024-04-17T06:04:29.802 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.799+0000 7f3e095dd740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563e5d0590c0 0x563e5d061fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:04:29.802 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.799+0000 7f3e095dd740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e5d0599e0 0x563e5d06dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:04:29.802 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.799+0000 7f3e095dd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e5d0627b0 0x563e5d055540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:04:29.803 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.799+0000 7f3e095dd740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f3e001b03e0 con 0x563e5d0599e0 2024-04-17T06:04:29.803 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.799+0000 7f3e095dd740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3e001adc70 con 0x563e5d0590c0 2024-04-17T06:04:29.803 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.799+0000 7f3e095dd740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3e001abac0 con 0x563e5d0627b0 2024-04-17T06:04:29.803 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.800+0000 7f3dff7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e5d0599e0 0x563e5d06dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:04:29.803 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.800+0000 7f3dff7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e5d0599e0 0x563e5d06dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40734/0 (socket says 172.21.15.47:40734) 2024-04-17T06:04:29.803 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.800+0000 7f3dff7fe640 1 -- 172.21.15.47:0/8039989 learned_addr learned my addr 172.21.15.47:0/8039989 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:04:29.803 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.800+0000 7f3e04eb2640 1 --2- 172.21.15.47:0/8039989 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e5d0627b0 0x563e5d055540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:04:29.803 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.800+0000 7f3dfffff640 1 --2- 172.21.15.47:0/8039989 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563e5d0590c0 0x563e5d061fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:04:29.803 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.800+0000 7f3dff7fe640 1 -- 172.21.15.47:0/8039989 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e5d0627b0 msgr2=0x563e5d055540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:04:29.803 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.800+0000 7f3dff7fe640 1 --2- 172.21.15.47:0/8039989 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e5d0627b0 0x563e5d055540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:29.803 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.800+0000 7f3dff7fe640 1 -- 172.21.15.47:0/8039989 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563e5d0590c0 msgr2=0x563e5d061fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:04:29.803 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.800+0000 7f3dff7fe640 1 --2- 172.21.15.47:0/8039989 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563e5d0590c0 0x563e5d061fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:29.803 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.800+0000 7f3dff7fe640 1 -- 172.21.15.47:0/8039989 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e5d0545b0 con 0x563e5d0599e0 2024-04-17T06:04:29.803 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.800+0000 7f3dff7fe640 1 --2- 172.21.15.47:0/8039989 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e5d0599e0 0x563e5d06dee0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f3df4005fd0 tx=0x7f3df400b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b59a1541576d70e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:04:29.803 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.800+0000 7f3de5ffb640 1 -- 172.21.15.47:0/8039989 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df400bce0 con 0x563e5d0599e0 2024-04-17T06:04:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.800+0000 7f3de5ffb640 1 -- 172.21.15.47:0/8039989 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3df400be80 con 0x563e5d0599e0 2024-04-17T06:04:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.800+0000 7f3de5ffb640 1 -- 172.21.15.47:0/8039989 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df40201b0 con 0x563e5d0599e0 2024-04-17T06:04:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.801+0000 7f3e095dd740 1 -- 172.21.15.47:0/8039989 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e5d0599e0 msgr2=0x563e5d06dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:04:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.801+0000 7f3e095dd740 1 --2- 172.21.15.47:0/8039989 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e5d0599e0 0x563e5d06dee0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f3df4005fd0 tx=0x7f3df400b390 comp rx=0 tx=0).stop 2024-04-17T06:04:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.801+0000 7f3e095dd740 1 -- 172.21.15.47:0/8039989 shutdown_connections 2024-04-17T06:04:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.801+0000 7f3e095dd740 1 --2- 172.21.15.47:0/8039989 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e5d0627b0 0x563e5d055540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.801+0000 7f3e095dd740 1 --2- 172.21.15.47:0/8039989 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e5d0599e0 0x563e5d06dee0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.801+0000 7f3e095dd740 1 --2- 172.21.15.47:0/8039989 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563e5d0590c0 0x563e5d061fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.801+0000 7f3e095dd740 1 -- 172.21.15.47:0/8039989 >> 172.21.15.47:0/8039989 conn(0x563e5d0539e0 msgr2=0x563e5d054e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:04:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.801+0000 7f3e095dd740 1 -- 172.21.15.47:0/8039989 shutdown_connections 2024-04-17T06:04:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.801+0000 7f3e095dd740 1 -- 172.21.15.47:0/8039989 wait complete. 2024-04-17T06:04:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.801+0000 7f3e095dd740 1 -- start start 2024-04-17T06:04:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.801+0000 7f3e095dd740 1 Processor -- start 2024-04-17T06:04:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.802+0000 7f3e095dd740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e5d0590c0 0x563e5d065d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:04:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.802+0000 7f3e095dd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e5d0599e0 0x563e5d066270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:04:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.802+0000 7f3e095dd740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563e5d0627b0 0x563e5d0667b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:04:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.802+0000 7f3e095dd740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f3e001ac130 con 0x563e5d0590c0 2024-04-17T06:04:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.802+0000 7f3e095dd740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3e001ad5a0 con 0x563e5d0627b0 2024-04-17T06:04:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.802+0000 7f3e095dd740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3e001ac800 con 0x563e5d0599e0 2024-04-17T06:04:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.802+0000 7f3e095dd740 10 client.0 _ll_register_callbacks cb 0x7f3df4004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:04:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.802+0000 7f3dfffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e5d0590c0 0x563e5d065d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:04:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.802+0000 7f3dfffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e5d0590c0 0x563e5d065d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40742/0 (socket says 172.21.15.47:40742) 2024-04-17T06:04:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.802+0000 7f3dfffff640 1 -- 172.21.15.47:0/1933411769 learned_addr learned my addr 172.21.15.47:0/1933411769 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:04:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.802+0000 7f3dff7fe640 1 --2- 172.21.15.47:0/1933411769 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e5d0599e0 0x563e5d066270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:04:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.802+0000 7f3e04eb2640 1 --2- 172.21.15.47:0/1933411769 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563e5d0627b0 0x563e5d0667b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:04:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.802+0000 7f3dfffff640 1 -- 172.21.15.47:0/1933411769 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e5d0599e0 msgr2=0x563e5d066270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:04:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.802+0000 7f3dfffff640 1 --2- 172.21.15.47:0/1933411769 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e5d0599e0 0x563e5d066270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.802+0000 7f3dfffff640 1 -- 172.21.15.47:0/1933411769 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563e5d0627b0 msgr2=0x563e5d0667b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:04:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.802+0000 7f3dfffff640 1 --2- 172.21.15.47:0/1933411769 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563e5d0627b0 0x563e5d0667b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.802+0000 7f3dfffff640 1 -- 172.21.15.47:0/1933411769 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e5d0d7c60 con 0x563e5d0590c0 2024-04-17T06:04:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.802+0000 7f3e04eb2640 1 --2- 172.21.15.47:0/1933411769 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563e5d0627b0 0x563e5d0667b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:04:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.802+0000 7f3dfffff640 1 --2- 172.21.15.47:0/1933411769 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e5d0590c0 0x563e5d065d30 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f3df0005940 tx=0x7f3df00bc870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a998f9324dbf599b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:04:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.802+0000 7f3ddf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-17T06:04:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.802+0000 7f3ddf7fe640 1 -- 172.21.15.47:0/1933411769 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df004b070 con 0x563e5d0590c0 2024-04-17T06:04:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.802+0000 7f3ddf7fe640 1 -- 172.21.15.47:0/1933411769 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f3df00c6a90 con 0x563e5d0590c0 2024-04-17T06:04:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.802+0000 7f3e095dd740 1 -- 172.21.15.47:0/1933411769 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563e5cfc6060 con 0x563e5d0590c0 2024-04-17T06:04:29.807 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.803+0000 7f3ddf7fe640 1 -- 172.21.15.47:0/1933411769 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3df00c8020 con 0x563e5d0590c0 2024-04-17T06:04:29.807 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.803+0000 7f3e095dd740 10 client.15675 fetch_fsmap learned FSMap version 27 2024-04-17T06:04:29.807 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.803+0000 7f3e095dd740 1 -- 172.21.15.47:0/1933411769 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563e5d0d7c60 con 0x563e5d0590c0 2024-04-17T06:04:29.807 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.803+0000 7f3ddf7fe640 1 -- 172.21.15.47:0/1933411769 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x563e5cfc6060 con 0x563e5d0590c0 2024-04-17T06:04:29.807 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.803+0000 7f3ddf7fe640 1 -- 172.21.15.47:0/1933411769 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 27) ==== 8343+0+0 (secure 0 0 0) 0x7f3df00c22d0 con 0x563e5d0590c0 2024-04-17T06:04:29.807 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.803+0000 7f3e095dd740 10 client.15675 fetch_fsmap finished waiting for FSMap version 27 2024-04-17T06:04:29.807 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.803+0000 7f3e095dd740 10 client.15675 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-17T06:04:29.807 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.803+0000 7f3e095dd740 20 client.15675 populate_metadata read hostname 'smithi047' 2024-04-17T06:04:29.807 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.803+0000 7f3e095dd740 1 --2- 172.21.15.47:0/1933411769 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x563e5d0d9180 0x563e5d0db5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:04:29.807 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.803+0000 7f3e095dd740 4 client.15675 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-17T06:04:29.807 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.803+0000 7f3e095dd740 1 -- 172.21.15.47:0/1933411769 --> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563e5cf42670 con 0x563e5d0d9180 2024-04-17T06:04:29.807 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.803+0000 7f3dff7fe640 1 --2- 172.21.15.47:0/1933411769 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x563e5d0d9180 0x563e5d0db5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:04:29.807 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.804+0000 7f3ddf7fe640 10 client.15675 ms_handle_connect on v2:172.21.15.98:6838/2841287917 2024-04-17T06:04:29.807 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.804+0000 7f3dff7fe640 1 --2- 172.21.15.47:0/1933411769 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x563e5d0d9180 0x563e5d0db5a0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17e23b91abe1485b server_cookie=2ffe807a1b5045cb in_seq=0 out_seq=0 2024-04-17T06:04:29.810 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.811+0000 7f3ddf7fe640 1 -- 172.21.15.47:0/1933411769 <== mds.0 v2:172.21.15.98:6838/2841287917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563e5cf42670 con 0x563e5d0d9180 2024-04-17T06:04:29.810 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.811+0000 7f3ddf7fe640 10 client.15675 handle_command_reply: tid=0 2024-04-17T06:04:29.810 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.812+0000 7f3e095dd740 1 client.15675 shutdown 2024-04-17T06:04:29.810 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.812+0000 7f3e095dd740 1 -- 172.21.15.47:0/1933411769 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e5d0590c0 msgr2=0x563e5d065d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:04:29.810 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.812+0000 7f3e095dd740 1 --2- 172.21.15.47:0/1933411769 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e5d0590c0 0x563e5d065d30 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f3df0005940 tx=0x7f3df00bc870 comp rx=0 tx=0).stop 2024-04-17T06:04:29.810 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.812+0000 7f3e095dd740 1 -- 172.21.15.47:0/1933411769 shutdown_connections 2024-04-17T06:04:29.810 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.812+0000 7f3e095dd740 1 --2- 172.21.15.47:0/1933411769 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x563e5d0d9180 0x563e5d0db5a0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:29.810 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.812+0000 7f3e095dd740 1 --2- 172.21.15.47:0/1933411769 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563e5d0627b0 0x563e5d0667b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:29.810 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.812+0000 7f3e095dd740 1 --2- 172.21.15.47:0/1933411769 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e5d0599e0 0x563e5d066270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:29.810 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.812+0000 7f3ddf7fe640 0 client.15675 ms_handle_reset on v2:172.21.15.98:6838/2841287917 2024-04-17T06:04:29.810 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.812+0000 7f3e095dd740 1 --2- 172.21.15.47:0/1933411769 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e5d0590c0 0x563e5d065d30 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:29.811 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.812+0000 7f3e095dd740 1 -- 172.21.15.47:0/1933411769 >> 172.21.15.47:0/1933411769 conn(0x563e5d0539e0 msgr2=0x563e5d0556c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:04:29.811 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.812+0000 7f3e095dd740 1 -- 172.21.15.47:0/1933411769 shutdown_connections 2024-04-17T06:04:29.811 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.812+0000 7f3e095dd740 1 -- 172.21.15.47:0/1933411769 wait complete. 2024-04-17T06:04:29.811 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.812+0000 7f3e095dd740 20 client.15675 trim_cache size 0 max 16384 2024-04-17T06:04:29.811 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-04-17T06:04:29.811 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-04-17T06:04:29.811 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-04-17T06:04:29.811 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-04-17T06:04:29.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.813+0000 7f3e06915640 1 -- 172.21.15.47:0/3307560263 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f3de80a3e40 msgr2=0x7f3de80a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:04:29.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.813+0000 7f3e06915640 1 --2- 172.21.15.47:0/3307560263 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f3de80a3e40 0x7f3de80a6300 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3df8009fd0 tx=0x7f3df8004ab0 comp rx=0 tx=0).stop 2024-04-17T06:04:29.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.813+0000 7f3e06915640 1 -- 172.21.15.47:0/3307560263 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3e001021e0 msgr2=0x7f3e0019e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:04:29.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.813+0000 7f3e06915640 1 --2- 172.21.15.47:0/3307560263 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3e001021e0 0x7f3e0019e410 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f3df0006010 tx=0x7f3df002cc70 comp rx=0 tx=0).stop 2024-04-17T06:04:29.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.813+0000 7f3e06915640 1 -- 172.21.15.47:0/3307560263 shutdown_connections 2024-04-17T06:04:29.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.813+0000 7f3e06915640 1 --2- 172.21.15.47:0/3307560263 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3e001985c0 0x7f3e00198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:29.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.813+0000 7f3e06915640 1 --2- 172.21.15.47:0/3307560263 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3e001072b0 0x7f3e0019e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:29.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.813+0000 7f3e06915640 1 --2- 172.21.15.47:0/3307560263 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f3de80a3e40 0x7f3de80a6300 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:29.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.813+0000 7f3e06915640 1 --2- 172.21.15.47:0/3307560263 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3e001021e0 0x7f3e0019e410 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:29.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.813+0000 7f3e06915640 1 -- 172.21.15.47:0/3307560263 >> 172.21.15.47:0/3307560263 conn(0x7f3e000793f0 msgr2=0x7f3e000ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:04:29.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.813+0000 7f3e06915640 1 -- 172.21.15.47:0/3307560263 shutdown_connections 2024-04-17T06:04:29.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:29.813+0000 7f3e06915640 1 -- 172.21.15.47:0/3307560263 wait complete. 2024-04-17T06:04:29.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:04:29 smithi098 ceph-mon[36516]: pgmap v100: 97 pgs: 97 active+clean; 1.9 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.7 KiB/s wr, 12 op/s 2024-04-17T06:04:29.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:04:29 smithi047 ceph-mon[32223]: pgmap v100: 97 pgs: 97 active+clean; 1.9 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.7 KiB/s wr, 12 op/s 2024-04-17T06:04:31.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:04:31 smithi103 ceph-mon[36243]: pgmap v101: 97 pgs: 97 active+clean; 1.9 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-17T06:04:31.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:04:31 smithi098 ceph-mon[36516]: pgmap v101: 97 pgs: 97 active+clean; 1.9 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-17T06:04:31.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:04:31 smithi047 ceph-mon[32223]: pgmap v101: 97 pgs: 97 active+clean; 1.9 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-17T06:04:33.783 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:04:33 smithi103 ceph-mon[36243]: pgmap v102: 97 pgs: 97 active+clean; 1.9 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 6.9 KiB/s wr, 9 op/s 2024-04-17T06:04:33.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:04:33 smithi098 ceph-mon[36516]: pgmap v102: 97 pgs: 97 active+clean; 1.9 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 6.9 KiB/s wr, 9 op/s 2024-04-17T06:04:33.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:04:33 smithi047 ceph-mon[32223]: pgmap v102: 97 pgs: 97 active+clean; 1.9 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 6.9 KiB/s wr, 9 op/s 2024-04-17T06:04:35.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:04:35 smithi103 ceph-mon[36243]: pgmap v103: 97 pgs: 97 active+clean; 2.0 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-04-17T06:04:35.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:04:35 smithi098 ceph-mon[36516]: pgmap v103: 97 pgs: 97 active+clean; 2.0 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-04-17T06:04:35.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:04:35 smithi047 ceph-mon[32223]: pgmap v103: 97 pgs: 97 active+clean; 2.0 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-04-17T06:04:37.783 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:04:37 smithi103 ceph-mon[36243]: pgmap v104: 97 pgs: 97 active+clean; 2.0 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 9.6 KiB/s wr, 11 op/s 2024-04-17T06:04:37.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:04:37 smithi098 ceph-mon[36516]: pgmap v104: 97 pgs: 97 active+clean; 2.0 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 9.6 KiB/s wr, 11 op/s 2024-04-17T06:04:37.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:04:37 smithi047 ceph-mon[32223]: pgmap v104: 97 pgs: 97 active+clean; 2.0 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 9.6 KiB/s wr, 11 op/s 2024-04-17T06:04:39.783 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:04:39 smithi103 ceph-mon[36243]: pgmap v105: 97 pgs: 97 active+clean; 2.0 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 9.6 KiB/s wr, 11 op/s 2024-04-17T06:04:39.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:04:39 smithi098 ceph-mon[36516]: pgmap v105: 97 pgs: 97 active+clean; 2.0 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 9.6 KiB/s wr, 11 op/s 2024-04-17T06:04:39.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:04:39 smithi047 ceph-mon[32223]: pgmap v105: 97 pgs: 97 active+clean; 2.0 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 9.6 KiB/s wr, 11 op/s 2024-04-17T06:04:41.783 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:04:41 smithi103 ceph-mon[36243]: pgmap v106: 97 pgs: 97 active+clean; 2.0 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-17T06:04:41.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:04:41 smithi098 ceph-mon[36516]: pgmap v106: 97 pgs: 97 active+clean; 2.0 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-17T06:04:41.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:04:41 smithi047 ceph-mon[32223]: pgmap v106: 97 pgs: 97 active+clean; 2.0 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-17T06:04:42.911 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.912+0000 7f44fcb0e640 1 -- 172.21.15.47:0/1670092929 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44f8101ab0 msgr2=0x7f44f8105aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:04:42.911 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.912+0000 7f44fcb0e640 1 --2- 172.21.15.47:0/1670092929 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44f8101ab0 0x7f44f8105aa0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f44ec009900 tx=0x7f44ec02bc30 comp rx=0 tx=0).stop 2024-04-17T06:04:42.911 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.912+0000 7f44fcb0e640 1 -- 172.21.15.47:0/1670092929 shutdown_connections 2024-04-17T06:04:42.911 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.912+0000 7f44fcb0e640 1 --2- 172.21.15.47:0/1670092929 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f44f8106180 0x7f44f810dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:42.911 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.912+0000 7f44fcb0e640 1 --2- 172.21.15.47:0/1670092929 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44f8101ab0 0x7f44f8105aa0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:42.912 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.912+0000 7f44fcb0e640 1 --2- 172.21.15.47:0/1670092929 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f44f8101100 0x7f44f81014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:42.912 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.912+0000 7f44fcb0e640 1 -- 172.21.15.47:0/1670092929 >> 172.21.15.47:0/1670092929 conn(0x7f44f8077fe0 msgr2=0x7f44f80783f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:04:42.912 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.912+0000 7f44fcb0e640 1 -- 172.21.15.47:0/1670092929 shutdown_connections 2024-04-17T06:04:42.912 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.912+0000 7f44fcb0e640 1 -- 172.21.15.47:0/1670092929 wait complete. 2024-04-17T06:04:42.914 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.913+0000 7f44fcb0e640 1 Processor -- start 2024-04-17T06:04:42.914 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.913+0000 7f44fcb0e640 1 -- start start 2024-04-17T06:04:42.914 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.913+0000 7f44fcb0e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44f8101100 0x7f44f8102b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:04:42.914 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.913+0000 7f44fcb0e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f44f8106180 0x7f44f8103080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:04:42.914 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.913+0000 7f44fcb0e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f44f81035c0 0x7f44f81a69c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:04:42.914 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.913+0000 7f44fcb0e640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f44f8111210 con 0x7f44f8101100 2024-04-17T06:04:42.914 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.913+0000 7f44fcb0e640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f44f8111090 con 0x7f44f81035c0 2024-04-17T06:04:42.914 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.913+0000 7f44fcb0e640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f44f8111390 con 0x7f44f8106180 2024-04-17T06:04:42.914 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.913+0000 7f44f659c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44f8101100 0x7f44f8102b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:04:42.914 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.913+0000 7f44f659c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44f8101100 0x7f44f8102b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49822/0 (socket says 172.21.15.47:49822) 2024-04-17T06:04:42.914 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.913+0000 7f44f659c640 1 -- 172.21.15.47:0/591755037 learned_addr learned my addr 172.21.15.47:0/591755037 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:04:42.915 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.914+0000 7f44f6d9d640 1 --2- 172.21.15.47:0/591755037 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f44f81035c0 0x7f44f81a69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:04:42.915 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.914+0000 7f44f5d9b640 1 --2- 172.21.15.47:0/591755037 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f44f8106180 0x7f44f8103080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:04:42.915 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.914+0000 7f44f659c640 1 -- 172.21.15.47:0/591755037 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f44f8106180 msgr2=0x7f44f8103080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:04:42.915 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.914+0000 7f44f659c640 1 --2- 172.21.15.47:0/591755037 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f44f8106180 0x7f44f8103080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:42.915 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.914+0000 7f44f659c640 1 -- 172.21.15.47:0/591755037 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f44f81035c0 msgr2=0x7f44f81a69c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:04:42.915 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.914+0000 7f44f659c640 1 --2- 172.21.15.47:0/591755037 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f44f81035c0 0x7f44f81a69c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:42.915 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.914+0000 7f44f659c640 1 -- 172.21.15.47:0/591755037 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44f81a70c0 con 0x7f44f8101100 2024-04-17T06:04:42.915 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.914+0000 7f44f6d9d640 1 --2- 172.21.15.47:0/591755037 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f44f81035c0 0x7f44f81a69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:04:42.915 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.914+0000 7f44f659c640 1 --2- 172.21.15.47:0/591755037 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44f8101100 0x7f44f8102b40 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f44e800d770 tx=0x7f44e800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:04:42.915 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.914+0000 7f44e77fe640 1 -- 172.21.15.47:0/591755037 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e8017070 con 0x7f44f8101100 2024-04-17T06:04:42.915 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.914+0000 7f44e77fe640 1 -- 172.21.15.47:0/591755037 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44e8005500 con 0x7f44f8101100 2024-04-17T06:04:42.915 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.914+0000 7f44fcb0e640 1 -- 172.21.15.47:0/591755037 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44f81a73b0 con 0x7f44f8101100 2024-04-17T06:04:42.915 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.914+0000 7f44e77fe640 1 -- 172.21.15.47:0/591755037 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e8004e00 con 0x7f44f8101100 2024-04-17T06:04:42.915 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.914+0000 7f44fcb0e640 1 -- 172.21.15.47:0/591755037 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44f81a7880 con 0x7f44f8101100 2024-04-17T06:04:42.921 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.917+0000 7f44e77fe640 1 -- 172.21.15.47:0/591755037 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f44e8004fa0 con 0x7f44f8101100 2024-04-17T06:04:42.921 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.917+0000 7f44ff7d5740 1 Processor -- start 2024-04-17T06:04:42.921 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.917+0000 7f44ff7d5740 1 -- start start 2024-04-17T06:04:42.921 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.917+0000 7f44e77fe640 1 --2- 172.21.15.47:0/591755037 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f44e00a3df0 0x7f44e00a62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:04:42.921 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.917+0000 7f44ff7d5740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56003f9000c0 0x56003f9004a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:04:42.921 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.917+0000 7f44ff7d5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56003f9009e0 0x56003f909270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:04:42.922 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.917+0000 7f44ff7d5740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56003f9097b0 0x56003f90bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:04:42.922 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.917+0000 7f44ff7d5740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x56003f8657d0 con 0x56003f9000c0 2024-04-17T06:04:42.922 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.917+0000 7f44ff7d5740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x56003f802f70 con 0x56003f9097b0 2024-04-17T06:04:42.922 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.917+0000 7f44ff7d5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56003f861790 con 0x56003f9009e0 2024-04-17T06:04:42.922 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.918+0000 7f44f659c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56003f9000c0 0x56003f9004a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:04:42.922 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.918+0000 7f44f6d9d640 1 --2- 172.21.15.47:0/591755037 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f44e00a3df0 0x7f44e00a62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:04:42.922 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.918+0000 7f44f659c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56003f9000c0 0x56003f9004a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49838/0 (socket says 172.21.15.47:49838) 2024-04-17T06:04:42.922 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.918+0000 7f44f659c640 1 -- 172.21.15.47:0/1542010695 learned_addr learned my addr 172.21.15.47:0/1542010695 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:04:42.922 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.918+0000 7f44e77fe640 1 -- 172.21.15.47:0/591755037 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f44e80d0320 con 0x7f44f8101100 2024-04-17T06:04:42.922 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.918+0000 7f44f5d9b640 1 --2- 172.21.15.47:0/1542010695 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56003f9009e0 0x56003f909270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:04:42.922 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.918+0000 7f44f6d9d640 1 --2- 172.21.15.47:0/1542010695 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56003f9097b0 0x56003f90bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:04:42.922 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.918+0000 7f44f659c640 1 -- 172.21.15.47:0/1542010695 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56003f9009e0 msgr2=0x56003f909270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:04:42.922 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.918+0000 7f44f659c640 1 --2- 172.21.15.47:0/1542010695 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56003f9009e0 0x56003f909270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:42.922 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.918+0000 7f44f659c640 1 -- 172.21.15.47:0/1542010695 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56003f9097b0 msgr2=0x56003f90bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:04:42.923 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.918+0000 7f44f659c640 1 --2- 172.21.15.47:0/1542010695 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56003f9097b0 0x56003f90bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:42.923 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.918+0000 7f44f659c640 1 -- 172.21.15.47:0/1542010695 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56003f578210 con 0x56003f9000c0 2024-04-17T06:04:42.923 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.918+0000 7f44f6d9d640 1 --2- 172.21.15.47:0/1542010695 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56003f9097b0 0x56003f90bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:04:42.923 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.918+0000 7f44f659c640 1 --2- 172.21.15.47:0/1542010695 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56003f9000c0 0x56003f9004a0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f44e80cd9e0 tx=0x7f44e80048a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a41fe487c344fb2 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:04:42.923 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.918+0000 7f44f6d9d640 1 --2- 172.21.15.47:0/591755037 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f44e00a3df0 0x7f44e00a62b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x56003f8fbe00 tx=0x7f44f0009ac0 comp rx=0 tx=0).ready entity=mgr.24923 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:04:42.923 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.918+0000 7f44e4ff9640 1 -- 172.21.15.47:0/1542010695 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e809c990 con 0x56003f9000c0 2024-04-17T06:04:42.923 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.918+0000 7f44e4ff9640 1 -- 172.21.15.47:0/1542010695 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44e80046c0 con 0x56003f9000c0 2024-04-17T06:04:42.923 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.918+0000 7f44e4ff9640 1 -- 172.21.15.47:0/1542010695 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e80a0640 con 0x56003f9000c0 2024-04-17T06:04:42.923 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.919+0000 7f44ff7d5740 1 -- 172.21.15.47:0/1542010695 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56003f9000c0 msgr2=0x56003f9004a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:04:42.923 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.919+0000 7f44ff7d5740 1 --2- 172.21.15.47:0/1542010695 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56003f9000c0 0x56003f9004a0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f44e80cd9e0 tx=0x7f44e80048a0 comp rx=0 tx=0).stop 2024-04-17T06:04:42.923 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.919+0000 7f44ff7d5740 1 -- 172.21.15.47:0/1542010695 shutdown_connections 2024-04-17T06:04:42.923 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.919+0000 7f44ff7d5740 1 --2- 172.21.15.47:0/1542010695 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56003f9097b0 0x56003f90bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:42.923 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.919+0000 7f44ff7d5740 1 --2- 172.21.15.47:0/1542010695 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56003f9009e0 0x56003f909270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:42.923 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.919+0000 7f44ff7d5740 1 --2- 172.21.15.47:0/1542010695 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56003f9000c0 0x56003f9004a0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:42.923 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.919+0000 7f44ff7d5740 1 -- 172.21.15.47:0/1542010695 >> 172.21.15.47:0/1542010695 conn(0x56003f8fa9e0 msgr2=0x56003f8fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:04:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.919+0000 7f44ff7d5740 1 -- 172.21.15.47:0/1542010695 shutdown_connections 2024-04-17T06:04:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.919+0000 7f44ff7d5740 1 -- 172.21.15.47:0/1542010695 wait complete. 2024-04-17T06:04:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.920+0000 7f44ff7d5740 1 -- start start 2024-04-17T06:04:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.920+0000 7f44ff7d5740 1 Processor -- start 2024-04-17T06:04:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.920+0000 7f44ff7d5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56003f9000c0 0x56003f9068e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:04:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.920+0000 7f44ff7d5740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56003f9009e0 0x56003f906e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:04:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.920+0000 7f44ff7d5740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56003f9097b0 0x56003f907360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:04:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.920+0000 7f44ff7d5740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x56003f90c380 con 0x56003f9097b0 2024-04-17T06:04:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.920+0000 7f44ff7d5740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x56003f7ede90 con 0x56003f9009e0 2024-04-17T06:04:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.920+0000 7f44ff7d5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56003f90c500 con 0x56003f9000c0 2024-04-17T06:04:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.920+0000 7f44ff7d5740 10 client.0 _ll_register_callbacks cb 0x56003f803d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:04:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.920+0000 7f44f6d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56003f9097b0 0x56003f907360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:04:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.920+0000 7f44f6d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56003f9097b0 0x56003f907360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49844/0 (socket says 172.21.15.47:49844) 2024-04-17T06:04:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.920+0000 7f44f6d9d640 1 -- 172.21.15.47:0/1052331798 learned_addr learned my addr 172.21.15.47:0/1052331798 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:04:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.920+0000 7f44f659c640 1 --2- 172.21.15.47:0/1052331798 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56003f9000c0 0x56003f9068e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:04:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.920+0000 7f44f5d9b640 1 --2- 172.21.15.47:0/1052331798 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56003f9009e0 0x56003f906e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:04:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.921+0000 7f44f6d9d640 1 -- 172.21.15.47:0/1052331798 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56003f9000c0 msgr2=0x56003f9068e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:04:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.921+0000 7f44f6d9d640 1 --2- 172.21.15.47:0/1052331798 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56003f9000c0 0x56003f9068e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.921+0000 7f44f6d9d640 1 -- 172.21.15.47:0/1052331798 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56003f9009e0 msgr2=0x56003f906e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:04:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.921+0000 7f44f6d9d640 1 --2- 172.21.15.47:0/1052331798 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56003f9009e0 0x56003f906e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.921+0000 7f44f6d9d640 1 -- 172.21.15.47:0/1052331798 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56003f644620 con 0x56003f9097b0 2024-04-17T06:04:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.921+0000 7f44f6d9d640 1 --2- 172.21.15.47:0/1052331798 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56003f9097b0 0x56003f907360 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f44f0014010 tx=0x7f44f0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9338f48a339bf218 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:04:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.921+0000 7f44ddffb640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-17T06:04:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.921+0000 7f44ddffb640 1 -- 172.21.15.47:0/1052331798 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44f0023070 con 0x56003f9097b0 2024-04-17T06:04:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.921+0000 7f44ddffb640 1 -- 172.21.15.47:0/1052331798 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f44f0028660 con 0x56003f9097b0 2024-04-17T06:04:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.921+0000 7f44ff7d5740 1 -- 172.21.15.47:0/1052331798 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56003f8731f0 con 0x56003f9097b0 2024-04-17T06:04:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.921+0000 7f44ddffb640 1 -- 172.21.15.47:0/1052331798 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f44f0019070 con 0x56003f9097b0 2024-04-17T06:04:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.921+0000 7f44ff7d5740 10 client.15687 fetch_fsmap learned FSMap version 27 2024-04-17T06:04:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.921+0000 7f44ff7d5740 1 -- 172.21.15.47:0/1052331798 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56003f918710 con 0x56003f9097b0 2024-04-17T06:04:42.926 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.921+0000 7f44ddffb640 1 -- 172.21.15.47:0/1052331798 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x56003f8731f0 con 0x56003f9097b0 2024-04-17T06:04:42.926 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.922+0000 7f44ddffb640 1 -- 172.21.15.47:0/1052331798 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 27) ==== 8343+0+0 (secure 0 0 0) 0x7f44f001f410 con 0x56003f9097b0 2024-04-17T06:04:42.926 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.922+0000 7f44ff7d5740 10 client.15687 fetch_fsmap finished waiting for FSMap version 27 2024-04-17T06:04:42.926 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.922+0000 7f44ff7d5740 10 client.15687 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-17T06:04:42.926 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.922+0000 7f44ff7d5740 20 client.15687 populate_metadata read hostname 'smithi047' 2024-04-17T06:04:42.926 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.922+0000 7f44ff7d5740 1 --2- 172.21.15.47:0/1052331798 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x56003f919b80 0x56003f91bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:04:42.926 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.922+0000 7f44ff7d5740 4 client.15687 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T06:04:42.926 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.922+0000 7f44ff7d5740 1 -- 172.21.15.47:0/1052331798 --> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56003f86d060 con 0x56003f919b80 2024-04-17T06:04:42.926 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.922+0000 7f44f659c640 1 --2- 172.21.15.47:0/1052331798 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x56003f919b80 0x56003f91bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:04:42.926 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.922+0000 7f44f659c640 1 --2- 172.21.15.47:0/1052331798 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x56003f919b80 0x56003f91bf90 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32a46a9fc617697f server_cookie=6a3203f04a7aa88c in_seq=0 out_seq=0 2024-04-17T06:04:42.926 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.922+0000 7f44ddffb640 10 client.15687 ms_handle_connect on v2:172.21.15.98:6838/2841287917 2024-04-17T06:04:42.927 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.923+0000 7f44ddffb640 1 -- 172.21.15.47:0/1052331798 <== mds.0 v2:172.21.15.98:6838/2841287917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56003f86d060 con 0x56003f919b80 2024-04-17T06:04:42.927 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.923+0000 7f44ddffb640 10 client.15687 handle_command_reply: tid=0 2024-04-17T06:04:42.927 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.923+0000 7f44ff7d5740 1 client.15687 shutdown 2024-04-17T06:04:42.927 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.923+0000 7f44ff7d5740 1 -- 172.21.15.47:0/1052331798 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56003f9097b0 msgr2=0x56003f907360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:04:42.928 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.923+0000 7f44ff7d5740 1 --2- 172.21.15.47:0/1052331798 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56003f9097b0 0x56003f907360 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f44f0014010 tx=0x7f44f0004a40 comp rx=0 tx=0).stop 2024-04-17T06:04:42.928 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.923+0000 7f44ff7d5740 1 -- 172.21.15.47:0/1052331798 shutdown_connections 2024-04-17T06:04:42.928 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.923+0000 7f44ff7d5740 1 --2- 172.21.15.47:0/1052331798 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x56003f919b80 0x56003f91bf90 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:42.928 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.923+0000 7f44ff7d5740 1 --2- 172.21.15.47:0/1052331798 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56003f9097b0 0x56003f907360 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:42.928 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.923+0000 7f44ff7d5740 1 --2- 172.21.15.47:0/1052331798 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56003f9009e0 0x56003f906e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:42.928 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.923+0000 7f44ff7d5740 1 --2- 172.21.15.47:0/1052331798 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56003f9000c0 0x56003f9068e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:42.928 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.923+0000 7f44ddffb640 0 client.15687 ms_handle_reset on v2:172.21.15.98:6838/2841287917 2024-04-17T06:04:42.928 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.923+0000 7f44ff7d5740 1 -- 172.21.15.47:0/1052331798 >> 172.21.15.47:0/1052331798 conn(0x56003f8fa9e0 msgr2=0x56003f8fc730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:04:42.928 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.923+0000 7f44ff7d5740 1 -- 172.21.15.47:0/1052331798 shutdown_connections 2024-04-17T06:04:42.928 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.923+0000 7f44ff7d5740 1 -- 172.21.15.47:0/1052331798 wait complete. 2024-04-17T06:04:42.928 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.924+0000 7f44ff7d5740 20 client.15687 trim_cache size 0 max 16384 2024-04-17T06:04:42.939 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.938+0000 7f44ff7d5740 1 Processor -- start 2024-04-17T06:04:42.939 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.938+0000 7f44ff7d5740 1 -- start start 2024-04-17T06:04:42.939 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.939+0000 7f44ff7d5740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56003f9000c0 0x56003f909010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:04:42.939 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.939+0000 7f44ff7d5740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56003f9009e0 0x56003f915240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:04:42.939 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.939+0000 7f44ff7d5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56003f9097b0 0x56003f8fc540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:04:42.939 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.939+0000 7f44ff7d5740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f44f81b2680 con 0x56003f9000c0 2024-04-17T06:04:42.939 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.939+0000 7f44ff7d5740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f44f81aff10 con 0x56003f9009e0 2024-04-17T06:04:42.939 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.939+0000 7f44ff7d5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f44f81add80 con 0x56003f9097b0 2024-04-17T06:04:42.939 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.939+0000 7f44f659c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56003f9000c0 0x56003f909010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:04:42.939 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.939+0000 7f44f659c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56003f9000c0 0x56003f909010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49850/0 (socket says 172.21.15.47:49850) 2024-04-17T06:04:42.940 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.939+0000 7f44f659c640 1 -- 172.21.15.47:0/3806754027 learned_addr learned my addr 172.21.15.47:0/3806754027 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:04:42.940 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.939+0000 7f44f5d9b640 1 --2- 172.21.15.47:0/3806754027 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56003f9009e0 0x56003f915240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:04:42.940 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.939+0000 7f44f659c640 1 -- 172.21.15.47:0/3806754027 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56003f9097b0 msgr2=0x56003f8fc540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:04:42.940 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.939+0000 7f44f659c640 1 --2- 172.21.15.47:0/3806754027 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56003f9097b0 0x56003f8fc540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:42.940 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.939+0000 7f44f659c640 1 -- 172.21.15.47:0/3806754027 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56003f9009e0 msgr2=0x56003f915240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:04:42.940 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.939+0000 7f44f659c640 1 --2- 172.21.15.47:0/3806754027 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56003f9009e0 0x56003f915240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:42.940 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.939+0000 7f44f659c640 1 -- 172.21.15.47:0/3806754027 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56003f8fb5b0 con 0x56003f9000c0 2024-04-17T06:04:42.940 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.939+0000 7f44f659c640 1 --2- 172.21.15.47:0/3806754027 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56003f9000c0 0x56003f909010 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f44e8009610 tx=0x7f44e800df90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a1d155dae580615 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:04:42.940 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.939+0000 7f44dffff640 1 -- 172.21.15.47:0/3806754027 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e80ba070 con 0x56003f9000c0 2024-04-17T06:04:42.940 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.939+0000 7f44dffff640 1 -- 172.21.15.47:0/3806754027 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44e809d5c0 con 0x56003f9000c0 2024-04-17T06:04:42.940 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.939+0000 7f44dffff640 1 -- 172.21.15.47:0/3806754027 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e809a330 con 0x56003f9000c0 2024-04-17T06:04:42.940 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.940+0000 7f44ff7d5740 1 -- 172.21.15.47:0/3806754027 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56003f9000c0 msgr2=0x56003f909010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:04:42.940 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.940+0000 7f44ff7d5740 1 --2- 172.21.15.47:0/3806754027 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56003f9000c0 0x56003f909010 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f44e8009610 tx=0x7f44e800df90 comp rx=0 tx=0).stop 2024-04-17T06:04:42.940 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.940+0000 7f44ff7d5740 1 -- 172.21.15.47:0/3806754027 shutdown_connections 2024-04-17T06:04:42.941 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.940+0000 7f44ff7d5740 1 --2- 172.21.15.47:0/3806754027 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56003f9097b0 0x56003f8fc540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:42.941 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.940+0000 7f44ff7d5740 1 --2- 172.21.15.47:0/3806754027 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56003f9009e0 0x56003f915240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:42.941 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.940+0000 7f44ff7d5740 1 --2- 172.21.15.47:0/3806754027 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56003f9000c0 0x56003f909010 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:42.941 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.940+0000 7f44ff7d5740 1 -- 172.21.15.47:0/3806754027 >> 172.21.15.47:0/3806754027 conn(0x56003f8fa9e0 msgr2=0x56003f8fbe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:04:42.941 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.940+0000 7f44ff7d5740 1 -- 172.21.15.47:0/3806754027 shutdown_connections 2024-04-17T06:04:42.941 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.940+0000 7f44ff7d5740 1 -- 172.21.15.47:0/3806754027 wait complete. 2024-04-17T06:04:42.941 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.940+0000 7f44ff7d5740 1 -- start start 2024-04-17T06:04:42.941 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.940+0000 7f44ff7d5740 1 Processor -- start 2024-04-17T06:04:42.944 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.941+0000 7f44ff7d5740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56003f9000c0 0x56003f90d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:04:42.944 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.941+0000 7f44ff7d5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56003f9009e0 0x56003f90dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:04:42.944 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.941+0000 7f44ff7d5740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56003f9097b0 0x56003f97f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:04:42.944 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.941+0000 7f44ff7d5740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f44f81ae3d0 con 0x56003f9000c0 2024-04-17T06:04:42.944 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.941+0000 7f44ff7d5740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f44f81af840 con 0x56003f9097b0 2024-04-17T06:04:42.944 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.941+0000 7f44ff7d5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f44f81aeaa0 con 0x56003f9009e0 2024-04-17T06:04:42.944 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.941+0000 7f44ff7d5740 10 client.0 _ll_register_callbacks cb 0x7f44f0009cd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:04:42.944 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.941+0000 7f44f659c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56003f9000c0 0x56003f90d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:04:42.944 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.941+0000 7f44f659c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56003f9000c0 0x56003f90d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49852/0 (socket says 172.21.15.47:49852) 2024-04-17T06:04:42.944 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.941+0000 7f44f659c640 1 -- 172.21.15.47:0/3864048863 learned_addr learned my addr 172.21.15.47:0/3864048863 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:04:42.945 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.941+0000 7f44f6d9d640 1 --2- 172.21.15.47:0/3864048863 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56003f9097b0 0x56003f97f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:04:42.945 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.941+0000 7f44f5d9b640 1 --2- 172.21.15.47:0/3864048863 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56003f9009e0 0x56003f90dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:04:42.945 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.941+0000 7f44f659c640 1 -- 172.21.15.47:0/3864048863 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56003f9009e0 msgr2=0x56003f90dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:04:42.945 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.941+0000 7f44f659c640 1 --2- 172.21.15.47:0/3864048863 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56003f9009e0 0x56003f90dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:42.945 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.941+0000 7f44f659c640 1 -- 172.21.15.47:0/3864048863 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56003f9097b0 msgr2=0x56003f97f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:04:42.945 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.941+0000 7f44f659c640 1 --2- 172.21.15.47:0/3864048863 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56003f9097b0 0x56003f97f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:42.945 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.941+0000 7f44f659c640 1 -- 172.21.15.47:0/3864048863 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56003f97f650 con 0x56003f9000c0 2024-04-17T06:04:42.945 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.941+0000 7f44f5d9b640 1 --2- 172.21.15.47:0/3864048863 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56003f9009e0 0x56003f90dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:04:42.945 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.941+0000 7f44f659c640 1 --2- 172.21.15.47:0/3864048863 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56003f9000c0 0x56003f90d810 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f44e80986f0 tx=0x7f44e80cdb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e0a122cd637af11 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:04:42.945 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.941+0000 7f44ddffb640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-17T06:04:42.945 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.941+0000 7f44ddffb640 1 -- 172.21.15.47:0/3864048863 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e80ba030 con 0x56003f9000c0 2024-04-17T06:04:42.945 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.941+0000 7f44ddffb640 1 -- 172.21.15.47:0/3864048863 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f44e8096e00 con 0x56003f9000c0 2024-04-17T06:04:42.945 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.941+0000 7f44ff7d5740 1 -- 172.21.15.47:0/3864048863 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56003f7e9670 con 0x56003f9000c0 2024-04-17T06:04:42.945 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.942+0000 7f44ddffb640 1 -- 172.21.15.47:0/3864048863 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f44e809c990 con 0x56003f9000c0 2024-04-17T06:04:42.946 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.942+0000 7f44ff7d5740 10 client.15693 fetch_fsmap learned FSMap version 27 2024-04-17T06:04:42.946 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.942+0000 7f44ff7d5740 1 -- 172.21.15.47:0/3864048863 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56003f980890 con 0x56003f9000c0 2024-04-17T06:04:42.946 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.942+0000 7f44ddffb640 1 -- 172.21.15.47:0/3864048863 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x56003f7e9670 con 0x56003f9000c0 2024-04-17T06:04:42.946 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.942+0000 7f44ddffb640 1 -- 172.21.15.47:0/3864048863 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 27) ==== 8343+0+0 (secure 0 0 0) 0x7f44e8098900 con 0x56003f9000c0 2024-04-17T06:04:42.946 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.942+0000 7f44ff7d5740 10 client.15693 fetch_fsmap finished waiting for FSMap version 27 2024-04-17T06:04:42.946 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.942+0000 7f44ff7d5740 10 client.15693 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-17T06:04:42.946 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.942+0000 7f44ff7d5740 20 client.15693 populate_metadata read hostname 'smithi047' 2024-04-17T06:04:42.946 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.942+0000 7f44ff7d5740 1 --2- 172.21.15.47:0/3864048863 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x56003f980ab0 0x56003f982ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:04:42.946 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.942+0000 7f44ff7d5740 4 client.15693 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-17T06:04:42.946 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.942+0000 7f44ff7d5740 1 -- 172.21.15.47:0/3864048863 --> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56003f8361e0 con 0x56003f980ab0 2024-04-17T06:04:42.946 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.942+0000 7f44f5d9b640 1 --2- 172.21.15.47:0/3864048863 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x56003f980ab0 0x56003f982ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:04:42.946 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.943+0000 7f44ddffb640 10 client.15693 ms_handle_connect on v2:172.21.15.98:6838/2841287917 2024-04-17T06:04:42.946 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.943+0000 7f44f5d9b640 1 --2- 172.21.15.47:0/3864048863 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x56003f980ab0 0x56003f982ea0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fcf25c01a327742d server_cookie=71a3ac6a9a8c1318 in_seq=0 out_seq=0 2024-04-17T06:04:42.950 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.951+0000 7f44ddffb640 1 -- 172.21.15.47:0/3864048863 <== mds.0 v2:172.21.15.98:6838/2841287917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56003f8361e0 con 0x56003f980ab0 2024-04-17T06:04:42.950 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.951+0000 7f44ddffb640 10 client.15693 handle_command_reply: tid=0 2024-04-17T06:04:42.950 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.951+0000 7f44ff7d5740 1 client.15693 shutdown 2024-04-17T06:04:42.951 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.951+0000 7f44ff7d5740 1 -- 172.21.15.47:0/3864048863 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56003f9000c0 msgr2=0x56003f90d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:04:42.951 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.951+0000 7f44ff7d5740 1 --2- 172.21.15.47:0/3864048863 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56003f9000c0 0x56003f90d810 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f44e80986f0 tx=0x7f44e80cdb20 comp rx=0 tx=0).stop 2024-04-17T06:04:42.951 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.951+0000 7f44ff7d5740 1 -- 172.21.15.47:0/3864048863 shutdown_connections 2024-04-17T06:04:42.951 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.951+0000 7f44ff7d5740 1 --2- 172.21.15.47:0/3864048863 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x56003f980ab0 0x56003f982ea0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:42.951 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.951+0000 7f44ff7d5740 1 --2- 172.21.15.47:0/3864048863 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56003f9097b0 0x56003f97f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:42.951 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.951+0000 7f44ff7d5740 1 --2- 172.21.15.47:0/3864048863 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56003f9009e0 0x56003f90dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:42.951 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.951+0000 7f44ddffb640 0 client.15693 ms_handle_reset on v2:172.21.15.98:6838/2841287917 2024-04-17T06:04:42.951 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.951+0000 7f44ff7d5740 1 --2- 172.21.15.47:0/3864048863 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56003f9000c0 0x56003f90d810 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:42.951 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.951+0000 7f44ff7d5740 1 -- 172.21.15.47:0/3864048863 >> 172.21.15.47:0/3864048863 conn(0x56003f8fa9e0 msgr2=0x56003f8fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:04:42.951 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.951+0000 7f44ff7d5740 1 -- 172.21.15.47:0/3864048863 shutdown_connections 2024-04-17T06:04:42.951 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.951+0000 7f44ff7d5740 1 -- 172.21.15.47:0/3864048863 wait complete. 2024-04-17T06:04:42.951 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.951+0000 7f44ff7d5740 20 client.15693 trim_cache size 0 max 16384 2024-04-17T06:04:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-04-17T06:04:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-04-17T06:04:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-04-17T06:04:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-04-17T06:04:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.952+0000 7f44fcb0e640 1 -- 172.21.15.47:0/591755037 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f44e00a3df0 msgr2=0x7f44e00a62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:04:42.953 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.952+0000 7f44fcb0e640 1 --2- 172.21.15.47:0/591755037 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f44e00a3df0 0x7f44e00a62b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x56003f8fbe00 tx=0x7f44f0009ac0 comp rx=0 tx=0).stop 2024-04-17T06:04:42.953 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.952+0000 7f44fcb0e640 1 -- 172.21.15.47:0/591755037 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44f8101100 msgr2=0x7f44f8102b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:04:42.953 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.952+0000 7f44fcb0e640 1 --2- 172.21.15.47:0/591755037 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44f8101100 0x7f44f8102b40 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f44e800d770 tx=0x7f44e800dc30 comp rx=0 tx=0).stop 2024-04-17T06:04:42.953 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.952+0000 7f44fcb0e640 1 -- 172.21.15.47:0/591755037 shutdown_connections 2024-04-17T06:04:42.953 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.952+0000 7f44fcb0e640 1 --2- 172.21.15.47:0/591755037 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f44f81035c0 0x7f44f81a69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:42.953 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.952+0000 7f44fcb0e640 1 --2- 172.21.15.47:0/591755037 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f44f8106180 0x7f44f8103080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:42.953 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.952+0000 7f44fcb0e640 1 --2- 172.21.15.47:0/591755037 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f44e00a3df0 0x7f44e00a62b0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:42.953 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.952+0000 7f44fcb0e640 1 --2- 172.21.15.47:0/591755037 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44f8101100 0x7f44f8102b40 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:42.953 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.952+0000 7f44fcb0e640 1 -- 172.21.15.47:0/591755037 >> 172.21.15.47:0/591755037 conn(0x7f44f8077fe0 msgr2=0x7f44f80699c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:04:42.953 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.952+0000 7f44fcb0e640 1 -- 172.21.15.47:0/591755037 shutdown_connections 2024-04-17T06:04:42.953 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:42.952+0000 7f44fcb0e640 1 -- 172.21.15.47:0/591755037 wait complete. 2024-04-17T06:04:43.783 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:04:43 smithi103 ceph-mon[36243]: pgmap v107: 97 pgs: 97 active+clean; 2.0 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 9.7 KiB/s wr, 11 op/s 2024-04-17T06:04:43.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:04:43 smithi098 ceph-mon[36516]: pgmap v107: 97 pgs: 97 active+clean; 2.0 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 9.7 KiB/s wr, 11 op/s 2024-04-17T06:04:43.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:04:43 smithi047 ceph-mon[32223]: pgmap v107: 97 pgs: 97 active+clean; 2.0 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 9.7 KiB/s wr, 11 op/s 2024-04-17T06:04:45.783 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:04:45 smithi103 ceph-mon[36243]: pgmap v108: 97 pgs: 97 active+clean; 2.0 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 18 op/s 2024-04-17T06:04:45.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:04:45 smithi098 ceph-mon[36516]: pgmap v108: 97 pgs: 97 active+clean; 2.0 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 18 op/s 2024-04-17T06:04:45.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:04:45 smithi047 ceph-mon[32223]: pgmap v108: 97 pgs: 97 active+clean; 2.0 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 18 op/s 2024-04-17T06:04:47.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:04:47 smithi103 ceph-mon[36243]: pgmap v109: 97 pgs: 97 active+clean; 2.0 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 9.4 KiB/s wr, 11 op/s 2024-04-17T06:04:47.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:04:47 smithi098 ceph-mon[36516]: pgmap v109: 97 pgs: 97 active+clean; 2.0 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 9.4 KiB/s wr, 11 op/s 2024-04-17T06:04:47.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:04:47 smithi047 ceph-mon[32223]: pgmap v109: 97 pgs: 97 active+clean; 2.0 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 9.4 KiB/s wr, 11 op/s 2024-04-17T06:04:49.783 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:04:49 smithi103 ceph-mon[36243]: pgmap v110: 97 pgs: 97 active+clean; 2.0 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 9.4 KiB/s wr, 11 op/s 2024-04-17T06:04:49.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:04:49 smithi098 ceph-mon[36516]: pgmap v110: 97 pgs: 97 active+clean; 2.0 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 9.4 KiB/s wr, 11 op/s 2024-04-17T06:04:49.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:04:49 smithi047 ceph-mon[32223]: pgmap v110: 97 pgs: 97 active+clean; 2.0 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 9.4 KiB/s wr, 11 op/s 2024-04-17T06:04:51.783 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:04:51 smithi103 ceph-mon[36243]: pgmap v111: 97 pgs: 97 active+clean; 2.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-17T06:04:51.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:04:51 smithi098 ceph-mon[36516]: pgmap v111: 97 pgs: 97 active+clean; 2.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-17T06:04:51.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:04:51 smithi047 ceph-mon[32223]: pgmap v111: 97 pgs: 97 active+clean; 2.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-17T06:04:53.783 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:04:53 smithi103 ceph-mon[36243]: pgmap v112: 97 pgs: 97 active+clean; 2.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 9.3 KiB/s wr, 11 op/s 2024-04-17T06:04:53.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:04:53 smithi098 ceph-mon[36516]: pgmap v112: 97 pgs: 97 active+clean; 2.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 9.3 KiB/s wr, 11 op/s 2024-04-17T06:04:53.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:04:53 smithi047 ceph-mon[32223]: pgmap v112: 97 pgs: 97 active+clean; 2.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 9.3 KiB/s wr, 11 op/s 2024-04-17T06:04:55.783 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:04:55 smithi103 ceph-mon[36243]: pgmap v113: 97 pgs: 97 active+clean; 2.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-04-17T06:04:55.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:04:55 smithi098 ceph-mon[36516]: pgmap v113: 97 pgs: 97 active+clean; 2.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-04-17T06:04:55.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:04:55 smithi047 ceph-mon[32223]: pgmap v113: 97 pgs: 97 active+clean; 2.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-04-17T06:04:56.052 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.052+0000 7f6814722640 1 -- 172.21.15.47:0/3641100028 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f680c103ab0 msgr2=0x7f680c10bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:04:56.052 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.052+0000 7f6814722640 1 --2- 172.21.15.47:0/3641100028 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f680c103ab0 0x7f680c10bf90 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f67fc009900 tx=0x7f67fc02bcd0 comp rx=0 tx=0).stop 2024-04-17T06:04:56.053 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.053+0000 7f6814722640 1 -- 172.21.15.47:0/3641100028 shutdown_connections 2024-04-17T06:04:56.053 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.053+0000 7f6814722640 1 --2- 172.21.15.47:0/3641100028 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f680c10c560 0x7f680c10e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:56.053 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.053+0000 7f6814722640 1 --2- 172.21.15.47:0/3641100028 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f680c103ab0 0x7f680c10bf90 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:56.053 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.053+0000 7f6814722640 1 --2- 172.21.15.47:0/3641100028 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f680c103190 0x7f680c103570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:56.053 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.053+0000 7f6814722640 1 -- 172.21.15.47:0/3641100028 >> 172.21.15.47:0/3641100028 conn(0x7f680c077fe0 msgr2=0x7f680c0783f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:04:56.053 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.053+0000 7f6814722640 1 -- 172.21.15.47:0/3641100028 shutdown_connections 2024-04-17T06:04:56.053 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.053+0000 7f6814722640 1 -- 172.21.15.47:0/3641100028 wait complete. 2024-04-17T06:04:56.053 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.053+0000 7f6814722640 1 Processor -- start 2024-04-17T06:04:56.053 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.053+0000 7f6814722640 1 -- start start 2024-04-17T06:04:56.053 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.053+0000 7f6814722640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f680c103190 0x7f680c19e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:04:56.053 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.053+0000 7f6814722640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f680c103ab0 0x7f680c19e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:04:56.053 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.053+0000 7f6814722640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f680c10c560 0x7f680c198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:04:56.053 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.053+0000 7f6814722640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f680c112330 con 0x7f680c103ab0 2024-04-17T06:04:56.054 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.054+0000 7f6814722640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f680c1121b0 con 0x7f680c103190 2024-04-17T06:04:56.054 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.054+0000 7f6814722640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f680c1124b0 con 0x7f680c10c560 2024-04-17T06:04:56.054 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.054+0000 7f6811cbd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f680c103ab0 0x7f680c19e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:04:56.054 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.054+0000 7f6811cbd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f680c103ab0 0x7f680c19e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38246/0 (socket says 172.21.15.47:38246) 2024-04-17T06:04:56.054 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.054+0000 7f6811cbd640 1 -- 172.21.15.47:0/1051463750 learned_addr learned my addr 172.21.15.47:0/1051463750 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:04:56.054 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.054+0000 7f68124be640 1 --2- 172.21.15.47:0/1051463750 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f680c103190 0x7f680c19e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:04:56.054 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.054+0000 7f6812cbf640 1 --2- 172.21.15.47:0/1051463750 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f680c10c560 0x7f680c198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:04:56.054 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.054+0000 7f6811cbd640 1 -- 172.21.15.47:0/1051463750 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f680c10c560 msgr2=0x7f680c198690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:04:56.054 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.054+0000 7f6811cbd640 1 --2- 172.21.15.47:0/1051463750 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f680c10c560 0x7f680c198690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:56.054 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.054+0000 7f6811cbd640 1 -- 172.21.15.47:0/1051463750 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f680c103190 msgr2=0x7f680c19e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:04:56.054 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.054+0000 7f6811cbd640 1 --2- 172.21.15.47:0/1051463750 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f680c103190 0x7f680c19e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:56.054 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.054+0000 7f6811cbd640 1 -- 172.21.15.47:0/1051463750 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f680c198e10 con 0x7f680c103ab0 2024-04-17T06:04:56.054 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.054+0000 7f6811cbd640 1 --2- 172.21.15.47:0/1051463750 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f680c103ab0 0x7f680c19e980 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f67fc005940 tx=0x7f67fc02cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:04:56.054 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.054+0000 7f68037fe640 1 -- 172.21.15.47:0/1051463750 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67fc041070 con 0x7f680c103ab0 2024-04-17T06:04:56.055 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.054+0000 7f68037fe640 1 -- 172.21.15.47:0/1051463750 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67fc004450 con 0x7f680c103ab0 2024-04-17T06:04:56.055 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.054+0000 7f6814722640 1 -- 172.21.15.47:0/1051463750 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f680c1990a0 con 0x7f680c103ab0 2024-04-17T06:04:56.055 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.054+0000 7f68037fe640 1 -- 172.21.15.47:0/1051463750 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67fc003df0 con 0x7f680c103ab0 2024-04-17T06:04:56.055 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.055+0000 7f6814722640 1 -- 172.21.15.47:0/1051463750 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f680c1a5300 con 0x7f680c103ab0 2024-04-17T06:04:56.062 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.057+0000 7f68037fe640 1 -- 172.21.15.47:0/1051463750 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f67fc044070 con 0x7f680c103ab0 2024-04-17T06:04:56.062 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.057+0000 7f68173e0740 1 Processor -- start 2024-04-17T06:04:56.062 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.058+0000 7f68173e0740 1 -- start start 2024-04-17T06:04:56.062 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.058+0000 7f68173e0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55898e1f30c0 0x55898e1f34a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:04:56.062 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.058+0000 7f68173e0740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55898e1f39e0 0x55898e1fc270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:04:56.062 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.058+0000 7f68037fe640 1 --2- 172.21.15.47:0/1051463750 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f67f80a3dc0 0x7f67f80a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:04:56.062 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.058+0000 7f68173e0740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55898e1fc7b0 0x55898e1feba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:04:56.062 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.058+0000 7f68173e0740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55898e1587d0 con 0x55898e1f39e0 2024-04-17T06:04:56.062 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.058+0000 7f68173e0740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55898e0f5f70 con 0x55898e1fc7b0 2024-04-17T06:04:56.062 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.058+0000 7f68173e0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55898e154790 con 0x55898e1f30c0 2024-04-17T06:04:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.058+0000 7f68124be640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55898e1f39e0 0x55898e1fc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:04:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.058+0000 7f68124be640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55898e1f39e0 0x55898e1fc270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38254/0 (socket says 172.21.15.47:38254) 2024-04-17T06:04:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.058+0000 7f68124be640 1 -- 172.21.15.47:0/2804852285 learned_addr learned my addr 172.21.15.47:0/2804852285 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:04:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.058+0000 7f68037fe640 1 -- 172.21.15.47:0/1051463750 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f67fc0f2250 con 0x7f680c103ab0 2024-04-17T06:04:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.058+0000 7f6811cbd640 1 --2- 172.21.15.47:0/1051463750 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f67f80a3dc0 0x7f67f80a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:04:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.058+0000 7f6812cbf640 1 --2- 172.21.15.47:0/2804852285 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55898e1f30c0 0x55898e1f34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:04:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.058+0000 7f6812cbf640 1 --2- 172.21.15.47:0/2804852285 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55898e1fc7b0 0x55898e1feba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:04:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.058+0000 7f68124be640 1 -- 172.21.15.47:0/2804852285 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55898e1f30c0 msgr2=0x55898e1f34a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:04:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.058+0000 7f68124be640 1 --2- 172.21.15.47:0/2804852285 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55898e1f30c0 0x55898e1f34a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.058+0000 7f68124be640 1 -- 172.21.15.47:0/2804852285 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55898e1fc7b0 msgr2=0x55898e1feba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:04:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.058+0000 7f68124be640 1 --2- 172.21.15.47:0/2804852285 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55898e1fc7b0 0x55898e1feba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.058+0000 7f68124be640 1 -- 172.21.15.47:0/2804852285 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55898de6b210 con 0x55898e1f39e0 2024-04-17T06:04:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.058+0000 7f6812cbf640 1 --2- 172.21.15.47:0/2804852285 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55898e1fc7b0 0x55898e1feba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:04:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.059+0000 7f6811cbd640 1 --2- 172.21.15.47:0/1051463750 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f67f80a3dc0 0x7f67f80a6280 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f67fc006010 tx=0x7f67fc088ed0 comp rx=0 tx=0).ready entity=mgr.24923 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:04:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.059+0000 7f68124be640 1 --2- 172.21.15.47:0/2804852285 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55898e1f39e0 0x55898e1fc270 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f68080103f0 tx=0x7f68080108b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfa51a27621e97b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:04:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.059+0000 7f6800ff9640 1 -- 172.21.15.47:0/2804852285 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6808016070 con 0x55898e1f39e0 2024-04-17T06:04:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.059+0000 7f6800ff9640 1 -- 172.21.15.47:0/2804852285 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6808004d20 con 0x55898e1f39e0 2024-04-17T06:04:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.059+0000 7f6800ff9640 1 -- 172.21.15.47:0/2804852285 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6808005400 con 0x55898e1f39e0 2024-04-17T06:04:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.059+0000 7f68173e0740 1 -- 172.21.15.47:0/2804852285 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55898e1f39e0 msgr2=0x55898e1fc270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:04:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.060+0000 7f68173e0740 1 --2- 172.21.15.47:0/2804852285 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55898e1f39e0 0x55898e1fc270 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f68080103f0 tx=0x7f68080108b0 comp rx=0 tx=0).stop 2024-04-17T06:04:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.060+0000 7f68173e0740 1 -- 172.21.15.47:0/2804852285 shutdown_connections 2024-04-17T06:04:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.060+0000 7f68173e0740 1 --2- 172.21.15.47:0/2804852285 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55898e1fc7b0 0x55898e1feba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.060+0000 7f68173e0740 1 --2- 172.21.15.47:0/2804852285 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55898e1f39e0 0x55898e1fc270 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.060+0000 7f68173e0740 1 --2- 172.21.15.47:0/2804852285 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55898e1f30c0 0x55898e1f34a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.060+0000 7f68173e0740 1 -- 172.21.15.47:0/2804852285 >> 172.21.15.47:0/2804852285 conn(0x55898e1ed9e0 msgr2=0x55898e1eddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:04:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.060+0000 7f68173e0740 1 -- 172.21.15.47:0/2804852285 shutdown_connections 2024-04-17T06:04:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.060+0000 7f68173e0740 1 -- 172.21.15.47:0/2804852285 wait complete. 2024-04-17T06:04:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.060+0000 7f68173e0740 1 -- start start 2024-04-17T06:04:56.065 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.060+0000 7f68173e0740 1 Processor -- start 2024-04-17T06:04:56.065 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.060+0000 7f68173e0740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55898e1f30c0 0x55898e1f9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:04:56.065 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.060+0000 7f68173e0740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55898e1f39e0 0x55898e1f9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:04:56.065 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.060+0000 7f68173e0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55898e1fc7b0 0x55898e1fa1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:04:56.065 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.060+0000 7f68173e0740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55898e1ff3e0 con 0x55898e1f39e0 2024-04-17T06:04:56.065 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.060+0000 7f68173e0740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55898e0e0e90 con 0x55898e1f30c0 2024-04-17T06:04:56.065 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.060+0000 7f68173e0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55898e1ff560 con 0x55898e1fc7b0 2024-04-17T06:04:56.065 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.061+0000 7f68173e0740 10 client.0 _ll_register_callbacks cb 0x55898e0f6d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:04:56.065 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.061+0000 7f68124be640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55898e1f39e0 0x55898e1f9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:04:56.065 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.061+0000 7f68124be640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55898e1f39e0 0x55898e1f9ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38258/0 (socket says 172.21.15.47:38258) 2024-04-17T06:04:56.065 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.061+0000 7f68124be640 1 -- 172.21.15.47:0/2063403103 learned_addr learned my addr 172.21.15.47:0/2063403103 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:04:56.065 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.061+0000 7f6812cbf640 1 --2- 172.21.15.47:0/2063403103 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55898e1f30c0 0x55898e1f9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:04:56.065 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.061+0000 7f68124be640 1 -- 172.21.15.47:0/2063403103 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55898e1fc7b0 msgr2=0x55898e1fa1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:04:56.066 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.061+0000 7f68124be640 1 --2- 172.21.15.47:0/2063403103 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55898e1fc7b0 0x55898e1fa1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:56.066 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.061+0000 7f68124be640 1 -- 172.21.15.47:0/2063403103 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55898e1f30c0 msgr2=0x55898e1f9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:04:56.066 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.061+0000 7f68124be640 1 --2- 172.21.15.47:0/2063403103 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55898e1f30c0 0x55898e1f9760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:56.066 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.061+0000 7f68124be640 1 -- 172.21.15.47:0/2063403103 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55898df37620 con 0x55898e1f39e0 2024-04-17T06:04:56.066 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.061+0000 7f6812cbf640 1 --2- 172.21.15.47:0/2063403103 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55898e1f30c0 0x55898e1f9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:04:56.066 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.061+0000 7f68124be640 1 --2- 172.21.15.47:0/2063403103 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55898e1f39e0 0x55898e1f9ca0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f6808006010 tx=0x7f6808004c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a40de00c85c1277 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:04:56.066 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.061+0000 7f67f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-17T06:04:56.066 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.061+0000 7f67f5ffb640 1 -- 172.21.15.47:0/2063403103 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6808029060 con 0x55898e1f39e0 2024-04-17T06:04:56.066 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.061+0000 7f67f5ffb640 1 -- 172.21.15.47:0/2063403103 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f6808012610 con 0x55898e1f39e0 2024-04-17T06:04:56.066 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.061+0000 7f68173e0740 1 -- 172.21.15.47:0/2063403103 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6808004d20 con 0x55898e1f39e0 2024-04-17T06:04:56.066 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.061+0000 7f67f5ffb640 1 -- 172.21.15.47:0/2063403103 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6808016030 con 0x55898e1f39e0 2024-04-17T06:04:56.066 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.061+0000 7f68173e0740 10 client.15705 fetch_fsmap learned FSMap version 27 2024-04-17T06:04:56.066 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.062+0000 7f68173e0740 1 -- 172.21.15.47:0/2063403103 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55898e20b600 con 0x55898e1f39e0 2024-04-17T06:04:56.066 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.062+0000 7f67f5ffb640 1 -- 172.21.15.47:0/2063403103 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6808004d20 con 0x55898e1f39e0 2024-04-17T06:04:56.067 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.062+0000 7f67f5ffb640 1 -- 172.21.15.47:0/2063403103 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 27) ==== 8343+0+0 (secure 0 0 0) 0x7f680800f040 con 0x55898e1f39e0 2024-04-17T06:04:56.067 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.062+0000 7f68173e0740 10 client.15705 fetch_fsmap finished waiting for FSMap version 27 2024-04-17T06:04:56.067 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.062+0000 7f68173e0740 10 client.15705 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-17T06:04:56.067 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.062+0000 7f68173e0740 20 client.15705 populate_metadata read hostname 'smithi047' 2024-04-17T06:04:56.067 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.062+0000 7f68173e0740 1 --2- 172.21.15.47:0/2063403103 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55898e20ca00 0x55898e20edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:04:56.067 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.062+0000 7f68173e0740 4 client.15705 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T06:04:56.067 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.062+0000 7f68173e0740 1 -- 172.21.15.47:0/2063403103 --> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55898e1661f0 con 0x55898e20ca00 2024-04-17T06:04:56.067 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.062+0000 7f6812cbf640 1 --2- 172.21.15.47:0/2063403103 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55898e20ca00 0x55898e20edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:04:56.067 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.063+0000 7f67f5ffb640 10 client.15705 ms_handle_connect on v2:172.21.15.98:6838/2841287917 2024-04-17T06:04:56.067 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.063+0000 7f6812cbf640 1 --2- 172.21.15.47:0/2063403103 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55898e20ca00 0x55898e20edf0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5339ddeaab83b0e server_cookie=435f372a32c1bbd8 in_seq=0 out_seq=0 2024-04-17T06:04:56.067 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.063+0000 7f67f5ffb640 1 -- 172.21.15.47:0/2063403103 <== mds.0 v2:172.21.15.98:6838/2841287917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55898e1661f0 con 0x55898e20ca00 2024-04-17T06:04:56.067 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.063+0000 7f67f5ffb640 10 client.15705 handle_command_reply: tid=0 2024-04-17T06:04:56.067 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.063+0000 7f68173e0740 1 client.15705 shutdown 2024-04-17T06:04:56.067 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.064+0000 7f68173e0740 1 -- 172.21.15.47:0/2063403103 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55898e1f39e0 msgr2=0x55898e1f9ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:04:56.068 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.064+0000 7f68173e0740 1 --2- 172.21.15.47:0/2063403103 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55898e1f39e0 0x55898e1f9ca0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f6808006010 tx=0x7f6808004c70 comp rx=0 tx=0).stop 2024-04-17T06:04:56.068 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.064+0000 7f68173e0740 1 -- 172.21.15.47:0/2063403103 shutdown_connections 2024-04-17T06:04:56.068 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.064+0000 7f68173e0740 1 --2- 172.21.15.47:0/2063403103 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55898e20ca00 0x55898e20edf0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:56.068 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.064+0000 7f68173e0740 1 --2- 172.21.15.47:0/2063403103 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55898e1fc7b0 0x55898e1fa1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:56.068 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.064+0000 7f68173e0740 1 --2- 172.21.15.47:0/2063403103 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55898e1f39e0 0x55898e1f9ca0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:56.068 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.064+0000 7f67f5ffb640 0 client.15705 ms_handle_reset on v2:172.21.15.98:6838/2841287917 2024-04-17T06:04:56.068 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.064+0000 7f68173e0740 1 --2- 172.21.15.47:0/2063403103 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55898e1f30c0 0x55898e1f9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:56.068 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.064+0000 7f68173e0740 1 -- 172.21.15.47:0/2063403103 >> 172.21.15.47:0/2063403103 conn(0x55898e1ed9e0 msgr2=0x55898e1ef730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:04:56.068 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.064+0000 7f68173e0740 1 -- 172.21.15.47:0/2063403103 shutdown_connections 2024-04-17T06:04:56.068 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.064+0000 7f68173e0740 1 -- 172.21.15.47:0/2063403103 wait complete. 2024-04-17T06:04:56.068 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.064+0000 7f68173e0740 20 client.15705 trim_cache size 0 max 16384 2024-04-17T06:04:56.081 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.079+0000 7f68173e0740 1 Processor -- start 2024-04-17T06:04:56.081 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.079+0000 7f68173e0740 1 -- start start 2024-04-17T06:04:56.081 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.079+0000 7f68173e0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55898e1f30c0 0x55898e1fbe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:04:56.082 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.079+0000 7f68173e0740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55898e1f39e0 0x55898e2082a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:04:56.082 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.079+0000 7f68173e0740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55898e1fc7b0 0x55898e2144a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:04:56.082 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.079+0000 7f68173e0740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f680c1b04f0 con 0x55898e1f39e0 2024-04-17T06:04:56.082 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.079+0000 7f68173e0740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f680c1add80 con 0x55898e1fc7b0 2024-04-17T06:04:56.082 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.079+0000 7f68173e0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f680c1abbd0 con 0x55898e1f30c0 2024-04-17T06:04:56.082 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.079+0000 7f68124be640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55898e1f39e0 0x55898e2082a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:04:56.082 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.079+0000 7f68124be640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55898e1f39e0 0x55898e2082a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38270/0 (socket says 172.21.15.47:38270) 2024-04-17T06:04:56.082 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.079+0000 7f68124be640 1 -- 172.21.15.47:0/4089867825 learned_addr learned my addr 172.21.15.47:0/4089867825 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:04:56.082 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.079+0000 7f6812cbf640 1 --2- 172.21.15.47:0/4089867825 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55898e1f30c0 0x55898e1fbe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:04:56.082 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.079+0000 7f6812cbf640 1 --2- 172.21.15.47:0/4089867825 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55898e1fc7b0 0x55898e2144a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:04:56.082 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.079+0000 7f68124be640 1 -- 172.21.15.47:0/4089867825 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55898e1f30c0 msgr2=0x55898e1fbe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:04:56.082 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.079+0000 7f68124be640 1 --2- 172.21.15.47:0/4089867825 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55898e1f30c0 0x55898e1fbe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:56.082 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.079+0000 7f68124be640 1 -- 172.21.15.47:0/4089867825 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55898e1fc7b0 msgr2=0x55898e2144a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:04:56.082 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.079+0000 7f68124be640 1 --2- 172.21.15.47:0/4089867825 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55898e1fc7b0 0x55898e2144a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:56.083 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.079+0000 7f68124be640 1 -- 172.21.15.47:0/4089867825 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55898e1eec10 con 0x55898e1f39e0 2024-04-17T06:04:56.083 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.080+0000 7f68124be640 1 --2- 172.21.15.47:0/4089867825 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55898e1f39e0 0x55898e2082a0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f68080108f0 tx=0x7f6808009ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c1e7639d74f91ea server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:04:56.083 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.080+0000 7f67f7fff640 1 -- 172.21.15.47:0/4089867825 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6808029070 con 0x55898e1f39e0 2024-04-17T06:04:56.083 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.080+0000 7f67f7fff640 1 -- 172.21.15.47:0/4089867825 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6808005420 con 0x55898e1f39e0 2024-04-17T06:04:56.083 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.080+0000 7f67f7fff640 1 -- 172.21.15.47:0/4089867825 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6808003ea0 con 0x55898e1f39e0 2024-04-17T06:04:56.083 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.080+0000 7f68173e0740 1 -- 172.21.15.47:0/4089867825 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55898e1f39e0 msgr2=0x55898e2082a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:04:56.083 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.080+0000 7f68173e0740 1 --2- 172.21.15.47:0/4089867825 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55898e1f39e0 0x55898e2082a0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f68080108f0 tx=0x7f6808009ce0 comp rx=0 tx=0).stop 2024-04-17T06:04:56.083 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.080+0000 7f68173e0740 1 -- 172.21.15.47:0/4089867825 shutdown_connections 2024-04-17T06:04:56.083 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.080+0000 7f68173e0740 1 --2- 172.21.15.47:0/4089867825 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55898e1fc7b0 0x55898e2144a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:56.083 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.080+0000 7f68173e0740 1 --2- 172.21.15.47:0/4089867825 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55898e1f39e0 0x55898e2082a0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:56.083 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.080+0000 7f68173e0740 1 --2- 172.21.15.47:0/4089867825 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55898e1f30c0 0x55898e1fbe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:56.083 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.080+0000 7f68173e0740 1 -- 172.21.15.47:0/4089867825 >> 172.21.15.47:0/4089867825 conn(0x55898e1ed9e0 msgr2=0x55898e1f3f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:04:56.083 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.080+0000 7f68173e0740 1 -- 172.21.15.47:0/4089867825 shutdown_connections 2024-04-17T06:04:56.084 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.080+0000 7f68173e0740 1 -- 172.21.15.47:0/4089867825 wait complete. 2024-04-17T06:04:56.084 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.081+0000 7f68173e0740 1 -- start start 2024-04-17T06:04:56.084 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.081+0000 7f68173e0740 1 Processor -- start 2024-04-17T06:04:56.084 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.081+0000 7f68173e0740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55898e1f30c0 0x55898e2049a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:04:56.084 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.081+0000 7f68173e0740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55898e1f39e0 0x55898e1f5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:04:56.084 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.081+0000 7f68173e0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55898e1fc7b0 0x55898e1f6510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:04:56.084 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.081+0000 7f68173e0740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f680c1ac240 con 0x55898e1f30c0 2024-04-17T06:04:56.084 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.081+0000 7f68173e0740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f680c1ad6b0 con 0x55898e1f39e0 2024-04-17T06:04:56.084 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.081+0000 7f68173e0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f680c1ac910 con 0x55898e1fc7b0 2024-04-17T06:04:56.084 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.081+0000 7f68173e0740 10 client.0 _ll_register_callbacks cb 0x7f6808009ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:04:56.084 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.081+0000 7f6812cbf640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55898e1f30c0 0x55898e2049a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:04:56.084 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.081+0000 7f6812cbf640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55898e1f30c0 0x55898e2049a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38280/0 (socket says 172.21.15.47:38280) 2024-04-17T06:04:56.084 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.081+0000 7f6812cbf640 1 -- 172.21.15.47:0/1105220937 learned_addr learned my addr 172.21.15.47:0/1105220937 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:04:56.084 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.081+0000 7f68124be640 1 --2- 172.21.15.47:0/1105220937 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55898e1f39e0 0x55898e1f5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:04:56.084 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.081+0000 7f6812cbf640 1 --2- 172.21.15.47:0/1105220937 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55898e1fc7b0 0x55898e1f6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:04:56.085 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.081+0000 7f6812cbf640 1 -- 172.21.15.47:0/1105220937 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55898e1fc7b0 msgr2=0x55898e1f6510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:04:56.085 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.081+0000 7f6812cbf640 1 --2- 172.21.15.47:0/1105220937 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55898e1fc7b0 0x55898e1f6510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:56.085 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.081+0000 7f6812cbf640 1 -- 172.21.15.47:0/1105220937 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55898e1f39e0 msgr2=0x55898e1f5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:04:56.085 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.081+0000 7f6812cbf640 1 --2- 172.21.15.47:0/1105220937 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55898e1f39e0 0x55898e1f5fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:56.085 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.081+0000 7f6812cbf640 1 -- 172.21.15.47:0/1105220937 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55898e1f6a50 con 0x55898e1f30c0 2024-04-17T06:04:56.085 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.082+0000 7f6812cbf640 1 --2- 172.21.15.47:0/1105220937 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55898e1f30c0 0x55898e2049a0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f6804009c90 tx=0x7f6808009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a4f12d5117fd3cd server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:04:56.085 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.082+0000 7f67f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-17T06:04:56.085 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.082+0000 7f67f5ffb640 1 -- 172.21.15.47:0/1105220937 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6804011910 con 0x55898e1f30c0 2024-04-17T06:04:56.085 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.082+0000 7f67f5ffb640 1 -- 172.21.15.47:0/1105220937 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f6804019070 con 0x55898e1f30c0 2024-04-17T06:04:56.085 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.082+0000 7f68173e0740 1 -- 172.21.15.47:0/1105220937 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6808005420 con 0x55898e1f30c0 2024-04-17T06:04:56.085 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.082+0000 7f67f5ffb640 1 -- 172.21.15.47:0/1105220937 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6804011ab0 con 0x55898e1f30c0 2024-04-17T06:04:56.085 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.082+0000 7f68173e0740 10 client.15711 fetch_fsmap learned FSMap version 27 2024-04-17T06:04:56.085 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.082+0000 7f68173e0740 1 -- 172.21.15.47:0/1105220937 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55898e2729f0 con 0x55898e1f30c0 2024-04-17T06:04:56.085 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.082+0000 7f67f5ffb640 1 -- 172.21.15.47:0/1105220937 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6808005420 con 0x55898e1f30c0 2024-04-17T06:04:56.086 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.082+0000 7f67f5ffb640 1 -- 172.21.15.47:0/1105220937 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 27) ==== 8343+0+0 (secure 0 0 0) 0x7f680401ece0 con 0x55898e1f30c0 2024-04-17T06:04:56.086 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.082+0000 7f68173e0740 10 client.15711 fetch_fsmap finished waiting for FSMap version 27 2024-04-17T06:04:56.086 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.082+0000 7f68173e0740 10 client.15711 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-17T06:04:56.086 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.082+0000 7f68173e0740 20 client.15711 populate_metadata read hostname 'smithi047' 2024-04-17T06:04:56.086 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.082+0000 7f68173e0740 1 --2- 172.21.15.47:0/1105220937 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55898e2730f0 0x55898e275510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:04:56.086 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.082+0000 7f68173e0740 4 client.15711 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-17T06:04:56.086 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.082+0000 7f68173e0740 1 -- 172.21.15.47:0/1105220937 --> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55898e160060 con 0x55898e2730f0 2024-04-17T06:04:56.086 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.083+0000 7f68124be640 1 --2- 172.21.15.47:0/1105220937 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55898e2730f0 0x55898e275510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:04:56.086 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.083+0000 7f67f5ffb640 10 client.15711 ms_handle_connect on v2:172.21.15.98:6838/2841287917 2024-04-17T06:04:56.086 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.083+0000 7f68124be640 1 --2- 172.21.15.47:0/1105220937 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55898e2730f0 0x55898e275510 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3935856aece817b server_cookie=f8236ac0d9e16f74 in_seq=0 out_seq=0 2024-04-17T06:04:56.089 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.091+0000 7f67f5ffb640 1 -- 172.21.15.47:0/1105220937 <== mds.0 v2:172.21.15.98:6838/2841287917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55898e160060 con 0x55898e2730f0 2024-04-17T06:04:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.091+0000 7f67f5ffb640 10 client.15711 handle_command_reply: tid=0 2024-04-17T06:04:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.091+0000 7f68173e0740 1 client.15711 shutdown 2024-04-17T06:04:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.091+0000 7f68173e0740 1 -- 172.21.15.47:0/1105220937 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55898e1f30c0 msgr2=0x55898e2049a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:04:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.091+0000 7f68173e0740 1 --2- 172.21.15.47:0/1105220937 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55898e1f30c0 0x55898e2049a0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f6804009c90 tx=0x7f6808009da0 comp rx=0 tx=0).stop 2024-04-17T06:04:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.091+0000 7f68173e0740 1 -- 172.21.15.47:0/1105220937 shutdown_connections 2024-04-17T06:04:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.091+0000 7f68173e0740 1 --2- 172.21.15.47:0/1105220937 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55898e2730f0 0x55898e275510 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.091+0000 7f68173e0740 1 --2- 172.21.15.47:0/1105220937 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55898e1fc7b0 0x55898e1f6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.091+0000 7f68173e0740 1 --2- 172.21.15.47:0/1105220937 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55898e1f39e0 0x55898e1f5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.091+0000 7f67f5ffb640 0 client.15711 ms_handle_reset on v2:172.21.15.98:6838/2841287917 2024-04-17T06:04:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.091+0000 7f68173e0740 1 --2- 172.21.15.47:0/1105220937 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55898e1f30c0 0x55898e2049a0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.091+0000 7f68173e0740 1 -- 172.21.15.47:0/1105220937 >> 172.21.15.47:0/1105220937 conn(0x55898e1ed9e0 msgr2=0x55898e1eee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:04:56.091 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.091+0000 7f68173e0740 1 -- 172.21.15.47:0/1105220937 shutdown_connections 2024-04-17T06:04:56.091 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.091+0000 7f68173e0740 1 -- 172.21.15.47:0/1105220937 wait complete. 2024-04-17T06:04:56.091 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.091+0000 7f68173e0740 20 client.15711 trim_cache size 0 max 16384 2024-04-17T06:04:56.091 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-04-17T06:04:56.091 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-04-17T06:04:56.091 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-04-17T06:04:56.091 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-04-17T06:04:56.092 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.092+0000 7f6814722640 1 -- 172.21.15.47:0/1051463750 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f67f80a3dc0 msgr2=0x7f67f80a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:04:56.092 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.092+0000 7f6814722640 1 --2- 172.21.15.47:0/1051463750 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f67f80a3dc0 0x7f67f80a6280 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f67fc006010 tx=0x7f67fc088ed0 comp rx=0 tx=0).stop 2024-04-17T06:04:56.092 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.092+0000 7f6814722640 1 -- 172.21.15.47:0/1051463750 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f680c103ab0 msgr2=0x7f680c19e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:04:56.093 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.092+0000 7f6814722640 1 --2- 172.21.15.47:0/1051463750 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f680c103ab0 0x7f680c19e980 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f67fc005940 tx=0x7f67fc02cc40 comp rx=0 tx=0).stop 2024-04-17T06:04:56.093 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.092+0000 7f6814722640 1 -- 172.21.15.47:0/1051463750 shutdown_connections 2024-04-17T06:04:56.093 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.092+0000 7f6814722640 1 --2- 172.21.15.47:0/1051463750 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f680c10c560 0x7f680c198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:56.093 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.092+0000 7f6814722640 1 --2- 172.21.15.47:0/1051463750 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f67f80a3dc0 0x7f67f80a6280 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:56.093 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.092+0000 7f6814722640 1 --2- 172.21.15.47:0/1051463750 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f680c103ab0 0x7f680c19e980 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:56.093 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.092+0000 7f6814722640 1 --2- 172.21.15.47:0/1051463750 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f680c103190 0x7f680c19e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:04:56.093 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.092+0000 7f6814722640 1 -- 172.21.15.47:0/1051463750 >> 172.21.15.47:0/1051463750 conn(0x7f680c077fe0 msgr2=0x7f680c0fded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:04:56.093 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.092+0000 7f6814722640 1 -- 172.21.15.47:0/1051463750 shutdown_connections 2024-04-17T06:04:56.093 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:04:56.092+0000 7f6814722640 1 -- 172.21.15.47:0/1051463750 wait complete. 2024-04-17T06:04:57.783 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:04:57 smithi103 ceph-mon[36243]: pgmap v114: 97 pgs: 97 active+clean; 2.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.9 KiB/s wr, 9 op/s 2024-04-17T06:04:57.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:04:57 smithi098 ceph-mon[36516]: pgmap v114: 97 pgs: 97 active+clean; 2.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.9 KiB/s wr, 9 op/s 2024-04-17T06:04:57.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:04:57 smithi047 ceph-mon[32223]: pgmap v114: 97 pgs: 97 active+clean; 2.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.9 KiB/s wr, 9 op/s 2024-04-17T06:04:58.964 INFO:tasks.workunit.client.0.smithi047.stdout:64/600: open fd = 3 2024-04-17T06:04:58.964 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.965 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.965 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:04:58.965 INFO:tasks.workunit.client.0.smithi047.stdout:65/600: open fd = 3 2024-04-17T06:04:58.965 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.965 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.965 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:04:58.965 INFO:tasks.workunit.client.0.smithi047.stdout:66/600: open fd = 3 2024-04-17T06:04:58.965 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.965 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.965 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:04:58.965 INFO:tasks.workunit.client.0.smithi047.stdout:67/600: open fd = 3 2024-04-17T06:04:58.965 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.966 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.966 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:04:58.966 INFO:tasks.workunit.client.0.smithi047.stdout:68/600: open fd = 3 2024-04-17T06:04:58.966 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.966 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.966 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:04:58.966 INFO:tasks.workunit.client.0.smithi047.stdout:69/600: open fd = 3 2024-04-17T06:04:58.966 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.966 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.966 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:04:58.966 INFO:tasks.workunit.client.0.smithi047.stdout:70/600: open fd = 3 2024-04-17T06:04:58.966 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.966 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.967 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:04:58.967 INFO:tasks.workunit.client.0.smithi047.stdout:71/600: open fd = 3 2024-04-17T06:04:58.967 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.967 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.967 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:04:58.967 INFO:tasks.workunit.client.0.smithi047.stdout:72/600: open fd = 3 2024-04-17T06:04:58.967 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.967 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.967 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:04:58.967 INFO:tasks.workunit.client.0.smithi047.stdout:73/600: open fd = 3 2024-04-17T06:04:58.967 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.967 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.967 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:04:58.968 INFO:tasks.workunit.client.0.smithi047.stdout:74/600: open fd = 3 2024-04-17T06:04:58.968 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.968 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.968 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:04:58.968 INFO:tasks.workunit.client.0.smithi047.stdout:75/600: open fd = 3 2024-04-17T06:04:58.968 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.968 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.968 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:04:58.968 INFO:tasks.workunit.client.0.smithi047.stdout:76/600: open fd = 3 2024-04-17T06:04:58.968 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.968 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.968 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:04:58.968 INFO:tasks.workunit.client.0.smithi047.stdout:77/600: open fd = 3 2024-04-17T06:04:58.969 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.969 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.969 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:04:58.969 INFO:tasks.workunit.client.0.smithi047.stdout:78/600: open fd = 3 2024-04-17T06:04:58.969 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.969 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.969 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:04:58.969 INFO:tasks.workunit.client.0.smithi047.stdout:79/600: open fd = 3 2024-04-17T06:04:58.969 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.969 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.969 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:04:58.969 INFO:tasks.workunit.client.0.smithi047.stdout:80/600: open fd = 3 2024-04-17T06:04:58.969 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.969 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.969 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:04:58.970 INFO:tasks.workunit.client.0.smithi047.stdout:81/600: open fd = 3 2024-04-17T06:04:58.970 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.970 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.970 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:04:58.970 INFO:tasks.workunit.client.0.smithi047.stdout:82/600: open fd = 3 2024-04-17T06:04:58.970 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.970 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.970 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:04:58.970 INFO:tasks.workunit.client.0.smithi047.stdout:83/600: open fd = 3 2024-04-17T06:04:58.970 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.970 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.970 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:04:58.970 INFO:tasks.workunit.client.0.smithi047.stdout:84/600: open fd = 3 2024-04-17T06:04:58.971 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.971 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.971 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:04:58.971 INFO:tasks.workunit.client.0.smithi047.stdout:85/600: open fd = 3 2024-04-17T06:04:58.971 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.971 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.971 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:04:58.971 INFO:tasks.workunit.client.0.smithi047.stdout:86/600: open fd = 3 2024-04-17T06:04:58.971 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.971 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.971 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:04:58.971 INFO:tasks.workunit.client.0.smithi047.stdout:87/600: open fd = 3 2024-04-17T06:04:58.971 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.971 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.971 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:04:58.972 INFO:tasks.workunit.client.0.smithi047.stdout:88/600: open fd = 3 2024-04-17T06:04:58.972 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.972 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.972 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:04:58.972 INFO:tasks.workunit.client.0.smithi047.stdout:89/600: open fd = 3 2024-04-17T06:04:58.972 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.972 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.972 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:04:58.972 INFO:tasks.workunit.client.0.smithi047.stdout:90/600: open fd = 3 2024-04-17T06:04:58.972 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.972 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.972 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:04:58.972 INFO:tasks.workunit.client.0.smithi047.stdout:91/600: open fd = 3 2024-04-17T06:04:58.972 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.972 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.973 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:04:58.973 INFO:tasks.workunit.client.0.smithi047.stdout:92/600: open fd = 3 2024-04-17T06:04:58.973 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.973 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.973 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:04:58.973 INFO:tasks.workunit.client.0.smithi047.stdout:93/600: open fd = 3 2024-04-17T06:04:58.973 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.973 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.973 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:04:58.973 INFO:tasks.workunit.client.0.smithi047.stdout:94/600: open fd = 3 2024-04-17T06:04:58.973 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.973 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.973 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:04:58.973 INFO:tasks.workunit.client.0.smithi047.stdout:95/600: open fd = 3 2024-04-17T06:04:58.973 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.974 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.974 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:04:58.974 INFO:tasks.workunit.client.0.smithi047.stdout:96/600: open fd = 3 2024-04-17T06:04:58.974 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.974 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.974 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:04:58.974 INFO:tasks.workunit.client.0.smithi047.stdout:97/600: open fd = 3 2024-04-17T06:04:58.974 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.974 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.974 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:04:58.974 INFO:tasks.workunit.client.0.smithi047.stdout:98/600: open fd = 3 2024-04-17T06:04:58.974 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.974 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.975 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:04:58.975 INFO:tasks.workunit.client.0.smithi047.stdout:99/600: open fd = 3 2024-04-17T06:04:58.975 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.975 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.975 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:04:58.975 INFO:tasks.workunit.client.0.smithi047.stdout:100/600: open fd = 3 2024-04-17T06:04:58.975 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.975 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.975 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:04:58.975 INFO:tasks.workunit.client.0.smithi047.stdout:101/600: open fd = 3 2024-04-17T06:04:58.975 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.976 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.976 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:04:58.976 INFO:tasks.workunit.client.0.smithi047.stdout:102/600: open fd = 3 2024-04-17T06:04:58.976 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.976 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.976 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:04:58.976 INFO:tasks.workunit.client.0.smithi047.stdout:103/600: open fd = 3 2024-04-17T06:04:58.976 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.976 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.976 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:04:58.976 INFO:tasks.workunit.client.0.smithi047.stdout:104/600: open fd = 3 2024-04-17T06:04:58.976 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.976 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.976 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:04:58.976 INFO:tasks.workunit.client.0.smithi047.stdout:105/600: open fd = 3 2024-04-17T06:04:58.977 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.977 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.977 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:04:58.977 INFO:tasks.workunit.client.0.smithi047.stdout:106/600: open fd = 3 2024-04-17T06:04:58.977 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.977 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.977 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:04:58.977 INFO:tasks.workunit.client.0.smithi047.stdout:107/600: open fd = 3 2024-04-17T06:04:58.977 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.977 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.977 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:04:58.977 INFO:tasks.workunit.client.0.smithi047.stdout:108/600: open fd = 3 2024-04-17T06:04:58.977 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.977 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.978 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:04:58.978 INFO:tasks.workunit.client.0.smithi047.stdout:109/600: open fd = 3 2024-04-17T06:04:58.978 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.978 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.978 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:04:58.978 INFO:tasks.workunit.client.0.smithi047.stdout:110/600: open fd = 3 2024-04-17T06:04:58.978 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.978 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.978 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:04:58.978 INFO:tasks.workunit.client.0.smithi047.stdout:111/600: open fd = 3 2024-04-17T06:04:58.978 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.978 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.978 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:04:58.978 INFO:tasks.workunit.client.0.smithi047.stdout:112/600: open fd = 3 2024-04-17T06:04:58.979 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.979 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.979 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:04:58.979 INFO:tasks.workunit.client.0.smithi047.stdout:113/600: open fd = 3 2024-04-17T06:04:58.979 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.979 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.979 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:04:58.979 INFO:tasks.workunit.client.0.smithi047.stdout:114/600: open fd = 3 2024-04-17T06:04:58.979 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.979 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.979 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:04:58.979 INFO:tasks.workunit.client.0.smithi047.stdout:115/600: open fd = 3 2024-04-17T06:04:58.979 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.979 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.979 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:04:58.980 INFO:tasks.workunit.client.0.smithi047.stdout:116/600: open fd = 3 2024-04-17T06:04:58.980 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.980 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.980 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:04:58.980 INFO:tasks.workunit.client.0.smithi047.stdout:117/600: open fd = 3 2024-04-17T06:04:58.980 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.980 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.980 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:04:58.980 INFO:tasks.workunit.client.0.smithi047.stdout:118/600: open fd = 3 2024-04-17T06:04:58.980 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.980 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.981 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:04:58.981 INFO:tasks.workunit.client.0.smithi047.stdout:119/600: open fd = 3 2024-04-17T06:04:58.981 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.981 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.981 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:04:58.981 INFO:tasks.workunit.client.0.smithi047.stdout:120/600: open fd = 3 2024-04-17T06:04:58.981 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.981 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.981 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:04:58.981 INFO:tasks.workunit.client.0.smithi047.stdout:121/600: open fd = 3 2024-04-17T06:04:58.981 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.981 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.981 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:04:58.981 INFO:tasks.workunit.client.0.smithi047.stdout:122/600: open fd = 3 2024-04-17T06:04:58.982 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.982 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.982 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:04:58.982 INFO:tasks.workunit.client.0.smithi047.stdout:123/600: open fd = 3 2024-04-17T06:04:58.982 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.982 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.982 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:04:58.982 INFO:tasks.workunit.client.0.smithi047.stdout:124/600: open fd = 3 2024-04-17T06:04:58.982 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.982 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.982 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:04:58.982 INFO:tasks.workunit.client.0.smithi047.stdout:125/600: open fd = 3 2024-04-17T06:04:58.982 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.982 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:58.983 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:04:58.983 INFO:tasks.workunit.client.0.smithi047.stdout:126/600: open fd = 3 2024-04-17T06:04:58.983 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:04:59.783 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:04:59 smithi103 ceph-mon[36243]: pgmap v115: 97 pgs: 97 active+clean; 2.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.9 KiB/s wr, 9 op/s 2024-04-17T06:04:59.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:04:59 smithi098 ceph-mon[36516]: pgmap v115: 97 pgs: 97 active+clean; 2.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.9 KiB/s wr, 9 op/s 2024-04-17T06:04:59.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:04:59 smithi047 ceph-mon[32223]: pgmap v115: 97 pgs: 97 active+clean; 2.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.9 KiB/s wr, 9 op/s 2024-04-17T06:05:00.783 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:05:00 smithi103 ceph-mon[36243]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-17T06:05:00.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:05:00 smithi098 ceph-mon[36516]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-17T06:05:00.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:05:00 smithi047 ceph-mon[32223]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-17T06:05:01.783 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:05:01 smithi103 ceph-mon[36243]: pgmap v116: 97 pgs: 97 active+clean; 2.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-17T06:05:01.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:05:01 smithi098 ceph-mon[36516]: pgmap v116: 97 pgs: 97 active+clean; 2.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-17T06:05:01.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:05:01 smithi047 ceph-mon[32223]: pgmap v116: 97 pgs: 97 active+clean; 2.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-17T06:05:03.783 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:05:03 smithi103 ceph-mon[36243]: pgmap v117: 97 pgs: 97 active+clean; 2.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 9.7 KiB/s wr, 11 op/s 2024-04-17T06:05:03.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:05:03 smithi098 ceph-mon[36516]: pgmap v117: 97 pgs: 97 active+clean; 2.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 9.7 KiB/s wr, 11 op/s 2024-04-17T06:05:03.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:05:03 smithi047 ceph-mon[32223]: pgmap v117: 97 pgs: 97 active+clean; 2.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 9.7 KiB/s wr, 11 op/s 2024-04-17T06:05:05.783 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:05:05 smithi103 ceph-mon[36243]: pgmap v118: 97 pgs: 97 active+clean; 2.2 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 36 KiB/s wr, 31 op/s 2024-04-17T06:05:05.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:05:05 smithi098 ceph-mon[36516]: pgmap v118: 97 pgs: 97 active+clean; 2.2 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 36 KiB/s wr, 31 op/s 2024-04-17T06:05:05.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:05:05 smithi047 ceph-mon[32223]: pgmap v118: 97 pgs: 97 active+clean; 2.2 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 36 KiB/s wr, 31 op/s 2024-04-17T06:05:07.783 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:05:07 smithi103 ceph-mon[36243]: pgmap v119: 97 pgs: 97 active+clean; 2.2 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 32 KiB/s wr, 26 op/s 2024-04-17T06:05:07.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:05:07 smithi098 ceph-mon[36516]: pgmap v119: 97 pgs: 97 active+clean; 2.2 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 32 KiB/s wr, 26 op/s 2024-04-17T06:05:07.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:05:07 smithi047 ceph-mon[32223]: pgmap v119: 97 pgs: 97 active+clean; 2.2 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 32 KiB/s wr, 26 op/s 2024-04-17T06:05:09.189 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.191+0000 7fa4b0900640 1 -- 172.21.15.47:0/730839417 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4ac1021c0 msgr2=0x7fa4ac1025a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:05:09.190 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.191+0000 7fa4b0900640 1 --2- 172.21.15.47:0/730839417 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4ac1021c0 0x7fa4ac1025a0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fa4a4009ca0 tx=0x7fa4a402c0f0 comp rx=0 tx=0).stop 2024-04-17T06:05:09.190 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.191+0000 7fa4b0900640 1 -- 172.21.15.47:0/730839417 shutdown_connections 2024-04-17T06:05:09.190 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.191+0000 7fa4b0900640 1 --2- 172.21.15.47:0/730839417 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa4ac107290 0x7fa4ac10ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:09.190 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.191+0000 7fa4b0900640 1 --2- 172.21.15.47:0/730839417 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa4ac102b70 0x7fa4ac106bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:09.190 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.191+0000 7fa4b0900640 1 --2- 172.21.15.47:0/730839417 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4ac1021c0 0x7fa4ac1025a0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:09.190 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.191+0000 7fa4b0900640 1 -- 172.21.15.47:0/730839417 >> 172.21.15.47:0/730839417 conn(0x7fa4ac0fda20 msgr2=0x7fa4ac0ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:05:09.190 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.191+0000 7fa4b0900640 1 -- 172.21.15.47:0/730839417 shutdown_connections 2024-04-17T06:05:09.190 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.191+0000 7fa4b0900640 1 -- 172.21.15.47:0/730839417 wait complete. 2024-04-17T06:05:09.190 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.192+0000 7fa4b0900640 1 Processor -- start 2024-04-17T06:05:09.191 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.192+0000 7fa4b0900640 1 -- start start 2024-04-17T06:05:09.192 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.192+0000 7fa4b0900640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4ac1021c0 0x7fa4ac19e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:05:09.193 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.192+0000 7fa4b0900640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa4ac107290 0x7fa4ac19e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:05:09.193 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.192+0000 7fa4b0900640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa4ac198660 0x7fa4ac198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:05:09.193 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.192+0000 7fa4b0900640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ac112340 con 0x7fa4ac1021c0 2024-04-17T06:05:09.193 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.192+0000 7fa4b0900640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ac1121c0 con 0x7fa4ac198660 2024-04-17T06:05:09.193 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.192+0000 7fa4b0900640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ac1124c0 con 0x7fa4ac107290 2024-04-17T06:05:09.193 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.192+0000 7fa4aa59c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4ac1021c0 0x7fa4ac19e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:05:09.193 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.192+0000 7fa4aa59c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4ac1021c0 0x7fa4ac19e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50768/0 (socket says 172.21.15.47:50768) 2024-04-17T06:05:09.193 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.192+0000 7fa4aa59c640 1 -- 172.21.15.47:0/1382774609 learned_addr learned my addr 172.21.15.47:0/1382774609 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:05:09.193 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.192+0000 7fa4a9d9b640 1 --2- 172.21.15.47:0/1382774609 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa4ac107290 0x7fa4ac19e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:05:09.193 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.193+0000 7fa4aad9d640 1 --2- 172.21.15.47:0/1382774609 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa4ac198660 0x7fa4ac198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:05:09.193 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.193+0000 7fa4aa59c640 1 -- 172.21.15.47:0/1382774609 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa4ac107290 msgr2=0x7fa4ac19e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:05:09.193 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.193+0000 7fa4aa59c640 1 --2- 172.21.15.47:0/1382774609 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa4ac107290 0x7fa4ac19e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:09.194 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.193+0000 7fa4aa59c640 1 -- 172.21.15.47:0/1382774609 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa4ac198660 msgr2=0x7fa4ac198b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:05:09.194 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.193+0000 7fa4aa59c640 1 --2- 172.21.15.47:0/1382774609 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa4ac198660 0x7fa4ac198b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:09.194 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.193+0000 7fa4aa59c640 1 -- 172.21.15.47:0/1382774609 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4ac199180 con 0x7fa4ac1021c0 2024-04-17T06:05:09.194 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.193+0000 7fa4aad9d640 1 --2- 172.21.15.47:0/1382774609 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa4ac198660 0x7fa4ac198b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:05:09.194 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.193+0000 7fa4a9d9b640 1 --2- 172.21.15.47:0/1382774609 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa4ac107290 0x7fa4ac19e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:05:09.194 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.193+0000 7fa4aa59c640 1 --2- 172.21.15.47:0/1382774609 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4ac1021c0 0x7fa4ac19e460 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fa4a4005e30 tx=0x7fa4a4032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:05:09.194 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.193+0000 7fa4a37fe640 1 -- 172.21.15.47:0/1382774609 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a4041020 con 0x7fa4ac1021c0 2024-04-17T06:05:09.194 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.193+0000 7fa4a37fe640 1 -- 172.21.15.47:0/1382774609 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4a4034d90 con 0x7fa4ac1021c0 2024-04-17T06:05:09.194 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.193+0000 7fa4b0900640 1 -- 172.21.15.47:0/1382774609 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4ac199410 con 0x7fa4ac1021c0 2024-04-17T06:05:09.194 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.193+0000 7fa4a37fe640 1 -- 172.21.15.47:0/1382774609 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a4039040 con 0x7fa4ac1021c0 2024-04-17T06:05:09.194 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.193+0000 7fa4b0900640 1 -- 172.21.15.47:0/1382774609 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4ac1a55b0 con 0x7fa4ac1021c0 2024-04-17T06:05:09.201 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.196+0000 7fa4b35c4740 1 Processor -- start 2024-04-17T06:05:09.201 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.196+0000 7fa4a37fe640 1 -- 172.21.15.47:0/1382774609 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa4a4044020 con 0x7fa4ac1021c0 2024-04-17T06:05:09.201 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.197+0000 7fa4b35c4740 1 -- start start 2024-04-17T06:05:09.201 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.197+0000 7fa4b35c4740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ff8334b0c0 0x55ff8334b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:05:09.201 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.197+0000 7fa4b35c4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ff8334b9e0 0x55ff83354270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:05:09.201 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.197+0000 7fa4b35c4740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff833547b0 0x55ff83356ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:05:09.202 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.197+0000 7fa4b35c4740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55ff832b07d0 con 0x55ff833547b0 2024-04-17T06:05:09.202 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.197+0000 7fa4b35c4740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55ff8324df70 con 0x55ff8334b0c0 2024-04-17T06:05:09.202 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.197+0000 7fa4b35c4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ff832ac790 con 0x55ff8334b9e0 2024-04-17T06:05:09.202 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.197+0000 7fa4a9d9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff833547b0 0x55ff83356ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:05:09.202 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.197+0000 7fa4a9d9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff833547b0 0x55ff83356ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50778/0 (socket says 172.21.15.47:50778) 2024-04-17T06:05:09.202 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.197+0000 7fa4a9d9b640 1 -- 172.21.15.47:0/3097935649 learned_addr learned my addr 172.21.15.47:0/3097935649 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:05:09.202 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.197+0000 7fa4aa59c640 1 --2- 172.21.15.47:0/3097935649 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ff8334b9e0 0x55ff83354270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:05:09.202 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.197+0000 7fa4aad9d640 1 --2- 172.21.15.47:0/3097935649 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ff8334b0c0 0x55ff8334b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:05:09.202 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.197+0000 7fa4a37fe640 1 --2- 172.21.15.47:0/1382774609 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fa4980a3dc0 0x7fa4980a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:05:09.202 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.197+0000 7fa4a9d9b640 1 -- 172.21.15.47:0/3097935649 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ff8334b9e0 msgr2=0x55ff83354270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:05:09.202 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.197+0000 7fa4a9d9b640 1 --2- 172.21.15.47:0/3097935649 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ff8334b9e0 0x55ff83354270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:09.202 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.197+0000 7fa4a9d9b640 1 -- 172.21.15.47:0/3097935649 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ff8334b0c0 msgr2=0x55ff8334b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:05:09.203 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.197+0000 7fa4a9d9b640 1 --2- 172.21.15.47:0/3097935649 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ff8334b0c0 0x55ff8334b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:09.203 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.197+0000 7fa4a9d9b640 1 -- 172.21.15.47:0/3097935649 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff82fc3210 con 0x55ff833547b0 2024-04-17T06:05:09.203 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.197+0000 7fa4aad9d640 1 --2- 172.21.15.47:0/1382774609 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fa4980a3dc0 0x7fa4980a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:05:09.203 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.198+0000 7fa4a9d9b640 1 --2- 172.21.15.47:0/3097935649 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff833547b0 0x55ff83356ba0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fa494009c70 tx=0x7fa4940117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=416e3720d012da9b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:05:09.203 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.198+0000 7fa4a37fe640 1 -- 172.21.15.47:0/1382774609 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7fa4a40f2610 con 0x7fa4ac1021c0 2024-04-17T06:05:09.203 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.198+0000 7fa4a0ff9640 1 -- 172.21.15.47:0/3097935649 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa494012450 con 0x55ff833547b0 2024-04-17T06:05:09.203 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.198+0000 7fa4a0ff9640 1 -- 172.21.15.47:0/3097935649 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa494012ad0 con 0x55ff833547b0 2024-04-17T06:05:09.203 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.198+0000 7fa4a0ff9640 1 -- 172.21.15.47:0/3097935649 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa49400fc40 con 0x55ff833547b0 2024-04-17T06:05:09.203 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.199+0000 7fa4b35c4740 1 -- 172.21.15.47:0/3097935649 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff833547b0 msgr2=0x55ff83356ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:05:09.203 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.199+0000 7fa4aad9d640 1 --2- 172.21.15.47:0/1382774609 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fa4980a3dc0 0x7fa4980a6280 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa49c004600 tx=0x7fa49c0046c0 comp rx=0 tx=0).ready entity=mgr.24923 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:05:09.203 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.199+0000 7fa4b35c4740 1 --2- 172.21.15.47:0/3097935649 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff833547b0 0x55ff83356ba0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fa494009c70 tx=0x7fa4940117b0 comp rx=0 tx=0).stop 2024-04-17T06:05:09.203 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.199+0000 7fa4b35c4740 1 -- 172.21.15.47:0/3097935649 shutdown_connections 2024-04-17T06:05:09.203 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.199+0000 7fa4b35c4740 1 --2- 172.21.15.47:0/3097935649 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff833547b0 0x55ff83356ba0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:09.203 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.199+0000 7fa4b35c4740 1 --2- 172.21.15.47:0/3097935649 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ff8334b9e0 0x55ff83354270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:09.203 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.199+0000 7fa4b35c4740 1 --2- 172.21.15.47:0/3097935649 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ff8334b0c0 0x55ff8334b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:09.204 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.199+0000 7fa4b35c4740 1 -- 172.21.15.47:0/3097935649 >> 172.21.15.47:0/3097935649 conn(0x55ff833459e0 msgr2=0x55ff83345dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:05:09.204 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.199+0000 7fa4b35c4740 1 -- 172.21.15.47:0/3097935649 shutdown_connections 2024-04-17T06:05:09.204 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.199+0000 7fa4b35c4740 1 -- 172.21.15.47:0/3097935649 wait complete. 2024-04-17T06:05:09.204 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.199+0000 7fa4b35c4740 1 -- start start 2024-04-17T06:05:09.204 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.199+0000 7fa4b35c4740 1 Processor -- start 2024-04-17T06:05:09.204 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.200+0000 7fa4b35c4740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ff8334b0c0 0x55ff833517d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:05:09.204 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.200+0000 7fa4b35c4740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff8334b9e0 0x55ff83351d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:05:09.204 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.200+0000 7fa4b35c4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ff833547b0 0x55ff83352250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:05:09.204 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.200+0000 7fa4b35c4740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55ff833573e0 con 0x55ff8334b9e0 2024-04-17T06:05:09.204 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.200+0000 7fa4b35c4740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55ff83238e90 con 0x55ff8334b0c0 2024-04-17T06:05:09.204 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.200+0000 7fa4b35c4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ff83357560 con 0x55ff833547b0 2024-04-17T06:05:09.204 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.200+0000 7fa4b35c4740 10 client.0 _ll_register_callbacks cb 0x55ff8324ed40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:05:09.204 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.200+0000 7fa4a9d9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff8334b9e0 0x55ff83351d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:05:09.204 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.200+0000 7fa4a9d9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff8334b9e0 0x55ff83351d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50788/0 (socket says 172.21.15.47:50788) 2024-04-17T06:05:09.204 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.200+0000 7fa4a9d9b640 1 -- 172.21.15.47:0/1864111452 learned_addr learned my addr 172.21.15.47:0/1864111452 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:05:09.205 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.200+0000 7fa4aa59c640 1 --2- 172.21.15.47:0/1864111452 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ff8334b0c0 0x55ff833517d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:05:09.205 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.200+0000 7fa4aad9d640 1 --2- 172.21.15.47:0/1864111452 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ff833547b0 0x55ff83352250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:05:09.205 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.200+0000 7fa4a9d9b640 1 -- 172.21.15.47:0/1864111452 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ff833547b0 msgr2=0x55ff83352250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:05:09.205 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.200+0000 7fa4a9d9b640 1 --2- 172.21.15.47:0/1864111452 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ff833547b0 0x55ff83352250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:09.205 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.200+0000 7fa4a9d9b640 1 -- 172.21.15.47:0/1864111452 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ff8334b0c0 msgr2=0x55ff833517d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:05:09.205 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.200+0000 7fa4a9d9b640 1 --2- 172.21.15.47:0/1864111452 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ff8334b0c0 0x55ff833517d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:09.205 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.200+0000 7fa4a9d9b640 1 -- 172.21.15.47:0/1864111452 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff8308f620 con 0x55ff8334b9e0 2024-04-17T06:05:09.205 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.200+0000 7fa4a9d9b640 1 --2- 172.21.15.47:0/1864111452 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff8334b9e0 0x55ff83351d10 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fa49401c040 tx=0x7fa49400f5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=983110ff99e41002 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:05:09.205 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.200+0000 7fa491ffb640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-17T06:05:09.205 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.200+0000 7fa491ffb640 1 -- 172.21.15.47:0/1864111452 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa494029020 con 0x55ff8334b9e0 2024-04-17T06:05:09.205 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.200+0000 7fa491ffb640 1 -- 172.21.15.47:0/1864111452 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7fa494012450 con 0x55ff8334b9e0 2024-04-17T06:05:09.205 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.200+0000 7fa4b35c4740 1 -- 172.21.15.47:0/1864111452 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa494012ad0 con 0x55ff8334b9e0 2024-04-17T06:05:09.205 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.201+0000 7fa491ffb640 1 -- 172.21.15.47:0/1864111452 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa494034050 con 0x55ff8334b9e0 2024-04-17T06:05:09.206 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.201+0000 7fa4b35c4740 10 client.15723 fetch_fsmap learned FSMap version 28 2024-04-17T06:05:09.206 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.201+0000 7fa4b35c4740 1 -- 172.21.15.47:0/1864111452 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ff83363620 con 0x55ff8334b9e0 2024-04-17T06:05:09.206 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.201+0000 7fa491ffb640 1 -- 172.21.15.47:0/1864111452 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa494012ad0 con 0x55ff8334b9e0 2024-04-17T06:05:09.206 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.201+0000 7fa491ffb640 1 -- 172.21.15.47:0/1864111452 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 28) ==== 8339+0+0 (secure 0 0 0) 0x7fa4940053a0 con 0x55ff8334b9e0 2024-04-17T06:05:09.206 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.201+0000 7fa4b35c4740 10 client.15723 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T06:05:09.206 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.201+0000 7fa4b35c4740 10 client.15723 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-17T06:05:09.206 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.201+0000 7fa4b35c4740 20 client.15723 populate_metadata read hostname 'smithi047' 2024-04-17T06:05:09.206 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.201+0000 7fa4b35c4740 1 --2- 172.21.15.47:0/1864111452 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55ff83364a30 0x55ff83366e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:05:09.206 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.201+0000 7fa4b35c4740 4 client.15723 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T06:05:09.206 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.201+0000 7fa4b35c4740 1 -- 172.21.15.47:0/1864111452 --> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ff832be1f0 con 0x55ff83364a30 2024-04-17T06:05:09.206 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.201+0000 7fa4aa59c640 1 --2- 172.21.15.47:0/1864111452 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55ff83364a30 0x55ff83366e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:05:09.206 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.202+0000 7fa4aa59c640 1 --2- 172.21.15.47:0/1864111452 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55ff83364a30 0x55ff83366e20 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8332cb84ead5f3eb server_cookie=e12702088b54a90b in_seq=0 out_seq=0 2024-04-17T06:05:09.206 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.202+0000 7fa491ffb640 10 client.15723 ms_handle_connect on v2:172.21.15.98:6838/2841287917 2024-04-17T06:05:09.207 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.202+0000 7fa491ffb640 1 -- 172.21.15.47:0/1864111452 <== mds.0 v2:172.21.15.98:6838/2841287917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ff832be1f0 con 0x55ff83364a30 2024-04-17T06:05:09.207 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.202+0000 7fa491ffb640 10 client.15723 handle_command_reply: tid=0 2024-04-17T06:05:09.207 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.203+0000 7fa4b35c4740 1 client.15723 shutdown 2024-04-17T06:05:09.207 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.203+0000 7fa4b35c4740 1 -- 172.21.15.47:0/1864111452 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff8334b9e0 msgr2=0x55ff83351d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:05:09.207 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.203+0000 7fa4b35c4740 1 --2- 172.21.15.47:0/1864111452 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff8334b9e0 0x55ff83351d10 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fa49401c040 tx=0x7fa49400f5c0 comp rx=0 tx=0).stop 2024-04-17T06:05:09.207 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.203+0000 7fa4b35c4740 1 -- 172.21.15.47:0/1864111452 shutdown_connections 2024-04-17T06:05:09.207 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.203+0000 7fa4b35c4740 1 --2- 172.21.15.47:0/1864111452 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55ff83364a30 0x55ff83366e20 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:09.207 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.203+0000 7fa4b35c4740 1 --2- 172.21.15.47:0/1864111452 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ff833547b0 0x55ff83352250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:09.207 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.203+0000 7fa4b35c4740 1 --2- 172.21.15.47:0/1864111452 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff8334b9e0 0x55ff83351d10 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:09.207 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.203+0000 7fa4b35c4740 1 --2- 172.21.15.47:0/1864111452 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ff8334b0c0 0x55ff833517d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:09.207 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.203+0000 7fa491ffb640 0 client.15723 ms_handle_reset on v2:172.21.15.98:6838/2841287917 2024-04-17T06:05:09.207 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.203+0000 7fa4b35c4740 1 -- 172.21.15.47:0/1864111452 >> 172.21.15.47:0/1864111452 conn(0x55ff833459e0 msgr2=0x55ff83347730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:05:09.207 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.203+0000 7fa4b35c4740 1 -- 172.21.15.47:0/1864111452 shutdown_connections 2024-04-17T06:05:09.207 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.203+0000 7fa4b35c4740 1 -- 172.21.15.47:0/1864111452 wait complete. 2024-04-17T06:05:09.208 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.203+0000 7fa4b35c4740 20 client.15723 trim_cache size 0 max 16384 2024-04-17T06:05:09.221 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.218+0000 7fa4b35c4740 1 Processor -- start 2024-04-17T06:05:09.221 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.218+0000 7fa4b35c4740 1 -- start start 2024-04-17T06:05:09.221 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.218+0000 7fa4b35c4740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff8334b0c0 0x55ff83353e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:05:09.221 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.218+0000 7fa4b35c4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ff8334b9e0 0x55ff833602a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:05:09.221 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.218+0000 7fa4b35c4740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ff833547b0 0x55ff833474f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:05:09.221 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.218+0000 7fa4b35c4740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ac1b0440 con 0x55ff8334b0c0 2024-04-17T06:05:09.221 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.218+0000 7fa4b35c4740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ac1add00 con 0x55ff833547b0 2024-04-17T06:05:09.222 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.218+0000 7fa4b35c4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ac1abb50 con 0x55ff8334b9e0 2024-04-17T06:05:09.222 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.218+0000 7fa4aa59c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff8334b0c0 0x55ff83353e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:05:09.222 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.218+0000 7fa4aa59c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff8334b0c0 0x55ff83353e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50790/0 (socket says 172.21.15.47:50790) 2024-04-17T06:05:09.222 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.218+0000 7fa4aa59c640 1 -- 172.21.15.47:0/110995194 learned_addr learned my addr 172.21.15.47:0/110995194 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:05:09.222 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.218+0000 7fa4aa59c640 1 -- 172.21.15.47:0/110995194 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ff8334b9e0 msgr2=0x55ff833602a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:05:09.222 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.218+0000 7fa4aa59c640 1 --2- 172.21.15.47:0/110995194 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ff8334b9e0 0x55ff833602a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:09.222 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.218+0000 7fa4aa59c640 1 -- 172.21.15.47:0/110995194 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ff833547b0 msgr2=0x55ff833474f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:05:09.222 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.218+0000 7fa4aa59c640 1 --2- 172.21.15.47:0/110995194 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ff833547b0 0x55ff833474f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:09.222 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.218+0000 7fa4aa59c640 1 -- 172.21.15.47:0/110995194 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff833412a0 con 0x55ff8334b0c0 2024-04-17T06:05:09.222 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.219+0000 7fa4aa59c640 1 --2- 172.21.15.47:0/110995194 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff8334b0c0 0x55ff83353e90 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fa4a40f0390 tx=0x7fa4a40f4b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f469ccb8ca6cd3d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:05:09.222 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.219+0000 7fa493fff640 1 -- 172.21.15.47:0/110995194 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a40a90b0 con 0x55ff8334b0c0 2024-04-17T06:05:09.222 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.219+0000 7fa493fff640 1 -- 172.21.15.47:0/110995194 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4a40be6f0 con 0x55ff8334b0c0 2024-04-17T06:05:09.222 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.219+0000 7fa493fff640 1 -- 172.21.15.47:0/110995194 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a40bc820 con 0x55ff8334b0c0 2024-04-17T06:05:09.223 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.219+0000 7fa4b35c4740 1 -- 172.21.15.47:0/110995194 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff8334b0c0 msgr2=0x55ff83353e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:05:09.223 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.219+0000 7fa4b35c4740 1 --2- 172.21.15.47:0/110995194 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff8334b0c0 0x55ff83353e90 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fa4a40f0390 tx=0x7fa4a40f4b20 comp rx=0 tx=0).stop 2024-04-17T06:05:09.223 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.219+0000 7fa4b35c4740 1 -- 172.21.15.47:0/110995194 shutdown_connections 2024-04-17T06:05:09.223 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.219+0000 7fa4b35c4740 1 --2- 172.21.15.47:0/110995194 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ff833547b0 0x55ff833474f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:09.223 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.219+0000 7fa4b35c4740 1 --2- 172.21.15.47:0/110995194 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ff8334b9e0 0x55ff833602a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:09.223 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.219+0000 7fa4b35c4740 1 --2- 172.21.15.47:0/110995194 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff8334b0c0 0x55ff83353e90 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:09.223 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.219+0000 7fa4b35c4740 1 -- 172.21.15.47:0/110995194 >> 172.21.15.47:0/110995194 conn(0x55ff833459e0 msgr2=0x55ff83346de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:05:09.223 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.219+0000 7fa4b35c4740 1 -- 172.21.15.47:0/110995194 shutdown_connections 2024-04-17T06:05:09.223 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.219+0000 7fa4b35c4740 1 -- 172.21.15.47:0/110995194 wait complete. 2024-04-17T06:05:09.223 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.220+0000 7fa4b35c4740 1 -- start start 2024-04-17T06:05:09.223 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.220+0000 7fa4b35c4740 1 Processor -- start 2024-04-17T06:05:09.223 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.220+0000 7fa4b35c4740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ff8334b0c0 0x55ff8334e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:05:09.223 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.220+0000 7fa4b35c4740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff8334b9e0 0x55ff8334ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:05:09.223 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.220+0000 7fa4b35c4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ff833547b0 0x55ff8334f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:05:09.224 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.220+0000 7fa4b35c4740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ac1ac1c0 con 0x55ff8334b9e0 2024-04-17T06:05:09.224 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.220+0000 7fa4b35c4740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ac1ad630 con 0x55ff8334b0c0 2024-04-17T06:05:09.224 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.220+0000 7fa4b35c4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ac1ac890 con 0x55ff833547b0 2024-04-17T06:05:09.224 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.220+0000 7fa4b35c4740 10 client.0 _ll_register_callbacks cb 0x7fa49c0092a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:05:09.224 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.220+0000 7fa4a9d9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff8334b9e0 0x55ff8334ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:05:09.224 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.220+0000 7fa4a9d9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff8334b9e0 0x55ff8334ecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50798/0 (socket says 172.21.15.47:50798) 2024-04-17T06:05:09.224 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.220+0000 7fa4a9d9b640 1 -- 172.21.15.47:0/3369111713 learned_addr learned my addr 172.21.15.47:0/3369111713 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:05:09.224 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.220+0000 7fa4aa59c640 1 --2- 172.21.15.47:0/3369111713 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ff8334b0c0 0x55ff8334e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:05:09.224 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.220+0000 7fa4a9d9b640 1 -- 172.21.15.47:0/3369111713 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ff833547b0 msgr2=0x55ff8334f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:05:09.224 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.220+0000 7fa4aad9d640 1 --2- 172.21.15.47:0/3369111713 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ff833547b0 0x55ff8334f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:05:09.224 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.221+0000 7fa4a9d9b640 1 --2- 172.21.15.47:0/3369111713 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ff833547b0 0x55ff8334f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:09.224 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.221+0000 7fa4a9d9b640 1 -- 172.21.15.47:0/3369111713 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ff8334b0c0 msgr2=0x55ff8334e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:05:09.224 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.221+0000 7fa4a9d9b640 1 --2- 172.21.15.47:0/3369111713 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ff8334b0c0 0x55ff8334e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:09.224 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.221+0000 7fa4a9d9b640 1 -- 172.21.15.47:0/3369111713 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff833c9ff0 con 0x55ff8334b9e0 2024-04-17T06:05:09.225 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.221+0000 7fa4a9d9b640 1 --2- 172.21.15.47:0/3369111713 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff8334b9e0 0x55ff8334ecf0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fa494001eb0 tx=0x7fa494009b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b48470fdf431edb server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:05:09.225 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.221+0000 7fa491ffb640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-17T06:05:09.225 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.221+0000 7fa491ffb640 1 -- 172.21.15.47:0/3369111713 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa494029070 con 0x55ff8334b9e0 2024-04-17T06:05:09.225 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.221+0000 7fa491ffb640 1 -- 172.21.15.47:0/3369111713 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7fa494012650 con 0x55ff8334b9e0 2024-04-17T06:05:09.225 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.221+0000 7fa4b35c4740 1 -- 172.21.15.47:0/3369111713 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4a40be6f0 con 0x55ff8334b9e0 2024-04-17T06:05:09.225 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.221+0000 7fa491ffb640 1 -- 172.21.15.47:0/3369111713 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa494046050 con 0x55ff8334b9e0 2024-04-17T06:05:09.225 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.221+0000 7fa4b35c4740 10 client.15729 fetch_fsmap learned FSMap version 28 2024-04-17T06:05:09.225 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.221+0000 7fa4b35c4740 1 -- 172.21.15.47:0/3369111713 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ff833cb300 con 0x55ff8334b9e0 2024-04-17T06:05:09.225 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.221+0000 7fa491ffb640 1 -- 172.21.15.47:0/3369111713 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4a40be6f0 con 0x55ff8334b9e0 2024-04-17T06:05:09.225 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.221+0000 7fa491ffb640 1 -- 172.21.15.47:0/3369111713 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 28) ==== 8339+0+0 (secure 0 0 0) 0x7fa49400f510 con 0x55ff8334b9e0 2024-04-17T06:05:09.225 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.221+0000 7fa4b35c4740 10 client.15729 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T06:05:09.225 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.221+0000 7fa4b35c4740 10 client.15729 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-17T06:05:09.225 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.222+0000 7fa4b35c4740 20 client.15729 populate_metadata read hostname 'smithi047' 2024-04-17T06:05:09.225 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.222+0000 7fa4b35c4740 1 --2- 172.21.15.47:0/3369111713 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55ff833cba70 0x55ff833cde90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:05:09.225 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.222+0000 7fa4b35c4740 4 client.15729 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-17T06:05:09.226 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.222+0000 7fa4b35c4740 1 -- 172.21.15.47:0/3369111713 --> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ff832b8060 con 0x55ff833cba70 2024-04-17T06:05:09.226 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.222+0000 7fa4aa59c640 1 --2- 172.21.15.47:0/3369111713 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55ff833cba70 0x55ff833cde90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:05:09.226 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.222+0000 7fa491ffb640 10 client.15729 ms_handle_connect on v2:172.21.15.98:6838/2841287917 2024-04-17T06:05:09.226 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.222+0000 7fa4aa59c640 1 --2- 172.21.15.47:0/3369111713 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55ff833cba70 0x55ff833cde90 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36ef9ab87d6fde7b server_cookie=31947d3751004eb3 in_seq=0 out_seq=0 2024-04-17T06:05:09.228 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.230+0000 7fa491ffb640 1 -- 172.21.15.47:0/3369111713 <== mds.0 v2:172.21.15.98:6838/2841287917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ff832b8060 con 0x55ff833cba70 2024-04-17T06:05:09.229 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.230+0000 7fa491ffb640 10 client.15729 handle_command_reply: tid=0 2024-04-17T06:05:09.229 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.230+0000 7fa4b35c4740 1 client.15729 shutdown 2024-04-17T06:05:09.229 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.230+0000 7fa4b35c4740 1 -- 172.21.15.47:0/3369111713 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff8334b9e0 msgr2=0x55ff8334ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:05:09.229 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.230+0000 7fa4b35c4740 1 --2- 172.21.15.47:0/3369111713 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff8334b9e0 0x55ff8334ecf0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fa494001eb0 tx=0x7fa494009b40 comp rx=0 tx=0).stop 2024-04-17T06:05:09.229 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.230+0000 7fa4b35c4740 1 -- 172.21.15.47:0/3369111713 shutdown_connections 2024-04-17T06:05:09.229 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.230+0000 7fa4b35c4740 1 --2- 172.21.15.47:0/3369111713 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55ff833cba70 0x55ff833cde90 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:09.229 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.230+0000 7fa4b35c4740 1 --2- 172.21.15.47:0/3369111713 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ff833547b0 0x55ff8334f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:09.229 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.230+0000 7fa4b35c4740 1 --2- 172.21.15.47:0/3369111713 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff8334b9e0 0x55ff8334ecf0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:09.229 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.230+0000 7fa4b35c4740 1 --2- 172.21.15.47:0/3369111713 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ff8334b0c0 0x55ff8334e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:09.229 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.230+0000 7fa491ffb640 0 client.15729 ms_handle_reset on v2:172.21.15.98:6838/2841287917 2024-04-17T06:05:09.229 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.230+0000 7fa4b35c4740 1 -- 172.21.15.47:0/3369111713 >> 172.21.15.47:0/3369111713 conn(0x55ff833459e0 msgr2=0x55ff83351f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:05:09.229 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.231+0000 7fa4b35c4740 1 -- 172.21.15.47:0/3369111713 shutdown_connections 2024-04-17T06:05:09.229 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.231+0000 7fa4b35c4740 1 -- 172.21.15.47:0/3369111713 wait complete. 2024-04-17T06:05:09.229 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.231+0000 7fa4b35c4740 20 client.15729 trim_cache size 0 max 16384 2024-04-17T06:05:09.230 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-04-17T06:05:09.230 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-04-17T06:05:09.230 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-04-17T06:05:09.230 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-04-17T06:05:09.231 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.231+0000 7fa4b0900640 1 -- 172.21.15.47:0/1382774609 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fa4980a3dc0 msgr2=0x7fa4980a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:05:09.231 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.231+0000 7fa4b0900640 1 --2- 172.21.15.47:0/1382774609 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fa4980a3dc0 0x7fa4980a6280 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa49c004600 tx=0x7fa49c0046c0 comp rx=0 tx=0).stop 2024-04-17T06:05:09.231 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.231+0000 7fa4b0900640 1 -- 172.21.15.47:0/1382774609 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4ac1021c0 msgr2=0x7fa4ac19e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:05:09.231 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.231+0000 7fa4b0900640 1 --2- 172.21.15.47:0/1382774609 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4ac1021c0 0x7fa4ac19e460 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fa4a4005e30 tx=0x7fa4a4032f20 comp rx=0 tx=0).stop 2024-04-17T06:05:09.231 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.231+0000 7fa4b0900640 1 -- 172.21.15.47:0/1382774609 shutdown_connections 2024-04-17T06:05:09.231 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.231+0000 7fa4b0900640 1 --2- 172.21.15.47:0/1382774609 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa4ac198660 0x7fa4ac198b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:09.231 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.231+0000 7fa4b0900640 1 --2- 172.21.15.47:0/1382774609 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa4ac107290 0x7fa4ac19e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:09.231 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.231+0000 7fa4b0900640 1 --2- 172.21.15.47:0/1382774609 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fa4980a3dc0 0x7fa4980a6280 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:09.231 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.231+0000 7fa4b0900640 1 --2- 172.21.15.47:0/1382774609 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4ac1021c0 0x7fa4ac19e460 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:09.231 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.231+0000 7fa4b0900640 1 -- 172.21.15.47:0/1382774609 >> 172.21.15.47:0/1382774609 conn(0x7fa4ac0fda20 msgr2=0x7fa4ac0ffd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:05:09.231 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.231+0000 7fa4b0900640 1 -- 172.21.15.47:0/1382774609 shutdown_connections 2024-04-17T06:05:09.232 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:09.231+0000 7fa4b0900640 1 -- 172.21.15.47:0/1382774609 wait complete. 2024-04-17T06:05:09.783 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:05:09 smithi103 ceph-mon[36243]: pgmap v120: 97 pgs: 97 active+clean; 2.2 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 32 KiB/s wr, 26 op/s 2024-04-17T06:05:09.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:05:09 smithi098 ceph-mon[36516]: pgmap v120: 97 pgs: 97 active+clean; 2.2 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 32 KiB/s wr, 26 op/s 2024-04-17T06:05:09.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:05:09 smithi047 ceph-mon[32223]: pgmap v120: 97 pgs: 97 active+clean; 2.2 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 32 KiB/s wr, 26 op/s 2024-04-17T06:05:11.783 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:05:11 smithi103 ceph-mon[36243]: pgmap v121: 97 pgs: 97 active+clean; 2.2 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 38 KiB/s wr, 33 op/s 2024-04-17T06:05:11.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:05:11 smithi098 ceph-mon[36516]: pgmap v121: 97 pgs: 97 active+clean; 2.2 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 38 KiB/s wr, 33 op/s 2024-04-17T06:05:11.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:05:11 smithi047 ceph-mon[32223]: pgmap v121: 97 pgs: 97 active+clean; 2.2 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 38 KiB/s wr, 33 op/s 2024-04-17T06:05:13.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:05:13 smithi103 ceph-mon[36243]: pgmap v122: 97 pgs: 97 active+clean; 2.2 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 32 KiB/s wr, 26 op/s 2024-04-17T06:05:13.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:05:13 smithi098 ceph-mon[36516]: pgmap v122: 97 pgs: 97 active+clean; 2.2 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 32 KiB/s wr, 26 op/s 2024-04-17T06:05:13.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:05:13 smithi047 ceph-mon[32223]: pgmap v122: 97 pgs: 97 active+clean; 2.2 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 32 KiB/s wr, 26 op/s 2024-04-17T06:05:15.783 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:05:15 smithi103 ceph-mon[36243]: pgmap v123: 97 pgs: 97 active+clean; 2.2 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 37 KiB/s wr, 31 op/s 2024-04-17T06:05:15.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:05:15 smithi098 ceph-mon[36516]: pgmap v123: 97 pgs: 97 active+clean; 2.2 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 37 KiB/s wr, 31 op/s 2024-04-17T06:05:15.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:05:15 smithi047 ceph-mon[32223]: pgmap v123: 97 pgs: 97 active+clean; 2.2 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 37 KiB/s wr, 31 op/s 2024-04-17T06:05:17.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:05:17 smithi103 ceph-mon[36243]: pgmap v124: 97 pgs: 97 active+clean; 2.2 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 10 KiB/s wr, 12 op/s 2024-04-17T06:05:17.884 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:05:17 smithi098 ceph-mon[36516]: pgmap v124: 97 pgs: 97 active+clean; 2.2 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 10 KiB/s wr, 12 op/s 2024-04-17T06:05:17.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:05:17 smithi047 ceph-mon[32223]: pgmap v124: 97 pgs: 97 active+clean; 2.2 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 10 KiB/s wr, 12 op/s 2024-04-17T06:05:19.783 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:05:19 smithi103 ceph-mon[36243]: pgmap v125: 97 pgs: 97 active+clean; 2.2 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 10 KiB/s wr, 12 op/s 2024-04-17T06:05:19.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:05:19 smithi098 ceph-mon[36516]: pgmap v125: 97 pgs: 97 active+clean; 2.2 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 10 KiB/s wr, 12 op/s 2024-04-17T06:05:19.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:05:19 smithi047 ceph-mon[32223]: pgmap v125: 97 pgs: 97 active+clean; 2.2 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 10 KiB/s wr, 12 op/s 2024-04-17T06:05:21.783 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:05:21 smithi103 ceph-mon[36243]: pgmap v126: 97 pgs: 97 active+clean; 2.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-04-17T06:05:21.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:05:21 smithi098 ceph-mon[36516]: pgmap v126: 97 pgs: 97 active+clean; 2.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-04-17T06:05:21.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:05:21 smithi047 ceph-mon[32223]: pgmap v126: 97 pgs: 97 active+clean; 2.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-04-17T06:05:22.328 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.329+0000 7f78b1b0e640 1 -- 172.21.15.47:0/244176490 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78ac1010d0 msgr2=0x7f78ac1014b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:05:22.329 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.329+0000 7f78b1b0e640 1 --2- 172.21.15.47:0/244176490 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78ac1010d0 0x7f78ac1014b0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f789c009900 tx=0x7f789c02bc30 comp rx=0 tx=0).stop 2024-04-17T06:05:22.329 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.329+0000 7f78b1b0e640 1 -- 172.21.15.47:0/244176490 shutdown_connections 2024-04-17T06:05:22.329 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.329+0000 7f78b1b0e640 1 --2- 172.21.15.47:0/244176490 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f78ac1061a0 0x7f78ac10dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:22.329 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.329+0000 7f78b1b0e640 1 --2- 172.21.15.47:0/244176490 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f78ac101a80 0x7f78ac105ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:22.329 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.329+0000 7f78b1b0e640 1 --2- 172.21.15.47:0/244176490 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78ac1010d0 0x7f78ac1014b0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:22.329 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.329+0000 7f78b1b0e640 1 -- 172.21.15.47:0/244176490 >> 172.21.15.47:0/244176490 conn(0x7f78ac077fe0 msgr2=0x7f78ac0783f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:05:22.329 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.329+0000 7f78b1b0e640 1 -- 172.21.15.47:0/244176490 shutdown_connections 2024-04-17T06:05:22.329 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.329+0000 7f78b1b0e640 1 -- 172.21.15.47:0/244176490 wait complete. 2024-04-17T06:05:22.329 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.330+0000 7f78b1b0e640 1 Processor -- start 2024-04-17T06:05:22.331 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.330+0000 7f78b1b0e640 1 -- start start 2024-04-17T06:05:22.331 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.330+0000 7f78b1b0e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78ac1010d0 0x7f78ac199630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:05:22.332 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.330+0000 7f78b1b0e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f78ac1061a0 0x7f78ac199b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:05:22.332 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.330+0000 7f78b1b0e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f78ac19a100 0x7f78ac1a46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:05:22.332 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.330+0000 7f78b1b0e640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f78ac1112b0 con 0x7f78ac1010d0 2024-04-17T06:05:22.332 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.330+0000 7f78b1b0e640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f78ac111130 con 0x7f78ac19a100 2024-04-17T06:05:22.332 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.330+0000 7f78b1b0e640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f78ac111430 con 0x7f78ac1061a0 2024-04-17T06:05:22.332 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.331+0000 7f78ab7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78ac1010d0 0x7f78ac199630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:05:22.332 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.331+0000 7f78ab7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78ac1010d0 0x7f78ac199630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43688/0 (socket says 172.21.15.47:43688) 2024-04-17T06:05:22.332 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.331+0000 7f78ab7fe640 1 -- 172.21.15.47:0/1333320411 learned_addr learned my addr 172.21.15.47:0/1333320411 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:05:22.332 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.331+0000 7f78aaffd640 1 --2- 172.21.15.47:0/1333320411 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f78ac1061a0 0x7f78ac199b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:05:22.332 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.331+0000 7f78abfff640 1 --2- 172.21.15.47:0/1333320411 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f78ac19a100 0x7f78ac1a46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:05:22.332 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.331+0000 7f78ab7fe640 1 -- 172.21.15.47:0/1333320411 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f78ac1061a0 msgr2=0x7f78ac199b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:05:22.332 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.331+0000 7f78ab7fe640 1 --2- 172.21.15.47:0/1333320411 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f78ac1061a0 0x7f78ac199b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:22.332 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.331+0000 7f78ab7fe640 1 -- 172.21.15.47:0/1333320411 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f78ac19a100 msgr2=0x7f78ac1a46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:05:22.332 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.331+0000 7f78ab7fe640 1 --2- 172.21.15.47:0/1333320411 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f78ac19a100 0x7f78ac1a46b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:22.332 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.331+0000 7f78ab7fe640 1 -- 172.21.15.47:0/1333320411 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78ac1a4e80 con 0x7f78ac1010d0 2024-04-17T06:05:22.333 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.331+0000 7f78ab7fe640 1 --2- 172.21.15.47:0/1333320411 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78ac1010d0 0x7f78ac199630 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f789c006010 tx=0x7f789c02cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:05:22.333 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.331+0000 7f78a8ff9640 1 -- 172.21.15.47:0/1333320411 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f789c041070 con 0x7f78ac1010d0 2024-04-17T06:05:22.333 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.331+0000 7f78a8ff9640 1 -- 172.21.15.47:0/1333320411 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f789c004450 con 0x7f78ac1010d0 2024-04-17T06:05:22.333 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.331+0000 7f78b1b0e640 1 -- 172.21.15.47:0/1333320411 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78ac1a5110 con 0x7f78ac1010d0 2024-04-17T06:05:22.333 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.331+0000 7f78a8ff9640 1 -- 172.21.15.47:0/1333320411 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f789c003e20 con 0x7f78ac1010d0 2024-04-17T06:05:22.333 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.331+0000 7f78b1b0e640 1 -- 172.21.15.47:0/1333320411 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78ac1a5600 con 0x7f78ac1010d0 2024-04-17T06:05:22.341 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.334+0000 7f78a8ff9640 1 -- 172.21.15.47:0/1333320411 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f789c044070 con 0x7f78ac1010d0 2024-04-17T06:05:22.341 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.334+0000 7f78b47d1740 1 Processor -- start 2024-04-17T06:05:22.341 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.334+0000 7f78b47d1740 1 -- start start 2024-04-17T06:05:22.341 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.334+0000 7f78a8ff9640 1 --2- 172.21.15.47:0/1333320411 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f78940a3dc0 0x7f78940a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:05:22.341 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.334+0000 7f78b47d1740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d52cdbc0c0 0x55d52cdbc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:05:22.341 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.334+0000 7f78b47d1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d52cdbc9e0 0x55d52cdc5270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:05:22.341 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.334+0000 7f78b47d1740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d52cdc57b0 0x55d52cdc7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:05:22.341 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.334+0000 7f78b47d1740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55d52cd217d0 con 0x55d52cdbc0c0 2024-04-17T06:05:22.342 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.334+0000 7f78b47d1740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55d52ccbef70 con 0x55d52cdc57b0 2024-04-17T06:05:22.342 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.334+0000 7f78b47d1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d52cd1d790 con 0x55d52cdbc9e0 2024-04-17T06:05:22.342 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.335+0000 7f78abfff640 1 --2- 172.21.15.47:0/1333320411 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f78940a3dc0 0x7f78940a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:05:22.342 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.335+0000 7f78ab7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d52cdbc0c0 0x55d52cdbc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:05:22.342 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.335+0000 7f78ab7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d52cdbc0c0 0x55d52cdbc4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43692/0 (socket says 172.21.15.47:43692) 2024-04-17T06:05:22.342 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.335+0000 7f78a8ff9640 1 -- 172.21.15.47:0/1333320411 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f789c0f2250 con 0x7f78ac1010d0 2024-04-17T06:05:22.342 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.335+0000 7f78ab7fe640 1 -- 172.21.15.47:0/2732318200 learned_addr learned my addr 172.21.15.47:0/2732318200 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:05:22.342 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.335+0000 7f78aaffd640 1 --2- 172.21.15.47:0/2732318200 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d52cdbc9e0 0x55d52cdc5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:05:22.342 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.335+0000 7f78abfff640 1 --2- 172.21.15.47:0/2732318200 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d52cdc57b0 0x55d52cdc7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:05:22.342 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.335+0000 7f78ab7fe640 1 -- 172.21.15.47:0/2732318200 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d52cdbc9e0 msgr2=0x55d52cdc5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:05:22.342 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.335+0000 7f78ab7fe640 1 --2- 172.21.15.47:0/2732318200 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d52cdbc9e0 0x55d52cdc5270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:22.342 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.335+0000 7f78ab7fe640 1 -- 172.21.15.47:0/2732318200 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d52cdc57b0 msgr2=0x55d52cdc7ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:05:22.343 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.335+0000 7f78ab7fe640 1 --2- 172.21.15.47:0/2732318200 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d52cdc57b0 0x55d52cdc7ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:22.343 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.335+0000 7f78ab7fe640 1 -- 172.21.15.47:0/2732318200 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d52ca34210 con 0x55d52cdbc0c0 2024-04-17T06:05:22.343 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.335+0000 7f78abfff640 1 --2- 172.21.15.47:0/2732318200 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d52cdc57b0 0x55d52cdc7ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:05:22.343 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.335+0000 7f78abfff640 1 --2- 172.21.15.47:0/1333320411 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f78940a3dc0 0x7f78940a6280 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f78a4005fd0 tx=0x7f78a4009ae0 comp rx=0 tx=0).ready entity=mgr.24923 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:05:22.343 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.335+0000 7f78ab7fe640 1 --2- 172.21.15.47:0/2732318200 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d52cdbc0c0 0x55d52cdbc4a0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f789c0c6a70 tx=0x7f789c02bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad7499da4e7207c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:05:22.343 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.335+0000 7f7891ffb640 1 -- 172.21.15.47:0/2732318200 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f789c04b040 con 0x55d52cdbc0c0 2024-04-17T06:05:22.343 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.335+0000 7f7891ffb640 1 -- 172.21.15.47:0/2732318200 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f789c0bea30 con 0x55d52cdbc0c0 2024-04-17T06:05:22.343 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.335+0000 7f7891ffb640 1 -- 172.21.15.47:0/2732318200 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f789c0be330 con 0x55d52cdbc0c0 2024-04-17T06:05:22.343 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.336+0000 7f78b47d1740 1 -- 172.21.15.47:0/2732318200 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d52cdbc0c0 msgr2=0x55d52cdbc4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:05:22.343 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.336+0000 7f78b47d1740 1 --2- 172.21.15.47:0/2732318200 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d52cdbc0c0 0x55d52cdbc4a0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f789c0c6a70 tx=0x7f789c02bc30 comp rx=0 tx=0).stop 2024-04-17T06:05:22.343 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.336+0000 7f78b47d1740 1 -- 172.21.15.47:0/2732318200 shutdown_connections 2024-04-17T06:05:22.343 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.336+0000 7f78b47d1740 1 --2- 172.21.15.47:0/2732318200 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d52cdc57b0 0x55d52cdc7ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:22.343 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.336+0000 7f78b47d1740 1 --2- 172.21.15.47:0/2732318200 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d52cdbc9e0 0x55d52cdc5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:22.343 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.336+0000 7f78b47d1740 1 --2- 172.21.15.47:0/2732318200 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d52cdbc0c0 0x55d52cdbc4a0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:22.344 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.336+0000 7f78b47d1740 1 -- 172.21.15.47:0/2732318200 >> 172.21.15.47:0/2732318200 conn(0x55d52cdb69e0 msgr2=0x55d52cdb6dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:05:22.344 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.336+0000 7f78b47d1740 1 -- 172.21.15.47:0/2732318200 shutdown_connections 2024-04-17T06:05:22.344 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.336+0000 7f78b47d1740 1 -- 172.21.15.47:0/2732318200 wait complete. 2024-04-17T06:05:22.344 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.336+0000 7f78b47d1740 1 -- start start 2024-04-17T06:05:22.344 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.337+0000 7f78b47d1740 1 Processor -- start 2024-04-17T06:05:22.344 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.337+0000 7f78b47d1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d52cdbc0c0 0x55d52cdc27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:05:22.344 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.337+0000 7f78b47d1740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d52cdbc9e0 0x55d52cdc2d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:05:22.344 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.337+0000 7f78b47d1740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d52cdc57b0 0x55d52cdc3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:05:22.344 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.337+0000 7f78b47d1740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55d52cdc81a0 con 0x55d52cdbc9e0 2024-04-17T06:05:22.344 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.337+0000 7f78b47d1740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55d52cca9e90 con 0x55d52cdc57b0 2024-04-17T06:05:22.344 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.337+0000 7f78b47d1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d52cdc8320 con 0x55d52cdbc0c0 2024-04-17T06:05:22.344 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.337+0000 7f78b47d1740 10 client.0 _ll_register_callbacks cb 0x55d52ccbfd40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:05:22.344 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.337+0000 7f78aaffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d52cdbc9e0 0x55d52cdc2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:05:22.345 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.337+0000 7f78aaffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d52cdbc9e0 0x55d52cdc2d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43696/0 (socket says 172.21.15.47:43696) 2024-04-17T06:05:22.345 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.337+0000 7f78aaffd640 1 -- 172.21.15.47:0/2827637468 learned_addr learned my addr 172.21.15.47:0/2827637468 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:05:22.345 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.337+0000 7f78ab7fe640 1 --2- 172.21.15.47:0/2827637468 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d52cdbc0c0 0x55d52cdc27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:05:22.345 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.338+0000 7f78abfff640 1 --2- 172.21.15.47:0/2827637468 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d52cdc57b0 0x55d52cdc3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:05:22.345 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.338+0000 7f78aaffd640 1 -- 172.21.15.47:0/2827637468 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d52cdbc0c0 msgr2=0x55d52cdc27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:05:22.345 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.338+0000 7f78aaffd640 1 --2- 172.21.15.47:0/2827637468 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d52cdbc0c0 0x55d52cdc27e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:22.345 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.338+0000 7f78aaffd640 1 -- 172.21.15.47:0/2827637468 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d52cdc57b0 msgr2=0x55d52cdc3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:05:22.345 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.338+0000 7f78aaffd640 1 --2- 172.21.15.47:0/2827637468 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d52cdc57b0 0x55d52cdc3260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:22.345 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.338+0000 7f78aaffd640 1 -- 172.21.15.47:0/2827637468 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d52cb00620 con 0x55d52cdbc9e0 2024-04-17T06:05:22.345 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.338+0000 7f78aaffd640 1 --2- 172.21.15.47:0/2827637468 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d52cdbc9e0 0x55d52cdc2d20 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f78a00115a0 tx=0x7f78a0011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1aaf9fc895886821 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:05:22.345 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.338+0000 7f788affd640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-17T06:05:22.345 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.338+0000 7f788affd640 1 -- 172.21.15.47:0/2827637468 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78a0013670 con 0x55d52cdbc9e0 2024-04-17T06:05:22.345 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.338+0000 7f788affd640 1 -- 172.21.15.47:0/2827637468 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f78a0013810 con 0x55d52cdbc9e0 2024-04-17T06:05:22.345 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.338+0000 7f78b47d1740 1 -- 172.21.15.47:0/2827637468 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f789c0bea30 con 0x55d52cdbc9e0 2024-04-17T06:05:22.346 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.338+0000 7f788affd640 1 -- 172.21.15.47:0/2827637468 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f78a0013a10 con 0x55d52cdbc9e0 2024-04-17T06:05:22.346 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.338+0000 7f78b47d1740 10 client.15741 fetch_fsmap learned FSMap version 28 2024-04-17T06:05:22.346 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.338+0000 7f78b47d1740 1 -- 172.21.15.47:0/2827637468 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d52cdd4340 con 0x55d52cdbc9e0 2024-04-17T06:05:22.346 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.338+0000 7f788affd640 1 -- 172.21.15.47:0/2827637468 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f789c0bea30 con 0x55d52cdbc9e0 2024-04-17T06:05:22.346 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.339+0000 7f788affd640 1 -- 172.21.15.47:0/2827637468 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 28) ==== 8339+0+0 (secure 0 0 0) 0x7f78a0014440 con 0x55d52cdbc9e0 2024-04-17T06:05:22.346 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.339+0000 7f78b47d1740 10 client.15741 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T06:05:22.346 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.339+0000 7f78b47d1740 10 client.15741 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-17T06:05:22.346 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.339+0000 7f78b47d1740 20 client.15741 populate_metadata read hostname 'smithi047' 2024-04-17T06:05:22.346 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.339+0000 7f78b47d1740 1 --2- 172.21.15.47:0/2827637468 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55d52cdd5800 0x55d52cdd7c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:05:22.346 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.339+0000 7f78b47d1740 4 client.15741 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T06:05:22.346 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.339+0000 7f78b47d1740 1 -- 172.21.15.47:0/2827637468 --> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d52cd2f1f0 con 0x55d52cdd5800 2024-04-17T06:05:22.346 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.339+0000 7f78ab7fe640 1 --2- 172.21.15.47:0/2827637468 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55d52cdd5800 0x55d52cdd7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:05:22.346 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.339+0000 7f788affd640 10 client.15741 ms_handle_connect on v2:172.21.15.98:6838/2841287917 2024-04-17T06:05:22.346 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.339+0000 7f78ab7fe640 1 --2- 172.21.15.47:0/2827637468 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55d52cdd5800 0x55d52cdd7c10 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5055c5ee9d159b71 server_cookie=63da1e0a7a6dbd34 in_seq=0 out_seq=0 2024-04-17T06:05:22.347 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.340+0000 7f788affd640 1 -- 172.21.15.47:0/2827637468 <== mds.0 v2:172.21.15.98:6838/2841287917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d52cd2f1f0 con 0x55d52cdd5800 2024-04-17T06:05:22.347 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.340+0000 7f788affd640 10 client.15741 handle_command_reply: tid=0 2024-04-17T06:05:22.347 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.340+0000 7f78b47d1740 1 client.15741 shutdown 2024-04-17T06:05:22.347 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.340+0000 7f78b47d1740 1 -- 172.21.15.47:0/2827637468 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d52cdbc9e0 msgr2=0x55d52cdc2d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:05:22.347 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.340+0000 7f78b47d1740 1 --2- 172.21.15.47:0/2827637468 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d52cdbc9e0 0x55d52cdc2d20 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f78a00115a0 tx=0x7f78a0011a60 comp rx=0 tx=0).stop 2024-04-17T06:05:22.347 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.340+0000 7f78b47d1740 1 -- 172.21.15.47:0/2827637468 shutdown_connections 2024-04-17T06:05:22.347 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.340+0000 7f78b47d1740 1 --2- 172.21.15.47:0/2827637468 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55d52cdd5800 0x55d52cdd7c10 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:22.347 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.340+0000 7f78b47d1740 1 --2- 172.21.15.47:0/2827637468 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d52cdc57b0 0x55d52cdc3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:22.347 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.340+0000 7f78b47d1740 1 --2- 172.21.15.47:0/2827637468 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d52cdbc9e0 0x55d52cdc2d20 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:22.347 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.340+0000 7f78b47d1740 1 --2- 172.21.15.47:0/2827637468 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d52cdbc0c0 0x55d52cdc27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:22.347 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.340+0000 7f788affd640 0 client.15741 ms_handle_reset on v2:172.21.15.98:6838/2841287917 2024-04-17T06:05:22.347 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.340+0000 7f78b47d1740 1 -- 172.21.15.47:0/2827637468 >> 172.21.15.47:0/2827637468 conn(0x55d52cdb69e0 msgr2=0x55d52cdb8730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:05:22.347 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.341+0000 7f78b47d1740 1 -- 172.21.15.47:0/2827637468 shutdown_connections 2024-04-17T06:05:22.347 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.341+0000 7f78b47d1740 1 -- 172.21.15.47:0/2827637468 wait complete. 2024-04-17T06:05:22.347 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.341+0000 7f78b47d1740 20 client.15741 trim_cache size 0 max 16384 2024-04-17T06:05:22.356 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.356+0000 7f78b47d1740 1 Processor -- start 2024-04-17T06:05:22.356 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.356+0000 7f78b47d1740 1 -- start start 2024-04-17T06:05:22.356 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.356+0000 7f78b47d1740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d52cdbc0c0 0x55d52cdc4f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:05:22.356 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.356+0000 7f78b47d1740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d52cdbc9e0 0x55d52cdd0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:05:22.357 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.356+0000 7f78b47d1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d52cdc57b0 0x55d52cdb8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:05:22.357 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.356+0000 7f78b47d1740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f78ac1b04a0 con 0x55d52cdbc0c0 2024-04-17T06:05:22.357 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.356+0000 7f78b47d1740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f78ac1add60 con 0x55d52cdbc9e0 2024-04-17T06:05:22.357 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.356+0000 7f78b47d1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f78ac1abbd0 con 0x55d52cdc57b0 2024-04-17T06:05:22.357 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.356+0000 7f78ab7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d52cdbc0c0 0x55d52cdc4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:05:22.357 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.356+0000 7f78ab7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d52cdbc0c0 0x55d52cdc4f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43700/0 (socket says 172.21.15.47:43700) 2024-04-17T06:05:22.357 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.356+0000 7f78ab7fe640 1 -- 172.21.15.47:0/1176128078 learned_addr learned my addr 172.21.15.47:0/1176128078 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:05:22.357 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.356+0000 7f78abfff640 1 --2- 172.21.15.47:0/1176128078 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d52cdc57b0 0x55d52cdb8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:05:22.357 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.356+0000 7f78ab7fe640 1 -- 172.21.15.47:0/1176128078 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d52cdc57b0 msgr2=0x55d52cdb8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:05:22.357 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.356+0000 7f78ab7fe640 1 --2- 172.21.15.47:0/1176128078 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d52cdc57b0 0x55d52cdb8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:22.357 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.356+0000 7f78aaffd640 1 --2- 172.21.15.47:0/1176128078 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d52cdbc9e0 0x55d52cdd0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:05:22.357 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.356+0000 7f78ab7fe640 1 -- 172.21.15.47:0/1176128078 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d52cdbc9e0 msgr2=0x55d52cdd0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:05:22.357 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.356+0000 7f78ab7fe640 1 --2- 172.21.15.47:0/1176128078 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d52cdbc9e0 0x55d52cdd0ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.356+0000 7f78ab7fe640 1 -- 172.21.15.47:0/1176128078 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d52cdbb680 con 0x55d52cdbc0c0 2024-04-17T06:05:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.356+0000 7f78ab7fe640 1 --2- 172.21.15.47:0/1176128078 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d52cdbc0c0 0x55d52cdc4f10 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f789c0effd0 tx=0x7f789c0ba650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10cb8184fe1ad253 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:05:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.357+0000 7f78917fa640 1 -- 172.21.15.47:0/1176128078 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f789c0d7070 con 0x55d52cdbc0c0 2024-04-17T06:05:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.357+0000 7f78917fa640 1 -- 172.21.15.47:0/1176128078 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f789c0bea30 con 0x55d52cdbc0c0 2024-04-17T06:05:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.357+0000 7f78917fa640 1 -- 172.21.15.47:0/1176128078 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f789c04b040 con 0x55d52cdbc0c0 2024-04-17T06:05:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.357+0000 7f78b47d1740 1 -- 172.21.15.47:0/1176128078 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d52cdbc0c0 msgr2=0x55d52cdc4f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:05:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.357+0000 7f78b47d1740 1 --2- 172.21.15.47:0/1176128078 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d52cdbc0c0 0x55d52cdc4f10 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f789c0effd0 tx=0x7f789c0ba650 comp rx=0 tx=0).stop 2024-04-17T06:05:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.357+0000 7f78b47d1740 1 -- 172.21.15.47:0/1176128078 shutdown_connections 2024-04-17T06:05:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.357+0000 7f78b47d1740 1 --2- 172.21.15.47:0/1176128078 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d52cdc57b0 0x55d52cdb8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.357+0000 7f78b47d1740 1 --2- 172.21.15.47:0/1176128078 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d52cdbc9e0 0x55d52cdd0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.357+0000 7f78b47d1740 1 --2- 172.21.15.47:0/1176128078 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d52cdbc0c0 0x55d52cdc4f10 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.357+0000 7f78b47d1740 1 -- 172.21.15.47:0/1176128078 >> 172.21.15.47:0/1176128078 conn(0x55d52cdb69e0 msgr2=0x55d52cdb7e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:05:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.357+0000 7f78b47d1740 1 -- 172.21.15.47:0/1176128078 shutdown_connections 2024-04-17T06:05:22.359 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.357+0000 7f78b47d1740 1 -- 172.21.15.47:0/1176128078 wait complete. 2024-04-17T06:05:22.359 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.357+0000 7f78b47d1740 1 -- start start 2024-04-17T06:05:22.359 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.358+0000 7f78b47d1740 1 Processor -- start 2024-04-17T06:05:22.361 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.358+0000 7f78b47d1740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d52cdbc0c0 0x55d52cdc8e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:05:22.362 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.358+0000 7f78b47d1740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d52cdbc9e0 0x55d52cdc9370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:05:22.362 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.358+0000 7f78b47d1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d52cdc57b0 0x55d52cdc98b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:05:22.362 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.358+0000 7f78b47d1740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f78ac1ac220 con 0x55d52cdbc0c0 2024-04-17T06:05:22.362 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.358+0000 7f78b47d1740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f78ac1ad690 con 0x55d52cdbc9e0 2024-04-17T06:05:22.362 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.358+0000 7f78b47d1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f78ac1ac8f0 con 0x55d52cdc57b0 2024-04-17T06:05:22.362 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.358+0000 7f78b47d1740 10 client.0 _ll_register_callbacks cb 0x7f78a0009e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:05:22.362 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.358+0000 7f78ab7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d52cdbc0c0 0x55d52cdc8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:05:22.362 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.358+0000 7f78ab7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d52cdbc0c0 0x55d52cdc8e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43704/0 (socket says 172.21.15.47:43704) 2024-04-17T06:05:22.362 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.358+0000 7f78ab7fe640 1 -- 172.21.15.47:0/3835799303 learned_addr learned my addr 172.21.15.47:0/3835799303 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:05:22.363 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.358+0000 7f78abfff640 1 --2- 172.21.15.47:0/3835799303 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d52cdc57b0 0x55d52cdc98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:05:22.363 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.358+0000 7f78ab7fe640 1 -- 172.21.15.47:0/3835799303 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d52cdc57b0 msgr2=0x55d52cdc98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:05:22.363 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.358+0000 7f78ab7fe640 1 --2- 172.21.15.47:0/3835799303 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d52cdc57b0 0x55d52cdc98b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:22.363 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.358+0000 7f78aaffd640 1 --2- 172.21.15.47:0/3835799303 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d52cdbc9e0 0x55d52cdc9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:05:22.363 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.358+0000 7f78ab7fe640 1 -- 172.21.15.47:0/3835799303 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d52cdbc9e0 msgr2=0x55d52cdc9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:05:22.363 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.358+0000 7f78ab7fe640 1 --2- 172.21.15.47:0/3835799303 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d52cdbc9e0 0x55d52cdc9370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:22.363 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.358+0000 7f78ab7fe640 1 -- 172.21.15.47:0/3835799303 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d52cdbbaa0 con 0x55d52cdbc0c0 2024-04-17T06:05:22.363 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.358+0000 7f78abfff640 1 --2- 172.21.15.47:0/3835799303 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d52cdc57b0 0x55d52cdc98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:05:22.363 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.359+0000 7f78ab7fe640 1 --2- 172.21.15.47:0/3835799303 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d52cdbc0c0 0x55d52cdc8e30 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f789c02c820 tx=0x7f789c02bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=121749dfa5890d50 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:05:22.363 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.359+0000 7f788affd640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-17T06:05:22.363 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.359+0000 7f788affd640 1 -- 172.21.15.47:0/3835799303 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f789c04b070 con 0x55d52cdbc0c0 2024-04-17T06:05:22.363 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.359+0000 7f788affd640 1 -- 172.21.15.47:0/3835799303 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f789c0efb90 con 0x55d52cdbc0c0 2024-04-17T06:05:22.363 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.359+0000 7f78b47d1740 1 -- 172.21.15.47:0/3835799303 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f789c0bea30 con 0x55d52cdbc0c0 2024-04-17T06:05:22.363 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.359+0000 7f788affd640 1 -- 172.21.15.47:0/3835799303 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f789c0d7040 con 0x55d52cdbc0c0 2024-04-17T06:05:22.363 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.359+0000 7f78b47d1740 10 client.15747 fetch_fsmap learned FSMap version 28 2024-04-17T06:05:22.364 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.359+0000 7f78b47d1740 1 -- 172.21.15.47:0/3835799303 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d52cdc7ba0 con 0x55d52cdbc0c0 2024-04-17T06:05:22.364 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.359+0000 7f788affd640 1 -- 172.21.15.47:0/3835799303 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f789c0bea30 con 0x55d52cdbc0c0 2024-04-17T06:05:22.364 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.359+0000 7f788affd640 1 -- 172.21.15.47:0/3835799303 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 28) ==== 8339+0+0 (secure 0 0 0) 0x7f789c0c6b10 con 0x55d52cdbc0c0 2024-04-17T06:05:22.364 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.359+0000 7f78b47d1740 10 client.15747 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T06:05:22.364 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.359+0000 7f78b47d1740 10 client.15747 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-17T06:05:22.364 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.359+0000 7f78b47d1740 20 client.15747 populate_metadata read hostname 'smithi047' 2024-04-17T06:05:22.364 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.359+0000 7f78b47d1740 1 --2- 172.21.15.47:0/3835799303 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55d52ce3c020 0x55d52ce3e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:05:22.364 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.359+0000 7f78b47d1740 4 client.15747 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-17T06:05:22.364 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.359+0000 7f78b47d1740 1 -- 172.21.15.47:0/3835799303 --> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d52cdbe3e0 con 0x55d52ce3c020 2024-04-17T06:05:22.364 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.360+0000 7f78aaffd640 1 --2- 172.21.15.47:0/3835799303 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55d52ce3c020 0x55d52ce3e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:05:22.364 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.360+0000 7f788affd640 10 client.15747 ms_handle_connect on v2:172.21.15.98:6838/2841287917 2024-04-17T06:05:22.364 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.360+0000 7f78aaffd640 1 --2- 172.21.15.47:0/3835799303 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55d52ce3c020 0x55d52ce3e440 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b6e56c0a44b9da5 server_cookie=d6d214361914e673 in_seq=0 out_seq=0 2024-04-17T06:05:22.367 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.368+0000 7f788affd640 1 -- 172.21.15.47:0/3835799303 <== mds.0 v2:172.21.15.98:6838/2841287917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d52cdbe3e0 con 0x55d52ce3c020 2024-04-17T06:05:22.368 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.368+0000 7f788affd640 10 client.15747 handle_command_reply: tid=0 2024-04-17T06:05:22.368 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.368+0000 7f78b47d1740 1 client.15747 shutdown 2024-04-17T06:05:22.368 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.368+0000 7f78b47d1740 1 -- 172.21.15.47:0/3835799303 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d52cdbc0c0 msgr2=0x55d52cdc8e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:05:22.368 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.368+0000 7f78b47d1740 1 --2- 172.21.15.47:0/3835799303 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d52cdbc0c0 0x55d52cdc8e30 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f789c02c820 tx=0x7f789c02bc30 comp rx=0 tx=0).stop 2024-04-17T06:05:22.368 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.368+0000 7f78b47d1740 1 -- 172.21.15.47:0/3835799303 shutdown_connections 2024-04-17T06:05:22.368 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.368+0000 7f78b47d1740 1 --2- 172.21.15.47:0/3835799303 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55d52ce3c020 0x55d52ce3e440 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:22.368 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.368+0000 7f78b47d1740 1 --2- 172.21.15.47:0/3835799303 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d52cdc57b0 0x55d52cdc98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:22.368 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.369+0000 7f78b47d1740 1 --2- 172.21.15.47:0/3835799303 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d52cdbc9e0 0x55d52cdc9370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:22.368 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.369+0000 7f78b47d1740 1 --2- 172.21.15.47:0/3835799303 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d52cdbc0c0 0x55d52cdc8e30 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:22.368 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.369+0000 7f788affd640 0 client.15747 ms_handle_reset on v2:172.21.15.98:6838/2841287917 2024-04-17T06:05:22.368 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.369+0000 7f78b47d1740 1 -- 172.21.15.47:0/3835799303 >> 172.21.15.47:0/3835799303 conn(0x55d52cdb69e0 msgr2=0x55d52cdb86c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:05:22.368 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.369+0000 7f78b47d1740 1 -- 172.21.15.47:0/3835799303 shutdown_connections 2024-04-17T06:05:22.368 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.369+0000 7f78b47d1740 1 -- 172.21.15.47:0/3835799303 wait complete. 2024-04-17T06:05:22.368 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.369+0000 7f78b47d1740 20 client.15747 trim_cache size 0 max 16384 2024-04-17T06:05:22.369 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-04-17T06:05:22.369 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-04-17T06:05:22.369 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-04-17T06:05:22.369 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-04-17T06:05:22.370 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.369+0000 7f78b1b0e640 1 -- 172.21.15.47:0/1333320411 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f78940a3dc0 msgr2=0x7f78940a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:05:22.370 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.369+0000 7f78b1b0e640 1 --2- 172.21.15.47:0/1333320411 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f78940a3dc0 0x7f78940a6280 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f78a4005fd0 tx=0x7f78a4009ae0 comp rx=0 tx=0).stop 2024-04-17T06:05:22.370 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.369+0000 7f78b1b0e640 1 -- 172.21.15.47:0/1333320411 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78ac1010d0 msgr2=0x7f78ac199630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:05:22.370 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.369+0000 7f78b1b0e640 1 --2- 172.21.15.47:0/1333320411 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78ac1010d0 0x7f78ac199630 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f789c006010 tx=0x7f789c02cc70 comp rx=0 tx=0).stop 2024-04-17T06:05:22.370 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.369+0000 7f78b1b0e640 1 -- 172.21.15.47:0/1333320411 shutdown_connections 2024-04-17T06:05:22.370 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.369+0000 7f78b1b0e640 1 --2- 172.21.15.47:0/1333320411 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f78ac19a100 0x7f78ac1a46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:22.370 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.369+0000 7f78b1b0e640 1 --2- 172.21.15.47:0/1333320411 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f78ac1061a0 0x7f78ac199b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:22.370 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.369+0000 7f78b1b0e640 1 --2- 172.21.15.47:0/1333320411 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f78940a3dc0 0x7f78940a6280 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:22.370 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.369+0000 7f78b1b0e640 1 --2- 172.21.15.47:0/1333320411 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78ac1010d0 0x7f78ac199630 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:22.370 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.369+0000 7f78b1b0e640 1 -- 172.21.15.47:0/1333320411 >> 172.21.15.47:0/1333320411 conn(0x7f78ac077fe0 msgr2=0x7f78ac0fee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:05:22.370 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.369+0000 7f78b1b0e640 1 -- 172.21.15.47:0/1333320411 shutdown_connections 2024-04-17T06:05:22.370 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:22.369+0000 7f78b1b0e640 1 -- 172.21.15.47:0/1333320411 wait complete. 2024-04-17T06:05:23.783 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:05:23 smithi103 ceph-mon[36243]: pgmap v127: 97 pgs: 97 active+clean; 2.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 6.9 KiB/s wr, 9 op/s 2024-04-17T06:05:23.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:05:23 smithi098 ceph-mon[36516]: pgmap v127: 97 pgs: 97 active+clean; 2.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 6.9 KiB/s wr, 9 op/s 2024-04-17T06:05:23.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:05:23 smithi047 ceph-mon[32223]: pgmap v127: 97 pgs: 97 active+clean; 2.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 6.9 KiB/s wr, 9 op/s 2024-04-17T06:05:25.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:05:25 smithi103 ceph-mon[36243]: pgmap v128: 97 pgs: 97 active+clean; 2.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-04-17T06:05:25.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:05:25 smithi098 ceph-mon[36516]: pgmap v128: 97 pgs: 97 active+clean; 2.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-04-17T06:05:25.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:05:25 smithi047 ceph-mon[32223]: pgmap v128: 97 pgs: 97 active+clean; 2.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-04-17T06:05:27.783 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:05:27 smithi103 ceph-mon[36243]: pgmap v129: 97 pgs: 97 active+clean; 2.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 10 KiB/s wr, 11 op/s 2024-04-17T06:05:27.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:05:27 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T06:05:27.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:05:27 smithi098 ceph-mon[36516]: pgmap v129: 97 pgs: 97 active+clean; 2.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 10 KiB/s wr, 11 op/s 2024-04-17T06:05:27.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:05:27 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T06:05:27.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:05:27 smithi047 ceph-mon[32223]: pgmap v129: 97 pgs: 97 active+clean; 2.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 10 KiB/s wr, 11 op/s 2024-04-17T06:05:27.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:05:27 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T06:05:28.783 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:05:28 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T06:05:28.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:05:28 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T06:05:28.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:05:28 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:05:28.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:05:28 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T06:05:28.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:05:28 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T06:05:28.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:05:28 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:05:28.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:05:28 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T06:05:28.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:05:28 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T06:05:28.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:05:28 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:05:29.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:05:29 smithi098 ceph-mon[36516]: pgmap v130: 97 pgs: 97 active+clean; 2.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 10 KiB/s wr, 11 op/s 2024-04-17T06:05:29.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:05:29 smithi047 ceph-mon[32223]: pgmap v130: 97 pgs: 97 active+clean; 2.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 10 KiB/s wr, 11 op/s 2024-04-17T06:05:30.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:05:29 smithi103 ceph-mon[36243]: pgmap v130: 97 pgs: 97 active+clean; 2.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 10 KiB/s wr, 11 op/s 2024-04-17T06:05:31.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:05:31 smithi098 ceph-mon[36516]: pgmap v131: 97 pgs: 97 active+clean; 2.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-04-17T06:05:31.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:05:31 smithi047 ceph-mon[32223]: pgmap v131: 97 pgs: 97 active+clean; 2.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-04-17T06:05:32.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:05:31 smithi103 ceph-mon[36243]: pgmap v131: 97 pgs: 97 active+clean; 2.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-04-17T06:05:33.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:05:33 smithi098 ceph-mon[36516]: pgmap v132: 97 pgs: 97 active+clean; 2.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-04-17T06:05:33.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:05:33 smithi047 ceph-mon[32223]: pgmap v132: 97 pgs: 97 active+clean; 2.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-04-17T06:05:34.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:05:33 smithi103 ceph-mon[36243]: pgmap v132: 97 pgs: 97 active+clean; 2.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-04-17T06:05:35.471 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.471+0000 7f8440500640 1 -- 172.21.15.47:0/42045814 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f84381061c0 msgr2=0x7f843810dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:05:35.471 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.471+0000 7f8440500640 1 --2- 172.21.15.47:0/42045814 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f84381061c0 0x7f843810dd90 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f8430009ca0 tx=0x7f843002c0f0 comp rx=0 tx=0).stop 2024-04-17T06:05:35.472 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.471+0000 7f8440500640 1 -- 172.21.15.47:0/42045814 shutdown_connections 2024-04-17T06:05:35.472 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.471+0000 7f8440500640 1 --2- 172.21.15.47:0/42045814 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f84381061c0 0x7f843810dd90 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:35.472 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.471+0000 7f8440500640 1 --2- 172.21.15.47:0/42045814 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8438101aa0 0x7f8438105ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:35.472 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.471+0000 7f8440500640 1 --2- 172.21.15.47:0/42045814 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f84381010f0 0x7f84381014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:35.472 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.471+0000 7f8440500640 1 -- 172.21.15.47:0/42045814 >> 172.21.15.47:0/42045814 conn(0x7f84380771e0 msgr2=0x7f84380775f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:05:35.472 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.471+0000 7f8440500640 1 -- 172.21.15.47:0/42045814 shutdown_connections 2024-04-17T06:05:35.472 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.471+0000 7f8440500640 1 -- 172.21.15.47:0/42045814 wait complete. 2024-04-17T06:05:35.472 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.471+0000 7f8440500640 1 Processor -- start 2024-04-17T06:05:35.472 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.471+0000 7f8440500640 1 -- start start 2024-04-17T06:05:35.472 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.472+0000 7f8440500640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f84381010f0 0x7f8438198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:05:35.472 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.472+0000 7f8440500640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f84381061c0 0x7f8438198cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:05:35.472 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.472+0000 7f8440500640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f843819cfe0 0x7f843819d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:05:35.472 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.472+0000 7f8440500640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f8438111280 con 0x7f84381010f0 2024-04-17T06:05:35.473 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.472+0000 7f8440500640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8438111100 con 0x7f843819cfe0 2024-04-17T06:05:35.473 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.472+0000 7f8440500640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8438111400 con 0x7f84381061c0 2024-04-17T06:05:35.473 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.472+0000 7f843e29c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f84381010f0 0x7f8438198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:05:35.473 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.472+0000 7f843e29c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f84381010f0 0x7f8438198770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44968/0 (socket says 172.21.15.47:44968) 2024-04-17T06:05:35.473 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.472+0000 7f843e29c640 1 -- 172.21.15.47:0/2793426350 learned_addr learned my addr 172.21.15.47:0/2793426350 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:05:35.473 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.472+0000 7f843da9b640 1 --2- 172.21.15.47:0/2793426350 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f84381061c0 0x7f8438198cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:05:35.473 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.472+0000 7f843ea9d640 1 --2- 172.21.15.47:0/2793426350 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f843819cfe0 0x7f843819d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:05:35.473 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.472+0000 7f843e29c640 1 -- 172.21.15.47:0/2793426350 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f84381061c0 msgr2=0x7f8438198cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:05:35.473 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.472+0000 7f843e29c640 1 --2- 172.21.15.47:0/2793426350 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f84381061c0 0x7f8438198cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:35.473 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.472+0000 7f843e29c640 1 -- 172.21.15.47:0/2793426350 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f843819cfe0 msgr2=0x7f843819d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:05:35.473 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.472+0000 7f843e29c640 1 --2- 172.21.15.47:0/2793426350 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f843819cfe0 0x7f843819d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:35.474 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.472+0000 7f843e29c640 1 -- 172.21.15.47:0/2793426350 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f843819dad0 con 0x7f84381010f0 2024-04-17T06:05:35.474 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.472+0000 7f843ea9d640 1 --2- 172.21.15.47:0/2793426350 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f843819cfe0 0x7f843819d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:05:35.474 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.472+0000 7f843da9b640 1 --2- 172.21.15.47:0/2793426350 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f84381061c0 0x7f8438198cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:05:35.474 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.472+0000 7f843e29c640 1 --2- 172.21.15.47:0/2793426350 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f84381010f0 0x7f8438198770 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f843400d770 tx=0x7f843400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:05:35.474 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.472+0000 7f842f7fe640 1 -- 172.21.15.47:0/2793426350 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8434017070 con 0x7f84381010f0 2024-04-17T06:05:35.474 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.473+0000 7f8440500640 1 -- 172.21.15.47:0/2793426350 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f843819ddc0 con 0x7f84381010f0 2024-04-17T06:05:35.474 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.473+0000 7f842f7fe640 1 -- 172.21.15.47:0/2793426350 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8434005500 con 0x7f84381010f0 2024-04-17T06:05:35.474 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.473+0000 7f842f7fe640 1 -- 172.21.15.47:0/2793426350 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8434004e00 con 0x7f84381010f0 2024-04-17T06:05:35.474 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.473+0000 7f8440500640 1 -- 172.21.15.47:0/2793426350 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84381a5560 con 0x7f84381010f0 2024-04-17T06:05:35.481 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.475+0000 7f842f7fe640 1 -- 172.21.15.47:0/2793426350 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8434004fa0 con 0x7f84381010f0 2024-04-17T06:05:35.482 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.476+0000 7f84431be740 1 Processor -- start 2024-04-17T06:05:35.482 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.476+0000 7f84431be740 1 -- start start 2024-04-17T06:05:35.482 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.476+0000 7f84431be740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557478848b60 0x557478848f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:05:35.482 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.476+0000 7f84431be740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557478849480 0x557478856680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:05:35.482 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.476+0000 7f84431be740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557478856bc0 0x557478858fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:05:35.482 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.476+0000 7f84431be740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5574787ae7d0 con 0x557478848b60 2024-04-17T06:05:35.482 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.476+0000 7f84431be740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55747874bf70 con 0x557478849480 2024-04-17T06:05:35.482 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.476+0000 7f84431be740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5574787aa790 con 0x557478856bc0 2024-04-17T06:05:35.482 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.476+0000 7f843ea9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557478848b60 0x557478848f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:05:35.482 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.476+0000 7f843ea9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557478848b60 0x557478848f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44976/0 (socket says 172.21.15.47:44976) 2024-04-17T06:05:35.482 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.476+0000 7f843ea9d640 1 -- 172.21.15.47:0/613657499 learned_addr learned my addr 172.21.15.47:0/613657499 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:05:35.482 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.476+0000 7f842f7fe640 1 --2- 172.21.15.47:0/2793426350 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f84240a3dc0 0x7f84240a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:05:35.482 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.476+0000 7f843ea9d640 1 --2- 172.21.15.47:0/2793426350 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f84240a3dc0 0x7f84240a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:05:35.483 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.476+0000 7f843da9b640 1 --2- 172.21.15.47:0/613657499 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557478856bc0 0x557478858fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:05:35.483 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.476+0000 7f843e29c640 1 --2- 172.21.15.47:0/613657499 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557478849480 0x557478856680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:05:35.483 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.477+0000 7f843ea9d640 1 -- 172.21.15.47:0/613657499 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557478856bc0 msgr2=0x557478858fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:05:35.483 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.477+0000 7f843ea9d640 1 --2- 172.21.15.47:0/613657499 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557478856bc0 0x557478858fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:35.483 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.477+0000 7f843ea9d640 1 -- 172.21.15.47:0/613657499 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557478849480 msgr2=0x557478856680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:05:35.483 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.477+0000 7f843ea9d640 1 --2- 172.21.15.47:0/613657499 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557478849480 0x557478856680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:35.483 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.477+0000 7f843ea9d640 1 -- 172.21.15.47:0/613657499 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574784c1210 con 0x557478848b60 2024-04-17T06:05:35.483 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.477+0000 7f842f7fe640 1 -- 172.21.15.47:0/2793426350 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f84340d0300 con 0x7f84381010f0 2024-04-17T06:05:35.483 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.477+0000 7f843da9b640 1 --2- 172.21.15.47:0/613657499 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557478856bc0 0x557478858fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:05:35.483 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.477+0000 7f843ea9d640 1 --2- 172.21.15.47:0/613657499 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557478848b60 0x557478848f40 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f843002cef0 tx=0x7f843002cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73dda921eba108ec server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:05:35.483 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.477+0000 7f842cbfe640 1 -- 172.21.15.47:0/613657499 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8430039040 con 0x557478848b60 2024-04-17T06:05:35.483 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.477+0000 7f842cbfe640 1 -- 172.21.15.47:0/613657499 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8430003b70 con 0x557478848b60 2024-04-17T06:05:35.483 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.477+0000 7f842cbfe640 1 -- 172.21.15.47:0/613657499 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8430034440 con 0x557478848b60 2024-04-17T06:05:35.483 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.478+0000 7f84431be740 1 -- 172.21.15.47:0/613657499 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557478848b60 msgr2=0x557478848f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:05:35.484 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.478+0000 7f843ea9d640 1 --2- 172.21.15.47:0/2793426350 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f84240a3dc0 0x7f84240a6280 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f843004c000 tx=0x7f8430048000 comp rx=0 tx=0).ready entity=mgr.24923 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:05:35.484 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.478+0000 7f84431be740 1 --2- 172.21.15.47:0/613657499 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557478848b60 0x557478848f40 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f843002cef0 tx=0x7f843002cf20 comp rx=0 tx=0).stop 2024-04-17T06:05:35.484 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.478+0000 7f84431be740 1 -- 172.21.15.47:0/613657499 shutdown_connections 2024-04-17T06:05:35.484 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.478+0000 7f84431be740 1 --2- 172.21.15.47:0/613657499 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557478856bc0 0x557478858fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:35.484 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.478+0000 7f84431be740 1 --2- 172.21.15.47:0/613657499 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557478849480 0x557478856680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:35.484 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.478+0000 7f84431be740 1 --2- 172.21.15.47:0/613657499 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557478848b60 0x557478848f40 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:35.484 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.478+0000 7f84431be740 1 -- 172.21.15.47:0/613657499 >> 172.21.15.47:0/613657499 conn(0x5574788439e0 msgr2=0x557478843dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:05:35.484 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.478+0000 7f84431be740 1 -- 172.21.15.47:0/613657499 shutdown_connections 2024-04-17T06:05:35.484 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.478+0000 7f84431be740 1 -- 172.21.15.47:0/613657499 wait complete. 2024-04-17T06:05:35.484 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.478+0000 7f84431be740 1 -- start start 2024-04-17T06:05:35.484 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.479+0000 7f84431be740 1 Processor -- start 2024-04-17T06:05:35.484 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.479+0000 7f84431be740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557478848b60 0x557478853bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:05:35.484 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.479+0000 7f84431be740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557478849480 0x557478854130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:05:35.484 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.479+0000 7f84431be740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557478856bc0 0x557478854670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:05:35.485 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.479+0000 7f84431be740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5574788597f0 con 0x557478848b60 2024-04-17T06:05:35.485 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.479+0000 7f84431be740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x557478736e90 con 0x557478856bc0 2024-04-17T06:05:35.485 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.479+0000 7f84431be740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x557478859970 con 0x557478849480 2024-04-17T06:05:35.485 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.479+0000 7f84431be740 10 client.0 _ll_register_callbacks cb 0x55747874cd40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:05:35.485 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.479+0000 7f843e29c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557478848b60 0x557478853bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:05:35.485 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.479+0000 7f843e29c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557478848b60 0x557478853bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44980/0 (socket says 172.21.15.47:44980) 2024-04-17T06:05:35.485 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.479+0000 7f843e29c640 1 -- 172.21.15.47:0/3063222419 learned_addr learned my addr 172.21.15.47:0/3063222419 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:05:35.485 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.479+0000 7f843da9b640 1 --2- 172.21.15.47:0/3063222419 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557478849480 0x557478854130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:05:35.485 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.479+0000 7f843ea9d640 1 --2- 172.21.15.47:0/3063222419 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557478856bc0 0x557478854670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:05:35.485 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.479+0000 7f843e29c640 1 -- 172.21.15.47:0/3063222419 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557478849480 msgr2=0x557478854130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:05:35.485 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.479+0000 7f843e29c640 1 --2- 172.21.15.47:0/3063222419 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557478849480 0x557478854130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:35.485 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.479+0000 7f843e29c640 1 -- 172.21.15.47:0/3063222419 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557478856bc0 msgr2=0x557478854670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:05:35.485 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.479+0000 7f843e29c640 1 --2- 172.21.15.47:0/3063222419 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557478856bc0 0x557478854670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:35.485 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.479+0000 7f843e29c640 1 -- 172.21.15.47:0/3063222419 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55747858d620 con 0x557478848b60 2024-04-17T06:05:35.486 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.479+0000 7f843ea9d640 1 --2- 172.21.15.47:0/3063222419 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557478856bc0 0x557478854670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:05:35.486 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.480+0000 7f843e29c640 1 --2- 172.21.15.47:0/3063222419 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557478848b60 0x557478853bf0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f84340cdb30 tx=0x7f84340209d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cba40453cba35c02 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:05:35.486 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.480+0000 7f8421ffb640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-17T06:05:35.486 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.480+0000 7f8421ffb640 1 -- 172.21.15.47:0/3063222419 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8434020040 con 0x557478848b60 2024-04-17T06:05:35.486 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.480+0000 7f8421ffb640 1 -- 172.21.15.47:0/3063222419 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f8434009cc0 con 0x557478848b60 2024-04-17T06:05:35.486 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.480+0000 7f84431be740 1 -- 172.21.15.47:0/3063222419 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8430003b70 con 0x557478848b60 2024-04-17T06:05:35.486 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.480+0000 7f8421ffb640 1 -- 172.21.15.47:0/3063222419 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f843409c9d0 con 0x557478848b60 2024-04-17T06:05:35.486 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.480+0000 7f84431be740 10 client.15759 fetch_fsmap learned FSMap version 28 2024-04-17T06:05:35.486 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.480+0000 7f84431be740 1 -- 172.21.15.47:0/3063222419 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557478865910 con 0x557478848b60 2024-04-17T06:05:35.486 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.480+0000 7f8421ffb640 1 -- 172.21.15.47:0/3063222419 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8430003b70 con 0x557478848b60 2024-04-17T06:05:35.486 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.480+0000 7f8421ffb640 1 -- 172.21.15.47:0/3063222419 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 28) ==== 8339+0+0 (secure 0 0 0) 0x7f84340cdd00 con 0x557478848b60 2024-04-17T06:05:35.486 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.480+0000 7f84431be740 10 client.15759 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T06:05:35.486 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.480+0000 7f84431be740 10 client.15759 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-17T06:05:35.487 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.480+0000 7f84431be740 20 client.15759 populate_metadata read hostname 'smithi047' 2024-04-17T06:05:35.487 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.480+0000 7f84431be740 1 --2- 172.21.15.47:0/3063222419 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x557478866e20 0x557478869210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:05:35.487 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.480+0000 7f84431be740 4 client.15759 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T06:05:35.487 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.480+0000 7f84431be740 1 -- 172.21.15.47:0/3063222419 --> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5574787bc1f0 con 0x557478866e20 2024-04-17T06:05:35.487 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.481+0000 7f843da9b640 1 --2- 172.21.15.47:0/3063222419 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x557478866e20 0x557478869210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:05:35.487 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.481+0000 7f843da9b640 1 --2- 172.21.15.47:0/3063222419 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x557478866e20 0x557478869210 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53894540e41fa413 server_cookie=36299b9e72a9d22c in_seq=0 out_seq=0 2024-04-17T06:05:35.487 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.481+0000 7f8421ffb640 10 client.15759 ms_handle_connect on v2:172.21.15.98:6838/2841287917 2024-04-17T06:05:35.487 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.482+0000 7f8421ffb640 1 -- 172.21.15.47:0/3063222419 <== mds.0 v2:172.21.15.98:6838/2841287917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5574787bc1f0 con 0x557478866e20 2024-04-17T06:05:35.487 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.482+0000 7f8421ffb640 10 client.15759 handle_command_reply: tid=0 2024-04-17T06:05:35.487 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.482+0000 7f84431be740 1 client.15759 shutdown 2024-04-17T06:05:35.487 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.482+0000 7f84431be740 1 -- 172.21.15.47:0/3063222419 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557478848b60 msgr2=0x557478853bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:05:35.487 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.482+0000 7f84431be740 1 --2- 172.21.15.47:0/3063222419 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557478848b60 0x557478853bf0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f84340cdb30 tx=0x7f84340209d0 comp rx=0 tx=0).stop 2024-04-17T06:05:35.487 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.482+0000 7f84431be740 1 -- 172.21.15.47:0/3063222419 shutdown_connections 2024-04-17T06:05:35.487 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.482+0000 7f84431be740 1 --2- 172.21.15.47:0/3063222419 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x557478866e20 0x557478869210 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:35.488 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.482+0000 7f84431be740 1 --2- 172.21.15.47:0/3063222419 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557478856bc0 0x557478854670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:35.488 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.482+0000 7f84431be740 1 --2- 172.21.15.47:0/3063222419 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557478849480 0x557478854130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:35.488 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.482+0000 7f8421ffb640 0 client.15759 ms_handle_reset on v2:172.21.15.98:6838/2841287917 2024-04-17T06:05:35.488 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.482+0000 7f84431be740 1 --2- 172.21.15.47:0/3063222419 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557478848b60 0x557478853bf0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:35.488 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.482+0000 7f84431be740 1 -- 172.21.15.47:0/3063222419 >> 172.21.15.47:0/3063222419 conn(0x5574788439e0 msgr2=0x557478845730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:05:35.488 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.482+0000 7f84431be740 1 -- 172.21.15.47:0/3063222419 shutdown_connections 2024-04-17T06:05:35.488 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.482+0000 7f84431be740 1 -- 172.21.15.47:0/3063222419 wait complete. 2024-04-17T06:05:35.488 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.482+0000 7f84431be740 20 client.15759 trim_cache size 0 max 16384 2024-04-17T06:05:35.500 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.497+0000 7f84431be740 1 Processor -- start 2024-04-17T06:05:35.500 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.497+0000 7f84431be740 1 -- start start 2024-04-17T06:05:35.500 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.498+0000 7f84431be740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557478848b60 0x557478856320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:05:35.500 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.498+0000 7f84431be740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557478849480 0x5574788626f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:05:35.500 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.498+0000 7f84431be740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557478856bc0 0x557478845540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:05:35.500 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.498+0000 7f84431be740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f84381ae220 con 0x557478856bc0 2024-04-17T06:05:35.500 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.498+0000 7f84431be740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f84381abae0 con 0x557478849480 2024-04-17T06:05:35.500 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.498+0000 7f84431be740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f84381a9930 con 0x557478848b60 2024-04-17T06:05:35.501 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.498+0000 7f843ea9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557478856bc0 0x557478845540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:05:35.501 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.498+0000 7f843ea9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557478856bc0 0x557478845540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44990/0 (socket says 172.21.15.47:44990) 2024-04-17T06:05:35.501 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.498+0000 7f843ea9d640 1 -- 172.21.15.47:0/1758148081 learned_addr learned my addr 172.21.15.47:0/1758148081 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:05:35.501 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.498+0000 7f843e29c640 1 --2- 172.21.15.47:0/1758148081 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557478848b60 0x557478856320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:05:35.501 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.498+0000 7f843da9b640 1 --2- 172.21.15.47:0/1758148081 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557478849480 0x5574788626f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:05:35.501 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.498+0000 7f843ea9d640 1 -- 172.21.15.47:0/1758148081 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557478848b60 msgr2=0x557478856320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:05:35.501 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.498+0000 7f843ea9d640 1 --2- 172.21.15.47:0/1758148081 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557478848b60 0x557478856320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:35.501 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.498+0000 7f843ea9d640 1 -- 172.21.15.47:0/1758148081 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557478849480 msgr2=0x5574788626f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:05:35.501 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.498+0000 7f843ea9d640 1 --2- 172.21.15.47:0/1758148081 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557478849480 0x5574788626f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:35.501 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.498+0000 7f843ea9d640 1 -- 172.21.15.47:0/1758148081 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557478849f80 con 0x557478856bc0 2024-04-17T06:05:35.501 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.498+0000 7f843ea9d640 1 --2- 172.21.15.47:0/1758148081 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557478856bc0 0x557478845540 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f843002cfd0 tx=0x7f843002cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=956eeaedf44a179f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:05:35.501 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.498+0000 7f8423fff640 1 -- 172.21.15.47:0/1758148081 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8430039050 con 0x557478856bc0 2024-04-17T06:05:35.501 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.499+0000 7f8423fff640 1 -- 172.21.15.47:0/1758148081 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8430034bf0 con 0x557478856bc0 2024-04-17T06:05:35.501 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.499+0000 7f8423fff640 1 -- 172.21.15.47:0/1758148081 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8430004040 con 0x557478856bc0 2024-04-17T06:05:35.502 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.499+0000 7f84431be740 1 -- 172.21.15.47:0/1758148081 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557478856bc0 msgr2=0x557478845540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:05:35.502 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.499+0000 7f84431be740 1 --2- 172.21.15.47:0/1758148081 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557478856bc0 0x557478845540 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f843002cfd0 tx=0x7f843002cca0 comp rx=0 tx=0).stop 2024-04-17T06:05:35.502 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.499+0000 7f84431be740 1 -- 172.21.15.47:0/1758148081 shutdown_connections 2024-04-17T06:05:35.502 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.499+0000 7f84431be740 1 --2- 172.21.15.47:0/1758148081 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557478856bc0 0x557478845540 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:35.502 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.499+0000 7f84431be740 1 --2- 172.21.15.47:0/1758148081 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557478849480 0x5574788626f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:35.502 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.499+0000 7f84431be740 1 --2- 172.21.15.47:0/1758148081 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557478848b60 0x557478856320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:35.502 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.499+0000 7f84431be740 1 -- 172.21.15.47:0/1758148081 >> 172.21.15.47:0/1758148081 conn(0x5574788439e0 msgr2=0x557478844e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:05:35.502 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.499+0000 7f84431be740 1 -- 172.21.15.47:0/1758148081 shutdown_connections 2024-04-17T06:05:35.502 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.499+0000 7f84431be740 1 -- 172.21.15.47:0/1758148081 wait complete. 2024-04-17T06:05:35.502 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.499+0000 7f84431be740 1 -- start start 2024-04-17T06:05:35.502 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.499+0000 7f84431be740 1 Processor -- start 2024-04-17T06:05:35.502 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.500+0000 7f84431be740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557478848b60 0x557478850f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:05:35.502 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.500+0000 7f84431be740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557478849480 0x557478851470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:05:35.502 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.500+0000 7f84431be740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557478856bc0 0x5574788cc340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:05:35.503 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.500+0000 7f84431be740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f84381a9fa0 con 0x557478856bc0 2024-04-17T06:05:35.503 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.500+0000 7f84431be740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f84381ab410 con 0x557478848b60 2024-04-17T06:05:35.503 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.500+0000 7f84431be740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f84381aa670 con 0x557478849480 2024-04-17T06:05:35.503 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.500+0000 7f84431be740 10 client.0 _ll_register_callbacks cb 0x7f84340056b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:05:35.503 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.500+0000 7f843ea9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557478856bc0 0x5574788cc340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:05:35.503 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.500+0000 7f843ea9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557478856bc0 0x5574788cc340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45002/0 (socket says 172.21.15.47:45002) 2024-04-17T06:05:35.503 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.500+0000 7f843ea9d640 1 -- 172.21.15.47:0/1405594953 learned_addr learned my addr 172.21.15.47:0/1405594953 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:05:35.503 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.500+0000 7f843da9b640 1 --2- 172.21.15.47:0/1405594953 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557478849480 0x557478851470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:05:35.503 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.500+0000 7f843e29c640 1 --2- 172.21.15.47:0/1405594953 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557478848b60 0x557478850f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:05:35.503 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.500+0000 7f843ea9d640 1 -- 172.21.15.47:0/1405594953 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557478849480 msgr2=0x557478851470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:05:35.503 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.500+0000 7f843ea9d640 1 --2- 172.21.15.47:0/1405594953 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557478849480 0x557478851470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:35.503 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.500+0000 7f843ea9d640 1 -- 172.21.15.47:0/1405594953 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557478848b60 msgr2=0x557478850f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:05:35.503 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.500+0000 7f843ea9d640 1 --2- 172.21.15.47:0/1405594953 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557478848b60 0x557478850f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:35.503 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.500+0000 7f843ea9d640 1 -- 172.21.15.47:0/1405594953 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574788cc880 con 0x557478856bc0 2024-04-17T06:05:35.504 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.500+0000 7f843ea9d640 1 --2- 172.21.15.47:0/1405594953 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557478856bc0 0x5574788cc340 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f84300344c0 tx=0x7f8430032f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf1a77a8c8e4e311 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:05:35.504 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.500+0000 7f8421ffb640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-17T06:05:35.504 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.501+0000 7f8421ffb640 1 -- 172.21.15.47:0/1405594953 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8430034710 con 0x557478856bc0 2024-04-17T06:05:35.504 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.501+0000 7f8421ffb640 1 -- 172.21.15.47:0/1405594953 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f84300348b0 con 0x557478856bc0 2024-04-17T06:05:35.504 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.501+0000 7f84431be740 1 -- 172.21.15.47:0/1405594953 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8430034bf0 con 0x557478856bc0 2024-04-17T06:05:35.504 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.501+0000 7f8421ffb640 1 -- 172.21.15.47:0/1405594953 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8430039020 con 0x557478856bc0 2024-04-17T06:05:35.504 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.501+0000 7f84431be740 10 client.15765 fetch_fsmap learned FSMap version 28 2024-04-17T06:05:35.504 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.501+0000 7f84431be740 1 -- 172.21.15.47:0/1405594953 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5574788cdac0 con 0x557478856bc0 2024-04-17T06:05:35.504 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.501+0000 7f8421ffb640 1 -- 172.21.15.47:0/1405594953 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8430034bf0 con 0x557478856bc0 2024-04-17T06:05:35.504 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.501+0000 7f8421ffb640 1 -- 172.21.15.47:0/1405594953 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 28) ==== 8339+0+0 (secure 0 0 0) 0x7f843004c530 con 0x557478856bc0 2024-04-17T06:05:35.504 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.501+0000 7f84431be740 10 client.15765 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T06:05:35.504 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.501+0000 7f84431be740 10 client.15765 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-17T06:05:35.504 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.501+0000 7f84431be740 20 client.15765 populate_metadata read hostname 'smithi047' 2024-04-17T06:05:35.504 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.501+0000 7f84431be740 1 --2- 172.21.15.47:0/1405594953 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x5574788cddf0 0x5574788d01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:05:35.505 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.501+0000 7f84431be740 4 client.15765 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-17T06:05:35.505 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.501+0000 7f84431be740 1 -- 172.21.15.47:0/1405594953 --> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5574787b6060 con 0x5574788cddf0 2024-04-17T06:05:35.505 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.502+0000 7f843e29c640 1 --2- 172.21.15.47:0/1405594953 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x5574788cddf0 0x5574788d01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:05:35.505 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.502+0000 7f843e29c640 1 --2- 172.21.15.47:0/1405594953 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x5574788cddf0 0x5574788d01e0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b615dfbe53d3f088 server_cookie=15c5818adc61eb2a in_seq=0 out_seq=0 2024-04-17T06:05:35.505 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.502+0000 7f8421ffb640 10 client.15765 ms_handle_connect on v2:172.21.15.98:6838/2841287917 2024-04-17T06:05:35.508 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.510+0000 7f8421ffb640 1 -- 172.21.15.47:0/1405594953 <== mds.0 v2:172.21.15.98:6838/2841287917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5574787b6060 con 0x5574788cddf0 2024-04-17T06:05:35.508 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.510+0000 7f8421ffb640 10 client.15765 handle_command_reply: tid=0 2024-04-17T06:05:35.509 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.510+0000 7f84431be740 1 client.15765 shutdown 2024-04-17T06:05:35.509 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.510+0000 7f84431be740 1 -- 172.21.15.47:0/1405594953 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557478856bc0 msgr2=0x5574788cc340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:05:35.509 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.510+0000 7f84431be740 1 --2- 172.21.15.47:0/1405594953 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557478856bc0 0x5574788cc340 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f84300344c0 tx=0x7f8430032f30 comp rx=0 tx=0).stop 2024-04-17T06:05:35.509 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.510+0000 7f84431be740 1 -- 172.21.15.47:0/1405594953 shutdown_connections 2024-04-17T06:05:35.509 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.510+0000 7f84431be740 1 --2- 172.21.15.47:0/1405594953 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x5574788cddf0 0x5574788d01e0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:35.509 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.510+0000 7f84431be740 1 --2- 172.21.15.47:0/1405594953 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557478856bc0 0x5574788cc340 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:35.509 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.510+0000 7f84431be740 1 --2- 172.21.15.47:0/1405594953 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557478849480 0x557478851470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:35.509 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.510+0000 7f8421ffb640 0 client.15765 ms_handle_reset on v2:172.21.15.98:6838/2841287917 2024-04-17T06:05:35.509 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.510+0000 7f84431be740 1 --2- 172.21.15.47:0/1405594953 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557478848b60 0x557478850f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:35.509 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.510+0000 7f84431be740 1 -- 172.21.15.47:0/1405594953 >> 172.21.15.47:0/1405594953 conn(0x5574788439e0 msgr2=0x557478842b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:05:35.509 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.510+0000 7f84431be740 1 -- 172.21.15.47:0/1405594953 shutdown_connections 2024-04-17T06:05:35.509 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.510+0000 7f84431be740 1 -- 172.21.15.47:0/1405594953 wait complete. 2024-04-17T06:05:35.509 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.510+0000 7f84431be740 20 client.15765 trim_cache size 0 max 16384 2024-04-17T06:05:35.510 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-04-17T06:05:35.510 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-04-17T06:05:35.510 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-04-17T06:05:35.510 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-04-17T06:05:35.511 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.511+0000 7f8440500640 1 -- 172.21.15.47:0/2793426350 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f84240a3dc0 msgr2=0x7f84240a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:05:35.511 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.511+0000 7f8440500640 1 --2- 172.21.15.47:0/2793426350 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f84240a3dc0 0x7f84240a6280 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f843004c000 tx=0x7f8430048000 comp rx=0 tx=0).stop 2024-04-17T06:05:35.511 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.511+0000 7f8440500640 1 -- 172.21.15.47:0/2793426350 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f84381010f0 msgr2=0x7f8438198770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:05:35.511 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.511+0000 7f8440500640 1 --2- 172.21.15.47:0/2793426350 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f84381010f0 0x7f8438198770 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f843400d770 tx=0x7f843400dc30 comp rx=0 tx=0).stop 2024-04-17T06:05:35.511 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.511+0000 7f8440500640 1 -- 172.21.15.47:0/2793426350 shutdown_connections 2024-04-17T06:05:35.511 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.511+0000 7f8440500640 1 --2- 172.21.15.47:0/2793426350 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f843819cfe0 0x7f843819d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:35.511 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.511+0000 7f8440500640 1 --2- 172.21.15.47:0/2793426350 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f84381061c0 0x7f8438198cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:35.511 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.511+0000 7f8440500640 1 --2- 172.21.15.47:0/2793426350 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f84240a3dc0 0x7f84240a6280 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:35.511 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.511+0000 7f8440500640 1 --2- 172.21.15.47:0/2793426350 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f84381010f0 0x7f8438198770 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:35.511 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.511+0000 7f8440500640 1 -- 172.21.15.47:0/2793426350 >> 172.21.15.47:0/2793426350 conn(0x7f84380771e0 msgr2=0x7f843806a970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:05:35.511 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.511+0000 7f8440500640 1 -- 172.21.15.47:0/2793426350 shutdown_connections 2024-04-17T06:05:35.511 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:35.511+0000 7f8440500640 1 -- 172.21.15.47:0/2793426350 wait complete. 2024-04-17T06:05:35.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:05:35 smithi098 ceph-mon[36516]: pgmap v133: 97 pgs: 97 active+clean; 2.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-04-17T06:05:35.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:05:35 smithi047 ceph-mon[32223]: pgmap v133: 97 pgs: 97 active+clean; 2.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-04-17T06:05:36.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:05:35 smithi103 ceph-mon[36243]: pgmap v133: 97 pgs: 97 active+clean; 2.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-04-17T06:05:37.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:05:37 smithi098 ceph-mon[36516]: pgmap v134: 97 pgs: 97 active+clean; 2.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.2 KiB/s wr, 9 op/s 2024-04-17T06:05:37.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:05:37 smithi047 ceph-mon[32223]: pgmap v134: 97 pgs: 97 active+clean; 2.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.2 KiB/s wr, 9 op/s 2024-04-17T06:05:38.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:05:37 smithi103 ceph-mon[36243]: pgmap v134: 97 pgs: 97 active+clean; 2.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.2 KiB/s wr, 9 op/s 2024-04-17T06:05:39.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:05:39 smithi098 ceph-mon[36516]: pgmap v135: 97 pgs: 97 active+clean; 2.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.2 KiB/s wr, 9 op/s 2024-04-17T06:05:39.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:05:39 smithi047 ceph-mon[32223]: pgmap v135: 97 pgs: 97 active+clean; 2.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.2 KiB/s wr, 9 op/s 2024-04-17T06:05:40.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:05:39 smithi103 ceph-mon[36243]: pgmap v135: 97 pgs: 97 active+clean; 2.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.2 KiB/s wr, 9 op/s 2024-04-17T06:05:41.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:05:41 smithi098 ceph-mon[36516]: pgmap v136: 97 pgs: 97 active+clean; 2.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-04-17T06:05:41.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:05:41 smithi047 ceph-mon[32223]: pgmap v136: 97 pgs: 97 active+clean; 2.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-04-17T06:05:42.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:05:41 smithi103 ceph-mon[36243]: pgmap v136: 97 pgs: 97 active+clean; 2.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-04-17T06:05:43.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:05:43 smithi098 ceph-mon[36516]: pgmap v137: 97 pgs: 97 active+clean; 2.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-04-17T06:05:43.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:05:43 smithi047 ceph-mon[32223]: pgmap v137: 97 pgs: 97 active+clean; 2.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-04-17T06:05:44.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:05:43 smithi103 ceph-mon[36243]: pgmap v137: 97 pgs: 97 active+clean; 2.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-04-17T06:05:45.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:05:45 smithi098 ceph-mon[36516]: pgmap v138: 97 pgs: 97 active+clean; 2.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-04-17T06:05:45.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:05:45 smithi047 ceph-mon[32223]: pgmap v138: 97 pgs: 97 active+clean; 2.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-04-17T06:05:46.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:05:45 smithi103 ceph-mon[36243]: pgmap v138: 97 pgs: 97 active+clean; 2.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-04-17T06:05:47.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:05:47 smithi098 ceph-mon[36516]: pgmap v139: 97 pgs: 97 active+clean; 2.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 10 KiB/s wr, 11 op/s 2024-04-17T06:05:47.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:05:47 smithi047 ceph-mon[32223]: pgmap v139: 97 pgs: 97 active+clean; 2.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 10 KiB/s wr, 11 op/s 2024-04-17T06:05:48.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:05:47 smithi103 ceph-mon[36243]: pgmap v139: 97 pgs: 97 active+clean; 2.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 10 KiB/s wr, 11 op/s 2024-04-17T06:05:48.613 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.610+0000 7fc4b8574640 1 -- 172.21.15.47:0/2775336864 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4b01072a0 msgr2=0x7fc4b010ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:05:48.613 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.610+0000 7fc4b8574640 1 --2- 172.21.15.47:0/2775336864 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4b01072a0 0x7fc4b010ee70 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fc4a8009ca0 tx=0x7fc4a802c0f0 comp rx=0 tx=0).stop 2024-04-17T06:05:48.613 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.610+0000 7fc4b8574640 1 -- 172.21.15.47:0/2775336864 shutdown_connections 2024-04-17T06:05:48.614 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.611+0000 7fc4b8574640 1 --2- 172.21.15.47:0/2775336864 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4b01072a0 0x7fc4b010ee70 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:48.614 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.611+0000 7fc4b8574640 1 --2- 172.21.15.47:0/2775336864 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc4b0102b80 0x7fc4b0106bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:48.614 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.611+0000 7fc4b8574640 1 --2- 172.21.15.47:0/2775336864 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc4b01021d0 0x7fc4b01025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:48.614 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.611+0000 7fc4b8574640 1 -- 172.21.15.47:0/2775336864 >> 172.21.15.47:0/2775336864 conn(0x7fc4b00fda30 msgr2=0x7fc4b00ffe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:05:48.614 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.611+0000 7fc4b8574640 1 -- 172.21.15.47:0/2775336864 shutdown_connections 2024-04-17T06:05:48.614 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.611+0000 7fc4b8574640 1 -- 172.21.15.47:0/2775336864 wait complete. 2024-04-17T06:05:48.614 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.611+0000 7fc4b8574640 1 Processor -- start 2024-04-17T06:05:48.614 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.611+0000 7fc4b8574640 1 -- start start 2024-04-17T06:05:48.614 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.611+0000 7fc4b8574640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4b01021d0 0x7fc4b0074260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:05:48.614 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.611+0000 7fc4b8574640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc4b01072a0 0x7fc4b00747a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:05:48.614 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.611+0000 7fc4b8574640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc4b0074ce0 0x7fc4b01a6890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:05:48.614 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.612+0000 7fc4b8574640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b0112340 con 0x7fc4b01021d0 2024-04-17T06:05:48.614 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.612+0000 7fc4b8574640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b01121c0 con 0x7fc4b01072a0 2024-04-17T06:05:48.615 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.612+0000 7fc4b8574640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b01124c0 con 0x7fc4b0074ce0 2024-04-17T06:05:48.615 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.612+0000 7fc4b6310640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4b01021d0 0x7fc4b0074260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:05:48.615 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.612+0000 7fc4b6310640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4b01021d0 0x7fc4b0074260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:42496/0 (socket says 172.21.15.47:42496) 2024-04-17T06:05:48.615 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.612+0000 7fc4b6310640 1 -- 172.21.15.47:0/119522826 learned_addr learned my addr 172.21.15.47:0/119522826 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:05:48.615 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.612+0000 7fc4b5b0f640 1 --2- 172.21.15.47:0/119522826 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc4b01072a0 0x7fc4b00747a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:05:48.615 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.612+0000 7fc4b6b11640 1 --2- 172.21.15.47:0/119522826 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc4b0074ce0 0x7fc4b01a6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:05:48.615 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.612+0000 7fc4b6310640 1 -- 172.21.15.47:0/119522826 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc4b0074ce0 msgr2=0x7fc4b01a6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:05:48.615 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.612+0000 7fc4b6310640 1 --2- 172.21.15.47:0/119522826 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc4b0074ce0 0x7fc4b01a6890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:48.615 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.612+0000 7fc4b6310640 1 -- 172.21.15.47:0/119522826 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc4b01072a0 msgr2=0x7fc4b00747a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:05:48.615 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.612+0000 7fc4b6310640 1 --2- 172.21.15.47:0/119522826 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc4b01072a0 0x7fc4b00747a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:48.615 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.612+0000 7fc4b6310640 1 -- 172.21.15.47:0/119522826 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4b01a6f90 con 0x7fc4b01021d0 2024-04-17T06:05:48.615 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.612+0000 7fc4b6b11640 1 --2- 172.21.15.47:0/119522826 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc4b0074ce0 0x7fc4b01a6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:05:48.615 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.612+0000 7fc4b5b0f640 1 --2- 172.21.15.47:0/119522826 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc4b01072a0 0x7fc4b00747a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:05:48.616 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.612+0000 7fc4b6310640 1 --2- 172.21.15.47:0/119522826 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4b01021d0 0x7fc4b0074260 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac00c920 tx=0x7fc4ac00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:05:48.616 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.612+0000 7fc4a77fe640 1 -- 172.21.15.47:0/119522826 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4ac007c70 con 0x7fc4b01021d0 2024-04-17T06:05:48.616 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.612+0000 7fc4a77fe640 1 -- 172.21.15.47:0/119522826 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4ac007e10 con 0x7fc4b01021d0 2024-04-17T06:05:48.616 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.612+0000 7fc4b8574640 1 -- 172.21.15.47:0/119522826 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4b01a7280 con 0x7fc4b01021d0 2024-04-17T06:05:48.616 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.612+0000 7fc4a77fe640 1 -- 172.21.15.47:0/119522826 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4ac018890 con 0x7fc4b01021d0 2024-04-17T06:05:48.616 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.613+0000 7fc4b8574640 1 -- 172.21.15.47:0/119522826 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4b01a77d0 con 0x7fc4b01021d0 2024-04-17T06:05:48.616 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.615+0000 7fc4bb235740 1 Processor -- start 2024-04-17T06:05:48.616 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.615+0000 7fc4a77fe640 1 -- 172.21.15.47:0/119522826 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc4ac018a30 con 0x7fc4b01021d0 2024-04-17T06:05:48.616 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.616+0000 7fc4bb235740 1 -- start start 2024-04-17T06:05:48.622 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.616+0000 7fc4bb235740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56400fb830c0 0x56400fb834a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:05:48.623 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.616+0000 7fc4bb235740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56400fb839e0 0x56400fb8c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:05:48.623 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.616+0000 7fc4bb235740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56400fb8c7b0 0x56400fb8eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:05:48.623 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.616+0000 7fc4bb235740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x56400fae87d0 con 0x56400fb839e0 2024-04-17T06:05:48.623 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.616+0000 7fc4bb235740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x56400fa85f70 con 0x56400fb830c0 2024-04-17T06:05:48.623 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.616+0000 7fc4bb235740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56400fae4790 con 0x56400fb8c7b0 2024-04-17T06:05:48.623 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.616+0000 7fc4b6310640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56400fb839e0 0x56400fb8c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:05:48.623 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.616+0000 7fc4b6310640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56400fb839e0 0x56400fb8c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:42502/0 (socket says 172.21.15.47:42502) 2024-04-17T06:05:48.623 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.616+0000 7fc4b6310640 1 -- 172.21.15.47:0/104452108 learned_addr learned my addr 172.21.15.47:0/104452108 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:05:48.623 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.616+0000 7fc4b6b11640 1 --2- 172.21.15.47:0/104452108 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56400fb830c0 0x56400fb834a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:05:48.623 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.616+0000 7fc4b5b0f640 1 --2- 172.21.15.47:0/104452108 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56400fb8c7b0 0x56400fb8eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:05:48.623 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.617+0000 7fc4b6310640 1 -- 172.21.15.47:0/104452108 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56400fb8c7b0 msgr2=0x56400fb8eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:05:48.623 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.617+0000 7fc4b6310640 1 --2- 172.21.15.47:0/104452108 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56400fb8c7b0 0x56400fb8eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:48.624 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.617+0000 7fc4b6310640 1 -- 172.21.15.47:0/104452108 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56400fb830c0 msgr2=0x56400fb834a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:05:48.624 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.617+0000 7fc4b6310640 1 --2- 172.21.15.47:0/104452108 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56400fb830c0 0x56400fb834a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:48.624 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.617+0000 7fc4b6310640 1 -- 172.21.15.47:0/104452108 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56400f7fb210 con 0x56400fb839e0 2024-04-17T06:05:48.624 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.617+0000 7fc4b5b0f640 1 --2- 172.21.15.47:0/104452108 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56400fb8c7b0 0x56400fb8eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:05:48.624 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.617+0000 7fc4a77fe640 1 --2- 172.21.15.47:0/119522826 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fc49c0a3df0 0x7fc49c0a62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:05:48.624 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.617+0000 7fc4b6b11640 1 --2- 172.21.15.47:0/119522826 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fc49c0a3df0 0x7fc49c0a62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:05:48.624 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.617+0000 7fc4b6310640 1 --2- 172.21.15.47:0/104452108 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56400fb839e0 0x56400fb8c270 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac0ce090 tx=0x7fc4ac0d6cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73888396c0ce0d67 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:05:48.624 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.617+0000 7fc4a77fe640 1 -- 172.21.15.47:0/119522826 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7fc4ac0d09a0 con 0x7fc4b01021d0 2024-04-17T06:05:48.624 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.617+0000 7fc4a4bfe640 1 -- 172.21.15.47:0/104452108 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4ac098c10 con 0x56400fb839e0 2024-04-17T06:05:48.624 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.617+0000 7fc4a4bfe640 1 -- 172.21.15.47:0/104452108 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4ac0d0e50 con 0x56400fb839e0 2024-04-17T06:05:48.624 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.617+0000 7fc4a4bfe640 1 -- 172.21.15.47:0/104452108 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4ac09c2d0 con 0x56400fb839e0 2024-04-17T06:05:48.624 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.618+0000 7fc4bb235740 1 -- 172.21.15.47:0/104452108 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56400fb839e0 msgr2=0x56400fb8c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:05:48.624 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.618+0000 7fc4bb235740 1 --2- 172.21.15.47:0/104452108 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56400fb839e0 0x56400fb8c270 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac0ce090 tx=0x7fc4ac0d6cd0 comp rx=0 tx=0).stop 2024-04-17T06:05:48.624 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.618+0000 7fc4b6b11640 1 --2- 172.21.15.47:0/119522826 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fc49c0a3df0 0x7fc49c0a62b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc4a8006010 tx=0x7fc4a8005ef0 comp rx=0 tx=0).ready entity=mgr.24923 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:05:48.625 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.618+0000 7fc4bb235740 1 -- 172.21.15.47:0/104452108 shutdown_connections 2024-04-17T06:05:48.625 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.618+0000 7fc4bb235740 1 --2- 172.21.15.47:0/104452108 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56400fb8c7b0 0x56400fb8eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:48.625 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.618+0000 7fc4bb235740 1 --2- 172.21.15.47:0/104452108 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56400fb839e0 0x56400fb8c270 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:48.625 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.618+0000 7fc4bb235740 1 --2- 172.21.15.47:0/104452108 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56400fb830c0 0x56400fb834a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:48.625 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.618+0000 7fc4bb235740 1 -- 172.21.15.47:0/104452108 >> 172.21.15.47:0/104452108 conn(0x56400fb7d9e0 msgr2=0x56400fb7ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:05:48.625 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.618+0000 7fc4bb235740 1 -- 172.21.15.47:0/104452108 shutdown_connections 2024-04-17T06:05:48.625 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.618+0000 7fc4bb235740 1 -- 172.21.15.47:0/104452108 wait complete. 2024-04-17T06:05:48.625 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.619+0000 7fc4bb235740 1 -- start start 2024-04-17T06:05:48.625 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.619+0000 7fc4bb235740 1 Processor -- start 2024-04-17T06:05:48.625 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.619+0000 7fc4bb235740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56400fb830c0 0x56400fb89900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:05:48.625 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.619+0000 7fc4bb235740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56400fb839e0 0x56400fb89e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:05:48.625 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.619+0000 7fc4bb235740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56400fb8c7b0 0x56400fb8a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:05:48.625 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.619+0000 7fc4bb235740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x56400fb8f3e0 con 0x56400fb8c7b0 2024-04-17T06:05:48.625 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.619+0000 7fc4bb235740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x56400fa70e90 con 0x56400fb830c0 2024-04-17T06:05:48.625 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.619+0000 7fc4bb235740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56400fb8f560 con 0x56400fb839e0 2024-04-17T06:05:48.626 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.619+0000 7fc4bb235740 10 client.0 _ll_register_callbacks cb 0x56400fa86d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:05:48.626 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.619+0000 7fc4b6b11640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56400fb8c7b0 0x56400fb8a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:05:48.626 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.619+0000 7fc4b6b11640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56400fb8c7b0 0x56400fb8a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:42508/0 (socket says 172.21.15.47:42508) 2024-04-17T06:05:48.626 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.619+0000 7fc4b6310640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56400fb830c0 0x56400fb89900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:05:48.626 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.619+0000 7fc4b6b11640 1 -- 172.21.15.47:0/1488990370 learned_addr learned my addr 172.21.15.47:0/1488990370 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:05:48.626 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.619+0000 7fc4b5b0f640 1 --2- 172.21.15.47:0/1488990370 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56400fb839e0 0x56400fb89e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:05:48.626 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.619+0000 7fc4b6310640 1 -- 172.21.15.47:0/1488990370 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56400fb839e0 msgr2=0x56400fb89e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:05:48.626 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.619+0000 7fc4b6310640 1 --2- 172.21.15.47:0/1488990370 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56400fb839e0 0x56400fb89e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:48.626 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.619+0000 7fc4b6310640 1 -- 172.21.15.47:0/1488990370 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56400fb8c7b0 msgr2=0x56400fb8a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:05:48.626 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.619+0000 7fc4b6310640 1 --2- 172.21.15.47:0/1488990370 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56400fb8c7b0 0x56400fb8a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:48.626 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.619+0000 7fc4b6310640 1 -- 172.21.15.47:0/1488990370 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56400f8c7620 con 0x56400fb830c0 2024-04-17T06:05:48.626 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.619+0000 7fc4b6b11640 1 --2- 172.21.15.47:0/1488990370 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56400fb8c7b0 0x56400fb8a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T06:05:48.626 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.620+0000 7fc4b6310640 1 --2- 172.21.15.47:0/1488990370 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56400fb830c0 0x56400fb89900 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x56400fb82b70 tx=0x7fc4ac0d0d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85d0eb7496e301fb server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:05:48.627 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.620+0000 7fc499ffb640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T06:05:48.627 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.620+0000 7fc499ffb640 1 -- 172.21.15.47:0/1488990370 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4ac09c4b0 con 0x56400fb830c0 2024-04-17T06:05:48.627 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.620+0000 7fc4bb235740 1 -- 172.21.15.47:0/1488990370 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4ac0d0e50 con 0x56400fb830c0 2024-04-17T06:05:48.627 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.620+0000 7fc499ffb640 1 -- 172.21.15.47:0/1488990370 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7fc4ac018d20 con 0x56400fb830c0 2024-04-17T06:05:48.627 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.620+0000 7fc499ffb640 1 -- 172.21.15.47:0/1488990370 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc4ac09c740 con 0x56400fb830c0 2024-04-17T06:05:48.627 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.620+0000 7fc4bb235740 10 client.25283 fetch_fsmap learned FSMap version 28 2024-04-17T06:05:48.627 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.620+0000 7fc4bb235740 1 -- 172.21.15.47:0/1488990370 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56400fb9b770 con 0x56400fb830c0 2024-04-17T06:05:48.627 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.620+0000 7fc499ffb640 1 -- 172.21.15.47:0/1488990370 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4ac0d0e50 con 0x56400fb830c0 2024-04-17T06:05:48.627 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.620+0000 7fc499ffb640 1 -- 172.21.15.47:0/1488990370 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8339+0+0 (secure 0 0 0) 0x7fc4ac098fe0 con 0x56400fb830c0 2024-04-17T06:05:48.627 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.620+0000 7fc4bb235740 10 client.25283 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T06:05:48.627 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.620+0000 7fc4bb235740 10 client.25283 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-17T06:05:48.627 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.620+0000 7fc4bb235740 20 client.25283 populate_metadata read hostname 'smithi047' 2024-04-17T06:05:48.627 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.620+0000 7fc4bb235740 1 --2- 172.21.15.47:0/1488990370 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x56400fb9cac0 0x56400fb9eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:05:48.627 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.620+0000 7fc4bb235740 4 client.25283 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T06:05:48.628 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.620+0000 7fc4bb235740 1 -- 172.21.15.47:0/1488990370 --> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56400faf61f0 con 0x56400fb9cac0 2024-04-17T06:05:48.628 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.621+0000 7fc4b5b0f640 1 --2- 172.21.15.47:0/1488990370 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x56400fb9cac0 0x56400fb9eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:05:48.628 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.621+0000 7fc4b5b0f640 1 --2- 172.21.15.47:0/1488990370 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x56400fb9cac0 0x56400fb9eeb0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46f4f5b7dc8d3cec server_cookie=22a0682c97ad892e in_seq=0 out_seq=0 2024-04-17T06:05:48.628 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.621+0000 7fc499ffb640 10 client.25283 ms_handle_connect on v2:172.21.15.98:6838/2841287917 2024-04-17T06:05:48.628 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.622+0000 7fc499ffb640 1 -- 172.21.15.47:0/1488990370 <== mds.0 v2:172.21.15.98:6838/2841287917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56400faf61f0 con 0x56400fb9cac0 2024-04-17T06:05:48.628 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.622+0000 7fc499ffb640 10 client.25283 handle_command_reply: tid=0 2024-04-17T06:05:48.628 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.622+0000 7fc4bb235740 1 client.25283 shutdown 2024-04-17T06:05:48.628 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.622+0000 7fc4bb235740 1 -- 172.21.15.47:0/1488990370 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56400fb830c0 msgr2=0x56400fb89900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:05:48.628 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.622+0000 7fc4bb235740 1 --2- 172.21.15.47:0/1488990370 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56400fb830c0 0x56400fb89900 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x56400fb82b70 tx=0x7fc4ac0d0d10 comp rx=0 tx=0).stop 2024-04-17T06:05:48.628 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.622+0000 7fc4bb235740 1 -- 172.21.15.47:0/1488990370 shutdown_connections 2024-04-17T06:05:48.628 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.622+0000 7fc4bb235740 1 --2- 172.21.15.47:0/1488990370 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x56400fb9cac0 0x56400fb9eeb0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:48.628 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.622+0000 7fc4bb235740 1 --2- 172.21.15.47:0/1488990370 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56400fb8c7b0 0x56400fb8a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:48.628 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.622+0000 7fc4bb235740 1 --2- 172.21.15.47:0/1488990370 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56400fb839e0 0x56400fb89e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:48.629 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.622+0000 7fc4bb235740 1 --2- 172.21.15.47:0/1488990370 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56400fb830c0 0x56400fb89900 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:48.629 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.622+0000 7fc499ffb640 0 client.25283 ms_handle_reset on v2:172.21.15.98:6838/2841287917 2024-04-17T06:05:48.629 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.622+0000 7fc4bb235740 1 -- 172.21.15.47:0/1488990370 >> 172.21.15.47:0/1488990370 conn(0x56400fb7d9e0 msgr2=0x56400fb7f730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:05:48.629 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.622+0000 7fc4bb235740 1 -- 172.21.15.47:0/1488990370 shutdown_connections 2024-04-17T06:05:48.629 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.622+0000 7fc4bb235740 1 -- 172.21.15.47:0/1488990370 wait complete. 2024-04-17T06:05:48.629 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.622+0000 7fc4bb235740 20 client.25283 trim_cache size 0 max 16384 2024-04-17T06:05:48.639 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.637+0000 7fc4bb235740 1 Processor -- start 2024-04-17T06:05:48.639 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.637+0000 7fc4bb235740 1 -- start start 2024-04-17T06:05:48.640 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.637+0000 7fc4bb235740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56400fb830c0 0x56400fb8c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:05:48.640 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.637+0000 7fc4bb235740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56400fb839e0 0x56400fb887f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:05:48.640 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.637+0000 7fc4bb235740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56400fb8c7b0 0x56400fb86bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:05:48.640 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.637+0000 7fc4bb235740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b01b2680 con 0x56400fb830c0 2024-04-17T06:05:48.640 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.637+0000 7fc4bb235740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b01aff40 con 0x56400fb839e0 2024-04-17T06:05:48.640 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.637+0000 7fc4bb235740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b01add90 con 0x56400fb8c7b0 2024-04-17T06:05:48.640 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.637+0000 7fc4b6310640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56400fb830c0 0x56400fb8c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:05:48.640 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.637+0000 7fc4b6310640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56400fb830c0 0x56400fb8c030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:42512/0 (socket says 172.21.15.47:42512) 2024-04-17T06:05:48.640 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.637+0000 7fc4b6310640 1 -- 172.21.15.47:0/2008051637 learned_addr learned my addr 172.21.15.47:0/2008051637 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:05:48.640 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.638+0000 7fc4b6b11640 1 --2- 172.21.15.47:0/2008051637 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56400fb8c7b0 0x56400fb86bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:05:48.640 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.638+0000 7fc4b6310640 1 -- 172.21.15.47:0/2008051637 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56400fb8c7b0 msgr2=0x56400fb86bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:05:48.640 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.638+0000 7fc4b6310640 1 --2- 172.21.15.47:0/2008051637 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56400fb8c7b0 0x56400fb86bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:48.640 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.638+0000 7fc4b6310640 1 -- 172.21.15.47:0/2008051637 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56400fb839e0 msgr2=0x56400fb887f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:05:48.640 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.638+0000 7fc4b6310640 1 --2- 172.21.15.47:0/2008051637 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56400fb839e0 0x56400fb887f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:48.641 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.638+0000 7fc4b6310640 1 -- 172.21.15.47:0/2008051637 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56400fb87e90 con 0x56400fb830c0 2024-04-17T06:05:48.641 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.638+0000 7fc4b6b11640 1 --2- 172.21.15.47:0/2008051637 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56400fb8c7b0 0x56400fb86bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:05:48.641 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.638+0000 7fc4b6310640 1 --2- 172.21.15.47:0/2008051637 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56400fb830c0 0x56400fb8c030 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac099340 tx=0x7fc4ac000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6178662c4917b8f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:05:48.641 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.638+0000 7fc49bfff640 1 -- 172.21.15.47:0/2008051637 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4ac09c2d0 con 0x56400fb830c0 2024-04-17T06:05:48.641 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.638+0000 7fc49bfff640 1 -- 172.21.15.47:0/2008051637 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4ac01f300 con 0x56400fb830c0 2024-04-17T06:05:48.641 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.638+0000 7fc49bfff640 1 -- 172.21.15.47:0/2008051637 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4ac098c10 con 0x56400fb830c0 2024-04-17T06:05:48.641 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.639+0000 7fc4bb235740 1 -- 172.21.15.47:0/2008051637 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56400fb830c0 msgr2=0x56400fb8c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:05:48.641 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.639+0000 7fc4bb235740 1 --2- 172.21.15.47:0/2008051637 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56400fb830c0 0x56400fb8c030 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac099340 tx=0x7fc4ac000bc0 comp rx=0 tx=0).stop 2024-04-17T06:05:48.641 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.639+0000 7fc4bb235740 1 -- 172.21.15.47:0/2008051637 shutdown_connections 2024-04-17T06:05:48.641 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.639+0000 7fc4bb235740 1 --2- 172.21.15.47:0/2008051637 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56400fb8c7b0 0x56400fb86bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:48.641 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.639+0000 7fc4bb235740 1 --2- 172.21.15.47:0/2008051637 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56400fb839e0 0x56400fb887f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:48.641 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.639+0000 7fc4bb235740 1 --2- 172.21.15.47:0/2008051637 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56400fb830c0 0x56400fb8c030 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:48.641 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.639+0000 7fc4bb235740 1 -- 172.21.15.47:0/2008051637 >> 172.21.15.47:0/2008051637 conn(0x56400fb7d9e0 msgr2=0x56400fb880e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:05:48.642 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.639+0000 7fc4bb235740 1 -- 172.21.15.47:0/2008051637 shutdown_connections 2024-04-17T06:05:48.642 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.639+0000 7fc4bb235740 1 -- 172.21.15.47:0/2008051637 wait complete. 2024-04-17T06:05:48.642 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.639+0000 7fc4bb235740 1 -- start start 2024-04-17T06:05:48.642 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.639+0000 7fc4bb235740 1 Processor -- start 2024-04-17T06:05:48.642 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.639+0000 7fc4bb235740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56400fb830c0 0x56400fb9f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:05:48.642 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.639+0000 7fc4bb235740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56400fb839e0 0x56400fb9f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:05:48.642 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.639+0000 7fc4bb235740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56400fb8c7b0 0x56400fc00c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:05:48.642 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.639+0000 7fc4bb235740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b01ae400 con 0x56400fb839e0 2024-04-17T06:05:48.642 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.639+0000 7fc4bb235740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b01af870 con 0x56400fb8c7b0 2024-04-17T06:05:48.642 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.639+0000 7fc4bb235740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b01aead0 con 0x56400fb830c0 2024-04-17T06:05:48.642 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.639+0000 7fc4bb235740 10 client.0 _ll_register_callbacks cb 0x7fc4a8037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:05:48.642 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.639+0000 7fc4b5b0f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56400fb839e0 0x56400fb9f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:05:48.642 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.639+0000 7fc4b5b0f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56400fb839e0 0x56400fb9f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:42516/0 (socket says 172.21.15.47:42516) 2024-04-17T06:05:48.643 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.639+0000 7fc4b5b0f640 1 -- 172.21.15.47:0/3395546170 learned_addr learned my addr 172.21.15.47:0/3395546170 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:05:48.643 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.639+0000 7fc4b6310640 1 --2- 172.21.15.47:0/3395546170 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56400fb830c0 0x56400fb9f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:05:48.643 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.640+0000 7fc4b5b0f640 1 -- 172.21.15.47:0/3395546170 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56400fb830c0 msgr2=0x56400fb9f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:05:48.643 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.640+0000 7fc4b5b0f640 1 --2- 172.21.15.47:0/3395546170 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56400fb830c0 0x56400fb9f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:48.643 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.640+0000 7fc4b5b0f640 1 -- 172.21.15.47:0/3395546170 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56400fb8c7b0 msgr2=0x56400fc00c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:05:48.643 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.640+0000 7fc4b6b11640 1 --2- 172.21.15.47:0/3395546170 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56400fb8c7b0 0x56400fc00c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:05:48.643 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.640+0000 7fc4b5b0f640 1 --2- 172.21.15.47:0/3395546170 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56400fb8c7b0 0x56400fc00c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:48.643 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.640+0000 7fc4b5b0f640 1 -- 172.21.15.47:0/3395546170 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56400fc011d0 con 0x56400fb839e0 2024-04-17T06:05:48.643 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.640+0000 7fc4b5b0f640 1 --2- 172.21.15.47:0/3395546170 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56400fb839e0 0x56400fb9f5e0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fc4a0005fd0 tx=0x7fc4a000def0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6482526d3ddb49d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:05:48.643 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.640+0000 7fc499ffb640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-17T06:05:48.643 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.640+0000 7fc499ffb640 1 -- 172.21.15.47:0/3395546170 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a0005020 con 0x56400fb839e0 2024-04-17T06:05:48.643 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.640+0000 7fc499ffb640 1 -- 172.21.15.47:0/3395546170 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7fc4a00051c0 con 0x56400fb839e0 2024-04-17T06:05:48.643 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.640+0000 7fc4bb235740 1 -- 172.21.15.47:0/3395546170 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4ac01f300 con 0x56400fb839e0 2024-04-17T06:05:48.643 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.640+0000 7fc499ffb640 1 -- 172.21.15.47:0/3395546170 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc4a002d050 con 0x56400fb839e0 2024-04-17T06:05:48.644 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.640+0000 7fc4bb235740 10 client.15783 fetch_fsmap learned FSMap version 28 2024-04-17T06:05:48.644 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.640+0000 7fc4bb235740 1 -- 172.21.15.47:0/3395546170 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56400fc024e0 con 0x56400fb839e0 2024-04-17T06:05:48.644 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.640+0000 7fc499ffb640 1 -- 172.21.15.47:0/3395546170 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4ac01f300 con 0x56400fb839e0 2024-04-17T06:05:48.644 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.641+0000 7fc499ffb640 1 -- 172.21.15.47:0/3395546170 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 28) ==== 8339+0+0 (secure 0 0 0) 0x7fc4a001e350 con 0x56400fb839e0 2024-04-17T06:05:48.644 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.641+0000 7fc4bb235740 10 client.15783 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T06:05:48.644 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.641+0000 7fc4bb235740 10 client.15783 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-17T06:05:48.644 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.641+0000 7fc4bb235740 20 client.15783 populate_metadata read hostname 'smithi047' 2024-04-17T06:05:48.644 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.641+0000 7fc4bb235740 1 --2- 172.21.15.47:0/3395546170 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x56400fc02ce0 0x56400fc05100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:05:48.644 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.641+0000 7fc4bb235740 4 client.15783 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-17T06:05:48.644 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.641+0000 7fc4bb235740 1 -- 172.21.15.47:0/3395546170 --> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56400faf0060 con 0x56400fc02ce0 2024-04-17T06:05:48.644 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.641+0000 7fc4b6310640 1 --2- 172.21.15.47:0/3395546170 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x56400fc02ce0 0x56400fc05100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:05:48.644 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.641+0000 7fc499ffb640 10 client.15783 ms_handle_connect on v2:172.21.15.98:6838/2841287917 2024-04-17T06:05:48.644 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.641+0000 7fc4b6310640 1 --2- 172.21.15.47:0/3395546170 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x56400fc02ce0 0x56400fc05100 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd81fbe4cb7cd55e server_cookie=82a75ec6e9e34a70 in_seq=0 out_seq=0 2024-04-17T06:05:48.647 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.649+0000 7fc499ffb640 1 -- 172.21.15.47:0/3395546170 <== mds.0 v2:172.21.15.98:6838/2841287917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56400faf0060 con 0x56400fc02ce0 2024-04-17T06:05:48.648 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.649+0000 7fc499ffb640 10 client.15783 handle_command_reply: tid=0 2024-04-17T06:05:48.648 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.649+0000 7fc4bb235740 1 client.15783 shutdown 2024-04-17T06:05:48.648 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.649+0000 7fc4bb235740 1 -- 172.21.15.47:0/3395546170 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56400fb839e0 msgr2=0x56400fb9f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:05:48.648 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.649+0000 7fc4bb235740 1 --2- 172.21.15.47:0/3395546170 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56400fb839e0 0x56400fb9f5e0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fc4a0005fd0 tx=0x7fc4a000def0 comp rx=0 tx=0).stop 2024-04-17T06:05:48.648 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.649+0000 7fc4bb235740 1 -- 172.21.15.47:0/3395546170 shutdown_connections 2024-04-17T06:05:48.648 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.649+0000 7fc4bb235740 1 --2- 172.21.15.47:0/3395546170 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x56400fc02ce0 0x56400fc05100 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:48.648 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.649+0000 7fc4bb235740 1 --2- 172.21.15.47:0/3395546170 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56400fb8c7b0 0x56400fc00c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:48.648 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.649+0000 7fc4bb235740 1 --2- 172.21.15.47:0/3395546170 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56400fb839e0 0x56400fb9f5e0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:48.648 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.649+0000 7fc4bb235740 1 --2- 172.21.15.47:0/3395546170 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56400fb830c0 0x56400fb9f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:48.648 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.649+0000 7fc499ffb640 0 client.15783 ms_handle_reset on v2:172.21.15.98:6838/2841287917 2024-04-17T06:05:48.648 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.649+0000 7fc4bb235740 1 -- 172.21.15.47:0/3395546170 >> 172.21.15.47:0/3395546170 conn(0x56400fb7d9e0 msgr2=0x56400fb98040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:05:48.648 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.649+0000 7fc4bb235740 1 -- 172.21.15.47:0/3395546170 shutdown_connections 2024-04-17T06:05:48.648 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.649+0000 7fc4bb235740 1 -- 172.21.15.47:0/3395546170 wait complete. 2024-04-17T06:05:48.648 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.650+0000 7fc4bb235740 20 client.15783 trim_cache size 0 max 16384 2024-04-17T06:05:48.649 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-04-17T06:05:48.649 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-04-17T06:05:48.649 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-04-17T06:05:48.649 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-04-17T06:05:48.650 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.650+0000 7fc4b8574640 1 -- 172.21.15.47:0/119522826 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fc49c0a3df0 msgr2=0x7fc49c0a62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:05:48.650 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.650+0000 7fc4b8574640 1 --2- 172.21.15.47:0/119522826 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fc49c0a3df0 0x7fc49c0a62b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc4a8006010 tx=0x7fc4a8005ef0 comp rx=0 tx=0).stop 2024-04-17T06:05:48.650 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.650+0000 7fc4b8574640 1 -- 172.21.15.47:0/119522826 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4b01021d0 msgr2=0x7fc4b0074260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:05:48.650 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.650+0000 7fc4b8574640 1 --2- 172.21.15.47:0/119522826 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4b01021d0 0x7fc4b0074260 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac00c920 tx=0x7fc4ac00cde0 comp rx=0 tx=0).stop 2024-04-17T06:05:48.650 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.650+0000 7fc4b8574640 1 -- 172.21.15.47:0/119522826 shutdown_connections 2024-04-17T06:05:48.650 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.650+0000 7fc4b8574640 1 --2- 172.21.15.47:0/119522826 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc4b0074ce0 0x7fc4b01a6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:48.650 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.650+0000 7fc4b8574640 1 --2- 172.21.15.47:0/119522826 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc4b01072a0 0x7fc4b00747a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:48.650 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.650+0000 7fc4b8574640 1 --2- 172.21.15.47:0/119522826 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fc49c0a3df0 0x7fc49c0a62b0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:48.650 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.650+0000 7fc4b8574640 1 --2- 172.21.15.47:0/119522826 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4b01021d0 0x7fc4b0074260 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:05:48.650 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.650+0000 7fc4b8574640 1 -- 172.21.15.47:0/119522826 >> 172.21.15.47:0/119522826 conn(0x7fc4b00fda30 msgr2=0x7fc4b00fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:05:48.651 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.650+0000 7fc4b8574640 1 -- 172.21.15.47:0/119522826 shutdown_connections 2024-04-17T06:05:48.651 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:05:48.650+0000 7fc4b8574640 1 -- 172.21.15.47:0/119522826 wait complete. 2024-04-17T06:05:49.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:05:49 smithi098 ceph-mon[36516]: pgmap v140: 97 pgs: 97 active+clean; 2.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 10 KiB/s wr, 11 op/s 2024-04-17T06:05:49.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:05:49 smithi047 ceph-mon[32223]: pgmap v140: 97 pgs: 97 active+clean; 2.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 10 KiB/s wr, 11 op/s 2024-04-17T06:05:50.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:05:49 smithi103 ceph-mon[36243]: pgmap v140: 97 pgs: 97 active+clean; 2.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 10 KiB/s wr, 11 op/s 2024-04-17T06:05:51.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:05:51 smithi098 ceph-mon[36516]: pgmap v141: 97 pgs: 97 active+clean; 2.5 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 17 KiB/s wr, 18 op/s 2024-04-17T06:05:51.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:05:51 smithi047 ceph-mon[32223]: pgmap v141: 97 pgs: 97 active+clean; 2.5 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 17 KiB/s wr, 18 op/s 2024-04-17T06:05:52.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:05:51 smithi103 ceph-mon[36243]: pgmap v141: 97 pgs: 97 active+clean; 2.5 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 17 KiB/s wr, 18 op/s 2024-04-17T06:05:53.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:05:53 smithi098 ceph-mon[36516]: pgmap v142: 97 pgs: 97 active+clean; 2.5 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 10 KiB/s wr, 11 op/s 2024-04-17T06:05:53.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:05:53 smithi047 ceph-mon[32223]: pgmap v142: 97 pgs: 97 active+clean; 2.5 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 10 KiB/s wr, 11 op/s 2024-04-17T06:05:54.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:05:53 smithi103 ceph-mon[36243]: pgmap v142: 97 pgs: 97 active+clean; 2.5 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 10 KiB/s wr, 11 op/s 2024-04-17T06:05:55.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:05:55 smithi098 ceph-mon[36516]: pgmap v143: 97 pgs: 97 active+clean; 2.5 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-04-17T06:05:55.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:05:55 smithi047 ceph-mon[32223]: pgmap v143: 97 pgs: 97 active+clean; 2.5 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-04-17T06:05:56.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:05:55 smithi103 ceph-mon[36243]: pgmap v143: 97 pgs: 97 active+clean; 2.5 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-04-17T06:05:57.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:05:57 smithi098 ceph-mon[36516]: pgmap v144: 97 pgs: 97 active+clean; 2.5 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 10 KiB/s wr, 11 op/s 2024-04-17T06:05:57.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:05:57 smithi047 ceph-mon[32223]: pgmap v144: 97 pgs: 97 active+clean; 2.5 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 10 KiB/s wr, 11 op/s 2024-04-17T06:05:58.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:05:57 smithi103 ceph-mon[36243]: pgmap v144: 97 pgs: 97 active+clean; 2.5 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 10 KiB/s wr, 11 op/s 2024-04-17T06:05:59.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:05:59 smithi098 ceph-mon[36516]: pgmap v145: 97 pgs: 97 active+clean; 2.5 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 10 KiB/s wr, 11 op/s 2024-04-17T06:05:59.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:05:59 smithi047 ceph-mon[32223]: pgmap v145: 97 pgs: 97 active+clean; 2.5 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 10 KiB/s wr, 11 op/s 2024-04-17T06:06:00.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:05:59 smithi103 ceph-mon[36243]: pgmap v145: 97 pgs: 97 active+clean; 2.5 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 10 KiB/s wr, 11 op/s 2024-04-17T06:06:01.364 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.364 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:06:01.364 INFO:tasks.workunit.client.0.smithi047.stdout:127/600: open fd = 3 2024-04-17T06:06:01.364 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.365 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.365 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:06:01.365 INFO:tasks.workunit.client.0.smithi047.stdout:128/600: open fd = 3 2024-04-17T06:06:01.365 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.365 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.365 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:06:01.365 INFO:tasks.workunit.client.0.smithi047.stdout:129/600: open fd = 3 2024-04-17T06:06:01.365 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.365 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.365 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:06:01.365 INFO:tasks.workunit.client.0.smithi047.stdout:130/600: open fd = 3 2024-04-17T06:06:01.365 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.365 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.365 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:06:01.366 INFO:tasks.workunit.client.0.smithi047.stdout:131/600: open fd = 3 2024-04-17T06:06:01.366 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.366 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.366 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:06:01.366 INFO:tasks.workunit.client.0.smithi047.stdout:132/600: open fd = 3 2024-04-17T06:06:01.366 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.366 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.366 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:06:01.366 INFO:tasks.workunit.client.0.smithi047.stdout:133/600: open fd = 3 2024-04-17T06:06:01.366 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.366 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.366 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:06:01.366 INFO:tasks.workunit.client.0.smithi047.stdout:134/600: open fd = 3 2024-04-17T06:06:01.366 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.367 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.367 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:06:01.367 INFO:tasks.workunit.client.0.smithi047.stdout:135/600: open fd = 3 2024-04-17T06:06:01.367 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.367 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.367 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:06:01.367 INFO:tasks.workunit.client.0.smithi047.stdout:136/600: open fd = 3 2024-04-17T06:06:01.367 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.367 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.367 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:06:01.367 INFO:tasks.workunit.client.0.smithi047.stdout:137/600: open fd = 3 2024-04-17T06:06:01.367 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.367 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.367 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:06:01.367 INFO:tasks.workunit.client.0.smithi047.stdout:138/600: open fd = 3 2024-04-17T06:06:01.368 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.368 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.368 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:06:01.368 INFO:tasks.workunit.client.0.smithi047.stdout:139/600: open fd = 3 2024-04-17T06:06:01.368 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.368 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.368 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:06:01.368 INFO:tasks.workunit.client.0.smithi047.stdout:140/600: open fd = 3 2024-04-17T06:06:01.368 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.368 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.368 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:06:01.368 INFO:tasks.workunit.client.0.smithi047.stdout:141/600: open fd = 3 2024-04-17T06:06:01.368 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.368 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.368 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:06:01.369 INFO:tasks.workunit.client.0.smithi047.stdout:142/600: open fd = 3 2024-04-17T06:06:01.369 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.369 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.369 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:06:01.369 INFO:tasks.workunit.client.0.smithi047.stdout:143/600: open fd = 3 2024-04-17T06:06:01.369 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.369 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.369 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:06:01.369 INFO:tasks.workunit.client.0.smithi047.stdout:144/600: open fd = 3 2024-04-17T06:06:01.369 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.369 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.369 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:06:01.369 INFO:tasks.workunit.client.0.smithi047.stdout:145/600: open fd = 3 2024-04-17T06:06:01.369 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.369 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.370 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:06:01.370 INFO:tasks.workunit.client.0.smithi047.stdout:146/600: open fd = 3 2024-04-17T06:06:01.370 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.370 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.370 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:06:01.370 INFO:tasks.workunit.client.0.smithi047.stdout:147/600: open fd = 3 2024-04-17T06:06:01.370 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.370 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.370 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:06:01.370 INFO:tasks.workunit.client.0.smithi047.stdout:148/600: open fd = 3 2024-04-17T06:06:01.370 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.370 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.370 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:06:01.370 INFO:tasks.workunit.client.0.smithi047.stdout:149/600: open fd = 3 2024-04-17T06:06:01.371 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.371 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.371 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:06:01.371 INFO:tasks.workunit.client.0.smithi047.stdout:150/600: open fd = 3 2024-04-17T06:06:01.371 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.371 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.371 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:06:01.371 INFO:tasks.workunit.client.0.smithi047.stdout:151/600: open fd = 3 2024-04-17T06:06:01.371 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.371 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.371 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:06:01.371 INFO:tasks.workunit.client.0.smithi047.stdout:152/600: open fd = 3 2024-04-17T06:06:01.371 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.371 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.371 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:06:01.372 INFO:tasks.workunit.client.0.smithi047.stdout:153/600: open fd = 3 2024-04-17T06:06:01.372 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.372 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.372 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:06:01.372 INFO:tasks.workunit.client.0.smithi047.stdout:154/600: open fd = 3 2024-04-17T06:06:01.372 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.372 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.372 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:06:01.372 INFO:tasks.workunit.client.0.smithi047.stdout:155/600: open fd = 3 2024-04-17T06:06:01.372 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.372 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.372 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:06:01.372 INFO:tasks.workunit.client.0.smithi047.stdout:156/600: open fd = 3 2024-04-17T06:06:01.372 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.373 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.373 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:06:01.373 INFO:tasks.workunit.client.0.smithi047.stdout:157/600: open fd = 3 2024-04-17T06:06:01.373 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.373 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.373 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:06:01.373 INFO:tasks.workunit.client.0.smithi047.stdout:158/600: open fd = 3 2024-04-17T06:06:01.373 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.373 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.373 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:06:01.373 INFO:tasks.workunit.client.0.smithi047.stdout:159/600: open fd = 3 2024-04-17T06:06:01.373 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.373 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.373 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:06:01.373 INFO:tasks.workunit.client.0.smithi047.stdout:160/600: open fd = 3 2024-04-17T06:06:01.373 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.374 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.374 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:06:01.374 INFO:tasks.workunit.client.0.smithi047.stdout:161/600: open fd = 3 2024-04-17T06:06:01.374 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.374 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.374 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:06:01.374 INFO:tasks.workunit.client.0.smithi047.stdout:162/600: open fd = 3 2024-04-17T06:06:01.374 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.374 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.374 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:06:01.374 INFO:tasks.workunit.client.0.smithi047.stdout:163/600: open fd = 3 2024-04-17T06:06:01.374 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.374 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.374 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:06:01.375 INFO:tasks.workunit.client.0.smithi047.stdout:164/600: open fd = 3 2024-04-17T06:06:01.375 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.375 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.375 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:06:01.375 INFO:tasks.workunit.client.0.smithi047.stdout:165/600: open fd = 3 2024-04-17T06:06:01.375 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.375 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.375 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:06:01.375 INFO:tasks.workunit.client.0.smithi047.stdout:166/600: open fd = 3 2024-04-17T06:06:01.375 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.375 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.375 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:06:01.375 INFO:tasks.workunit.client.0.smithi047.stdout:167/600: open fd = 3 2024-04-17T06:06:01.375 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.376 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.376 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:06:01.376 INFO:tasks.workunit.client.0.smithi047.stdout:168/600: open fd = 3 2024-04-17T06:06:01.376 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.376 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.376 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:06:01.376 INFO:tasks.workunit.client.0.smithi047.stdout:169/600: open fd = 3 2024-04-17T06:06:01.376 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.376 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.376 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:06:01.376 INFO:tasks.workunit.client.0.smithi047.stdout:170/600: open fd = 3 2024-04-17T06:06:01.376 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.376 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.376 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:06:01.376 INFO:tasks.workunit.client.0.smithi047.stdout:171/600: open fd = 3 2024-04-17T06:06:01.377 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.377 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.377 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:06:01.377 INFO:tasks.workunit.client.0.smithi047.stdout:172/600: open fd = 3 2024-04-17T06:06:01.377 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.377 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.377 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:06:01.377 INFO:tasks.workunit.client.0.smithi047.stdout:173/600: open fd = 3 2024-04-17T06:06:01.377 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.377 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.377 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:06:01.377 INFO:tasks.workunit.client.0.smithi047.stdout:174/600: open fd = 3 2024-04-17T06:06:01.377 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.377 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.378 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:06:01.378 INFO:tasks.workunit.client.0.smithi047.stdout:175/600: open fd = 3 2024-04-17T06:06:01.378 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.378 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.378 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:06:01.378 INFO:tasks.workunit.client.0.smithi047.stdout:176/600: open fd = 3 2024-04-17T06:06:01.378 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.378 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.378 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:06:01.378 INFO:tasks.workunit.client.0.smithi047.stdout:177/600: open fd = 3 2024-04-17T06:06:01.378 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.378 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.378 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:06:01.378 INFO:tasks.workunit.client.0.smithi047.stdout:178/600: open fd = 3 2024-04-17T06:06:01.379 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.379 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.379 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:06:01.379 INFO:tasks.workunit.client.0.smithi047.stdout:179/600: open fd = 3 2024-04-17T06:06:01.379 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.379 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.379 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:06:01.379 INFO:tasks.workunit.client.0.smithi047.stdout:180/600: open fd = 3 2024-04-17T06:06:01.379 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.379 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.379 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:06:01.379 INFO:tasks.workunit.client.0.smithi047.stdout:181/600: open fd = 3 2024-04-17T06:06:01.379 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.379 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.379 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:06:01.380 INFO:tasks.workunit.client.0.smithi047.stdout:182/600: open fd = 3 2024-04-17T06:06:01.380 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.380 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.380 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:06:01.380 INFO:tasks.workunit.client.0.smithi047.stdout:183/600: open fd = 3 2024-04-17T06:06:01.380 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.380 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.380 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:06:01.380 INFO:tasks.workunit.client.0.smithi047.stdout:184/600: open fd = 3 2024-04-17T06:06:01.380 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.380 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.380 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:06:01.380 INFO:tasks.workunit.client.0.smithi047.stdout:185/600: open fd = 3 2024-04-17T06:06:01.380 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.380 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.381 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:06:01.381 INFO:tasks.workunit.client.0.smithi047.stdout:186/600: open fd = 3 2024-04-17T06:06:01.381 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.381 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.381 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:06:01.381 INFO:tasks.workunit.client.0.smithi047.stdout:187/600: open fd = 3 2024-04-17T06:06:01.381 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.381 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.381 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:06:01.381 INFO:tasks.workunit.client.0.smithi047.stdout:188/600: open fd = 3 2024-04-17T06:06:01.381 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.381 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:06:01.751 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.750+0000 7fab3a952640 1 -- 172.21.15.47:0/182353443 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab341021b0 msgr2=0x7fab34105cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:06:01.751 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.750+0000 7fab3a952640 1 --2- 172.21.15.47:0/182353443 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab341021b0 0x7fab34105cd0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fab28009900 tx=0x7fab2802bc30 comp rx=0 tx=0).stop 2024-04-17T06:06:01.751 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.750+0000 7fab3a952640 1 -- 172.21.15.47:0/182353443 shutdown_connections 2024-04-17T06:06:01.751 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.750+0000 7fab3a952640 1 --2- 172.21.15.47:0/182353443 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fab341063b0 0x7fab3410df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:01.751 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.750+0000 7fab3a952640 1 --2- 172.21.15.47:0/182353443 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab341021b0 0x7fab34105cd0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:01.751 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.750+0000 7fab3a952640 1 --2- 172.21.15.47:0/182353443 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fab34101800 0x7fab34101be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:01.751 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.750+0000 7fab3a952640 1 -- 172.21.15.47:0/182353443 >> 172.21.15.47:0/182353443 conn(0x7fab34077380 msgr2=0x7fab340fe770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:06:01.751 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.750+0000 7fab3a952640 1 -- 172.21.15.47:0/182353443 shutdown_connections 2024-04-17T06:06:01.751 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.750+0000 7fab3a952640 1 -- 172.21.15.47:0/182353443 wait complete. 2024-04-17T06:06:01.752 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.751+0000 7fab3a952640 1 Processor -- start 2024-04-17T06:06:01.752 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.751+0000 7fab3a952640 1 -- start start 2024-04-17T06:06:01.752 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.751+0000 7fab3a952640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fab34101800 0x7fab341a0860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:06:01.752 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.751+0000 7fab3a952640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fab341063b0 0x7fab341a0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:06:01.752 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.751+0000 7fab3a952640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab3419aa10 0x7fab3419aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:06:01.752 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.751+0000 7fab3a952640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fab34111440 con 0x7fab3419aa10 2024-04-17T06:06:01.752 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.751+0000 7fab3a952640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fab341112c0 con 0x7fab34101800 2024-04-17T06:06:01.752 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.751+0000 7fab3a952640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fab341115c0 con 0x7fab341063b0 2024-04-17T06:06:01.752 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.752+0000 7fab38eef640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab3419aa10 0x7fab3419aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:06:01.752 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.752+0000 7fab38eef640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab3419aa10 0x7fab3419aec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43866/0 (socket says 172.21.15.47:43866) 2024-04-17T06:06:01.752 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.752+0000 7fab38eef640 1 -- 172.21.15.47:0/3163058983 learned_addr learned my addr 172.21.15.47:0/3163058983 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:06:01.752 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.752+0000 7fab337fe640 1 --2- 172.21.15.47:0/3163058983 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fab341063b0 0x7fab341a0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:06:01.752 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.752+0000 7fab33fff640 1 --2- 172.21.15.47:0/3163058983 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fab34101800 0x7fab341a0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:06:01.752 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.752+0000 7fab38eef640 1 -- 172.21.15.47:0/3163058983 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fab341063b0 msgr2=0x7fab341a0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:06:01.753 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.752+0000 7fab38eef640 1 --2- 172.21.15.47:0/3163058983 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fab341063b0 0x7fab341a0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:01.753 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.752+0000 7fab38eef640 1 -- 172.21.15.47:0/3163058983 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fab34101800 msgr2=0x7fab341a0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:06:01.753 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.752+0000 7fab38eef640 1 --2- 172.21.15.47:0/3163058983 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fab34101800 0x7fab341a0860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:01.753 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.752+0000 7fab38eef640 1 -- 172.21.15.47:0/3163058983 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab3419b530 con 0x7fab3419aa10 2024-04-17T06:06:01.753 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.752+0000 7fab33fff640 1 --2- 172.21.15.47:0/3163058983 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fab34101800 0x7fab341a0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:06:01.753 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.752+0000 7fab38eef640 1 --2- 172.21.15.47:0/3163058983 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab3419aa10 0x7fab3419aec0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fab2c00eaa0 tx=0x7fab2c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:06:01.753 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.752+0000 7fab317fa640 1 -- 172.21.15.47:0/3163058983 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab2c00cdb0 con 0x7fab3419aa10 2024-04-17T06:06:01.753 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.752+0000 7fab317fa640 1 -- 172.21.15.47:0/3163058983 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab2c005020 con 0x7fab3419aa10 2024-04-17T06:06:01.753 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.752+0000 7fab3a952640 1 -- 172.21.15.47:0/3163058983 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab3419b820 con 0x7fab3419aa10 2024-04-17T06:06:01.753 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.752+0000 7fab317fa640 1 -- 172.21.15.47:0/3163058983 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab2c0116b0 con 0x7fab3419aa10 2024-04-17T06:06:01.753 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.752+0000 7fab3a952640 1 -- 172.21.15.47:0/3163058983 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab341a7a70 con 0x7fab3419aa10 2024-04-17T06:06:01.759 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.755+0000 7fab317fa640 1 -- 172.21.15.47:0/3163058983 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fab2c005560 con 0x7fab3419aa10 2024-04-17T06:06:01.759 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.755+0000 7fab3d61d740 1 Processor -- start 2024-04-17T06:06:01.759 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.755+0000 7fab3d61d740 1 -- start start 2024-04-17T06:06:01.759 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.756+0000 7fab3d61d740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557ecc1b60c0 0x557ecc1b64a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:06:01.759 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.756+0000 7fab3d61d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ecc1b69e0 0x557ecc1bf270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:06:01.759 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.756+0000 7fab3d61d740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557ecc1bf7b0 0x557ecc1c1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:06:01.759 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.756+0000 7fab3d61d740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x557ecc11b7d0 con 0x557ecc1b60c0 2024-04-17T06:06:01.759 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.756+0000 7fab3d61d740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x557ecc0b8f70 con 0x557ecc1bf7b0 2024-04-17T06:06:01.760 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.756+0000 7fab3d61d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x557ecc117790 con 0x557ecc1b69e0 2024-04-17T06:06:01.760 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.756+0000 7fab337fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557ecc1b60c0 0x557ecc1b64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:06:01.760 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.756+0000 7fab337fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557ecc1b60c0 0x557ecc1b64a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43878/0 (socket says 172.21.15.47:43878) 2024-04-17T06:06:01.760 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.756+0000 7fab337fe640 1 -- 172.21.15.47:0/3386823841 learned_addr learned my addr 172.21.15.47:0/3386823841 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:06:01.760 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.756+0000 7fab33fff640 1 --2- 172.21.15.47:0/3386823841 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ecc1b69e0 0x557ecc1bf270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:06:01.760 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.756+0000 7fab317fa640 1 --2- 172.21.15.47:0/3163058983 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fab1c0a3dc0 0x7fab1c0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:06:01.760 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.756+0000 7fab337fe640 1 --2- 172.21.15.47:0/3386823841 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557ecc1bf7b0 0x557ecc1c1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:06:01.760 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.756+0000 7fab337fe640 1 -- 172.21.15.47:0/3386823841 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ecc1b69e0 msgr2=0x557ecc1bf270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:06:01.760 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.756+0000 7fab337fe640 1 --2- 172.21.15.47:0/3386823841 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ecc1b69e0 0x557ecc1bf270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:01.760 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.756+0000 7fab337fe640 1 -- 172.21.15.47:0/3386823841 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557ecc1bf7b0 msgr2=0x557ecc1c1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:06:01.760 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.756+0000 7fab337fe640 1 --2- 172.21.15.47:0/3386823841 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557ecc1bf7b0 0x557ecc1c1ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:01.760 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.756+0000 7fab337fe640 1 -- 172.21.15.47:0/3386823841 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ecbe2e210 con 0x557ecc1b60c0 2024-04-17T06:06:01.760 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.756+0000 7fab317fa640 1 -- 172.21.15.47:0/3163058983 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7fab2c0d0980 con 0x7fab3419aa10 2024-04-17T06:06:01.761 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.756+0000 7fab38eef640 1 --2- 172.21.15.47:0/3163058983 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fab1c0a3dc0 0x7fab1c0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:06:01.761 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.756+0000 7fab337fe640 1 --2- 172.21.15.47:0/3386823841 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557ecc1b60c0 0x557ecc1b64a0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fab2802cfd0 tx=0x7fab28005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eecc980438f1c43d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:06:01.761 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.757+0000 7fab1a7fc640 1 -- 172.21.15.47:0/3386823841 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab28049070 con 0x557ecc1b60c0 2024-04-17T06:06:01.761 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.757+0000 7fab1a7fc640 1 -- 172.21.15.47:0/3386823841 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab2802cc20 con 0x557ecc1b60c0 2024-04-17T06:06:01.761 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.757+0000 7fab38eef640 1 --2- 172.21.15.47:0/3163058983 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fab1c0a3dc0 0x7fab1c0a6280 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fab2c099320 tx=0x7fab2c087040 comp rx=0 tx=0).ready entity=mgr.24923 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:06:01.761 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.757+0000 7fab1a7fc640 1 -- 172.21.15.47:0/3386823841 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab28005120 con 0x557ecc1b60c0 2024-04-17T06:06:01.761 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.757+0000 7fab3d61d740 1 -- 172.21.15.47:0/3386823841 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557ecc1b60c0 msgr2=0x557ecc1b64a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:06:01.761 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.757+0000 7fab3d61d740 1 --2- 172.21.15.47:0/3386823841 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557ecc1b60c0 0x557ecc1b64a0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fab2802cfd0 tx=0x7fab28005dd0 comp rx=0 tx=0).stop 2024-04-17T06:06:01.761 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.757+0000 7fab3d61d740 1 -- 172.21.15.47:0/3386823841 shutdown_connections 2024-04-17T06:06:01.761 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.757+0000 7fab3d61d740 1 --2- 172.21.15.47:0/3386823841 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557ecc1bf7b0 0x557ecc1c1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:01.761 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.757+0000 7fab3d61d740 1 --2- 172.21.15.47:0/3386823841 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ecc1b69e0 0x557ecc1bf270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:01.761 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.757+0000 7fab3d61d740 1 --2- 172.21.15.47:0/3386823841 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557ecc1b60c0 0x557ecc1b64a0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:01.761 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.757+0000 7fab3d61d740 1 -- 172.21.15.47:0/3386823841 >> 172.21.15.47:0/3386823841 conn(0x557ecc1b09e0 msgr2=0x557ecc1b0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:06:01.761 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.757+0000 7fab3d61d740 1 -- 172.21.15.47:0/3386823841 shutdown_connections 2024-04-17T06:06:01.762 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.757+0000 7fab3d61d740 1 -- 172.21.15.47:0/3386823841 wait complete. 2024-04-17T06:06:01.762 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.758+0000 7fab3d61d740 1 -- start start 2024-04-17T06:06:01.762 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.758+0000 7fab3d61d740 1 Processor -- start 2024-04-17T06:06:01.762 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.758+0000 7fab3d61d740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557ecc1b60c0 0x557ecc1bc880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:06:01.762 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.758+0000 7fab3d61d740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557ecc1b69e0 0x557ecc1bcdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:06:01.762 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.758+0000 7fab3d61d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ecc1bf7b0 0x557ecc1bd300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:06:01.762 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.758+0000 7fab3d61d740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x557ecc1c2380 con 0x557ecc1b60c0 2024-04-17T06:06:01.762 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.758+0000 7fab3d61d740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x557ecc0a3e90 con 0x557ecc1b69e0 2024-04-17T06:06:01.762 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.758+0000 7fab3d61d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x557ecc1c2500 con 0x557ecc1bf7b0 2024-04-17T06:06:01.762 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.758+0000 7fab3d61d740 10 client.0 _ll_register_callbacks cb 0x557ecc0b9d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:06:01.762 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.758+0000 7fab337fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557ecc1b60c0 0x557ecc1bc880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:06:01.762 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.759+0000 7fab337fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557ecc1b60c0 0x557ecc1bc880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43886/0 (socket says 172.21.15.47:43886) 2024-04-17T06:06:01.762 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.759+0000 7fab337fe640 1 -- 172.21.15.47:0/3380576699 learned_addr learned my addr 172.21.15.47:0/3380576699 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:06:01.763 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.759+0000 7fab33fff640 1 --2- 172.21.15.47:0/3380576699 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557ecc1b69e0 0x557ecc1bcdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:06:01.763 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.759+0000 7fab337fe640 1 --2- 172.21.15.47:0/3380576699 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ecc1bf7b0 0x557ecc1bd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:06:01.763 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.759+0000 7fab337fe640 1 -- 172.21.15.47:0/3380576699 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ecc1bf7b0 msgr2=0x557ecc1bd300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:06:01.763 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.759+0000 7fab337fe640 1 --2- 172.21.15.47:0/3380576699 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ecc1bf7b0 0x557ecc1bd300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:01.763 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.759+0000 7fab337fe640 1 -- 172.21.15.47:0/3380576699 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557ecc1b69e0 msgr2=0x557ecc1bcdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:06:01.763 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.759+0000 7fab337fe640 1 --2- 172.21.15.47:0/3380576699 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557ecc1b69e0 0x557ecc1bcdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:01.763 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.759+0000 7fab337fe640 1 -- 172.21.15.47:0/3380576699 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ecbefa620 con 0x557ecc1b60c0 2024-04-17T06:06:01.763 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.759+0000 7fab337fe640 1 --2- 172.21.15.47:0/3380576699 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557ecc1b60c0 0x557ecc1bc880 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fab28004bc0 tx=0x7fab2802c5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1133fa4e921cd97d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:06:01.763 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.759+0000 7fab137fe640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-17T06:06:01.763 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.759+0000 7fab137fe640 1 -- 172.21.15.47:0/3380576699 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab28049040 con 0x557ecc1b60c0 2024-04-17T06:06:01.763 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.759+0000 7fab137fe640 1 -- 172.21.15.47:0/3380576699 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7fab28004440 con 0x557ecc1b60c0 2024-04-17T06:06:01.763 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.759+0000 7fab3d61d740 1 -- 172.21.15.47:0/3380576699 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab2802cc20 con 0x557ecc1b60c0 2024-04-17T06:06:01.763 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.760+0000 7fab137fe640 1 -- 172.21.15.47:0/3380576699 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fab28004e30 con 0x557ecc1b60c0 2024-04-17T06:06:01.763 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.760+0000 7fab3d61d740 10 client.15795 fetch_fsmap learned FSMap version 28 2024-04-17T06:06:01.763 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.760+0000 7fab3d61d740 1 -- 172.21.15.47:0/3380576699 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557ecc1ce670 con 0x557ecc1b60c0 2024-04-17T06:06:01.764 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.760+0000 7fab137fe640 1 -- 172.21.15.47:0/3380576699 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab28045030 con 0x557ecc1b60c0 2024-04-17T06:06:01.764 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.760+0000 7fab137fe640 1 -- 172.21.15.47:0/3380576699 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 28) ==== 8339+0+0 (secure 0 0 0) 0x7fab2804c070 con 0x557ecc1b60c0 2024-04-17T06:06:01.764 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.760+0000 7fab3d61d740 10 client.15795 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T06:06:01.764 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.760+0000 7fab3d61d740 10 client.15795 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-17T06:06:01.764 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.760+0000 7fab3d61d740 20 client.15795 populate_metadata read hostname 'smithi047' 2024-04-17T06:06:01.764 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.760+0000 7fab3d61d740 1 --2- 172.21.15.47:0/3380576699 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x557ecc1cf9c0 0x557ecc1d1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:06:01.764 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.760+0000 7fab3d61d740 4 client.15795 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T06:06:01.764 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.760+0000 7fab3d61d740 1 -- 172.21.15.47:0/3380576699 --> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fab2802cc20 con 0x557ecc1cf9c0 2024-04-17T06:06:01.764 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.760+0000 7fab33fff640 1 --2- 172.21.15.47:0/3380576699 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x557ecc1cf9c0 0x557ecc1d1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:06:01.764 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.761+0000 7fab137fe640 10 client.15795 ms_handle_connect on v2:172.21.15.98:6838/2841287917 2024-04-17T06:06:01.764 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.761+0000 7fab33fff640 1 --2- 172.21.15.47:0/3380576699 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x557ecc1cf9c0 0x557ecc1d1db0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ded34de28a09b2c server_cookie=58ae14073693023b in_seq=0 out_seq=0 2024-04-17T06:06:01.764 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.762+0000 7fab137fe640 1 -- 172.21.15.47:0/3380576699 <== mds.0 v2:172.21.15.98:6838/2841287917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7fab2802cc20 con 0x557ecc1cf9c0 2024-04-17T06:06:01.764 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.762+0000 7fab137fe640 10 client.15795 handle_command_reply: tid=0 2024-04-17T06:06:01.764 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.762+0000 7fab3d61d740 1 client.15795 shutdown 2024-04-17T06:06:01.765 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.762+0000 7fab3d61d740 1 -- 172.21.15.47:0/3380576699 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557ecc1b60c0 msgr2=0x557ecc1bc880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:06:01.765 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.762+0000 7fab3d61d740 1 --2- 172.21.15.47:0/3380576699 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557ecc1b60c0 0x557ecc1bc880 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fab28004bc0 tx=0x7fab2802c5d0 comp rx=0 tx=0).stop 2024-04-17T06:06:01.766 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.762+0000 7fab3d61d740 1 -- 172.21.15.47:0/3380576699 shutdown_connections 2024-04-17T06:06:01.766 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.762+0000 7fab3d61d740 1 --2- 172.21.15.47:0/3380576699 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x557ecc1cf9c0 0x557ecc1d1db0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:01.766 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.762+0000 7fab3d61d740 1 --2- 172.21.15.47:0/3380576699 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ecc1bf7b0 0x557ecc1bd300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:01.766 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.762+0000 7fab3d61d740 1 --2- 172.21.15.47:0/3380576699 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557ecc1b69e0 0x557ecc1bcdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:01.766 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.762+0000 7fab3d61d740 1 --2- 172.21.15.47:0/3380576699 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557ecc1b60c0 0x557ecc1bc880 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:01.766 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.762+0000 7fab137fe640 0 client.15795 ms_handle_reset on v2:172.21.15.98:6838/2841287917 2024-04-17T06:06:01.766 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.762+0000 7fab3d61d740 1 -- 172.21.15.47:0/3380576699 >> 172.21.15.47:0/3380576699 conn(0x557ecc1b09e0 msgr2=0x557ecc1b2730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:06:01.766 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.762+0000 7fab3d61d740 1 -- 172.21.15.47:0/3380576699 shutdown_connections 2024-04-17T06:06:01.766 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.762+0000 7fab3d61d740 1 -- 172.21.15.47:0/3380576699 wait complete. 2024-04-17T06:06:01.766 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.762+0000 7fab3d61d740 20 client.15795 trim_cache size 0 max 16384 2024-04-17T06:06:01.779 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.777+0000 7fab3d61d740 1 Processor -- start 2024-04-17T06:06:01.779 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.777+0000 7fab3d61d740 1 -- start start 2024-04-17T06:06:01.780 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.777+0000 7fab3d61d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ecc1b60c0 0x557ecc1befb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:06:01.780 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.777+0000 7fab3d61d740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557ecc1b69e0 0x557ecc1cb240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:06:01.780 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.777+0000 7fab3d61d740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557ecc1bf7b0 0x557ecc1d73b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:06:01.780 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.777+0000 7fab3d61d740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fab341b25b0 con 0x557ecc1b69e0 2024-04-17T06:06:01.780 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.777+0000 7fab3d61d740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fab341afe70 con 0x557ecc1bf7b0 2024-04-17T06:06:01.780 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.777+0000 7fab3d61d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fab341adce0 con 0x557ecc1b60c0 2024-04-17T06:06:01.780 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.778+0000 7fab33fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557ecc1b69e0 0x557ecc1cb240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:06:01.780 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.778+0000 7fab33fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557ecc1b69e0 0x557ecc1cb240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43902/0 (socket says 172.21.15.47:43902) 2024-04-17T06:06:01.780 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.778+0000 7fab33fff640 1 -- 172.21.15.47:0/3406125744 learned_addr learned my addr 172.21.15.47:0/3406125744 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:06:01.780 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.778+0000 7fab337fe640 1 --2- 172.21.15.47:0/3406125744 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ecc1b60c0 0x557ecc1befb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:06:01.780 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.778+0000 7fab33fff640 1 -- 172.21.15.47:0/3406125744 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ecc1b60c0 msgr2=0x557ecc1befb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:06:01.780 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.778+0000 7fab33fff640 1 --2- 172.21.15.47:0/3406125744 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ecc1b60c0 0x557ecc1befb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:01.780 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.778+0000 7fab33fff640 1 -- 172.21.15.47:0/3406125744 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557ecc1bf7b0 msgr2=0x557ecc1d73b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:06:01.780 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.778+0000 7fab33fff640 1 --2- 172.21.15.47:0/3406125744 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557ecc1bf7b0 0x557ecc1d73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:01.781 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.778+0000 7fab33fff640 1 -- 172.21.15.47:0/3406125744 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ecc1ac2a0 con 0x557ecc1b69e0 2024-04-17T06:06:01.781 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.778+0000 7fab337fe640 1 --2- 172.21.15.47:0/3406125744 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ecc1b60c0 0x557ecc1befb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:06:01.781 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.778+0000 7fab33fff640 1 --2- 172.21.15.47:0/3406125744 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557ecc1b69e0 0x557ecc1cb240 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x557ecc1bc0e0 tx=0x7fab24005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83ad42f3395a05da server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:06:01.781 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.778+0000 7fab19ffb640 1 -- 172.21.15.47:0/3406125744 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab24025020 con 0x557ecc1b69e0 2024-04-17T06:06:01.781 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.778+0000 7fab19ffb640 1 -- 172.21.15.47:0/3406125744 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab24003de0 con 0x557ecc1b69e0 2024-04-17T06:06:01.781 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.778+0000 7fab19ffb640 1 -- 172.21.15.47:0/3406125744 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab24012a80 con 0x557ecc1b69e0 2024-04-17T06:06:01.781 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.779+0000 7fab3d61d740 1 -- 172.21.15.47:0/3406125744 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557ecc1b69e0 msgr2=0x557ecc1cb240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:06:01.781 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.779+0000 7fab3d61d740 1 --2- 172.21.15.47:0/3406125744 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557ecc1b69e0 0x557ecc1cb240 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x557ecc1bc0e0 tx=0x7fab24005f90 comp rx=0 tx=0).stop 2024-04-17T06:06:01.781 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.779+0000 7fab3d61d740 1 -- 172.21.15.47:0/3406125744 shutdown_connections 2024-04-17T06:06:01.781 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.779+0000 7fab3d61d740 1 --2- 172.21.15.47:0/3406125744 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557ecc1bf7b0 0x557ecc1d73b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:01.781 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.779+0000 7fab3d61d740 1 --2- 172.21.15.47:0/3406125744 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557ecc1b69e0 0x557ecc1cb240 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:01.781 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.779+0000 7fab3d61d740 1 --2- 172.21.15.47:0/3406125744 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ecc1b60c0 0x557ecc1befb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:01.781 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.779+0000 7fab3d61d740 1 -- 172.21.15.47:0/3406125744 >> 172.21.15.47:0/3406125744 conn(0x557ecc1b09e0 msgr2=0x557ecc1b2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:06:01.781 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.779+0000 7fab3d61d740 1 -- 172.21.15.47:0/3406125744 shutdown_connections 2024-04-17T06:06:01.782 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.779+0000 7fab3d61d740 1 -- 172.21.15.47:0/3406125744 wait complete. 2024-04-17T06:06:01.782 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.779+0000 7fab3d61d740 1 -- start start 2024-04-17T06:06:01.782 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.780+0000 7fab3d61d740 1 Processor -- start 2024-04-17T06:06:01.782 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.780+0000 7fab3d61d740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557ecc1b60c0 0x557ecc1c3890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:06:01.782 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.780+0000 7fab3d61d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ecc1b69e0 0x557ecc1c3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:06:01.782 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.780+0000 7fab3d61d740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557ecc1bf7b0 0x557ecc234c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:06:01.782 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.780+0000 7fab3d61d740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fab341ae330 con 0x557ecc1bf7b0 2024-04-17T06:06:01.782 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.780+0000 7fab3d61d740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fab341af7a0 con 0x557ecc1b60c0 2024-04-17T06:06:01.782 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.780+0000 7fab3d61d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fab341aea00 con 0x557ecc1b69e0 2024-04-17T06:06:01.782 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.780+0000 7fab3d61d740 10 client.0 _ll_register_callbacks cb 0x7fab28047070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:06:01.782 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.780+0000 7fab337fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557ecc1bf7b0 0x557ecc234c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:06:01.782 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.780+0000 7fab337fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557ecc1bf7b0 0x557ecc234c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43904/0 (socket says 172.21.15.47:43904) 2024-04-17T06:06:01.782 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.780+0000 7fab337fe640 1 -- 172.21.15.47:0/1451406988 learned_addr learned my addr 172.21.15.47:0/1451406988 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:06:01.782 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.780+0000 7fab33fff640 1 --2- 172.21.15.47:0/1451406988 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ecc1b69e0 0x557ecc1c3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:06:01.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.780+0000 7fab337fe640 1 --2- 172.21.15.47:0/1451406988 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557ecc1b60c0 0x557ecc1c3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:06:01.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.780+0000 7fab337fe640 1 -- 172.21.15.47:0/1451406988 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ecc1b69e0 msgr2=0x557ecc1c3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:06:01.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.780+0000 7fab337fe640 1 --2- 172.21.15.47:0/1451406988 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ecc1b69e0 0x557ecc1c3dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:01.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.780+0000 7fab337fe640 1 -- 172.21.15.47:0/1451406988 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557ecc1b60c0 msgr2=0x557ecc1c3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:06:01.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.780+0000 7fab337fe640 1 --2- 172.21.15.47:0/1451406988 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557ecc1b60c0 0x557ecc1c3890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:01.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.780+0000 7fab337fe640 1 -- 172.21.15.47:0/1451406988 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ecc235160 con 0x557ecc1bf7b0 2024-04-17T06:06:01.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.780+0000 7fab337fe640 1 --2- 172.21.15.47:0/1451406988 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557ecc1bf7b0 0x557ecc234c20 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fab2802c170 tx=0x7fab28004bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79611e4cc895ee02 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:06:01.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.780+0000 7fab137fe640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-17T06:06:01.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.781+0000 7fab137fe640 1 -- 172.21.15.47:0/1451406988 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab28060020 con 0x557ecc1bf7b0 2024-04-17T06:06:01.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.781+0000 7fab137fe640 1 -- 172.21.15.47:0/1451406988 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7fab28004240 con 0x557ecc1bf7b0 2024-04-17T06:06:01.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.781+0000 7fab3d61d740 1 -- 172.21.15.47:0/1451406988 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab24003de0 con 0x557ecc1bf7b0 2024-04-17T06:06:01.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.781+0000 7fab137fe640 1 -- 172.21.15.47:0/1451406988 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fab28049040 con 0x557ecc1bf7b0 2024-04-17T06:06:01.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.781+0000 7fab3d61d740 10 client.15801 fetch_fsmap learned FSMap version 28 2024-04-17T06:06:01.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.781+0000 7fab3d61d740 1 -- 172.21.15.47:0/1451406988 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557ecc2363e0 con 0x557ecc1bf7b0 2024-04-17T06:06:01.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.781+0000 7fab137fe640 1 -- 172.21.15.47:0/1451406988 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab24003de0 con 0x557ecc1bf7b0 2024-04-17T06:06:01.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.781+0000 7fab137fe640 1 -- 172.21.15.47:0/1451406988 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 28) ==== 8339+0+0 (secure 0 0 0) 0x7fab2804c040 con 0x557ecc1bf7b0 2024-04-17T06:06:01.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.781+0000 7fab3d61d740 10 client.15801 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T06:06:01.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.781+0000 7fab3d61d740 10 client.15801 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-17T06:06:01.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.781+0000 7fab3d61d740 20 client.15801 populate_metadata read hostname 'smithi047' 2024-04-17T06:06:01.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.781+0000 7fab3d61d740 1 --2- 172.21.15.47:0/1451406988 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x557ecc2369e0 0x557ecc238dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:06:01.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.781+0000 7fab3d61d740 4 client.15801 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-17T06:06:01.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.781+0000 7fab3d61d740 1 -- 172.21.15.47:0/1451406988 --> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557ecc1291f0 con 0x557ecc2369e0 2024-04-17T06:06:01.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.782+0000 7fab33fff640 1 --2- 172.21.15.47:0/1451406988 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x557ecc2369e0 0x557ecc238dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:06:01.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.782+0000 7fab33fff640 1 --2- 172.21.15.47:0/1451406988 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x557ecc2369e0 0x557ecc238dd0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=efe2d6966b66e07 server_cookie=46d1adb3c4805fa0 in_seq=0 out_seq=0 2024-04-17T06:06:01.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:01.782+0000 7fab137fe640 10 client.15801 ms_handle_connect on v2:172.21.15.98:6838/2841287917 2024-04-17T06:06:01.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:06:01 smithi098 ceph-mon[36516]: pgmap v146: 97 pgs: 97 active+clean; 2.5 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-04-17T06:06:01.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:06:01 smithi047 ceph-mon[32223]: pgmap v146: 97 pgs: 97 active+clean; 2.5 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-04-17T06:06:02.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:06:01 smithi103 ceph-mon[36243]: pgmap v146: 97 pgs: 97 active+clean; 2.5 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-04-17T06:06:03.531 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:03.532+0000 7fab137fe640 1 -- 172.21.15.47:0/1451406988 <== mds.0 v2:172.21.15.98:6838/2841287917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557ecc1291f0 con 0x557ecc2369e0 2024-04-17T06:06:03.531 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:03.532+0000 7fab137fe640 10 client.15801 handle_command_reply: tid=0 2024-04-17T06:06:03.531 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:03.532+0000 7fab3d61d740 1 client.15801 shutdown 2024-04-17T06:06:03.531 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:03.532+0000 7fab3d61d740 1 -- 172.21.15.47:0/1451406988 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557ecc1bf7b0 msgr2=0x557ecc234c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:06:03.531 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:03.532+0000 7fab3d61d740 1 --2- 172.21.15.47:0/1451406988 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557ecc1bf7b0 0x557ecc234c20 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fab2802c170 tx=0x7fab28004bf0 comp rx=0 tx=0).stop 2024-04-17T06:06:03.531 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:03.532+0000 7fab3d61d740 1 -- 172.21.15.47:0/1451406988 shutdown_connections 2024-04-17T06:06:03.531 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:03.532+0000 7fab3d61d740 1 --2- 172.21.15.47:0/1451406988 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x557ecc2369e0 0x557ecc238dd0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:03.531 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:03.532+0000 7fab3d61d740 1 --2- 172.21.15.47:0/1451406988 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557ecc1bf7b0 0x557ecc234c20 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:03.531 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:03.532+0000 7fab3d61d740 1 --2- 172.21.15.47:0/1451406988 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ecc1b69e0 0x557ecc1c3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:03.532 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:03.532+0000 7fab3d61d740 1 --2- 172.21.15.47:0/1451406988 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557ecc1b60c0 0x557ecc1c3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:03.532 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:03.532+0000 7fab137fe640 0 client.15801 ms_handle_reset on v2:172.21.15.98:6838/2841287917 2024-04-17T06:06:03.532 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:03.532+0000 7fab3d61d740 1 -- 172.21.15.47:0/1451406988 >> 172.21.15.47:0/1451406988 conn(0x557ecc1b09e0 msgr2=0x557ecc1b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:06:03.532 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:03.532+0000 7fab3d61d740 1 -- 172.21.15.47:0/1451406988 shutdown_connections 2024-04-17T06:06:03.532 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:03.532+0000 7fab3d61d740 1 -- 172.21.15.47:0/1451406988 wait complete. 2024-04-17T06:06:03.532 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:03.533+0000 7fab3d61d740 20 client.15801 trim_cache size 0 max 16384 2024-04-17T06:06:03.532 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-04-17T06:06:03.532 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-04-17T06:06:03.532 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-04-17T06:06:03.532 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-04-17T06:06:03.533 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:03.533+0000 7fab3a952640 1 -- 172.21.15.47:0/3163058983 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fab1c0a3dc0 msgr2=0x7fab1c0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:06:03.533 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:03.533+0000 7fab3a952640 1 --2- 172.21.15.47:0/3163058983 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fab1c0a3dc0 0x7fab1c0a6280 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fab2c099320 tx=0x7fab2c087040 comp rx=0 tx=0).stop 2024-04-17T06:06:03.533 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:03.533+0000 7fab3a952640 1 -- 172.21.15.47:0/3163058983 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab3419aa10 msgr2=0x7fab3419aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:06:03.533 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:03.533+0000 7fab3a952640 1 --2- 172.21.15.47:0/3163058983 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab3419aa10 0x7fab3419aec0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fab2c00eaa0 tx=0x7fab2c00ef60 comp rx=0 tx=0).stop 2024-04-17T06:06:03.533 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:03.534+0000 7fab3a952640 1 -- 172.21.15.47:0/3163058983 shutdown_connections 2024-04-17T06:06:03.534 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:03.534+0000 7fab3a952640 1 --2- 172.21.15.47:0/3163058983 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fab1c0a3dc0 0x7fab1c0a6280 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:03.534 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:03.534+0000 7fab3a952640 1 --2- 172.21.15.47:0/3163058983 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab3419aa10 0x7fab3419aec0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:03.534 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:03.534+0000 7fab3a952640 1 --2- 172.21.15.47:0/3163058983 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fab341063b0 0x7fab341a0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:03.534 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:03.534+0000 7fab3a952640 1 --2- 172.21.15.47:0/3163058983 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fab34101800 0x7fab341a0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:03.534 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:03.534+0000 7fab3a952640 1 -- 172.21.15.47:0/3163058983 >> 172.21.15.47:0/3163058983 conn(0x7fab34077380 msgr2=0x7fab34104b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:06:03.534 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:03.534+0000 7fab3a952640 1 -- 172.21.15.47:0/3163058983 shutdown_connections 2024-04-17T06:06:03.534 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:03.534+0000 7fab3a952640 1 -- 172.21.15.47:0/3163058983 wait complete. 2024-04-17T06:06:03.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:06:03 smithi098 ceph-mon[36516]: pgmap v147: 97 pgs: 97 active+clean; 2.5 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.7 KiB/s wr, 9 op/s 2024-04-17T06:06:03.883 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:06:03 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[70913]: *** Caught signal (Segmentation fault) ** 2024-04-17T06:06:03.883 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:06:03 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[70913]: in thread 7fef9b626700 thread_name:md_log_replay 2024-04-17T06:06:03.884 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:06:03 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[70913]: ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev) 2024-04-17T06:06:03.884 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:06:03 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[70913]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fefac9e7d20] 2024-04-17T06:06:03.884 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:06:03 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[70913]: 2: /usr/bin/ceph-mds(+0x54962f) [0x5636225e062f] 2024-04-17T06:06:03.884 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:06:03 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[70913]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x5636225e3dc3] 2024-04-17T06:06:03.884 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:06:03 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[70913]: 4: (MDLog::_replay_thread()+0x617) [0x56362263e8e7] 2024-04-17T06:06:03.884 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:06:03 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[70913]: 5: (MDLog::ReplayThread::entry()+0x11) [0x563622647fd1] 2024-04-17T06:06:03.884 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:06:03 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[70913]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7fefac9dd1ca] 2024-04-17T06:06:03.884 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:06:03 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[70913]: 7: clone() 2024-04-17T06:06:03.884 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:06:03 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[70913]: 2024-04-17T06:06:03.592+0000 7fef9b626700 -1 *** Caught signal (Segmentation fault) ** 2024-04-17T06:06:03.884 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:06:03 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[70913]: in thread 7fef9b626700 thread_name:md_log_replay 2024-04-17T06:06:03.884 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:06:03 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[70913]: 2024-04-17T06:06:03.884 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:06:03 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[70913]: ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev) 2024-04-17T06:06:03.884 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:06:03 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[70913]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fefac9e7d20] 2024-04-17T06:06:03.885 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:06:03 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[70913]: 2: /usr/bin/ceph-mds(+0x54962f) [0x5636225e062f] 2024-04-17T06:06:03.885 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:06:03 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[70913]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x5636225e3dc3] 2024-04-17T06:06:03.885 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:06:03 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[70913]: 4: (MDLog::_replay_thread()+0x617) [0x56362263e8e7] 2024-04-17T06:06:03.885 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:06:03 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[70913]: 5: (MDLog::ReplayThread::entry()+0x11) [0x563622647fd1] 2024-04-17T06:06:03.885 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:06:03 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[70913]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7fefac9dd1ca] 2024-04-17T06:06:03.885 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:06:03 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[70913]: 7: clone() 2024-04-17T06:06:03.885 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:06:03 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[70913]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-17T06:06:03.885 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:06:03 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[70913]: 2024-04-17T06:06:03.885 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:06:03 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[70913]: 0> 2024-04-17T06:06:03.592+0000 7fef9b626700 -1 *** Caught signal (Segmentation fault) ** 2024-04-17T06:06:03.885 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:06:03 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[70913]: in thread 7fef9b626700 thread_name:md_log_replay 2024-04-17T06:06:03.885 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:06:03 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[70913]: 2024-04-17T06:06:03.885 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:06:03 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[70913]: ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev) 2024-04-17T06:06:03.885 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:06:03 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[70913]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fefac9e7d20] 2024-04-17T06:06:03.885 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:06:03 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[70913]: 2: /usr/bin/ceph-mds(+0x54962f) [0x5636225e062f] 2024-04-17T06:06:03.886 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:06:03 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[70913]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x5636225e3dc3] 2024-04-17T06:06:03.886 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:06:03 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[70913]: 4: (MDLog::_replay_thread()+0x617) [0x56362263e8e7] 2024-04-17T06:06:03.886 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:06:03 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[70913]: 5: (MDLog::ReplayThread::entry()+0x11) [0x563622647fd1] 2024-04-17T06:06:03.886 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:06:03 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[70913]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7fefac9dd1ca] 2024-04-17T06:06:03.886 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:06:03 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[70913]: 7: clone() 2024-04-17T06:06:03.886 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:06:03 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[70913]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-17T06:06:03.886 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:06:03 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[70913]: 2024-04-17T06:06:03.886 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:06:03 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[70913]: 0> 2024-04-17T06:06:03.592+0000 7fef9b626700 -1 *** Caught signal (Segmentation fault) ** 2024-04-17T06:06:03.886 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:06:03 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[70913]: in thread 7fef9b626700 thread_name:md_log_replay 2024-04-17T06:06:03.886 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:06:03 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[70913]: 2024-04-17T06:06:03.886 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:06:03 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[70913]: ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev) 2024-04-17T06:06:03.886 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:06:03 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[70913]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fefac9e7d20] 2024-04-17T06:06:03.886 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:06:03 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[70913]: 2: /usr/bin/ceph-mds(+0x54962f) [0x5636225e062f] 2024-04-17T06:06:03.886 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:06:03 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[70913]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x5636225e3dc3] 2024-04-17T06:06:03.886 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:06:03 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[70913]: 4: (MDLog::_replay_thread()+0x617) [0x56362263e8e7] 2024-04-17T06:06:03.887 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:06:03 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[70913]: 5: (MDLog::ReplayThread::entry()+0x11) [0x563622647fd1] 2024-04-17T06:06:03.887 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:06:03 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[70913]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7fefac9dd1ca] 2024-04-17T06:06:03.887 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:06:03 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[70913]: 7: clone() 2024-04-17T06:06:03.887 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:06:03 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[70913]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-17T06:06:03.887 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:06:03 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[70913]: 2024-04-17T06:06:03.887 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:06:03 smithi098 podman[71229]: 2024-04-17 06:06:03.647542658 +0000 UTC m=+0.021738820 container died 7f9c4de4da23047896a6ce319c90b24489fd46448e31d3c2037eb0332d37acae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux ) 2024-04-17T06:06:03.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:06:03 smithi047 ceph-mon[32223]: pgmap v147: 97 pgs: 97 active+clean; 2.5 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.7 KiB/s wr, 9 op/s 2024-04-17T06:06:04.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:06:03 smithi103 ceph-mon[36243]: pgmap v147: 97 pgs: 97 active+clean; 2.5 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.7 KiB/s wr, 9 op/s 2024-04-17T06:06:04.882 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:06:04 smithi098 podman[71229]: 2024-04-17 06:06:04.38998365 +0000 UTC m=+0.764179814 container remove 7f9c4de4da23047896a6ce319c90b24489fd46448e31d3c2037eb0332d37acae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416) 2024-04-17T06:06:04.883 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:06:04 smithi098 systemd[1]: ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@mds.e.service: Main process exited, code=exited, status=139/n/a 2024-04-17T06:06:04.883 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:06:04 smithi098 systemd[1]: ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@mds.e.service: Failed with result 'exit-code'. 2024-04-17T06:06:05.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:06:05 smithi047 ceph-mon[32223]: pgmap v148: 97 pgs: 97 active+clean; 2.6 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 37 KiB/s wr, 31 op/s 2024-04-17T06:06:06.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:06:05 smithi103 ceph-mon[36243]: pgmap v148: 97 pgs: 97 active+clean; 2.6 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 37 KiB/s wr, 31 op/s 2024-04-17T06:06:06.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:06:05 smithi098 ceph-mon[36516]: pgmap v148: 97 pgs: 97 active+clean; 2.6 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 37 KiB/s wr, 31 op/s 2024-04-17T06:06:08.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:06:07 smithi103 ceph-mon[36243]: pgmap v149: 97 pgs: 97 active+clean; 2.6 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 33 KiB/s wr, 26 op/s 2024-04-17T06:06:08.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:06:07 smithi098 ceph-mon[36516]: pgmap v149: 97 pgs: 97 active+clean; 2.6 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 33 KiB/s wr, 26 op/s 2024-04-17T06:06:08.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:06:07 smithi047 ceph-mon[32223]: pgmap v149: 97 pgs: 97 active+clean; 2.6 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 33 KiB/s wr, 26 op/s 2024-04-17T06:06:10.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:06:09 smithi103 ceph-mon[36243]: pgmap v150: 97 pgs: 97 active+clean; 2.6 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 33 KiB/s wr, 26 op/s 2024-04-17T06:06:10.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:06:09 smithi098 ceph-mon[36516]: pgmap v150: 97 pgs: 97 active+clean; 2.6 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 33 KiB/s wr, 26 op/s 2024-04-17T06:06:10.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:06:09 smithi047 ceph-mon[32223]: pgmap v150: 97 pgs: 97 active+clean; 2.6 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 33 KiB/s wr, 26 op/s 2024-04-17T06:06:12.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:06:11 smithi103 ceph-mon[36243]: pgmap v151: 97 pgs: 97 active+clean; 2.6 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 36 KiB/s wr, 30 op/s 2024-04-17T06:06:12.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:06:11 smithi098 ceph-mon[36516]: pgmap v151: 97 pgs: 97 active+clean; 2.6 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 36 KiB/s wr, 30 op/s 2024-04-17T06:06:12.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:06:11 smithi047 ceph-mon[32223]: pgmap v151: 97 pgs: 97 active+clean; 2.6 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 36 KiB/s wr, 30 op/s 2024-04-17T06:06:14.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:06:13 smithi103 ceph-mon[36243]: pgmap v152: 97 pgs: 97 active+clean; 2.6 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 33 KiB/s wr, 26 op/s 2024-04-17T06:06:14.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:06:13 smithi098 ceph-mon[36516]: pgmap v152: 97 pgs: 97 active+clean; 2.6 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 33 KiB/s wr, 26 op/s 2024-04-17T06:06:14.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:06:13 smithi047 ceph-mon[32223]: pgmap v152: 97 pgs: 97 active+clean; 2.6 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 33 KiB/s wr, 26 op/s 2024-04-17T06:06:14.914 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:06:14 smithi098 systemd[1]: ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@mds.e.service: Scheduled restart job, restart counter is at 3. 2024-04-17T06:06:14.914 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:06:14 smithi098 systemd[1]: Stopped Ceph mds.e for 565db2d8-fc7e-11ee-bc91-c7b262605968. 2024-04-17T06:06:14.914 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:06:14 smithi098 systemd[1]: Starting Ceph mds.e for 565db2d8-fc7e-11ee-bc91-c7b262605968... 2024-04-17T06:06:14.914 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:06:14 smithi098 podman[71330]: 2024-04-17 06:06:14.771827514 +0000 UTC m=+0.018159288 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-17T06:06:14.914 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:06:14 smithi098 podman[71330]: 2024-04-17 06:06:14.916360967 +0000 UTC m=+0.162692770 container create c10b1e0b1416beb3d91d8206ac63000f89b005e2582c3e40b474bb19a69df613 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2) 2024-04-17T06:06:15.217 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:06:15 smithi098 podman[71330]: 2024-04-17 06:06:15.127672851 +0000 UTC m=+0.374004615 container init c10b1e0b1416beb3d91d8206ac63000f89b005e2582c3e40b474bb19a69df613 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-04-17T06:06:15.217 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:06:15 smithi098 podman[71330]: 2024-04-17 06:06:15.130277233 +0000 UTC m=+0.376609003 container start c10b1e0b1416beb3d91d8206ac63000f89b005e2582c3e40b474bb19a69df613 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-17T06:06:15.217 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:06:15 smithi098 ceph-mds[71349]: -- 172.21.15.98:0/3144846584 <== mon.2 v2:172.21.15.98:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fcef7c9a40 con 0x55fcef79cc00 2024-04-17T06:06:15.217 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:06:15 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[71345]: starting mds.e at 2024-04-17T06:06:15.482 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:06:15 smithi098 bash[71330]: c10b1e0b1416beb3d91d8206ac63000f89b005e2582c3e40b474bb19a69df613 2024-04-17T06:06:15.482 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:06:15 smithi098 systemd[1]: Started Ceph mds.e for 565db2d8-fc7e-11ee-bc91-c7b262605968. 2024-04-17T06:06:15.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:06:15 smithi098 ceph-mon[36516]: pgmap v153: 97 pgs: 97 active+clean; 2.6 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 38 KiB/s wr, 30 op/s 2024-04-17T06:06:15.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:06:15 smithi098 ceph-mon[36516]: MDS daemon mds.e is removed because it is dead or otherwise unavailable. 2024-04-17T06:06:15.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:06:15 smithi098 ceph-mon[36516]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 5 up:standby 2024-04-17T06:06:16.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:06:15 smithi103 ceph-mon[36243]: pgmap v153: 97 pgs: 97 active+clean; 2.6 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 38 KiB/s wr, 30 op/s 2024-04-17T06:06:16.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:06:15 smithi103 ceph-mon[36243]: MDS daemon mds.e is removed because it is dead or otherwise unavailable. 2024-04-17T06:06:16.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:06:15 smithi103 ceph-mon[36243]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 5 up:standby 2024-04-17T06:06:16.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:06:15 smithi047 ceph-mon[32223]: pgmap v153: 97 pgs: 97 active+clean; 2.6 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 38 KiB/s wr, 30 op/s 2024-04-17T06:06:16.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:06:15 smithi047 ceph-mon[32223]: MDS daemon mds.e is removed because it is dead or otherwise unavailable. 2024-04-17T06:06:16.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:06:15 smithi047 ceph-mon[32223]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 5 up:standby 2024-04-17T06:06:16.636 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.634+0000 7f6a09635640 1 -- 172.21.15.47:0/1040321642 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a04106fc0 msgr2=0x7f6a041073a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:06:16.636 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.634+0000 7f6a09635640 1 --2- 172.21.15.47:0/1040321642 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a04106fc0 0x7f6a041073a0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f69f4009900 tx=0x7f69f402bc80 comp rx=0 tx=0).stop 2024-04-17T06:06:16.636 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.635+0000 7f6a09635640 1 -- 172.21.15.47:0/1040321642 shutdown_connections 2024-04-17T06:06:16.636 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.635+0000 7f6a09635640 1 --2- 172.21.15.47:0/1040321642 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6a04101b70 0x7f6a0410dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:16.636 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.635+0000 7f6a09635640 1 --2- 172.21.15.47:0/1040321642 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6a041011b0 0x7f6a04101630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:16.636 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.635+0000 7f6a09635640 1 --2- 172.21.15.47:0/1040321642 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a04106fc0 0x7f6a041073a0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:16.637 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.635+0000 7f6a09635640 1 -- 172.21.15.47:0/1040321642 >> 172.21.15.47:0/1040321642 conn(0x7f6a04076c60 msgr2=0x7f6a04077070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:06:16.637 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.635+0000 7f6a09635640 1 -- 172.21.15.47:0/1040321642 shutdown_connections 2024-04-17T06:06:16.637 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.635+0000 7f6a09635640 1 -- 172.21.15.47:0/1040321642 wait complete. 2024-04-17T06:06:16.637 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.635+0000 7f6a09635640 1 Processor -- start 2024-04-17T06:06:16.637 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.635+0000 7f6a09635640 1 -- start start 2024-04-17T06:06:16.637 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.636+0000 7f6a09635640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6a041011b0 0x7f6a0410e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:06:16.637 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.636+0000 7f6a09635640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a04101b70 0x7f6a0410ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:06:16.637 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.636+0000 7f6a09635640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6a04106fc0 0x7f6a0410f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:06:16.637 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.636+0000 7f6a09635640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f6a04112340 con 0x7f6a04101b70 2024-04-17T06:06:16.637 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.636+0000 7f6a09635640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6a041121c0 con 0x7f6a04106fc0 2024-04-17T06:06:16.637 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.636+0000 7f6a09635640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6a041124c0 con 0x7f6a041011b0 2024-04-17T06:06:16.637 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.636+0000 7f69fa7fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a04101b70 0x7f6a0410ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:06:16.638 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.636+0000 7f69fa7fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a04101b70 0x7f6a0410ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35370/0 (socket says 172.21.15.47:35370) 2024-04-17T06:06:16.638 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.636+0000 7f69fa7fc640 1 -- 172.21.15.47:0/1934365955 learned_addr learned my addr 172.21.15.47:0/1934365955 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:06:16.638 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.636+0000 7f6a037fe640 1 --2- 172.21.15.47:0/1934365955 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6a04106fc0 0x7f6a0410f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:06:16.638 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.636+0000 7f6a02ffd640 1 --2- 172.21.15.47:0/1934365955 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6a041011b0 0x7f6a0410e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:06:16.638 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.636+0000 7f69fa7fc640 1 -- 172.21.15.47:0/1934365955 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6a041011b0 msgr2=0x7f6a0410e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:06:16.638 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.636+0000 7f69fa7fc640 1 --2- 172.21.15.47:0/1934365955 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6a041011b0 0x7f6a0410e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:16.638 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.636+0000 7f69fa7fc640 1 -- 172.21.15.47:0/1934365955 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6a04106fc0 msgr2=0x7f6a0410f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:06:16.638 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.636+0000 7f69fa7fc640 1 --2- 172.21.15.47:0/1934365955 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6a04106fc0 0x7f6a0410f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:16.638 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.636+0000 7f69fa7fc640 1 -- 172.21.15.47:0/1934365955 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a041a6be0 con 0x7f6a04101b70 2024-04-17T06:06:16.638 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.636+0000 7f6a02ffd640 1 --2- 172.21.15.47:0/1934365955 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6a041011b0 0x7f6a0410e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:06:16.638 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.636+0000 7f6a037fe640 1 --2- 172.21.15.47:0/1934365955 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6a04106fc0 0x7f6a0410f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T06:06:16.638 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.636+0000 7f69fa7fc640 1 --2- 172.21.15.47:0/1934365955 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a04101b70 0x7f6a0410ec50 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f69ec00e9f0 tx=0x7f69ec00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:06:16.638 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.636+0000 7f6a00ff9640 1 -- 172.21.15.47:0/1934365955 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69ec00cd80 con 0x7f6a04101b70 2024-04-17T06:06:16.639 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.637+0000 7f6a00ff9640 1 -- 172.21.15.47:0/1934365955 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69ec010910 con 0x7f6a04101b70 2024-04-17T06:06:16.639 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.637+0000 7f6a09635640 1 -- 172.21.15.47:0/1934365955 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a041a6ed0 con 0x7f6a04101b70 2024-04-17T06:06:16.639 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.637+0000 7f6a00ff9640 1 -- 172.21.15.47:0/1934365955 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69ec018990 con 0x7f6a04101b70 2024-04-17T06:06:16.639 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.637+0000 7f6a09635640 1 -- 172.21.15.47:0/1934365955 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a041a7390 con 0x7f6a04101b70 2024-04-17T06:06:16.645 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.639+0000 7f6a00ff9640 1 -- 172.21.15.47:0/1934365955 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f69ec010ac0 con 0x7f6a04101b70 2024-04-17T06:06:16.645 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.639+0000 7f6a0c21e740 1 Processor -- start 2024-04-17T06:06:16.645 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.639+0000 7f6a0c21e740 1 -- start start 2024-04-17T06:06:16.645 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.640+0000 7f6a0c21e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555fa4b330c0 0x555fa4b334a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:06:16.645 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.640+0000 7f6a0c21e740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555fa4b339e0 0x555fa4b3c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:06:16.645 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.640+0000 7f6a0c21e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555fa4b3c7b0 0x555fa4b3eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:06:16.645 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.640+0000 7f6a0c21e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x555fa4a987d0 con 0x555fa4b3c7b0 2024-04-17T06:06:16.645 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.640+0000 7f6a0c21e740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x555fa4a35f70 con 0x555fa4b339e0 2024-04-17T06:06:16.645 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.640+0000 7f6a0c21e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x555fa4a94790 con 0x555fa4b330c0 2024-04-17T06:06:16.646 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.640+0000 7f6a00ff9640 1 --2- 172.21.15.47:0/1934365955 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f69f00a3df0 0x7f69f00a62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:06:16.646 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.640+0000 7f69fa7fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555fa4b3c7b0 0x555fa4b3eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:06:16.646 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.640+0000 7f6a02ffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555fa4b339e0 0x555fa4b3c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:06:16.646 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.640+0000 7f69fa7fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555fa4b3c7b0 0x555fa4b3eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35374/0 (socket says 172.21.15.47:35374) 2024-04-17T06:06:16.646 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.640+0000 7f6a037fe640 1 --2- 172.21.15.47:0/1934365955 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f69f00a3df0 0x7f69f00a62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:06:16.646 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.640+0000 7f69fa7fc640 1 -- 172.21.15.47:0/3120904491 learned_addr learned my addr 172.21.15.47:0/3120904491 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:06:16.646 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.640+0000 7f6a037fe640 1 --2- 172.21.15.47:0/3120904491 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555fa4b330c0 0x555fa4b334a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:06:16.646 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.640+0000 7f6a00ff9640 1 -- 172.21.15.47:0/1934365955 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f69ec0d0b10 con 0x7f6a04101b70 2024-04-17T06:06:16.646 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.641+0000 7f69fa7fc640 1 -- 172.21.15.47:0/3120904491 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555fa4b330c0 msgr2=0x555fa4b334a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:06:16.646 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.641+0000 7f69fa7fc640 1 --2- 172.21.15.47:0/3120904491 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555fa4b330c0 0x555fa4b334a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:16.646 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.641+0000 7f69fa7fc640 1 -- 172.21.15.47:0/3120904491 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555fa4b339e0 msgr2=0x555fa4b3c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:06:16.646 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.641+0000 7f69fa7fc640 1 --2- 172.21.15.47:0/3120904491 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555fa4b339e0 0x555fa4b3c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:16.646 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.641+0000 7f69fa7fc640 1 -- 172.21.15.47:0/3120904491 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555fa47ab210 con 0x555fa4b3c7b0 2024-04-17T06:06:16.646 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.641+0000 7f6a02ffd640 1 --2- 172.21.15.47:0/3120904491 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555fa4b339e0 0x555fa4b3c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T06:06:16.647 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.641+0000 7f6a037fe640 1 --2- 172.21.15.47:0/1934365955 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f69f00a3df0 0x7f69f00a62b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f69fc005fd0 tx=0x7f69fc011440 comp rx=0 tx=0).ready entity=mgr.24923 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:06:16.647 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.641+0000 7f69fa7fc640 1 --2- 172.21.15.47:0/3120904491 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555fa4b3c7b0 0x555fa4b3eba0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f69ec09c7e0 tx=0x7f69ec06f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ee53264370c6d15 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:06:16.647 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.641+0000 7f69f97fa640 1 -- 172.21.15.47:0/3120904491 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69ec09d1e0 con 0x555fa4b3c7b0 2024-04-17T06:06:16.647 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.641+0000 7f69f97fa640 1 -- 172.21.15.47:0/3120904491 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69ec09dbd0 con 0x555fa4b3c7b0 2024-04-17T06:06:16.647 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.641+0000 7f69f97fa640 1 -- 172.21.15.47:0/3120904491 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69ec0a5750 con 0x555fa4b3c7b0 2024-04-17T06:06:16.647 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.642+0000 7f6a0c21e740 1 -- 172.21.15.47:0/3120904491 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555fa4b3c7b0 msgr2=0x555fa4b3eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:06:16.647 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.642+0000 7f6a0c21e740 1 --2- 172.21.15.47:0/3120904491 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555fa4b3c7b0 0x555fa4b3eba0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f69ec09c7e0 tx=0x7f69ec06f690 comp rx=0 tx=0).stop 2024-04-17T06:06:16.647 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.642+0000 7f6a0c21e740 1 -- 172.21.15.47:0/3120904491 shutdown_connections 2024-04-17T06:06:16.647 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.642+0000 7f6a0c21e740 1 --2- 172.21.15.47:0/3120904491 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555fa4b3c7b0 0x555fa4b3eba0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:16.647 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.642+0000 7f6a0c21e740 1 --2- 172.21.15.47:0/3120904491 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555fa4b339e0 0x555fa4b3c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:16.647 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.642+0000 7f6a0c21e740 1 --2- 172.21.15.47:0/3120904491 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555fa4b330c0 0x555fa4b334a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:16.647 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.642+0000 7f6a0c21e740 1 -- 172.21.15.47:0/3120904491 >> 172.21.15.47:0/3120904491 conn(0x555fa4b2d9e0 msgr2=0x555fa4b2ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:06:16.647 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.642+0000 7f6a0c21e740 1 -- 172.21.15.47:0/3120904491 shutdown_connections 2024-04-17T06:06:16.647 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.642+0000 7f6a0c21e740 1 -- 172.21.15.47:0/3120904491 wait complete. 2024-04-17T06:06:16.648 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.642+0000 7f6a0c21e740 1 -- start start 2024-04-17T06:06:16.648 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.642+0000 7f6a0c21e740 1 Processor -- start 2024-04-17T06:06:16.648 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.643+0000 7f6a0c21e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555fa4b330c0 0x555fa4b39870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:06:16.648 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.643+0000 7f6a0c21e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555fa4b339e0 0x555fa4b39db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:06:16.648 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.643+0000 7f6a0c21e740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555fa4b3c7b0 0x555fa4b3a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:06:16.648 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.643+0000 7f6a0c21e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x555fa4b3f3e0 con 0x555fa4b339e0 2024-04-17T06:06:16.648 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.643+0000 7f6a0c21e740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x555fa4a20e90 con 0x555fa4b3c7b0 2024-04-17T06:06:16.648 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.643+0000 7f6a0c21e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x555fa4b3f560 con 0x555fa4b330c0 2024-04-17T06:06:16.648 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.643+0000 7f6a0c21e740 10 client.0 _ll_register_callbacks cb 0x555fa4a36d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:06:16.648 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.643+0000 7f69fa7fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555fa4b339e0 0x555fa4b39db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:06:16.648 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.643+0000 7f69fa7fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555fa4b339e0 0x555fa4b39db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35386/0 (socket says 172.21.15.47:35386) 2024-04-17T06:06:16.648 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.643+0000 7f69fa7fc640 1 -- 172.21.15.47:0/1788816748 learned_addr learned my addr 172.21.15.47:0/1788816748 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:06:16.648 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.643+0000 7f6a037fe640 1 --2- 172.21.15.47:0/1788816748 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555fa4b3c7b0 0x555fa4b3a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:06:16.648 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.643+0000 7f6a02ffd640 1 --2- 172.21.15.47:0/1788816748 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555fa4b330c0 0x555fa4b39870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:06:16.649 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.643+0000 7f69fa7fc640 1 -- 172.21.15.47:0/1788816748 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555fa4b330c0 msgr2=0x555fa4b39870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:06:16.649 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.643+0000 7f69fa7fc640 1 --2- 172.21.15.47:0/1788816748 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555fa4b330c0 0x555fa4b39870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:16.649 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.643+0000 7f69fa7fc640 1 -- 172.21.15.47:0/1788816748 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555fa4b3c7b0 msgr2=0x555fa4b3a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:06:16.649 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.643+0000 7f69fa7fc640 1 --2- 172.21.15.47:0/1788816748 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555fa4b3c7b0 0x555fa4b3a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:16.649 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.643+0000 7f69fa7fc640 1 -- 172.21.15.47:0/1788816748 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555fa4877620 con 0x555fa4b339e0 2024-04-17T06:06:16.649 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.643+0000 7f6a02ffd640 1 --2- 172.21.15.47:0/1788816748 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555fa4b330c0 0x555fa4b39870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:06:16.649 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.643+0000 7f69fa7fc640 1 --2- 172.21.15.47:0/1788816748 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555fa4b339e0 0x555fa4b39db0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f69ec09ca70 tx=0x7f69ec0a0f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e913fb7b197a63c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:06:16.649 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.643+0000 7f69ea7fc640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-17T06:06:16.649 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.643+0000 7f69ea7fc640 1 -- 172.21.15.47:0/1788816748 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69ec09cd40 con 0x555fa4b339e0 2024-04-17T06:06:16.649 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.643+0000 7f69ea7fc640 1 -- 172.21.15.47:0/1788816748 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f69ec09d470 con 0x555fa4b339e0 2024-04-17T06:06:16.649 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.643+0000 7f6a0c21e740 1 -- 172.21.15.47:0/1788816748 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69ec09dbd0 con 0x555fa4b339e0 2024-04-17T06:06:16.649 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.644+0000 7f69ea7fc640 1 -- 172.21.15.47:0/1788816748 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f69ec09cf80 con 0x555fa4b339e0 2024-04-17T06:06:16.649 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.644+0000 7f6a0c21e740 10 client.15819 fetch_fsmap learned FSMap version 30 2024-04-17T06:06:16.649 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.644+0000 7f6a0c21e740 1 -- 172.21.15.47:0/1788816748 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555fa4b4b6d0 con 0x555fa4b339e0 2024-04-17T06:06:16.650 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.644+0000 7f69ea7fc640 1 -- 172.21.15.47:0/1788816748 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69ec09dbd0 con 0x555fa4b339e0 2024-04-17T06:06:16.650 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.644+0000 7f69ea7fc640 1 -- 172.21.15.47:0/1788816748 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 30) ==== 8339+0+0 (secure 0 0 0) 0x7f69ec072120 con 0x555fa4b339e0 2024-04-17T06:06:16.650 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.644+0000 7f6a0c21e740 10 client.15819 fetch_fsmap finished waiting for FSMap version 30 2024-04-17T06:06:16.650 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.644+0000 7f6a0c21e740 10 client.15819 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-17T06:06:16.650 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.644+0000 7f6a0c21e740 20 client.15819 populate_metadata read hostname 'smithi047' 2024-04-17T06:06:16.650 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.644+0000 7f6a0c21e740 1 --2- 172.21.15.47:0/1788816748 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x555fa4b4ca20 0x555fa4b4ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:06:16.650 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.644+0000 7f6a0c21e740 4 client.15819 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T06:06:16.650 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.644+0000 7f6a0c21e740 1 -- 172.21.15.47:0/1788816748 --> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555fa4aa61f0 con 0x555fa4b4ca20 2024-04-17T06:06:16.650 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.644+0000 7f6a02ffd640 1 --2- 172.21.15.47:0/1788816748 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x555fa4b4ca20 0x555fa4b4ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:06:16.650 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.645+0000 7f6a02ffd640 1 --2- 172.21.15.47:0/1788816748 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x555fa4b4ca20 0x555fa4b4ee10 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44dd5ff23017df30 server_cookie=1ab392891be5154a in_seq=0 out_seq=0 2024-04-17T06:06:16.650 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.645+0000 7f69ea7fc640 10 client.15819 ms_handle_connect on v2:172.21.15.98:6838/2841287917 2024-04-17T06:06:16.650 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.645+0000 7f69ea7fc640 1 -- 172.21.15.47:0/1788816748 <== mds.0 v2:172.21.15.98:6838/2841287917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555fa4aa61f0 con 0x555fa4b4ca20 2024-04-17T06:06:16.650 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.645+0000 7f69ea7fc640 10 client.15819 handle_command_reply: tid=0 2024-04-17T06:06:16.650 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.645+0000 7f6a0c21e740 1 client.15819 shutdown 2024-04-17T06:06:16.651 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.645+0000 7f6a0c21e740 1 -- 172.21.15.47:0/1788816748 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555fa4b339e0 msgr2=0x555fa4b39db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:06:16.651 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.645+0000 7f6a0c21e740 1 --2- 172.21.15.47:0/1788816748 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555fa4b339e0 0x555fa4b39db0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f69ec09ca70 tx=0x7f69ec0a0f00 comp rx=0 tx=0).stop 2024-04-17T06:06:16.651 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.645+0000 7f6a0c21e740 1 -- 172.21.15.47:0/1788816748 shutdown_connections 2024-04-17T06:06:16.651 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.645+0000 7f6a0c21e740 1 --2- 172.21.15.47:0/1788816748 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x555fa4b4ca20 0x555fa4b4ee10 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:16.651 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.645+0000 7f6a0c21e740 1 --2- 172.21.15.47:0/1788816748 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555fa4b3c7b0 0x555fa4b3a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:16.651 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.645+0000 7f6a0c21e740 1 --2- 172.21.15.47:0/1788816748 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555fa4b339e0 0x555fa4b39db0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:16.651 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.645+0000 7f6a0c21e740 1 --2- 172.21.15.47:0/1788816748 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555fa4b330c0 0x555fa4b39870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:16.651 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.645+0000 7f69ea7fc640 0 client.15819 ms_handle_reset on v2:172.21.15.98:6838/2841287917 2024-04-17T06:06:16.651 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.645+0000 7f6a0c21e740 1 -- 172.21.15.47:0/1788816748 >> 172.21.15.47:0/1788816748 conn(0x555fa4b2d9e0 msgr2=0x555fa4b2f730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:06:16.651 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.646+0000 7f6a0c21e740 1 -- 172.21.15.47:0/1788816748 shutdown_connections 2024-04-17T06:06:16.651 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.646+0000 7f6a0c21e740 1 -- 172.21.15.47:0/1788816748 wait complete. 2024-04-17T06:06:16.651 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.646+0000 7f6a0c21e740 20 client.15819 trim_cache size 0 max 16384 2024-04-17T06:06:16.659 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.661+0000 7f6a0c21e740 1 Processor -- start 2024-04-17T06:06:16.659 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.661+0000 7f6a0c21e740 1 -- start start 2024-04-17T06:06:16.660 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.661+0000 7f6a0c21e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555fa4b330c0 0x555fa4b3bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:06:16.661 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.661+0000 7f6a0c21e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555fa4b339e0 0x555fa4b47ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:06:16.661 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.661+0000 7f6a0c21e740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555fa4b3c7b0 0x555fa4b51e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:06:16.661 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.661+0000 7f6a0c21e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f6a041b2610 con 0x555fa4b339e0 2024-04-17T06:06:16.661 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.661+0000 7f6a0c21e740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6a041afed0 con 0x555fa4b3c7b0 2024-04-17T06:06:16.661 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.661+0000 7f6a0c21e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6a041add20 con 0x555fa4b330c0 2024-04-17T06:06:16.661 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.661+0000 7f69fa7fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555fa4b339e0 0x555fa4b47ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:06:16.661 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.661+0000 7f69fa7fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555fa4b339e0 0x555fa4b47ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35398/0 (socket says 172.21.15.47:35398) 2024-04-17T06:06:16.661 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.661+0000 7f69fa7fc640 1 -- 172.21.15.47:0/4099015894 learned_addr learned my addr 172.21.15.47:0/4099015894 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:06:16.661 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.661+0000 7f6a037fe640 1 --2- 172.21.15.47:0/4099015894 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555fa4b3c7b0 0x555fa4b51e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:06:16.661 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.661+0000 7f69fa7fc640 1 -- 172.21.15.47:0/4099015894 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555fa4b330c0 msgr2=0x555fa4b3bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:06:16.661 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.661+0000 7f69fa7fc640 1 --2- 172.21.15.47:0/4099015894 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555fa4b330c0 0x555fa4b3bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:16.661 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.661+0000 7f69fa7fc640 1 -- 172.21.15.47:0/4099015894 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555fa4b3c7b0 msgr2=0x555fa4b51e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:06:16.661 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.661+0000 7f69fa7fc640 1 --2- 172.21.15.47:0/4099015894 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555fa4b3c7b0 0x555fa4b51e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:16.661 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.661+0000 7f69fa7fc640 1 -- 172.21.15.47:0/4099015894 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555fa4b3f500 con 0x555fa4b339e0 2024-04-17T06:06:16.662 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.662+0000 7f69fa7fc640 1 --2- 172.21.15.47:0/4099015894 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555fa4b339e0 0x555fa4b47ec0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f69ec09ad90 tx=0x7f69ec018420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1d3c3b4645f0556 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:06:16.662 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.662+0000 7f69f8ff9640 1 -- 172.21.15.47:0/4099015894 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69ec09c440 con 0x555fa4b339e0 2024-04-17T06:06:16.662 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.662+0000 7f69f8ff9640 1 -- 172.21.15.47:0/4099015894 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69ec09d1e0 con 0x555fa4b339e0 2024-04-17T06:06:16.662 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.662+0000 7f69f8ff9640 1 -- 172.21.15.47:0/4099015894 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69ec099150 con 0x555fa4b339e0 2024-04-17T06:06:16.665 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.662+0000 7f6a0c21e740 1 -- 172.21.15.47:0/4099015894 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555fa4b339e0 msgr2=0x555fa4b47ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:06:16.665 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.662+0000 7f6a0c21e740 1 --2- 172.21.15.47:0/4099015894 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555fa4b339e0 0x555fa4b47ec0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f69ec09ad90 tx=0x7f69ec018420 comp rx=0 tx=0).stop 2024-04-17T06:06:16.665 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.662+0000 7f6a0c21e740 1 -- 172.21.15.47:0/4099015894 shutdown_connections 2024-04-17T06:06:16.665 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.662+0000 7f6a0c21e740 1 --2- 172.21.15.47:0/4099015894 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555fa4b3c7b0 0x555fa4b51e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:16.665 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.662+0000 7f6a0c21e740 1 --2- 172.21.15.47:0/4099015894 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555fa4b339e0 0x555fa4b47ec0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:16.665 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.662+0000 7f6a0c21e740 1 --2- 172.21.15.47:0/4099015894 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555fa4b330c0 0x555fa4b3bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:16.665 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.662+0000 7f6a0c21e740 1 -- 172.21.15.47:0/4099015894 >> 172.21.15.47:0/4099015894 conn(0x555fa4b2d9e0 msgr2=0x555fa4b477b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:06:16.665 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.663+0000 7f6a0c21e740 1 -- 172.21.15.47:0/4099015894 shutdown_connections 2024-04-17T06:06:16.665 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.663+0000 7f6a0c21e740 1 -- 172.21.15.47:0/4099015894 wait complete. 2024-04-17T06:06:16.666 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.663+0000 7f6a0c21e740 1 -- start start 2024-04-17T06:06:16.666 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.663+0000 7f6a0c21e740 1 Processor -- start 2024-04-17T06:06:16.666 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.663+0000 7f6a0c21e740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555fa4b330c0 0x555fa4b52200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:06:16.666 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.663+0000 7f6a0c21e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555fa4b339e0 0x555fa4b4f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:06:16.666 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.663+0000 7f6a0c21e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555fa4b3c7b0 0x555fa4b4fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:06:16.666 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.663+0000 7f6a0c21e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f6a041ae390 con 0x555fa4b339e0 2024-04-17T06:06:16.666 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.663+0000 7f6a0c21e740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6a041af800 con 0x555fa4b330c0 2024-04-17T06:06:16.666 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.663+0000 7f6a0c21e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6a041aea60 con 0x555fa4b3c7b0 2024-04-17T06:06:16.666 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.663+0000 7f6a0c21e740 10 client.0 _ll_register_callbacks cb 0x7f69ec005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:06:16.666 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.663+0000 7f69fa7fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555fa4b339e0 0x555fa4b4f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:06:16.666 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.663+0000 7f69fa7fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555fa4b339e0 0x555fa4b4f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35404/0 (socket says 172.21.15.47:35404) 2024-04-17T06:06:16.666 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.663+0000 7f69fa7fc640 1 -- 172.21.15.47:0/480154999 learned_addr learned my addr 172.21.15.47:0/480154999 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:06:16.666 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.663+0000 7f6a02ffd640 1 --2- 172.21.15.47:0/480154999 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555fa4b330c0 0x555fa4b52200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:06:16.667 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.663+0000 7f6a037fe640 1 --2- 172.21.15.47:0/480154999 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555fa4b3c7b0 0x555fa4b4fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:06:16.667 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.664+0000 7f69fa7fc640 1 -- 172.21.15.47:0/480154999 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555fa4b3c7b0 msgr2=0x555fa4b4fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:06:16.667 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.664+0000 7f69fa7fc640 1 --2- 172.21.15.47:0/480154999 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555fa4b3c7b0 0x555fa4b4fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:16.667 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.664+0000 7f69fa7fc640 1 -- 172.21.15.47:0/480154999 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555fa4b330c0 msgr2=0x555fa4b52200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:06:16.667 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.664+0000 7f69fa7fc640 1 --2- 172.21.15.47:0/480154999 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555fa4b330c0 0x555fa4b52200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:16.667 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.664+0000 7f69fa7fc640 1 -- 172.21.15.47:0/480154999 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555fa4b502b0 con 0x555fa4b339e0 2024-04-17T06:06:16.667 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.664+0000 7f6a02ffd640 1 --2- 172.21.15.47:0/480154999 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555fa4b330c0 0x555fa4b52200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T06:06:16.667 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.664+0000 7f69fa7fc640 1 --2- 172.21.15.47:0/480154999 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555fa4b339e0 0x555fa4b4f590 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f69ec00e9c0 tx=0x7f69ec09cd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c00fce069b0236e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:06:16.667 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.664+0000 7f69ea7fc640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-17T06:06:16.667 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.664+0000 7f69ea7fc640 1 -- 172.21.15.47:0/480154999 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69ec09c440 con 0x555fa4b339e0 2024-04-17T06:06:16.667 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.664+0000 7f69ea7fc640 1 -- 172.21.15.47:0/480154999 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f69ec072120 con 0x555fa4b339e0 2024-04-17T06:06:16.667 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.664+0000 7f6a0c21e740 1 -- 172.21.15.47:0/480154999 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69ec09d1e0 con 0x555fa4b339e0 2024-04-17T06:06:16.667 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.664+0000 7f69ea7fc640 1 -- 172.21.15.47:0/480154999 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f69ec072320 con 0x555fa4b339e0 2024-04-17T06:06:16.667 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.664+0000 7f6a0c21e740 10 client.15825 fetch_fsmap learned FSMap version 30 2024-04-17T06:06:16.668 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.664+0000 7f6a0c21e740 1 -- 172.21.15.47:0/480154999 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555fa4b502b0 con 0x555fa4b339e0 2024-04-17T06:06:16.668 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.664+0000 7f69ea7fc640 1 -- 172.21.15.47:0/480154999 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69ec09d1e0 con 0x555fa4b339e0 2024-04-17T06:06:16.668 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.664+0000 7f69ea7fc640 1 -- 172.21.15.47:0/480154999 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 30) ==== 8339+0+0 (secure 0 0 0) 0x7f69ec098830 con 0x555fa4b339e0 2024-04-17T06:06:16.668 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.664+0000 7f6a0c21e740 10 client.15825 fetch_fsmap finished waiting for FSMap version 30 2024-04-17T06:06:16.668 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.664+0000 7f6a0c21e740 10 client.15825 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-17T06:06:16.668 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.664+0000 7f6a0c21e740 20 client.15825 populate_metadata read hostname 'smithi047' 2024-04-17T06:06:16.668 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.664+0000 7f6a0c21e740 1 --2- 172.21.15.47:0/480154999 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x555fa4bb2f00 0x555fa4bb5320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:06:16.668 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.664+0000 7f6a0c21e740 4 client.15825 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-17T06:06:16.668 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.664+0000 7f6a0c21e740 1 -- 172.21.15.47:0/480154999 --> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555fa4aa0060 con 0x555fa4bb2f00 2024-04-17T06:06:16.668 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.665+0000 7f6a02ffd640 1 --2- 172.21.15.47:0/480154999 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x555fa4bb2f00 0x555fa4bb5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:06:16.668 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.665+0000 7f69ea7fc640 10 client.15825 ms_handle_connect on v2:172.21.15.98:6838/2841287917 2024-04-17T06:06:16.668 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.665+0000 7f6a02ffd640 1 --2- 172.21.15.47:0/480154999 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x555fa4bb2f00 0x555fa4bb5320 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1808bdfd6bbffe15 server_cookie=7cbfc41ab0ca4903 in_seq=0 out_seq=0 2024-04-17T06:06:16.671 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.672+0000 7f69ea7fc640 1 -- 172.21.15.47:0/480154999 <== mds.0 v2:172.21.15.98:6838/2841287917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555fa4aa0060 con 0x555fa4bb2f00 2024-04-17T06:06:16.671 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.672+0000 7f69ea7fc640 10 client.15825 handle_command_reply: tid=0 2024-04-17T06:06:16.671 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.672+0000 7f6a0c21e740 1 client.15825 shutdown 2024-04-17T06:06:16.671 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.672+0000 7f6a0c21e740 1 -- 172.21.15.47:0/480154999 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555fa4b339e0 msgr2=0x555fa4b4f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:06:16.671 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.672+0000 7f6a0c21e740 1 --2- 172.21.15.47:0/480154999 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555fa4b339e0 0x555fa4b4f590 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f69ec00e9c0 tx=0x7f69ec09cd40 comp rx=0 tx=0).stop 2024-04-17T06:06:16.671 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.672+0000 7f6a0c21e740 1 -- 172.21.15.47:0/480154999 shutdown_connections 2024-04-17T06:06:16.671 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.672+0000 7f6a0c21e740 1 --2- 172.21.15.47:0/480154999 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x555fa4bb2f00 0x555fa4bb5320 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:16.671 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.672+0000 7f6a0c21e740 1 --2- 172.21.15.47:0/480154999 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555fa4b3c7b0 0x555fa4b4fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:16.672 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.672+0000 7f6a0c21e740 1 --2- 172.21.15.47:0/480154999 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555fa4b339e0 0x555fa4b4f590 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:16.672 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.672+0000 7f69ea7fc640 0 client.15825 ms_handle_reset on v2:172.21.15.98:6838/2841287917 2024-04-17T06:06:16.672 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.672+0000 7f6a0c21e740 1 --2- 172.21.15.47:0/480154999 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555fa4b330c0 0x555fa4b52200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:16.672 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.672+0000 7f6a0c21e740 1 -- 172.21.15.47:0/480154999 >> 172.21.15.47:0/480154999 conn(0x555fa4b2d9e0 msgr2=0x555fa4b2cb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:06:16.672 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.672+0000 7f6a0c21e740 1 -- 172.21.15.47:0/480154999 shutdown_connections 2024-04-17T06:06:16.672 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.672+0000 7f6a0c21e740 1 -- 172.21.15.47:0/480154999 wait complete. 2024-04-17T06:06:16.672 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.672+0000 7f6a0c21e740 20 client.15825 trim_cache size 0 max 16384 2024-04-17T06:06:16.672 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-04-17T06:06:16.672 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-04-17T06:06:16.672 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-04-17T06:06:16.672 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-04-17T06:06:16.673 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.672+0000 7f6a09635640 1 -- 172.21.15.47:0/1934365955 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f69f00a3df0 msgr2=0x7f69f00a62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:06:16.673 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.672+0000 7f6a09635640 1 --2- 172.21.15.47:0/1934365955 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f69f00a3df0 0x7f69f00a62b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f69fc005fd0 tx=0x7f69fc011440 comp rx=0 tx=0).stop 2024-04-17T06:06:16.673 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.672+0000 7f6a09635640 1 -- 172.21.15.47:0/1934365955 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a04101b70 msgr2=0x7f6a0410ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:06:16.673 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.672+0000 7f6a09635640 1 --2- 172.21.15.47:0/1934365955 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a04101b70 0x7f6a0410ec50 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f69ec00e9f0 tx=0x7f69ec00eeb0 comp rx=0 tx=0).stop 2024-04-17T06:06:16.674 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.673+0000 7f6a09635640 1 -- 172.21.15.47:0/1934365955 shutdown_connections 2024-04-17T06:06:16.674 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.673+0000 7f6a09635640 1 --2- 172.21.15.47:0/1934365955 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6a04106fc0 0x7f6a0410f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:16.674 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.673+0000 7f6a09635640 1 --2- 172.21.15.47:0/1934365955 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f69f00a3df0 0x7f69f00a62b0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:16.674 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.673+0000 7f6a09635640 1 --2- 172.21.15.47:0/1934365955 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a04101b70 0x7f6a0410ec50 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:16.674 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.673+0000 7f6a09635640 1 --2- 172.21.15.47:0/1934365955 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6a041011b0 0x7f6a0410e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:16.674 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.673+0000 7f6a09635640 1 -- 172.21.15.47:0/1934365955 >> 172.21.15.47:0/1934365955 conn(0x7f6a04076c60 msgr2=0x7f6a040ffe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:06:16.674 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.673+0000 7f6a09635640 1 -- 172.21.15.47:0/1934365955 shutdown_connections 2024-04-17T06:06:16.674 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:16.673+0000 7f6a09635640 1 -- 172.21.15.47:0/1934365955 wait complete. 2024-04-17T06:06:17.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:06:16 smithi103 ceph-mon[36243]: mds.? [v2:172.21.15.98:6834/194247764,v1:172.21.15.98:6835/194247764] up:boot 2024-04-17T06:06:17.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:06:16 smithi103 ceph-mon[36243]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-17T06:06:17.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:06:16 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-17T06:06:17.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:06:16 smithi047 ceph-mon[32223]: mds.? [v2:172.21.15.98:6834/194247764,v1:172.21.15.98:6835/194247764] up:boot 2024-04-17T06:06:17.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:06:16 smithi047 ceph-mon[32223]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-17T06:06:17.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:06:16 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-17T06:06:17.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:06:16 smithi098 ceph-mon[36516]: mds.? [v2:172.21.15.98:6834/194247764,v1:172.21.15.98:6835/194247764] up:boot 2024-04-17T06:06:17.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:06:16 smithi098 ceph-mon[36516]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-17T06:06:17.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:06:16 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-17T06:06:18.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:06:17 smithi103 ceph-mon[36243]: pgmap v154: 97 pgs: 97 active+clean; 2.6 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 7.5 KiB/s wr, 7 op/s 2024-04-17T06:06:18.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:06:17 smithi098 ceph-mon[36516]: pgmap v154: 97 pgs: 97 active+clean; 2.6 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 7.5 KiB/s wr, 7 op/s 2024-04-17T06:06:18.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:06:17 smithi047 ceph-mon[32223]: pgmap v154: 97 pgs: 97 active+clean; 2.6 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 7.5 KiB/s wr, 7 op/s 2024-04-17T06:06:20.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:06:19 smithi103 ceph-mon[36243]: pgmap v155: 97 pgs: 97 active+clean; 2.6 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 7.5 KiB/s wr, 7 op/s 2024-04-17T06:06:20.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:06:19 smithi098 ceph-mon[36516]: pgmap v155: 97 pgs: 97 active+clean; 2.6 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 7.5 KiB/s wr, 7 op/s 2024-04-17T06:06:20.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:06:19 smithi047 ceph-mon[32223]: pgmap v155: 97 pgs: 97 active+clean; 2.6 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 7.5 KiB/s wr, 7 op/s 2024-04-17T06:06:22.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:06:21 smithi103 ceph-mon[36243]: pgmap v156: 97 pgs: 97 active+clean; 2.7 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-04-17T06:06:22.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:06:21 smithi098 ceph-mon[36516]: pgmap v156: 97 pgs: 97 active+clean; 2.7 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-04-17T06:06:22.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:06:21 smithi047 ceph-mon[32223]: pgmap v156: 97 pgs: 97 active+clean; 2.7 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-04-17T06:06:24.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:06:23 smithi103 ceph-mon[36243]: pgmap v157: 97 pgs: 97 active+clean; 2.7 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-04-17T06:06:24.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:06:23 smithi098 ceph-mon[36516]: pgmap v157: 97 pgs: 97 active+clean; 2.7 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-04-17T06:06:24.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:06:23 smithi047 ceph-mon[32223]: pgmap v157: 97 pgs: 97 active+clean; 2.7 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-04-17T06:06:26.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:06:25 smithi103 ceph-mon[36243]: pgmap v158: 97 pgs: 97 active+clean; 2.7 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 KiB/s wr, 16 op/s 2024-04-17T06:06:26.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:06:25 smithi098 ceph-mon[36516]: pgmap v158: 97 pgs: 97 active+clean; 2.7 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 KiB/s wr, 16 op/s 2024-04-17T06:06:26.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:06:25 smithi047 ceph-mon[32223]: pgmap v158: 97 pgs: 97 active+clean; 2.7 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 KiB/s wr, 16 op/s 2024-04-17T06:06:28.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:06:27 smithi103 ceph-mon[36243]: pgmap v159: 97 pgs: 97 active+clean; 2.7 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 12 op/s 2024-04-17T06:06:28.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:06:27 smithi098 ceph-mon[36516]: pgmap v159: 97 pgs: 97 active+clean; 2.7 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 12 op/s 2024-04-17T06:06:28.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:06:27 smithi047 ceph-mon[32223]: pgmap v159: 97 pgs: 97 active+clean; 2.7 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 12 op/s 2024-04-17T06:06:29.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:06:28 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T06:06:29.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:06:28 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T06:06:29.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:06:28 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T06:06:29.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:06:28 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:06:29.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:06:28 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T06:06:29.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:06:28 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T06:06:29.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:06:28 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T06:06:29.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:06:28 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:06:29.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:06:28 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T06:06:29.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:06:28 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T06:06:29.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:06:28 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T06:06:29.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:06:28 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:06:29.775 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.775+0000 7f3383f8f640 1 -- 172.21.15.47:0/693975525 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f337c102b90 msgr2=0x7f337c106bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:06:29.775 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.775+0000 7f3383f8f640 1 --2- 172.21.15.47:0/693975525 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f337c102b90 0x7f337c106bd0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f336c009900 tx=0x7f336c02bc30 comp rx=0 tx=0).stop 2024-04-17T06:06:29.775 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.775+0000 7f3383f8f640 1 -- 172.21.15.47:0/693975525 shutdown_connections 2024-04-17T06:06:29.775 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.775+0000 7f3383f8f640 1 --2- 172.21.15.47:0/693975525 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f337c1072b0 0x7f337c10ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:29.775 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.775+0000 7f3383f8f640 1 --2- 172.21.15.47:0/693975525 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f337c102b90 0x7f337c106bd0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:29.775 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.775+0000 7f3383f8f640 1 --2- 172.21.15.47:0/693975525 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f337c1021e0 0x7f337c1025c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:29.775 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.775+0000 7f3383f8f640 1 -- 172.21.15.47:0/693975525 >> 172.21.15.47:0/693975525 conn(0x7f337c0793f0 msgr2=0x7f337c079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:06:29.776 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.775+0000 7f3383f8f640 1 -- 172.21.15.47:0/693975525 shutdown_connections 2024-04-17T06:06:29.776 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.775+0000 7f3383f8f640 1 -- 172.21.15.47:0/693975525 wait complete. 2024-04-17T06:06:29.776 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.775+0000 7f3383f8f640 1 Processor -- start 2024-04-17T06:06:29.776 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.775+0000 7f3383f8f640 1 -- start start 2024-04-17T06:06:29.776 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.776+0000 7f3383f8f640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f337c1021e0 0x7f337c19e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:06:29.776 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.776+0000 7f3383f8f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f337c1072b0 0x7f337c19e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:06:29.776 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.776+0000 7f3383f8f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f337c1985a0 0x7f337c198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:06:29.776 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.776+0000 7f3383f8f640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f337c112340 con 0x7f337c1072b0 2024-04-17T06:06:29.776 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.776+0000 7f3383f8f640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f337c1121c0 con 0x7f337c1021e0 2024-04-17T06:06:29.776 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.776+0000 7f3383f8f640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f337c1124c0 con 0x7f337c1985a0 2024-04-17T06:06:29.776 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.776+0000 7f338152a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f337c1072b0 0x7f337c19e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:06:29.776 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.776+0000 7f338152a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f337c1072b0 0x7f337c19e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43364/0 (socket says 172.21.15.47:43364) 2024-04-17T06:06:29.777 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.776+0000 7f338152a640 1 -- 172.21.15.47:0/1924738857 learned_addr learned my addr 172.21.15.47:0/1924738857 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:06:29.777 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.776+0000 7f3381d2b640 1 --2- 172.21.15.47:0/1924738857 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f337c1021e0 0x7f337c19e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:06:29.777 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.776+0000 7f338252c640 1 --2- 172.21.15.47:0/1924738857 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f337c1985a0 0x7f337c198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:06:29.777 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.776+0000 7f338152a640 1 -- 172.21.15.47:0/1924738857 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f337c1985a0 msgr2=0x7f337c198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:06:29.777 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.776+0000 7f338152a640 1 --2- 172.21.15.47:0/1924738857 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f337c1985a0 0x7f337c198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:29.777 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.776+0000 7f338152a640 1 -- 172.21.15.47:0/1924738857 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f337c1021e0 msgr2=0x7f337c19e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:06:29.777 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.776+0000 7f338152a640 1 --2- 172.21.15.47:0/1924738857 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f337c1021e0 0x7f337c19e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:29.777 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.776+0000 7f338152a640 1 -- 172.21.15.47:0/1924738857 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f337c1990c0 con 0x7f337c1072b0 2024-04-17T06:06:29.777 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.776+0000 7f338152a640 1 --2- 172.21.15.47:0/1924738857 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f337c1072b0 0x7f337c19e930 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f336c036040 tx=0x7f336c02cb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:06:29.777 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.777+0000 7f3372ffd640 1 -- 172.21.15.47:0/1924738857 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f336c041070 con 0x7f337c1072b0 2024-04-17T06:06:29.777 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.777+0000 7f3372ffd640 1 -- 172.21.15.47:0/1924738857 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f336c004450 con 0x7f337c1072b0 2024-04-17T06:06:29.778 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.777+0000 7f3383f8f640 1 -- 172.21.15.47:0/1924738857 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f337c199350 con 0x7f337c1072b0 2024-04-17T06:06:29.778 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.777+0000 7f3372ffd640 1 -- 172.21.15.47:0/1924738857 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f336c003dc0 con 0x7f337c1072b0 2024-04-17T06:06:29.778 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.777+0000 7f3383f8f640 1 -- 172.21.15.47:0/1924738857 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f337c1a5540 con 0x7f337c1072b0 2024-04-17T06:06:29.785 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.779+0000 7f3372ffd640 1 -- 172.21.15.47:0/1924738857 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f336c044070 con 0x7f337c1072b0 2024-04-17T06:06:29.785 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.779+0000 7f3386b79740 1 Processor -- start 2024-04-17T06:06:29.785 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.780+0000 7f3386b79740 1 -- start start 2024-04-17T06:06:29.785 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.780+0000 7f3386b79740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a1136630c0 0x55a1136634a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:06:29.785 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.780+0000 7f3386b79740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a1136639e0 0x55a11366c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:06:29.785 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.780+0000 7f3386b79740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a11366c7b0 0x55a11366eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:06:29.785 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.780+0000 7f3386b79740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55a1135c87d0 con 0x55a1136630c0 2024-04-17T06:06:29.785 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.780+0000 7f3386b79740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55a113565f70 con 0x55a11366c7b0 2024-04-17T06:06:29.785 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.780+0000 7f3386b79740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55a1135c4790 con 0x55a1136639e0 2024-04-17T06:06:29.786 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.780+0000 7f338252c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a1136630c0 0x55a1136634a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:06:29.786 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.780+0000 7f338252c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a1136630c0 0x55a1136634a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43370/0 (socket says 172.21.15.47:43370) 2024-04-17T06:06:29.786 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.780+0000 7f338252c640 1 -- 172.21.15.47:0/1734747831 learned_addr learned my addr 172.21.15.47:0/1734747831 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:06:29.786 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.780+0000 7f3372ffd640 1 --2- 172.21.15.47:0/1924738857 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f33680a3dc0 0x7f33680a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:06:29.786 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.780+0000 7f3381d2b640 1 --2- 172.21.15.47:0/1734747831 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a1136639e0 0x55a11366c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:06:29.786 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.780+0000 7f338252c640 1 --2- 172.21.15.47:0/1924738857 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f33680a3dc0 0x7f33680a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:06:29.786 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.780+0000 7f3372ffd640 1 -- 172.21.15.47:0/1924738857 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f336c0f2290 con 0x7f337c1072b0 2024-04-17T06:06:29.786 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.780+0000 7f338152a640 1 --2- 172.21.15.47:0/1734747831 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a11366c7b0 0x55a11366eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:06:29.786 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.780+0000 7f338252c640 1 -- 172.21.15.47:0/1734747831 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a1136639e0 msgr2=0x55a11366c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:06:29.786 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.780+0000 7f338252c640 1 --2- 172.21.15.47:0/1734747831 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a1136639e0 0x55a11366c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:29.786 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.780+0000 7f338252c640 1 -- 172.21.15.47:0/1734747831 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a11366c7b0 msgr2=0x55a11366eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:06:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.780+0000 7f338252c640 1 --2- 172.21.15.47:0/1734747831 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a11366c7b0 0x55a11366eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.780+0000 7f338252c640 1 -- 172.21.15.47:0/1734747831 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1132db210 con 0x55a1136630c0 2024-04-17T06:06:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.780+0000 7f3381d2b640 1 --2- 172.21.15.47:0/1734747831 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a1136639e0 0x55a11366c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T06:06:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.780+0000 7f338152a640 1 --2- 172.21.15.47:0/1734747831 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a11366c7b0 0x55a11366eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:06:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.781+0000 7f338252c640 1 --2- 172.21.15.47:0/1734747831 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a1136630c0 0x55a1136634a0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f33740139d0 tx=0x7f3374013e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22491267cd8aeee2 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:06:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.781+0000 7f3367fff640 1 -- 172.21.15.47:0/1734747831 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3374004230 con 0x55a1136630c0 2024-04-17T06:06:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.781+0000 7f3367fff640 1 -- 172.21.15.47:0/1734747831 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f337401f480 con 0x55a1136630c0 2024-04-17T06:06:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.781+0000 7f3367fff640 1 -- 172.21.15.47:0/1734747831 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f337401e620 con 0x55a1136630c0 2024-04-17T06:06:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.782+0000 7f3386b79740 1 -- 172.21.15.47:0/1734747831 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a1136630c0 msgr2=0x55a1136634a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:06:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.782+0000 7f338252c640 1 --2- 172.21.15.47:0/1924738857 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f33680a3dc0 0x7f33680a6280 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3374023000 tx=0x7f3374021000 comp rx=0 tx=0).ready entity=mgr.24923 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:06:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.782+0000 7f3386b79740 1 --2- 172.21.15.47:0/1734747831 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a1136630c0 0x55a1136634a0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f33740139d0 tx=0x7f3374013e90 comp rx=0 tx=0).stop 2024-04-17T06:06:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.782+0000 7f3386b79740 1 -- 172.21.15.47:0/1734747831 shutdown_connections 2024-04-17T06:06:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.782+0000 7f3386b79740 1 --2- 172.21.15.47:0/1734747831 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a11366c7b0 0x55a11366eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:29.788 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.782+0000 7f3386b79740 1 --2- 172.21.15.47:0/1734747831 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a1136639e0 0x55a11366c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:29.788 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.782+0000 7f3386b79740 1 --2- 172.21.15.47:0/1734747831 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a1136630c0 0x55a1136634a0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:29.788 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.782+0000 7f3386b79740 1 -- 172.21.15.47:0/1734747831 >> 172.21.15.47:0/1734747831 conn(0x55a11365d9e0 msgr2=0x55a11365ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:06:29.788 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.782+0000 7f3386b79740 1 -- 172.21.15.47:0/1734747831 shutdown_connections 2024-04-17T06:06:29.788 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.782+0000 7f3386b79740 1 -- 172.21.15.47:0/1734747831 wait complete. 2024-04-17T06:06:29.788 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.782+0000 7f3386b79740 1 -- start start 2024-04-17T06:06:29.788 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.782+0000 7f3386b79740 1 Processor -- start 2024-04-17T06:06:29.788 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.783+0000 7f3386b79740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a1136630c0 0x55a113669850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:06:29.788 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.783+0000 7f3386b79740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a1136639e0 0x55a113669d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:06:29.788 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.783+0000 7f3386b79740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a11366c7b0 0x55a11366a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:06:29.788 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.783+0000 7f3386b79740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55a11366f3e0 con 0x55a11366c7b0 2024-04-17T06:06:29.788 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.783+0000 7f3386b79740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55a113550e90 con 0x55a1136639e0 2024-04-17T06:06:29.789 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.783+0000 7f3386b79740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55a11366f560 con 0x55a1136630c0 2024-04-17T06:06:29.789 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.783+0000 7f3386b79740 10 client.0 _ll_register_callbacks cb 0x55a113566d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:06:29.789 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.783+0000 7f338252c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a11366c7b0 0x55a11366a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:06:29.789 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.783+0000 7f338252c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a11366c7b0 0x55a11366a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43374/0 (socket says 172.21.15.47:43374) 2024-04-17T06:06:29.789 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.783+0000 7f338252c640 1 -- 172.21.15.47:0/935895325 learned_addr learned my addr 172.21.15.47:0/935895325 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:06:29.789 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.783+0000 7f3381d2b640 1 --2- 172.21.15.47:0/935895325 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a1136630c0 0x55a113669850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:06:29.789 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.783+0000 7f338152a640 1 --2- 172.21.15.47:0/935895325 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a1136639e0 0x55a113669d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:06:29.789 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.783+0000 7f338252c640 1 -- 172.21.15.47:0/935895325 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a1136630c0 msgr2=0x55a113669850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:06:29.789 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.783+0000 7f338252c640 1 --2- 172.21.15.47:0/935895325 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a1136630c0 0x55a113669850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:29.789 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.783+0000 7f338252c640 1 -- 172.21.15.47:0/935895325 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a1136639e0 msgr2=0x55a113669d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:06:29.789 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.783+0000 7f338252c640 1 --2- 172.21.15.47:0/935895325 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a1136639e0 0x55a113669d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:29.790 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.783+0000 7f338252c640 1 -- 172.21.15.47:0/935895325 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1133a7620 con 0x55a11366c7b0 2024-04-17T06:06:29.790 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.783+0000 7f338252c640 1 --2- 172.21.15.47:0/935895325 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a11366c7b0 0x55a11366a2d0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f3374014fd0 tx=0x7f337401f8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fe6e1f39e5df161 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:06:29.790 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.783+0000 7f33657fa640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-17T06:06:29.790 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.783+0000 7f33657fa640 1 -- 172.21.15.47:0/935895325 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3374038060 con 0x55a11366c7b0 2024-04-17T06:06:29.790 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.784+0000 7f33657fa640 1 -- 172.21.15.47:0/935895325 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f3374011880 con 0x55a11366c7b0 2024-04-17T06:06:29.790 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.784+0000 7f3386b79740 1 -- 172.21.15.47:0/935895325 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f337401f480 con 0x55a11366c7b0 2024-04-17T06:06:29.790 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.784+0000 7f33657fa640 1 -- 172.21.15.47:0/935895325 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3374011a80 con 0x55a11366c7b0 2024-04-17T06:06:29.790 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.784+0000 7f3386b79740 10 client.15837 fetch_fsmap learned FSMap version 30 2024-04-17T06:06:29.790 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.784+0000 7f3386b79740 1 -- 172.21.15.47:0/935895325 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a11367b490 con 0x55a11366c7b0 2024-04-17T06:06:29.790 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.784+0000 7f33657fa640 1 -- 172.21.15.47:0/935895325 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f337401f480 con 0x55a11366c7b0 2024-04-17T06:06:29.790 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.784+0000 7f33657fa640 1 -- 172.21.15.47:0/935895325 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 30) ==== 8339+0+0 (secure 0 0 0) 0x7f3374028940 con 0x55a11366c7b0 2024-04-17T06:06:29.791 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.784+0000 7f3386b79740 10 client.15837 fetch_fsmap finished waiting for FSMap version 30 2024-04-17T06:06:29.791 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.784+0000 7f3386b79740 10 client.15837 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-17T06:06:29.791 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.784+0000 7f3386b79740 20 client.15837 populate_metadata read hostname 'smithi047' 2024-04-17T06:06:29.791 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.784+0000 7f3386b79740 1 --2- 172.21.15.47:0/935895325 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55a11367c7e0 0x55a11367ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:06:29.791 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.784+0000 7f3386b79740 4 client.15837 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T06:06:29.791 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.784+0000 7f3386b79740 1 -- 172.21.15.47:0/935895325 --> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a1135d61f0 con 0x55a11367c7e0 2024-04-17T06:06:29.791 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.785+0000 7f3381d2b640 1 --2- 172.21.15.47:0/935895325 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55a11367c7e0 0x55a11367ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:06:29.791 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.785+0000 7f3381d2b640 1 --2- 172.21.15.47:0/935895325 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55a11367c7e0 0x55a11367ebd0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59b8033f98dbd17b server_cookie=10ae81ec9eede6a0 in_seq=0 out_seq=0 2024-04-17T06:06:29.791 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.785+0000 7f33657fa640 10 client.15837 ms_handle_connect on v2:172.21.15.98:6838/2841287917 2024-04-17T06:06:29.791 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.786+0000 7f33657fa640 1 -- 172.21.15.47:0/935895325 <== mds.0 v2:172.21.15.98:6838/2841287917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a1135d61f0 con 0x55a11367c7e0 2024-04-17T06:06:29.791 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.786+0000 7f33657fa640 10 client.15837 handle_command_reply: tid=0 2024-04-17T06:06:29.791 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.786+0000 7f3386b79740 1 client.15837 shutdown 2024-04-17T06:06:29.791 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.786+0000 7f3386b79740 1 -- 172.21.15.47:0/935895325 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a11366c7b0 msgr2=0x55a11366a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:06:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.786+0000 7f3386b79740 1 --2- 172.21.15.47:0/935895325 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a11366c7b0 0x55a11366a2d0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f3374014fd0 tx=0x7f337401f8e0 comp rx=0 tx=0).stop 2024-04-17T06:06:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.786+0000 7f3386b79740 1 -- 172.21.15.47:0/935895325 shutdown_connections 2024-04-17T06:06:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.786+0000 7f3386b79740 1 --2- 172.21.15.47:0/935895325 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55a11367c7e0 0x55a11367ebd0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.786+0000 7f3386b79740 1 --2- 172.21.15.47:0/935895325 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a11366c7b0 0x55a11366a2d0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.786+0000 7f3386b79740 1 --2- 172.21.15.47:0/935895325 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a1136639e0 0x55a113669d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.786+0000 7f33657fa640 0 client.15837 ms_handle_reset on v2:172.21.15.98:6838/2841287917 2024-04-17T06:06:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.786+0000 7f3386b79740 1 --2- 172.21.15.47:0/935895325 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a1136630c0 0x55a113669850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.786+0000 7f3386b79740 1 -- 172.21.15.47:0/935895325 >> 172.21.15.47:0/935895325 conn(0x55a11365d9e0 msgr2=0x55a11365f730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:06:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.786+0000 7f3386b79740 1 -- 172.21.15.47:0/935895325 shutdown_connections 2024-04-17T06:06:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.786+0000 7f3386b79740 1 -- 172.21.15.47:0/935895325 wait complete. 2024-04-17T06:06:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.786+0000 7f3386b79740 20 client.15837 trim_cache size 0 max 16384 2024-04-17T06:06:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.801+0000 7f3386b79740 1 Processor -- start 2024-04-17T06:06:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.801+0000 7f3386b79740 1 -- start start 2024-04-17T06:06:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.801+0000 7f3386b79740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a1136630c0 0x55a1136782a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:06:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.801+0000 7f3386b79740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a1136639e0 0x55a11366a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:06:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.801+0000 7f3386b79740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a11366c7b0 0x55a113684400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:06:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.801+0000 7f3386b79740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f337c1b0430 con 0x55a1136630c0 2024-04-17T06:06:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.801+0000 7f3386b79740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f337c1adcf0 con 0x55a11366c7b0 2024-04-17T06:06:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.801+0000 7f3386b79740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f337c1abb60 con 0x55a1136639e0 2024-04-17T06:06:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.801+0000 7f3381d2b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a1136630c0 0x55a1136782a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:06:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.801+0000 7f3381d2b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a1136630c0 0x55a1136782a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43384/0 (socket says 172.21.15.47:43384) 2024-04-17T06:06:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.801+0000 7f3381d2b640 1 -- 172.21.15.47:0/592258375 learned_addr learned my addr 172.21.15.47:0/592258375 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:06:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.802+0000 7f3381d2b640 1 -- 172.21.15.47:0/592258375 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a1136639e0 msgr2=0x55a11366a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:06:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.802+0000 7f3381d2b640 1 --2- 172.21.15.47:0/592258375 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a1136639e0 0x55a11366a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.802+0000 7f3381d2b640 1 -- 172.21.15.47:0/592258375 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a11366c7b0 msgr2=0x55a113684400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:06:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.802+0000 7f3381d2b640 1 --2- 172.21.15.47:0/592258375 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a11366c7b0 0x55a113684400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.802+0000 7f3381d2b640 1 -- 172.21.15.47:0/592258375 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a113668820 con 0x55a1136630c0 2024-04-17T06:06:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.802+0000 7f3381d2b640 1 --2- 172.21.15.47:0/592258375 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a1136630c0 0x55a1136782a0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f3378004700 tx=0x7f3378005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bb660d95648b00b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:06:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.802+0000 7f33677fe640 1 -- 172.21.15.47:0/592258375 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3378024020 con 0x55a1136630c0 2024-04-17T06:06:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.802+0000 7f33677fe640 1 -- 172.21.15.47:0/592258375 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3378005020 con 0x55a1136630c0 2024-04-17T06:06:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.802+0000 7f33677fe640 1 -- 172.21.15.47:0/592258375 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3378004b20 con 0x55a1136630c0 2024-04-17T06:06:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.803+0000 7f3386b79740 1 -- 172.21.15.47:0/592258375 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a1136630c0 msgr2=0x55a1136782a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:06:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.803+0000 7f3386b79740 1 --2- 172.21.15.47:0/592258375 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a1136630c0 0x55a1136782a0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f3378004700 tx=0x7f3378005f90 comp rx=0 tx=0).stop 2024-04-17T06:06:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.803+0000 7f3386b79740 1 -- 172.21.15.47:0/592258375 shutdown_connections 2024-04-17T06:06:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.803+0000 7f3386b79740 1 --2- 172.21.15.47:0/592258375 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a11366c7b0 0x55a113684400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.803+0000 7f3386b79740 1 --2- 172.21.15.47:0/592258375 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a1136639e0 0x55a11366a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.803+0000 7f3386b79740 1 --2- 172.21.15.47:0/592258375 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a1136630c0 0x55a1136782a0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.803+0000 7f3386b79740 1 -- 172.21.15.47:0/592258375 >> 172.21.15.47:0/592258375 conn(0x55a11365d9e0 msgr2=0x55a11365cb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:06:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.803+0000 7f3386b79740 1 -- 172.21.15.47:0/592258375 shutdown_connections 2024-04-17T06:06:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.803+0000 7f3386b79740 1 -- 172.21.15.47:0/592258375 wait complete. 2024-04-17T06:06:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.803+0000 7f3386b79740 1 -- start start 2024-04-17T06:06:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.803+0000 7f3386b79740 1 Processor -- start 2024-04-17T06:06:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.803+0000 7f3386b79740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a1136630c0 0x55a113674ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:06:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.804+0000 7f3386b79740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a1136639e0 0x55a113666280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:06:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.804+0000 7f3386b79740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a11366c7b0 0x55a1136667c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:06:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.804+0000 7f3386b79740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f337c1ac1b0 con 0x55a11366c7b0 2024-04-17T06:06:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.804+0000 7f3386b79740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f337c1ad620 con 0x55a1136639e0 2024-04-17T06:06:29.807 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.804+0000 7f3386b79740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f337c1ac880 con 0x55a1136630c0 2024-04-17T06:06:29.807 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.804+0000 7f3386b79740 10 client.0 _ll_register_callbacks cb 0x7f33780097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:06:29.807 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.804+0000 7f338252c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a11366c7b0 0x55a1136667c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:06:29.807 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.804+0000 7f338252c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a11366c7b0 0x55a1136667c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43392/0 (socket says 172.21.15.47:43392) 2024-04-17T06:06:29.807 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.804+0000 7f338252c640 1 -- 172.21.15.47:0/948570637 learned_addr learned my addr 172.21.15.47:0/948570637 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:06:29.807 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.804+0000 7f338152a640 1 --2- 172.21.15.47:0/948570637 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a1136639e0 0x55a113666280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:06:29.807 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.804+0000 7f3381d2b640 1 --2- 172.21.15.47:0/948570637 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a1136630c0 0x55a113674ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:06:29.807 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.804+0000 7f338252c640 1 -- 172.21.15.47:0/948570637 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a1136630c0 msgr2=0x55a113674ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:06:29.807 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.804+0000 7f338252c640 1 --2- 172.21.15.47:0/948570637 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a1136630c0 0x55a113674ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:29.807 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.804+0000 7f338252c640 1 -- 172.21.15.47:0/948570637 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a1136639e0 msgr2=0x55a113666280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:06:29.807 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.804+0000 7f338252c640 1 --2- 172.21.15.47:0/948570637 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a1136639e0 0x55a113666280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:29.807 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.804+0000 7f338252c640 1 -- 172.21.15.47:0/948570637 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a113663810 con 0x55a11366c7b0 2024-04-17T06:06:29.807 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.804+0000 7f3381d2b640 1 --2- 172.21.15.47:0/948570637 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a1136630c0 0x55a113674ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:06:29.807 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.804+0000 7f338252c640 1 --2- 172.21.15.47:0/948570637 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a11366c7b0 0x55a1136667c0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f3374002230 tx=0x7f3374005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de9045ed298033f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:06:29.808 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.804+0000 7f33657fa640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-17T06:06:29.808 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.804+0000 7f33657fa640 1 -- 172.21.15.47:0/948570637 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3374038070 con 0x55a11366c7b0 2024-04-17T06:06:29.808 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.804+0000 7f33657fa640 1 -- 172.21.15.47:0/948570637 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f337401faf0 con 0x55a11366c7b0 2024-04-17T06:06:29.808 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.804+0000 7f3386b79740 1 -- 172.21.15.47:0/948570637 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3378005020 con 0x55a11366c7b0 2024-04-17T06:06:29.808 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.805+0000 7f33657fa640 1 -- 172.21.15.47:0/948570637 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f337401f480 con 0x55a11366c7b0 2024-04-17T06:06:29.808 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.805+0000 7f3386b79740 10 client.15843 fetch_fsmap learned FSMap version 30 2024-04-17T06:06:29.808 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.805+0000 7f3386b79740 1 -- 172.21.15.47:0/948570637 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a1136e2b10 con 0x55a11366c7b0 2024-04-17T06:06:29.808 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.805+0000 7f33657fa640 1 -- 172.21.15.47:0/948570637 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3378005020 con 0x55a11366c7b0 2024-04-17T06:06:29.808 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.805+0000 7f33657fa640 1 -- 172.21.15.47:0/948570637 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 30) ==== 8339+0+0 (secure 0 0 0) 0x7f3374010600 con 0x55a11366c7b0 2024-04-17T06:06:29.808 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.805+0000 7f3386b79740 10 client.15843 fetch_fsmap finished waiting for FSMap version 30 2024-04-17T06:06:29.808 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.805+0000 7f3386b79740 10 client.15843 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-17T06:06:29.808 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.805+0000 7f3386b79740 20 client.15843 populate_metadata read hostname 'smithi047' 2024-04-17T06:06:29.808 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.805+0000 7f3386b79740 1 --2- 172.21.15.47:0/948570637 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55a1136e3290 0x55a1136e56b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:06:29.808 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.805+0000 7f3386b79740 4 client.15843 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-17T06:06:29.809 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.805+0000 7f3386b79740 1 -- 172.21.15.47:0/948570637 --> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a1135d0060 con 0x55a1136e3290 2024-04-17T06:06:29.809 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.805+0000 7f3381d2b640 1 --2- 172.21.15.47:0/948570637 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55a1136e3290 0x55a1136e56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:06:29.809 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.806+0000 7f33657fa640 10 client.15843 ms_handle_connect on v2:172.21.15.98:6838/2841287917 2024-04-17T06:06:29.809 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.806+0000 7f3381d2b640 1 --2- 172.21.15.47:0/948570637 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55a1136e3290 0x55a1136e56b0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98e077470dee37e1 server_cookie=f03d2f53c0eb72d8 in_seq=0 out_seq=0 2024-04-17T06:06:29.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.814+0000 7f33657fa640 1 -- 172.21.15.47:0/948570637 <== mds.0 v2:172.21.15.98:6838/2841287917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a1135d0060 con 0x55a1136e3290 2024-04-17T06:06:29.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.814+0000 7f33657fa640 10 client.15843 handle_command_reply: tid=0 2024-04-17T06:06:29.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.814+0000 7f3386b79740 1 client.15843 shutdown 2024-04-17T06:06:29.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.814+0000 7f3386b79740 1 -- 172.21.15.47:0/948570637 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a11366c7b0 msgr2=0x55a1136667c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:06:29.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.814+0000 7f3386b79740 1 --2- 172.21.15.47:0/948570637 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a11366c7b0 0x55a1136667c0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f3374002230 tx=0x7f3374005fd0 comp rx=0 tx=0).stop 2024-04-17T06:06:29.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.814+0000 7f3386b79740 1 -- 172.21.15.47:0/948570637 shutdown_connections 2024-04-17T06:06:29.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.814+0000 7f3386b79740 1 --2- 172.21.15.47:0/948570637 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55a1136e3290 0x55a1136e56b0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:29.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.814+0000 7f3386b79740 1 --2- 172.21.15.47:0/948570637 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a11366c7b0 0x55a1136667c0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:29.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.814+0000 7f3386b79740 1 --2- 172.21.15.47:0/948570637 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a1136639e0 0x55a113666280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:29.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.814+0000 7f33657fa640 0 client.15843 ms_handle_reset on v2:172.21.15.98:6838/2841287917 2024-04-17T06:06:29.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.814+0000 7f3386b79740 1 --2- 172.21.15.47:0/948570637 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a1136630c0 0x55a113674ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:29.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.814+0000 7f3386b79740 1 -- 172.21.15.47:0/948570637 >> 172.21.15.47:0/948570637 conn(0x55a11365d9e0 msgr2=0x55a11365ecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:06:29.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.814+0000 7f3386b79740 1 -- 172.21.15.47:0/948570637 shutdown_connections 2024-04-17T06:06:29.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.814+0000 7f3386b79740 1 -- 172.21.15.47:0/948570637 wait complete. 2024-04-17T06:06:29.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.814+0000 7f3386b79740 20 client.15843 trim_cache size 0 max 16384 2024-04-17T06:06:29.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-04-17T06:06:29.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-04-17T06:06:29.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-04-17T06:06:29.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-04-17T06:06:29.815 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.815+0000 7f3383f8f640 1 -- 172.21.15.47:0/1924738857 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f33680a3dc0 msgr2=0x7f33680a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:06:29.815 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.815+0000 7f3383f8f640 1 --2- 172.21.15.47:0/1924738857 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f33680a3dc0 0x7f33680a6280 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3374023000 tx=0x7f3374021000 comp rx=0 tx=0).stop 2024-04-17T06:06:29.815 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.815+0000 7f3383f8f640 1 -- 172.21.15.47:0/1924738857 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f337c1072b0 msgr2=0x7f337c19e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:06:29.815 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.815+0000 7f3383f8f640 1 --2- 172.21.15.47:0/1924738857 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f337c1072b0 0x7f337c19e930 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f336c036040 tx=0x7f336c02cb20 comp rx=0 tx=0).stop 2024-04-17T06:06:29.815 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.815+0000 7f3383f8f640 1 -- 172.21.15.47:0/1924738857 shutdown_connections 2024-04-17T06:06:29.815 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.815+0000 7f3383f8f640 1 --2- 172.21.15.47:0/1924738857 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f337c1985a0 0x7f337c198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:29.815 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.815+0000 7f3383f8f640 1 --2- 172.21.15.47:0/1924738857 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f33680a3dc0 0x7f33680a6280 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:29.815 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.815+0000 7f3383f8f640 1 --2- 172.21.15.47:0/1924738857 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f337c1072b0 0x7f337c19e930 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:29.815 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.815+0000 7f3383f8f640 1 --2- 172.21.15.47:0/1924738857 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f337c1021e0 0x7f337c19e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:29.815 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.815+0000 7f3383f8f640 1 -- 172.21.15.47:0/1924738857 >> 172.21.15.47:0/1924738857 conn(0x7f337c0793f0 msgr2=0x7f337c0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:06:29.815 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.815+0000 7f3383f8f640 1 -- 172.21.15.47:0/1924738857 shutdown_connections 2024-04-17T06:06:29.816 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:29.815+0000 7f3383f8f640 1 -- 172.21.15.47:0/1924738857 wait complete. 2024-04-17T06:06:30.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:06:29 smithi103 ceph-mon[36243]: pgmap v160: 97 pgs: 97 active+clean; 2.7 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 12 op/s 2024-04-17T06:06:30.134 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:06:29 smithi098 ceph-mon[36516]: pgmap v160: 97 pgs: 97 active+clean; 2.7 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 12 op/s 2024-04-17T06:06:30.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:06:29 smithi047 ceph-mon[32223]: pgmap v160: 97 pgs: 97 active+clean; 2.7 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 12 op/s 2024-04-17T06:06:32.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:06:31 smithi103 ceph-mon[36243]: pgmap v161: 97 pgs: 97 active+clean; 2.7 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-04-17T06:06:32.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:06:31 smithi098 ceph-mon[36516]: pgmap v161: 97 pgs: 97 active+clean; 2.7 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-04-17T06:06:32.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:06:31 smithi047 ceph-mon[32223]: pgmap v161: 97 pgs: 97 active+clean; 2.7 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-04-17T06:06:34.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:06:33 smithi103 ceph-mon[36243]: pgmap v162: 97 pgs: 97 active+clean; 2.7 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 7.5 KiB/s wr, 9 op/s 2024-04-17T06:06:34.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:06:33 smithi098 ceph-mon[36516]: pgmap v162: 97 pgs: 97 active+clean; 2.7 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 7.5 KiB/s wr, 9 op/s 2024-04-17T06:06:34.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:06:33 smithi047 ceph-mon[32223]: pgmap v162: 97 pgs: 97 active+clean; 2.7 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 7.5 KiB/s wr, 9 op/s 2024-04-17T06:06:36.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:06:35 smithi103 ceph-mon[36243]: pgmap v163: 97 pgs: 97 active+clean; 2.8 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 16 KiB/s wr, 16 op/s 2024-04-17T06:06:36.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:06:35 smithi098 ceph-mon[36516]: pgmap v163: 97 pgs: 97 active+clean; 2.8 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 16 KiB/s wr, 16 op/s 2024-04-17T06:06:36.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:06:35 smithi047 ceph-mon[32223]: pgmap v163: 97 pgs: 97 active+clean; 2.8 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 16 KiB/s wr, 16 op/s 2024-04-17T06:06:38.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:06:37 smithi103 ceph-mon[36243]: pgmap v164: 97 pgs: 97 active+clean; 2.8 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-04-17T06:06:38.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:06:37 smithi098 ceph-mon[36516]: pgmap v164: 97 pgs: 97 active+clean; 2.8 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-04-17T06:06:38.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:06:37 smithi047 ceph-mon[32223]: pgmap v164: 97 pgs: 97 active+clean; 2.8 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-04-17T06:06:40.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:06:39 smithi103 ceph-mon[36243]: pgmap v165: 97 pgs: 97 active+clean; 2.8 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-04-17T06:06:40.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:06:39 smithi098 ceph-mon[36516]: pgmap v165: 97 pgs: 97 active+clean; 2.8 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-04-17T06:06:40.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:06:39 smithi047 ceph-mon[32223]: pgmap v165: 97 pgs: 97 active+clean; 2.8 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-04-17T06:06:42.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:06:41 smithi103 ceph-mon[36243]: pgmap v166: 97 pgs: 97 active+clean; 2.8 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-04-17T06:06:42.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:06:41 smithi098 ceph-mon[36516]: pgmap v166: 97 pgs: 97 active+clean; 2.8 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-04-17T06:06:42.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:06:41 smithi047 ceph-mon[32223]: pgmap v166: 97 pgs: 97 active+clean; 2.8 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-04-17T06:06:42.915 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.915+0000 7f9fb89f4640 1 -- 172.21.15.47:0/794549010 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9fb406a790 msgr2=0x7f9fb406abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:06:42.915 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.915+0000 7f9fb89f4640 1 --2- 172.21.15.47:0/794549010 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9fb406a790 0x7f9fb406abf0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c009900 tx=0x7f9f9c02bcd0 comp rx=0 tx=0).stop 2024-04-17T06:06:42.915 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.915+0000 7f9fb89f4640 1 -- 172.21.15.47:0/794549010 shutdown_connections 2024-04-17T06:06:42.915 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.915+0000 7f9fb89f4640 1 --2- 172.21.15.47:0/794549010 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9fb406b130 0x7f9fb410ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:42.915 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.915+0000 7f9fb89f4640 1 --2- 172.21.15.47:0/794549010 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9fb406a790 0x7f9fb406abf0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:42.915 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.915+0000 7f9fb89f4640 1 --2- 172.21.15.47:0/794549010 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9fb41080f0 0x7f9fb41084d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:42.915 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.915+0000 7f9fb89f4640 1 -- 172.21.15.47:0/794549010 >> 172.21.15.47:0/794549010 conn(0x7f9fb4076c60 msgr2=0x7f9fb4077070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:06:42.915 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.915+0000 7f9fb89f4640 1 -- 172.21.15.47:0/794549010 shutdown_connections 2024-04-17T06:06:42.916 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.915+0000 7f9fb89f4640 1 -- 172.21.15.47:0/794549010 wait complete. 2024-04-17T06:06:42.916 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.915+0000 7f9fb89f4640 1 Processor -- start 2024-04-17T06:06:42.916 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.916+0000 7f9fb89f4640 1 -- start start 2024-04-17T06:06:42.916 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.916+0000 7f9fb89f4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9fb406a790 0x7f9fb41986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:06:42.916 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.916+0000 7f9fb89f4640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9fb406b130 0x7f9fb4198c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:06:42.916 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.916+0000 7f9fb89f4640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9fb41080f0 0x7f9fb419cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:06:42.916 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.916+0000 7f9fb89f4640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb41122e0 con 0x7f9fb406b130 2024-04-17T06:06:42.916 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.916+0000 7f9fb89f4640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb4112160 con 0x7f9fb41080f0 2024-04-17T06:06:42.916 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.916+0000 7f9fb89f4640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb4112460 con 0x7f9fb406a790 2024-04-17T06:06:42.916 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.916+0000 7f9fabfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9fb406b130 0x7f9fb4198c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:06:42.916 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.916+0000 7f9fabfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9fb406b130 0x7f9fb4198c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54042/0 (socket says 172.21.15.47:54042) 2024-04-17T06:06:42.916 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.916+0000 7f9fabfff640 1 -- 172.21.15.47:0/4129112992 learned_addr learned my addr 172.21.15.47:0/4129112992 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:06:42.916 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.916+0000 7f9fb2d9d640 1 --2- 172.21.15.47:0/4129112992 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9fb41080f0 0x7f9fb419cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:06:42.916 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.916+0000 7f9fb259c640 1 --2- 172.21.15.47:0/4129112992 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9fb406a790 0x7f9fb41986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:06:42.917 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.916+0000 7f9fabfff640 1 -- 172.21.15.47:0/4129112992 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9fb406a790 msgr2=0x7f9fb41986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:06:42.917 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.916+0000 7f9fabfff640 1 --2- 172.21.15.47:0/4129112992 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9fb406a790 0x7f9fb41986c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:42.917 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.916+0000 7f9fabfff640 1 -- 172.21.15.47:0/4129112992 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9fb41080f0 msgr2=0x7f9fb419cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:06:42.917 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.916+0000 7f9fabfff640 1 --2- 172.21.15.47:0/4129112992 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9fb41080f0 0x7f9fb419cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:42.917 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.916+0000 7f9fabfff640 1 -- 172.21.15.47:0/4129112992 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fb419d6b0 con 0x7f9fb406b130 2024-04-17T06:06:42.917 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.916+0000 7f9fb259c640 1 --2- 172.21.15.47:0/4129112992 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9fb406a790 0x7f9fb41986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:06:42.917 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.917+0000 7f9fabfff640 1 --2- 172.21.15.47:0/4129112992 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9fb406b130 0x7f9fb4198c00 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c005940 tx=0x7f9f9c02cd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:06:42.917 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.917+0000 7f9fab7fe640 1 -- 172.21.15.47:0/4129112992 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f9c041070 con 0x7f9fb406b130 2024-04-17T06:06:42.917 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.917+0000 7f9fab7fe640 1 -- 172.21.15.47:0/4129112992 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f9c004450 con 0x7f9fb406b130 2024-04-17T06:06:42.917 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.917+0000 7f9fb89f4640 1 -- 172.21.15.47:0/4129112992 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fb419d940 con 0x7f9fb406b130 2024-04-17T06:06:42.917 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.917+0000 7f9fab7fe640 1 -- 172.21.15.47:0/4129112992 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f9c034440 con 0x7f9fb406b130 2024-04-17T06:06:42.917 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.917+0000 7f9fb89f4640 1 -- 172.21.15.47:0/4129112992 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fb41a5230 con 0x7f9fb406b130 2024-04-17T06:06:42.923 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.919+0000 7f9fab7fe640 1 -- 172.21.15.47:0/4129112992 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9f9c044070 con 0x7f9fb406b130 2024-04-17T06:06:42.923 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.920+0000 7f9fbb5e4740 1 Processor -- start 2024-04-17T06:06:42.923 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.920+0000 7f9fbb5e4740 1 -- start start 2024-04-17T06:06:42.923 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.920+0000 7f9fab7fe640 1 --2- 172.21.15.47:0/4129112992 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f9fa00a3dc0 0x7f9fa00a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:06:42.923 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.920+0000 7f9fbb5e4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4094400c0 0x55c4094404a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:06:42.923 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.920+0000 7f9fbb5e4740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4094409e0 0x55c409449270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:06:42.923 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.920+0000 7f9fbb5e4740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c4094497b0 0x55c40944bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:06:42.923 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.920+0000 7f9fbb5e4740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55c4093a57d0 con 0x55c4094409e0 2024-04-17T06:06:42.923 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.920+0000 7f9fbb5e4740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55c409342f70 con 0x55c4094497b0 2024-04-17T06:06:42.923 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.920+0000 7f9fbb5e4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c4093a1790 con 0x55c4094400c0 2024-04-17T06:06:42.923 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.920+0000 7f9fab7fe640 1 -- 172.21.15.47:0/4129112992 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f9f9c0f2300 con 0x7f9fb406b130 2024-04-17T06:06:42.923 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.920+0000 7f9fb2d9d640 1 --2- 172.21.15.47:0/4129112992 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f9fa00a3dc0 0x7f9fa00a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:06:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.920+0000 7f9fabfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4094409e0 0x55c409449270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:06:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.920+0000 7f9fabfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4094409e0 0x55c409449270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54054/0 (socket says 172.21.15.47:54054) 2024-04-17T06:06:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.920+0000 7f9fabfff640 1 -- 172.21.15.47:0/3170838776 learned_addr learned my addr 172.21.15.47:0/3170838776 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:06:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.920+0000 7f9fb259c640 1 --2- 172.21.15.47:0/3170838776 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4094400c0 0x55c4094404a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:06:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.921+0000 7f9fabfff640 1 -- 172.21.15.47:0/3170838776 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4094400c0 msgr2=0x55c4094404a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:06:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.921+0000 7f9fb2d9d640 1 --2- 172.21.15.47:0/3170838776 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c4094497b0 0x55c40944bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:06:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.921+0000 7f9fabfff640 1 --2- 172.21.15.47:0/3170838776 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4094400c0 0x55c4094404a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.921+0000 7f9fabfff640 1 -- 172.21.15.47:0/3170838776 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c4094497b0 msgr2=0x55c40944bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:06:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.921+0000 7f9fabfff640 1 --2- 172.21.15.47:0/3170838776 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c4094497b0 0x55c40944bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.921+0000 7f9fabfff640 1 -- 172.21.15.47:0/3170838776 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4090b8210 con 0x55c4094409e0 2024-04-17T06:06:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.921+0000 7f9fb2d9d640 1 --2- 172.21.15.47:0/3170838776 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c4094497b0 0x55c40944bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:06:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.921+0000 7f9fb2d9d640 1 --2- 172.21.15.47:0/4129112992 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f9fa00a3dc0 0x7f9fa00a6280 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9fac007f60 tx=0x7f9fac0079c0 comp rx=0 tx=0).ready entity=mgr.24923 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:06:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.921+0000 7f9fabfff640 1 --2- 172.21.15.47:0/3170838776 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4094409e0 0x55c409449270 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c0be370 tx=0x7f9f9c0ba680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46fc0dc67fe9de4a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:06:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.921+0000 7f9fa8ff9640 1 -- 172.21.15.47:0/3170838776 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f9c04b070 con 0x55c4094409e0 2024-04-17T06:06:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.921+0000 7f9fa8ff9640 1 -- 172.21.15.47:0/3170838776 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f9c0beae0 con 0x55c4094409e0 2024-04-17T06:06:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.921+0000 7f9fa8ff9640 1 -- 172.21.15.47:0/3170838776 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f9c09b5f0 con 0x55c4094409e0 2024-04-17T06:06:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.922+0000 7f9fbb5e4740 1 -- 172.21.15.47:0/3170838776 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4094409e0 msgr2=0x55c409449270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:06:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.922+0000 7f9fbb5e4740 1 --2- 172.21.15.47:0/3170838776 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4094409e0 0x55c409449270 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c0be370 tx=0x7f9f9c0ba680 comp rx=0 tx=0).stop 2024-04-17T06:06:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.922+0000 7f9fbb5e4740 1 -- 172.21.15.47:0/3170838776 shutdown_connections 2024-04-17T06:06:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.922+0000 7f9fbb5e4740 1 --2- 172.21.15.47:0/3170838776 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c4094497b0 0x55c40944bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.922+0000 7f9fbb5e4740 1 --2- 172.21.15.47:0/3170838776 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4094409e0 0x55c409449270 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.922+0000 7f9fbb5e4740 1 --2- 172.21.15.47:0/3170838776 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4094400c0 0x55c4094404a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.922+0000 7f9fbb5e4740 1 -- 172.21.15.47:0/3170838776 >> 172.21.15.47:0/3170838776 conn(0x55c40943a9e0 msgr2=0x55c40943adc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:06:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.922+0000 7f9fbb5e4740 1 -- 172.21.15.47:0/3170838776 shutdown_connections 2024-04-17T06:06:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.922+0000 7f9fbb5e4740 1 -- 172.21.15.47:0/3170838776 wait complete. 2024-04-17T06:06:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.922+0000 7f9fbb5e4740 1 -- start start 2024-04-17T06:06:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.922+0000 7f9fbb5e4740 1 Processor -- start 2024-04-17T06:06:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.923+0000 7f9fbb5e4740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c4094400c0 0x55c4094466f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:06:42.926 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.923+0000 7f9fbb5e4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4094409e0 0x55c409446c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:06:42.926 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.923+0000 7f9fbb5e4740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4094497b0 0x55c409447170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:06:42.926 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.923+0000 7f9fbb5e4740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55c40944c1a0 con 0x55c4094497b0 2024-04-17T06:06:42.926 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.923+0000 7f9fbb5e4740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55c40932de90 con 0x55c4094400c0 2024-04-17T06:06:42.926 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.923+0000 7f9fbb5e4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c40944c320 con 0x55c4094409e0 2024-04-17T06:06:42.926 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.923+0000 7f9fbb5e4740 10 client.0 _ll_register_callbacks cb 0x55c409343d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:06:42.926 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.923+0000 7f9fb2d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4094497b0 0x55c409447170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:06:42.926 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.923+0000 7f9fb2d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4094497b0 0x55c409447170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54064/0 (socket says 172.21.15.47:54064) 2024-04-17T06:06:42.926 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.923+0000 7f9fb2d9d640 1 -- 172.21.15.47:0/795862489 learned_addr learned my addr 172.21.15.47:0/795862489 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:06:42.926 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.923+0000 7f9fb259c640 1 --2- 172.21.15.47:0/795862489 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c4094400c0 0x55c4094466f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:06:42.926 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.923+0000 7f9fabfff640 1 --2- 172.21.15.47:0/795862489 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4094409e0 0x55c409446c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:06:42.926 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.923+0000 7f9fb2d9d640 1 -- 172.21.15.47:0/795862489 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4094409e0 msgr2=0x55c409446c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:06:42.926 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.923+0000 7f9fb2d9d640 1 --2- 172.21.15.47:0/795862489 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4094409e0 0x55c409446c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:42.927 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.923+0000 7f9fb2d9d640 1 -- 172.21.15.47:0/795862489 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c4094400c0 msgr2=0x55c4094466f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:06:42.927 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.923+0000 7f9fb2d9d640 1 --2- 172.21.15.47:0/795862489 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c4094400c0 0x55c4094466f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:42.927 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.923+0000 7f9fb2d9d640 1 -- 172.21.15.47:0/795862489 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c409184620 con 0x55c4094497b0 2024-04-17T06:06:42.927 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.923+0000 7f9fb259c640 1 --2- 172.21.15.47:0/795862489 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c4094400c0 0x55c4094466f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T06:06:42.927 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.923+0000 7f9fb2d9d640 1 --2- 172.21.15.47:0/795862489 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4094497b0 0x55c409447170 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f9fac012e40 tx=0x7f9fac004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8efebf95b1dff93 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:06:42.927 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.923+0000 7f9f99ffb640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-17T06:06:42.927 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.923+0000 7f9f99ffb640 1 -- 172.21.15.47:0/795862489 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fac01f620 con 0x55c4094497b0 2024-04-17T06:06:42.927 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.924+0000 7f9fbb5e4740 1 -- 172.21.15.47:0/795862489 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f9c0beae0 con 0x55c4094497b0 2024-04-17T06:06:42.927 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.924+0000 7f9f99ffb640 1 -- 172.21.15.47:0/795862489 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f9fac01f7c0 con 0x55c4094497b0 2024-04-17T06:06:42.927 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.924+0000 7f9f99ffb640 1 -- 172.21.15.47:0/795862489 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9fac01f9c0 con 0x55c4094497b0 2024-04-17T06:06:42.927 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.924+0000 7f9fbb5e4740 10 client.15855 fetch_fsmap learned FSMap version 30 2024-04-17T06:06:42.927 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.924+0000 7f9fbb5e4740 1 -- 172.21.15.47:0/795862489 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c4094582f0 con 0x55c4094497b0 2024-04-17T06:06:42.927 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.924+0000 7f9f99ffb640 1 -- 172.21.15.47:0/795862489 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f9c0beae0 con 0x55c4094497b0 2024-04-17T06:06:42.927 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.924+0000 7f9f99ffb640 1 -- 172.21.15.47:0/795862489 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 30) ==== 8339+0+0 (secure 0 0 0) 0x7f9fac027400 con 0x55c4094497b0 2024-04-17T06:06:42.928 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.924+0000 7f9fbb5e4740 10 client.15855 fetch_fsmap finished waiting for FSMap version 30 2024-04-17T06:06:42.928 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.924+0000 7f9fbb5e4740 10 client.15855 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-17T06:06:42.928 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.924+0000 7f9fbb5e4740 20 client.15855 populate_metadata read hostname 'smithi047' 2024-04-17T06:06:42.928 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.924+0000 7f9fbb5e4740 1 --2- 172.21.15.47:0/795862489 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55c409459760 0x55c40945bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:06:42.928 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.924+0000 7f9fbb5e4740 4 client.15855 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T06:06:42.928 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.924+0000 7f9fbb5e4740 1 -- 172.21.15.47:0/795862489 --> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c4093b31f0 con 0x55c409459760 2024-04-17T06:06:42.928 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.925+0000 7f9fb259c640 1 --2- 172.21.15.47:0/795862489 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55c409459760 0x55c40945bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:06:42.928 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.925+0000 7f9f99ffb640 10 client.15855 ms_handle_connect on v2:172.21.15.98:6838/2841287917 2024-04-17T06:06:42.928 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.925+0000 7f9fb259c640 1 --2- 172.21.15.47:0/795862489 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55c409459760 0x55c40945bb50 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=286259e3006dd157 server_cookie=5e2e644276b869ed in_seq=0 out_seq=0 2024-04-17T06:06:42.929 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.926+0000 7f9f99ffb640 1 -- 172.21.15.47:0/795862489 <== mds.0 v2:172.21.15.98:6838/2841287917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c4093b31f0 con 0x55c409459760 2024-04-17T06:06:42.929 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.926+0000 7f9f99ffb640 10 client.15855 handle_command_reply: tid=0 2024-04-17T06:06:42.929 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.926+0000 7f9fbb5e4740 1 client.15855 shutdown 2024-04-17T06:06:42.929 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.926+0000 7f9fbb5e4740 1 -- 172.21.15.47:0/795862489 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4094497b0 msgr2=0x55c409447170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:06:42.930 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.926+0000 7f9fbb5e4740 1 --2- 172.21.15.47:0/795862489 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4094497b0 0x55c409447170 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f9fac012e40 tx=0x7f9fac004ab0 comp rx=0 tx=0).stop 2024-04-17T06:06:42.930 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.926+0000 7f9fbb5e4740 1 -- 172.21.15.47:0/795862489 shutdown_connections 2024-04-17T06:06:42.930 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.926+0000 7f9fbb5e4740 1 --2- 172.21.15.47:0/795862489 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55c409459760 0x55c40945bb50 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:42.930 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.926+0000 7f9fbb5e4740 1 --2- 172.21.15.47:0/795862489 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4094497b0 0x55c409447170 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:42.930 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.926+0000 7f9fbb5e4740 1 --2- 172.21.15.47:0/795862489 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4094409e0 0x55c409446c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:42.930 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.926+0000 7f9fbb5e4740 1 --2- 172.21.15.47:0/795862489 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c4094400c0 0x55c4094466f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:42.930 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.926+0000 7f9f99ffb640 0 client.15855 ms_handle_reset on v2:172.21.15.98:6838/2841287917 2024-04-17T06:06:42.930 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.926+0000 7f9fbb5e4740 1 -- 172.21.15.47:0/795862489 >> 172.21.15.47:0/795862489 conn(0x55c40943a9e0 msgr2=0x55c40943c730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:06:42.930 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.926+0000 7f9fbb5e4740 1 -- 172.21.15.47:0/795862489 shutdown_connections 2024-04-17T06:06:42.930 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.926+0000 7f9fbb5e4740 1 -- 172.21.15.47:0/795862489 wait complete. 2024-04-17T06:06:42.930 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.926+0000 7f9fbb5e4740 20 client.15855 trim_cache size 0 max 16384 2024-04-17T06:06:42.943 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.941+0000 7f9fbb5e4740 1 Processor -- start 2024-04-17T06:06:42.943 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.941+0000 7f9fbb5e4740 1 -- start start 2024-04-17T06:06:42.943 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.941+0000 7f9fbb5e4740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c4094400c0 0x55c409448e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:06:42.943 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.941+0000 7f9fbb5e4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4094409e0 0x55c409454fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:06:42.943 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.941+0000 7f9fbb5e4740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4094497b0 0x55c409461420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:06:42.943 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.941+0000 7f9fbb5e4740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb41b03e0 con 0x55c4094497b0 2024-04-17T06:06:42.943 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.941+0000 7f9fbb5e4740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb41adca0 con 0x55c4094400c0 2024-04-17T06:06:42.943 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.941+0000 7f9fbb5e4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb41abaf0 con 0x55c4094409e0 2024-04-17T06:06:42.943 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.941+0000 7f9fb2d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4094497b0 0x55c409461420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:06:42.944 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.941+0000 7f9fb2d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4094497b0 0x55c409461420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54072/0 (socket says 172.21.15.47:54072) 2024-04-17T06:06:42.944 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.941+0000 7f9fb2d9d640 1 -- 172.21.15.47:0/1863442146 learned_addr learned my addr 172.21.15.47:0/1863442146 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:06:42.944 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.941+0000 7f9fb259c640 1 --2- 172.21.15.47:0/1863442146 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c4094400c0 0x55c409448e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:06:42.944 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.941+0000 7f9fabfff640 1 --2- 172.21.15.47:0/1863442146 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4094409e0 0x55c409454fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:06:42.944 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.942+0000 7f9fb2d9d640 1 -- 172.21.15.47:0/1863442146 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4094409e0 msgr2=0x55c409454fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:06:42.944 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.942+0000 7f9fb2d9d640 1 --2- 172.21.15.47:0/1863442146 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4094409e0 0x55c409454fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:42.944 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.942+0000 7f9fb2d9d640 1 -- 172.21.15.47:0/1863442146 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c4094400c0 msgr2=0x55c409448e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:06:42.944 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.942+0000 7f9fb2d9d640 1 --2- 172.21.15.47:0/1863442146 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c4094400c0 0x55c409448e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:42.944 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.942+0000 7f9fb2d9d640 1 -- 172.21.15.47:0/1863442146 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c40943a810 con 0x55c4094497b0 2024-04-17T06:06:42.944 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.942+0000 7f9fb259c640 1 --2- 172.21.15.47:0/1863442146 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c4094400c0 0x55c409448e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:06:42.944 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.942+0000 7f9fb2d9d640 1 --2- 172.21.15.47:0/1863442146 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4094497b0 0x55c409461420 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f9fac017040 tx=0x7f9fac007860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8659e8bd581852fc server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:06:42.944 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.942+0000 7f9f9bfff640 1 -- 172.21.15.47:0/1863442146 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fac03f070 con 0x55c4094497b0 2024-04-17T06:06:42.944 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.942+0000 7f9f9bfff640 1 -- 172.21.15.47:0/1863442146 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9fac027c90 con 0x55c4094497b0 2024-04-17T06:06:42.944 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.942+0000 7f9f9bfff640 1 -- 172.21.15.47:0/1863442146 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fac036040 con 0x55c4094497b0 2024-04-17T06:06:42.944 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.942+0000 7f9fbb5e4740 1 -- 172.21.15.47:0/1863442146 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4094497b0 msgr2=0x55c409461420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:06:42.945 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.942+0000 7f9fbb5e4740 1 --2- 172.21.15.47:0/1863442146 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4094497b0 0x55c409461420 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f9fac017040 tx=0x7f9fac007860 comp rx=0 tx=0).stop 2024-04-17T06:06:42.945 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.943+0000 7f9fbb5e4740 1 -- 172.21.15.47:0/1863442146 shutdown_connections 2024-04-17T06:06:42.945 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.943+0000 7f9fbb5e4740 1 --2- 172.21.15.47:0/1863442146 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4094497b0 0x55c409461420 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:42.945 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.943+0000 7f9fbb5e4740 1 --2- 172.21.15.47:0/1863442146 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4094409e0 0x55c409454fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:42.945 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.943+0000 7f9fbb5e4740 1 --2- 172.21.15.47:0/1863442146 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c4094400c0 0x55c409448e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:42.945 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.943+0000 7f9fbb5e4740 1 -- 172.21.15.47:0/1863442146 >> 172.21.15.47:0/1863442146 conn(0x55c40943a9e0 msgr2=0x55c40943c630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:06:42.945 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.943+0000 7f9fbb5e4740 1 -- 172.21.15.47:0/1863442146 shutdown_connections 2024-04-17T06:06:42.945 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.943+0000 7f9fbb5e4740 1 -- 172.21.15.47:0/1863442146 wait complete. 2024-04-17T06:06:42.945 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.943+0000 7f9fbb5e4740 1 -- start start 2024-04-17T06:06:42.945 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.943+0000 7f9fbb5e4740 1 Processor -- start 2024-04-17T06:06:42.945 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.943+0000 7f9fbb5e4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4094400c0 0x55c4094514e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:06:42.945 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.943+0000 7f9fbb5e4740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4094409e0 0x55c409451a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:06:42.945 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.943+0000 7f9fbb5e4740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c4094497b0 0x55c40944e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:06:42.945 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.943+0000 7f9fbb5e4740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb41ac160 con 0x55c4094409e0 2024-04-17T06:06:42.945 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.943+0000 7f9fbb5e4740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb41ad5d0 con 0x55c4094497b0 2024-04-17T06:06:42.946 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.943+0000 7f9fbb5e4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb41ac830 con 0x55c4094400c0 2024-04-17T06:06:42.946 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.943+0000 7f9fbb5e4740 10 client.0 _ll_register_callbacks cb 0x7f9fac012b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:06:42.946 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.943+0000 7f9fabfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4094409e0 0x55c409451a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:06:42.946 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.943+0000 7f9fabfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4094409e0 0x55c409451a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54074/0 (socket says 172.21.15.47:54074) 2024-04-17T06:06:42.946 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.943+0000 7f9fabfff640 1 -- 172.21.15.47:0/1784337490 learned_addr learned my addr 172.21.15.47:0/1784337490 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:06:42.946 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.943+0000 7f9fb259c640 1 --2- 172.21.15.47:0/1784337490 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4094400c0 0x55c4094514e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:06:42.946 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.943+0000 7f9fb2d9d640 1 --2- 172.21.15.47:0/1784337490 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c4094497b0 0x55c40944e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:06:42.946 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.944+0000 7f9fabfff640 1 -- 172.21.15.47:0/1784337490 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4094400c0 msgr2=0x55c4094514e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:06:42.946 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.944+0000 7f9fabfff640 1 --2- 172.21.15.47:0/1784337490 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4094400c0 0x55c4094514e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:42.946 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.944+0000 7f9fabfff640 1 -- 172.21.15.47:0/1784337490 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c4094497b0 msgr2=0x55c40944e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:06:42.946 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.944+0000 7f9fabfff640 1 --2- 172.21.15.47:0/1784337490 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c4094497b0 0x55c40944e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:42.946 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.944+0000 7f9fabfff640 1 -- 172.21.15.47:0/1784337490 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c40944bba0 con 0x55c4094409e0 2024-04-17T06:06:42.946 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.944+0000 7f9fabfff640 1 --2- 172.21.15.47:0/1784337490 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4094409e0 0x55c409451a20 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c0efc40 tx=0x7f9f9c02bcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb0920ecbc809f25 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:06:42.947 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.944+0000 7f9f99ffb640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-17T06:06:42.947 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.944+0000 7f9f99ffb640 1 -- 172.21.15.47:0/1784337490 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f9c09a060 con 0x55c4094409e0 2024-04-17T06:06:42.947 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.944+0000 7f9f99ffb640 1 -- 172.21.15.47:0/1784337490 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f9f9c09e7c0 con 0x55c4094409e0 2024-04-17T06:06:42.947 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.944+0000 7f9fbb5e4740 1 -- 172.21.15.47:0/1784337490 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9fac027c90 con 0x55c4094409e0 2024-04-17T06:06:42.947 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.944+0000 7f9f99ffb640 1 -- 172.21.15.47:0/1784337490 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9f9c04b040 con 0x55c4094409e0 2024-04-17T06:06:42.947 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.944+0000 7f9fbb5e4740 10 client.15861 fetch_fsmap learned FSMap version 30 2024-04-17T06:06:42.947 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.944+0000 7f9fbb5e4740 1 -- 172.21.15.47:0/1784337490 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c4094c05d0 con 0x55c4094409e0 2024-04-17T06:06:42.947 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.944+0000 7f9f99ffb640 1 -- 172.21.15.47:0/1784337490 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9fac027c90 con 0x55c4094409e0 2024-04-17T06:06:42.947 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.944+0000 7f9f99ffb640 1 -- 172.21.15.47:0/1784337490 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 30) ==== 8339+0+0 (secure 0 0 0) 0x7f9f9c0035f0 con 0x55c4094409e0 2024-04-17T06:06:42.947 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.944+0000 7f9fbb5e4740 10 client.15861 fetch_fsmap finished waiting for FSMap version 30 2024-04-17T06:06:42.947 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.944+0000 7f9fbb5e4740 10 client.15861 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-17T06:06:42.947 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.944+0000 7f9fbb5e4740 20 client.15861 populate_metadata read hostname 'smithi047' 2024-04-17T06:06:42.947 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.944+0000 7f9fbb5e4740 1 --2- 172.21.15.47:0/1784337490 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55c4094c0b60 0x55c4094c2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:06:42.947 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.944+0000 7f9fbb5e4740 4 client.15861 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-17T06:06:42.947 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.944+0000 7f9fbb5e4740 1 -- 172.21.15.47:0/1784337490 --> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c4093ad060 con 0x55c4094c0b60 2024-04-17T06:06:42.947 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.945+0000 7f9fb259c640 1 --2- 172.21.15.47:0/1784337490 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55c4094c0b60 0x55c4094c2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:06:42.948 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.945+0000 7f9f99ffb640 10 client.15861 ms_handle_connect on v2:172.21.15.98:6838/2841287917 2024-04-17T06:06:42.948 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.945+0000 7f9fb259c640 1 --2- 172.21.15.47:0/1784337490 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55c4094c0b60 0x55c4094c2f80 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6cbc87d7e83eba7 server_cookie=64d274c2d696f762 in_seq=0 out_seq=0 2024-04-17T06:06:42.951 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.953+0000 7f9f99ffb640 1 -- 172.21.15.47:0/1784337490 <== mds.0 v2:172.21.15.98:6838/2841287917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c4093ad060 con 0x55c4094c0b60 2024-04-17T06:06:42.951 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.953+0000 7f9f99ffb640 10 client.15861 handle_command_reply: tid=0 2024-04-17T06:06:42.951 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.953+0000 7f9fbb5e4740 1 client.15861 shutdown 2024-04-17T06:06:42.951 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.953+0000 7f9fbb5e4740 1 -- 172.21.15.47:0/1784337490 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4094409e0 msgr2=0x55c409451a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:06:42.951 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.953+0000 7f9fbb5e4740 1 --2- 172.21.15.47:0/1784337490 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4094409e0 0x55c409451a20 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c0efc40 tx=0x7f9f9c02bcd0 comp rx=0 tx=0).stop 2024-04-17T06:06:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.953+0000 7f9fbb5e4740 1 -- 172.21.15.47:0/1784337490 shutdown_connections 2024-04-17T06:06:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.953+0000 7f9fbb5e4740 1 --2- 172.21.15.47:0/1784337490 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55c4094c0b60 0x55c4094c2f80 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.953+0000 7f9fbb5e4740 1 --2- 172.21.15.47:0/1784337490 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c4094497b0 0x55c40944e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.953+0000 7f9fbb5e4740 1 --2- 172.21.15.47:0/1784337490 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4094409e0 0x55c409451a20 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.953+0000 7f9fbb5e4740 1 --2- 172.21.15.47:0/1784337490 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4094400c0 0x55c4094514e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.953+0000 7f9f99ffb640 0 client.15861 ms_handle_reset on v2:172.21.15.98:6838/2841287917 2024-04-17T06:06:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.953+0000 7f9fbb5e4740 1 -- 172.21.15.47:0/1784337490 >> 172.21.15.47:0/1784337490 conn(0x55c40943a9e0 msgr2=0x55c40943bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:06:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.953+0000 7f9fbb5e4740 1 -- 172.21.15.47:0/1784337490 shutdown_connections 2024-04-17T06:06:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.953+0000 7f9fbb5e4740 1 -- 172.21.15.47:0/1784337490 wait complete. 2024-04-17T06:06:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.953+0000 7f9fbb5e4740 20 client.15861 trim_cache size 0 max 16384 2024-04-17T06:06:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-04-17T06:06:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-04-17T06:06:42.953 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-04-17T06:06:42.953 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-04-17T06:06:42.953 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.954+0000 7f9fb89f4640 1 -- 172.21.15.47:0/4129112992 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f9fa00a3dc0 msgr2=0x7f9fa00a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:06:42.953 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.954+0000 7f9fb89f4640 1 --2- 172.21.15.47:0/4129112992 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f9fa00a3dc0 0x7f9fa00a6280 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9fac007f60 tx=0x7f9fac0079c0 comp rx=0 tx=0).stop 2024-04-17T06:06:42.954 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.954+0000 7f9fb89f4640 1 -- 172.21.15.47:0/4129112992 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9fb406b130 msgr2=0x7f9fb4198c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:06:42.954 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.954+0000 7f9fb89f4640 1 --2- 172.21.15.47:0/4129112992 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9fb406b130 0x7f9fb4198c00 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c005940 tx=0x7f9f9c02cd10 comp rx=0 tx=0).stop 2024-04-17T06:06:42.954 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.954+0000 7f9fb89f4640 1 -- 172.21.15.47:0/4129112992 shutdown_connections 2024-04-17T06:06:42.954 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.954+0000 7f9fb89f4640 1 --2- 172.21.15.47:0/4129112992 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9fb41080f0 0x7f9fb419cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:42.954 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.954+0000 7f9fb89f4640 1 --2- 172.21.15.47:0/4129112992 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f9fa00a3dc0 0x7f9fa00a6280 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:42.954 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.954+0000 7f9fb89f4640 1 --2- 172.21.15.47:0/4129112992 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9fb406b130 0x7f9fb4198c00 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:42.954 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.954+0000 7f9fb89f4640 1 --2- 172.21.15.47:0/4129112992 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9fb406a790 0x7f9fb41986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:42.954 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.954+0000 7f9fb89f4640 1 -- 172.21.15.47:0/4129112992 >> 172.21.15.47:0/4129112992 conn(0x7f9fb4076c60 msgr2=0x7f9fb40ffc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:06:42.954 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.954+0000 7f9fb89f4640 1 -- 172.21.15.47:0/4129112992 shutdown_connections 2024-04-17T06:06:42.954 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:42.954+0000 7f9fb89f4640 1 -- 172.21.15.47:0/4129112992 wait complete. 2024-04-17T06:06:44.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:06:43 smithi103 ceph-mon[36243]: pgmap v167: 97 pgs: 97 active+clean; 2.8 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-04-17T06:06:44.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:06:43 smithi098 ceph-mon[36516]: pgmap v167: 97 pgs: 97 active+clean; 2.8 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-04-17T06:06:44.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:06:43 smithi047 ceph-mon[32223]: pgmap v167: 97 pgs: 97 active+clean; 2.8 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-04-17T06:06:46.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:06:45 smithi103 ceph-mon[36243]: pgmap v168: 97 pgs: 97 active+clean; 2.9 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 20 KiB/s wr, 18 op/s 2024-04-17T06:06:46.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:06:45 smithi098 ceph-mon[36516]: pgmap v168: 97 pgs: 97 active+clean; 2.9 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 20 KiB/s wr, 18 op/s 2024-04-17T06:06:46.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:06:45 smithi047 ceph-mon[32223]: pgmap v168: 97 pgs: 97 active+clean; 2.9 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 20 KiB/s wr, 18 op/s 2024-04-17T06:06:48.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:06:47 smithi103 ceph-mon[36243]: pgmap v169: 97 pgs: 97 active+clean; 2.9 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-04-17T06:06:48.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:06:47 smithi098 ceph-mon[36516]: pgmap v169: 97 pgs: 97 active+clean; 2.9 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-04-17T06:06:48.142 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:06:47 smithi047 ceph-mon[32223]: pgmap v169: 97 pgs: 97 active+clean; 2.9 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-04-17T06:06:50.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:06:49 smithi103 ceph-mon[36243]: pgmap v170: 97 pgs: 97 active+clean; 2.9 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-04-17T06:06:50.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:06:49 smithi098 ceph-mon[36516]: pgmap v170: 97 pgs: 97 active+clean; 2.9 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-04-17T06:06:50.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:06:49 smithi047 ceph-mon[32223]: pgmap v170: 97 pgs: 97 active+clean; 2.9 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-04-17T06:06:52.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:06:51 smithi103 ceph-mon[36243]: pgmap v171: 97 pgs: 97 active+clean; 2.9 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-04-17T06:06:52.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:06:51 smithi098 ceph-mon[36516]: pgmap v171: 97 pgs: 97 active+clean; 2.9 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-04-17T06:06:52.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:06:51 smithi047 ceph-mon[32223]: pgmap v171: 97 pgs: 97 active+clean; 2.9 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-04-17T06:06:54.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:06:53 smithi103 ceph-mon[36243]: pgmap v172: 97 pgs: 97 active+clean; 2.9 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-04-17T06:06:54.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:06:53 smithi098 ceph-mon[36516]: pgmap v172: 97 pgs: 97 active+clean; 2.9 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-04-17T06:06:54.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:06:53 smithi047 ceph-mon[32223]: pgmap v172: 97 pgs: 97 active+clean; 2.9 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-04-17T06:06:56.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:06:55 smithi103 ceph-mon[36243]: pgmap v173: 97 pgs: 97 active+clean; 2.9 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 16 KiB/s wr, 16 op/s 2024-04-17T06:06:56.051 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.052+0000 7f6a30dc8640 1 -- 172.21.15.47:0/2648471846 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a2c1006d0 msgr2=0x7f6a2c100ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:06:56.051 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.052+0000 7f6a30dc8640 1 --2- 172.21.15.47:0/2648471846 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a2c1006d0 0x7f6a2c100ab0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c009900 tx=0x7f6a1c02bc30 comp rx=0 tx=0).stop 2024-04-17T06:06:56.052 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.053+0000 7f6a30dc8640 1 -- 172.21.15.47:0/2648471846 shutdown_connections 2024-04-17T06:06:56.052 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.053+0000 7f6a30dc8640 1 --2- 172.21.15.47:0/2648471846 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6a2c107290 0x7f6a2c10ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:56.052 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.053+0000 7f6a30dc8640 1 --2- 172.21.15.47:0/2648471846 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6a2c101080 0x7f6a2c106bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:56.052 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.053+0000 7f6a30dc8640 1 --2- 172.21.15.47:0/2648471846 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a2c1006d0 0x7f6a2c100ab0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:56.052 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.053+0000 7f6a30dc8640 1 -- 172.21.15.47:0/2648471846 >> 172.21.15.47:0/2648471846 conn(0x7f6a2c0773a0 msgr2=0x7f6a2c0fe770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:06:56.052 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.053+0000 7f6a30dc8640 1 -- 172.21.15.47:0/2648471846 shutdown_connections 2024-04-17T06:06:56.052 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.053+0000 7f6a30dc8640 1 -- 172.21.15.47:0/2648471846 wait complete. 2024-04-17T06:06:56.054 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.053+0000 7f6a30dc8640 1 Processor -- start 2024-04-17T06:06:56.054 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.053+0000 7f6a30dc8640 1 -- start start 2024-04-17T06:06:56.054 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.054+0000 7f6a30dc8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6a2c1006d0 0x7f6a2c101640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:06:56.055 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.054+0000 7f6a30dc8640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6a2c107290 0x7f6a2c101b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:06:56.055 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.054+0000 7f6a30dc8640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a2c1020c0 0x7f6a2c1a6950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:06:56.055 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.054+0000 7f6a30dc8640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f6a2c112340 con 0x7f6a2c1020c0 2024-04-17T06:06:56.055 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.054+0000 7f6a30dc8640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6a2c1121c0 con 0x7f6a2c107290 2024-04-17T06:06:56.055 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.054+0000 7f6a30dc8640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6a2c1124c0 con 0x7f6a2c1006d0 2024-04-17T06:06:56.055 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.054+0000 7f6a2ad9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a2c1020c0 0x7f6a2c1a6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:06:56.055 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.054+0000 7f6a2ad9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a2c1020c0 0x7f6a2c1a6950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55632/0 (socket says 172.21.15.47:55632) 2024-04-17T06:06:56.055 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.054+0000 7f6a2ad9d640 1 -- 172.21.15.47:0/4268494183 learned_addr learned my addr 172.21.15.47:0/4268494183 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:06:56.055 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.054+0000 7f6a2a59c640 1 --2- 172.21.15.47:0/4268494183 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6a2c1006d0 0x7f6a2c101640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:06:56.055 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.054+0000 7f6a29d9b640 1 --2- 172.21.15.47:0/4268494183 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6a2c107290 0x7f6a2c101b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:06:56.055 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.054+0000 7f6a2ad9d640 1 -- 172.21.15.47:0/4268494183 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6a2c1006d0 msgr2=0x7f6a2c101640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:06:56.055 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.054+0000 7f6a2ad9d640 1 --2- 172.21.15.47:0/4268494183 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6a2c1006d0 0x7f6a2c101640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:56.055 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.054+0000 7f6a2ad9d640 1 -- 172.21.15.47:0/4268494183 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6a2c107290 msgr2=0x7f6a2c101b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:06:56.055 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.054+0000 7f6a2ad9d640 1 --2- 172.21.15.47:0/4268494183 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6a2c107290 0x7f6a2c101b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:56.056 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.054+0000 7f6a2ad9d640 1 -- 172.21.15.47:0/4268494183 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a2c1a7050 con 0x7f6a2c1020c0 2024-04-17T06:06:56.056 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.054+0000 7f6a29d9b640 1 --2- 172.21.15.47:0/4268494183 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6a2c107290 0x7f6a2c101b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:06:56.056 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.054+0000 7f6a2ad9d640 1 --2- 172.21.15.47:0/4268494183 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a2c1020c0 0x7f6a2c1a6950 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f6a2400c970 tx=0x7f6a2400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:06:56.056 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.055+0000 7f6a1b7fe640 1 -- 172.21.15.47:0/4268494183 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a24007bb0 con 0x7f6a2c1020c0 2024-04-17T06:06:56.056 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.055+0000 7f6a1b7fe640 1 -- 172.21.15.47:0/4268494183 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a24007d50 con 0x7f6a2c1020c0 2024-04-17T06:06:56.056 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.055+0000 7f6a30dc8640 1 -- 172.21.15.47:0/4268494183 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a2c1a7340 con 0x7f6a2c1020c0 2024-04-17T06:06:56.056 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.055+0000 7f6a1b7fe640 1 -- 172.21.15.47:0/4268494183 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a24018800 con 0x7f6a2c1020c0 2024-04-17T06:06:56.056 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.055+0000 7f6a30dc8640 1 -- 172.21.15.47:0/4268494183 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a2c1a7840 con 0x7f6a2c1020c0 2024-04-17T06:06:56.061 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.057+0000 7f6a1b7fe640 1 -- 172.21.15.47:0/4268494183 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6a2400fa50 con 0x7f6a2c1020c0 2024-04-17T06:06:56.061 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.057+0000 7f6a339b3740 1 Processor -- start 2024-04-17T06:06:56.061 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.058+0000 7f6a339b3740 1 -- start start 2024-04-17T06:06:56.061 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.058+0000 7f6a339b3740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56330feae0c0 0x56330feae4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:06:56.062 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.058+0000 7f6a339b3740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56330feae9e0 0x56330feb7270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:06:56.062 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.058+0000 7f6a339b3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56330feb77b0 0x56330feb9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:06:56.062 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.058+0000 7f6a339b3740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x56330fe137d0 con 0x56330feae9e0 2024-04-17T06:06:56.062 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.058+0000 7f6a339b3740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x56330fdb0f70 con 0x56330feae0c0 2024-04-17T06:06:56.062 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.058+0000 7f6a339b3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56330fe0f790 con 0x56330feb77b0 2024-04-17T06:06:56.062 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.058+0000 7f6a1b7fe640 1 --2- 172.21.15.47:0/4268494183 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f6a140a3df0 0x7f6a140a62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:06:56.062 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.058+0000 7f6a2a59c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56330feae9e0 0x56330feb7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:06:56.062 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.058+0000 7f6a2a59c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56330feae9e0 0x56330feb7270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55636/0 (socket says 172.21.15.47:55636) 2024-04-17T06:06:56.062 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.058+0000 7f6a2a59c640 1 -- 172.21.15.47:0/4288819013 learned_addr learned my addr 172.21.15.47:0/4288819013 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:06:56.062 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.058+0000 7f6a29d9b640 1 --2- 172.21.15.47:0/4288819013 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56330feae0c0 0x56330feae4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:06:56.062 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.058+0000 7f6a2ad9d640 1 --2- 172.21.15.47:0/4268494183 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f6a140a3df0 0x7f6a140a62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:06:56.062 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.058+0000 7f6a29d9b640 1 --2- 172.21.15.47:0/4288819013 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56330feb77b0 0x56330feb9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:06:56.062 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.058+0000 7f6a1b7fe640 1 -- 172.21.15.47:0/4268494183 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f6a240d07a0 con 0x7f6a2c1020c0 2024-04-17T06:06:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.058+0000 7f6a2a59c640 1 -- 172.21.15.47:0/4288819013 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56330feb77b0 msgr2=0x56330feb9ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:06:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.058+0000 7f6a2a59c640 1 --2- 172.21.15.47:0/4288819013 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56330feb77b0 0x56330feb9ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.058+0000 7f6a2a59c640 1 -- 172.21.15.47:0/4288819013 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56330feae0c0 msgr2=0x56330feae4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:06:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.058+0000 7f6a2a59c640 1 --2- 172.21.15.47:0/4288819013 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56330feae0c0 0x56330feae4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.058+0000 7f6a2a59c640 1 -- 172.21.15.47:0/4288819013 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56330fb26210 con 0x56330feae9e0 2024-04-17T06:06:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.059+0000 7f6a2ad9d640 1 --2- 172.21.15.47:0/4268494183 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f6a140a3df0 0x7f6a140a62b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6a2409c700 tx=0x7f6a240c7880 comp rx=0 tx=0).ready entity=mgr.24923 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:06:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.059+0000 7f6a2a59c640 1 --2- 172.21.15.47:0/4288819013 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56330feae9e0 0x56330feb7270 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c02c170 tx=0x7f6a1c02c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aed07113b2206aa4 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:06:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.059+0000 7f6a18ff9640 1 -- 172.21.15.47:0/4288819013 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a1c042070 con 0x56330feae9e0 2024-04-17T06:06:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.059+0000 7f6a18ff9640 1 -- 172.21.15.47:0/4288819013 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a1c02ca80 con 0x56330feae9e0 2024-04-17T06:06:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.059+0000 7f6a18ff9640 1 -- 172.21.15.47:0/4288819013 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a1c005180 con 0x56330feae9e0 2024-04-17T06:06:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.060+0000 7f6a339b3740 1 -- 172.21.15.47:0/4288819013 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56330feae9e0 msgr2=0x56330feb7270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:06:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.060+0000 7f6a339b3740 1 --2- 172.21.15.47:0/4288819013 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56330feae9e0 0x56330feb7270 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c02c170 tx=0x7f6a1c02c680 comp rx=0 tx=0).stop 2024-04-17T06:06:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.060+0000 7f6a339b3740 1 -- 172.21.15.47:0/4288819013 shutdown_connections 2024-04-17T06:06:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.060+0000 7f6a339b3740 1 --2- 172.21.15.47:0/4288819013 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56330feb77b0 0x56330feb9ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.060+0000 7f6a339b3740 1 --2- 172.21.15.47:0/4288819013 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56330feae9e0 0x56330feb7270 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.060+0000 7f6a339b3740 1 --2- 172.21.15.47:0/4288819013 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56330feae0c0 0x56330feae4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.060+0000 7f6a339b3740 1 -- 172.21.15.47:0/4288819013 >> 172.21.15.47:0/4288819013 conn(0x56330fea89e0 msgr2=0x56330fea8dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:06:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.060+0000 7f6a339b3740 1 -- 172.21.15.47:0/4288819013 shutdown_connections 2024-04-17T06:06:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.060+0000 7f6a339b3740 1 -- 172.21.15.47:0/4288819013 wait complete. 2024-04-17T06:06:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.060+0000 7f6a339b3740 1 -- start start 2024-04-17T06:06:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.060+0000 7f6a339b3740 1 Processor -- start 2024-04-17T06:06:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.060+0000 7f6a339b3740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56330feae0c0 0x56330feb47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:06:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.060+0000 7f6a339b3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56330feae9e0 0x56330feb4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:06:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.060+0000 7f6a339b3740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56330feb77b0 0x56330feb5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:06:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.060+0000 7f6a339b3740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x56330feba380 con 0x56330feae0c0 2024-04-17T06:06:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.060+0000 7f6a339b3740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x56330fd9be90 con 0x56330feb77b0 2024-04-17T06:06:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.060+0000 7f6a339b3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56330feba500 con 0x56330feae9e0 2024-04-17T06:06:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.060+0000 7f6a339b3740 10 client.0 _ll_register_callbacks cb 0x56330fdb1d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:06:56.065 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.061+0000 7f6a29d9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56330feae0c0 0x56330feb47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:06:56.065 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.061+0000 7f6a29d9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56330feae0c0 0x56330feb47e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55648/0 (socket says 172.21.15.47:55648) 2024-04-17T06:06:56.065 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.061+0000 7f6a29d9b640 1 -- 172.21.15.47:0/3908354885 learned_addr learned my addr 172.21.15.47:0/3908354885 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:06:56.065 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.061+0000 7f6a2a59c640 1 --2- 172.21.15.47:0/3908354885 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56330feae9e0 0x56330feb4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:06:56.065 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.061+0000 7f6a29d9b640 1 -- 172.21.15.47:0/3908354885 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56330feae9e0 msgr2=0x56330feb4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:06:56.065 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.061+0000 7f6a29d9b640 1 --2- 172.21.15.47:0/3908354885 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56330feae9e0 0x56330feb4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:56.065 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.061+0000 7f6a29d9b640 1 -- 172.21.15.47:0/3908354885 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56330feb77b0 msgr2=0x56330feb5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:06:56.065 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.061+0000 7f6a29d9b640 1 --2- 172.21.15.47:0/3908354885 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56330feb77b0 0x56330feb5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:56.065 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.061+0000 7f6a29d9b640 1 -- 172.21.15.47:0/3908354885 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56330fbf2620 con 0x56330feae0c0 2024-04-17T06:06:56.065 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.061+0000 7f6a29d9b640 1 --2- 172.21.15.47:0/3908354885 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56330feae0c0 0x56330feb47e0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f6a20005fd0 tx=0x7f6a20014c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c23620b9d68951d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:06:56.065 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.061+0000 7f6a11ffb640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-17T06:06:56.065 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.061+0000 7f6a11ffb640 1 -- 172.21.15.47:0/3908354885 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a20012d40 con 0x56330feae0c0 2024-04-17T06:06:56.065 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.061+0000 7f6a11ffb640 1 -- 172.21.15.47:0/3908354885 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f6a20004b20 con 0x56330feae0c0 2024-04-17T06:06:56.065 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.061+0000 7f6a339b3740 1 -- 172.21.15.47:0/3908354885 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a1c02ca80 con 0x56330feae0c0 2024-04-17T06:06:56.065 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.061+0000 7f6a11ffb640 1 -- 172.21.15.47:0/3908354885 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6a20004d20 con 0x56330feae0c0 2024-04-17T06:06:56.065 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.061+0000 7f6a339b3740 10 client.15873 fetch_fsmap learned FSMap version 30 2024-04-17T06:06:56.066 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.061+0000 7f6a339b3740 1 -- 172.21.15.47:0/3908354885 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56330fec6680 con 0x56330feae0c0 2024-04-17T06:06:56.066 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.062+0000 7f6a11ffb640 1 -- 172.21.15.47:0/3908354885 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a1c02ca80 con 0x56330feae0c0 2024-04-17T06:06:56.066 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.062+0000 7f6a11ffb640 1 -- 172.21.15.47:0/3908354885 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 30) ==== 8339+0+0 (secure 0 0 0) 0x7f6a2000c050 con 0x56330feae0c0 2024-04-17T06:06:56.066 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.062+0000 7f6a339b3740 10 client.15873 fetch_fsmap finished waiting for FSMap version 30 2024-04-17T06:06:56.066 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.062+0000 7f6a339b3740 10 client.15873 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-17T06:06:56.066 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.062+0000 7f6a339b3740 20 client.15873 populate_metadata read hostname 'smithi047' 2024-04-17T06:06:56.066 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.062+0000 7f6a339b3740 1 --2- 172.21.15.47:0/3908354885 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x56330fec7af0 0x56330fec9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:06:56.066 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.062+0000 7f6a339b3740 4 client.15873 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T06:06:56.066 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.062+0000 7f6a339b3740 1 -- 172.21.15.47:0/3908354885 --> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56330fe211f0 con 0x56330fec7af0 2024-04-17T06:06:56.066 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.062+0000 7f6a2a59c640 1 --2- 172.21.15.47:0/3908354885 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x56330fec7af0 0x56330fec9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:06:56.066 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.062+0000 7f6a11ffb640 10 client.15873 ms_handle_connect on v2:172.21.15.98:6838/2841287917 2024-04-17T06:06:56.066 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.062+0000 7f6a2a59c640 1 --2- 172.21.15.47:0/3908354885 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x56330fec7af0 0x56330fec9ee0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31fa52a98aa9580f server_cookie=41dadb9787a34a71 in_seq=0 out_seq=0 2024-04-17T06:06:56.068 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.063+0000 7f6a11ffb640 1 -- 172.21.15.47:0/3908354885 <== mds.0 v2:172.21.15.98:6838/2841287917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56330fe211f0 con 0x56330fec7af0 2024-04-17T06:06:56.068 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.063+0000 7f6a11ffb640 10 client.15873 handle_command_reply: tid=0 2024-04-17T06:06:56.068 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.063+0000 7f6a339b3740 1 client.15873 shutdown 2024-04-17T06:06:56.068 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.063+0000 7f6a339b3740 1 -- 172.21.15.47:0/3908354885 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56330feae0c0 msgr2=0x56330feb47e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:06:56.068 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.063+0000 7f6a339b3740 1 --2- 172.21.15.47:0/3908354885 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56330feae0c0 0x56330feb47e0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f6a20005fd0 tx=0x7f6a20014c90 comp rx=0 tx=0).stop 2024-04-17T06:06:56.068 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.063+0000 7f6a339b3740 1 -- 172.21.15.47:0/3908354885 shutdown_connections 2024-04-17T06:06:56.068 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.063+0000 7f6a339b3740 1 --2- 172.21.15.47:0/3908354885 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x56330fec7af0 0x56330fec9ee0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:56.068 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.063+0000 7f6a339b3740 1 --2- 172.21.15.47:0/3908354885 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56330feb77b0 0x56330feb5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:56.068 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.063+0000 7f6a339b3740 1 --2- 172.21.15.47:0/3908354885 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56330feae9e0 0x56330feb4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:56.068 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.063+0000 7f6a11ffb640 0 client.15873 ms_handle_reset on v2:172.21.15.98:6838/2841287917 2024-04-17T06:06:56.068 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.063+0000 7f6a339b3740 1 --2- 172.21.15.47:0/3908354885 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56330feae0c0 0x56330feb47e0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:56.068 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.063+0000 7f6a339b3740 1 -- 172.21.15.47:0/3908354885 >> 172.21.15.47:0/3908354885 conn(0x56330fea89e0 msgr2=0x56330feaa730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:06:56.068 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.064+0000 7f6a339b3740 1 -- 172.21.15.47:0/3908354885 shutdown_connections 2024-04-17T06:06:56.068 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.064+0000 7f6a339b3740 1 -- 172.21.15.47:0/3908354885 wait complete. 2024-04-17T06:06:56.068 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.064+0000 7f6a339b3740 20 client.15873 trim_cache size 0 max 16384 2024-04-17T06:06:56.081 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.079+0000 7f6a339b3740 1 Processor -- start 2024-04-17T06:06:56.081 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.079+0000 7f6a339b3740 1 -- start start 2024-04-17T06:06:56.082 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.079+0000 7f6a339b3740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56330feae0c0 0x56330feb6f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:06:56.082 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.079+0000 7f6a339b3740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56330feae9e0 0x56330fec3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:06:56.082 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.079+0000 7f6a339b3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56330feb77b0 0x56330fea7980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:06:56.082 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.079+0000 7f6a339b3740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f6a2c1b25a0 con 0x56330feae9e0 2024-04-17T06:06:56.082 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.079+0000 7f6a339b3740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6a2c1afe60 con 0x56330feae0c0 2024-04-17T06:06:56.082 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.079+0000 7f6a339b3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6a2c1adcb0 con 0x56330feb77b0 2024-04-17T06:06:56.082 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.079+0000 7f6a2a59c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56330feae9e0 0x56330fec3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:06:56.082 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.079+0000 7f6a2a59c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56330feae9e0 0x56330fec3240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55652/0 (socket says 172.21.15.47:55652) 2024-04-17T06:06:56.082 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.079+0000 7f6a2a59c640 1 -- 172.21.15.47:0/2384857360 learned_addr learned my addr 172.21.15.47:0/2384857360 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:06:56.082 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.079+0000 7f6a29d9b640 1 --2- 172.21.15.47:0/2384857360 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56330feae0c0 0x56330feb6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:06:56.082 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.079+0000 7f6a2a59c640 1 -- 172.21.15.47:0/2384857360 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56330feb77b0 msgr2=0x56330fea7980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:06:56.082 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.079+0000 7f6a2a59c640 1 --2- 172.21.15.47:0/2384857360 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56330feb77b0 0x56330fea7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:56.083 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.079+0000 7f6a2a59c640 1 -- 172.21.15.47:0/2384857360 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56330feae0c0 msgr2=0x56330feb6f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:06:56.083 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.079+0000 7f6a2a59c640 1 --2- 172.21.15.47:0/2384857360 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56330feae0c0 0x56330feb6f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:56.083 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.079+0000 7f6a2a59c640 1 -- 172.21.15.47:0/2384857360 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56330feb9ba0 con 0x56330feae9e0 2024-04-17T06:06:56.083 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.079+0000 7f6a2a59c640 1 --2- 172.21.15.47:0/2384857360 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56330feae9e0 0x56330fec3240 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x56330feadb70 tx=0x7f6a1c005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16e63570ebe49ece server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:06:56.083 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.080+0000 7f6a13fff640 1 -- 172.21.15.47:0/2384857360 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a1c042040 con 0x56330feae9e0 2024-04-17T06:06:56.083 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.080+0000 7f6a13fff640 1 -- 172.21.15.47:0/2384857360 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a1c04ebc0 con 0x56330feae9e0 2024-04-17T06:06:56.083 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.080+0000 7f6a13fff640 1 -- 172.21.15.47:0/2384857360 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a1c04e3b0 con 0x56330feae9e0 2024-04-17T06:06:56.083 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.080+0000 7f6a339b3740 1 -- 172.21.15.47:0/2384857360 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56330feae9e0 msgr2=0x56330fec3240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:06:56.083 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.080+0000 7f6a339b3740 1 --2- 172.21.15.47:0/2384857360 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56330feae9e0 0x56330fec3240 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x56330feadb70 tx=0x7f6a1c005c00 comp rx=0 tx=0).stop 2024-04-17T06:06:56.083 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.080+0000 7f6a339b3740 1 -- 172.21.15.47:0/2384857360 shutdown_connections 2024-04-17T06:06:56.083 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.080+0000 7f6a339b3740 1 --2- 172.21.15.47:0/2384857360 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56330feb77b0 0x56330fea7980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:56.083 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.080+0000 7f6a339b3740 1 --2- 172.21.15.47:0/2384857360 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56330feae9e0 0x56330fec3240 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:56.084 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.080+0000 7f6a339b3740 1 --2- 172.21.15.47:0/2384857360 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56330feae0c0 0x56330feb6f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:56.084 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.080+0000 7f6a339b3740 1 -- 172.21.15.47:0/2384857360 >> 172.21.15.47:0/2384857360 conn(0x56330fea89e0 msgr2=0x56330fea7270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:06:56.084 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.080+0000 7f6a339b3740 1 -- 172.21.15.47:0/2384857360 shutdown_connections 2024-04-17T06:06:56.084 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.080+0000 7f6a339b3740 1 -- 172.21.15.47:0/2384857360 wait complete. 2024-04-17T06:06:56.084 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.081+0000 7f6a339b3740 1 -- start start 2024-04-17T06:06:56.084 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.081+0000 7f6a339b3740 1 Processor -- start 2024-04-17T06:06:56.084 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.081+0000 7f6a339b3740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56330feae0c0 0x56330febf820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:06:56.084 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.081+0000 7f6a339b3740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56330feae9e0 0x56330febad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:06:56.084 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.081+0000 7f6a339b3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56330feb77b0 0x56330febb270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:06:56.084 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.081+0000 7f6a339b3740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f6a2c1ae320 con 0x56330feae9e0 2024-04-17T06:06:56.084 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.081+0000 7f6a339b3740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6a2c1af790 con 0x56330feae0c0 2024-04-17T06:06:56.084 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.081+0000 7f6a339b3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6a2c1ae9f0 con 0x56330feb77b0 2024-04-17T06:06:56.084 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.081+0000 7f6a339b3740 10 client.0 _ll_register_callbacks cb 0x7f6a20014750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:06:56.084 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.081+0000 7f6a2a59c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56330feae9e0 0x56330febad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:06:56.085 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.081+0000 7f6a2a59c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56330feae9e0 0x56330febad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55656/0 (socket says 172.21.15.47:55656) 2024-04-17T06:06:56.085 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.081+0000 7f6a2a59c640 1 -- 172.21.15.47:0/3490048298 learned_addr learned my addr 172.21.15.47:0/3490048298 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:06:56.085 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.081+0000 7f6a29d9b640 1 --2- 172.21.15.47:0/3490048298 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56330feae0c0 0x56330febf820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:06:56.085 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.081+0000 7f6a2a59c640 1 -- 172.21.15.47:0/3490048298 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56330feb77b0 msgr2=0x56330febb270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:06:56.085 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.081+0000 7f6a2a59c640 1 --2- 172.21.15.47:0/3490048298 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56330feb77b0 0x56330febb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:56.085 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.081+0000 7f6a2a59c640 1 -- 172.21.15.47:0/3490048298 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56330feae0c0 msgr2=0x56330febf820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:06:56.085 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.081+0000 7f6a2a59c640 1 --2- 172.21.15.47:0/3490048298 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56330feae0c0 0x56330febf820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:56.085 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.081+0000 7f6a2a59c640 1 -- 172.21.15.47:0/3490048298 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56330febb7b0 con 0x56330feae9e0 2024-04-17T06:06:56.085 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.081+0000 7f6a2a59c640 1 --2- 172.21.15.47:0/3490048298 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56330feae9e0 0x56330febad30 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c009900 tx=0x7f6a1c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0b136860223bb4a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:06:56.085 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.081+0000 7f6a11ffb640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-17T06:06:56.085 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.082+0000 7f6a11ffb640 1 -- 172.21.15.47:0/3490048298 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a1c042050 con 0x56330feae9e0 2024-04-17T06:06:56.085 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.082+0000 7f6a11ffb640 1 -- 172.21.15.47:0/3490048298 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f6a1c02c880 con 0x56330feae9e0 2024-04-17T06:06:56.085 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.082+0000 7f6a339b3740 1 -- 172.21.15.47:0/3490048298 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a1c04ebc0 con 0x56330feae9e0 2024-04-17T06:06:56.085 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.082+0000 7f6a11ffb640 1 -- 172.21.15.47:0/3490048298 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6a1c0044b0 con 0x56330feae9e0 2024-04-17T06:06:56.086 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.082+0000 7f6a339b3740 10 client.15879 fetch_fsmap learned FSMap version 30 2024-04-17T06:06:56.086 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.082+0000 7f6a339b3740 1 -- 172.21.15.47:0/3490048298 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56330ff2dd80 con 0x56330feae9e0 2024-04-17T06:06:56.086 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.082+0000 7f6a11ffb640 1 -- 172.21.15.47:0/3490048298 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a1c04ebc0 con 0x56330feae9e0 2024-04-17T06:06:56.086 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.082+0000 7f6a11ffb640 1 -- 172.21.15.47:0/3490048298 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 30) ==== 8339+0+0 (secure 0 0 0) 0x7f6a1c005130 con 0x56330feae9e0 2024-04-17T06:06:56.086 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.082+0000 7f6a339b3740 10 client.15879 fetch_fsmap finished waiting for FSMap version 30 2024-04-17T06:06:56.086 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.082+0000 7f6a339b3740 10 client.15879 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-17T06:06:56.086 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.082+0000 7f6a339b3740 20 client.15879 populate_metadata read hostname 'smithi047' 2024-04-17T06:06:56.086 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.082+0000 7f6a339b3740 1 --2- 172.21.15.47:0/3490048298 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x56330ff2e050 0x56330ff30440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:06:56.086 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.082+0000 7f6a339b3740 4 client.15879 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-17T06:06:56.086 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.082+0000 7f6a339b3740 1 -- 172.21.15.47:0/3490048298 --> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56330fe1b060 con 0x56330ff2e050 2024-04-17T06:06:56.086 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.083+0000 7f6a29d9b640 1 --2- 172.21.15.47:0/3490048298 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x56330ff2e050 0x56330ff30440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:06:56.086 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.083+0000 7f6a29d9b640 1 --2- 172.21.15.47:0/3490048298 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x56330ff2e050 0x56330ff30440 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f27cd40ef89cc434 server_cookie=1a8c6ba407ac664a in_seq=0 out_seq=0 2024-04-17T06:06:56.086 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.083+0000 7f6a11ffb640 10 client.15879 ms_handle_connect on v2:172.21.15.98:6838/2841287917 2024-04-17T06:06:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.091+0000 7f6a11ffb640 1 -- 172.21.15.47:0/3490048298 <== mds.0 v2:172.21.15.98:6838/2841287917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56330fe1b060 con 0x56330ff2e050 2024-04-17T06:06:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.091+0000 7f6a11ffb640 10 client.15879 handle_command_reply: tid=0 2024-04-17T06:06:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.091+0000 7f6a339b3740 1 client.15879 shutdown 2024-04-17T06:06:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.091+0000 7f6a339b3740 1 -- 172.21.15.47:0/3490048298 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56330feae9e0 msgr2=0x56330febad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:06:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.091+0000 7f6a339b3740 1 --2- 172.21.15.47:0/3490048298 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56330feae9e0 0x56330febad30 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c009900 tx=0x7f6a1c0047c0 comp rx=0 tx=0).stop 2024-04-17T06:06:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.091+0000 7f6a339b3740 1 -- 172.21.15.47:0/3490048298 shutdown_connections 2024-04-17T06:06:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.091+0000 7f6a339b3740 1 --2- 172.21.15.47:0/3490048298 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x56330ff2e050 0x56330ff30440 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.091+0000 7f6a339b3740 1 --2- 172.21.15.47:0/3490048298 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56330feb77b0 0x56330febb270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:56.091 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.091+0000 7f6a339b3740 1 --2- 172.21.15.47:0/3490048298 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56330feae9e0 0x56330febad30 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:56.091 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.091+0000 7f6a11ffb640 0 client.15879 ms_handle_reset on v2:172.21.15.98:6838/2841287917 2024-04-17T06:06:56.091 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.091+0000 7f6a339b3740 1 --2- 172.21.15.47:0/3490048298 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56330feae0c0 0x56330febf820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:56.091 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.091+0000 7f6a339b3740 1 -- 172.21.15.47:0/3490048298 >> 172.21.15.47:0/3490048298 conn(0x56330fea89e0 msgr2=0x56330feb2cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:06:56.091 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.091+0000 7f6a339b3740 1 -- 172.21.15.47:0/3490048298 shutdown_connections 2024-04-17T06:06:56.091 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.091+0000 7f6a339b3740 1 -- 172.21.15.47:0/3490048298 wait complete. 2024-04-17T06:06:56.091 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.091+0000 7f6a339b3740 20 client.15879 trim_cache size 0 max 16384 2024-04-17T06:06:56.091 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-04-17T06:06:56.091 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-04-17T06:06:56.091 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-04-17T06:06:56.091 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-04-17T06:06:56.092 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.092+0000 7f6a30dc8640 1 -- 172.21.15.47:0/4268494183 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f6a140a3df0 msgr2=0x7f6a140a62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:06:56.092 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.092+0000 7f6a30dc8640 1 --2- 172.21.15.47:0/4268494183 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f6a140a3df0 0x7f6a140a62b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6a2409c700 tx=0x7f6a240c7880 comp rx=0 tx=0).stop 2024-04-17T06:06:56.092 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.092+0000 7f6a30dc8640 1 -- 172.21.15.47:0/4268494183 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a2c1020c0 msgr2=0x7f6a2c1a6950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:06:56.092 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.092+0000 7f6a30dc8640 1 --2- 172.21.15.47:0/4268494183 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a2c1020c0 0x7f6a2c1a6950 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f6a2400c970 tx=0x7f6a2400ce30 comp rx=0 tx=0).stop 2024-04-17T06:06:56.092 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.092+0000 7f6a30dc8640 1 -- 172.21.15.47:0/4268494183 shutdown_connections 2024-04-17T06:06:56.093 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.092+0000 7f6a30dc8640 1 --2- 172.21.15.47:0/4268494183 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f6a140a3df0 0x7f6a140a62b0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:56.093 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.092+0000 7f6a30dc8640 1 --2- 172.21.15.47:0/4268494183 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a2c1020c0 0x7f6a2c1a6950 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:56.093 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.092+0000 7f6a30dc8640 1 --2- 172.21.15.47:0/4268494183 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6a2c107290 0x7f6a2c101b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:56.093 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.092+0000 7f6a30dc8640 1 --2- 172.21.15.47:0/4268494183 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6a2c1006d0 0x7f6a2c101640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:06:56.093 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.092+0000 7f6a30dc8640 1 -- 172.21.15.47:0/4268494183 >> 172.21.15.47:0/4268494183 conn(0x7f6a2c0773a0 msgr2=0x7f6a2c105280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:06:56.093 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.092+0000 7f6a30dc8640 1 -- 172.21.15.47:0/4268494183 shutdown_connections 2024-04-17T06:06:56.093 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:06:56.092+0000 7f6a30dc8640 1 -- 172.21.15.47:0/4268494183 wait complete. 2024-04-17T06:06:56.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:06:55 smithi098 ceph-mon[36516]: pgmap v173: 97 pgs: 97 active+clean; 2.9 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 16 KiB/s wr, 16 op/s 2024-04-17T06:06:56.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:06:55 smithi047 ceph-mon[32223]: pgmap v173: 97 pgs: 97 active+clean; 2.9 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 16 KiB/s wr, 16 op/s 2024-04-17T06:06:58.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:06:57 smithi103 ceph-mon[36243]: pgmap v174: 97 pgs: 97 active+clean; 2.9 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 7.2 KiB/s wr, 9 op/s 2024-04-17T06:06:58.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:06:57 smithi098 ceph-mon[36516]: pgmap v174: 97 pgs: 97 active+clean; 2.9 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 7.2 KiB/s wr, 9 op/s 2024-04-17T06:06:58.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:06:57 smithi047 ceph-mon[32223]: pgmap v174: 97 pgs: 97 active+clean; 2.9 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 7.2 KiB/s wr, 9 op/s 2024-04-17T06:07:00.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:06:59 smithi103 ceph-mon[36243]: pgmap v175: 97 pgs: 97 active+clean; 2.9 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 7.2 KiB/s wr, 9 op/s 2024-04-17T06:07:00.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:06:59 smithi098 ceph-mon[36516]: pgmap v175: 97 pgs: 97 active+clean; 2.9 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 7.2 KiB/s wr, 9 op/s 2024-04-17T06:07:00.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:06:59 smithi047 ceph-mon[32223]: pgmap v175: 97 pgs: 97 active+clean; 2.9 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 7.2 KiB/s wr, 9 op/s 2024-04-17T06:07:02.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:07:01 smithi103 ceph-mon[36243]: pgmap v176: 97 pgs: 97 active+clean; 3.0 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-04-17T06:07:02.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:07:01 smithi098 ceph-mon[36516]: pgmap v176: 97 pgs: 97 active+clean; 3.0 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-04-17T06:07:02.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:07:01 smithi047 ceph-mon[32223]: pgmap v176: 97 pgs: 97 active+clean; 3.0 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-04-17T06:07:03.763 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:07:03.763 INFO:tasks.workunit.client.0.smithi047.stdout:189/600: open fd = 3 2024-04-17T06:07:03.764 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.764 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.764 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:07:03.764 INFO:tasks.workunit.client.0.smithi047.stdout:190/600: open fd = 3 2024-04-17T06:07:03.764 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.764 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.764 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:07:03.764 INFO:tasks.workunit.client.0.smithi047.stdout:191/600: open fd = 3 2024-04-17T06:07:03.764 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.764 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.764 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:07:03.764 INFO:tasks.workunit.client.0.smithi047.stdout:192/600: open fd = 3 2024-04-17T06:07:03.764 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.764 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.765 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:07:03.765 INFO:tasks.workunit.client.0.smithi047.stdout:193/600: open fd = 3 2024-04-17T06:07:03.765 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.765 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.765 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:07:03.765 INFO:tasks.workunit.client.0.smithi047.stdout:194/600: open fd = 3 2024-04-17T06:07:03.765 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.765 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.765 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:07:03.765 INFO:tasks.workunit.client.0.smithi047.stdout:195/600: open fd = 3 2024-04-17T06:07:03.765 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.765 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.765 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:07:03.765 INFO:tasks.workunit.client.0.smithi047.stdout:196/600: open fd = 3 2024-04-17T06:07:03.765 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.766 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.766 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:07:03.766 INFO:tasks.workunit.client.0.smithi047.stdout:197/600: open fd = 3 2024-04-17T06:07:03.766 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.766 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.766 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:07:03.766 INFO:tasks.workunit.client.0.smithi047.stdout:198/600: open fd = 3 2024-04-17T06:07:03.766 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.766 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.766 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:07:03.766 INFO:tasks.workunit.client.0.smithi047.stdout:199/600: open fd = 3 2024-04-17T06:07:03.766 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.766 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.766 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:07:03.767 INFO:tasks.workunit.client.0.smithi047.stdout:200/600: open fd = 3 2024-04-17T06:07:03.767 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.767 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.767 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:07:03.767 INFO:tasks.workunit.client.0.smithi047.stdout:201/600: open fd = 3 2024-04-17T06:07:03.767 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.767 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.767 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:07:03.767 INFO:tasks.workunit.client.0.smithi047.stdout:202/600: open fd = 3 2024-04-17T06:07:03.767 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.767 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.767 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:07:03.767 INFO:tasks.workunit.client.0.smithi047.stdout:203/600: open fd = 3 2024-04-17T06:07:03.767 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.767 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.768 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:07:03.768 INFO:tasks.workunit.client.0.smithi047.stdout:204/600: open fd = 3 2024-04-17T06:07:03.768 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.768 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.768 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:07:03.768 INFO:tasks.workunit.client.0.smithi047.stdout:205/600: open fd = 3 2024-04-17T06:07:03.768 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.768 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.768 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:07:03.768 INFO:tasks.workunit.client.0.smithi047.stdout:206/600: open fd = 3 2024-04-17T06:07:03.768 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.768 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.768 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:07:03.768 INFO:tasks.workunit.client.0.smithi047.stdout:207/600: open fd = 3 2024-04-17T06:07:03.768 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.768 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.769 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:07:03.769 INFO:tasks.workunit.client.0.smithi047.stdout:208/600: open fd = 3 2024-04-17T06:07:03.769 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.769 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.769 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:07:03.769 INFO:tasks.workunit.client.0.smithi047.stdout:209/600: open fd = 3 2024-04-17T06:07:03.769 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.769 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.769 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:07:03.769 INFO:tasks.workunit.client.0.smithi047.stdout:210/600: open fd = 3 2024-04-17T06:07:03.769 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.769 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.769 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:07:03.770 INFO:tasks.workunit.client.0.smithi047.stdout:211/600: open fd = 3 2024-04-17T06:07:03.770 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.770 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.770 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:07:03.770 INFO:tasks.workunit.client.0.smithi047.stdout:212/600: open fd = 3 2024-04-17T06:07:03.770 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.770 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.770 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:07:03.770 INFO:tasks.workunit.client.0.smithi047.stdout:213/600: open fd = 3 2024-04-17T06:07:03.770 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.770 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.770 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:07:03.770 INFO:tasks.workunit.client.0.smithi047.stdout:214/600: open fd = 3 2024-04-17T06:07:03.770 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.770 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.771 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:07:03.771 INFO:tasks.workunit.client.0.smithi047.stdout:215/600: open fd = 3 2024-04-17T06:07:03.771 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.771 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.771 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:07:03.771 INFO:tasks.workunit.client.0.smithi047.stdout:216/600: open fd = 3 2024-04-17T06:07:03.771 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.771 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.771 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:07:03.771 INFO:tasks.workunit.client.0.smithi047.stdout:217/600: open fd = 3 2024-04-17T06:07:03.771 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.771 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.771 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:07:03.771 INFO:tasks.workunit.client.0.smithi047.stdout:218/600: open fd = 3 2024-04-17T06:07:03.771 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.772 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.772 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:07:03.772 INFO:tasks.workunit.client.0.smithi047.stdout:219/600: open fd = 3 2024-04-17T06:07:03.772 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.772 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.772 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:07:03.772 INFO:tasks.workunit.client.0.smithi047.stdout:220/600: open fd = 3 2024-04-17T06:07:03.772 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.772 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.772 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:07:03.772 INFO:tasks.workunit.client.0.smithi047.stdout:221/600: open fd = 3 2024-04-17T06:07:03.772 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.772 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.772 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:07:03.772 INFO:tasks.workunit.client.0.smithi047.stdout:222/600: open fd = 3 2024-04-17T06:07:03.773 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.773 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.773 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:07:03.773 INFO:tasks.workunit.client.0.smithi047.stdout:223/600: open fd = 3 2024-04-17T06:07:03.773 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.773 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.773 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:07:03.773 INFO:tasks.workunit.client.0.smithi047.stdout:224/600: open fd = 3 2024-04-17T06:07:03.773 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.773 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.773 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:07:03.773 INFO:tasks.workunit.client.0.smithi047.stdout:225/600: open fd = 3 2024-04-17T06:07:03.773 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.773 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.773 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:07:03.774 INFO:tasks.workunit.client.0.smithi047.stdout:226/600: open fd = 3 2024-04-17T06:07:03.774 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.774 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.774 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:07:03.774 INFO:tasks.workunit.client.0.smithi047.stdout:227/600: open fd = 3 2024-04-17T06:07:03.774 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.774 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.774 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:07:03.774 INFO:tasks.workunit.client.0.smithi047.stdout:228/600: open fd = 3 2024-04-17T06:07:03.774 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.774 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.774 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:07:03.774 INFO:tasks.workunit.client.0.smithi047.stdout:229/600: open fd = 3 2024-04-17T06:07:03.775 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.775 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.775 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:07:03.775 INFO:tasks.workunit.client.0.smithi047.stdout:230/600: open fd = 3 2024-04-17T06:07:03.775 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.775 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.775 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:07:03.775 INFO:tasks.workunit.client.0.smithi047.stdout:231/600: open fd = 3 2024-04-17T06:07:03.775 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.775 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.775 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:07:03.775 INFO:tasks.workunit.client.0.smithi047.stdout:232/600: open fd = 3 2024-04-17T06:07:03.775 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.775 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.775 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:07:03.776 INFO:tasks.workunit.client.0.smithi047.stdout:233/600: open fd = 3 2024-04-17T06:07:03.776 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.776 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.776 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:07:03.776 INFO:tasks.workunit.client.0.smithi047.stdout:234/600: open fd = 3 2024-04-17T06:07:03.776 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.776 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.776 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:07:03.776 INFO:tasks.workunit.client.0.smithi047.stdout:235/600: open fd = 3 2024-04-17T06:07:03.776 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.776 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.776 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:07:03.776 INFO:tasks.workunit.client.0.smithi047.stdout:236/600: open fd = 3 2024-04-17T06:07:03.776 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.777 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.777 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:07:03.777 INFO:tasks.workunit.client.0.smithi047.stdout:237/600: open fd = 3 2024-04-17T06:07:03.777 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.777 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.777 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:07:03.777 INFO:tasks.workunit.client.0.smithi047.stdout:238/600: open fd = 3 2024-04-17T06:07:03.777 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.777 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.777 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:07:03.777 INFO:tasks.workunit.client.0.smithi047.stdout:239/600: open fd = 3 2024-04-17T06:07:03.777 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.777 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.777 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:07:03.777 INFO:tasks.workunit.client.0.smithi047.stdout:240/600: open fd = 3 2024-04-17T06:07:03.778 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.778 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.778 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:07:03.778 INFO:tasks.workunit.client.0.smithi047.stdout:241/600: open fd = 3 2024-04-17T06:07:03.778 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.778 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.778 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:07:03.778 INFO:tasks.workunit.client.0.smithi047.stdout:242/600: open fd = 3 2024-04-17T06:07:03.778 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.778 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.778 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:07:03.778 INFO:tasks.workunit.client.0.smithi047.stdout:243/600: open fd = 3 2024-04-17T06:07:03.778 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.778 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.779 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:07:03.779 INFO:tasks.workunit.client.0.smithi047.stdout:244/600: open fd = 3 2024-04-17T06:07:03.779 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.779 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.779 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:07:03.779 INFO:tasks.workunit.client.0.smithi047.stdout:245/600: open fd = 3 2024-04-17T06:07:03.779 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.779 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.779 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:07:03.779 INFO:tasks.workunit.client.0.smithi047.stdout:246/600: open fd = 3 2024-04-17T06:07:03.779 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.779 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.779 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:07:03.779 INFO:tasks.workunit.client.0.smithi047.stdout:247/600: open fd = 3 2024-04-17T06:07:03.780 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.780 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.780 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:07:03.780 INFO:tasks.workunit.client.0.smithi047.stdout:248/600: open fd = 3 2024-04-17T06:07:03.780 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.780 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.780 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:07:03.780 INFO:tasks.workunit.client.0.smithi047.stdout:249/600: open fd = 3 2024-04-17T06:07:03.780 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.780 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.780 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:07:03.780 INFO:tasks.workunit.client.0.smithi047.stdout:250/600: open fd = 3 2024-04-17T06:07:03.780 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:03.780 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:07:04.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:07:03 smithi103 ceph-mon[36243]: pgmap v177: 97 pgs: 97 active+clean; 3.0 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-04-17T06:07:04.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:07:03 smithi098 ceph-mon[36516]: pgmap v177: 97 pgs: 97 active+clean; 3.0 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-04-17T06:07:04.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:07:03 smithi047 ceph-mon[32223]: pgmap v177: 97 pgs: 97 active+clean; 3.0 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-04-17T06:07:06.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:07:05 smithi103 ceph-mon[36243]: pgmap v178: 97 pgs: 97 active+clean; 3.0 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 38 KiB/s wr, 31 op/s 2024-04-17T06:07:06.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:07:05 smithi098 ceph-mon[36516]: pgmap v178: 97 pgs: 97 active+clean; 3.0 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 38 KiB/s wr, 31 op/s 2024-04-17T06:07:06.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:07:05 smithi047 ceph-mon[32223]: pgmap v178: 97 pgs: 97 active+clean; 3.0 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 38 KiB/s wr, 31 op/s 2024-04-17T06:07:08.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:07:07 smithi103 ceph-mon[36243]: pgmap v179: 97 pgs: 97 active+clean; 3.0 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 34 KiB/s wr, 26 op/s 2024-04-17T06:07:08.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:07:07 smithi098 ceph-mon[36516]: pgmap v179: 97 pgs: 97 active+clean; 3.0 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 34 KiB/s wr, 26 op/s 2024-04-17T06:07:08.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:07:07 smithi047 ceph-mon[32223]: pgmap v179: 97 pgs: 97 active+clean; 3.0 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 34 KiB/s wr, 26 op/s 2024-04-17T06:07:09.191 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.191+0000 7f9acefd4640 1 -- 172.21.15.47:0/1513497403 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ac8101100 msgr2=0x7f9ac81014e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:07:09.192 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.191+0000 7f9acefd4640 1 --2- 172.21.15.47:0/1513497403 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ac8101100 0x7f9ac81014e0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f9abc009900 tx=0x7f9abc02bc30 comp rx=0 tx=0).stop 2024-04-17T06:07:09.192 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.191+0000 7f9acefd4640 1 -- 172.21.15.47:0/1513497403 shutdown_connections 2024-04-17T06:07:09.192 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.191+0000 7f9acefd4640 1 --2- 172.21.15.47:0/1513497403 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ac8106180 0x7f9ac810dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:09.192 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.191+0000 7f9acefd4640 1 --2- 172.21.15.47:0/1513497403 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9ac8101ab0 0x7f9ac8105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:09.192 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.191+0000 7f9acefd4640 1 --2- 172.21.15.47:0/1513497403 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ac8101100 0x7f9ac81014e0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:09.192 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.191+0000 7f9acefd4640 1 -- 172.21.15.47:0/1513497403 >> 172.21.15.47:0/1513497403 conn(0x7f9ac8077fe0 msgr2=0x7f9ac80783f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:07:09.192 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.192+0000 7f9acefd4640 1 -- 172.21.15.47:0/1513497403 shutdown_connections 2024-04-17T06:07:09.192 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.192+0000 7f9acefd4640 1 -- 172.21.15.47:0/1513497403 wait complete. 2024-04-17T06:07:09.192 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.192+0000 7f9acefd4640 1 Processor -- start 2024-04-17T06:07:09.192 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.192+0000 7f9acefd4640 1 -- start start 2024-04-17T06:07:09.192 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.192+0000 7f9acefd4640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ac8101100 0x7f9ac819e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:07:09.192 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.192+0000 7f9acefd4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ac8106180 0x7f9ac819e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:07:09.192 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.192+0000 7f9acefd4640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9ac8198610 0x7f9ac8198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:07:09.193 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.192+0000 7f9acefd4640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f9ac8111240 con 0x7f9ac8101100 2024-04-17T06:07:09.193 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.192+0000 7f9acefd4640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9ac81110c0 con 0x7f9ac8198610 2024-04-17T06:07:09.193 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.192+0000 7f9acefd4640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9ac81113c0 con 0x7f9ac8106180 2024-04-17T06:07:09.193 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.193+0000 7f9accd70640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ac8101100 0x7f9ac819e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:07:09.193 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.193+0000 7f9accd70640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ac8101100 0x7f9ac819e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47124/0 (socket says 172.21.15.47:47124) 2024-04-17T06:07:09.193 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.193+0000 7f9accd70640 1 -- 172.21.15.47:0/1220511775 learned_addr learned my addr 172.21.15.47:0/1220511775 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:07:09.193 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.193+0000 7f9ac7fff640 1 --2- 172.21.15.47:0/1220511775 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ac8106180 0x7f9ac819e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:07:09.193 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.193+0000 7f9acd571640 1 --2- 172.21.15.47:0/1220511775 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9ac8198610 0x7f9ac8198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:07:09.193 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.193+0000 7f9accd70640 1 -- 172.21.15.47:0/1220511775 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ac8106180 msgr2=0x7f9ac819e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:07:09.193 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.193+0000 7f9accd70640 1 --2- 172.21.15.47:0/1220511775 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ac8106180 0x7f9ac819e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:09.193 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.193+0000 7f9accd70640 1 -- 172.21.15.47:0/1220511775 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9ac8198610 msgr2=0x7f9ac8198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:07:09.193 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.193+0000 7f9accd70640 1 --2- 172.21.15.47:0/1220511775 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9ac8198610 0x7f9ac8198ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:09.193 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.193+0000 7f9accd70640 1 -- 172.21.15.47:0/1220511775 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ac8199130 con 0x7f9ac8101100 2024-04-17T06:07:09.193 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.193+0000 7f9acd571640 1 --2- 172.21.15.47:0/1220511775 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9ac8198610 0x7f9ac8198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:07:09.194 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.193+0000 7f9accd70640 1 --2- 172.21.15.47:0/1220511775 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ac8101100 0x7f9ac819e460 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f9abc006010 tx=0x7f9abc02cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:07:09.194 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.193+0000 7f9ac5ffb640 1 -- 172.21.15.47:0/1220511775 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9abc041070 con 0x7f9ac8101100 2024-04-17T06:07:09.194 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.193+0000 7f9ac5ffb640 1 -- 172.21.15.47:0/1220511775 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9abc004450 con 0x7f9ac8101100 2024-04-17T06:07:09.194 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.193+0000 7f9acefd4640 1 -- 172.21.15.47:0/1220511775 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ac81993c0 con 0x7f9ac8101100 2024-04-17T06:07:09.194 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.193+0000 7f9ac5ffb640 1 -- 172.21.15.47:0/1220511775 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9abc034440 con 0x7f9ac8101100 2024-04-17T06:07:09.194 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.194+0000 7f9acefd4640 1 -- 172.21.15.47:0/1220511775 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ac81a55b0 con 0x7f9ac8101100 2024-04-17T06:07:09.199 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.196+0000 7f9ac5ffb640 1 -- 172.21.15.47:0/1220511775 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9abc044070 con 0x7f9ac8101100 2024-04-17T06:07:09.199 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.196+0000 7f9ad1bbe740 1 Processor -- start 2024-04-17T06:07:09.200 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.196+0000 7f9ad1bbe740 1 -- start start 2024-04-17T06:07:09.200 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.197+0000 7f9ad1bbe740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5610107ea0c0 0x5610107ea4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:07:09.200 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.197+0000 7f9ad1bbe740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610107ea9e0 0x5610107f3270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:07:09.200 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.197+0000 7f9ad1bbe740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5610107f37b0 0x5610107f5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:07:09.200 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.197+0000 7f9ad1bbe740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x56101074f7d0 con 0x5610107f37b0 2024-04-17T06:07:09.200 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.197+0000 7f9ad1bbe740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5610106ecf70 con 0x5610107ea9e0 2024-04-17T06:07:09.200 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.197+0000 7f9ad1bbe740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56101074b790 con 0x5610107ea0c0 2024-04-17T06:07:09.200 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.197+0000 7f9ac5ffb640 1 --2- 172.21.15.47:0/1220511775 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f9ab80a3dc0 0x7f9ab80a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:07:09.200 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.197+0000 7f9ac7fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5610107f37b0 0x5610107f5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:07:09.200 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.197+0000 7f9ac7fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5610107f37b0 0x5610107f5ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47132/0 (socket says 172.21.15.47:47132) 2024-04-17T06:07:09.200 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.197+0000 7f9ac7fff640 1 -- 172.21.15.47:0/3820564457 learned_addr learned my addr 172.21.15.47:0/3820564457 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:07:09.200 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.197+0000 7f9acd571640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5610107ea0c0 0x5610107ea4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:07:09.201 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.197+0000 7f9acd571640 1 --2- 172.21.15.47:0/1220511775 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f9ab80a3dc0 0x7f9ab80a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:07:09.201 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.197+0000 7f9ac5ffb640 1 -- 172.21.15.47:0/1220511775 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f9abc0f2330 con 0x7f9ac8101100 2024-04-17T06:07:09.201 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.197+0000 7f9accd70640 1 --2- 172.21.15.47:0/3820564457 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610107ea9e0 0x5610107f3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:07:09.201 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.197+0000 7f9ac7fff640 1 -- 172.21.15.47:0/3820564457 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5610107ea0c0 msgr2=0x5610107ea4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:07:09.201 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.197+0000 7f9ac7fff640 1 --2- 172.21.15.47:0/3820564457 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5610107ea0c0 0x5610107ea4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:09.201 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.197+0000 7f9ac7fff640 1 -- 172.21.15.47:0/3820564457 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610107ea9e0 msgr2=0x5610107f3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:07:09.201 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.197+0000 7f9ac7fff640 1 --2- 172.21.15.47:0/3820564457 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610107ea9e0 0x5610107f3270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:09.201 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.197+0000 7f9ac7fff640 1 -- 172.21.15.47:0/3820564457 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561010462210 con 0x5610107f37b0 2024-04-17T06:07:09.201 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.198+0000 7f9acd571640 1 --2- 172.21.15.47:0/1220511775 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f9ab80a3dc0 0x7f9ab80a6280 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9ac0009fd0 tx=0x7f9ac0005bd0 comp rx=0 tx=0).ready entity=mgr.24923 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:07:09.201 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.198+0000 7f9ac7fff640 1 --2- 172.21.15.47:0/3820564457 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5610107f37b0 0x5610107f5ba0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f9ab400bc60 tx=0x7f9ab40107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95893159edebb6f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:07:09.201 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.198+0000 7f9ab2ffd640 1 -- 172.21.15.47:0/3820564457 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab401c480 con 0x5610107f37b0 2024-04-17T06:07:09.201 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.198+0000 7f9ab2ffd640 1 -- 172.21.15.47:0/3820564457 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ab401cb00 con 0x5610107f37b0 2024-04-17T06:07:09.201 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.198+0000 7f9ab2ffd640 1 -- 172.21.15.47:0/3820564457 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab4011cb0 con 0x5610107f37b0 2024-04-17T06:07:09.201 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.198+0000 7f9ad1bbe740 1 -- 172.21.15.47:0/3820564457 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5610107f37b0 msgr2=0x5610107f5ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:07:09.201 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.198+0000 7f9ad1bbe740 1 --2- 172.21.15.47:0/3820564457 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5610107f37b0 0x5610107f5ba0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f9ab400bc60 tx=0x7f9ab40107a0 comp rx=0 tx=0).stop 2024-04-17T06:07:09.202 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.199+0000 7f9ad1bbe740 1 -- 172.21.15.47:0/3820564457 shutdown_connections 2024-04-17T06:07:09.202 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.199+0000 7f9ad1bbe740 1 --2- 172.21.15.47:0/3820564457 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5610107f37b0 0x5610107f5ba0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:09.202 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.199+0000 7f9ad1bbe740 1 --2- 172.21.15.47:0/3820564457 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610107ea9e0 0x5610107f3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:09.202 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.199+0000 7f9ad1bbe740 1 --2- 172.21.15.47:0/3820564457 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5610107ea0c0 0x5610107ea4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:09.202 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.199+0000 7f9ad1bbe740 1 -- 172.21.15.47:0/3820564457 >> 172.21.15.47:0/3820564457 conn(0x5610107e49e0 msgr2=0x5610107e4dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:07:09.202 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.199+0000 7f9ad1bbe740 1 -- 172.21.15.47:0/3820564457 shutdown_connections 2024-04-17T06:07:09.202 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.199+0000 7f9ad1bbe740 1 -- 172.21.15.47:0/3820564457 wait complete. 2024-04-17T06:07:09.202 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.199+0000 7f9ad1bbe740 1 -- start start 2024-04-17T06:07:09.202 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.199+0000 7f9ad1bbe740 1 Processor -- start 2024-04-17T06:07:09.202 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.199+0000 7f9ad1bbe740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5610107ea0c0 0x5610107f0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:07:09.202 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.199+0000 7f9ad1bbe740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610107ea9e0 0x5610107f0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:07:09.202 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.200+0000 7f9ad1bbe740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5610107f37b0 0x5610107f11f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:07:09.202 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.200+0000 7f9ad1bbe740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5610107f63e0 con 0x5610107f37b0 2024-04-17T06:07:09.203 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.200+0000 7f9ad1bbe740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5610106d7e90 con 0x5610107ea9e0 2024-04-17T06:07:09.203 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.200+0000 7f9ad1bbe740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5610107f6560 con 0x5610107ea0c0 2024-04-17T06:07:09.203 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.200+0000 7f9ad1bbe740 10 client.0 _ll_register_callbacks cb 0x5610106edd40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:07:09.203 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.200+0000 7f9acd571640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5610107f37b0 0x5610107f11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:07:09.203 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.200+0000 7f9acd571640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5610107f37b0 0x5610107f11f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47134/0 (socket says 172.21.15.47:47134) 2024-04-17T06:07:09.203 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.200+0000 7f9acd571640 1 -- 172.21.15.47:0/400975197 learned_addr learned my addr 172.21.15.47:0/400975197 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:07:09.203 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.200+0000 7f9accd70640 1 --2- 172.21.15.47:0/400975197 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5610107ea0c0 0x5610107f0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:07:09.203 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.200+0000 7f9ac7fff640 1 --2- 172.21.15.47:0/400975197 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610107ea9e0 0x5610107f0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:07:09.203 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.200+0000 7f9acd571640 1 -- 172.21.15.47:0/400975197 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5610107ea0c0 msgr2=0x5610107f0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:07:09.203 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.200+0000 7f9acd571640 1 --2- 172.21.15.47:0/400975197 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5610107ea0c0 0x5610107f0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:09.203 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.200+0000 7f9acd571640 1 -- 172.21.15.47:0/400975197 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610107ea9e0 msgr2=0x5610107f0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:07:09.203 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.200+0000 7f9acd571640 1 --2- 172.21.15.47:0/400975197 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610107ea9e0 0x5610107f0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:09.203 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.200+0000 7f9acd571640 1 -- 172.21.15.47:0/400975197 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56101052e620 con 0x5610107f37b0 2024-04-17T06:07:09.203 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.200+0000 7f9accd70640 1 --2- 172.21.15.47:0/400975197 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5610107ea0c0 0x5610107f0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:07:09.204 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.200+0000 7f9ac7fff640 1 --2- 172.21.15.47:0/400975197 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610107ea9e0 0x5610107f0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:07:09.204 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.200+0000 7f9acd571640 1 --2- 172.21.15.47:0/400975197 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5610107f37b0 0x5610107f11f0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f9ac0012f90 tx=0x7f9ac0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46e12bffb69bc5a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:07:09.204 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.200+0000 7f9aa7fff640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-17T06:07:09.204 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.200+0000 7f9aa7fff640 1 -- 172.21.15.47:0/400975197 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ac0025070 con 0x5610107f37b0 2024-04-17T06:07:09.205 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.200+0000 7f9aa7fff640 1 -- 172.21.15.47:0/400975197 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f9ac000fc00 con 0x5610107f37b0 2024-04-17T06:07:09.205 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.200+0000 7f9ad1bbe740 1 -- 172.21.15.47:0/400975197 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ab401cb00 con 0x5610107f37b0 2024-04-17T06:07:09.205 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.201+0000 7f9aa7fff640 1 -- 172.21.15.47:0/400975197 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9ac002a460 con 0x5610107f37b0 2024-04-17T06:07:09.205 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.201+0000 7f9ad1bbe740 10 client.15891 fetch_fsmap learned FSMap version 30 2024-04-17T06:07:09.205 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.201+0000 7f9ad1bbe740 1 -- 172.21.15.47:0/400975197 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5610108025e0 con 0x5610107f37b0 2024-04-17T06:07:09.205 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.201+0000 7f9aa7fff640 1 -- 172.21.15.47:0/400975197 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ab401cb00 con 0x5610107f37b0 2024-04-17T06:07:09.206 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.201+0000 7f9aa7fff640 1 -- 172.21.15.47:0/400975197 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 30) ==== 8339+0+0 (secure 0 0 0) 0x7f9ac001e040 con 0x5610107f37b0 2024-04-17T06:07:09.206 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.201+0000 7f9ad1bbe740 10 client.15891 fetch_fsmap finished waiting for FSMap version 30 2024-04-17T06:07:09.206 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.201+0000 7f9ad1bbe740 10 client.15891 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-17T06:07:09.206 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.201+0000 7f9ad1bbe740 20 client.15891 populate_metadata read hostname 'smithi047' 2024-04-17T06:07:09.206 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.201+0000 7f9ad1bbe740 1 --2- 172.21.15.47:0/400975197 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x561010803b10 0x561010805f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:07:09.206 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.201+0000 7f9ad1bbe740 4 client.15891 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T06:07:09.206 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.201+0000 7f9ad1bbe740 1 -- 172.21.15.47:0/400975197 --> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56101075d1f0 con 0x561010803b10 2024-04-17T06:07:09.206 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.201+0000 7f9accd70640 1 --2- 172.21.15.47:0/400975197 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x561010803b10 0x561010805f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:07:09.206 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.202+0000 7f9aa7fff640 10 client.15891 ms_handle_connect on v2:172.21.15.98:6838/2841287917 2024-04-17T06:07:09.206 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.202+0000 7f9accd70640 1 --2- 172.21.15.47:0/400975197 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x561010803b10 0x561010805f40 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a5ecf3b63751f77 server_cookie=5000adc4f91f629b in_seq=0 out_seq=0 2024-04-17T06:07:09.207 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.203+0000 7f9aa7fff640 1 -- 172.21.15.47:0/400975197 <== mds.0 v2:172.21.15.98:6838/2841287917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56101075d1f0 con 0x561010803b10 2024-04-17T06:07:09.207 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.203+0000 7f9aa7fff640 10 client.15891 handle_command_reply: tid=0 2024-04-17T06:07:09.208 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.203+0000 7f9ad1bbe740 1 client.15891 shutdown 2024-04-17T06:07:09.208 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.203+0000 7f9ad1bbe740 1 -- 172.21.15.47:0/400975197 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5610107f37b0 msgr2=0x5610107f11f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:07:09.208 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.203+0000 7f9ad1bbe740 1 --2- 172.21.15.47:0/400975197 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5610107f37b0 0x5610107f11f0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f9ac0012f90 tx=0x7f9ac0004ab0 comp rx=0 tx=0).stop 2024-04-17T06:07:09.208 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.203+0000 7f9ad1bbe740 1 -- 172.21.15.47:0/400975197 shutdown_connections 2024-04-17T06:07:09.208 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.203+0000 7f9ad1bbe740 1 --2- 172.21.15.47:0/400975197 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x561010803b10 0x561010805f40 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:09.208 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.203+0000 7f9ad1bbe740 1 --2- 172.21.15.47:0/400975197 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5610107f37b0 0x5610107f11f0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:09.208 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.203+0000 7f9ad1bbe740 1 --2- 172.21.15.47:0/400975197 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610107ea9e0 0x5610107f0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:09.208 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.203+0000 7f9ad1bbe740 1 --2- 172.21.15.47:0/400975197 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5610107ea0c0 0x5610107f0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:09.208 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.203+0000 7f9aa7fff640 0 client.15891 ms_handle_reset on v2:172.21.15.98:6838/2841287917 2024-04-17T06:07:09.208 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.203+0000 7f9ad1bbe740 1 -- 172.21.15.47:0/400975197 >> 172.21.15.47:0/400975197 conn(0x5610107e49e0 msgr2=0x5610107e6730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:07:09.208 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.203+0000 7f9ad1bbe740 1 -- 172.21.15.47:0/400975197 shutdown_connections 2024-04-17T06:07:09.208 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.203+0000 7f9ad1bbe740 1 -- 172.21.15.47:0/400975197 wait complete. 2024-04-17T06:07:09.208 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.203+0000 7f9ad1bbe740 20 client.15891 trim_cache size 0 max 16384 2024-04-17T06:07:09.218 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.218+0000 7f9ad1bbe740 1 Processor -- start 2024-04-17T06:07:09.218 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.218+0000 7f9ad1bbe740 1 -- start start 2024-04-17T06:07:09.218 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.218+0000 7f9ad1bbe740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610107ea0c0 0x5610107f2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:07:09.218 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.218+0000 7f9ad1bbe740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5610107ea9e0 0x5610107ff2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:07:09.218 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.218+0000 7f9ad1bbe740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5610107f37b0 0x56101080b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:07:09.218 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.218+0000 7f9ad1bbe740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f9ac81b0470 con 0x5610107ea9e0 2024-04-17T06:07:09.218 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.218+0000 7f9ad1bbe740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9ac81add30 con 0x5610107ea0c0 2024-04-17T06:07:09.218 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.218+0000 7f9ad1bbe740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9ac81abb80 con 0x5610107f37b0 2024-04-17T06:07:09.218 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.218+0000 7f9ac7fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5610107ea9e0 0x5610107ff2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:07:09.219 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.218+0000 7f9ac7fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5610107ea9e0 0x5610107ff2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47138/0 (socket says 172.21.15.47:47138) 2024-04-17T06:07:09.219 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.218+0000 7f9ac7fff640 1 -- 172.21.15.47:0/3028362721 learned_addr learned my addr 172.21.15.47:0/3028362721 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:07:09.219 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.219+0000 7f9accd70640 1 --2- 172.21.15.47:0/3028362721 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610107ea0c0 0x5610107f2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:07:09.219 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.219+0000 7f9acd571640 1 --2- 172.21.15.47:0/3028362721 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5610107f37b0 0x56101080b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:07:09.219 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.219+0000 7f9ac7fff640 1 -- 172.21.15.47:0/3028362721 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5610107f37b0 msgr2=0x56101080b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:07:09.219 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.219+0000 7f9ac7fff640 1 --2- 172.21.15.47:0/3028362721 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5610107f37b0 0x56101080b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:09.219 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.219+0000 7f9ac7fff640 1 -- 172.21.15.47:0/3028362721 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610107ea0c0 msgr2=0x5610107f2ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:07:09.219 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.219+0000 7f9ac7fff640 1 --2- 172.21.15.47:0/3028362721 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610107ea0c0 0x5610107f2ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:09.219 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.219+0000 7f9ac7fff640 1 -- 172.21.15.47:0/3028362721 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610107e4810 con 0x5610107ea9e0 2024-04-17T06:07:09.219 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.219+0000 7f9ac7fff640 1 --2- 172.21.15.47:0/3028362721 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5610107ea9e0 0x5610107ff2a0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f9ab400bc60 tx=0x7f9ab400b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa2734d93f630d73 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:07:09.219 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.219+0000 7f9ab27fc640 1 -- 172.21.15.47:0/3028362721 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab400be10 con 0x5610107ea9e0 2024-04-17T06:07:09.219 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.219+0000 7f9ab27fc640 1 -- 172.21.15.47:0/3028362721 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ab4005040 con 0x5610107ea9e0 2024-04-17T06:07:09.219 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.219+0000 7f9ab27fc640 1 -- 172.21.15.47:0/3028362721 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab4004c50 con 0x5610107ea9e0 2024-04-17T06:07:09.219 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.220+0000 7f9ad1bbe740 1 -- 172.21.15.47:0/3028362721 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5610107ea9e0 msgr2=0x5610107ff2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:07:09.220 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.220+0000 7f9ad1bbe740 1 --2- 172.21.15.47:0/3028362721 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5610107ea9e0 0x5610107ff2a0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f9ab400bc60 tx=0x7f9ab400b9a0 comp rx=0 tx=0).stop 2024-04-17T06:07:09.220 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.220+0000 7f9ad1bbe740 1 -- 172.21.15.47:0/3028362721 shutdown_connections 2024-04-17T06:07:09.220 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.220+0000 7f9ad1bbe740 1 --2- 172.21.15.47:0/3028362721 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5610107f37b0 0x56101080b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:09.220 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.220+0000 7f9ad1bbe740 1 --2- 172.21.15.47:0/3028362721 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5610107ea9e0 0x5610107ff2a0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:09.220 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.220+0000 7f9ad1bbe740 1 --2- 172.21.15.47:0/3028362721 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610107ea0c0 0x5610107f2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:09.220 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.220+0000 7f9ad1bbe740 1 -- 172.21.15.47:0/3028362721 >> 172.21.15.47:0/3028362721 conn(0x5610107e49e0 msgr2=0x5610107eba90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:07:09.220 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.220+0000 7f9ad1bbe740 1 -- 172.21.15.47:0/3028362721 shutdown_connections 2024-04-17T06:07:09.220 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.220+0000 7f9ad1bbe740 1 -- 172.21.15.47:0/3028362721 wait complete. 2024-04-17T06:07:09.220 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.220+0000 7f9ad1bbe740 1 -- start start 2024-04-17T06:07:09.220 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.220+0000 7f9ad1bbe740 1 Processor -- start 2024-04-17T06:07:09.223 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.221+0000 7f9ad1bbe740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610107ea0c0 0x5610107f7450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:07:09.223 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.221+0000 7f9ad1bbe740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5610107ea9e0 0x5610107f7990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:07:09.223 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.221+0000 7f9ad1bbe740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5610107f37b0 0x5610107f7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:07:09.223 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.221+0000 7f9ad1bbe740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f9ac81ac1f0 con 0x5610107ea9e0 2024-04-17T06:07:09.223 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.221+0000 7f9ad1bbe740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9ac81ad660 con 0x5610107ea0c0 2024-04-17T06:07:09.223 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.221+0000 7f9ad1bbe740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9ac81ac8c0 con 0x5610107f37b0 2024-04-17T06:07:09.223 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.221+0000 7f9ad1bbe740 10 client.0 _ll_register_callbacks cb 0x7f9ac0009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:07:09.223 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.221+0000 7f9ac7fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5610107ea9e0 0x5610107f7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:07:09.223 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.221+0000 7f9ac7fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5610107ea9e0 0x5610107f7990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47144/0 (socket says 172.21.15.47:47144) 2024-04-17T06:07:09.223 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.221+0000 7f9ac7fff640 1 -- 172.21.15.47:0/3081526000 learned_addr learned my addr 172.21.15.47:0/3081526000 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:07:09.224 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.221+0000 7f9accd70640 1 --2- 172.21.15.47:0/3081526000 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610107ea0c0 0x5610107f7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:07:09.224 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.221+0000 7f9ac7fff640 1 -- 172.21.15.47:0/3081526000 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5610107f37b0 msgr2=0x5610107f7ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:07:09.224 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.221+0000 7f9ac7fff640 1 --2- 172.21.15.47:0/3081526000 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5610107f37b0 0x5610107f7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:09.224 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.221+0000 7f9ac7fff640 1 -- 172.21.15.47:0/3081526000 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610107ea0c0 msgr2=0x5610107f7450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:07:09.224 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.221+0000 7f9ac7fff640 1 --2- 172.21.15.47:0/3081526000 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610107ea0c0 0x5610107f7450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:09.224 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.221+0000 7f9ac7fff640 1 -- 172.21.15.47:0/3081526000 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561010869360 con 0x5610107ea9e0 2024-04-17T06:07:09.224 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.221+0000 7f9ac7fff640 1 --2- 172.21.15.47:0/3081526000 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5610107ea9e0 0x5610107f7990 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f9ab4004870 tx=0x7f9ab4005420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0152ede9d1e1936 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:07:09.224 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.221+0000 7f9aa7fff640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-17T06:07:09.224 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.221+0000 7f9aa7fff640 1 -- 172.21.15.47:0/3081526000 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab4005610 con 0x5610107ea9e0 2024-04-17T06:07:09.224 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.221+0000 7f9aa7fff640 1 -- 172.21.15.47:0/3081526000 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f9ab4023040 con 0x5610107ea9e0 2024-04-17T06:07:09.224 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.222+0000 7f9ad1bbe740 1 -- 172.21.15.47:0/3081526000 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ab4005040 con 0x5610107ea9e0 2024-04-17T06:07:09.224 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.222+0000 7f9aa7fff640 1 -- 172.21.15.47:0/3081526000 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9ab400be10 con 0x5610107ea9e0 2024-04-17T06:07:09.224 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.222+0000 7f9ad1bbe740 10 client.15897 fetch_fsmap learned FSMap version 30 2024-04-17T06:07:09.224 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.222+0000 7f9ad1bbe740 1 -- 172.21.15.47:0/3081526000 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56101086a650 con 0x5610107ea9e0 2024-04-17T06:07:09.225 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.222+0000 7f9aa7fff640 1 -- 172.21.15.47:0/3081526000 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ab4005040 con 0x5610107ea9e0 2024-04-17T06:07:09.225 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.222+0000 7f9aa7fff640 1 -- 172.21.15.47:0/3081526000 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 30) ==== 8339+0+0 (secure 0 0 0) 0x7f9ab4003820 con 0x5610107ea9e0 2024-04-17T06:07:09.225 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.222+0000 7f9ad1bbe740 10 client.15897 fetch_fsmap finished waiting for FSMap version 30 2024-04-17T06:07:09.225 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.222+0000 7f9ad1bbe740 10 client.15897 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-17T06:07:09.225 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.222+0000 7f9ad1bbe740 20 client.15897 populate_metadata read hostname 'smithi047' 2024-04-17T06:07:09.225 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.222+0000 7f9ad1bbe740 1 --2- 172.21.15.47:0/3081526000 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x56101086aa70 0x56101086ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:07:09.225 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.222+0000 7f9ad1bbe740 4 client.15897 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-17T06:07:09.225 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.222+0000 7f9ad1bbe740 1 -- 172.21.15.47:0/3081526000 --> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561010757060 con 0x56101086aa70 2024-04-17T06:07:09.225 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.222+0000 7f9accd70640 1 --2- 172.21.15.47:0/3081526000 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x56101086aa70 0x56101086ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:07:09.225 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.223+0000 7f9accd70640 1 --2- 172.21.15.47:0/3081526000 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x56101086aa70 0x56101086ce60 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd6cfa3ec594141 server_cookie=2647ef53e0439333 in_seq=0 out_seq=0 2024-04-17T06:07:09.225 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.223+0000 7f9aa7fff640 10 client.15897 ms_handle_connect on v2:172.21.15.98:6838/2841287917 2024-04-17T06:07:09.229 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.231+0000 7f9aa7fff640 1 -- 172.21.15.47:0/3081526000 <== mds.0 v2:172.21.15.98:6838/2841287917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561010757060 con 0x56101086aa70 2024-04-17T06:07:09.229 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.231+0000 7f9aa7fff640 10 client.15897 handle_command_reply: tid=0 2024-04-17T06:07:09.229 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.231+0000 7f9ad1bbe740 1 client.15897 shutdown 2024-04-17T06:07:09.229 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.231+0000 7f9ad1bbe740 1 -- 172.21.15.47:0/3081526000 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5610107ea9e0 msgr2=0x5610107f7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:07:09.229 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.231+0000 7f9ad1bbe740 1 --2- 172.21.15.47:0/3081526000 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5610107ea9e0 0x5610107f7990 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f9ab4004870 tx=0x7f9ab4005420 comp rx=0 tx=0).stop 2024-04-17T06:07:09.229 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.231+0000 7f9ad1bbe740 1 -- 172.21.15.47:0/3081526000 shutdown_connections 2024-04-17T06:07:09.230 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.231+0000 7f9ad1bbe740 1 --2- 172.21.15.47:0/3081526000 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x56101086aa70 0x56101086ce60 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:09.230 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.231+0000 7f9ad1bbe740 1 --2- 172.21.15.47:0/3081526000 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5610107f37b0 0x5610107f7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:09.230 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.231+0000 7f9ad1bbe740 1 --2- 172.21.15.47:0/3081526000 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5610107ea9e0 0x5610107f7990 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:09.230 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.231+0000 7f9ad1bbe740 1 --2- 172.21.15.47:0/3081526000 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610107ea0c0 0x5610107f7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:09.230 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.231+0000 7f9aa7fff640 0 client.15897 ms_handle_reset on v2:172.21.15.98:6838/2841287917 2024-04-17T06:07:09.230 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.231+0000 7f9ad1bbe740 1 -- 172.21.15.47:0/3081526000 >> 172.21.15.47:0/3081526000 conn(0x5610107e49e0 msgr2=0x5610107e3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:07:09.230 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.231+0000 7f9ad1bbe740 1 -- 172.21.15.47:0/3081526000 shutdown_connections 2024-04-17T06:07:09.230 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.231+0000 7f9ad1bbe740 1 -- 172.21.15.47:0/3081526000 wait complete. 2024-04-17T06:07:09.230 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.231+0000 7f9ad1bbe740 20 client.15897 trim_cache size 0 max 16384 2024-04-17T06:07:09.230 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-04-17T06:07:09.231 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-04-17T06:07:09.231 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-04-17T06:07:09.231 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-04-17T06:07:09.231 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.232+0000 7f9acefd4640 1 -- 172.21.15.47:0/1220511775 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f9ab80a3dc0 msgr2=0x7f9ab80a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:07:09.232 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.232+0000 7f9acefd4640 1 --2- 172.21.15.47:0/1220511775 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f9ab80a3dc0 0x7f9ab80a6280 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9ac0009fd0 tx=0x7f9ac0005bd0 comp rx=0 tx=0).stop 2024-04-17T06:07:09.232 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.232+0000 7f9acefd4640 1 -- 172.21.15.47:0/1220511775 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ac8101100 msgr2=0x7f9ac819e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:07:09.232 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.232+0000 7f9acefd4640 1 --2- 172.21.15.47:0/1220511775 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ac8101100 0x7f9ac819e460 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f9abc006010 tx=0x7f9abc02cc70 comp rx=0 tx=0).stop 2024-04-17T06:07:09.232 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.232+0000 7f9acefd4640 1 -- 172.21.15.47:0/1220511775 shutdown_connections 2024-04-17T06:07:09.232 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.232+0000 7f9acefd4640 1 --2- 172.21.15.47:0/1220511775 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9ac8198610 0x7f9ac8198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:09.232 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.232+0000 7f9acefd4640 1 --2- 172.21.15.47:0/1220511775 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ac8106180 0x7f9ac819e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:09.232 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.232+0000 7f9acefd4640 1 --2- 172.21.15.47:0/1220511775 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f9ab80a3dc0 0x7f9ab80a6280 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:09.232 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.232+0000 7f9acefd4640 1 --2- 172.21.15.47:0/1220511775 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ac8101100 0x7f9ac819e460 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:09.232 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.232+0000 7f9acefd4640 1 -- 172.21.15.47:0/1220511775 >> 172.21.15.47:0/1220511775 conn(0x7f9ac8077fe0 msgr2=0x7f9ac80699a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:07:09.232 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.232+0000 7f9acefd4640 1 -- 172.21.15.47:0/1220511775 shutdown_connections 2024-04-17T06:07:09.232 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:09.232+0000 7f9acefd4640 1 -- 172.21.15.47:0/1220511775 wait complete. 2024-04-17T06:07:10.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:07:09 smithi103 ceph-mon[36243]: pgmap v180: 97 pgs: 97 active+clean; 3.0 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 34 KiB/s wr, 26 op/s 2024-04-17T06:07:10.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:07:09 smithi098 ceph-mon[36516]: pgmap v180: 97 pgs: 97 active+clean; 3.0 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 34 KiB/s wr, 26 op/s 2024-04-17T06:07:10.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:07:09 smithi047 ceph-mon[32223]: pgmap v180: 97 pgs: 97 active+clean; 3.0 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 34 KiB/s wr, 26 op/s 2024-04-17T06:07:12.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:07:11 smithi103 ceph-mon[36243]: pgmap v181: 97 pgs: 97 active+clean; 3.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 41 KiB/s wr, 33 op/s 2024-04-17T06:07:12.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:07:11 smithi098 ceph-mon[36516]: pgmap v181: 97 pgs: 97 active+clean; 3.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 41 KiB/s wr, 33 op/s 2024-04-17T06:07:12.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:07:11 smithi047 ceph-mon[32223]: pgmap v181: 97 pgs: 97 active+clean; 3.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 41 KiB/s wr, 33 op/s 2024-04-17T06:07:14.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:07:13 smithi103 ceph-mon[36243]: pgmap v182: 97 pgs: 97 active+clean; 3.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 34 KiB/s wr, 26 op/s 2024-04-17T06:07:14.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:07:13 smithi098 ceph-mon[36516]: pgmap v182: 97 pgs: 97 active+clean; 3.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 34 KiB/s wr, 26 op/s 2024-04-17T06:07:14.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:07:13 smithi047 ceph-mon[32223]: pgmap v182: 97 pgs: 97 active+clean; 3.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 34 KiB/s wr, 26 op/s 2024-04-17T06:07:16.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:07:15 smithi103 ceph-mon[36243]: pgmap v183: 97 pgs: 97 active+clean; 3.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 39 KiB/s wr, 31 op/s 2024-04-17T06:07:16.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:07:15 smithi098 ceph-mon[36516]: pgmap v183: 97 pgs: 97 active+clean; 3.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 39 KiB/s wr, 31 op/s 2024-04-17T06:07:16.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:07:15 smithi047 ceph-mon[32223]: pgmap v183: 97 pgs: 97 active+clean; 3.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 39 KiB/s wr, 31 op/s 2024-04-17T06:07:18.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:07:17 smithi098 ceph-mon[36516]: pgmap v184: 97 pgs: 97 active+clean; 3.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-04-17T06:07:18.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:07:17 smithi047 ceph-mon[32223]: pgmap v184: 97 pgs: 97 active+clean; 3.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-04-17T06:07:18.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:07:17 smithi103 ceph-mon[36243]: pgmap v184: 97 pgs: 97 active+clean; 3.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-04-17T06:07:20.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:07:19 smithi098 ceph-mon[36516]: pgmap v185: 97 pgs: 97 active+clean; 3.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-04-17T06:07:20.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:07:19 smithi047 ceph-mon[32223]: pgmap v185: 97 pgs: 97 active+clean; 3.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-04-17T06:07:20.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:07:19 smithi103 ceph-mon[36243]: pgmap v185: 97 pgs: 97 active+clean; 3.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-04-17T06:07:22.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:07:21 smithi098 ceph-mon[36516]: pgmap v186: 97 pgs: 97 active+clean; 3.1 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-04-17T06:07:22.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:07:21 smithi047 ceph-mon[32223]: pgmap v186: 97 pgs: 97 active+clean; 3.1 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-04-17T06:07:22.285 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:07:21 smithi103 ceph-mon[36243]: pgmap v186: 97 pgs: 97 active+clean; 3.1 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-04-17T06:07:22.332 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.331+0000 7fead9ed3640 1 -- 172.21.15.47:0/2732412848 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fead406aa40 msgr2=0x7fead406ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:07:22.332 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.331+0000 7fead9ed3640 1 --2- 172.21.15.47:0/2732412848 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fead406aa40 0x7fead406ae20 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7feac4009ca0 tx=0x7feac402c0f0 comp rx=0 tx=0).stop 2024-04-17T06:07:22.332 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.332+0000 7fead9ed3640 1 -- 172.21.15.47:0/2732412848 shutdown_connections 2024-04-17T06:07:22.332 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.332+0000 7fead9ed3640 1 --2- 172.21.15.47:0/2732412848 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fead41072f0 0x7fead410ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:22.332 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.332+0000 7fead9ed3640 1 --2- 172.21.15.47:0/2732412848 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fead406b3f0 0x7fead4106bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:22.332 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.332+0000 7fead9ed3640 1 --2- 172.21.15.47:0/2732412848 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fead406aa40 0x7fead406ae20 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:22.333 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.332+0000 7fead9ed3640 1 -- 172.21.15.47:0/2732412848 >> 172.21.15.47:0/2732412848 conn(0x7fead4077bb0 msgr2=0x7fead4077fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:07:22.333 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.332+0000 7fead9ed3640 1 -- 172.21.15.47:0/2732412848 shutdown_connections 2024-04-17T06:07:22.333 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.332+0000 7fead9ed3640 1 -- 172.21.15.47:0/2732412848 wait complete. 2024-04-17T06:07:22.333 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.332+0000 7fead9ed3640 1 Processor -- start 2024-04-17T06:07:22.333 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.332+0000 7fead9ed3640 1 -- start start 2024-04-17T06:07:22.333 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.333+0000 7fead9ed3640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fead406aa40 0x7fead419e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:07:22.333 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.333+0000 7fead9ed3640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fead41072f0 0x7fead419e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:07:22.333 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.333+0000 7fead9ed3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fead4198660 0x7fead4198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:07:22.333 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.333+0000 7fead9ed3640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fead4112380 con 0x7fead41072f0 2024-04-17T06:07:22.333 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.333+0000 7fead9ed3640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fead4112200 con 0x7fead406aa40 2024-04-17T06:07:22.334 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.333+0000 7fead9ed3640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fead4112500 con 0x7fead4198660 2024-04-17T06:07:22.334 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.333+0000 7fead2ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fead41072f0 0x7fead419e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:07:22.334 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.333+0000 7fead2ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fead41072f0 0x7fead419e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44950/0 (socket says 172.21.15.47:44950) 2024-04-17T06:07:22.334 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.333+0000 7fead2ffd640 1 -- 172.21.15.47:0/2300448496 learned_addr learned my addr 172.21.15.47:0/2300448496 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:07:22.334 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.333+0000 7fead37fe640 1 --2- 172.21.15.47:0/2300448496 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fead406aa40 0x7fead419e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:07:22.334 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.333+0000 7fead3fff640 1 --2- 172.21.15.47:0/2300448496 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fead4198660 0x7fead4198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:07:22.334 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.333+0000 7fead2ffd640 1 -- 172.21.15.47:0/2300448496 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fead4198660 msgr2=0x7fead4198b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:07:22.334 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.333+0000 7fead2ffd640 1 --2- 172.21.15.47:0/2300448496 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fead4198660 0x7fead4198b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:22.334 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.333+0000 7fead2ffd640 1 -- 172.21.15.47:0/2300448496 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fead406aa40 msgr2=0x7fead419e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:07:22.334 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.333+0000 7fead2ffd640 1 --2- 172.21.15.47:0/2300448496 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fead406aa40 0x7fead419e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:22.334 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.333+0000 7fead2ffd640 1 -- 172.21.15.47:0/2300448496 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fead4199180 con 0x7fead41072f0 2024-04-17T06:07:22.334 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.333+0000 7fead3fff640 1 --2- 172.21.15.47:0/2300448496 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fead4198660 0x7fead4198b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:07:22.334 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.333+0000 7fead2ffd640 1 --2- 172.21.15.47:0/2300448496 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fead41072f0 0x7fead419e9a0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7feac800d740 tx=0x7feac800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:07:22.335 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.334+0000 7fead0ff9640 1 -- 172.21.15.47:0/2300448496 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feac8017070 con 0x7fead41072f0 2024-04-17T06:07:22.335 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.334+0000 7fead0ff9640 1 -- 172.21.15.47:0/2300448496 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feac800de50 con 0x7fead41072f0 2024-04-17T06:07:22.335 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.334+0000 7fead9ed3640 1 -- 172.21.15.47:0/2300448496 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fead4199470 con 0x7fead41072f0 2024-04-17T06:07:22.335 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.334+0000 7fead0ff9640 1 -- 172.21.15.47:0/2300448496 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feac800f6b0 con 0x7fead41072f0 2024-04-17T06:07:22.335 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.334+0000 7fead9ed3640 1 -- 172.21.15.47:0/2300448496 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fead41a5670 con 0x7fead41072f0 2024-04-17T06:07:22.341 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.336+0000 7fead0ff9640 1 -- 172.21.15.47:0/2300448496 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7feac801a070 con 0x7fead41072f0 2024-04-17T06:07:22.341 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.336+0000 7feadcb97740 1 Processor -- start 2024-04-17T06:07:22.341 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.336+0000 7feadcb97740 1 -- start start 2024-04-17T06:07:22.341 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.337+0000 7feadcb97740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb2fbe50c0 0x55eb2fbe54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:07:22.341 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.337+0000 7feadcb97740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eb2fbe59e0 0x55eb2fbee270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:07:22.342 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.337+0000 7feadcb97740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb2fbee7b0 0x55eb2fbf0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:07:22.342 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.337+0000 7feadcb97740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55eb2fb4a7d0 con 0x55eb2fbe59e0 2024-04-17T06:07:22.342 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.337+0000 7feadcb97740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55eb2fae7f70 con 0x55eb2fbe50c0 2024-04-17T06:07:22.342 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.337+0000 7feadcb97740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55eb2fb46790 con 0x55eb2fbee7b0 2024-04-17T06:07:22.342 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.337+0000 7fead0ff9640 1 --2- 172.21.15.47:0/2300448496 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7feabc0a3dc0 0x7feabc0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:07:22.342 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.337+0000 7fead37fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eb2fbe59e0 0x55eb2fbee270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:07:22.342 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.337+0000 7fead37fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eb2fbe59e0 0x55eb2fbee270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44962/0 (socket says 172.21.15.47:44962) 2024-04-17T06:07:22.342 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.337+0000 7fead37fe640 1 -- 172.21.15.47:0/487200687 learned_addr learned my addr 172.21.15.47:0/487200687 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:07:22.342 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.337+0000 7fead3fff640 1 --2- 172.21.15.47:0/487200687 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb2fbe50c0 0x55eb2fbe54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:07:22.342 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.337+0000 7fead3fff640 1 --2- 172.21.15.47:0/2300448496 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7feabc0a3dc0 0x7feabc0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:07:22.342 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.337+0000 7fead0ff9640 1 -- 172.21.15.47:0/2300448496 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7feac80d0210 con 0x7fead41072f0 2024-04-17T06:07:22.342 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.337+0000 7fead2ffd640 1 --2- 172.21.15.47:0/487200687 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb2fbee7b0 0x55eb2fbf0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:07:22.342 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.337+0000 7fead37fe640 1 -- 172.21.15.47:0/487200687 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb2fbee7b0 msgr2=0x55eb2fbf0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:07:22.342 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.337+0000 7fead37fe640 1 --2- 172.21.15.47:0/487200687 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb2fbee7b0 0x55eb2fbf0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:22.343 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.337+0000 7fead37fe640 1 -- 172.21.15.47:0/487200687 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb2fbe50c0 msgr2=0x55eb2fbe54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:07:22.343 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.337+0000 7fead37fe640 1 --2- 172.21.15.47:0/487200687 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb2fbe50c0 0x55eb2fbe54a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:22.343 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.337+0000 7fead37fe640 1 -- 172.21.15.47:0/487200687 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb2f85d210 con 0x55eb2fbe59e0 2024-04-17T06:07:22.343 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.337+0000 7fead3fff640 1 --2- 172.21.15.47:0/487200687 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb2fbe50c0 0x55eb2fbe54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:07:22.343 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.337+0000 7fead37fe640 1 --2- 172.21.15.47:0/487200687 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eb2fbe59e0 0x55eb2fbee270 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7feac402c700 tx=0x7feac40099f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd7ef92fe03afb27 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:07:22.343 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.337+0000 7fead3fff640 1 --2- 172.21.15.47:0/2300448496 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7feabc0a3dc0 0x7feabc0a6280 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7feacc0058c0 tx=0x7feacc004730 comp rx=0 tx=0).ready entity=mgr.24923 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:07:22.343 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.338+0000 7feab9ffb640 1 -- 172.21.15.47:0/487200687 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feac4043070 con 0x55eb2fbe59e0 2024-04-17T06:07:22.343 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.338+0000 7feab9ffb640 1 -- 172.21.15.47:0/487200687 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feac4032c90 con 0x55eb2fbe59e0 2024-04-17T06:07:22.343 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.338+0000 7feab9ffb640 1 -- 172.21.15.47:0/487200687 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feac4039040 con 0x55eb2fbe59e0 2024-04-17T06:07:22.343 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.338+0000 7feadcb97740 1 -- 172.21.15.47:0/487200687 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eb2fbe59e0 msgr2=0x55eb2fbee270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:07:22.343 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.338+0000 7feadcb97740 1 --2- 172.21.15.47:0/487200687 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eb2fbe59e0 0x55eb2fbee270 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7feac402c700 tx=0x7feac40099f0 comp rx=0 tx=0).stop 2024-04-17T06:07:22.343 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.338+0000 7feadcb97740 1 -- 172.21.15.47:0/487200687 shutdown_connections 2024-04-17T06:07:22.343 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.338+0000 7feadcb97740 1 --2- 172.21.15.47:0/487200687 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb2fbee7b0 0x55eb2fbf0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:22.343 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.338+0000 7feadcb97740 1 --2- 172.21.15.47:0/487200687 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eb2fbe59e0 0x55eb2fbee270 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:22.344 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.338+0000 7feadcb97740 1 --2- 172.21.15.47:0/487200687 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb2fbe50c0 0x55eb2fbe54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:22.344 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.338+0000 7feadcb97740 1 -- 172.21.15.47:0/487200687 >> 172.21.15.47:0/487200687 conn(0x55eb2fbdf9e0 msgr2=0x55eb2fbdfdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:07:22.344 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.338+0000 7feadcb97740 1 -- 172.21.15.47:0/487200687 shutdown_connections 2024-04-17T06:07:22.344 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.338+0000 7feadcb97740 1 -- 172.21.15.47:0/487200687 wait complete. 2024-04-17T06:07:22.344 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.339+0000 7feadcb97740 1 -- start start 2024-04-17T06:07:22.344 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.339+0000 7feadcb97740 1 Processor -- start 2024-04-17T06:07:22.344 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.339+0000 7feadcb97740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb2fbe50c0 0x55eb2fbeb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:07:22.344 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.339+0000 7feadcb97740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb2fbe59e0 0x55eb2fbebd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:07:22.344 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.339+0000 7feadcb97740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eb2fbee7b0 0x55eb2fbec240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:07:22.344 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.339+0000 7feadcb97740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55eb2fbf1380 con 0x55eb2fbee7b0 2024-04-17T06:07:22.344 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.339+0000 7feadcb97740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55eb2fad2e90 con 0x55eb2fbe59e0 2024-04-17T06:07:22.344 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.339+0000 7feadcb97740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55eb2fbf1500 con 0x55eb2fbe50c0 2024-04-17T06:07:22.344 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.339+0000 7feadcb97740 10 client.0 _ll_register_callbacks cb 0x55eb2fae8d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:07:22.344 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.339+0000 7fead3fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eb2fbee7b0 0x55eb2fbec240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:07:22.345 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.339+0000 7fead3fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eb2fbee7b0 0x55eb2fbec240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44970/0 (socket says 172.21.15.47:44970) 2024-04-17T06:07:22.345 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.339+0000 7fead3fff640 1 -- 172.21.15.47:0/1209174801 learned_addr learned my addr 172.21.15.47:0/1209174801 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:07:22.345 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.340+0000 7fead37fe640 1 --2- 172.21.15.47:0/1209174801 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb2fbe50c0 0x55eb2fbeb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:07:22.345 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.340+0000 7fead2ffd640 1 --2- 172.21.15.47:0/1209174801 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb2fbe59e0 0x55eb2fbebd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:07:22.345 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.340+0000 7fead3fff640 1 -- 172.21.15.47:0/1209174801 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb2fbe50c0 msgr2=0x55eb2fbeb7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:07:22.345 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.340+0000 7fead3fff640 1 --2- 172.21.15.47:0/1209174801 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb2fbe50c0 0x55eb2fbeb7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:22.345 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.340+0000 7fead3fff640 1 -- 172.21.15.47:0/1209174801 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb2fbe59e0 msgr2=0x55eb2fbebd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:07:22.345 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.340+0000 7fead3fff640 1 --2- 172.21.15.47:0/1209174801 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb2fbe59e0 0x55eb2fbebd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:22.345 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.340+0000 7fead3fff640 1 -- 172.21.15.47:0/1209174801 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb2f929620 con 0x55eb2fbee7b0 2024-04-17T06:07:22.345 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.340+0000 7fead3fff640 1 --2- 172.21.15.47:0/1209174801 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eb2fbee7b0 0x55eb2fbec240 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7feacc011570 tx=0x7feacc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6596b029f9b61a97 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:07:22.345 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.340+0000 7feab2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-17T06:07:22.345 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.340+0000 7feab2ffd640 1 -- 172.21.15.47:0/1209174801 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feacc005220 con 0x55eb2fbee7b0 2024-04-17T06:07:22.345 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.340+0000 7feab2ffd640 1 -- 172.21.15.47:0/1209174801 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7feacc0053c0 con 0x55eb2fbee7b0 2024-04-17T06:07:22.345 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.340+0000 7feadcb97740 1 -- 172.21.15.47:0/1209174801 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feac4032c90 con 0x55eb2fbee7b0 2024-04-17T06:07:22.345 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.340+0000 7feab2ffd640 1 -- 172.21.15.47:0/1209174801 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7feacc005600 con 0x55eb2fbee7b0 2024-04-17T06:07:22.346 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.340+0000 7feadcb97740 10 client.15909 fetch_fsmap learned FSMap version 30 2024-04-17T06:07:22.346 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.340+0000 7feadcb97740 1 -- 172.21.15.47:0/1209174801 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55eb2fbfd5b0 con 0x55eb2fbee7b0 2024-04-17T06:07:22.346 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.341+0000 7feab2ffd640 1 -- 172.21.15.47:0/1209174801 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feac4032c90 con 0x55eb2fbee7b0 2024-04-17T06:07:22.346 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.341+0000 7feab2ffd640 1 -- 172.21.15.47:0/1209174801 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 30) ==== 8339+0+0 (secure 0 0 0) 0x7feacc01c040 con 0x55eb2fbee7b0 2024-04-17T06:07:22.346 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.341+0000 7feadcb97740 10 client.15909 fetch_fsmap finished waiting for FSMap version 30 2024-04-17T06:07:22.346 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.341+0000 7feadcb97740 10 client.15909 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-17T06:07:22.346 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.341+0000 7feadcb97740 20 client.15909 populate_metadata read hostname 'smithi047' 2024-04-17T06:07:22.346 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.341+0000 7feadcb97740 1 --2- 172.21.15.47:0/1209174801 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55eb2fbfea70 0x55eb2fc00ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:07:22.346 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.341+0000 7feadcb97740 4 client.15909 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T06:07:22.346 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.341+0000 7feadcb97740 1 -- 172.21.15.47:0/1209174801 --> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eb2fb581f0 con 0x55eb2fbfea70 2024-04-17T06:07:22.346 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.341+0000 7fead37fe640 1 --2- 172.21.15.47:0/1209174801 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55eb2fbfea70 0x55eb2fc00ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:07:22.346 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.342+0000 7fead37fe640 1 --2- 172.21.15.47:0/1209174801 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55eb2fbfea70 0x55eb2fc00ea0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bee2ae372caf9389 server_cookie=722e85f5d001587a in_seq=0 out_seq=0 2024-04-17T06:07:22.346 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.342+0000 7feab2ffd640 10 client.15909 ms_handle_connect on v2:172.21.15.98:6838/2841287917 2024-04-17T06:07:22.347 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.342+0000 7feab2ffd640 1 -- 172.21.15.47:0/1209174801 <== mds.0 v2:172.21.15.98:6838/2841287917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55eb2fb581f0 con 0x55eb2fbfea70 2024-04-17T06:07:22.347 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.342+0000 7feab2ffd640 10 client.15909 handle_command_reply: tid=0 2024-04-17T06:07:22.347 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.342+0000 7feadcb97740 1 client.15909 shutdown 2024-04-17T06:07:22.347 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.343+0000 7feadcb97740 1 -- 172.21.15.47:0/1209174801 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eb2fbee7b0 msgr2=0x55eb2fbec240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:07:22.347 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.343+0000 7feadcb97740 1 --2- 172.21.15.47:0/1209174801 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eb2fbee7b0 0x55eb2fbec240 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7feacc011570 tx=0x7feacc0046c0 comp rx=0 tx=0).stop 2024-04-17T06:07:22.347 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.343+0000 7feadcb97740 1 -- 172.21.15.47:0/1209174801 shutdown_connections 2024-04-17T06:07:22.347 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.343+0000 7feadcb97740 1 --2- 172.21.15.47:0/1209174801 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55eb2fbfea70 0x55eb2fc00ea0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:22.347 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.343+0000 7feadcb97740 1 --2- 172.21.15.47:0/1209174801 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eb2fbee7b0 0x55eb2fbec240 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:22.347 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.343+0000 7feadcb97740 1 --2- 172.21.15.47:0/1209174801 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb2fbe59e0 0x55eb2fbebd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:22.347 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.343+0000 7feadcb97740 1 --2- 172.21.15.47:0/1209174801 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb2fbe50c0 0x55eb2fbeb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:22.347 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.343+0000 7feab2ffd640 0 client.15909 ms_handle_reset on v2:172.21.15.98:6838/2841287917 2024-04-17T06:07:22.347 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.343+0000 7feadcb97740 1 -- 172.21.15.47:0/1209174801 >> 172.21.15.47:0/1209174801 conn(0x55eb2fbdf9e0 msgr2=0x55eb2fbe1730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:07:22.347 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.343+0000 7feadcb97740 1 -- 172.21.15.47:0/1209174801 shutdown_connections 2024-04-17T06:07:22.347 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.343+0000 7feadcb97740 1 -- 172.21.15.47:0/1209174801 wait complete. 2024-04-17T06:07:22.348 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.343+0000 7feadcb97740 20 client.15909 trim_cache size 0 max 16384 2024-04-17T06:07:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.358+0000 7feadcb97740 1 Processor -- start 2024-04-17T06:07:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.358+0000 7feadcb97740 1 -- start start 2024-04-17T06:07:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.358+0000 7feadcb97740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eb2fbe50c0 0x55eb2fbfa240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:07:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.358+0000 7feadcb97740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb2fbe59e0 0x55eb2fbed5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:07:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.358+0000 7feadcb97740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb2fbee7b0 0x55eb2fc067b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:07:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.358+0000 7feadcb97740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fead41b04a0 con 0x55eb2fbe50c0 2024-04-17T06:07:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.358+0000 7feadcb97740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fead41add60 con 0x55eb2fbee7b0 2024-04-17T06:07:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.358+0000 7feadcb97740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fead41abbb0 con 0x55eb2fbe59e0 2024-04-17T06:07:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.358+0000 7fead37fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eb2fbe50c0 0x55eb2fbfa240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:07:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.359+0000 7fead37fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eb2fbe50c0 0x55eb2fbfa240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44972/0 (socket says 172.21.15.47:44972) 2024-04-17T06:07:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.359+0000 7fead37fe640 1 -- 172.21.15.47:0/1215665279 learned_addr learned my addr 172.21.15.47:0/1215665279 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:07:22.359 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.359+0000 7fead2ffd640 1 --2- 172.21.15.47:0/1215665279 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb2fbe59e0 0x55eb2fbed5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:07:22.359 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.359+0000 7fead37fe640 1 -- 172.21.15.47:0/1215665279 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb2fbe59e0 msgr2=0x55eb2fbed5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:07:22.359 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.359+0000 7fead3fff640 1 --2- 172.21.15.47:0/1215665279 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb2fbee7b0 0x55eb2fc067b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:07:22.359 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.359+0000 7fead37fe640 1 --2- 172.21.15.47:0/1215665279 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb2fbe59e0 0x55eb2fbed5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:22.359 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.359+0000 7fead37fe640 1 -- 172.21.15.47:0/1215665279 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb2fbee7b0 msgr2=0x55eb2fc067b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:07:22.359 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.359+0000 7fead37fe640 1 --2- 172.21.15.47:0/1215665279 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb2fbee7b0 0x55eb2fc067b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:22.359 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.359+0000 7fead37fe640 1 -- 172.21.15.47:0/1215665279 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb2fbe09e0 con 0x55eb2fbe50c0 2024-04-17T06:07:22.359 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.359+0000 7fead2ffd640 1 --2- 172.21.15.47:0/1215665279 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb2fbe59e0 0x55eb2fbed5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:07:22.359 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.359+0000 7fead3fff640 1 --2- 172.21.15.47:0/1215665279 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb2fbee7b0 0x55eb2fc067b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:07:22.359 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.359+0000 7fead37fe640 1 --2- 172.21.15.47:0/1215665279 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eb2fbe50c0 0x55eb2fbfa240 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7feac402c700 tx=0x7feac402c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aa7987c977a788a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:07:22.359 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.359+0000 7feab97fa640 1 -- 172.21.15.47:0/1215665279 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feac404f070 con 0x55eb2fbe50c0 2024-04-17T06:07:22.359 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.359+0000 7feab97fa640 1 -- 172.21.15.47:0/1215665279 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feac4003970 con 0x55eb2fbe50c0 2024-04-17T06:07:22.359 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.359+0000 7feab97fa640 1 -- 172.21.15.47:0/1215665279 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feac4043040 con 0x55eb2fbe50c0 2024-04-17T06:07:22.359 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.360+0000 7feadcb97740 1 -- 172.21.15.47:0/1215665279 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eb2fbe50c0 msgr2=0x55eb2fbfa240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:07:22.360 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.360+0000 7feadcb97740 1 --2- 172.21.15.47:0/1215665279 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eb2fbe50c0 0x55eb2fbfa240 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7feac402c700 tx=0x7feac402c0f0 comp rx=0 tx=0).stop 2024-04-17T06:07:22.360 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.360+0000 7feadcb97740 1 -- 172.21.15.47:0/1215665279 shutdown_connections 2024-04-17T06:07:22.360 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.360+0000 7feadcb97740 1 --2- 172.21.15.47:0/1215665279 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb2fbee7b0 0x55eb2fc067b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:22.360 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.360+0000 7feadcb97740 1 --2- 172.21.15.47:0/1215665279 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb2fbe59e0 0x55eb2fbed5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:22.360 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.360+0000 7feadcb97740 1 --2- 172.21.15.47:0/1215665279 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eb2fbe50c0 0x55eb2fbfa240 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:22.360 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.360+0000 7feadcb97740 1 -- 172.21.15.47:0/1215665279 >> 172.21.15.47:0/1215665279 conn(0x55eb2fbdf9e0 msgr2=0x55eb2fbecee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:07:22.360 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.360+0000 7feadcb97740 1 -- 172.21.15.47:0/1215665279 shutdown_connections 2024-04-17T06:07:22.360 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.360+0000 7feadcb97740 1 -- 172.21.15.47:0/1215665279 wait complete. 2024-04-17T06:07:22.360 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.360+0000 7feadcb97740 1 -- start start 2024-04-17T06:07:22.363 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.361+0000 7feadcb97740 1 Processor -- start 2024-04-17T06:07:22.363 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.361+0000 7feadcb97740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb2fbe50c0 0x55eb2fbe87b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:07:22.363 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.361+0000 7feadcb97740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb2fbe59e0 0x55eb2fbe8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:07:22.363 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.361+0000 7feadcb97740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eb2fbee7b0 0x55eb2fbe9270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:07:22.363 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.361+0000 7feadcb97740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fead41ac220 con 0x55eb2fbee7b0 2024-04-17T06:07:22.363 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.361+0000 7feadcb97740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fead41ad690 con 0x55eb2fbe59e0 2024-04-17T06:07:22.363 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.361+0000 7feadcb97740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fead41ac8f0 con 0x55eb2fbe50c0 2024-04-17T06:07:22.363 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.361+0000 7feadcb97740 10 client.0 _ll_register_callbacks cb 0x7feacc00ea10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:07:22.363 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.361+0000 7fead3fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eb2fbee7b0 0x55eb2fbe9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:07:22.364 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.361+0000 7fead3fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eb2fbee7b0 0x55eb2fbe9270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44984/0 (socket says 172.21.15.47:44984) 2024-04-17T06:07:22.364 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.361+0000 7fead3fff640 1 -- 172.21.15.47:0/518908188 learned_addr learned my addr 172.21.15.47:0/518908188 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:07:22.364 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.361+0000 7fead37fe640 1 --2- 172.21.15.47:0/518908188 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb2fbe50c0 0x55eb2fbe87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:07:22.364 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.361+0000 7fead2ffd640 1 --2- 172.21.15.47:0/518908188 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb2fbe59e0 0x55eb2fbe8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:07:22.364 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.361+0000 7fead3fff640 1 -- 172.21.15.47:0/518908188 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb2fbe50c0 msgr2=0x55eb2fbe87b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:07:22.364 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.361+0000 7fead3fff640 1 --2- 172.21.15.47:0/518908188 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb2fbe50c0 0x55eb2fbe87b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:22.364 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.361+0000 7fead3fff640 1 -- 172.21.15.47:0/518908188 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb2fbe59e0 msgr2=0x55eb2fbe8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:07:22.364 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.361+0000 7fead3fff640 1 --2- 172.21.15.47:0/518908188 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb2fbe59e0 0x55eb2fbe8cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:22.364 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.361+0000 7fead3fff640 1 -- 172.21.15.47:0/518908188 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb2fc64310 con 0x55eb2fbee7b0 2024-04-17T06:07:22.364 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.361+0000 7fead37fe640 1 --2- 172.21.15.47:0/518908188 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb2fbe50c0 0x55eb2fbe87b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:07:22.364 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.361+0000 7fead2ffd640 1 --2- 172.21.15.47:0/518908188 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb2fbe59e0 0x55eb2fbe8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:07:22.364 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.361+0000 7fead3fff640 1 --2- 172.21.15.47:0/518908188 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eb2fbee7b0 0x55eb2fbe9270 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7feacc004d60 tx=0x7feacc00e500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4508eeff06e2e01f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:07:22.364 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.361+0000 7feab2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-17T06:07:22.364 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.362+0000 7feab2ffd640 1 -- 172.21.15.47:0/518908188 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feacc0049a0 con 0x55eb2fbee7b0 2024-04-17T06:07:22.364 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.362+0000 7feab2ffd640 1 -- 172.21.15.47:0/518908188 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7feacc004fd0 con 0x55eb2fbee7b0 2024-04-17T06:07:22.365 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.362+0000 7feadcb97740 1 -- 172.21.15.47:0/518908188 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feac4003970 con 0x55eb2fbee7b0 2024-04-17T06:07:22.365 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.362+0000 7feab2ffd640 1 -- 172.21.15.47:0/518908188 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7feacc004b40 con 0x55eb2fbee7b0 2024-04-17T06:07:22.365 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.362+0000 7feadcb97740 10 client.15915 fetch_fsmap learned FSMap version 30 2024-04-17T06:07:22.365 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.362+0000 7feadcb97740 1 -- 172.21.15.47:0/518908188 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55eb2fc65590 con 0x55eb2fbee7b0 2024-04-17T06:07:22.365 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.362+0000 7feab2ffd640 1 -- 172.21.15.47:0/518908188 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feac4003970 con 0x55eb2fbee7b0 2024-04-17T06:07:22.365 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.362+0000 7feab2ffd640 1 -- 172.21.15.47:0/518908188 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 30) ==== 8339+0+0 (secure 0 0 0) 0x7feacc01c040 con 0x55eb2fbee7b0 2024-04-17T06:07:22.365 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.362+0000 7feadcb97740 10 client.15915 fetch_fsmap finished waiting for FSMap version 30 2024-04-17T06:07:22.365 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.362+0000 7feadcb97740 10 client.15915 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-17T06:07:22.365 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.362+0000 7feadcb97740 20 client.15915 populate_metadata read hostname 'smithi047' 2024-04-17T06:07:22.365 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.362+0000 7feadcb97740 1 --2- 172.21.15.47:0/518908188 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55eb2fc65d00 0x55eb2fc68120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:07:22.365 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.362+0000 7feadcb97740 4 client.15915 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-17T06:07:22.365 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.362+0000 7feadcb97740 1 -- 172.21.15.47:0/518908188 --> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55eb2fb52060 con 0x55eb2fc65d00 2024-04-17T06:07:22.365 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.363+0000 7fead37fe640 1 --2- 172.21.15.47:0/518908188 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55eb2fc65d00 0x55eb2fc68120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:07:22.365 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.363+0000 7fead37fe640 1 --2- 172.21.15.47:0/518908188 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55eb2fc65d00 0x55eb2fc68120 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3dc408672b2e42f server_cookie=965cc2ce6e8260f8 in_seq=0 out_seq=0 2024-04-17T06:07:22.365 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.363+0000 7feab2ffd640 10 client.15915 ms_handle_connect on v2:172.21.15.98:6838/2841287917 2024-04-17T06:07:22.369 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.371+0000 7feab2ffd640 1 -- 172.21.15.47:0/518908188 <== mds.0 v2:172.21.15.98:6838/2841287917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55eb2fb52060 con 0x55eb2fc65d00 2024-04-17T06:07:22.369 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.371+0000 7feab2ffd640 10 client.15915 handle_command_reply: tid=0 2024-04-17T06:07:22.370 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.371+0000 7feadcb97740 1 client.15915 shutdown 2024-04-17T06:07:22.370 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.371+0000 7feadcb97740 1 -- 172.21.15.47:0/518908188 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eb2fbee7b0 msgr2=0x55eb2fbe9270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:07:22.370 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.371+0000 7feadcb97740 1 --2- 172.21.15.47:0/518908188 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eb2fbee7b0 0x55eb2fbe9270 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7feacc004d60 tx=0x7feacc00e500 comp rx=0 tx=0).stop 2024-04-17T06:07:22.370 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.371+0000 7feadcb97740 1 -- 172.21.15.47:0/518908188 shutdown_connections 2024-04-17T06:07:22.370 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.371+0000 7feadcb97740 1 --2- 172.21.15.47:0/518908188 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55eb2fc65d00 0x55eb2fc68120 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:22.370 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.371+0000 7feadcb97740 1 --2- 172.21.15.47:0/518908188 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eb2fbee7b0 0x55eb2fbe9270 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:22.370 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.371+0000 7feadcb97740 1 --2- 172.21.15.47:0/518908188 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb2fbe59e0 0x55eb2fbe8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:22.370 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.371+0000 7feab2ffd640 0 client.15915 ms_handle_reset on v2:172.21.15.98:6838/2841287917 2024-04-17T06:07:22.370 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.371+0000 7feadcb97740 1 --2- 172.21.15.47:0/518908188 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb2fbe50c0 0x55eb2fbe87b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:22.370 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.371+0000 7feadcb97740 1 -- 172.21.15.47:0/518908188 >> 172.21.15.47:0/518908188 conn(0x55eb2fbdf9e0 msgr2=0x55eb2fbec010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:07:22.370 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.371+0000 7feadcb97740 1 -- 172.21.15.47:0/518908188 shutdown_connections 2024-04-17T06:07:22.370 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.371+0000 7feadcb97740 1 -- 172.21.15.47:0/518908188 wait complete. 2024-04-17T06:07:22.370 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.372+0000 7feadcb97740 20 client.15915 trim_cache size 0 max 16384 2024-04-17T06:07:22.371 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-04-17T06:07:22.371 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-04-17T06:07:22.371 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-04-17T06:07:22.371 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-04-17T06:07:22.372 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.372+0000 7fead9ed3640 1 -- 172.21.15.47:0/2300448496 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7feabc0a3dc0 msgr2=0x7feabc0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:07:22.372 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.372+0000 7fead9ed3640 1 --2- 172.21.15.47:0/2300448496 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7feabc0a3dc0 0x7feabc0a6280 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7feacc0058c0 tx=0x7feacc004730 comp rx=0 tx=0).stop 2024-04-17T06:07:22.372 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.372+0000 7fead9ed3640 1 -- 172.21.15.47:0/2300448496 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fead41072f0 msgr2=0x7fead419e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:07:22.372 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.372+0000 7fead9ed3640 1 --2- 172.21.15.47:0/2300448496 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fead41072f0 0x7fead419e9a0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7feac800d740 tx=0x7feac800dc00 comp rx=0 tx=0).stop 2024-04-17T06:07:22.372 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.372+0000 7fead9ed3640 1 -- 172.21.15.47:0/2300448496 shutdown_connections 2024-04-17T06:07:22.372 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.372+0000 7fead9ed3640 1 --2- 172.21.15.47:0/2300448496 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fead4198660 0x7fead4198b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:22.372 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.372+0000 7fead9ed3640 1 --2- 172.21.15.47:0/2300448496 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7feabc0a3dc0 0x7feabc0a6280 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:22.372 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.372+0000 7fead9ed3640 1 --2- 172.21.15.47:0/2300448496 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fead41072f0 0x7fead419e9a0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:22.372 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.372+0000 7fead9ed3640 1 --2- 172.21.15.47:0/2300448496 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fead406aa40 0x7fead419e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:22.372 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.372+0000 7fead9ed3640 1 -- 172.21.15.47:0/2300448496 >> 172.21.15.47:0/2300448496 conn(0x7fead4077bb0 msgr2=0x7fead40ff320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:07:22.372 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.372+0000 7fead9ed3640 1 -- 172.21.15.47:0/2300448496 shutdown_connections 2024-04-17T06:07:22.372 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:22.372+0000 7fead9ed3640 1 -- 172.21.15.47:0/2300448496 wait complete. 2024-04-17T06:07:24.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:07:23 smithi098 ceph-mon[36516]: pgmap v187: 97 pgs: 97 active+clean; 3.1 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.5 KiB/s wr, 9 op/s 2024-04-17T06:07:24.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:07:23 smithi047 ceph-mon[32223]: pgmap v187: 97 pgs: 97 active+clean; 3.1 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.5 KiB/s wr, 9 op/s 2024-04-17T06:07:24.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:07:23 smithi103 ceph-mon[36243]: pgmap v187: 97 pgs: 97 active+clean; 3.1 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.5 KiB/s wr, 9 op/s 2024-04-17T06:07:26.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:07:25 smithi098 ceph-mon[36516]: pgmap v188: 97 pgs: 97 active+clean; 3.2 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-04-17T06:07:26.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:07:25 smithi047 ceph-mon[32223]: pgmap v188: 97 pgs: 97 active+clean; 3.2 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-04-17T06:07:26.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:07:25 smithi103 ceph-mon[36243]: pgmap v188: 97 pgs: 97 active+clean; 3.2 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-04-17T06:07:28.086 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:07:27 smithi103 ceph-mon[36243]: pgmap v189: 97 pgs: 97 active+clean; 3.2 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:07:28.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:07:27 smithi098 ceph-mon[36516]: pgmap v189: 97 pgs: 97 active+clean; 3.2 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:07:28.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:07:27 smithi047 ceph-mon[32223]: pgmap v189: 97 pgs: 97 active+clean; 3.2 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:07:29.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:07:28 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T06:07:29.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:07:28 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T06:07:29.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:07:28 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T06:07:30.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:07:29 smithi098 ceph-mon[36516]: pgmap v190: 97 pgs: 97 active+clean; 3.2 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:07:30.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:07:29 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T06:07:30.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:07:29 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T06:07:30.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:07:29 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:07:30.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:07:29 smithi047 ceph-mon[32223]: pgmap v190: 97 pgs: 97 active+clean; 3.2 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:07:30.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:07:29 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T06:07:30.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:07:29 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T06:07:30.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:07:29 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:07:30.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:07:29 smithi103 ceph-mon[36243]: pgmap v190: 97 pgs: 97 active+clean; 3.2 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:07:30.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:07:29 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T06:07:30.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:07:29 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T06:07:30.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:07:29 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:07:32.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:07:31 smithi098 ceph-mon[36516]: pgmap v191: 97 pgs: 97 active+clean; 3.2 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-04-17T06:07:32.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:07:31 smithi047 ceph-mon[32223]: pgmap v191: 97 pgs: 97 active+clean; 3.2 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-04-17T06:07:32.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:07:31 smithi103 ceph-mon[36243]: pgmap v191: 97 pgs: 97 active+clean; 3.2 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-04-17T06:07:34.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:07:33 smithi098 ceph-mon[36516]: pgmap v192: 97 pgs: 97 active+clean; 3.2 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:07:34.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:07:33 smithi047 ceph-mon[32223]: pgmap v192: 97 pgs: 97 active+clean; 3.2 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:07:34.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:07:33 smithi103 ceph-mon[36243]: pgmap v192: 97 pgs: 97 active+clean; 3.2 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:07:35.472 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.472+0000 7fbeb45e9640 1 -- 172.21.15.47:0/680672839 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbeac106180 msgr2=0x7fbeac10dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:07:35.472 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.472+0000 7fbeb45e9640 1 --2- 172.21.15.47:0/680672839 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbeac106180 0x7fbeac10dd50 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fbea4009ca0 tx=0x7fbea402c0f0 comp rx=0 tx=0).stop 2024-04-17T06:07:35.472 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.472+0000 7fbeb45e9640 1 -- 172.21.15.47:0/680672839 shutdown_connections 2024-04-17T06:07:35.472 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.472+0000 7fbeb45e9640 1 --2- 172.21.15.47:0/680672839 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbeac106180 0x7fbeac10dd50 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:35.472 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.472+0000 7fbeb45e9640 1 --2- 172.21.15.47:0/680672839 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbeac101ab0 0x7fbeac105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:35.473 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.472+0000 7fbeb45e9640 1 --2- 172.21.15.47:0/680672839 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbeac101100 0x7fbeac1014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:35.473 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.472+0000 7fbeb45e9640 1 -- 172.21.15.47:0/680672839 >> 172.21.15.47:0/680672839 conn(0x7fbeac077fe0 msgr2=0x7fbeac0783f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:07:35.473 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.472+0000 7fbeb45e9640 1 -- 172.21.15.47:0/680672839 shutdown_connections 2024-04-17T06:07:35.473 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.472+0000 7fbeb45e9640 1 -- 172.21.15.47:0/680672839 wait complete. 2024-04-17T06:07:35.473 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.473+0000 7fbeb45e9640 1 Processor -- start 2024-04-17T06:07:35.473 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.473+0000 7fbeb45e9640 1 -- start start 2024-04-17T06:07:35.473 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.473+0000 7fbeb45e9640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbeac101100 0x7fbeac19e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:07:35.473 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.473+0000 7fbeb45e9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbeac106180 0x7fbeac19eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:07:35.473 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.473+0000 7fbeb45e9640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbeac198630 0x7fbeac198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:07:35.473 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.473+0000 7fbeb45e9640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fbeac112390 con 0x7fbeac101100 2024-04-17T06:07:35.473 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.473+0000 7fbeb45e9640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fbeac112210 con 0x7fbeac198630 2024-04-17T06:07:35.473 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.473+0000 7fbeb45e9640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fbeac112510 con 0x7fbeac106180 2024-04-17T06:07:35.473 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.473+0000 7fbeb2385640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbeac101100 0x7fbeac19e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:07:35.473 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.473+0000 7fbeb2385640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbeac101100 0x7fbeac19e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41094/0 (socket says 172.21.15.47:41094) 2024-04-17T06:07:35.474 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.473+0000 7fbeb2385640 1 -- 172.21.15.47:0/2972741162 learned_addr learned my addr 172.21.15.47:0/2972741162 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:07:35.474 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.473+0000 7fbeb2385640 1 -- 172.21.15.47:0/2972741162 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbeac106180 msgr2=0x7fbeac19eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:07:35.474 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.473+0000 7fbeb2385640 1 --2- 172.21.15.47:0/2972741162 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbeac106180 0x7fbeac19eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:35.474 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.474+0000 7fbeb2385640 1 -- 172.21.15.47:0/2972741162 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbeac198630 msgr2=0x7fbeac198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:07:35.474 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.474+0000 7fbeb2385640 1 --2- 172.21.15.47:0/2972741162 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbeac198630 0x7fbeac198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:35.474 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.474+0000 7fbeb2385640 1 -- 172.21.15.47:0/2972741162 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbeac199230 con 0x7fbeac101100 2024-04-17T06:07:35.474 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.474+0000 7fbeb2385640 1 --2- 172.21.15.47:0/2972741162 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbeac101100 0x7fbeac19e570 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fbea800d950 tx=0x7fbea800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:07:35.474 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.474+0000 7fbea37fe640 1 -- 172.21.15.47:0/2972741162 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea8017030 con 0x7fbeac101100 2024-04-17T06:07:35.474 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.474+0000 7fbeb45e9640 1 -- 172.21.15.47:0/2972741162 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbeac1a5310 con 0x7fbeac101100 2024-04-17T06:07:35.474 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.474+0000 7fbea37fe640 1 -- 172.21.15.47:0/2972741162 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbea8004020 con 0x7fbeac101100 2024-04-17T06:07:35.474 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.474+0000 7fbea37fe640 1 -- 172.21.15.47:0/2972741162 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea80047a0 con 0x7fbeac101100 2024-04-17T06:07:35.474 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.474+0000 7fbeb45e9640 1 -- 172.21.15.47:0/2972741162 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbeac1a5710 con 0x7fbeac101100 2024-04-17T06:07:35.475 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.477+0000 7fbeb71d4740 1 Processor -- start 2024-04-17T06:07:35.475 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.477+0000 7fbea37fe640 1 -- 172.21.15.47:0/2972741162 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fbea80041d0 con 0x7fbeac101100 2024-04-17T06:07:35.475 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.478+0000 7fbeb71d4740 1 -- start start 2024-04-17T06:07:35.481 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.478+0000 7fbeb71d4740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559bf0bc30c0 0x559bf0bc34a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:07:35.481 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.478+0000 7fbeb71d4740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559bf0bc39e0 0x559bf0bcc270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:07:35.481 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.478+0000 7fbeb71d4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559bf0bcc7b0 0x559bf0bceba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:07:35.481 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.478+0000 7fbeb71d4740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x559bf0b287d0 con 0x559bf0bc39e0 2024-04-17T06:07:35.481 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.478+0000 7fbeb71d4740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x559bf0ac5f70 con 0x559bf0bc30c0 2024-04-17T06:07:35.481 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.478+0000 7fbeb71d4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x559bf0b24790 con 0x559bf0bcc7b0 2024-04-17T06:07:35.481 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.478+0000 7fbeb2385640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559bf0bc39e0 0x559bf0bcc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:07:35.481 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.478+0000 7fbeb2385640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559bf0bc39e0 0x559bf0bcc270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41108/0 (socket says 172.21.15.47:41108) 2024-04-17T06:07:35.481 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.478+0000 7fbeb2385640 1 -- 172.21.15.47:0/2909192731 learned_addr learned my addr 172.21.15.47:0/2909192731 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:07:35.481 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.478+0000 7fbeb2b86640 1 --2- 172.21.15.47:0/2909192731 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559bf0bc30c0 0x559bf0bc34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:07:35.481 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.478+0000 7fbeb1b84640 1 --2- 172.21.15.47:0/2909192731 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559bf0bcc7b0 0x559bf0bceba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:07:35.482 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.478+0000 7fbeb2385640 1 -- 172.21.15.47:0/2909192731 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559bf0bcc7b0 msgr2=0x559bf0bceba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:07:35.482 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.478+0000 7fbeb2385640 1 --2- 172.21.15.47:0/2909192731 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559bf0bcc7b0 0x559bf0bceba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:35.482 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.478+0000 7fbeb2385640 1 -- 172.21.15.47:0/2909192731 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559bf0bc30c0 msgr2=0x559bf0bc34a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:07:35.482 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.478+0000 7fbeb2385640 1 --2- 172.21.15.47:0/2909192731 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559bf0bc30c0 0x559bf0bc34a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:35.482 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.478+0000 7fbeb2385640 1 -- 172.21.15.47:0/2909192731 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559bf083b210 con 0x559bf0bc39e0 2024-04-17T06:07:35.482 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.478+0000 7fbeb1b84640 1 --2- 172.21.15.47:0/2909192731 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559bf0bcc7b0 0x559bf0bceba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:07:35.482 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.478+0000 7fbea37fe640 1 --2- 172.21.15.47:0/2972741162 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fbe980a3dc0 0x7fbe980a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:07:35.482 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.479+0000 7fbeb2b86640 1 --2- 172.21.15.47:0/2972741162 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fbe980a3dc0 0x7fbe980a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:07:35.482 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.479+0000 7fbeb2385640 1 --2- 172.21.15.47:0/2909192731 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559bf0bc39e0 0x559bf0bcc270 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fbea80cd930 tx=0x7fbea80d87a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc56ffaff600cdd9 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:07:35.482 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.479+0000 7fbea37fe640 1 -- 172.21.15.47:0/2972741162 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7fbea80d0240 con 0x7fbeac101100 2024-04-17T06:07:35.482 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.479+0000 7fbea0bfe640 1 -- 172.21.15.47:0/2909192731 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea809bfe0 con 0x559bf0bc39e0 2024-04-17T06:07:35.482 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.479+0000 7fbea0bfe640 1 -- 172.21.15.47:0/2909192731 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbea80d8e20 con 0x559bf0bc39e0 2024-04-17T06:07:35.482 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.479+0000 7fbea0bfe640 1 -- 172.21.15.47:0/2909192731 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea809bb70 con 0x559bf0bc39e0 2024-04-17T06:07:35.483 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.480+0000 7fbeb71d4740 1 -- 172.21.15.47:0/2909192731 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559bf0bc39e0 msgr2=0x559bf0bcc270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:07:35.483 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.480+0000 7fbeb2b86640 1 --2- 172.21.15.47:0/2972741162 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fbe980a3dc0 0x7fbe980a6280 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fbea4009430 tx=0x7fbea4005fd0 comp rx=0 tx=0).ready entity=mgr.24923 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:07:35.483 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.480+0000 7fbeb71d4740 1 --2- 172.21.15.47:0/2909192731 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559bf0bc39e0 0x559bf0bcc270 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fbea80cd930 tx=0x7fbea80d87a0 comp rx=0 tx=0).stop 2024-04-17T06:07:35.483 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.480+0000 7fbeb71d4740 1 -- 172.21.15.47:0/2909192731 shutdown_connections 2024-04-17T06:07:35.483 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.480+0000 7fbeb71d4740 1 --2- 172.21.15.47:0/2909192731 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559bf0bcc7b0 0x559bf0bceba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:35.483 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.480+0000 7fbeb71d4740 1 --2- 172.21.15.47:0/2909192731 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559bf0bc39e0 0x559bf0bcc270 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:35.483 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.480+0000 7fbeb71d4740 1 --2- 172.21.15.47:0/2909192731 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559bf0bc30c0 0x559bf0bc34a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:35.483 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.480+0000 7fbeb71d4740 1 -- 172.21.15.47:0/2909192731 >> 172.21.15.47:0/2909192731 conn(0x559bf0bbd9e0 msgr2=0x559bf0bbddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:07:35.483 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.480+0000 7fbeb71d4740 1 -- 172.21.15.47:0/2909192731 shutdown_connections 2024-04-17T06:07:35.483 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.480+0000 7fbeb71d4740 1 -- 172.21.15.47:0/2909192731 wait complete. 2024-04-17T06:07:35.483 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.480+0000 7fbeb71d4740 1 -- start start 2024-04-17T06:07:35.483 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.480+0000 7fbeb71d4740 1 Processor -- start 2024-04-17T06:07:35.483 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.481+0000 7fbeb71d4740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559bf0bc30c0 0x559bf0bc97f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:07:35.483 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.481+0000 7fbeb71d4740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559bf0bc39e0 0x559bf0bc9d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:07:35.484 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.481+0000 7fbeb71d4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559bf0bcc7b0 0x559bf0bca270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:07:35.484 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.481+0000 7fbeb71d4740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x559bf0bcf3e0 con 0x559bf0bc39e0 2024-04-17T06:07:35.484 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.481+0000 7fbeb71d4740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x559bf0ab0e90 con 0x559bf0bc30c0 2024-04-17T06:07:35.484 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.481+0000 7fbeb71d4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x559bf0bcf560 con 0x559bf0bcc7b0 2024-04-17T06:07:35.484 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.481+0000 7fbeb71d4740 10 client.0 _ll_register_callbacks cb 0x559bf0ac6d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:07:35.484 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.481+0000 7fbeb1b84640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559bf0bc39e0 0x559bf0bc9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:07:35.484 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.481+0000 7fbeb1b84640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559bf0bc39e0 0x559bf0bc9d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41124/0 (socket says 172.21.15.47:41124) 2024-04-17T06:07:35.484 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.481+0000 7fbeb1b84640 1 -- 172.21.15.47:0/1171380119 learned_addr learned my addr 172.21.15.47:0/1171380119 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:07:35.484 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.481+0000 7fbeb2385640 1 --2- 172.21.15.47:0/1171380119 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559bf0bc30c0 0x559bf0bc97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:07:35.484 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.481+0000 7fbeb2b86640 1 --2- 172.21.15.47:0/1171380119 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559bf0bcc7b0 0x559bf0bca270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:07:35.484 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.481+0000 7fbeb1b84640 1 -- 172.21.15.47:0/1171380119 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559bf0bcc7b0 msgr2=0x559bf0bca270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:07:35.484 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.481+0000 7fbeb1b84640 1 --2- 172.21.15.47:0/1171380119 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559bf0bcc7b0 0x559bf0bca270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:35.484 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.481+0000 7fbeb1b84640 1 -- 172.21.15.47:0/1171380119 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559bf0bc30c0 msgr2=0x559bf0bc97f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:07:35.485 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.481+0000 7fbeb1b84640 1 --2- 172.21.15.47:0/1171380119 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559bf0bc30c0 0x559bf0bc97f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:35.485 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.481+0000 7fbeb1b84640 1 -- 172.21.15.47:0/1171380119 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559bf0907620 con 0x559bf0bc39e0 2024-04-17T06:07:35.485 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.481+0000 7fbeb2385640 1 --2- 172.21.15.47:0/1171380119 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559bf0bc30c0 0x559bf0bc97f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T06:07:35.485 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.481+0000 7fbeb1b84640 1 --2- 172.21.15.47:0/1171380119 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559bf0bc39e0 0x559bf0bc9d30 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c00c150 tx=0x7fbe9c00c610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36d26af805908f38 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:07:35.485 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.481+0000 7fbe95ffb640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-17T06:07:35.485 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.481+0000 7fbe95ffb640 1 -- 172.21.15.47:0/1171380119 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe9c015070 con 0x559bf0bc39e0 2024-04-17T06:07:35.485 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.482+0000 7fbeb71d4740 1 -- 172.21.15.47:0/1171380119 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbea80d8e20 con 0x559bf0bc39e0 2024-04-17T06:07:35.485 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.482+0000 7fbe95ffb640 1 -- 172.21.15.47:0/1171380119 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7fbe9c011800 con 0x559bf0bc39e0 2024-04-17T06:07:35.485 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.482+0000 7fbe95ffb640 1 -- 172.21.15.47:0/1171380119 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fbe9c021050 con 0x559bf0bc39e0 2024-04-17T06:07:35.485 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.482+0000 7fbeb71d4740 10 client.15927 fetch_fsmap learned FSMap version 30 2024-04-17T06:07:35.485 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.482+0000 7fbeb71d4740 1 -- 172.21.15.47:0/1171380119 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559bf0bdb670 con 0x559bf0bc39e0 2024-04-17T06:07:35.485 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.482+0000 7fbe95ffb640 1 -- 172.21.15.47:0/1171380119 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbea80d8e20 con 0x559bf0bc39e0 2024-04-17T06:07:35.485 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.482+0000 7fbe95ffb640 1 -- 172.21.15.47:0/1171380119 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 30) ==== 8339+0+0 (secure 0 0 0) 0x7fbe9c019a60 con 0x559bf0bc39e0 2024-04-17T06:07:35.485 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.482+0000 7fbeb71d4740 10 client.15927 fetch_fsmap finished waiting for FSMap version 30 2024-04-17T06:07:35.486 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.482+0000 7fbeb71d4740 10 client.15927 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-17T06:07:35.486 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.482+0000 7fbeb71d4740 20 client.15927 populate_metadata read hostname 'smithi047' 2024-04-17T06:07:35.486 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.482+0000 7fbeb71d4740 1 --2- 172.21.15.47:0/1171380119 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x559bf0bdcb30 0x559bf0bdefa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:07:35.486 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.482+0000 7fbeb71d4740 4 client.15927 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T06:07:35.486 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.482+0000 7fbeb71d4740 1 -- 172.21.15.47:0/1171380119 --> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559bf0b361f0 con 0x559bf0bdcb30 2024-04-17T06:07:35.486 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.483+0000 7fbeb2385640 1 --2- 172.21.15.47:0/1171380119 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x559bf0bdcb30 0x559bf0bdefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:07:35.486 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.483+0000 7fbe95ffb640 10 client.15927 ms_handle_connect on v2:172.21.15.98:6838/2841287917 2024-04-17T06:07:35.486 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.483+0000 7fbeb2385640 1 --2- 172.21.15.47:0/1171380119 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x559bf0bdcb30 0x559bf0bdefa0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ccbf9413384fdd0 server_cookie=bd4191c4c28ec144 in_seq=0 out_seq=0 2024-04-17T06:07:35.487 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.484+0000 7fbe95ffb640 1 -- 172.21.15.47:0/1171380119 <== mds.0 v2:172.21.15.98:6838/2841287917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559bf0b361f0 con 0x559bf0bdcb30 2024-04-17T06:07:35.487 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.484+0000 7fbe95ffb640 10 client.15927 handle_command_reply: tid=0 2024-04-17T06:07:35.487 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.484+0000 7fbeb71d4740 1 client.15927 shutdown 2024-04-17T06:07:35.487 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.484+0000 7fbeb71d4740 1 -- 172.21.15.47:0/1171380119 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559bf0bc39e0 msgr2=0x559bf0bc9d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:07:35.487 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.484+0000 7fbeb71d4740 1 --2- 172.21.15.47:0/1171380119 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559bf0bc39e0 0x559bf0bc9d30 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c00c150 tx=0x7fbe9c00c610 comp rx=0 tx=0).stop 2024-04-17T06:07:35.488 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.484+0000 7fbeb71d4740 1 -- 172.21.15.47:0/1171380119 shutdown_connections 2024-04-17T06:07:35.488 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.484+0000 7fbeb71d4740 1 --2- 172.21.15.47:0/1171380119 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x559bf0bdcb30 0x559bf0bdefa0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:35.488 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.484+0000 7fbeb71d4740 1 --2- 172.21.15.47:0/1171380119 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559bf0bcc7b0 0x559bf0bca270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:35.488 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.484+0000 7fbeb71d4740 1 --2- 172.21.15.47:0/1171380119 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559bf0bc39e0 0x559bf0bc9d30 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:35.488 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.484+0000 7fbeb71d4740 1 --2- 172.21.15.47:0/1171380119 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559bf0bc30c0 0x559bf0bc97f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:35.488 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.484+0000 7fbe95ffb640 0 client.15927 ms_handle_reset on v2:172.21.15.98:6838/2841287917 2024-04-17T06:07:35.488 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.484+0000 7fbeb71d4740 1 -- 172.21.15.47:0/1171380119 >> 172.21.15.47:0/1171380119 conn(0x559bf0bbd9e0 msgr2=0x559bf0bbf730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:07:35.488 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.484+0000 7fbeb71d4740 1 -- 172.21.15.47:0/1171380119 shutdown_connections 2024-04-17T06:07:35.488 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.484+0000 7fbeb71d4740 1 -- 172.21.15.47:0/1171380119 wait complete. 2024-04-17T06:07:35.488 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.484+0000 7fbeb71d4740 20 client.15927 trim_cache size 0 max 16384 2024-04-17T06:07:35.501 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.499+0000 7fbeb71d4740 1 Processor -- start 2024-04-17T06:07:35.501 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.499+0000 7fbeb71d4740 1 -- start start 2024-04-17T06:07:35.501 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.499+0000 7fbeb71d4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559bf0bc30c0 0x559bf0bcbf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:07:35.501 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.499+0000 7fbeb71d4740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559bf0bc39e0 0x559bf0bd82a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:07:35.502 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.499+0000 7fbeb71d4740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559bf0bcc7b0 0x559bf0bd4b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:07:35.502 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.499+0000 7fbeb71d4740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fbeac1b0510 con 0x559bf0bcc7b0 2024-04-17T06:07:35.502 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.499+0000 7fbeb71d4740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fbeac1addd0 con 0x559bf0bc39e0 2024-04-17T06:07:35.502 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.499+0000 7fbeb71d4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fbeac1abc20 con 0x559bf0bc30c0 2024-04-17T06:07:35.502 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.500+0000 7fbeb2b86640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559bf0bcc7b0 0x559bf0bd4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:07:35.502 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.500+0000 7fbeb2b86640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559bf0bcc7b0 0x559bf0bd4b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41136/0 (socket says 172.21.15.47:41136) 2024-04-17T06:07:35.502 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.500+0000 7fbeb2b86640 1 -- 172.21.15.47:0/2245290482 learned_addr learned my addr 172.21.15.47:0/2245290482 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:07:35.502 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.500+0000 7fbeb2385640 1 --2- 172.21.15.47:0/2245290482 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559bf0bc30c0 0x559bf0bcbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:07:35.502 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.500+0000 7fbeb1b84640 1 --2- 172.21.15.47:0/2245290482 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559bf0bc39e0 0x559bf0bd82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:07:35.502 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.500+0000 7fbeb2b86640 1 -- 172.21.15.47:0/2245290482 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559bf0bc30c0 msgr2=0x559bf0bcbf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:07:35.502 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.500+0000 7fbeb2b86640 1 --2- 172.21.15.47:0/2245290482 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559bf0bc30c0 0x559bf0bcbf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:35.502 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.500+0000 7fbeb2b86640 1 -- 172.21.15.47:0/2245290482 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559bf0bc39e0 msgr2=0x559bf0bd82a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:07:35.502 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.500+0000 7fbeb2b86640 1 --2- 172.21.15.47:0/2245290482 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559bf0bc39e0 0x559bf0bd82a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:35.503 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.500+0000 7fbeb2b86640 1 -- 172.21.15.47:0/2245290482 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559bf0bc87a0 con 0x559bf0bcc7b0 2024-04-17T06:07:35.503 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.500+0000 7fbeb2b86640 1 --2- 172.21.15.47:0/2245290482 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559bf0bcc7b0 0x559bf0bd4b90 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fbea4009990 tx=0x7fbea403e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9eea4d1f5533d828 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:07:35.503 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.500+0000 7fbe97fff640 1 -- 172.21.15.47:0/2245290482 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea4039040 con 0x559bf0bcc7b0 2024-04-17T06:07:35.503 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.500+0000 7fbe97fff640 1 -- 172.21.15.47:0/2245290482 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbea4051070 con 0x559bf0bcc7b0 2024-04-17T06:07:35.503 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.500+0000 7fbe97fff640 1 -- 172.21.15.47:0/2245290482 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea404c6b0 con 0x559bf0bcc7b0 2024-04-17T06:07:35.503 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.501+0000 7fbeb71d4740 1 -- 172.21.15.47:0/2245290482 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559bf0bcc7b0 msgr2=0x559bf0bd4b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:07:35.503 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.501+0000 7fbeb71d4740 1 --2- 172.21.15.47:0/2245290482 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559bf0bcc7b0 0x559bf0bd4b90 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fbea4009990 tx=0x7fbea403e000 comp rx=0 tx=0).stop 2024-04-17T06:07:35.503 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.501+0000 7fbeb71d4740 1 -- 172.21.15.47:0/2245290482 shutdown_connections 2024-04-17T06:07:35.503 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.501+0000 7fbeb71d4740 1 --2- 172.21.15.47:0/2245290482 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559bf0bcc7b0 0x559bf0bd4b90 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:35.503 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.501+0000 7fbeb71d4740 1 --2- 172.21.15.47:0/2245290482 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559bf0bc39e0 0x559bf0bd82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:35.503 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.501+0000 7fbeb71d4740 1 --2- 172.21.15.47:0/2245290482 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559bf0bc30c0 0x559bf0bcbf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:35.503 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.501+0000 7fbeb71d4740 1 -- 172.21.15.47:0/2245290482 >> 172.21.15.47:0/2245290482 conn(0x559bf0bbd9e0 msgr2=0x559bf0bbddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:07:35.503 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.501+0000 7fbeb71d4740 1 -- 172.21.15.47:0/2245290482 shutdown_connections 2024-04-17T06:07:35.504 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.501+0000 7fbeb71d4740 1 -- 172.21.15.47:0/2245290482 wait complete. 2024-04-17T06:07:35.504 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.501+0000 7fbeb71d4740 1 -- start start 2024-04-17T06:07:35.504 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.501+0000 7fbeb71d4740 1 Processor -- start 2024-04-17T06:07:35.504 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.501+0000 7fbeb71d4740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559bf0bc30c0 0x559bf0c3b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:07:35.504 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.501+0000 7fbeb71d4740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559bf0bc39e0 0x559bf0c3b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:07:35.504 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.501+0000 7fbeb71d4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559bf0bcc7b0 0x559bf0c46120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:07:35.504 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.501+0000 7fbeb71d4740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fbeac1ac290 con 0x559bf0bc39e0 2024-04-17T06:07:35.504 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.501+0000 7fbeb71d4740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fbeac1ad700 con 0x559bf0bc30c0 2024-04-17T06:07:35.504 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.501+0000 7fbeb71d4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fbeac1ac960 con 0x559bf0bcc7b0 2024-04-17T06:07:35.504 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.502+0000 7fbeb71d4740 10 client.0 _ll_register_callbacks cb 0x7fbe9c00ab60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:07:35.504 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.502+0000 7fbeb1b84640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559bf0bc39e0 0x559bf0c3b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:07:35.504 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.502+0000 7fbeb1b84640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559bf0bc39e0 0x559bf0c3b5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41148/0 (socket says 172.21.15.47:41148) 2024-04-17T06:07:35.504 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.502+0000 7fbeb1b84640 1 -- 172.21.15.47:0/3617814486 learned_addr learned my addr 172.21.15.47:0/3617814486 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:07:35.504 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.502+0000 7fbeb2b86640 1 --2- 172.21.15.47:0/3617814486 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559bf0bcc7b0 0x559bf0c46120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:07:35.505 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.502+0000 7fbeb2385640 1 --2- 172.21.15.47:0/3617814486 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559bf0bc30c0 0x559bf0c3b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:07:35.505 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.502+0000 7fbeb1b84640 1 -- 172.21.15.47:0/3617814486 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559bf0bcc7b0 msgr2=0x559bf0c46120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:07:35.505 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.502+0000 7fbeb1b84640 1 --2- 172.21.15.47:0/3617814486 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559bf0bcc7b0 0x559bf0c46120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:35.505 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.502+0000 7fbeb1b84640 1 -- 172.21.15.47:0/3617814486 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559bf0bc30c0 msgr2=0x559bf0c3b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:07:35.505 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.502+0000 7fbeb1b84640 1 --2- 172.21.15.47:0/3617814486 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559bf0bc30c0 0x559bf0c3b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:35.505 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.502+0000 7fbeb1b84640 1 -- 172.21.15.47:0/3617814486 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559bf0c46660 con 0x559bf0bc39e0 2024-04-17T06:07:35.505 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.502+0000 7fbeb2b86640 1 --2- 172.21.15.47:0/3617814486 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559bf0bcc7b0 0x559bf0c46120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:07:35.505 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.502+0000 7fbeb1b84640 1 --2- 172.21.15.47:0/3617814486 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559bf0bc39e0 0x559bf0c3b5e0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c00ac70 tx=0x7fbe9c00a3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c67d5ca206a708eb server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:07:35.505 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.502+0000 7fbe95ffb640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-17T06:07:35.505 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.502+0000 7fbe95ffb640 1 -- 172.21.15.47:0/3617814486 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe9c02b070 con 0x559bf0bc39e0 2024-04-17T06:07:35.505 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.502+0000 7fbe95ffb640 1 -- 172.21.15.47:0/3617814486 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7fbe9c011a10 con 0x559bf0bc39e0 2024-04-17T06:07:35.505 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.502+0000 7fbeb71d4740 1 -- 172.21.15.47:0/3617814486 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbea4051070 con 0x559bf0bc39e0 2024-04-17T06:07:35.505 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.502+0000 7fbe95ffb640 1 -- 172.21.15.47:0/3617814486 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fbe9c037050 con 0x559bf0bc39e0 2024-04-17T06:07:35.505 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.502+0000 7fbeb71d4740 10 client.15933 fetch_fsmap learned FSMap version 30 2024-04-17T06:07:35.506 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.502+0000 7fbeb71d4740 1 -- 172.21.15.47:0/3617814486 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559bf0c478e0 con 0x559bf0bc39e0 2024-04-17T06:07:35.506 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.503+0000 7fbe95ffb640 1 -- 172.21.15.47:0/3617814486 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbea4051070 con 0x559bf0bc39e0 2024-04-17T06:07:35.506 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.503+0000 7fbe95ffb640 1 -- 172.21.15.47:0/3617814486 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 30) ==== 8339+0+0 (secure 0 0 0) 0x7fbe9c0304d0 con 0x559bf0bc39e0 2024-04-17T06:07:35.506 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.503+0000 7fbeb71d4740 10 client.15933 fetch_fsmap finished waiting for FSMap version 30 2024-04-17T06:07:35.506 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.503+0000 7fbeb71d4740 10 client.15933 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-17T06:07:35.506 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.503+0000 7fbeb71d4740 20 client.15933 populate_metadata read hostname 'smithi047' 2024-04-17T06:07:35.506 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.503+0000 7fbeb71d4740 1 --2- 172.21.15.47:0/3617814486 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x559bf0c48090 0x559bf0c4a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:07:35.506 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.503+0000 7fbeb71d4740 4 client.15933 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-17T06:07:35.506 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.503+0000 7fbeb71d4740 1 -- 172.21.15.47:0/3617814486 --> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559bf0b30060 con 0x559bf0c48090 2024-04-17T06:07:35.506 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.503+0000 7fbeb2385640 1 --2- 172.21.15.47:0/3617814486 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x559bf0c48090 0x559bf0c4a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:07:35.506 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.503+0000 7fbe95ffb640 10 client.15933 ms_handle_connect on v2:172.21.15.98:6838/2841287917 2024-04-17T06:07:35.506 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.503+0000 7fbeb2385640 1 --2- 172.21.15.47:0/3617814486 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x559bf0c48090 0x559bf0c4a4b0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7127d8a71963d196 server_cookie=203760072625fb5e in_seq=0 out_seq=0 2024-04-17T06:07:35.510 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.511+0000 7fbe95ffb640 1 -- 172.21.15.47:0/3617814486 <== mds.0 v2:172.21.15.98:6838/2841287917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559bf0b30060 con 0x559bf0c48090 2024-04-17T06:07:35.510 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.511+0000 7fbe95ffb640 10 client.15933 handle_command_reply: tid=0 2024-04-17T06:07:35.510 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.512+0000 7fbeb71d4740 1 client.15933 shutdown 2024-04-17T06:07:35.510 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.512+0000 7fbeb71d4740 1 -- 172.21.15.47:0/3617814486 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559bf0bc39e0 msgr2=0x559bf0c3b5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:07:35.510 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.512+0000 7fbeb71d4740 1 --2- 172.21.15.47:0/3617814486 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559bf0bc39e0 0x559bf0c3b5e0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c00ac70 tx=0x7fbe9c00a3b0 comp rx=0 tx=0).stop 2024-04-17T06:07:35.510 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.512+0000 7fbeb71d4740 1 -- 172.21.15.47:0/3617814486 shutdown_connections 2024-04-17T06:07:35.510 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.512+0000 7fbeb71d4740 1 --2- 172.21.15.47:0/3617814486 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x559bf0c48090 0x559bf0c4a4b0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:35.511 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.512+0000 7fbeb71d4740 1 --2- 172.21.15.47:0/3617814486 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559bf0bcc7b0 0x559bf0c46120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:35.511 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.512+0000 7fbeb71d4740 1 --2- 172.21.15.47:0/3617814486 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559bf0bc39e0 0x559bf0c3b5e0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:35.511 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.512+0000 7fbe95ffb640 0 client.15933 ms_handle_reset on v2:172.21.15.98:6838/2841287917 2024-04-17T06:07:35.511 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.512+0000 7fbeb71d4740 1 --2- 172.21.15.47:0/3617814486 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559bf0bc30c0 0x559bf0c3b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:35.511 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.512+0000 7fbeb71d4740 1 -- 172.21.15.47:0/3617814486 >> 172.21.15.47:0/3617814486 conn(0x559bf0bbd9e0 msgr2=0x559bf0bbcb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:07:35.511 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.512+0000 7fbeb71d4740 1 -- 172.21.15.47:0/3617814486 shutdown_connections 2024-04-17T06:07:35.511 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.512+0000 7fbeb71d4740 1 -- 172.21.15.47:0/3617814486 wait complete. 2024-04-17T06:07:35.511 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.512+0000 7fbeb71d4740 20 client.15933 trim_cache size 0 max 16384 2024-04-17T06:07:35.511 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-04-17T06:07:35.511 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-04-17T06:07:35.511 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-04-17T06:07:35.512 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-04-17T06:07:35.512 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.513+0000 7fbeb45e9640 1 -- 172.21.15.47:0/2972741162 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fbe980a3dc0 msgr2=0x7fbe980a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:07:35.512 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.513+0000 7fbeb45e9640 1 --2- 172.21.15.47:0/2972741162 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fbe980a3dc0 0x7fbe980a6280 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fbea4009430 tx=0x7fbea4005fd0 comp rx=0 tx=0).stop 2024-04-17T06:07:35.513 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.513+0000 7fbeb45e9640 1 -- 172.21.15.47:0/2972741162 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbeac101100 msgr2=0x7fbeac19e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:07:35.513 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.513+0000 7fbeb45e9640 1 --2- 172.21.15.47:0/2972741162 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbeac101100 0x7fbeac19e570 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fbea800d950 tx=0x7fbea800de10 comp rx=0 tx=0).stop 2024-04-17T06:07:35.513 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.513+0000 7fbeb45e9640 1 -- 172.21.15.47:0/2972741162 shutdown_connections 2024-04-17T06:07:35.513 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.513+0000 7fbeb45e9640 1 --2- 172.21.15.47:0/2972741162 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbeac198630 0x7fbeac198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:35.513 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.513+0000 7fbeb45e9640 1 --2- 172.21.15.47:0/2972741162 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbeac106180 0x7fbeac19eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:35.513 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.513+0000 7fbeb45e9640 1 --2- 172.21.15.47:0/2972741162 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fbe980a3dc0 0x7fbe980a6280 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:35.513 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.513+0000 7fbeb45e9640 1 --2- 172.21.15.47:0/2972741162 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbeac101100 0x7fbeac19e570 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:35.513 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.513+0000 7fbeb45e9640 1 -- 172.21.15.47:0/2972741162 >> 172.21.15.47:0/2972741162 conn(0x7fbeac077fe0 msgr2=0x7fbeac0feef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:07:35.513 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.513+0000 7fbeb45e9640 1 -- 172.21.15.47:0/2972741162 shutdown_connections 2024-04-17T06:07:35.513 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:35.513+0000 7fbeb45e9640 1 -- 172.21.15.47:0/2972741162 wait complete. 2024-04-17T06:07:36.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:07:35 smithi098 ceph-mon[36516]: pgmap v193: 97 pgs: 97 active+clean; 3.2 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-04-17T06:07:36.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:07:35 smithi047 ceph-mon[32223]: pgmap v193: 97 pgs: 97 active+clean; 3.2 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-04-17T06:07:36.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:07:35 smithi103 ceph-mon[36243]: pgmap v193: 97 pgs: 97 active+clean; 3.2 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-04-17T06:07:38.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:07:37 smithi098 ceph-mon[36516]: pgmap v194: 97 pgs: 97 active+clean; 3.2 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.3 KiB/s wr, 9 op/s 2024-04-17T06:07:38.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:07:37 smithi047 ceph-mon[32223]: pgmap v194: 97 pgs: 97 active+clean; 3.2 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.3 KiB/s wr, 9 op/s 2024-04-17T06:07:38.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:07:37 smithi103 ceph-mon[36243]: pgmap v194: 97 pgs: 97 active+clean; 3.2 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.3 KiB/s wr, 9 op/s 2024-04-17T06:07:40.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:07:39 smithi098 ceph-mon[36516]: pgmap v195: 97 pgs: 97 active+clean; 3.2 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.3 KiB/s wr, 9 op/s 2024-04-17T06:07:40.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:07:39 smithi047 ceph-mon[32223]: pgmap v195: 97 pgs: 97 active+clean; 3.2 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.3 KiB/s wr, 9 op/s 2024-04-17T06:07:40.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:07:39 smithi103 ceph-mon[36243]: pgmap v195: 97 pgs: 97 active+clean; 3.2 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.3 KiB/s wr, 9 op/s 2024-04-17T06:07:42.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:07:41 smithi098 ceph-mon[36516]: pgmap v196: 97 pgs: 97 active+clean; 3.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-04-17T06:07:42.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:07:41 smithi047 ceph-mon[32223]: pgmap v196: 97 pgs: 97 active+clean; 3.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-04-17T06:07:42.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:07:41 smithi103 ceph-mon[36243]: pgmap v196: 97 pgs: 97 active+clean; 3.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-04-17T06:07:44.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:07:43 smithi098 ceph-mon[36516]: pgmap v197: 97 pgs: 97 active+clean; 3.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:07:44.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:07:43 smithi047 ceph-mon[32223]: pgmap v197: 97 pgs: 97 active+clean; 3.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:07:44.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:07:43 smithi103 ceph-mon[36243]: pgmap v197: 97 pgs: 97 active+clean; 3.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:07:46.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:07:45 smithi098 ceph-mon[36516]: pgmap v198: 97 pgs: 97 active+clean; 3.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-04-17T06:07:46.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:07:45 smithi047 ceph-mon[32223]: pgmap v198: 97 pgs: 97 active+clean; 3.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-04-17T06:07:46.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:07:45 smithi103 ceph-mon[36243]: pgmap v198: 97 pgs: 97 active+clean; 3.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-04-17T06:07:48.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:07:47 smithi098 ceph-mon[36516]: pgmap v199: 97 pgs: 97 active+clean; 3.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:07:48.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:07:47 smithi047 ceph-mon[32223]: pgmap v199: 97 pgs: 97 active+clean; 3.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:07:48.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:07:47 smithi103 ceph-mon[36243]: pgmap v199: 97 pgs: 97 active+clean; 3.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:07:48.612 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.611+0000 7f74d69a8640 1 -- 172.21.15.47:0/1383988188 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f74d0106180 msgr2=0x7f74d010dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:07:48.613 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.611+0000 7f74d69a8640 1 --2- 172.21.15.47:0/1383988188 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f74d0106180 0x7f74d010dd50 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f74c8009ca0 tx=0x7f74c802c0f0 comp rx=0 tx=0).stop 2024-04-17T06:07:48.613 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.611+0000 7f74d69a8640 1 -- 172.21.15.47:0/1383988188 shutdown_connections 2024-04-17T06:07:48.613 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.611+0000 7f74d69a8640 1 --2- 172.21.15.47:0/1383988188 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f74d0106180 0x7f74d010dd50 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:48.613 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.611+0000 7f74d69a8640 1 --2- 172.21.15.47:0/1383988188 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f74d0101ab0 0x7f74d0105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:48.613 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.611+0000 7f74d69a8640 1 --2- 172.21.15.47:0/1383988188 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f74d0101100 0x7f74d01014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:48.613 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.611+0000 7f74d69a8640 1 -- 172.21.15.47:0/1383988188 >> 172.21.15.47:0/1383988188 conn(0x7f74d0077fe0 msgr2=0x7f74d00783f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:07:48.613 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.611+0000 7f74d69a8640 1 -- 172.21.15.47:0/1383988188 shutdown_connections 2024-04-17T06:07:48.613 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.611+0000 7f74d69a8640 1 -- 172.21.15.47:0/1383988188 wait complete. 2024-04-17T06:07:48.613 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.612+0000 7f74d69a8640 1 Processor -- start 2024-04-17T06:07:48.613 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.612+0000 7f74d69a8640 1 -- start start 2024-04-17T06:07:48.613 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.612+0000 7f74d69a8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f74d0101100 0x7f74d0199560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:07:48.613 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.612+0000 7f74d69a8640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f74d0106180 0x7f74d0199aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:07:48.614 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.612+0000 7f74d69a8640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f74d019a080 0x7f74d01a45e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:07:48.614 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.612+0000 7f74d69a8640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f74d0111240 con 0x7f74d0106180 2024-04-17T06:07:48.614 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.612+0000 7f74d69a8640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f74d01110c0 con 0x7f74d019a080 2024-04-17T06:07:48.614 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.612+0000 7f74d69a8640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f74d01113c0 con 0x7f74d0101100 2024-04-17T06:07:48.614 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.612+0000 7f74c7fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f74d0106180 0x7f74d0199aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:07:48.614 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.612+0000 7f74c7fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f74d0106180 0x7f74d0199aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60752/0 (socket says 172.21.15.47:60752) 2024-04-17T06:07:48.614 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.612+0000 7f74c7fff640 1 -- 172.21.15.47:0/2425337349 learned_addr learned my addr 172.21.15.47:0/2425337349 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:07:48.614 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.613+0000 7f74d4f45640 1 --2- 172.21.15.47:0/2425337349 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f74d019a080 0x7f74d01a45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:07:48.614 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.613+0000 7f74cffff640 1 --2- 172.21.15.47:0/2425337349 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f74d0101100 0x7f74d0199560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:07:48.614 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.613+0000 7f74c7fff640 1 -- 172.21.15.47:0/2425337349 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f74d0101100 msgr2=0x7f74d0199560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:07:48.614 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.613+0000 7f74c7fff640 1 --2- 172.21.15.47:0/2425337349 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f74d0101100 0x7f74d0199560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:48.614 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.613+0000 7f74c7fff640 1 -- 172.21.15.47:0/2425337349 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f74d019a080 msgr2=0x7f74d01a45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:07:48.614 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.613+0000 7f74c7fff640 1 --2- 172.21.15.47:0/2425337349 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f74d019a080 0x7f74d01a45e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:48.615 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.613+0000 7f74c7fff640 1 -- 172.21.15.47:0/2425337349 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74d01a4e00 con 0x7f74d0106180 2024-04-17T06:07:48.615 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.613+0000 7f74cffff640 1 --2- 172.21.15.47:0/2425337349 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f74d0101100 0x7f74d0199560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:07:48.615 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.613+0000 7f74c7fff640 1 --2- 172.21.15.47:0/2425337349 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f74d0106180 0x7f74d0199aa0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f74b800d770 tx=0x7f74b800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:07:48.615 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.613+0000 7f74cdffb640 1 -- 172.21.15.47:0/2425337349 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74b8017070 con 0x7f74d0106180 2024-04-17T06:07:48.615 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.613+0000 7f74cdffb640 1 -- 172.21.15.47:0/2425337349 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74b8009c30 con 0x7f74d0106180 2024-04-17T06:07:48.615 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.613+0000 7f74cdffb640 1 -- 172.21.15.47:0/2425337349 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74b800f620 con 0x7f74d0106180 2024-04-17T06:07:48.615 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.613+0000 7f74d69a8640 1 -- 172.21.15.47:0/2425337349 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74d01a50f0 con 0x7f74d0106180 2024-04-17T06:07:48.615 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.613+0000 7f74d69a8640 1 -- 172.21.15.47:0/2425337349 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74d01a5690 con 0x7f74d0106180 2024-04-17T06:07:48.621 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.616+0000 7f74cdffb640 1 -- 172.21.15.47:0/2425337349 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f74b801a070 con 0x7f74d0106180 2024-04-17T06:07:48.621 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.616+0000 7f74d959a740 1 Processor -- start 2024-04-17T06:07:48.622 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.616+0000 7f74d959a740 1 -- start start 2024-04-17T06:07:48.622 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.616+0000 7f74d959a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5616ba854be0 0x5616ba854fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:07:48.622 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.616+0000 7f74d959a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5616ba855500 0x5616ba862680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:07:48.622 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.616+0000 7f74d959a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5616ba862bc0 0x5616ba864fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:07:48.622 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.616+0000 7f74d959a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5616ba7ba7d0 con 0x5616ba855500 2024-04-17T06:07:48.622 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.616+0000 7f74d959a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5616ba757f70 con 0x5616ba854be0 2024-04-17T06:07:48.622 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.616+0000 7f74d959a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5616ba7b6790 con 0x5616ba862bc0 2024-04-17T06:07:48.622 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.616+0000 7f74cffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5616ba855500 0x5616ba862680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:07:48.622 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.616+0000 7f74cdffb640 1 --2- 172.21.15.47:0/2425337349 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f74bc0a3e40 0x7f74bc0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:07:48.622 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.616+0000 7f74cffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5616ba855500 0x5616ba862680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60758/0 (socket says 172.21.15.47:60758) 2024-04-17T06:07:48.622 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.616+0000 7f74cffff640 1 -- 172.21.15.47:0/1863156502 learned_addr learned my addr 172.21.15.47:0/1863156502 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:07:48.622 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.616+0000 7f74d4f45640 1 --2- 172.21.15.47:0/2425337349 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f74bc0a3e40 0x7f74bc0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:07:48.622 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.617+0000 7f74cdffb640 1 -- 172.21.15.47:0/2425337349 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f74b80d0240 con 0x7f74d0106180 2024-04-17T06:07:48.622 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.617+0000 7f74c7fff640 1 --2- 172.21.15.47:0/1863156502 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5616ba862bc0 0x5616ba864fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:07:48.623 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.617+0000 7f74cffff640 1 -- 172.21.15.47:0/1863156502 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5616ba862bc0 msgr2=0x5616ba864fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:07:48.623 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.617+0000 7f74cffff640 1 --2- 172.21.15.47:0/1863156502 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5616ba862bc0 0x5616ba864fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:48.623 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.617+0000 7f74d4f45640 1 --2- 172.21.15.47:0/1863156502 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5616ba854be0 0x5616ba854fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:07:48.623 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.617+0000 7f74cffff640 1 -- 172.21.15.47:0/1863156502 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5616ba854be0 msgr2=0x5616ba854fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:07:48.623 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.617+0000 7f74cffff640 1 --2- 172.21.15.47:0/1863156502 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5616ba854be0 0x5616ba854fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:48.623 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.617+0000 7f74cffff640 1 -- 172.21.15.47:0/1863156502 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616ba4cd210 con 0x5616ba855500 2024-04-17T06:07:48.623 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.617+0000 7f74c7fff640 1 --2- 172.21.15.47:0/1863156502 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5616ba862bc0 0x5616ba864fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:07:48.623 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.617+0000 7f74d4f45640 1 --2- 172.21.15.47:0/1863156502 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5616ba854be0 0x5616ba854fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:07:48.623 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.617+0000 7f74cffff640 1 --2- 172.21.15.47:0/1863156502 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5616ba855500 0x5616ba862680 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f74c000bd40 tx=0x7f74c000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35d1f09bfcd01b99 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:07:48.623 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.617+0000 7f74c67fc640 1 -- 172.21.15.47:0/1863156502 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c0015070 con 0x5616ba855500 2024-04-17T06:07:48.623 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.617+0000 7f74d4f45640 1 --2- 172.21.15.47:0/2425337349 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f74bc0a3e40 0x7f74bc0a6300 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f74c8004af0 tx=0x7f74c802e040 comp rx=0 tx=0).ready entity=mgr.24923 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:07:48.623 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.617+0000 7f74c67fc640 1 -- 172.21.15.47:0/1863156502 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74c000be30 con 0x5616ba855500 2024-04-17T06:07:48.623 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.617+0000 7f74c67fc640 1 -- 172.21.15.47:0/1863156502 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c0011aa0 con 0x5616ba855500 2024-04-17T06:07:48.623 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.618+0000 7f74d959a740 1 -- 172.21.15.47:0/1863156502 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5616ba855500 msgr2=0x5616ba862680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:07:48.624 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.618+0000 7f74d959a740 1 --2- 172.21.15.47:0/1863156502 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5616ba855500 0x5616ba862680 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f74c000bd40 tx=0x7f74c000f730 comp rx=0 tx=0).stop 2024-04-17T06:07:48.624 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.618+0000 7f74d959a740 1 -- 172.21.15.47:0/1863156502 shutdown_connections 2024-04-17T06:07:48.624 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.618+0000 7f74d959a740 1 --2- 172.21.15.47:0/1863156502 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5616ba862bc0 0x5616ba864fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:48.624 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.618+0000 7f74d959a740 1 --2- 172.21.15.47:0/1863156502 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5616ba855500 0x5616ba862680 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:48.624 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.618+0000 7f74d959a740 1 --2- 172.21.15.47:0/1863156502 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5616ba854be0 0x5616ba854fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:48.624 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.618+0000 7f74d959a740 1 -- 172.21.15.47:0/1863156502 >> 172.21.15.47:0/1863156502 conn(0x5616ba84f9e0 msgr2=0x5616ba84fdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:07:48.624 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.618+0000 7f74d959a740 1 -- 172.21.15.47:0/1863156502 shutdown_connections 2024-04-17T06:07:48.624 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.618+0000 7f74d959a740 1 -- 172.21.15.47:0/1863156502 wait complete. 2024-04-17T06:07:48.624 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.618+0000 7f74d959a740 1 -- start start 2024-04-17T06:07:48.624 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.618+0000 7f74d959a740 1 Processor -- start 2024-04-17T06:07:48.624 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.619+0000 7f74d959a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5616ba854be0 0x5616ba85fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:07:48.624 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.619+0000 7f74d959a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5616ba855500 0x5616ba860080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:07:48.624 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.619+0000 7f74d959a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5616ba862bc0 0x5616ba8605c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:07:48.624 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.619+0000 7f74d959a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5616ba865780 con 0x5616ba854be0 2024-04-17T06:07:48.624 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.619+0000 7f74d959a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5616ba742e90 con 0x5616ba855500 2024-04-17T06:07:48.625 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.619+0000 7f74d959a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5616ba865900 con 0x5616ba862bc0 2024-04-17T06:07:48.625 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.619+0000 7f74d959a740 10 client.0 _ll_register_callbacks cb 0x5616ba758d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:07:48.625 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.619+0000 7f74cffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5616ba854be0 0x5616ba85fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:07:48.625 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.619+0000 7f74cffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5616ba854be0 0x5616ba85fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60766/0 (socket says 172.21.15.47:60766) 2024-04-17T06:07:48.625 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.619+0000 7f74cffff640 1 -- 172.21.15.47:0/2290556907 learned_addr learned my addr 172.21.15.47:0/2290556907 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:07:48.625 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.619+0000 7f74c7fff640 1 --2- 172.21.15.47:0/2290556907 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5616ba855500 0x5616ba860080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:07:48.625 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.619+0000 7f74d4f45640 1 --2- 172.21.15.47:0/2290556907 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5616ba862bc0 0x5616ba8605c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:07:48.625 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.619+0000 7f74cffff640 1 -- 172.21.15.47:0/2290556907 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5616ba862bc0 msgr2=0x5616ba8605c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:07:48.625 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.619+0000 7f74cffff640 1 --2- 172.21.15.47:0/2290556907 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5616ba862bc0 0x5616ba8605c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:48.625 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.619+0000 7f74cffff640 1 -- 172.21.15.47:0/2290556907 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5616ba855500 msgr2=0x5616ba860080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:07:48.625 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.619+0000 7f74cffff640 1 --2- 172.21.15.47:0/2290556907 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5616ba855500 0x5616ba860080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:48.625 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.619+0000 7f74cffff640 1 -- 172.21.15.47:0/2290556907 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616ba599620 con 0x5616ba854be0 2024-04-17T06:07:48.625 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.619+0000 7f74d4f45640 1 --2- 172.21.15.47:0/2290556907 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5616ba862bc0 0x5616ba8605c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:07:48.625 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.619+0000 7f74cffff640 1 --2- 172.21.15.47:0/2290556907 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5616ba854be0 0x5616ba85fb40 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f74c0011780 tx=0x7f74c0005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9c9f7b02e240859 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:07:48.626 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.619+0000 7f74af7fe640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-17T06:07:48.626 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.620+0000 7f74af7fe640 1 -- 172.21.15.47:0/2290556907 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c0015070 con 0x5616ba854be0 2024-04-17T06:07:48.626 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.620+0000 7f74af7fe640 1 -- 172.21.15.47:0/2290556907 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f74c00055f0 con 0x5616ba854be0 2024-04-17T06:07:48.626 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.620+0000 7f74d959a740 1 -- 172.21.15.47:0/2290556907 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74c000be30 con 0x5616ba854be0 2024-04-17T06:07:48.626 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.620+0000 7f74af7fe640 1 -- 172.21.15.47:0/2290556907 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f74c0011820 con 0x5616ba854be0 2024-04-17T06:07:48.626 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.620+0000 7f74d959a740 10 client.15945 fetch_fsmap learned FSMap version 30 2024-04-17T06:07:48.626 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.620+0000 7f74d959a740 1 -- 172.21.15.47:0/2290556907 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5616ba871910 con 0x5616ba854be0 2024-04-17T06:07:48.626 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.620+0000 7f74af7fe640 1 -- 172.21.15.47:0/2290556907 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74c000be30 con 0x5616ba854be0 2024-04-17T06:07:48.626 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.620+0000 7f74af7fe640 1 -- 172.21.15.47:0/2290556907 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 30) ==== 8339+0+0 (secure 0 0 0) 0x7f74c001a460 con 0x5616ba854be0 2024-04-17T06:07:48.626 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.620+0000 7f74d959a740 10 client.15945 fetch_fsmap finished waiting for FSMap version 30 2024-04-17T06:07:48.626 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.620+0000 7f74d959a740 10 client.15945 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-17T06:07:48.626 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.620+0000 7f74d959a740 20 client.15945 populate_metadata read hostname 'smithi047' 2024-04-17T06:07:48.626 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.620+0000 7f74d959a740 1 --2- 172.21.15.47:0/2290556907 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x5616ba872d70 0x5616ba875160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:07:48.626 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.620+0000 7f74d959a740 4 client.15945 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T06:07:48.627 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.620+0000 7f74d959a740 1 -- 172.21.15.47:0/2290556907 --> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5616ba7c81f0 con 0x5616ba872d70 2024-04-17T06:07:48.627 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.621+0000 7f74c7fff640 1 --2- 172.21.15.47:0/2290556907 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x5616ba872d70 0x5616ba875160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:07:48.627 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.621+0000 7f74c7fff640 1 --2- 172.21.15.47:0/2290556907 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x5616ba872d70 0x5616ba875160 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bec58fa8bda1d2ba server_cookie=c185897445f36e86 in_seq=0 out_seq=0 2024-04-17T06:07:48.627 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.621+0000 7f74af7fe640 10 client.15945 ms_handle_connect on v2:172.21.15.98:6838/2841287917 2024-04-17T06:07:48.627 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.622+0000 7f74af7fe640 1 -- 172.21.15.47:0/2290556907 <== mds.0 v2:172.21.15.98:6838/2841287917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5616ba7c81f0 con 0x5616ba872d70 2024-04-17T06:07:48.627 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.622+0000 7f74af7fe640 10 client.15945 handle_command_reply: tid=0 2024-04-17T06:07:48.627 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.622+0000 7f74d959a740 1 client.15945 shutdown 2024-04-17T06:07:48.627 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.622+0000 7f74d959a740 1 -- 172.21.15.47:0/2290556907 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5616ba854be0 msgr2=0x5616ba85fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:07:48.627 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.622+0000 7f74d959a740 1 --2- 172.21.15.47:0/2290556907 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5616ba854be0 0x5616ba85fb40 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f74c0011780 tx=0x7f74c0005270 comp rx=0 tx=0).stop 2024-04-17T06:07:48.627 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.622+0000 7f74d959a740 1 -- 172.21.15.47:0/2290556907 shutdown_connections 2024-04-17T06:07:48.627 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.622+0000 7f74d959a740 1 --2- 172.21.15.47:0/2290556907 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x5616ba872d70 0x5616ba875160 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:48.627 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.622+0000 7f74d959a740 1 --2- 172.21.15.47:0/2290556907 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5616ba862bc0 0x5616ba8605c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:48.627 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.622+0000 7f74d959a740 1 --2- 172.21.15.47:0/2290556907 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5616ba855500 0x5616ba860080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:48.627 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.622+0000 7f74d959a740 1 --2- 172.21.15.47:0/2290556907 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5616ba854be0 0x5616ba85fb40 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:48.628 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.622+0000 7f74af7fe640 0 client.15945 ms_handle_reset on v2:172.21.15.98:6838/2841287917 2024-04-17T06:07:48.628 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.622+0000 7f74d959a740 1 -- 172.21.15.47:0/2290556907 >> 172.21.15.47:0/2290556907 conn(0x5616ba84f9e0 msgr2=0x5616ba851730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:07:48.628 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.622+0000 7f74d959a740 1 -- 172.21.15.47:0/2290556907 shutdown_connections 2024-04-17T06:07:48.628 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.622+0000 7f74d959a740 1 -- 172.21.15.47:0/2290556907 wait complete. 2024-04-17T06:07:48.628 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.622+0000 7f74d959a740 20 client.15945 trim_cache size 0 max 16384 2024-04-17T06:07:48.638 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.637+0000 7f74d959a740 1 Processor -- start 2024-04-17T06:07:48.638 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.637+0000 7f74d959a740 1 -- start start 2024-04-17T06:07:48.638 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.638+0000 7f74d959a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5616ba854be0 0x5616ba862270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:07:48.638 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.638+0000 7f74d959a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5616ba855500 0x5616ba86e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:07:48.638 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.638+0000 7f74d959a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5616ba862bc0 0x5616ba85eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:07:48.638 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.638+0000 7f74d959a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f74d01ae1f0 con 0x5616ba862bc0 2024-04-17T06:07:48.638 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.638+0000 7f74d959a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f74d01abab0 con 0x5616ba855500 2024-04-17T06:07:48.639 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.638+0000 7f74d959a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f74d01a9920 con 0x5616ba854be0 2024-04-17T06:07:48.639 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.638+0000 7f74d4f45640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5616ba862bc0 0x5616ba85eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:07:48.639 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.638+0000 7f74d4f45640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5616ba862bc0 0x5616ba85eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60774/0 (socket says 172.21.15.47:60774) 2024-04-17T06:07:48.639 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.638+0000 7f74d4f45640 1 -- 172.21.15.47:0/2499113788 learned_addr learned my addr 172.21.15.47:0/2499113788 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:07:48.639 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.638+0000 7f74c7fff640 1 --2- 172.21.15.47:0/2499113788 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5616ba855500 0x5616ba86e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:07:48.639 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.638+0000 7f74cffff640 1 --2- 172.21.15.47:0/2499113788 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5616ba854be0 0x5616ba862270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:07:48.639 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.638+0000 7f74d4f45640 1 -- 172.21.15.47:0/2499113788 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5616ba854be0 msgr2=0x5616ba862270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:07:48.639 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.638+0000 7f74d4f45640 1 --2- 172.21.15.47:0/2499113788 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5616ba854be0 0x5616ba862270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:48.639 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.638+0000 7f74d4f45640 1 -- 172.21.15.47:0/2499113788 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5616ba855500 msgr2=0x5616ba86e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:07:48.639 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.638+0000 7f74d4f45640 1 --2- 172.21.15.47:0/2499113788 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5616ba855500 0x5616ba86e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:48.639 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.638+0000 7f74d4f45640 1 -- 172.21.15.47:0/2499113788 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616ba855330 con 0x5616ba862bc0 2024-04-17T06:07:48.639 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.638+0000 7f74d4f45640 1 --2- 172.21.15.47:0/2499113788 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5616ba862bc0 0x5616ba85eca0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f74c8009ca0 tx=0x7f74c8040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40f5d768944c06f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:07:48.639 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.638+0000 7f74c5ffb640 1 -- 172.21.15.47:0/2499113788 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c8044040 con 0x5616ba862bc0 2024-04-17T06:07:48.639 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.639+0000 7f74c5ffb640 1 -- 172.21.15.47:0/2499113788 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74c8003e50 con 0x5616ba862bc0 2024-04-17T06:07:48.639 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.639+0000 7f74c5ffb640 1 -- 172.21.15.47:0/2499113788 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c8039040 con 0x5616ba862bc0 2024-04-17T06:07:48.640 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.639+0000 7f74d959a740 1 -- 172.21.15.47:0/2499113788 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5616ba862bc0 msgr2=0x5616ba85eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:07:48.640 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.639+0000 7f74d959a740 1 --2- 172.21.15.47:0/2499113788 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5616ba862bc0 0x5616ba85eca0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f74c8009ca0 tx=0x7f74c8040000 comp rx=0 tx=0).stop 2024-04-17T06:07:48.640 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.639+0000 7f74d959a740 1 -- 172.21.15.47:0/2499113788 shutdown_connections 2024-04-17T06:07:48.640 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.639+0000 7f74d959a740 1 --2- 172.21.15.47:0/2499113788 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5616ba862bc0 0x5616ba85eca0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:48.640 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.639+0000 7f74d959a740 1 --2- 172.21.15.47:0/2499113788 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5616ba855500 0x5616ba86e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:48.640 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.639+0000 7f74d959a740 1 --2- 172.21.15.47:0/2499113788 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5616ba854be0 0x5616ba862270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:48.640 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.639+0000 7f74d959a740 1 -- 172.21.15.47:0/2499113788 >> 172.21.15.47:0/2499113788 conn(0x5616ba84f9e0 msgr2=0x5616ba85fb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:07:48.640 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.639+0000 7f74d959a740 1 -- 172.21.15.47:0/2499113788 shutdown_connections 2024-04-17T06:07:48.640 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.639+0000 7f74d959a740 1 -- 172.21.15.47:0/2499113788 wait complete. 2024-04-17T06:07:48.640 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.640+0000 7f74d959a740 1 -- start start 2024-04-17T06:07:48.643 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.640+0000 7f74d959a740 1 Processor -- start 2024-04-17T06:07:48.643 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.640+0000 7f74d959a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5616ba854be0 0x5616ba86ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:07:48.643 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.640+0000 7f74d959a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5616ba855500 0x5616ba85c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:07:48.643 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.640+0000 7f74d959a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5616ba862bc0 0x5616ba85c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:07:48.643 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.640+0000 7f74d959a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f74d01a9f70 con 0x5616ba855500 2024-04-17T06:07:48.643 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.640+0000 7f74d959a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f74d01ab3e0 con 0x5616ba854be0 2024-04-17T06:07:48.644 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.640+0000 7f74d959a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f74d01aa640 con 0x5616ba862bc0 2024-04-17T06:07:48.644 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.640+0000 7f74d959a740 10 client.0 _ll_register_callbacks cb 0x7f74c000bbc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:07:48.644 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.640+0000 7f74c7fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5616ba855500 0x5616ba85c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:07:48.644 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.640+0000 7f74c7fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5616ba855500 0x5616ba85c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60776/0 (socket says 172.21.15.47:60776) 2024-04-17T06:07:48.644 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.640+0000 7f74c7fff640 1 -- 172.21.15.47:0/757374999 learned_addr learned my addr 172.21.15.47:0/757374999 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:07:48.644 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.640+0000 7f74cffff640 1 --2- 172.21.15.47:0/757374999 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5616ba854be0 0x5616ba86ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:07:48.644 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.640+0000 7f74d4f45640 1 --2- 172.21.15.47:0/757374999 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5616ba862bc0 0x5616ba85c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:07:48.644 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.641+0000 7f74c7fff640 1 -- 172.21.15.47:0/757374999 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5616ba862bc0 msgr2=0x5616ba85c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:07:48.644 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.641+0000 7f74c7fff640 1 --2- 172.21.15.47:0/757374999 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5616ba862bc0 0x5616ba85c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:48.645 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.641+0000 7f74c7fff640 1 -- 172.21.15.47:0/757374999 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5616ba854be0 msgr2=0x5616ba86ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:07:48.645 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.641+0000 7f74c7fff640 1 --2- 172.21.15.47:0/757374999 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5616ba854be0 0x5616ba86ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:48.645 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.641+0000 7f74c7fff640 1 -- 172.21.15.47:0/757374999 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616ba85a990 con 0x5616ba855500 2024-04-17T06:07:48.645 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.641+0000 7f74c7fff640 1 --2- 172.21.15.47:0/757374999 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5616ba855500 0x5616ba85c3f0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f74b8010fd0 tx=0x7f74b8098500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91a355162dc33b4a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:07:48.645 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.641+0000 7f74af7fe640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-17T06:07:48.645 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.641+0000 7f74af7fe640 1 -- 172.21.15.47:0/757374999 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74b8021040 con 0x5616ba855500 2024-04-17T06:07:48.645 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.641+0000 7f74af7fe640 1 -- 172.21.15.47:0/757374999 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f74b80cdbe0 con 0x5616ba855500 2024-04-17T06:07:48.645 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.641+0000 7f74d959a740 1 -- 172.21.15.47:0/757374999 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74c8003e50 con 0x5616ba855500 2024-04-17T06:07:48.645 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.641+0000 7f74af7fe640 1 -- 172.21.15.47:0/757374999 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f74b80cdde0 con 0x5616ba855500 2024-04-17T06:07:48.645 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.641+0000 7f74d959a740 10 client.15951 fetch_fsmap learned FSMap version 30 2024-04-17T06:07:48.645 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.641+0000 7f74d959a740 1 -- 172.21.15.47:0/757374999 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5616ba8d8f00 con 0x5616ba855500 2024-04-17T06:07:48.645 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.641+0000 7f74af7fe640 1 -- 172.21.15.47:0/757374999 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74c8003e50 con 0x5616ba855500 2024-04-17T06:07:48.646 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.641+0000 7f74af7fe640 1 -- 172.21.15.47:0/757374999 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 30) ==== 8339+0+0 (secure 0 0 0) 0x7f74b809b690 con 0x5616ba855500 2024-04-17T06:07:48.646 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.642+0000 7f74d959a740 10 client.15951 fetch_fsmap finished waiting for FSMap version 30 2024-04-17T06:07:48.646 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.642+0000 7f74d959a740 10 client.15951 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-17T06:07:48.646 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.642+0000 7f74d959a740 20 client.15951 populate_metadata read hostname 'smithi047' 2024-04-17T06:07:48.646 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.642+0000 7f74d959a740 1 --2- 172.21.15.47:0/757374999 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x5616ba8d95e0 0x5616ba8dba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:07:48.646 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.642+0000 7f74d959a740 4 client.15951 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-17T06:07:48.646 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.642+0000 7f74d959a740 1 -- 172.21.15.47:0/757374999 --> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5616ba7c2060 con 0x5616ba8d95e0 2024-04-17T06:07:48.646 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.642+0000 7f74cffff640 1 --2- 172.21.15.47:0/757374999 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x5616ba8d95e0 0x5616ba8dba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:07:48.646 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.642+0000 7f74cffff640 1 --2- 172.21.15.47:0/757374999 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x5616ba8d95e0 0x5616ba8dba00 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fbfee812c874a31d server_cookie=3a47ace986343713 in_seq=0 out_seq=0 2024-04-17T06:07:48.646 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.642+0000 7f74af7fe640 10 client.15951 ms_handle_connect on v2:172.21.15.98:6838/2841287917 2024-04-17T06:07:48.650 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.650+0000 7f74af7fe640 1 -- 172.21.15.47:0/757374999 <== mds.0 v2:172.21.15.98:6838/2841287917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5616ba7c2060 con 0x5616ba8d95e0 2024-04-17T06:07:48.650 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.650+0000 7f74af7fe640 10 client.15951 handle_command_reply: tid=0 2024-04-17T06:07:48.650 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.650+0000 7f74d959a740 1 client.15951 shutdown 2024-04-17T06:07:48.650 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.651+0000 7f74d959a740 1 -- 172.21.15.47:0/757374999 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5616ba855500 msgr2=0x5616ba85c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:07:48.650 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.651+0000 7f74d959a740 1 --2- 172.21.15.47:0/757374999 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5616ba855500 0x5616ba85c3f0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f74b8010fd0 tx=0x7f74b8098500 comp rx=0 tx=0).stop 2024-04-17T06:07:48.650 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.651+0000 7f74d959a740 1 -- 172.21.15.47:0/757374999 shutdown_connections 2024-04-17T06:07:48.651 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.651+0000 7f74d959a740 1 --2- 172.21.15.47:0/757374999 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x5616ba8d95e0 0x5616ba8dba00 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:48.651 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.651+0000 7f74d959a740 1 --2- 172.21.15.47:0/757374999 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5616ba862bc0 0x5616ba85c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:48.651 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.651+0000 7f74d959a740 1 --2- 172.21.15.47:0/757374999 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5616ba855500 0x5616ba85c3f0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:48.651 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.651+0000 7f74af7fe640 0 client.15951 ms_handle_reset on v2:172.21.15.98:6838/2841287917 2024-04-17T06:07:48.651 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.651+0000 7f74d959a740 1 --2- 172.21.15.47:0/757374999 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5616ba854be0 0x5616ba86ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:48.651 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.651+0000 7f74d959a740 1 -- 172.21.15.47:0/757374999 >> 172.21.15.47:0/757374999 conn(0x5616ba84f9e0 msgr2=0x5616ba84e270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:07:48.651 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.651+0000 7f74d959a740 1 -- 172.21.15.47:0/757374999 shutdown_connections 2024-04-17T06:07:48.651 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.651+0000 7f74d959a740 1 -- 172.21.15.47:0/757374999 wait complete. 2024-04-17T06:07:48.651 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.651+0000 7f74d959a740 20 client.15951 trim_cache size 0 max 16384 2024-04-17T06:07:48.651 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-04-17T06:07:48.652 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-04-17T06:07:48.652 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-04-17T06:07:48.652 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-04-17T06:07:48.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.651+0000 7f74d69a8640 1 -- 172.21.15.47:0/2425337349 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f74bc0a3e40 msgr2=0x7f74bc0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:07:48.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.651+0000 7f74d69a8640 1 --2- 172.21.15.47:0/2425337349 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f74bc0a3e40 0x7f74bc0a6300 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f74c8004af0 tx=0x7f74c802e040 comp rx=0 tx=0).stop 2024-04-17T06:07:48.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.651+0000 7f74d69a8640 1 -- 172.21.15.47:0/2425337349 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f74d0106180 msgr2=0x7f74d0199aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:07:48.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.651+0000 7f74d69a8640 1 --2- 172.21.15.47:0/2425337349 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f74d0106180 0x7f74d0199aa0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f74b800d770 tx=0x7f74b800dc30 comp rx=0 tx=0).stop 2024-04-17T06:07:48.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.651+0000 7f74d69a8640 1 -- 172.21.15.47:0/2425337349 shutdown_connections 2024-04-17T06:07:48.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.651+0000 7f74d69a8640 1 --2- 172.21.15.47:0/2425337349 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f74d019a080 0x7f74d01a45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:48.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.651+0000 7f74d69a8640 1 --2- 172.21.15.47:0/2425337349 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f74bc0a3e40 0x7f74bc0a6300 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:48.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.651+0000 7f74d69a8640 1 --2- 172.21.15.47:0/2425337349 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f74d0106180 0x7f74d0199aa0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:48.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.651+0000 7f74d69a8640 1 --2- 172.21.15.47:0/2425337349 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f74d0101100 0x7f74d0199560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:07:48.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.651+0000 7f74d69a8640 1 -- 172.21.15.47:0/2425337349 >> 172.21.15.47:0/2425337349 conn(0x7f74d0077fe0 msgr2=0x7f74d00fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:07:48.654 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.652+0000 7f74d69a8640 1 -- 172.21.15.47:0/2425337349 shutdown_connections 2024-04-17T06:07:48.654 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:07:48.652+0000 7f74d69a8640 1 -- 172.21.15.47:0/2425337349 wait complete. 2024-04-17T06:07:50.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:07:49 smithi098 ceph-mon[36516]: pgmap v200: 97 pgs: 97 active+clean; 3.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:07:50.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:07:49 smithi047 ceph-mon[32223]: pgmap v200: 97 pgs: 97 active+clean; 3.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:07:50.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:07:49 smithi103 ceph-mon[36243]: pgmap v200: 97 pgs: 97 active+clean; 3.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:07:52.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:07:51 smithi098 ceph-mon[36516]: pgmap v201: 97 pgs: 97 active+clean; 3.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 19 KiB/s wr, 18 op/s 2024-04-17T06:07:52.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:07:51 smithi047 ceph-mon[32223]: pgmap v201: 97 pgs: 97 active+clean; 3.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 19 KiB/s wr, 18 op/s 2024-04-17T06:07:52.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:07:51 smithi103 ceph-mon[36243]: pgmap v201: 97 pgs: 97 active+clean; 3.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 19 KiB/s wr, 18 op/s 2024-04-17T06:07:54.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:07:53 smithi098 ceph-mon[36516]: pgmap v202: 97 pgs: 97 active+clean; 3.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:07:54.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:07:53 smithi047 ceph-mon[32223]: pgmap v202: 97 pgs: 97 active+clean; 3.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:07:54.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:07:53 smithi103 ceph-mon[36243]: pgmap v202: 97 pgs: 97 active+clean; 3.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:07:56.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:07:55 smithi098 ceph-mon[36516]: pgmap v203: 97 pgs: 97 active+clean; 3.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-04-17T06:07:56.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:07:55 smithi047 ceph-mon[32223]: pgmap v203: 97 pgs: 97 active+clean; 3.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-04-17T06:07:56.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:07:55 smithi103 ceph-mon[36243]: pgmap v203: 97 pgs: 97 active+clean; 3.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-04-17T06:07:58.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:07:57 smithi098 ceph-mon[36516]: pgmap v204: 97 pgs: 97 active+clean; 3.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:07:58.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:07:57 smithi047 ceph-mon[32223]: pgmap v204: 97 pgs: 97 active+clean; 3.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:07:58.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:07:57 smithi103 ceph-mon[36243]: pgmap v204: 97 pgs: 97 active+clean; 3.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:08:00.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:07:59 smithi098 ceph-mon[36516]: pgmap v205: 97 pgs: 97 active+clean; 3.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:08:00.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:07:59 smithi047 ceph-mon[32223]: pgmap v205: 97 pgs: 97 active+clean; 3.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:08:00.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:07:59 smithi103 ceph-mon[36243]: pgmap v205: 97 pgs: 97 active+clean; 3.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:08:01.752 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.752+0000 7fb197a7b640 1 -- 172.21.15.47:0/1753492954 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1901021e0 msgr2=0x7fb1901025c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:08:01.752 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.752+0000 7fb197a7b640 1 --2- 172.21.15.47:0/1753492954 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1901021e0 0x7fb1901025c0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fb18c009900 tx=0x7fb18c02bc30 comp rx=0 tx=0).stop 2024-04-17T06:08:01.752 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.752+0000 7fb197a7b640 1 -- 172.21.15.47:0/1753492954 shutdown_connections 2024-04-17T06:08:01.752 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.752+0000 7fb197a7b640 1 --2- 172.21.15.47:0/1753492954 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb1901072b0 0x7fb19010ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:01.752 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.752+0000 7fb197a7b640 1 --2- 172.21.15.47:0/1753492954 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb190102b90 0x7fb190106bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:01.752 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.752+0000 7fb197a7b640 1 --2- 172.21.15.47:0/1753492954 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1901021e0 0x7fb1901025c0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:01.752 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.752+0000 7fb197a7b640 1 -- 172.21.15.47:0/1753492954 >> 172.21.15.47:0/1753492954 conn(0x7fb1900793f0 msgr2=0x7fb190079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:08:01.752 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.752+0000 7fb197a7b640 1 -- 172.21.15.47:0/1753492954 shutdown_connections 2024-04-17T06:08:01.752 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.752+0000 7fb197a7b640 1 -- 172.21.15.47:0/1753492954 wait complete. 2024-04-17T06:08:01.753 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.752+0000 7fb197a7b640 1 Processor -- start 2024-04-17T06:08:01.753 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.752+0000 7fb197a7b640 1 -- start start 2024-04-17T06:08:01.753 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.753+0000 7fb197a7b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb1901021e0 0x7fb190198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:01.753 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.753+0000 7fb197a7b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb1901072b0 0x7fb190198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:01.753 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.753+0000 7fb197a7b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb19019d000 0x7fb19019d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:01.753 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.753+0000 7fb197a7b640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb190112360 con 0x7fb19019d000 2024-04-17T06:08:01.753 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.753+0000 7fb197a7b640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb1901121e0 con 0x7fb1901021e0 2024-04-17T06:08:01.753 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.753+0000 7fb197a7b640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb1901124e0 con 0x7fb1901072b0 2024-04-17T06:08:01.753 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.753+0000 7fb196018640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb19019d000 0x7fb19019d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:01.753 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.753+0000 7fb196018640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb19019d000 0x7fb19019d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47784/0 (socket says 172.21.15.47:47784) 2024-04-17T06:08:01.753 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.753+0000 7fb196018640 1 -- 172.21.15.47:0/4214823065 learned_addr learned my addr 172.21.15.47:0/4214823065 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:08:01.753 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.753+0000 7fb195016640 1 --2- 172.21.15.47:0/4214823065 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb1901072b0 0x7fb190198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:01.753 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.753+0000 7fb195817640 1 --2- 172.21.15.47:0/4214823065 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb1901021e0 0x7fb190198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:01.754 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.753+0000 7fb196018640 1 -- 172.21.15.47:0/4214823065 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb1901072b0 msgr2=0x7fb190198c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:08:01.754 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.753+0000 7fb196018640 1 --2- 172.21.15.47:0/4214823065 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb1901072b0 0x7fb190198c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:01.754 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.753+0000 7fb196018640 1 -- 172.21.15.47:0/4214823065 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb1901021e0 msgr2=0x7fb190198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:08:01.754 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.753+0000 7fb196018640 1 --2- 172.21.15.47:0/4214823065 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb1901021e0 0x7fb190198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:01.754 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.753+0000 7fb196018640 1 -- 172.21.15.47:0/4214823065 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb19019daf0 con 0x7fb19019d000 2024-04-17T06:08:01.754 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.753+0000 7fb195817640 1 --2- 172.21.15.47:0/4214823065 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb1901021e0 0x7fb190198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:08:01.754 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.753+0000 7fb196018640 1 --2- 172.21.15.47:0/4214823065 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb19019d000 0x7fb19019d4b0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fb18800cab0 tx=0x7fb18800cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:08:01.754 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.753+0000 7fb186ffd640 1 -- 172.21.15.47:0/4214823065 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb18800ae20 con 0x7fb19019d000 2024-04-17T06:08:01.754 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.754+0000 7fb186ffd640 1 -- 172.21.15.47:0/4214823065 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb18800f930 con 0x7fb19019d000 2024-04-17T06:08:01.754 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.754+0000 7fb197a7b640 1 -- 172.21.15.47:0/4214823065 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb19019dde0 con 0x7fb19019d000 2024-04-17T06:08:01.754 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.754+0000 7fb186ffd640 1 -- 172.21.15.47:0/4214823065 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb188018b00 con 0x7fb19019d000 2024-04-17T06:08:01.754 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.754+0000 7fb197a7b640 1 -- 172.21.15.47:0/4214823065 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1901a5580 con 0x7fb19019d000 2024-04-17T06:08:01.757 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.757+0000 7fb186ffd640 1 -- 172.21.15.47:0/4214823065 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb18800fae0 con 0x7fb19019d000 2024-04-17T06:08:01.757 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.757+0000 7fb19a66e740 1 Processor -- start 2024-04-17T06:08:01.758 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.757+0000 7fb19a66e740 1 -- start start 2024-04-17T06:08:01.758 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.757+0000 7fb19a66e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5570424510c0 0x5570424514a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:01.758 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.757+0000 7fb19a66e740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5570424519e0 0x55704245a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:01.758 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.757+0000 7fb19a66e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55704245a7b0 0x55704245cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:01.758 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.757+0000 7fb19a66e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5570423b67d0 con 0x5570424510c0 2024-04-17T06:08:01.758 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.757+0000 7fb19a66e740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x557042353f70 con 0x5570424519e0 2024-04-17T06:08:01.758 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.757+0000 7fb19a66e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5570423b2790 con 0x55704245a7b0 2024-04-17T06:08:01.758 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.758+0000 7fb195016640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5570424510c0 0x5570424514a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:01.758 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.758+0000 7fb195016640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5570424510c0 0x5570424514a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47788/0 (socket says 172.21.15.47:47788) 2024-04-17T06:08:01.758 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.758+0000 7fb195016640 1 -- 172.21.15.47:0/3685468179 learned_addr learned my addr 172.21.15.47:0/3685468179 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:08:01.758 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.758+0000 7fb195817640 1 --2- 172.21.15.47:0/3685468179 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5570424519e0 0x55704245a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:01.758 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.758+0000 7fb186ffd640 1 --2- 172.21.15.47:0/4214823065 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fb17c0a3dc0 0x7fb17c0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:01.758 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.758+0000 7fb195016640 1 --2- 172.21.15.47:0/3685468179 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55704245a7b0 0x55704245cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:01.759 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.758+0000 7fb195016640 1 -- 172.21.15.47:0/3685468179 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55704245a7b0 msgr2=0x55704245cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:08:01.759 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.758+0000 7fb195016640 1 --2- 172.21.15.47:0/3685468179 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55704245a7b0 0x55704245cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:01.759 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.758+0000 7fb196018640 1 --2- 172.21.15.47:0/4214823065 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fb17c0a3dc0 0x7fb17c0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:01.759 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.758+0000 7fb195016640 1 -- 172.21.15.47:0/3685468179 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5570424519e0 msgr2=0x55704245a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:08:01.759 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.758+0000 7fb195016640 1 --2- 172.21.15.47:0/3685468179 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5570424519e0 0x55704245a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:01.759 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.758+0000 7fb195016640 1 -- 172.21.15.47:0/3685468179 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570420c9210 con 0x5570424510c0 2024-04-17T06:08:01.759 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.758+0000 7fb186ffd640 1 -- 172.21.15.47:0/4214823065 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7fb1880d0e60 con 0x7fb19019d000 2024-04-17T06:08:01.759 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.758+0000 7fb195016640 1 --2- 172.21.15.47:0/3685468179 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5570424510c0 0x5570424514a0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fb18000bb40 tx=0x7fb180004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1aed2ccfa67dc166 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:08:01.759 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.758+0000 7fb17bfff640 1 -- 172.21.15.47:0/3685468179 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb180005020 con 0x5570424510c0 2024-04-17T06:08:01.759 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.759+0000 7fb17bfff640 1 -- 172.21.15.47:0/3685468179 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1800056a0 con 0x5570424510c0 2024-04-17T06:08:01.759 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.759+0000 7fb17bfff640 1 -- 172.21.15.47:0/3685468179 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb180010660 con 0x5570424510c0 2024-04-17T06:08:01.759 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.759+0000 7fb19a66e740 1 -- 172.21.15.47:0/3685468179 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5570424510c0 msgr2=0x5570424514a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:08:01.759 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.759+0000 7fb196018640 1 --2- 172.21.15.47:0/4214823065 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fb17c0a3dc0 0x7fb17c0a6280 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb1880ce690 tx=0x7fb1880922c0 comp rx=0 tx=0).ready entity=mgr.24923 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:08:01.759 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.759+0000 7fb19a66e740 1 --2- 172.21.15.47:0/3685468179 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5570424510c0 0x5570424514a0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fb18000bb40 tx=0x7fb180004610 comp rx=0 tx=0).stop 2024-04-17T06:08:01.760 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.759+0000 7fb19a66e740 1 -- 172.21.15.47:0/3685468179 shutdown_connections 2024-04-17T06:08:01.760 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.759+0000 7fb19a66e740 1 --2- 172.21.15.47:0/3685468179 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55704245a7b0 0x55704245cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:01.760 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.759+0000 7fb19a66e740 1 --2- 172.21.15.47:0/3685468179 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5570424519e0 0x55704245a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:01.760 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.759+0000 7fb19a66e740 1 --2- 172.21.15.47:0/3685468179 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5570424510c0 0x5570424514a0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:01.760 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.759+0000 7fb19a66e740 1 -- 172.21.15.47:0/3685468179 >> 172.21.15.47:0/3685468179 conn(0x55704244b9e0 msgr2=0x55704244bdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:08:01.760 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.759+0000 7fb19a66e740 1 -- 172.21.15.47:0/3685468179 shutdown_connections 2024-04-17T06:08:01.760 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.759+0000 7fb19a66e740 1 -- 172.21.15.47:0/3685468179 wait complete. 2024-04-17T06:08:01.760 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.760+0000 7fb19a66e740 1 -- start start 2024-04-17T06:08:01.760 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.760+0000 7fb19a66e740 1 Processor -- start 2024-04-17T06:08:01.764 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.760+0000 7fb19a66e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5570424510c0 0x557042457870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:01.764 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.760+0000 7fb19a66e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5570424519e0 0x557042457db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:01.764 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.760+0000 7fb19a66e740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55704245a7b0 0x5570424582f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:01.764 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.760+0000 7fb19a66e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55704245d380 con 0x5570424510c0 2024-04-17T06:08:01.764 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.760+0000 7fb19a66e740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55704233ee90 con 0x55704245a7b0 2024-04-17T06:08:01.764 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.760+0000 7fb19a66e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55704245d500 con 0x5570424519e0 2024-04-17T06:08:01.764 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.760+0000 7fb19a66e740 10 client.0 _ll_register_callbacks cb 0x557042354d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:08:01.764 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.760+0000 7fb195016640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5570424510c0 0x557042457870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:01.765 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.760+0000 7fb195016640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5570424510c0 0x557042457870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47802/0 (socket says 172.21.15.47:47802) 2024-04-17T06:08:01.765 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.760+0000 7fb195016640 1 -- 172.21.15.47:0/2838171835 learned_addr learned my addr 172.21.15.47:0/2838171835 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:08:01.765 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.761+0000 7fb195817640 1 --2- 172.21.15.47:0/2838171835 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5570424519e0 0x557042457db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:01.765 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.761+0000 7fb195016640 1 --2- 172.21.15.47:0/2838171835 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55704245a7b0 0x5570424582f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:01.765 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.761+0000 7fb195016640 1 -- 172.21.15.47:0/2838171835 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5570424519e0 msgr2=0x557042457db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:08:01.765 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.761+0000 7fb195016640 1 --2- 172.21.15.47:0/2838171835 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5570424519e0 0x557042457db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:01.765 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.761+0000 7fb195016640 1 -- 172.21.15.47:0/2838171835 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55704245a7b0 msgr2=0x5570424582f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:08:01.765 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.761+0000 7fb195016640 1 --2- 172.21.15.47:0/2838171835 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55704245a7b0 0x5570424582f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:01.765 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.761+0000 7fb195016640 1 -- 172.21.15.47:0/2838171835 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557042195620 con 0x5570424510c0 2024-04-17T06:08:01.765 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.761+0000 7fb195016640 1 --2- 172.21.15.47:0/2838171835 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5570424510c0 0x557042457870 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x557042446af0 tx=0x7fb180005f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1afd3e77642b927 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:08:01.765 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.761+0000 7fb1797fa640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-17T06:08:01.765 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.761+0000 7fb1797fa640 1 -- 172.21.15.47:0/2838171835 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1800051e0 con 0x5570424510c0 2024-04-17T06:08:01.765 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.761+0000 7fb1797fa640 1 -- 172.21.15.47:0/2838171835 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7fb180005380 con 0x5570424510c0 2024-04-17T06:08:01.765 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.761+0000 7fb19a66e740 1 -- 172.21.15.47:0/2838171835 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1800056a0 con 0x5570424510c0 2024-04-17T06:08:01.766 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.761+0000 7fb1797fa640 1 -- 172.21.15.47:0/2838171835 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb18002f050 con 0x5570424510c0 2024-04-17T06:08:01.766 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.761+0000 7fb19a66e740 10 client.15963 fetch_fsmap learned FSMap version 30 2024-04-17T06:08:01.766 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.761+0000 7fb19a66e740 1 -- 172.21.15.47:0/2838171835 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557042469670 con 0x5570424510c0 2024-04-17T06:08:01.766 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.762+0000 7fb1797fa640 1 -- 172.21.15.47:0/2838171835 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1800056a0 con 0x5570424510c0 2024-04-17T06:08:01.766 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.762+0000 7fb1797fa640 1 -- 172.21.15.47:0/2838171835 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 30) ==== 8339+0+0 (secure 0 0 0) 0x7fb18001f3e0 con 0x5570424510c0 2024-04-17T06:08:01.766 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.762+0000 7fb19a66e740 10 client.15963 fetch_fsmap finished waiting for FSMap version 30 2024-04-17T06:08:01.766 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.762+0000 7fb19a66e740 10 client.15963 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-17T06:08:01.766 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.762+0000 7fb19a66e740 20 client.15963 populate_metadata read hostname 'smithi047' 2024-04-17T06:08:01.766 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.762+0000 7fb19a66e740 1 --2- 172.21.15.47:0/2838171835 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55704246aad0 0x55704246cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:01.766 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.762+0000 7fb19a66e740 4 client.15963 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T06:08:01.766 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.762+0000 7fb19a66e740 1 -- 172.21.15.47:0/2838171835 --> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5570423c41f0 con 0x55704246aad0 2024-04-17T06:08:01.766 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.762+0000 7fb195817640 1 --2- 172.21.15.47:0/2838171835 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55704246aad0 0x55704246cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:01.766 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.763+0000 7fb195817640 1 --2- 172.21.15.47:0/2838171835 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55704246aad0 0x55704246cec0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb977e805f89b3f8 server_cookie=469a03a0b8617ce9 in_seq=0 out_seq=0 2024-04-17T06:08:01.766 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.763+0000 7fb1797fa640 10 client.15963 ms_handle_connect on v2:172.21.15.98:6838/2841287917 2024-04-17T06:08:01.767 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.763+0000 7fb1797fa640 1 -- 172.21.15.47:0/2838171835 <== mds.0 v2:172.21.15.98:6838/2841287917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5570423c41f0 con 0x55704246aad0 2024-04-17T06:08:01.767 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.763+0000 7fb1797fa640 10 client.15963 handle_command_reply: tid=0 2024-04-17T06:08:01.767 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.763+0000 7fb19a66e740 1 client.15963 shutdown 2024-04-17T06:08:01.767 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.763+0000 7fb19a66e740 1 -- 172.21.15.47:0/2838171835 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5570424510c0 msgr2=0x557042457870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:08:01.767 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.763+0000 7fb19a66e740 1 --2- 172.21.15.47:0/2838171835 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5570424510c0 0x557042457870 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x557042446af0 tx=0x7fb180005f70 comp rx=0 tx=0).stop 2024-04-17T06:08:01.767 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.763+0000 7fb19a66e740 1 -- 172.21.15.47:0/2838171835 shutdown_connections 2024-04-17T06:08:01.767 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.763+0000 7fb19a66e740 1 --2- 172.21.15.47:0/2838171835 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55704246aad0 0x55704246cec0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:01.767 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.764+0000 7fb19a66e740 1 --2- 172.21.15.47:0/2838171835 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55704245a7b0 0x5570424582f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:01.767 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.764+0000 7fb1797fa640 0 client.15963 ms_handle_reset on v2:172.21.15.98:6838/2841287917 2024-04-17T06:08:01.767 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.764+0000 7fb19a66e740 1 --2- 172.21.15.47:0/2838171835 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5570424519e0 0x557042457db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:01.767 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.764+0000 7fb19a66e740 1 --2- 172.21.15.47:0/2838171835 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5570424510c0 0x557042457870 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:01.767 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.764+0000 7fb19a66e740 1 -- 172.21.15.47:0/2838171835 >> 172.21.15.47:0/2838171835 conn(0x55704244b9e0 msgr2=0x55704244d730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:08:01.767 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.764+0000 7fb19a66e740 1 -- 172.21.15.47:0/2838171835 shutdown_connections 2024-04-17T06:08:01.768 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.764+0000 7fb19a66e740 1 -- 172.21.15.47:0/2838171835 wait complete. 2024-04-17T06:08:01.768 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.764+0000 7fb19a66e740 20 client.15963 trim_cache size 0 max 16384 2024-04-17T06:08:01.781 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.779+0000 7fb19a66e740 1 Processor -- start 2024-04-17T06:08:01.781 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.779+0000 7fb19a66e740 1 -- start start 2024-04-17T06:08:01.781 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.779+0000 7fb19a66e740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5570424510c0 0x557042459fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:01.781 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.779+0000 7fb19a66e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5570424519e0 0x557042466240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:01.781 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.779+0000 7fb19a66e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55704245a7b0 0x55704244a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:01.781 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.779+0000 7fb19a66e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb1901b03c0 con 0x5570424519e0 2024-04-17T06:08:01.782 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.779+0000 7fb19a66e740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb1901adc80 con 0x5570424510c0 2024-04-17T06:08:01.782 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.779+0000 7fb19a66e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb1901abad0 con 0x55704245a7b0 2024-04-17T06:08:01.782 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.779+0000 7fb195817640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5570424519e0 0x557042466240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:01.782 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.779+0000 7fb195817640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5570424519e0 0x557042466240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47816/0 (socket says 172.21.15.47:47816) 2024-04-17T06:08:01.782 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.779+0000 7fb195817640 1 -- 172.21.15.47:0/2735873687 learned_addr learned my addr 172.21.15.47:0/2735873687 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:08:01.782 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.779+0000 7fb195817640 1 -- 172.21.15.47:0/2735873687 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55704245a7b0 msgr2=0x55704244a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:08:01.782 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.779+0000 7fb195817640 1 --2- 172.21.15.47:0/2735873687 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55704245a7b0 0x55704244a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:01.782 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.779+0000 7fb195817640 1 -- 172.21.15.47:0/2735873687 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5570424510c0 msgr2=0x557042459fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:08:01.782 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.779+0000 7fb195817640 1 --2- 172.21.15.47:0/2735873687 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5570424510c0 0x557042459fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:01.782 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.779+0000 7fb195817640 1 -- 172.21.15.47:0/2735873687 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570424472a0 con 0x5570424519e0 2024-04-17T06:08:01.782 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.780+0000 7fb195817640 1 --2- 172.21.15.47:0/2735873687 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5570424519e0 0x557042466240 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fb18c02cfd0 tx=0x7fb18c0058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcd920f2f2a3525 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:08:01.782 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.780+0000 7fb17b7fe640 1 -- 172.21.15.47:0/2735873687 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb18c02cd60 con 0x5570424519e0 2024-04-17T06:08:01.782 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.780+0000 7fb17b7fe640 1 -- 172.21.15.47:0/2735873687 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb18c04b070 con 0x5570424519e0 2024-04-17T06:08:01.782 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.780+0000 7fb17b7fe640 1 -- 172.21.15.47:0/2735873687 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb18c005460 con 0x5570424519e0 2024-04-17T06:08:01.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.780+0000 7fb19a66e740 1 -- 172.21.15.47:0/2735873687 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5570424519e0 msgr2=0x557042466240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:08:01.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.780+0000 7fb19a66e740 1 --2- 172.21.15.47:0/2735873687 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5570424519e0 0x557042466240 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fb18c02cfd0 tx=0x7fb18c0058b0 comp rx=0 tx=0).stop 2024-04-17T06:08:01.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.780+0000 7fb19a66e740 1 -- 172.21.15.47:0/2735873687 shutdown_connections 2024-04-17T06:08:01.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.780+0000 7fb19a66e740 1 --2- 172.21.15.47:0/2735873687 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55704245a7b0 0x55704244a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:01.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.780+0000 7fb19a66e740 1 --2- 172.21.15.47:0/2735873687 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5570424519e0 0x557042466240 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:01.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.780+0000 7fb19a66e740 1 --2- 172.21.15.47:0/2735873687 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5570424510c0 0x557042459fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:01.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.780+0000 7fb19a66e740 1 -- 172.21.15.47:0/2735873687 >> 172.21.15.47:0/2735873687 conn(0x55704244b9e0 msgr2=0x55704244a270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:08:01.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.780+0000 7fb19a66e740 1 -- 172.21.15.47:0/2735873687 shutdown_connections 2024-04-17T06:08:01.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.780+0000 7fb19a66e740 1 -- 172.21.15.47:0/2735873687 wait complete. 2024-04-17T06:08:01.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.781+0000 7fb19a66e740 1 -- start start 2024-04-17T06:08:01.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.781+0000 7fb19a66e740 1 Processor -- start 2024-04-17T06:08:01.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.781+0000 7fb19a66e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5570424510c0 0x5570424544a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:01.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.781+0000 7fb19a66e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5570424519e0 0x5570424549e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:01.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.781+0000 7fb19a66e740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55704245a7b0 0x557042454f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:01.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.781+0000 7fb19a66e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb1901ac140 con 0x5570424510c0 2024-04-17T06:08:01.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.781+0000 7fb19a66e740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb1901ad5b0 con 0x55704245a7b0 2024-04-17T06:08:01.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.781+0000 7fb19a66e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb1901ac810 con 0x5570424519e0 2024-04-17T06:08:01.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.781+0000 7fb19a66e740 10 client.0 _ll_register_callbacks cb 0x7fb18c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:08:01.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.781+0000 7fb195016640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5570424510c0 0x5570424544a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:01.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.781+0000 7fb195016640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5570424510c0 0x5570424544a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47824/0 (socket says 172.21.15.47:47824) 2024-04-17T06:08:01.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.781+0000 7fb195016640 1 -- 172.21.15.47:0/3138141987 learned_addr learned my addr 172.21.15.47:0/3138141987 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:08:01.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.781+0000 7fb195817640 1 --2- 172.21.15.47:0/3138141987 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5570424519e0 0x5570424549e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:01.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.782+0000 7fb195016640 1 -- 172.21.15.47:0/3138141987 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5570424519e0 msgr2=0x5570424549e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:08:01.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.782+0000 7fb195016640 1 --2- 172.21.15.47:0/3138141987 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5570424519e0 0x5570424549e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:01.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.782+0000 7fb195016640 1 -- 172.21.15.47:0/3138141987 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55704245a7b0 msgr2=0x557042454f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:08:01.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.782+0000 7fb195016640 1 --2- 172.21.15.47:0/3138141987 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55704245a7b0 0x557042454f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:01.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.782+0000 7fb195016640 1 -- 172.21.15.47:0/3138141987 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570424cfd90 con 0x5570424510c0 2024-04-17T06:08:01.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.782+0000 7fb195016640 1 --2- 172.21.15.47:0/3138141987 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5570424510c0 0x5570424544a0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fb18000b8d0 tx=0x7fb180005960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=959c3d51b28d14b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:08:01.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.782+0000 7fb1797fa640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-17T06:08:01.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.782+0000 7fb1797fa640 1 -- 172.21.15.47:0/3138141987 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb180003a80 con 0x5570424510c0 2024-04-17T06:08:01.785 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.782+0000 7fb1797fa640 1 -- 172.21.15.47:0/3138141987 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7fb18001f3e0 con 0x5570424510c0 2024-04-17T06:08:01.785 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.782+0000 7fb19a66e740 1 -- 172.21.15.47:0/3138141987 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb18c04b070 con 0x5570424510c0 2024-04-17T06:08:01.785 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.782+0000 7fb1797fa640 1 -- 172.21.15.47:0/3138141987 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb18000b390 con 0x5570424510c0 2024-04-17T06:08:01.785 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.782+0000 7fb19a66e740 10 client.15969 fetch_fsmap learned FSMap version 30 2024-04-17T06:08:01.785 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.782+0000 7fb19a66e740 1 -- 172.21.15.47:0/3138141987 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5570424d1000 con 0x5570424510c0 2024-04-17T06:08:01.785 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.782+0000 7fb1797fa640 1 -- 172.21.15.47:0/3138141987 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb18c04b070 con 0x5570424510c0 2024-04-17T06:08:01.785 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.782+0000 7fb1797fa640 1 -- 172.21.15.47:0/3138141987 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 30) ==== 8339+0+0 (secure 0 0 0) 0x7fb1800054c0 con 0x5570424510c0 2024-04-17T06:08:01.785 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.783+0000 7fb19a66e740 10 client.15969 fetch_fsmap finished waiting for FSMap version 30 2024-04-17T06:08:01.785 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.783+0000 7fb19a66e740 10 client.15969 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-17T06:08:01.785 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.783+0000 7fb19a66e740 20 client.15969 populate_metadata read hostname 'smithi047' 2024-04-17T06:08:01.785 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.783+0000 7fb19a66e740 1 --2- 172.21.15.47:0/3138141987 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x5570424d1660 0x5570424d3a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:01.785 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.783+0000 7fb19a66e740 4 client.15969 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-17T06:08:01.785 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.783+0000 7fb19a66e740 1 -- 172.21.15.47:0/3138141987 --> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5570423be060 con 0x5570424d1660 2024-04-17T06:08:01.785 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.783+0000 7fb195817640 1 --2- 172.21.15.47:0/3138141987 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x5570424d1660 0x5570424d3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:01.786 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.783+0000 7fb195817640 1 --2- 172.21.15.47:0/3138141987 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x5570424d1660 0x5570424d3a80 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=785e0446675f1397 server_cookie=518094fddb6a659f in_seq=0 out_seq=0 2024-04-17T06:08:01.786 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:01.783+0000 7fb1797fa640 10 client.15969 ms_handle_connect on v2:172.21.15.98:6838/2841287917 2024-04-17T06:08:02.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:08:01 smithi098 ceph-mon[36516]: pgmap v206: 97 pgs: 97 active+clean; 3.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-04-17T06:08:02.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:08:01 smithi047 ceph-mon[32223]: pgmap v206: 97 pgs: 97 active+clean; 3.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-04-17T06:08:02.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:08:01 smithi103 ceph-mon[36243]: pgmap v206: 97 pgs: 97 active+clean; 3.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-04-17T06:08:03.533 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:03.534+0000 7fb1797fa640 1 -- 172.21.15.47:0/3138141987 <== mds.0 v2:172.21.15.98:6838/2841287917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5570423be060 con 0x5570424d1660 2024-04-17T06:08:03.533 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:03.534+0000 7fb1797fa640 10 client.15969 handle_command_reply: tid=0 2024-04-17T06:08:03.533 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:03.534+0000 7fb19a66e740 1 client.15969 shutdown 2024-04-17T06:08:03.533 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:03.535+0000 7fb19a66e740 1 -- 172.21.15.47:0/3138141987 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5570424510c0 msgr2=0x5570424544a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:08:03.533 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:03.535+0000 7fb19a66e740 1 --2- 172.21.15.47:0/3138141987 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5570424510c0 0x5570424544a0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fb18000b8d0 tx=0x7fb180005960 comp rx=0 tx=0).stop 2024-04-17T06:08:03.533 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:03.535+0000 7fb19a66e740 1 -- 172.21.15.47:0/3138141987 shutdown_connections 2024-04-17T06:08:03.533 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:03.535+0000 7fb19a66e740 1 --2- 172.21.15.47:0/3138141987 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x5570424d1660 0x5570424d3a80 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:03.533 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:03.535+0000 7fb19a66e740 1 --2- 172.21.15.47:0/3138141987 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55704245a7b0 0x557042454f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:03.533 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:03.535+0000 7fb19a66e740 1 --2- 172.21.15.47:0/3138141987 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5570424519e0 0x5570424549e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:03.534 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:03.535+0000 7fb19a66e740 1 --2- 172.21.15.47:0/3138141987 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5570424510c0 0x5570424544a0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:03.534 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:03.535+0000 7fb1797fa640 0 client.15969 ms_handle_reset on v2:172.21.15.98:6838/2841287917 2024-04-17T06:08:03.534 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:03.535+0000 7fb19a66e740 1 -- 172.21.15.47:0/3138141987 >> 172.21.15.47:0/3138141987 conn(0x55704244b9e0 msgr2=0x55704244d550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:08:03.534 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:03.535+0000 7fb19a66e740 1 -- 172.21.15.47:0/3138141987 shutdown_connections 2024-04-17T06:08:03.534 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:03.535+0000 7fb19a66e740 1 -- 172.21.15.47:0/3138141987 wait complete. 2024-04-17T06:08:03.534 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:03.535+0000 7fb19a66e740 20 client.15969 trim_cache size 0 max 16384 2024-04-17T06:08:03.534 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-04-17T06:08:03.534 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-04-17T06:08:03.534 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-04-17T06:08:03.534 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-04-17T06:08:03.535 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:03.536+0000 7fb197a7b640 1 -- 172.21.15.47:0/4214823065 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fb17c0a3dc0 msgr2=0x7fb17c0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:08:03.535 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:03.536+0000 7fb197a7b640 1 --2- 172.21.15.47:0/4214823065 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fb17c0a3dc0 0x7fb17c0a6280 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb1880ce690 tx=0x7fb1880922c0 comp rx=0 tx=0).stop 2024-04-17T06:08:03.536 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:03.536+0000 7fb197a7b640 1 -- 172.21.15.47:0/4214823065 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb19019d000 msgr2=0x7fb19019d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:08:03.536 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:03.536+0000 7fb197a7b640 1 --2- 172.21.15.47:0/4214823065 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb19019d000 0x7fb19019d4b0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fb18800cab0 tx=0x7fb18800cf70 comp rx=0 tx=0).stop 2024-04-17T06:08:03.536 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:03.536+0000 7fb197a7b640 1 -- 172.21.15.47:0/4214823065 shutdown_connections 2024-04-17T06:08:03.536 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:03.536+0000 7fb197a7b640 1 --2- 172.21.15.47:0/4214823065 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fb17c0a3dc0 0x7fb17c0a6280 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:03.536 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:03.536+0000 7fb197a7b640 1 --2- 172.21.15.47:0/4214823065 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb19019d000 0x7fb19019d4b0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:03.536 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:03.536+0000 7fb197a7b640 1 --2- 172.21.15.47:0/4214823065 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb1901072b0 0x7fb190198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:03.536 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:03.536+0000 7fb197a7b640 1 --2- 172.21.15.47:0/4214823065 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb1901021e0 0x7fb190198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:03.536 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:03.536+0000 7fb197a7b640 1 -- 172.21.15.47:0/4214823065 >> 172.21.15.47:0/4214823065 conn(0x7fb1900793f0 msgr2=0x7fb1900ffc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:08:03.536 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:03.536+0000 7fb197a7b640 1 -- 172.21.15.47:0/4214823065 shutdown_connections 2024-04-17T06:08:03.536 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:03.536+0000 7fb197a7b640 1 -- 172.21.15.47:0/4214823065 wait complete. 2024-04-17T06:08:04.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:08:03 smithi098 ceph-mon[36516]: pgmap v207: 97 pgs: 97 active+clean; 3.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.3 KiB/s wr, 9 op/s 2024-04-17T06:08:04.133 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:08:03 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[66417]: *** Caught signal (Segmentation fault) ** 2024-04-17T06:08:04.133 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:08:03 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[66417]: in thread 7f13270bc700 thread_name:md_log_replay 2024-04-17T06:08:04.133 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:08:03 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[66417]: ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev) 2024-04-17T06:08:04.133 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:08:03 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[66417]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f133847dd20] 2024-04-17T06:08:04.133 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:08:03 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[66417]: 2: /usr/bin/ceph-mds(+0x54962f) [0x56289f0ce62f] 2024-04-17T06:08:04.134 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:08:03 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[66417]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x56289f0d1dc3] 2024-04-17T06:08:04.134 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:08:03 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[66417]: 4: (MDLog::_replay_thread()+0x617) [0x56289f12c8e7] 2024-04-17T06:08:04.134 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:08:03 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[66417]: 5: (MDLog::ReplayThread::entry()+0x11) [0x56289f135fd1] 2024-04-17T06:08:04.134 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:08:03 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[66417]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f13384731ca] 2024-04-17T06:08:04.134 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:08:03 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[66417]: 7: clone() 2024-04-17T06:08:04.134 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:08:03 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[66417]: 2024-04-17T06:08:03.857+0000 7f13270bc700 -1 *** Caught signal (Segmentation fault) ** 2024-04-17T06:08:04.134 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:08:03 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[66417]: in thread 7f13270bc700 thread_name:md_log_replay 2024-04-17T06:08:04.134 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:08:03 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[66417]: 2024-04-17T06:08:04.134 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:08:03 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[66417]: ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev) 2024-04-17T06:08:04.134 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:08:03 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[66417]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f133847dd20] 2024-04-17T06:08:04.134 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:08:03 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[66417]: 2: /usr/bin/ceph-mds(+0x54962f) [0x56289f0ce62f] 2024-04-17T06:08:04.134 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:08:03 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[66417]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x56289f0d1dc3] 2024-04-17T06:08:04.134 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:08:03 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[66417]: 4: (MDLog::_replay_thread()+0x617) [0x56289f12c8e7] 2024-04-17T06:08:04.135 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:08:03 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[66417]: 5: (MDLog::ReplayThread::entry()+0x11) [0x56289f135fd1] 2024-04-17T06:08:04.135 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:08:03 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[66417]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f13384731ca] 2024-04-17T06:08:04.135 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:08:03 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[66417]: 7: clone() 2024-04-17T06:08:04.135 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:08:03 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[66417]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-17T06:08:04.135 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:08:03 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[66417]: 2024-04-17T06:08:04.135 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:08:03 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[66417]: 0> 2024-04-17T06:08:03.857+0000 7f13270bc700 -1 *** Caught signal (Segmentation fault) ** 2024-04-17T06:08:04.135 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:08:03 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[66417]: in thread 7f13270bc700 thread_name:md_log_replay 2024-04-17T06:08:04.135 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:08:03 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[66417]: 2024-04-17T06:08:04.135 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:08:03 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[66417]: ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev) 2024-04-17T06:08:04.135 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:08:03 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[66417]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f133847dd20] 2024-04-17T06:08:04.135 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:08:03 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[66417]: 2: /usr/bin/ceph-mds(+0x54962f) [0x56289f0ce62f] 2024-04-17T06:08:04.135 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:08:03 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[66417]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x56289f0d1dc3] 2024-04-17T06:08:04.135 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:08:03 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[66417]: 4: (MDLog::_replay_thread()+0x617) [0x56289f12c8e7] 2024-04-17T06:08:04.135 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:08:03 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[66417]: 5: (MDLog::ReplayThread::entry()+0x11) [0x56289f135fd1] 2024-04-17T06:08:04.135 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:08:03 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[66417]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f13384731ca] 2024-04-17T06:08:04.136 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:08:03 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[66417]: 7: clone() 2024-04-17T06:08:04.136 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:08:03 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[66417]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-17T06:08:04.136 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:08:03 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[66417]: 2024-04-17T06:08:04.136 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:08:03 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[66417]: 0> 2024-04-17T06:08:03.857+0000 7f13270bc700 -1 *** Caught signal (Segmentation fault) ** 2024-04-17T06:08:04.136 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:08:03 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[66417]: in thread 7f13270bc700 thread_name:md_log_replay 2024-04-17T06:08:04.136 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:08:03 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[66417]: 2024-04-17T06:08:04.136 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:08:03 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[66417]: ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev) 2024-04-17T06:08:04.136 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:08:03 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[66417]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f133847dd20] 2024-04-17T06:08:04.136 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:08:03 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[66417]: 2: /usr/bin/ceph-mds(+0x54962f) [0x56289f0ce62f] 2024-04-17T06:08:04.136 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:08:03 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[66417]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x56289f0d1dc3] 2024-04-17T06:08:04.136 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:08:03 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[66417]: 4: (MDLog::_replay_thread()+0x617) [0x56289f12c8e7] 2024-04-17T06:08:04.136 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:08:03 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[66417]: 5: (MDLog::ReplayThread::entry()+0x11) [0x56289f135fd1] 2024-04-17T06:08:04.136 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:08:03 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[66417]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f13384731ca] 2024-04-17T06:08:04.136 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:08:03 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[66417]: 7: clone() 2024-04-17T06:08:04.136 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:08:03 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[66417]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-17T06:08:04.137 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:08:03 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[66417]: 2024-04-17T06:08:04.137 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:08:03 smithi098 podman[71666]: 2024-04-17 06:08:03.918093991 +0000 UTC m=+0.020790979 container died 3f6debd36fb7e3aa462cbb0d74d76a322b2e30e193182f61c0da29c7c7a2f6ec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k, ceph=True, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-17T06:08:04.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:08:03 smithi047 ceph-mon[32223]: pgmap v207: 97 pgs: 97 active+clean; 3.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.3 KiB/s wr, 9 op/s 2024-04-17T06:08:04.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:08:03 smithi103 ceph-mon[36243]: pgmap v207: 97 pgs: 97 active+clean; 3.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.3 KiB/s wr, 9 op/s 2024-04-17T06:08:04.822 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:08:04 smithi098 podman[71666]: 2024-04-17 06:08:04.574341551 +0000 UTC m=+0.677038556 container remove 3f6debd36fb7e3aa462cbb0d74d76a322b2e30e193182f61c0da29c7c7a2f6ec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-17T06:08:04.822 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:08:04 smithi098 systemd[1]: ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@mds.k.service: Main process exited, code=exited, status=139/n/a 2024-04-17T06:08:05.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:08:04 smithi098 ceph-mon[36516]: osdmap e92: 12 total, 12 up, 12 in 2024-04-17T06:08:05.133 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:08:04 smithi098 systemd[1]: ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@mds.k.service: Failed with result 'exit-code'. 2024-04-17T06:08:05.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:08:04 smithi047 ceph-mon[32223]: osdmap e92: 12 total, 12 up, 12 in 2024-04-17T06:08:05.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:08:04 smithi103 ceph-mon[36243]: osdmap e92: 12 total, 12 up, 12 in 2024-04-17T06:08:06.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:08:05 smithi098 ceph-mon[36516]: pgmap v209: 97 pgs: 97 active+clean; 3.5 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 68 KiB/s wr, 34 op/s 2024-04-17T06:08:06.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:08:05 smithi047 ceph-mon[32223]: pgmap v209: 97 pgs: 97 active+clean; 3.5 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 68 KiB/s wr, 34 op/s 2024-04-17T06:08:06.187 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:08:06.188 INFO:tasks.workunit.client.0.smithi047.stdout:251/600: open fd = 3 2024-04-17T06:08:06.188 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.188 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.188 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:08:06.188 INFO:tasks.workunit.client.0.smithi047.stdout:252/600: open fd = 3 2024-04-17T06:08:06.188 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.188 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.188 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:08:06.188 INFO:tasks.workunit.client.0.smithi047.stdout:253/600: open fd = 3 2024-04-17T06:08:06.188 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.188 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.188 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:08:06.188 INFO:tasks.workunit.client.0.smithi047.stdout:254/600: open fd = 3 2024-04-17T06:08:06.189 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.189 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.189 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:08:06.189 INFO:tasks.workunit.client.0.smithi047.stdout:255/600: open fd = 3 2024-04-17T06:08:06.189 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.189 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.189 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:08:06.189 INFO:tasks.workunit.client.0.smithi047.stdout:256/600: open fd = 3 2024-04-17T06:08:06.189 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.189 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.189 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:08:06.189 INFO:tasks.workunit.client.0.smithi047.stdout:257/600: open fd = 3 2024-04-17T06:08:06.189 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.189 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.189 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:08:06.190 INFO:tasks.workunit.client.0.smithi047.stdout:258/600: open fd = 3 2024-04-17T06:08:06.190 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.190 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.190 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:08:06.190 INFO:tasks.workunit.client.0.smithi047.stdout:259/600: open fd = 3 2024-04-17T06:08:06.190 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.190 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.190 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:08:06.190 INFO:tasks.workunit.client.0.smithi047.stdout:260/600: open fd = 3 2024-04-17T06:08:06.190 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.190 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.190 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:08:06.190 INFO:tasks.workunit.client.0.smithi047.stdout:261/600: open fd = 3 2024-04-17T06:08:06.190 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.191 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.191 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:08:06.191 INFO:tasks.workunit.client.0.smithi047.stdout:262/600: open fd = 3 2024-04-17T06:08:06.191 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.191 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.191 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:08:06.191 INFO:tasks.workunit.client.0.smithi047.stdout:263/600: open fd = 3 2024-04-17T06:08:06.191 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.191 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.191 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:08:06.191 INFO:tasks.workunit.client.0.smithi047.stdout:264/600: open fd = 3 2024-04-17T06:08:06.191 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.191 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.191 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:08:06.192 INFO:tasks.workunit.client.0.smithi047.stdout:265/600: open fd = 3 2024-04-17T06:08:06.192 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.192 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.192 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:08:06.192 INFO:tasks.workunit.client.0.smithi047.stdout:266/600: open fd = 3 2024-04-17T06:08:06.192 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.192 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.192 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:08:06.192 INFO:tasks.workunit.client.0.smithi047.stdout:267/600: open fd = 3 2024-04-17T06:08:06.192 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.192 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.192 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:08:06.192 INFO:tasks.workunit.client.0.smithi047.stdout:268/600: open fd = 3 2024-04-17T06:08:06.192 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.193 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.193 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:08:06.193 INFO:tasks.workunit.client.0.smithi047.stdout:269/600: open fd = 3 2024-04-17T06:08:06.193 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.193 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.193 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:08:06.193 INFO:tasks.workunit.client.0.smithi047.stdout:270/600: open fd = 3 2024-04-17T06:08:06.193 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.193 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.193 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:08:06.193 INFO:tasks.workunit.client.0.smithi047.stdout:271/600: open fd = 3 2024-04-17T06:08:06.193 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.193 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.193 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:08:06.194 INFO:tasks.workunit.client.0.smithi047.stdout:272/600: open fd = 3 2024-04-17T06:08:06.194 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.194 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.194 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:08:06.194 INFO:tasks.workunit.client.0.smithi047.stdout:273/600: open fd = 3 2024-04-17T06:08:06.194 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.194 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.194 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:08:06.194 INFO:tasks.workunit.client.0.smithi047.stdout:274/600: open fd = 3 2024-04-17T06:08:06.194 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.194 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.194 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:08:06.194 INFO:tasks.workunit.client.0.smithi047.stdout:275/600: open fd = 3 2024-04-17T06:08:06.195 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.195 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.195 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:08:06.195 INFO:tasks.workunit.client.0.smithi047.stdout:276/600: open fd = 3 2024-04-17T06:08:06.195 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.195 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.195 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:08:06.195 INFO:tasks.workunit.client.0.smithi047.stdout:277/600: open fd = 3 2024-04-17T06:08:06.195 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.195 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.195 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:08:06.195 INFO:tasks.workunit.client.0.smithi047.stdout:278/600: open fd = 3 2024-04-17T06:08:06.195 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.196 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.196 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:08:06.196 INFO:tasks.workunit.client.0.smithi047.stdout:279/600: open fd = 3 2024-04-17T06:08:06.196 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.196 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.196 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:08:06.196 INFO:tasks.workunit.client.0.smithi047.stdout:280/600: open fd = 3 2024-04-17T06:08:06.196 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.196 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.196 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:08:06.196 INFO:tasks.workunit.client.0.smithi047.stdout:281/600: open fd = 3 2024-04-17T06:08:06.196 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.196 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.196 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:08:06.197 INFO:tasks.workunit.client.0.smithi047.stdout:282/600: open fd = 3 2024-04-17T06:08:06.197 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.197 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.197 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:08:06.197 INFO:tasks.workunit.client.0.smithi047.stdout:283/600: open fd = 3 2024-04-17T06:08:06.197 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.197 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.197 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:08:06.197 INFO:tasks.workunit.client.0.smithi047.stdout:284/600: open fd = 3 2024-04-17T06:08:06.197 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.197 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.197 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:08:06.197 INFO:tasks.workunit.client.0.smithi047.stdout:285/600: open fd = 3 2024-04-17T06:08:06.198 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.198 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.198 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:08:06.198 INFO:tasks.workunit.client.0.smithi047.stdout:286/600: open fd = 3 2024-04-17T06:08:06.198 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.198 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.198 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:08:06.198 INFO:tasks.workunit.client.0.smithi047.stdout:287/600: open fd = 3 2024-04-17T06:08:06.198 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.198 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.198 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:08:06.198 INFO:tasks.workunit.client.0.smithi047.stdout:288/600: open fd = 3 2024-04-17T06:08:06.198 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.198 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.199 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:08:06.199 INFO:tasks.workunit.client.0.smithi047.stdout:289/600: open fd = 3 2024-04-17T06:08:06.199 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.199 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.199 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:08:06.199 INFO:tasks.workunit.client.0.smithi047.stdout:290/600: open fd = 3 2024-04-17T06:08:06.199 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.199 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.199 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:08:06.199 INFO:tasks.workunit.client.0.smithi047.stdout:291/600: open fd = 3 2024-04-17T06:08:06.199 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.199 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.199 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:08:06.200 INFO:tasks.workunit.client.0.smithi047.stdout:292/600: open fd = 3 2024-04-17T06:08:06.200 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.200 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.200 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:08:06.200 INFO:tasks.workunit.client.0.smithi047.stdout:293/600: open fd = 3 2024-04-17T06:08:06.200 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.200 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.200 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:08:06.200 INFO:tasks.workunit.client.0.smithi047.stdout:294/600: open fd = 3 2024-04-17T06:08:06.200 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.200 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.200 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:08:06.200 INFO:tasks.workunit.client.0.smithi047.stdout:295/600: open fd = 3 2024-04-17T06:08:06.200 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.201 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.201 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:08:06.201 INFO:tasks.workunit.client.0.smithi047.stdout:296/600: open fd = 3 2024-04-17T06:08:06.201 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.201 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.201 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:08:06.201 INFO:tasks.workunit.client.0.smithi047.stdout:297/600: open fd = 3 2024-04-17T06:08:06.201 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.201 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.201 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:08:06.201 INFO:tasks.workunit.client.0.smithi047.stdout:298/600: open fd = 3 2024-04-17T06:08:06.201 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.201 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.201 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:08:06.202 INFO:tasks.workunit.client.0.smithi047.stdout:299/600: open fd = 3 2024-04-17T06:08:06.202 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.202 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.203 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:08:06.203 INFO:tasks.workunit.client.0.smithi047.stdout:300/600: open fd = 3 2024-04-17T06:08:06.203 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.203 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.203 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:08:06.203 INFO:tasks.workunit.client.0.smithi047.stdout:301/600: open fd = 3 2024-04-17T06:08:06.203 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.203 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.203 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:08:06.203 INFO:tasks.workunit.client.0.smithi047.stdout:302/600: open fd = 3 2024-04-17T06:08:06.203 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.203 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.203 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:08:06.204 INFO:tasks.workunit.client.0.smithi047.stdout:303/600: open fd = 3 2024-04-17T06:08:06.204 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.204 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.204 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:08:06.204 INFO:tasks.workunit.client.0.smithi047.stdout:304/600: open fd = 3 2024-04-17T06:08:06.204 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.204 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.204 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:08:06.204 INFO:tasks.workunit.client.0.smithi047.stdout:305/600: open fd = 3 2024-04-17T06:08:06.204 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.204 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.204 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:08:06.204 INFO:tasks.workunit.client.0.smithi047.stdout:306/600: open fd = 3 2024-04-17T06:08:06.204 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.204 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.205 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:08:06.205 INFO:tasks.workunit.client.0.smithi047.stdout:307/600: open fd = 3 2024-04-17T06:08:06.205 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.205 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.205 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:08:06.205 INFO:tasks.workunit.client.0.smithi047.stdout:308/600: open fd = 3 2024-04-17T06:08:06.205 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.205 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.205 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:08:06.205 INFO:tasks.workunit.client.0.smithi047.stdout:309/600: open fd = 3 2024-04-17T06:08:06.205 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.205 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.205 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:08:06.205 INFO:tasks.workunit.client.0.smithi047.stdout:310/600: open fd = 3 2024-04-17T06:08:06.206 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.206 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.206 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:08:06.206 INFO:tasks.workunit.client.0.smithi047.stdout:311/600: open fd = 3 2024-04-17T06:08:06.206 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.206 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.206 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:08:06.206 INFO:tasks.workunit.client.0.smithi047.stdout:312/600: open fd = 3 2024-04-17T06:08:06.206 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.206 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:08:06.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:08:05 smithi103 ceph-mon[36243]: pgmap v209: 97 pgs: 97 active+clean; 3.5 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 68 KiB/s wr, 34 op/s 2024-04-17T06:08:08.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:08:07 smithi098 ceph-mon[36516]: pgmap v210: 97 pgs: 97 active+clean; 3.5 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 68 KiB/s wr, 34 op/s 2024-04-17T06:08:08.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:08:07 smithi047 ceph-mon[32223]: pgmap v210: 97 pgs: 97 active+clean; 3.5 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 68 KiB/s wr, 34 op/s 2024-04-17T06:08:08.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:08:07 smithi103 ceph-mon[36243]: pgmap v210: 97 pgs: 97 active+clean; 3.5 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 68 KiB/s wr, 34 op/s 2024-04-17T06:08:10.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:08:09 smithi098 ceph-mon[36516]: pgmap v211: 97 pgs: 97 active+clean; 3.5 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 68 KiB/s wr, 34 op/s 2024-04-17T06:08:10.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:08:09 smithi098 ceph-mon[36516]: osdmap e93: 12 total, 12 up, 12 in 2024-04-17T06:08:10.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:08:09 smithi047 ceph-mon[32223]: pgmap v211: 97 pgs: 97 active+clean; 3.5 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 68 KiB/s wr, 34 op/s 2024-04-17T06:08:10.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:08:09 smithi047 ceph-mon[32223]: osdmap e93: 12 total, 12 up, 12 in 2024-04-17T06:08:10.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:08:09 smithi103 ceph-mon[36243]: pgmap v211: 97 pgs: 97 active+clean; 3.5 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 68 KiB/s wr, 34 op/s 2024-04-17T06:08:10.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:08:09 smithi103 ceph-mon[36243]: osdmap e93: 12 total, 12 up, 12 in 2024-04-17T06:08:12.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:08:11 smithi098 ceph-mon[36516]: pgmap v213: 97 pgs: 97 active+clean; 3.5 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 85 KiB/s wr, 42 op/s 2024-04-17T06:08:12.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:08:11 smithi047 ceph-mon[32223]: pgmap v213: 97 pgs: 97 active+clean; 3.5 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 85 KiB/s wr, 42 op/s 2024-04-17T06:08:12.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:08:11 smithi103 ceph-mon[36243]: pgmap v213: 97 pgs: 97 active+clean; 3.5 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 85 KiB/s wr, 42 op/s 2024-04-17T06:08:14.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:08:13 smithi098 ceph-mon[36516]: pgmap v214: 97 pgs: 97 active+clean; 3.5 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 80 KiB/s wr, 39 op/s 2024-04-17T06:08:14.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:08:13 smithi047 ceph-mon[32223]: pgmap v214: 97 pgs: 97 active+clean; 3.5 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 80 KiB/s wr, 39 op/s 2024-04-17T06:08:14.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:08:13 smithi103 ceph-mon[36243]: pgmap v214: 97 pgs: 97 active+clean; 3.5 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 80 KiB/s wr, 39 op/s 2024-04-17T06:08:15.164 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:08:14 smithi098 systemd[1]: ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@mds.k.service: Scheduled restart job, restart counter is at 1. 2024-04-17T06:08:15.164 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:08:14 smithi098 systemd[1]: Stopped Ceph mds.k for 565db2d8-fc7e-11ee-bc91-c7b262605968. 2024-04-17T06:08:15.164 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:08:14 smithi098 systemd[1]: Starting Ceph mds.k for 565db2d8-fc7e-11ee-bc91-c7b262605968... 2024-04-17T06:08:15.164 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:08:15 smithi098 podman[71760]: 2024-04-17 06:08:15.020266682 +0000 UTC m=+0.018150903 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-17T06:08:15.417 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:08:15 smithi098 podman[71760]: 2024-04-17 06:08:15.167238179 +0000 UTC m=+0.165122381 container create 87698d00c9e06b8a3ba01b7441d686fa32afaca37d49cbb752c0977710c3b739 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-17T06:08:15.679 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:08:15 smithi098 podman[71760]: 2024-04-17 06:08:15.420092364 +0000 UTC m=+0.417976587 container init 87698d00c9e06b8a3ba01b7441d686fa32afaca37d49cbb752c0977710c3b739 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-17T06:08:15.679 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:08:15 smithi098 podman[71760]: 2024-04-17 06:08:15.422918724 +0000 UTC m=+0.420802938 container start 87698d00c9e06b8a3ba01b7441d686fa32afaca37d49cbb752c0977710c3b739 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.vendor=CentOS) 2024-04-17T06:08:15.679 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:08:15 smithi098 ceph-mds[71779]: -- 172.21.15.98:0/1770386249 <== mon.2 v2:172.21.15.98:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557e1b6a9a40 con 0x557e1b67cc00 2024-04-17T06:08:15.679 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:08:15 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[71775]: starting mds.k at 2024-04-17T06:08:15.679 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:08:15 smithi098 bash[71760]: 87698d00c9e06b8a3ba01b7441d686fa32afaca37d49cbb752c0977710c3b739 2024-04-17T06:08:15.680 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:08:15 smithi098 systemd[1]: Started Ceph mds.k for 565db2d8-fc7e-11ee-bc91-c7b262605968. 2024-04-17T06:08:16.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:08:15 smithi098 ceph-mon[36516]: pgmap v215: 97 pgs: 97 active+clean; 3.6 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 8.9 KiB/s wr, 9 op/s 2024-04-17T06:08:16.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:08:15 smithi098 ceph-mon[36516]: daemon mds.k restarted 2024-04-17T06:08:16.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:08:15 smithi103 ceph-mon[36243]: pgmap v215: 97 pgs: 97 active+clean; 3.6 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 8.9 KiB/s wr, 9 op/s 2024-04-17T06:08:16.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:08:15 smithi103 ceph-mon[36243]: daemon mds.k restarted 2024-04-17T06:08:16.388 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:08:15 smithi047 ceph-mon[32223]: pgmap v215: 97 pgs: 97 active+clean; 3.6 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 8.9 KiB/s wr, 9 op/s 2024-04-17T06:08:16.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:08:15 smithi047 ceph-mon[32223]: daemon mds.k restarted 2024-04-17T06:08:16.639 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.639+0000 7f150131c640 1 -- 172.21.15.47:0/3455319492 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f14fc06b170 msgr2=0x7f14fc10edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:08:16.639 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.639+0000 7f150131c640 1 --2- 172.21.15.47:0/3455319492 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f14fc06b170 0x7f14fc10edd0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f14f4009ca0 tx=0x7f14f402c180 comp rx=0 tx=0).stop 2024-04-17T06:08:16.639 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.639+0000 7f150131c640 1 -- 172.21.15.47:0/3455319492 shutdown_connections 2024-04-17T06:08:16.639 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.639+0000 7f150131c640 1 --2- 172.21.15.47:0/3455319492 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f14fc06b170 0x7f14fc10edd0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:16.640 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.639+0000 7f150131c640 1 --2- 172.21.15.47:0/3455319492 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14fc06a7d0 0x7f14fc06ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:16.640 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.639+0000 7f150131c640 1 --2- 172.21.15.47:0/3455319492 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f14fc108030 0x7f14fc108410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:16.640 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.639+0000 7f150131c640 1 -- 172.21.15.47:0/3455319492 >> 172.21.15.47:0/3455319492 conn(0x7f14fc076c60 msgr2=0x7f14fc077070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:08:16.640 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.639+0000 7f150131c640 1 -- 172.21.15.47:0/3455319492 shutdown_connections 2024-04-17T06:08:16.640 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.639+0000 7f150131c640 1 -- 172.21.15.47:0/3455319492 wait complete. 2024-04-17T06:08:16.640 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.640+0000 7f150131c640 1 Processor -- start 2024-04-17T06:08:16.640 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.640+0000 7f150131c640 1 -- start start 2024-04-17T06:08:16.640 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.640+0000 7f150131c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14fc06a7d0 0x7f14fc101280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:16.640 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.640+0000 7f150131c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f14fc06b170 0x7f14fc1017c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:16.640 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.640+0000 7f150131c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f14fc108030 0x7f14fc101d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:16.640 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.640+0000 7f150131c640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f14fc112300 con 0x7f14fc06b170 2024-04-17T06:08:16.640 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.640+0000 7f150131c640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f14fc112180 con 0x7f14fc06a7d0 2024-04-17T06:08:16.640 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.640+0000 7f150131c640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f14fc112480 con 0x7f14fc108030 2024-04-17T06:08:16.640 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.640+0000 7f14f27fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f14fc06b170 0x7f14fc1017c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:16.641 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.640+0000 7f14f27fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f14fc06b170 0x7f14fc1017c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50220/0 (socket says 172.21.15.47:50220) 2024-04-17T06:08:16.641 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.640+0000 7f14f27fc640 1 -- 172.21.15.47:0/3758528644 learned_addr learned my addr 172.21.15.47:0/3758528644 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:08:16.641 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.640+0000 7f14faffd640 1 --2- 172.21.15.47:0/3758528644 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14fc06a7d0 0x7f14fc101280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:16.641 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.641+0000 7f14fb7fe640 1 --2- 172.21.15.47:0/3758528644 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f14fc108030 0x7f14fc101d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:16.641 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.641+0000 7f14f27fc640 1 -- 172.21.15.47:0/3758528644 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f14fc108030 msgr2=0x7f14fc101d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:08:16.641 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.641+0000 7f14f27fc640 1 --2- 172.21.15.47:0/3758528644 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f14fc108030 0x7f14fc101d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:16.641 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.641+0000 7f14f27fc640 1 -- 172.21.15.47:0/3758528644 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14fc06a7d0 msgr2=0x7f14fc101280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:08:16.641 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.641+0000 7f14f27fc640 1 --2- 172.21.15.47:0/3758528644 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14fc06a7d0 0x7f14fc101280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:16.641 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.641+0000 7f14f27fc640 1 -- 172.21.15.47:0/3758528644 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14fc1a6ca0 con 0x7f14fc06b170 2024-04-17T06:08:16.641 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.641+0000 7f14f27fc640 1 --2- 172.21.15.47:0/3758528644 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f14fc06b170 0x7f14fc1017c0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f14e400e9f0 tx=0x7f14e400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:08:16.641 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.641+0000 7f14f8ff9640 1 -- 172.21.15.47:0/3758528644 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e400cd80 con 0x7f14fc06b170 2024-04-17T06:08:16.641 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.641+0000 7f14f8ff9640 1 -- 172.21.15.47:0/3758528644 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14e4010910 con 0x7f14fc06b170 2024-04-17T06:08:16.641 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.641+0000 7f150131c640 1 -- 172.21.15.47:0/3758528644 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14fc1a6f90 con 0x7f14fc06b170 2024-04-17T06:08:16.641 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.641+0000 7f14f8ff9640 1 -- 172.21.15.47:0/3758528644 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e4018990 con 0x7f14fc06b170 2024-04-17T06:08:16.641 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.641+0000 7f150131c640 1 -- 172.21.15.47:0/3758528644 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14fc1a7450 con 0x7f14fc06b170 2024-04-17T06:08:16.648 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.644+0000 7f14f8ff9640 1 -- 172.21.15.47:0/3758528644 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f14e4010ac0 con 0x7f14fc06b170 2024-04-17T06:08:16.649 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.644+0000 7f1503fe8740 1 Processor -- start 2024-04-17T06:08:16.649 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.644+0000 7f1503fe8740 1 -- start start 2024-04-17T06:08:16.649 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.644+0000 7f1503fe8740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f86e3d0c0 0x561f86e3d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:16.649 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.644+0000 7f1503fe8740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561f86e3d9e0 0x561f86e46270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:16.649 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.644+0000 7f1503fe8740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f86e467b0 0x561f86e48ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:16.649 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.644+0000 7f1503fe8740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x561f86da27d0 con 0x561f86e3d9e0 2024-04-17T06:08:16.649 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.644+0000 7f1503fe8740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x561f86d3ff70 con 0x561f86e467b0 2024-04-17T06:08:16.649 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.644+0000 7f1503fe8740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561f86d9e790 con 0x561f86e3d0c0 2024-04-17T06:08:16.649 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.645+0000 7f14f8ff9640 1 --2- 172.21.15.47:0/3758528644 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f14e80a3dc0 0x7f14e80a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:16.649 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.645+0000 7f14faffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561f86e3d9e0 0x561f86e46270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:16.649 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.645+0000 7f14faffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561f86e3d9e0 0x561f86e46270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50226/0 (socket says 172.21.15.47:50226) 2024-04-17T06:08:16.649 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.645+0000 7f14faffd640 1 -- 172.21.15.47:0/2656966434 learned_addr learned my addr 172.21.15.47:0/2656966434 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:08:16.649 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.645+0000 7f14f27fc640 1 --2- 172.21.15.47:0/2656966434 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f86e467b0 0x561f86e48ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:16.650 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.645+0000 7f14fb7fe640 1 --2- 172.21.15.47:0/2656966434 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f86e3d0c0 0x561f86e3d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:16.650 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.645+0000 7f14fb7fe640 1 --2- 172.21.15.47:0/3758528644 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f14e80a3dc0 0x7f14e80a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:16.650 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.645+0000 7f14f8ff9640 1 -- 172.21.15.47:0/3758528644 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8469+0+0 (secure 0 0 0) 0x7f14e40d0ba0 con 0x7f14fc06b170 2024-04-17T06:08:16.650 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.645+0000 7f14faffd640 1 -- 172.21.15.47:0/2656966434 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f86e3d0c0 msgr2=0x561f86e3d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:08:16.650 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.645+0000 7f14faffd640 1 --2- 172.21.15.47:0/2656966434 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f86e3d0c0 0x561f86e3d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:16.650 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.645+0000 7f14faffd640 1 -- 172.21.15.47:0/2656966434 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f86e467b0 msgr2=0x561f86e48ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:08:16.650 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.645+0000 7f14faffd640 1 --2- 172.21.15.47:0/2656966434 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f86e467b0 0x561f86e48ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:16.650 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.645+0000 7f14faffd640 1 -- 172.21.15.47:0/2656966434 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f86ab5210 con 0x561f86e3d9e0 2024-04-17T06:08:16.650 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.645+0000 7f14fb7fe640 1 --2- 172.21.15.47:0/3758528644 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f14e80a3dc0 0x7f14e80a6280 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f14f4005fa0 tx=0x7f14f40095c0 comp rx=0 tx=0).ready entity=mgr.24923 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:08:16.650 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.645+0000 7f14faffd640 1 --2- 172.21.15.47:0/2656966434 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561f86e3d9e0 0x561f86e46270 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f14ec00bc90 tx=0x7f14ec0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26764c79a757c49 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:08:16.650 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.645+0000 7f14f17fa640 1 -- 172.21.15.47:0/2656966434 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14ec01c480 con 0x561f86e3d9e0 2024-04-17T06:08:16.650 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.645+0000 7f14f17fa640 1 -- 172.21.15.47:0/2656966434 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14ec01cb00 con 0x561f86e3d9e0 2024-04-17T06:08:16.650 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.646+0000 7f14f17fa640 1 -- 172.21.15.47:0/2656966434 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14ec011c60 con 0x561f86e3d9e0 2024-04-17T06:08:16.651 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.646+0000 7f1503fe8740 1 -- 172.21.15.47:0/2656966434 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561f86e3d9e0 msgr2=0x561f86e46270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:08:16.651 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.646+0000 7f1503fe8740 1 --2- 172.21.15.47:0/2656966434 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561f86e3d9e0 0x561f86e46270 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f14ec00bc90 tx=0x7f14ec0107a0 comp rx=0 tx=0).stop 2024-04-17T06:08:16.651 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.646+0000 7f1503fe8740 1 -- 172.21.15.47:0/2656966434 shutdown_connections 2024-04-17T06:08:16.651 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.646+0000 7f1503fe8740 1 --2- 172.21.15.47:0/2656966434 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f86e467b0 0x561f86e48ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:16.651 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.646+0000 7f1503fe8740 1 --2- 172.21.15.47:0/2656966434 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561f86e3d9e0 0x561f86e46270 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:16.651 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.646+0000 7f1503fe8740 1 --2- 172.21.15.47:0/2656966434 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f86e3d0c0 0x561f86e3d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:16.651 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.646+0000 7f1503fe8740 1 -- 172.21.15.47:0/2656966434 >> 172.21.15.47:0/2656966434 conn(0x561f86e379e0 msgr2=0x561f86e37dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:08:16.651 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.646+0000 7f1503fe8740 1 -- 172.21.15.47:0/2656966434 shutdown_connections 2024-04-17T06:08:16.651 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.646+0000 7f1503fe8740 1 -- 172.21.15.47:0/2656966434 wait complete. 2024-04-17T06:08:16.651 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.647+0000 7f1503fe8740 1 -- start start 2024-04-17T06:08:16.651 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.647+0000 7f1503fe8740 1 Processor -- start 2024-04-17T06:08:16.651 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.647+0000 7f1503fe8740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f86e3d0c0 0x561f86e437d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:16.651 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.647+0000 7f1503fe8740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f86e3d9e0 0x561f86e43d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:16.651 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.647+0000 7f1503fe8740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561f86e467b0 0x561f86e44250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:16.652 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.647+0000 7f1503fe8740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x561f86e493e0 con 0x561f86e467b0 2024-04-17T06:08:16.652 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.647+0000 7f1503fe8740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x561f86d2ae90 con 0x561f86e3d9e0 2024-04-17T06:08:16.652 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.647+0000 7f1503fe8740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561f86e49560 con 0x561f86e3d0c0 2024-04-17T06:08:16.652 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.647+0000 7f1503fe8740 10 client.0 _ll_register_callbacks cb 0x561f86d40d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:08:16.652 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.647+0000 7f14fb7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561f86e467b0 0x561f86e44250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:16.652 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.647+0000 7f14f27fc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f86e3d9e0 0x561f86e43d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:16.652 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.647+0000 7f14fb7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561f86e467b0 0x561f86e44250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50238/0 (socket says 172.21.15.47:50238) 2024-04-17T06:08:16.652 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.647+0000 7f14fb7fe640 1 -- 172.21.15.47:0/1911961723 learned_addr learned my addr 172.21.15.47:0/1911961723 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:08:16.652 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.647+0000 7f14faffd640 1 --2- 172.21.15.47:0/1911961723 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f86e3d0c0 0x561f86e437d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:16.652 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.648+0000 7f14fb7fe640 1 -- 172.21.15.47:0/1911961723 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f86e3d0c0 msgr2=0x561f86e437d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:08:16.652 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.648+0000 7f14fb7fe640 1 --2- 172.21.15.47:0/1911961723 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f86e3d0c0 0x561f86e437d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:16.652 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.648+0000 7f14fb7fe640 1 -- 172.21.15.47:0/1911961723 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f86e3d9e0 msgr2=0x561f86e43d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:08:16.652 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.648+0000 7f14fb7fe640 1 --2- 172.21.15.47:0/1911961723 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f86e3d9e0 0x561f86e43d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:16.652 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.648+0000 7f14fb7fe640 1 -- 172.21.15.47:0/1911961723 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f86b81620 con 0x561f86e467b0 2024-04-17T06:08:16.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.648+0000 7f14faffd640 1 --2- 172.21.15.47:0/1911961723 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f86e3d0c0 0x561f86e437d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:08:16.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.648+0000 7f14f27fc640 1 --2- 172.21.15.47:0/1911961723 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f86e3d9e0 0x561f86e43d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T06:08:16.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.648+0000 7f14fb7fe640 1 --2- 172.21.15.47:0/1911961723 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561f86e467b0 0x561f86e44250 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f14f4036000 tx=0x7f14f402c180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bddec56fe1703253 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:08:16.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.648+0000 7f14e27fc640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-17T06:08:16.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.648+0000 7f14e27fc640 1 -- 172.21.15.47:0/1911961723 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f4039070 con 0x561f86e467b0 2024-04-17T06:08:16.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.648+0000 7f14e27fc640 1 -- 172.21.15.47:0/1911961723 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8469+0+0 (secure 0 0 0) 0x7f14f4004340 con 0x561f86e467b0 2024-04-17T06:08:16.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.648+0000 7f1503fe8740 1 -- 172.21.15.47:0/1911961723 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14ec01cb00 con 0x561f86e467b0 2024-04-17T06:08:16.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.648+0000 7f14e27fc640 1 -- 172.21.15.47:0/1911961723 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f14f405d050 con 0x561f86e467b0 2024-04-17T06:08:16.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.648+0000 7f1503fe8740 10 client.15984 fetch_fsmap learned FSMap version 32 2024-04-17T06:08:16.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.648+0000 7f1503fe8740 1 -- 172.21.15.47:0/1911961723 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x561f86e55670 con 0x561f86e467b0 2024-04-17T06:08:16.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.648+0000 7f14e27fc640 1 -- 172.21.15.47:0/1911961723 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14ec01cb00 con 0x561f86e467b0 2024-04-17T06:08:16.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.649+0000 7f14e27fc640 1 -- 172.21.15.47:0/1911961723 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) ==== 8339+0+0 (secure 0 0 0) 0x7f14f404e9d0 con 0x561f86e467b0 2024-04-17T06:08:16.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.649+0000 7f1503fe8740 10 client.15984 fetch_fsmap finished waiting for FSMap version 32 2024-04-17T06:08:16.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.649+0000 7f1503fe8740 10 client.15984 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-17T06:08:16.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.649+0000 7f1503fe8740 20 client.15984 populate_metadata read hostname 'smithi047' 2024-04-17T06:08:16.654 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.649+0000 7f1503fe8740 1 --2- 172.21.15.47:0/1911961723 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x561f86e56b80 0x561f86e58f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:16.654 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.649+0000 7f1503fe8740 4 client.15984 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T06:08:16.654 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.649+0000 7f1503fe8740 1 -- 172.21.15.47:0/1911961723 --> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561f86db01f0 con 0x561f86e56b80 2024-04-17T06:08:16.654 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.649+0000 7f14faffd640 1 --2- 172.21.15.47:0/1911961723 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x561f86e56b80 0x561f86e58f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:16.654 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.649+0000 7f14e27fc640 10 client.15984 ms_handle_connect on v2:172.21.15.98:6838/2841287917 2024-04-17T06:08:16.654 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.649+0000 7f14faffd640 1 --2- 172.21.15.47:0/1911961723 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x561f86e56b80 0x561f86e58f90 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce23cc2867d52125 server_cookie=be3aa1347d7346fe in_seq=0 out_seq=0 2024-04-17T06:08:16.654 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.650+0000 7f14e27fc640 1 -- 172.21.15.47:0/1911961723 <== mds.0 v2:172.21.15.98:6838/2841287917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561f86db01f0 con 0x561f86e56b80 2024-04-17T06:08:16.654 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.650+0000 7f14e27fc640 10 client.15984 handle_command_reply: tid=0 2024-04-17T06:08:16.654 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.650+0000 7f1503fe8740 1 client.15984 shutdown 2024-04-17T06:08:16.654 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.650+0000 7f1503fe8740 1 -- 172.21.15.47:0/1911961723 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561f86e467b0 msgr2=0x561f86e44250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:08:16.654 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.650+0000 7f1503fe8740 1 --2- 172.21.15.47:0/1911961723 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561f86e467b0 0x561f86e44250 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f14f4036000 tx=0x7f14f402c180 comp rx=0 tx=0).stop 2024-04-17T06:08:16.654 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.650+0000 7f1503fe8740 1 -- 172.21.15.47:0/1911961723 shutdown_connections 2024-04-17T06:08:16.654 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.650+0000 7f1503fe8740 1 --2- 172.21.15.47:0/1911961723 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x561f86e56b80 0x561f86e58f90 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:16.655 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.650+0000 7f1503fe8740 1 --2- 172.21.15.47:0/1911961723 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561f86e467b0 0x561f86e44250 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:16.655 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.650+0000 7f1503fe8740 1 --2- 172.21.15.47:0/1911961723 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f86e3d9e0 0x561f86e43d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:16.655 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.650+0000 7f14e27fc640 0 client.15984 ms_handle_reset on v2:172.21.15.98:6838/2841287917 2024-04-17T06:08:16.655 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.650+0000 7f1503fe8740 1 --2- 172.21.15.47:0/1911961723 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f86e3d0c0 0x561f86e437d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:16.655 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.650+0000 7f1503fe8740 1 -- 172.21.15.47:0/1911961723 >> 172.21.15.47:0/1911961723 conn(0x561f86e379e0 msgr2=0x561f86e39730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:08:16.655 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.651+0000 7f1503fe8740 1 -- 172.21.15.47:0/1911961723 shutdown_connections 2024-04-17T06:08:16.655 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.651+0000 7f1503fe8740 1 -- 172.21.15.47:0/1911961723 wait complete. 2024-04-17T06:08:16.655 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.651+0000 7f1503fe8740 20 client.15984 trim_cache size 0 max 16384 2024-04-17T06:08:16.664 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.665+0000 7f1503fe8740 1 Processor -- start 2024-04-17T06:08:16.664 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.666+0000 7f1503fe8740 1 -- start start 2024-04-17T06:08:16.665 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.666+0000 7f1503fe8740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f86e3d0c0 0x561f86e45f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:16.665 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.666+0000 7f1503fe8740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561f86e3d9e0 0x561f86e522a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:16.665 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.666+0000 7f1503fe8740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f86e467b0 0x561f86e5e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:16.665 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.666+0000 7f1503fe8740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f14fc1b2640 con 0x561f86e3d9e0 2024-04-17T06:08:16.665 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.666+0000 7f1503fe8740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f14fc1aff00 con 0x561f86e3d0c0 2024-04-17T06:08:16.665 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.666+0000 7f1503fe8740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f14fc1add50 con 0x561f86e467b0 2024-04-17T06:08:16.665 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.666+0000 7f14f27fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561f86e3d9e0 0x561f86e522a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:16.665 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.666+0000 7f14f27fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561f86e3d9e0 0x561f86e522a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50252/0 (socket says 172.21.15.47:50252) 2024-04-17T06:08:16.665 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.666+0000 7f14f27fc640 1 -- 172.21.15.47:0/1874428419 learned_addr learned my addr 172.21.15.47:0/1874428419 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:08:16.665 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.666+0000 7f14faffd640 1 --2- 172.21.15.47:0/1874428419 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f86e3d0c0 0x561f86e45f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:16.665 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.666+0000 7f14fb7fe640 1 --2- 172.21.15.47:0/1874428419 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f86e467b0 0x561f86e5e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:16.665 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.666+0000 7f14f27fc640 1 -- 172.21.15.47:0/1874428419 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f86e467b0 msgr2=0x561f86e5e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:08:16.665 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.666+0000 7f14f27fc640 1 --2- 172.21.15.47:0/1874428419 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f86e467b0 0x561f86e5e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:16.665 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.666+0000 7f14f27fc640 1 -- 172.21.15.47:0/1874428419 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f86e3d0c0 msgr2=0x561f86e45f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:08:16.666 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.666+0000 7f14f27fc640 1 --2- 172.21.15.47:0/1874428419 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f86e3d0c0 0x561f86e45f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:16.666 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.666+0000 7f14f27fc640 1 -- 172.21.15.47:0/1874428419 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f86e48ba0 con 0x561f86e3d9e0 2024-04-17T06:08:16.666 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.667+0000 7f14f27fc640 1 --2- 172.21.15.47:0/1874428419 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561f86e3d9e0 0x561f86e522a0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f14e4099540 tx=0x7f14e40bd040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61144e2473dcae3a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:08:16.666 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.667+0000 7f14f0ff9640 1 -- 172.21.15.47:0/1874428419 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e40a0f20 con 0x561f86e3d9e0 2024-04-17T06:08:16.666 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.667+0000 7f14f0ff9640 1 -- 172.21.15.47:0/1874428419 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14e40a57e0 con 0x561f86e3d9e0 2024-04-17T06:08:16.666 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.667+0000 7f14f0ff9640 1 -- 172.21.15.47:0/1874428419 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e409bfc0 con 0x561f86e3d9e0 2024-04-17T06:08:16.669 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.667+0000 7f1503fe8740 1 -- 172.21.15.47:0/1874428419 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561f86e3d9e0 msgr2=0x561f86e522a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:08:16.669 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.667+0000 7f1503fe8740 1 --2- 172.21.15.47:0/1874428419 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561f86e3d9e0 0x561f86e522a0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f14e4099540 tx=0x7f14e40bd040 comp rx=0 tx=0).stop 2024-04-17T06:08:16.669 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.667+0000 7f1503fe8740 1 -- 172.21.15.47:0/1874428419 shutdown_connections 2024-04-17T06:08:16.669 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.667+0000 7f1503fe8740 1 --2- 172.21.15.47:0/1874428419 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f86e467b0 0x561f86e5e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:16.669 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.667+0000 7f1503fe8740 1 --2- 172.21.15.47:0/1874428419 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561f86e3d9e0 0x561f86e522a0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:16.669 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.667+0000 7f1503fe8740 1 --2- 172.21.15.47:0/1874428419 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f86e3d0c0 0x561f86e45f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:16.669 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.667+0000 7f1503fe8740 1 -- 172.21.15.47:0/1874428419 >> 172.21.15.47:0/1874428419 conn(0x561f86e379e0 msgr2=0x561f86e37dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:08:16.670 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.667+0000 7f1503fe8740 1 -- 172.21.15.47:0/1874428419 shutdown_connections 2024-04-17T06:08:16.670 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.668+0000 7f1503fe8740 1 -- 172.21.15.47:0/1874428419 wait complete. 2024-04-17T06:08:16.670 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.668+0000 7f1503fe8740 1 -- start start 2024-04-17T06:08:16.670 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.668+0000 7f1503fe8740 1 Processor -- start 2024-04-17T06:08:16.670 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.668+0000 7f1503fe8740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f86e3d0c0 0x561f86e4a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:16.670 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.668+0000 7f1503fe8740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561f86e3d9e0 0x561f86e4a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:16.670 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.668+0000 7f1503fe8740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f86e467b0 0x561f86e4ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:16.670 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.668+0000 7f1503fe8740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f14fc1ae3c0 con 0x561f86e3d9e0 2024-04-17T06:08:16.670 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.668+0000 7f1503fe8740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f14fc1af830 con 0x561f86e3d0c0 2024-04-17T06:08:16.670 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.668+0000 7f1503fe8740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f14fc1aea90 con 0x561f86e467b0 2024-04-17T06:08:16.670 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.668+0000 7f1503fe8740 10 client.0 _ll_register_callbacks cb 0x7f14f4037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:08:16.670 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.668+0000 7f14f27fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561f86e3d9e0 0x561f86e4a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:16.670 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.668+0000 7f14f27fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561f86e3d9e0 0x561f86e4a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50258/0 (socket says 172.21.15.47:50258) 2024-04-17T06:08:16.671 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.668+0000 7f14f27fc640 1 -- 172.21.15.47:0/3118050761 learned_addr learned my addr 172.21.15.47:0/3118050761 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:08:16.671 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.668+0000 7f14faffd640 1 --2- 172.21.15.47:0/3118050761 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f86e3d0c0 0x561f86e4a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:16.671 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.668+0000 7f14f27fc640 1 -- 172.21.15.47:0/3118050761 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f86e467b0 msgr2=0x561f86e4ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:08:16.671 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.668+0000 7f14f27fc640 1 --2- 172.21.15.47:0/3118050761 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f86e467b0 0x561f86e4ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:16.671 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.668+0000 7f14f27fc640 1 -- 172.21.15.47:0/3118050761 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f86e3d0c0 msgr2=0x561f86e4a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:08:16.671 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.668+0000 7f14f27fc640 1 --2- 172.21.15.47:0/3118050761 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f86e3d0c0 0x561f86e4a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:16.671 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.668+0000 7f14f27fc640 1 -- 172.21.15.47:0/3118050761 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f86ebc110 con 0x561f86e3d9e0 2024-04-17T06:08:16.671 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.669+0000 7f14f27fc640 1 --2- 172.21.15.47:0/3118050761 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561f86e3d9e0 0x561f86e4a610 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f14e4097010 tx=0x7f14e40ce2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a23cbf17fb5b388b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:08:16.671 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.669+0000 7f14e27fc640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-17T06:08:16.671 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.669+0000 7f14e27fc640 1 -- 172.21.15.47:0/3118050761 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e4018d30 con 0x561f86e3d9e0 2024-04-17T06:08:16.671 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.669+0000 7f14e27fc640 1 -- 172.21.15.47:0/3118050761 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8469+0+0 (secure 0 0 0) 0x7f14e409bfc0 con 0x561f86e3d9e0 2024-04-17T06:08:16.671 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.669+0000 7f1503fe8740 1 -- 172.21.15.47:0/3118050761 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14e40a57e0 con 0x561f86e3d9e0 2024-04-17T06:08:16.671 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.669+0000 7f14e27fc640 1 -- 172.21.15.47:0/3118050761 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f14e409c1c0 con 0x561f86e3d9e0 2024-04-17T06:08:16.671 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.669+0000 7f1503fe8740 10 client.15990 fetch_fsmap learned FSMap version 32 2024-04-17T06:08:16.672 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.669+0000 7f1503fe8740 1 -- 172.21.15.47:0/3118050761 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x561f86ebd320 con 0x561f86e3d9e0 2024-04-17T06:08:16.672 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.669+0000 7f14e27fc640 1 -- 172.21.15.47:0/3118050761 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14e40a57e0 con 0x561f86e3d9e0 2024-04-17T06:08:16.672 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.669+0000 7f14e27fc640 1 -- 172.21.15.47:0/3118050761 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) ==== 8339+0+0 (secure 0 0 0) 0x7f14e40b7070 con 0x561f86e3d9e0 2024-04-17T06:08:16.672 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.669+0000 7f1503fe8740 10 client.15990 fetch_fsmap finished waiting for FSMap version 32 2024-04-17T06:08:16.672 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.669+0000 7f1503fe8740 10 client.15990 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-17T06:08:16.672 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.669+0000 7f1503fe8740 20 client.15990 populate_metadata read hostname 'smithi047' 2024-04-17T06:08:16.672 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.669+0000 7f1503fe8740 1 --2- 172.21.15.47:0/3118050761 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x561f86ebda50 0x561f86ebfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:16.672 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.669+0000 7f1503fe8740 4 client.15990 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-17T06:08:16.672 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.669+0000 7f1503fe8740 1 -- 172.21.15.47:0/3118050761 --> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561f86daa060 con 0x561f86ebda50 2024-04-17T06:08:16.672 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.670+0000 7f14faffd640 1 --2- 172.21.15.47:0/3118050761 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x561f86ebda50 0x561f86ebfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:16.672 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.670+0000 7f14faffd640 1 --2- 172.21.15.47:0/3118050761 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x561f86ebda50 0x561f86ebfe70 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a12fba9604529f4 server_cookie=23251082a02a3492 in_seq=0 out_seq=0 2024-04-17T06:08:16.672 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.670+0000 7f14e27fc640 10 client.15990 ms_handle_connect on v2:172.21.15.98:6838/2841287917 2024-04-17T06:08:16.676 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.678+0000 7f14e27fc640 1 -- 172.21.15.47:0/3118050761 <== mds.0 v2:172.21.15.98:6838/2841287917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561f86daa060 con 0x561f86ebda50 2024-04-17T06:08:16.676 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.678+0000 7f14e27fc640 10 client.15990 handle_command_reply: tid=0 2024-04-17T06:08:16.676 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.678+0000 7f1503fe8740 1 client.15990 shutdown 2024-04-17T06:08:16.676 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.678+0000 7f1503fe8740 1 -- 172.21.15.47:0/3118050761 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561f86e3d9e0 msgr2=0x561f86e4a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:08:16.676 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.678+0000 7f1503fe8740 1 --2- 172.21.15.47:0/3118050761 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561f86e3d9e0 0x561f86e4a610 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f14e4097010 tx=0x7f14e40ce2c0 comp rx=0 tx=0).stop 2024-04-17T06:08:16.676 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.678+0000 7f1503fe8740 1 -- 172.21.15.47:0/3118050761 shutdown_connections 2024-04-17T06:08:16.676 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.678+0000 7f1503fe8740 1 --2- 172.21.15.47:0/3118050761 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x561f86ebda50 0x561f86ebfe70 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:16.676 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.678+0000 7f1503fe8740 1 --2- 172.21.15.47:0/3118050761 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f86e467b0 0x561f86e4ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:16.677 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.678+0000 7f14e27fc640 0 client.15990 ms_handle_reset on v2:172.21.15.98:6838/2841287917 2024-04-17T06:08:16.677 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.678+0000 7f1503fe8740 1 --2- 172.21.15.47:0/3118050761 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561f86e3d9e0 0x561f86e4a610 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:16.677 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.678+0000 7f1503fe8740 1 --2- 172.21.15.47:0/3118050761 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f86e3d0c0 0x561f86e4a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:16.677 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.678+0000 7f1503fe8740 1 -- 172.21.15.47:0/3118050761 >> 172.21.15.47:0/3118050761 conn(0x561f86e379e0 msgr2=0x561f86e43790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:08:16.677 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.678+0000 7f1503fe8740 1 -- 172.21.15.47:0/3118050761 shutdown_connections 2024-04-17T06:08:16.677 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.678+0000 7f1503fe8740 1 -- 172.21.15.47:0/3118050761 wait complete. 2024-04-17T06:08:16.677 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.678+0000 7f1503fe8740 20 client.15990 trim_cache size 0 max 16384 2024-04-17T06:08:16.677 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-04-17T06:08:16.677 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-04-17T06:08:16.677 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-04-17T06:08:16.677 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-04-17T06:08:16.678 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.678+0000 7f150131c640 1 -- 172.21.15.47:0/3758528644 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f14e80a3dc0 msgr2=0x7f14e80a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:08:16.678 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.678+0000 7f150131c640 1 --2- 172.21.15.47:0/3758528644 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f14e80a3dc0 0x7f14e80a6280 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f14f4005fa0 tx=0x7f14f40095c0 comp rx=0 tx=0).stop 2024-04-17T06:08:16.678 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.678+0000 7f150131c640 1 -- 172.21.15.47:0/3758528644 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f14fc06b170 msgr2=0x7f14fc1017c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:08:16.678 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.678+0000 7f150131c640 1 --2- 172.21.15.47:0/3758528644 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f14fc06b170 0x7f14fc1017c0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f14e400e9f0 tx=0x7f14e400eeb0 comp rx=0 tx=0).stop 2024-04-17T06:08:16.679 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.679+0000 7f150131c640 1 -- 172.21.15.47:0/3758528644 shutdown_connections 2024-04-17T06:08:16.679 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.679+0000 7f150131c640 1 --2- 172.21.15.47:0/3758528644 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f14fc108030 0x7f14fc101d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:16.679 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.679+0000 7f150131c640 1 --2- 172.21.15.47:0/3758528644 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f14e80a3dc0 0x7f14e80a6280 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:16.679 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.679+0000 7f150131c640 1 --2- 172.21.15.47:0/3758528644 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f14fc06b170 0x7f14fc1017c0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:16.679 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.679+0000 7f150131c640 1 --2- 172.21.15.47:0/3758528644 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14fc06a7d0 0x7f14fc101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:16.679 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.679+0000 7f150131c640 1 -- 172.21.15.47:0/3758528644 >> 172.21.15.47:0/3758528644 conn(0x7f14fc076c60 msgr2=0x7f14fc0ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:08:16.679 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.679+0000 7f150131c640 1 -- 172.21.15.47:0/3758528644 shutdown_connections 2024-04-17T06:08:16.679 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:16.679+0000 7f150131c640 1 -- 172.21.15.47:0/3758528644 wait complete. 2024-04-17T06:08:17.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:08:16 smithi103 ceph-mon[36243]: osdmap e94: 12 total, 12 up, 12 in 2024-04-17T06:08:17.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:08:16 smithi103 ceph-mon[36243]: mds.? [v2:172.21.15.98:6832/3380333999,v1:172.21.15.98:6833/3380333999] up:boot 2024-04-17T06:08:17.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:08:16 smithi103 ceph-mon[36243]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-04-17T06:08:17.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:08:16 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-17T06:08:17.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:08:16 smithi103 ceph-mon[36243]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-17T06:08:17.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:08:16 smithi047 ceph-mon[32223]: osdmap e94: 12 total, 12 up, 12 in 2024-04-17T06:08:17.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:08:16 smithi047 ceph-mon[32223]: mds.? [v2:172.21.15.98:6832/3380333999,v1:172.21.15.98:6833/3380333999] up:boot 2024-04-17T06:08:17.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:08:16 smithi047 ceph-mon[32223]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-04-17T06:08:17.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:08:16 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-17T06:08:17.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:08:16 smithi047 ceph-mon[32223]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-17T06:08:17.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:08:16 smithi098 ceph-mon[36516]: osdmap e94: 12 total, 12 up, 12 in 2024-04-17T06:08:17.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:08:16 smithi098 ceph-mon[36516]: mds.? [v2:172.21.15.98:6832/3380333999,v1:172.21.15.98:6833/3380333999] up:boot 2024-04-17T06:08:17.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:08:16 smithi098 ceph-mon[36516]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-04-17T06:08:17.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:08:16 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-17T06:08:17.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:08:16 smithi098 ceph-mon[36516]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-17T06:08:18.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:08:17 smithi103 ceph-mon[36243]: pgmap v217: 97 pgs: 97 active+clean; 3.6 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-04-17T06:08:18.382 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:08:17 smithi098 ceph-mon[36516]: pgmap v217: 97 pgs: 97 active+clean; 3.6 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-04-17T06:08:18.388 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:08:17 smithi047 ceph-mon[32223]: pgmap v217: 97 pgs: 97 active+clean; 3.6 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-04-17T06:08:20.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:08:19 smithi103 ceph-mon[36243]: pgmap v218: 97 pgs: 97 active+clean; 3.6 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6.6 KiB/s wr, 5 op/s 2024-04-17T06:08:20.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:08:19 smithi098 ceph-mon[36516]: pgmap v218: 97 pgs: 97 active+clean; 3.6 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6.6 KiB/s wr, 5 op/s 2024-04-17T06:08:20.388 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:08:19 smithi047 ceph-mon[32223]: pgmap v218: 97 pgs: 97 active+clean; 3.6 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6.6 KiB/s wr, 5 op/s 2024-04-17T06:08:22.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:08:21 smithi103 ceph-mon[36243]: pgmap v219: 97 pgs: 97 active+clean; 3.6 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-17T06:08:22.382 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:08:21 smithi098 ceph-mon[36516]: pgmap v219: 97 pgs: 97 active+clean; 3.6 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-17T06:08:22.388 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:08:21 smithi047 ceph-mon[32223]: pgmap v219: 97 pgs: 97 active+clean; 3.6 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-17T06:08:24.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:08:23 smithi103 ceph-mon[36243]: pgmap v220: 97 pgs: 97 active+clean; 3.6 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-17T06:08:24.382 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:08:23 smithi098 ceph-mon[36516]: pgmap v220: 97 pgs: 97 active+clean; 3.6 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-17T06:08:24.388 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:08:23 smithi047 ceph-mon[32223]: pgmap v220: 97 pgs: 97 active+clean; 3.6 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-17T06:08:26.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:08:25 smithi103 ceph-mon[36243]: pgmap v221: 97 pgs: 97 active+clean; 3.6 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-17T06:08:26.382 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:08:25 smithi098 ceph-mon[36516]: pgmap v221: 97 pgs: 97 active+clean; 3.6 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-17T06:08:26.388 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:08:25 smithi047 ceph-mon[32223]: pgmap v221: 97 pgs: 97 active+clean; 3.6 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-17T06:08:28.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:08:27 smithi103 ceph-mon[36243]: pgmap v222: 97 pgs: 97 active+clean; 3.6 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-04-17T06:08:28.382 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:08:27 smithi098 ceph-mon[36516]: pgmap v222: 97 pgs: 97 active+clean; 3.6 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-04-17T06:08:28.388 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:08:27 smithi047 ceph-mon[32223]: pgmap v222: 97 pgs: 97 active+clean; 3.6 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-04-17T06:08:29.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.768+0000 7f8f5d196640 1 -- 172.21.15.47:0/1681510803 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8f58101100 msgr2=0x7f8f581014e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:08:29.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.768+0000 7f8f5d196640 1 --2- 172.21.15.47:0/1681510803 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8f58101100 0x7f8f581014e0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f8f48009900 tx=0x7f8f4802bc30 comp rx=0 tx=0).stop 2024-04-17T06:08:29.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.768+0000 7f8f5d196640 1 -- 172.21.15.47:0/1681510803 shutdown_connections 2024-04-17T06:08:29.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.768+0000 7f8f5d196640 1 --2- 172.21.15.47:0/1681510803 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f58106180 0x7f8f5810dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:29.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.768+0000 7f8f5d196640 1 --2- 172.21.15.47:0/1681510803 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f58101ab0 0x7f8f58105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:29.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.768+0000 7f8f5d196640 1 --2- 172.21.15.47:0/1681510803 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8f58101100 0x7f8f581014e0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:29.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.768+0000 7f8f5d196640 1 -- 172.21.15.47:0/1681510803 >> 172.21.15.47:0/1681510803 conn(0x7f8f58077fe0 msgr2=0x7f8f580783f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:08:29.785 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.768+0000 7f8f5d196640 1 -- 172.21.15.47:0/1681510803 shutdown_connections 2024-04-17T06:08:29.785 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.768+0000 7f8f5d196640 1 -- 172.21.15.47:0/1681510803 wait complete. 2024-04-17T06:08:29.785 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.768+0000 7f8f5d196640 1 Processor -- start 2024-04-17T06:08:29.785 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.769+0000 7f8f5d196640 1 -- start start 2024-04-17T06:08:29.785 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.769+0000 7f8f5d196640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f58101100 0x7f8f581996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:29.786 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.769+0000 7f8f5d196640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8f58106180 0x7f8f58199be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:29.786 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.769+0000 7f8f5d196640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f5819a170 0x7f8f581a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:29.786 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.769+0000 7f8f5d196640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f8f58111240 con 0x7f8f58106180 2024-04-17T06:08:29.786 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.769+0000 7f8f5d196640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8f581110c0 con 0x7f8f58101100 2024-04-17T06:08:29.786 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.769+0000 7f8f5d196640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8f581113c0 con 0x7f8f5819a170 2024-04-17T06:08:29.786 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.769+0000 7f8f5659c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8f58106180 0x7f8f58199be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.769+0000 7f8f5659c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8f58106180 0x7f8f58199be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51854/0 (socket says 172.21.15.47:51854) 2024-04-17T06:08:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.769+0000 7f8f5659c640 1 -- 172.21.15.47:0/1114953815 learned_addr learned my addr 172.21.15.47:0/1114953815 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:08:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.769+0000 7f8f56d9d640 1 --2- 172.21.15.47:0/1114953815 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f58101100 0x7f8f581996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.769+0000 7f8f5759e640 1 --2- 172.21.15.47:0/1114953815 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f5819a170 0x7f8f581a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.769+0000 7f8f5659c640 1 -- 172.21.15.47:0/1114953815 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f5819a170 msgr2=0x7f8f581a4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:08:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.769+0000 7f8f5659c640 1 --2- 172.21.15.47:0/1114953815 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f5819a170 0x7f8f581a4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:29.788 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.769+0000 7f8f5659c640 1 -- 172.21.15.47:0/1114953815 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f58101100 msgr2=0x7f8f581996a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:08:29.788 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.769+0000 7f8f5659c640 1 --2- 172.21.15.47:0/1114953815 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f58101100 0x7f8f581996a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:29.788 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.769+0000 7f8f5659c640 1 -- 172.21.15.47:0/1114953815 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f581a4ef0 con 0x7f8f58106180 2024-04-17T06:08:29.788 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.770+0000 7f8f5659c640 1 --2- 172.21.15.47:0/1114953815 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8f58106180 0x7f8f58199be0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c00e9c0 tx=0x7f8f4c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:08:29.788 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.770+0000 7f8f3ffff640 1 -- 172.21.15.47:0/1114953815 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f4c00cd80 con 0x7f8f58106180 2024-04-17T06:08:29.788 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.770+0000 7f8f3ffff640 1 -- 172.21.15.47:0/1114953815 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f4c005500 con 0x7f8f58106180 2024-04-17T06:08:29.789 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.770+0000 7f8f5d196640 1 -- 172.21.15.47:0/1114953815 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f581a51e0 con 0x7f8f58106180 2024-04-17T06:08:29.789 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.770+0000 7f8f3ffff640 1 -- 172.21.15.47:0/1114953815 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f4c010690 con 0x7f8f58106180 2024-04-17T06:08:29.789 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.770+0000 7f8f5d196640 1 -- 172.21.15.47:0/1114953815 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f581a5730 con 0x7f8f58106180 2024-04-17T06:08:29.789 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.772+0000 7f8f3ffff640 1 -- 172.21.15.47:0/1114953815 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8f4c0046c0 con 0x7f8f58106180 2024-04-17T06:08:29.790 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.772+0000 7f8f5fd76740 1 Processor -- start 2024-04-17T06:08:29.790 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.773+0000 7f8f5fd76740 1 -- start start 2024-04-17T06:08:29.790 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.773+0000 7f8f5fd76740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dc801ef0c0 0x55dc801ef4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:29.790 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.773+0000 7f8f3ffff640 1 --2- 172.21.15.47:0/1114953815 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f8f400a3dc0 0x7f8f400a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:29.790 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.773+0000 7f8f5fd76740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dc801ef9e0 0x55dc801fc680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:29.790 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.773+0000 7f8f5fd76740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc801fcbc0 0x55dc801fefb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:29.791 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.773+0000 7f8f5fd76740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55dc801547d0 con 0x55dc801ef9e0 2024-04-17T06:08:29.791 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.773+0000 7f8f5fd76740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55dc800f1f70 con 0x55dc801ef0c0 2024-04-17T06:08:29.791 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.773+0000 7f8f5fd76740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55dc80150790 con 0x55dc801fcbc0 2024-04-17T06:08:29.791 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.773+0000 7f8f56d9d640 1 --2- 172.21.15.47:0/1114953815 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f8f400a3dc0 0x7f8f400a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:29.791 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.773+0000 7f8f5659c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dc801ef9e0 0x55dc801fc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.773+0000 7f8f5759e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dc801ef0c0 0x55dc801ef4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.773+0000 7f8f5759e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dc801ef0c0 0x55dc801ef4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.47:53072/0 (socket says 172.21.15.47:53072) 2024-04-17T06:08:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.773+0000 7f8f5659c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dc801ef9e0 0x55dc801fc680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51860/0 (socket says 172.21.15.47:51860) 2024-04-17T06:08:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.773+0000 7f8f5759e640 1 -- 172.21.15.47:0/2576184502 learned_addr learned my addr 172.21.15.47:0/2576184502 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:08:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.773+0000 7f8f3ffff640 1 -- 172.21.15.47:0/1114953815 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8469+0+0 (secure 0 0 0) 0x7f8f4c0d0b40 con 0x7f8f58106180 2024-04-17T06:08:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.773+0000 7f8f5759e640 1 --2- 172.21.15.47:0/2576184502 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc801fcbc0 0x55dc801fefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:29.793 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.773+0000 7f8f5759e640 1 -- 172.21.15.47:0/2576184502 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc801fcbc0 msgr2=0x55dc801fefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:08:29.793 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.773+0000 7f8f5759e640 1 --2- 172.21.15.47:0/2576184502 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc801fcbc0 0x55dc801fefb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:29.793 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.773+0000 7f8f5759e640 1 -- 172.21.15.47:0/2576184502 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dc801ef9e0 msgr2=0x55dc801fc680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:08:29.793 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.773+0000 7f8f5759e640 1 --2- 172.21.15.47:0/2576184502 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dc801ef9e0 0x55dc801fc680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:29.793 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.773+0000 7f8f5759e640 1 -- 172.21.15.47:0/2576184502 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc7fe67210 con 0x55dc801ef0c0 2024-04-17T06:08:29.793 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.774+0000 7f8f56d9d640 1 --2- 172.21.15.47:0/1114953815 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f8f400a3dc0 0x7f8f400a6280 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8f48006010 tx=0x7f8f48005dd0 comp rx=0 tx=0).ready entity=mgr.24923 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:08:29.794 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.774+0000 7f8f5759e640 1 --2- 172.21.15.47:0/2576184502 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dc801ef0c0 0x55dc801ef4a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8f50005c40 tx=0x7f8f50005bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c82afec9dcd8068 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:08:29.794 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.774+0000 7f8f3d7fa640 1 -- 172.21.15.47:0/2576184502 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f50011130 con 0x55dc801ef0c0 2024-04-17T06:08:29.794 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.774+0000 7f8f3d7fa640 1 -- 172.21.15.47:0/2576184502 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f500112d0 con 0x55dc801ef0c0 2024-04-17T06:08:29.794 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.774+0000 7f8f3d7fa640 1 -- 172.21.15.47:0/2576184502 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f5000fb90 con 0x55dc801ef0c0 2024-04-17T06:08:29.794 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.774+0000 7f8f5fd76740 1 -- 172.21.15.47:0/2576184502 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dc801ef0c0 msgr2=0x55dc801ef4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:08:29.794 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.774+0000 7f8f5fd76740 1 --2- 172.21.15.47:0/2576184502 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dc801ef0c0 0x55dc801ef4a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8f50005c40 tx=0x7f8f50005bd0 comp rx=0 tx=0).stop 2024-04-17T06:08:29.795 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.775+0000 7f8f5fd76740 1 -- 172.21.15.47:0/2576184502 shutdown_connections 2024-04-17T06:08:29.795 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.775+0000 7f8f5fd76740 1 --2- 172.21.15.47:0/2576184502 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc801fcbc0 0x55dc801fefb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:29.795 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.775+0000 7f8f5fd76740 1 --2- 172.21.15.47:0/2576184502 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dc801ef9e0 0x55dc801fc680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:29.795 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.775+0000 7f8f5fd76740 1 --2- 172.21.15.47:0/2576184502 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dc801ef0c0 0x55dc801ef4a0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:29.795 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.775+0000 7f8f5fd76740 1 -- 172.21.15.47:0/2576184502 >> 172.21.15.47:0/2576184502 conn(0x55dc801e99e0 msgr2=0x55dc801e9dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:08:29.795 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.775+0000 7f8f5fd76740 1 -- 172.21.15.47:0/2576184502 shutdown_connections 2024-04-17T06:08:29.796 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.775+0000 7f8f5fd76740 1 -- 172.21.15.47:0/2576184502 wait complete. 2024-04-17T06:08:29.796 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.775+0000 7f8f5fd76740 1 -- start start 2024-04-17T06:08:29.796 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.775+0000 7f8f5fd76740 1 Processor -- start 2024-04-17T06:08:29.796 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.775+0000 7f8f5fd76740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dc801ef0c0 0x55dc801f9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:29.796 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.775+0000 7f8f5fd76740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc801ef9e0 0x55dc801fa130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:29.796 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.775+0000 7f8f5fd76740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dc801fcbc0 0x55dc801fa670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:29.797 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.775+0000 7f8f5fd76740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55dc801ff7f0 con 0x55dc801ef0c0 2024-04-17T06:08:29.797 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.775+0000 7f8f5fd76740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55dc800dce90 con 0x55dc801fcbc0 2024-04-17T06:08:29.797 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.775+0000 7f8f5fd76740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55dc801ff970 con 0x55dc801ef9e0 2024-04-17T06:08:29.797 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.775+0000 7f8f5fd76740 10 client.0 _ll_register_callbacks cb 0x55dc800f2d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:08:29.797 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.775+0000 7f8f56d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dc801ef0c0 0x55dc801f9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:29.797 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.775+0000 7f8f56d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dc801ef0c0 0x55dc801f9bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51862/0 (socket says 172.21.15.47:51862) 2024-04-17T06:08:29.798 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.775+0000 7f8f5759e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dc801fcbc0 0x55dc801fa670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:29.798 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.775+0000 7f8f56d9d640 1 -- 172.21.15.47:0/3727695766 learned_addr learned my addr 172.21.15.47:0/3727695766 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:08:29.798 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.775+0000 7f8f5659c640 1 --2- 172.21.15.47:0/3727695766 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc801ef9e0 0x55dc801fa130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:29.798 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.776+0000 7f8f5659c640 1 -- 172.21.15.47:0/3727695766 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dc801fcbc0 msgr2=0x55dc801fa670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:08:29.798 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.776+0000 7f8f5659c640 1 --2- 172.21.15.47:0/3727695766 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dc801fcbc0 0x55dc801fa670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:29.799 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.776+0000 7f8f5659c640 1 -- 172.21.15.47:0/3727695766 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dc801ef0c0 msgr2=0x55dc801f9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:08:29.799 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.776+0000 7f8f5659c640 1 --2- 172.21.15.47:0/3727695766 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dc801ef0c0 0x55dc801f9bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:29.799 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.776+0000 7f8f5659c640 1 -- 172.21.15.47:0/3727695766 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc7ff33620 con 0x55dc801ef9e0 2024-04-17T06:08:29.799 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.776+0000 7f8f56d9d640 1 --2- 172.21.15.47:0/3727695766 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dc801ef0c0 0x55dc801f9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T06:08:29.799 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.776+0000 7f8f5659c640 1 --2- 172.21.15.47:0/3727695766 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc801ef9e0 0x55dc801fa130 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c0ce2c0 tx=0x7f8f4c004b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c1300454ea1bd493 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:08:29.799 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.776+0000 7f8f3aff5640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T06:08:29.800 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.776+0000 7f8f3aff5640 1 -- 172.21.15.47:0/3727695766 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f4c0b1040 con 0x55dc801ef9e0 2024-04-17T06:08:29.800 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.776+0000 7f8f5fd76740 1 -- 172.21.15.47:0/3727695766 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f500112d0 con 0x55dc801ef9e0 2024-04-17T06:08:29.800 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.776+0000 7f8f3aff5640 1 -- 172.21.15.47:0/3727695766 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8469+0+0 (secure 0 0 0) 0x7f8f4c0af400 con 0x55dc801ef9e0 2024-04-17T06:08:29.800 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.776+0000 7f8f3aff5640 1 -- 172.21.15.47:0/3727695766 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8f4c0b6050 con 0x55dc801ef9e0 2024-04-17T06:08:29.800 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.776+0000 7f8f5fd76740 10 client.25432 fetch_fsmap learned FSMap version 32 2024-04-17T06:08:29.800 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.776+0000 7f8f5fd76740 1 -- 172.21.15.47:0/3727695766 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55dc8020bab0 con 0x55dc801ef9e0 2024-04-17T06:08:29.801 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.776+0000 7f8f3aff5640 1 -- 172.21.15.47:0/3727695766 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f500112d0 con 0x55dc801ef9e0 2024-04-17T06:08:29.801 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.777+0000 7f8f3aff5640 1 -- 172.21.15.47:0/3727695766 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 32) ==== 8339+0+0 (secure 0 0 0) 0x7f8f4c099180 con 0x55dc801ef9e0 2024-04-17T06:08:29.801 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.777+0000 7f8f5fd76740 10 client.25432 fetch_fsmap finished waiting for FSMap version 32 2024-04-17T06:08:29.801 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.777+0000 7f8f5fd76740 10 client.25432 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-17T06:08:29.801 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.777+0000 7f8f5fd76740 20 client.25432 populate_metadata read hostname 'smithi047' 2024-04-17T06:08:29.801 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.777+0000 7f8f5fd76740 1 --2- 172.21.15.47:0/3727695766 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55dc8020cf20 0x55dc8020f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:29.802 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.777+0000 7f8f5fd76740 4 client.25432 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T06:08:29.802 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.777+0000 7f8f5fd76740 1 -- 172.21.15.47:0/3727695766 --> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dc801621f0 con 0x55dc8020cf20 2024-04-17T06:08:29.802 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.777+0000 7f8f56d9d640 1 --2- 172.21.15.47:0/3727695766 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55dc8020cf20 0x55dc8020f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:29.802 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.777+0000 7f8f56d9d640 1 --2- 172.21.15.47:0/3727695766 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55dc8020cf20 0x55dc8020f350 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a62b86a24ea3831 server_cookie=2a12144be99d706c in_seq=0 out_seq=0 2024-04-17T06:08:29.802 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.777+0000 7f8f3aff5640 10 client.25432 ms_handle_connect on v2:172.21.15.98:6838/2841287917 2024-04-17T06:08:29.803 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.778+0000 7f8f3aff5640 1 -- 172.21.15.47:0/3727695766 <== mds.0 v2:172.21.15.98:6838/2841287917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55dc801621f0 con 0x55dc8020cf20 2024-04-17T06:08:29.803 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.778+0000 7f8f3aff5640 10 client.25432 handle_command_reply: tid=0 2024-04-17T06:08:29.803 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.778+0000 7f8f5fd76740 1 client.25432 shutdown 2024-04-17T06:08:29.803 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.778+0000 7f8f5fd76740 1 -- 172.21.15.47:0/3727695766 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc801ef9e0 msgr2=0x55dc801fa130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:08:29.803 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.778+0000 7f8f5fd76740 1 --2- 172.21.15.47:0/3727695766 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc801ef9e0 0x55dc801fa130 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c0ce2c0 tx=0x7f8f4c004b30 comp rx=0 tx=0).stop 2024-04-17T06:08:29.803 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.778+0000 7f8f5fd76740 1 -- 172.21.15.47:0/3727695766 shutdown_connections 2024-04-17T06:08:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.778+0000 7f8f5fd76740 1 --2- 172.21.15.47:0/3727695766 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55dc8020cf20 0x55dc8020f350 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.778+0000 7f8f5fd76740 1 --2- 172.21.15.47:0/3727695766 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dc801fcbc0 0x55dc801fa670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.778+0000 7f8f3aff5640 0 client.25432 ms_handle_reset on v2:172.21.15.98:6838/2841287917 2024-04-17T06:08:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.778+0000 7f8f5fd76740 1 --2- 172.21.15.47:0/3727695766 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc801ef9e0 0x55dc801fa130 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.778+0000 7f8f5fd76740 1 --2- 172.21.15.47:0/3727695766 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dc801ef0c0 0x55dc801f9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.778+0000 7f8f5fd76740 1 -- 172.21.15.47:0/3727695766 >> 172.21.15.47:0/3727695766 conn(0x55dc801e99e0 msgr2=0x55dc801eb730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:08:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.778+0000 7f8f5fd76740 1 -- 172.21.15.47:0/3727695766 shutdown_connections 2024-04-17T06:08:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.778+0000 7f8f5fd76740 1 -- 172.21.15.47:0/3727695766 wait complete. 2024-04-17T06:08:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.778+0000 7f8f5fd76740 20 client.25432 trim_cache size 0 max 16384 2024-04-17T06:08:29.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.793+0000 7f8f5fd76740 1 Processor -- start 2024-04-17T06:08:29.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.793+0000 7f8f5fd76740 1 -- start start 2024-04-17T06:08:29.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.793+0000 7f8f5fd76740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dc801ef0c0 0x55dc801fc320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:29.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.793+0000 7f8f5fd76740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dc801ef9e0 0x55dc802086b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:29.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.793+0000 7f8f5fd76740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc801fcbc0 0x55dc80214c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:29.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.793+0000 7f8f5fd76740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f8f581ae280 con 0x55dc801ef0c0 2024-04-17T06:08:29.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.793+0000 7f8f5fd76740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8f581abb40 con 0x55dc801ef9e0 2024-04-17T06:08:29.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.793+0000 7f8f5fd76740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8f581a9990 con 0x55dc801fcbc0 2024-04-17T06:08:29.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.793+0000 7f8f56d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dc801ef0c0 0x55dc801fc320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:29.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.793+0000 7f8f56d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dc801ef0c0 0x55dc801fc320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51868/0 (socket says 172.21.15.47:51868) 2024-04-17T06:08:29.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.793+0000 7f8f56d9d640 1 -- 172.21.15.47:0/100234275 learned_addr learned my addr 172.21.15.47:0/100234275 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:08:29.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.793+0000 7f8f5759e640 1 --2- 172.21.15.47:0/100234275 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc801fcbc0 0x55dc80214c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:29.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.793+0000 7f8f5659c640 1 --2- 172.21.15.47:0/100234275 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dc801ef9e0 0x55dc802086b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:29.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.793+0000 7f8f56d9d640 1 -- 172.21.15.47:0/100234275 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc801fcbc0 msgr2=0x55dc80214c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:08:29.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.793+0000 7f8f56d9d640 1 --2- 172.21.15.47:0/100234275 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc801fcbc0 0x55dc80214c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:29.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.793+0000 7f8f56d9d640 1 -- 172.21.15.47:0/100234275 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dc801ef9e0 msgr2=0x55dc802086b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:08:29.815 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.793+0000 7f8f56d9d640 1 --2- 172.21.15.47:0/100234275 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dc801ef9e0 0x55dc802086b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:29.815 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.793+0000 7f8f56d9d640 1 -- 172.21.15.47:0/100234275 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc8020f9d0 con 0x55dc801ef0c0 2024-04-17T06:08:29.815 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.794+0000 7f8f56d9d640 1 --2- 172.21.15.47:0/100234275 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dc801ef0c0 0x55dc801fc320 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f8f48000c00 tx=0x7f8f4803f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a8f9ec74bc57012 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:08:29.815 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.794+0000 7f8f3cff9640 1 -- 172.21.15.47:0/100234275 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f48004180 con 0x55dc801ef0c0 2024-04-17T06:08:29.815 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.794+0000 7f8f3cff9640 1 -- 172.21.15.47:0/100234275 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f48004ba0 con 0x55dc801ef0c0 2024-04-17T06:08:29.815 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.794+0000 7f8f3cff9640 1 -- 172.21.15.47:0/100234275 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f48052820 con 0x55dc801ef0c0 2024-04-17T06:08:29.815 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.794+0000 7f8f5fd76740 1 -- 172.21.15.47:0/100234275 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dc801ef0c0 msgr2=0x55dc801fc320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:08:29.815 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.794+0000 7f8f5fd76740 1 --2- 172.21.15.47:0/100234275 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dc801ef0c0 0x55dc801fc320 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f8f48000c00 tx=0x7f8f4803f000 comp rx=0 tx=0).stop 2024-04-17T06:08:29.815 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.795+0000 7f8f5fd76740 1 -- 172.21.15.47:0/100234275 shutdown_connections 2024-04-17T06:08:29.815 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.795+0000 7f8f5fd76740 1 --2- 172.21.15.47:0/100234275 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc801fcbc0 0x55dc80214c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:29.816 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.795+0000 7f8f5fd76740 1 --2- 172.21.15.47:0/100234275 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dc801ef9e0 0x55dc802086b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:29.816 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.795+0000 7f8f5fd76740 1 --2- 172.21.15.47:0/100234275 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dc801ef0c0 0x55dc801fc320 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:29.816 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.795+0000 7f8f5fd76740 1 -- 172.21.15.47:0/100234275 >> 172.21.15.47:0/100234275 conn(0x55dc801e99e0 msgr2=0x55dc801e8b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:08:29.816 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.795+0000 7f8f5fd76740 1 -- 172.21.15.47:0/100234275 shutdown_connections 2024-04-17T06:08:29.816 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.795+0000 7f8f5fd76740 1 -- 172.21.15.47:0/100234275 wait complete. 2024-04-17T06:08:29.816 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.795+0000 7f8f5fd76740 1 -- start start 2024-04-17T06:08:29.816 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.795+0000 7f8f5fd76740 1 Processor -- start 2024-04-17T06:08:29.816 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.795+0000 7f8f5fd76740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc801ef0c0 0x55dc80204db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:29.816 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.795+0000 7f8f5fd76740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dc801ef9e0 0x55dc80201b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:29.817 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.795+0000 7f8f5fd76740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dc801fcbc0 0x55dc802020d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:29.817 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.795+0000 7f8f5fd76740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f8f581aa000 con 0x55dc801ef9e0 2024-04-17T06:08:29.817 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.795+0000 7f8f5fd76740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8f581ab470 con 0x55dc801fcbc0 2024-04-17T06:08:29.817 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.795+0000 7f8f5fd76740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8f581aa6d0 con 0x55dc801ef0c0 2024-04-17T06:08:29.817 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.795+0000 7f8f5fd76740 10 client.0 _ll_register_callbacks cb 0x7f8f4c009790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:08:29.817 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.795+0000 7f8f5659c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dc801ef9e0 0x55dc80201b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:29.818 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.795+0000 7f8f5659c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dc801ef9e0 0x55dc80201b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51870/0 (socket says 172.21.15.47:51870) 2024-04-17T06:08:29.818 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.795+0000 7f8f5659c640 1 -- 172.21.15.47:0/3694159749 learned_addr learned my addr 172.21.15.47:0/3694159749 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:08:29.818 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.795+0000 7f8f5759e640 1 --2- 172.21.15.47:0/3694159749 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dc801fcbc0 0x55dc802020d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:29.818 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.795+0000 7f8f56d9d640 1 --2- 172.21.15.47:0/3694159749 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc801ef0c0 0x55dc80204db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:29.818 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.796+0000 7f8f5659c640 1 -- 172.21.15.47:0/3694159749 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc801ef0c0 msgr2=0x55dc80204db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:08:29.819 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.796+0000 7f8f5659c640 1 --2- 172.21.15.47:0/3694159749 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc801ef0c0 0x55dc80204db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:29.819 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.796+0000 7f8f5659c640 1 -- 172.21.15.47:0/3694159749 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dc801fcbc0 msgr2=0x55dc802020d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:08:29.819 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.796+0000 7f8f5659c640 1 --2- 172.21.15.47:0/3694159749 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dc801fcbc0 0x55dc802020d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:29.819 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.796+0000 7f8f5659c640 1 -- 172.21.15.47:0/3694159749 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc801fefb0 con 0x55dc801ef9e0 2024-04-17T06:08:29.819 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.796+0000 7f8f5659c640 1 --2- 172.21.15.47:0/3694159749 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dc801ef9e0 0x55dc80201b90 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c09adc0 tx=0x7f8f4c0acd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=558d5d5411a054a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:08:29.819 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.796+0000 7f8f3aff5640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-17T06:08:29.820 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.796+0000 7f8f3aff5640 1 -- 172.21.15.47:0/3694159749 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f4c098db0 con 0x55dc801ef9e0 2024-04-17T06:08:29.820 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.796+0000 7f8f3aff5640 1 -- 172.21.15.47:0/3694159749 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8469+0+0 (secure 0 0 0) 0x7f8f4c0a52f0 con 0x55dc801ef9e0 2024-04-17T06:08:29.820 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.796+0000 7f8f5fd76740 1 -- 172.21.15.47:0/3694159749 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f48004ba0 con 0x55dc801ef9e0 2024-04-17T06:08:29.820 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.796+0000 7f8f3aff5640 1 -- 172.21.15.47:0/3694159749 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8f4c09c650 con 0x55dc801ef9e0 2024-04-17T06:08:29.820 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.796+0000 7f8f5fd76740 10 client.16008 fetch_fsmap learned FSMap version 32 2024-04-17T06:08:29.820 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.796+0000 7f8f5fd76740 1 -- 172.21.15.47:0/3694159749 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55dc80274070 con 0x55dc801ef9e0 2024-04-17T06:08:29.821 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.796+0000 7f8f3aff5640 1 -- 172.21.15.47:0/3694159749 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f48004ba0 con 0x55dc801ef9e0 2024-04-17T06:08:29.821 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.796+0000 7f8f3aff5640 1 -- 172.21.15.47:0/3694159749 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) ==== 8339+0+0 (secure 0 0 0) 0x7f8f4c09bf60 con 0x55dc801ef9e0 2024-04-17T06:08:29.821 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.796+0000 7f8f5fd76740 10 client.16008 fetch_fsmap finished waiting for FSMap version 32 2024-04-17T06:08:29.821 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.796+0000 7f8f5fd76740 10 client.16008 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-17T06:08:29.821 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.796+0000 7f8f5fd76740 20 client.16008 populate_metadata read hostname 'smithi047' 2024-04-17T06:08:29.821 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.796+0000 7f8f5fd76740 1 --2- 172.21.15.47:0/3694159749 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55dc802747a0 0x55dc80276bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:29.822 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.796+0000 7f8f5fd76740 4 client.16008 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-17T06:08:29.822 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.796+0000 7f8f5fd76740 1 -- 172.21.15.47:0/3694159749 --> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55dc8015c060 con 0x55dc802747a0 2024-04-17T06:08:29.822 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.797+0000 7f8f56d9d640 1 --2- 172.21.15.47:0/3694159749 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55dc802747a0 0x55dc80276bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:29.822 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.797+0000 7f8f3aff5640 10 client.16008 ms_handle_connect on v2:172.21.15.98:6838/2841287917 2024-04-17T06:08:29.822 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.797+0000 7f8f56d9d640 1 --2- 172.21.15.47:0/3694159749 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55dc802747a0 0x55dc80276bc0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=caab4d5a3e1cca64 server_cookie=11cbdf26f81fca2e in_seq=0 out_seq=0 2024-04-17T06:08:29.822 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.805+0000 7f8f3aff5640 1 -- 172.21.15.47:0/3694159749 <== mds.0 v2:172.21.15.98:6838/2841287917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55dc8015c060 con 0x55dc802747a0 2024-04-17T06:08:29.823 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.805+0000 7f8f3aff5640 10 client.16008 handle_command_reply: tid=0 2024-04-17T06:08:29.823 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.805+0000 7f8f5fd76740 1 client.16008 shutdown 2024-04-17T06:08:29.823 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.805+0000 7f8f5fd76740 1 -- 172.21.15.47:0/3694159749 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dc801ef9e0 msgr2=0x55dc80201b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:08:29.823 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.805+0000 7f8f5fd76740 1 --2- 172.21.15.47:0/3694159749 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dc801ef9e0 0x55dc80201b90 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c09adc0 tx=0x7f8f4c0acd70 comp rx=0 tx=0).stop 2024-04-17T06:08:29.823 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.805+0000 7f8f5fd76740 1 -- 172.21.15.47:0/3694159749 shutdown_connections 2024-04-17T06:08:29.823 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.805+0000 7f8f5fd76740 1 --2- 172.21.15.47:0/3694159749 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55dc802747a0 0x55dc80276bc0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:29.824 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.805+0000 7f8f5fd76740 1 --2- 172.21.15.47:0/3694159749 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dc801fcbc0 0x55dc802020d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:29.824 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.805+0000 7f8f5fd76740 1 --2- 172.21.15.47:0/3694159749 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dc801ef9e0 0x55dc80201b90 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:29.824 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.805+0000 7f8f3aff5640 0 client.16008 ms_handle_reset on v2:172.21.15.98:6838/2841287917 2024-04-17T06:08:29.824 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.805+0000 7f8f5fd76740 1 --2- 172.21.15.47:0/3694159749 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc801ef0c0 0x55dc80204db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:29.824 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.805+0000 7f8f5fd76740 1 -- 172.21.15.47:0/3694159749 >> 172.21.15.47:0/3694159749 conn(0x55dc801e99e0 msgr2=0x55dc801f9bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:08:29.824 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.805+0000 7f8f5fd76740 1 -- 172.21.15.47:0/3694159749 shutdown_connections 2024-04-17T06:08:29.825 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.805+0000 7f8f5fd76740 1 -- 172.21.15.47:0/3694159749 wait complete. 2024-04-17T06:08:29.825 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.805+0000 7f8f5fd76740 20 client.16008 trim_cache size 0 max 16384 2024-04-17T06:08:29.825 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-04-17T06:08:29.825 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-04-17T06:08:29.826 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-04-17T06:08:29.826 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-04-17T06:08:29.827 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.806+0000 7f8f5d196640 1 -- 172.21.15.47:0/1114953815 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f8f400a3dc0 msgr2=0x7f8f400a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:08:29.827 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.806+0000 7f8f5d196640 1 --2- 172.21.15.47:0/1114953815 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f8f400a3dc0 0x7f8f400a6280 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8f48006010 tx=0x7f8f48005dd0 comp rx=0 tx=0).stop 2024-04-17T06:08:29.827 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.806+0000 7f8f5d196640 1 -- 172.21.15.47:0/1114953815 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8f58106180 msgr2=0x7f8f58199be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:08:29.827 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.806+0000 7f8f5d196640 1 --2- 172.21.15.47:0/1114953815 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8f58106180 0x7f8f58199be0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c00e9c0 tx=0x7f8f4c00ee80 comp rx=0 tx=0).stop 2024-04-17T06:08:29.827 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.806+0000 7f8f5d196640 1 -- 172.21.15.47:0/1114953815 shutdown_connections 2024-04-17T06:08:29.827 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.806+0000 7f8f5d196640 1 --2- 172.21.15.47:0/1114953815 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f5819a170 0x7f8f581a4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:29.827 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.806+0000 7f8f5d196640 1 --2- 172.21.15.47:0/1114953815 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f8f400a3dc0 0x7f8f400a6280 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:29.828 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.806+0000 7f8f5d196640 1 --2- 172.21.15.47:0/1114953815 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8f58106180 0x7f8f58199be0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:29.828 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.806+0000 7f8f5d196640 1 --2- 172.21.15.47:0/1114953815 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f58101100 0x7f8f581996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:29.828 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.806+0000 7f8f5d196640 1 -- 172.21.15.47:0/1114953815 >> 172.21.15.47:0/1114953815 conn(0x7f8f58077fe0 msgr2=0x7f8f580fefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:08:29.828 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.806+0000 7f8f5d196640 1 -- 172.21.15.47:0/1114953815 shutdown_connections 2024-04-17T06:08:29.828 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:29.806+0000 7f8f5d196640 1 -- 172.21.15.47:0/1114953815 wait complete. 2024-04-17T06:08:30.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:08:29 smithi103 ceph-mon[36243]: pgmap v223: 97 pgs: 97 active+clean; 3.6 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:08:30.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:08:29 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T06:08:30.382 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:08:29 smithi098 ceph-mon[36516]: pgmap v223: 97 pgs: 97 active+clean; 3.6 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:08:30.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:08:29 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T06:08:30.388 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:08:29 smithi047 ceph-mon[32223]: pgmap v223: 97 pgs: 97 active+clean; 3.6 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:08:30.388 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:08:29 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T06:08:31.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:08:30 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T06:08:31.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:08:30 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T06:08:31.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:08:30 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:08:31.382 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:08:30 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T06:08:31.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:08:30 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T06:08:31.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:08:30 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:08:31.388 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:08:30 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T06:08:31.388 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:08:30 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T06:08:31.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:08:30 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:08:32.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:08:32 smithi103 ceph-mon[36243]: pgmap v224: 97 pgs: 97 active+clean; 3.7 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-04-17T06:08:32.382 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:08:32 smithi098 ceph-mon[36516]: pgmap v224: 97 pgs: 97 active+clean; 3.7 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-04-17T06:08:32.388 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:08:32 smithi047 ceph-mon[32223]: pgmap v224: 97 pgs: 97 active+clean; 3.7 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-04-17T06:08:34.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:08:34 smithi103 ceph-mon[36243]: pgmap v225: 97 pgs: 97 active+clean; 3.7 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.8 KiB/s wr, 9 op/s 2024-04-17T06:08:34.382 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:08:34 smithi098 ceph-mon[36516]: pgmap v225: 97 pgs: 97 active+clean; 3.7 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.8 KiB/s wr, 9 op/s 2024-04-17T06:08:34.388 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:08:34 smithi047 ceph-mon[32223]: pgmap v225: 97 pgs: 97 active+clean; 3.7 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.8 KiB/s wr, 9 op/s 2024-04-17T06:08:36.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:08:36 smithi103 ceph-mon[36243]: pgmap v226: 97 pgs: 97 active+clean; 3.7 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 16 KiB/s wr, 15 op/s 2024-04-17T06:08:36.382 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:08:36 smithi098 ceph-mon[36516]: pgmap v226: 97 pgs: 97 active+clean; 3.7 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 16 KiB/s wr, 15 op/s 2024-04-17T06:08:36.388 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:08:36 smithi047 ceph-mon[32223]: pgmap v226: 97 pgs: 97 active+clean; 3.7 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 16 KiB/s wr, 15 op/s 2024-04-17T06:08:38.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:08:38 smithi103 ceph-mon[36243]: pgmap v227: 97 pgs: 97 active+clean; 3.7 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:08:38.382 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:08:38 smithi098 ceph-mon[36516]: pgmap v227: 97 pgs: 97 active+clean; 3.7 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:08:38.388 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:08:38 smithi047 ceph-mon[32223]: pgmap v227: 97 pgs: 97 active+clean; 3.7 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:08:40.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:08:40 smithi103 ceph-mon[36243]: pgmap v228: 97 pgs: 97 active+clean; 3.7 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:08:40.382 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:08:40 smithi098 ceph-mon[36516]: pgmap v228: 97 pgs: 97 active+clean; 3.7 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:08:40.388 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:08:40 smithi047 ceph-mon[32223]: pgmap v228: 97 pgs: 97 active+clean; 3.7 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:08:42.382 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:08:42 smithi098 ceph-mon[36516]: pgmap v229: 97 pgs: 97 active+clean; 3.7 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-04-17T06:08:42.388 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:08:42 smithi047 ceph-mon[32223]: pgmap v229: 97 pgs: 97 active+clean; 3.7 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-04-17T06:08:42.533 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:08:42 smithi103 ceph-mon[36243]: pgmap v229: 97 pgs: 97 active+clean; 3.7 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-04-17T06:08:42.907 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.907+0000 7feee2b04640 1 -- 172.21.15.47:0/2389198512 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feedc10e940 msgr2=0x7feedc110d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:08:42.907 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.907+0000 7feee2b04640 1 --2- 172.21.15.47:0/2389198512 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feedc10e940 0x7feedc110d30 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7feecc009900 tx=0x7feecc02bc30 comp rx=0 tx=0).stop 2024-04-17T06:08:42.907 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.907+0000 7feee2b04640 1 -- 172.21.15.47:0/2389198512 shutdown_connections 2024-04-17T06:08:42.907 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.907+0000 7feee2b04640 1 --2- 172.21.15.47:0/2389198512 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feedc10e940 0x7feedc110d30 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:42.907 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.907+0000 7feee2b04640 1 --2- 172.21.15.47:0/2389198512 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feedc1016b0 0x7feedc10e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:42.907 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.907+0000 7feee2b04640 1 --2- 172.21.15.47:0/2389198512 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feedc100d90 0x7feedc101170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:42.907 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.907+0000 7feee2b04640 1 -- 172.21.15.47:0/2389198512 >> 172.21.15.47:0/2389198512 conn(0x7feedc077fe0 msgr2=0x7feedc0783f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:08:42.908 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.907+0000 7feee2b04640 1 -- 172.21.15.47:0/2389198512 shutdown_connections 2024-04-17T06:08:42.908 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.907+0000 7feee2b04640 1 -- 172.21.15.47:0/2389198512 wait complete. 2024-04-17T06:08:42.908 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.908+0000 7feee2b04640 1 Processor -- start 2024-04-17T06:08:42.908 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.908+0000 7feee2b04640 1 -- start start 2024-04-17T06:08:42.908 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.908+0000 7feee2b04640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feedc100d90 0x7feedc1a0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:42.908 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.908+0000 7feee2b04640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feedc1016b0 0x7feedc1a0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:42.908 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.908+0000 7feee2b04640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feedc10e940 0x7feedc19a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:42.908 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.908+0000 7feee2b04640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7feedc1144f0 con 0x7feedc1016b0 2024-04-17T06:08:42.908 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.908+0000 7feee2b04640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7feedc114370 con 0x7feedc100d90 2024-04-17T06:08:42.908 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.908+0000 7feee2b04640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7feedc114670 con 0x7feedc10e940 2024-04-17T06:08:42.908 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.908+0000 7feedbfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feedc1016b0 0x7feedc1a0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:42.908 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.908+0000 7feedbfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feedc1016b0 0x7feedc1a0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39200/0 (socket says 172.21.15.47:39200) 2024-04-17T06:08:42.908 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.908+0000 7feedbfff640 1 -- 172.21.15.47:0/2010996717 learned_addr learned my addr 172.21.15.47:0/2010996717 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:08:42.908 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.908+0000 7feee08a0640 1 --2- 172.21.15.47:0/2010996717 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feedc100d90 0x7feedc1a0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:42.909 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.908+0000 7feee10a1640 1 --2- 172.21.15.47:0/2010996717 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feedc10e940 0x7feedc19a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:42.909 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.908+0000 7feedbfff640 1 -- 172.21.15.47:0/2010996717 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feedc10e940 msgr2=0x7feedc19a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:08:42.909 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.908+0000 7feedbfff640 1 --2- 172.21.15.47:0/2010996717 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feedc10e940 0x7feedc19a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:42.909 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.908+0000 7feedbfff640 1 -- 172.21.15.47:0/2010996717 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feedc100d90 msgr2=0x7feedc1a0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:08:42.909 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.908+0000 7feedbfff640 1 --2- 172.21.15.47:0/2010996717 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feedc100d90 0x7feedc1a0570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:42.909 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.908+0000 7feedbfff640 1 -- 172.21.15.47:0/2010996717 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feedc19aef0 con 0x7feedc1016b0 2024-04-17T06:08:42.909 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.909+0000 7feedbfff640 1 --2- 172.21.15.47:0/2010996717 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feedc1016b0 0x7feedc1a0ab0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7feed000c920 tx=0x7feed000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:08:42.909 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.909+0000 7feed9ffb640 1 -- 172.21.15.47:0/2010996717 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed0007c70 con 0x7feedc1016b0 2024-04-17T06:08:42.909 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.909+0000 7feed9ffb640 1 -- 172.21.15.47:0/2010996717 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feed0007e10 con 0x7feedc1016b0 2024-04-17T06:08:42.909 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.909+0000 7feee2b04640 1 -- 172.21.15.47:0/2010996717 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feedc19b1e0 con 0x7feedc1016b0 2024-04-17T06:08:42.909 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.909+0000 7feed9ffb640 1 -- 172.21.15.47:0/2010996717 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed0018890 con 0x7feedc1016b0 2024-04-17T06:08:42.909 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.909+0000 7feee2b04640 1 -- 172.21.15.47:0/2010996717 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feedc1a74f0 con 0x7feedc1016b0 2024-04-17T06:08:42.919 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.911+0000 7feed9ffb640 1 -- 172.21.15.47:0/2010996717 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7feed0018a70 con 0x7feedc1016b0 2024-04-17T06:08:42.920 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.912+0000 7feee57c9740 1 Processor -- start 2024-04-17T06:08:42.920 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.912+0000 7feee57c9740 1 -- start start 2024-04-17T06:08:42.920 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.912+0000 7feed9ffb640 1 --2- 172.21.15.47:0/2010996717 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7feec40a3dc0 0x7feec40a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:42.920 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.912+0000 7feee57c9740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e336a140c0 0x55e336a144a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:42.920 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.912+0000 7feee57c9740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e336a149e0 0x55e336a1d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:42.920 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.912+0000 7feee57c9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e336a1d7b0 0x55e336a1fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:42.920 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.912+0000 7feee57c9740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55e3369797d0 con 0x55e336a149e0 2024-04-17T06:08:42.920 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.912+0000 7feee57c9740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55e336916f70 con 0x55e336a140c0 2024-04-17T06:08:42.920 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.912+0000 7feee57c9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55e336975790 con 0x55e336a1d7b0 2024-04-17T06:08:42.920 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.912+0000 7feee10a1640 1 --2- 172.21.15.47:0/2010996717 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7feec40a3dc0 0x7feec40a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:42.920 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.912+0000 7feed9ffb640 1 -- 172.21.15.47:0/2010996717 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8469+0+0 (secure 0 0 0) 0x7feed00d0990 con 0x7feedc1016b0 2024-04-17T06:08:42.920 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.912+0000 7feedbfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e336a149e0 0x55e336a1d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:42.920 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.913+0000 7feedbfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e336a149e0 0x55e336a1d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39208/0 (socket says 172.21.15.47:39208) 2024-04-17T06:08:42.921 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.913+0000 7feedbfff640 1 -- 172.21.15.47:0/3538427784 learned_addr learned my addr 172.21.15.47:0/3538427784 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:08:42.921 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.913+0000 7feee08a0640 1 --2- 172.21.15.47:0/3538427784 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e336a140c0 0x55e336a144a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:42.921 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.913+0000 7feee10a1640 1 --2- 172.21.15.47:0/3538427784 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e336a1d7b0 0x55e336a1fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:42.921 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.913+0000 7feedbfff640 1 -- 172.21.15.47:0/3538427784 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e336a1d7b0 msgr2=0x55e336a1fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:08:42.921 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.913+0000 7feedbfff640 1 --2- 172.21.15.47:0/3538427784 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e336a1d7b0 0x55e336a1fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:42.921 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.913+0000 7feedbfff640 1 -- 172.21.15.47:0/3538427784 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e336a140c0 msgr2=0x55e336a144a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:08:42.921 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.913+0000 7feedbfff640 1 --2- 172.21.15.47:0/3538427784 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e336a140c0 0x55e336a144a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:42.921 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.913+0000 7feedbfff640 1 -- 172.21.15.47:0/3538427784 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e33668c210 con 0x55e336a149e0 2024-04-17T06:08:42.921 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.913+0000 7feee10a1640 1 --2- 172.21.15.47:0/2010996717 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7feec40a3dc0 0x7feec40a6280 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7feecc036040 tx=0x7feecc02cac0 comp rx=0 tx=0).ready entity=mgr.24923 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:08:42.921 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.913+0000 7feedbfff640 1 --2- 172.21.15.47:0/3538427784 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e336a149e0 0x55e336a1d270 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7feed00992d0 tx=0x7feed009c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c51682057d4f26 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:08:42.921 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.913+0000 7feec2ffd640 1 -- 172.21.15.47:0/3538427784 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed00a5570 con 0x55e336a149e0 2024-04-17T06:08:42.921 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.913+0000 7feec2ffd640 1 -- 172.21.15.47:0/3538427784 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feed009a9a0 con 0x55e336a149e0 2024-04-17T06:08:42.921 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.913+0000 7feec2ffd640 1 -- 172.21.15.47:0/3538427784 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed00ce330 con 0x55e336a149e0 2024-04-17T06:08:42.921 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.914+0000 7feee57c9740 1 -- 172.21.15.47:0/3538427784 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e336a149e0 msgr2=0x55e336a1d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:08:42.922 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.914+0000 7feee57c9740 1 --2- 172.21.15.47:0/3538427784 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e336a149e0 0x55e336a1d270 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7feed00992d0 tx=0x7feed009c390 comp rx=0 tx=0).stop 2024-04-17T06:08:42.922 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.914+0000 7feee57c9740 1 -- 172.21.15.47:0/3538427784 shutdown_connections 2024-04-17T06:08:42.922 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.914+0000 7feee57c9740 1 --2- 172.21.15.47:0/3538427784 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e336a1d7b0 0x55e336a1fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:42.922 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.914+0000 7feee57c9740 1 --2- 172.21.15.47:0/3538427784 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e336a149e0 0x55e336a1d270 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:42.922 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.914+0000 7feee57c9740 1 --2- 172.21.15.47:0/3538427784 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e336a140c0 0x55e336a144a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:42.922 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.914+0000 7feee57c9740 1 -- 172.21.15.47:0/3538427784 >> 172.21.15.47:0/3538427784 conn(0x55e336a0e9e0 msgr2=0x55e336a0edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:08:42.922 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.914+0000 7feee57c9740 1 -- 172.21.15.47:0/3538427784 shutdown_connections 2024-04-17T06:08:42.922 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.914+0000 7feee57c9740 1 -- 172.21.15.47:0/3538427784 wait complete. 2024-04-17T06:08:42.922 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.914+0000 7feee57c9740 1 -- start start 2024-04-17T06:08:42.922 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.914+0000 7feee57c9740 1 Processor -- start 2024-04-17T06:08:42.922 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.915+0000 7feee57c9740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e336a140c0 0x55e336a1a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:42.922 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.915+0000 7feee57c9740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e336a149e0 0x55e336a1acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:42.922 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.915+0000 7feee57c9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e336a1d7b0 0x55e336a1b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:42.922 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.915+0000 7feee57c9740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55e336a203e0 con 0x55e336a149e0 2024-04-17T06:08:42.923 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.915+0000 7feee57c9740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55e336901e90 con 0x55e336a140c0 2024-04-17T06:08:42.923 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.915+0000 7feee57c9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55e336a20560 con 0x55e336a1d7b0 2024-04-17T06:08:42.923 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.915+0000 7feee57c9740 10 client.0 _ll_register_callbacks cb 0x55e336917d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:08:42.923 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.915+0000 7feedbfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e336a149e0 0x55e336a1acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:42.923 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.915+0000 7feedbfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e336a149e0 0x55e336a1acf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39212/0 (socket says 172.21.15.47:39212) 2024-04-17T06:08:42.923 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.915+0000 7feedbfff640 1 -- 172.21.15.47:0/3802669879 learned_addr learned my addr 172.21.15.47:0/3802669879 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:08:42.923 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.915+0000 7feee10a1640 1 --2- 172.21.15.47:0/3802669879 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e336a1d7b0 0x55e336a1b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:42.923 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.915+0000 7feee08a0640 1 --2- 172.21.15.47:0/3802669879 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e336a140c0 0x55e336a1a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:42.923 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.915+0000 7feedbfff640 1 -- 172.21.15.47:0/3802669879 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e336a1d7b0 msgr2=0x55e336a1b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:08:42.923 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.915+0000 7feedbfff640 1 --2- 172.21.15.47:0/3802669879 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e336a1d7b0 0x55e336a1b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:42.923 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.915+0000 7feedbfff640 1 -- 172.21.15.47:0/3802669879 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e336a140c0 msgr2=0x55e336a1a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:08:42.923 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.915+0000 7feedbfff640 1 --2- 172.21.15.47:0/3802669879 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e336a140c0 0x55e336a1a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:42.923 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.915+0000 7feedbfff640 1 -- 172.21.15.47:0/3802669879 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e336758620 con 0x55e336a149e0 2024-04-17T06:08:42.923 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.915+0000 7feee10a1640 1 --2- 172.21.15.47:0/3802669879 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e336a1d7b0 0x55e336a1b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:08:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.915+0000 7feee08a0640 1 --2- 172.21.15.47:0/3802669879 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e336a140c0 0x55e336a1a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:08:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.916+0000 7feedbfff640 1 --2- 172.21.15.47:0/3802669879 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e336a149e0 0x55e336a1acf0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7feed00a5040 tx=0x7feed0018f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=657722f74b5630b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:08:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.916+0000 7feebbfff640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-17T06:08:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.916+0000 7feebbfff640 1 -- 172.21.15.47:0/3802669879 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed00a5570 con 0x55e336a149e0 2024-04-17T06:08:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.916+0000 7feebbfff640 1 -- 172.21.15.47:0/3802669879 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8469+0+0 (secure 0 0 0) 0x7feed00ce330 con 0x55e336a149e0 2024-04-17T06:08:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.916+0000 7feee57c9740 1 -- 172.21.15.47:0/3802669879 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feed009a9a0 con 0x55e336a149e0 2024-04-17T06:08:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.916+0000 7feebbfff640 1 -- 172.21.15.47:0/3802669879 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7feed00ce530 con 0x55e336a149e0 2024-04-17T06:08:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.916+0000 7feee57c9740 10 client.16020 fetch_fsmap learned FSMap version 32 2024-04-17T06:08:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.916+0000 7feee57c9740 1 -- 172.21.15.47:0/3802669879 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55e336a2c640 con 0x55e336a149e0 2024-04-17T06:08:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.916+0000 7feebbfff640 1 -- 172.21.15.47:0/3802669879 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feed009a9a0 con 0x55e336a149e0 2024-04-17T06:08:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.916+0000 7feebbfff640 1 -- 172.21.15.47:0/3802669879 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) ==== 8339+0+0 (secure 0 0 0) 0x7feed009bd50 con 0x55e336a149e0 2024-04-17T06:08:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.916+0000 7feee57c9740 10 client.16020 fetch_fsmap finished waiting for FSMap version 32 2024-04-17T06:08:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.916+0000 7feee57c9740 10 client.16020 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-17T06:08:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.916+0000 7feee57c9740 20 client.16020 populate_metadata read hostname 'smithi047' 2024-04-17T06:08:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.916+0000 7feee57c9740 1 --2- 172.21.15.47:0/3802669879 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55e336a2d990 0x55e336a2fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.916+0000 7feee57c9740 4 client.16020 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T06:08:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.916+0000 7feee57c9740 1 -- 172.21.15.47:0/3802669879 --> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e3369871f0 con 0x55e336a2d990 2024-04-17T06:08:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.917+0000 7feee08a0640 1 --2- 172.21.15.47:0/3802669879 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55e336a2d990 0x55e336a2fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.917+0000 7feee08a0640 1 --2- 172.21.15.47:0/3802669879 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55e336a2d990 0x55e336a2fd80 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73da3fdf2bc7528a server_cookie=6368069977742315 in_seq=0 out_seq=0 2024-04-17T06:08:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.917+0000 7feebbfff640 10 client.16020 ms_handle_connect on v2:172.21.15.98:6838/2841287917 2024-04-17T06:08:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.918+0000 7feebbfff640 1 -- 172.21.15.47:0/3802669879 <== mds.0 v2:172.21.15.98:6838/2841287917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e3369871f0 con 0x55e336a2d990 2024-04-17T06:08:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.918+0000 7feebbfff640 10 client.16020 handle_command_reply: tid=0 2024-04-17T06:08:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.918+0000 7feee57c9740 1 client.16020 shutdown 2024-04-17T06:08:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.918+0000 7feee57c9740 1 -- 172.21.15.47:0/3802669879 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e336a149e0 msgr2=0x55e336a1acf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:08:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.918+0000 7feee57c9740 1 --2- 172.21.15.47:0/3802669879 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e336a149e0 0x55e336a1acf0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7feed00a5040 tx=0x7feed0018f90 comp rx=0 tx=0).stop 2024-04-17T06:08:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.918+0000 7feee57c9740 1 -- 172.21.15.47:0/3802669879 shutdown_connections 2024-04-17T06:08:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.918+0000 7feee57c9740 1 --2- 172.21.15.47:0/3802669879 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55e336a2d990 0x55e336a2fd80 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.918+0000 7feee57c9740 1 --2- 172.21.15.47:0/3802669879 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e336a1d7b0 0x55e336a1b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.918+0000 7feee57c9740 1 --2- 172.21.15.47:0/3802669879 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e336a149e0 0x55e336a1acf0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:42.926 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.918+0000 7feee57c9740 1 --2- 172.21.15.47:0/3802669879 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e336a140c0 0x55e336a1a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:42.926 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.918+0000 7feebbfff640 0 client.16020 ms_handle_reset on v2:172.21.15.98:6838/2841287917 2024-04-17T06:08:42.926 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.918+0000 7feee57c9740 1 -- 172.21.15.47:0/3802669879 >> 172.21.15.47:0/3802669879 conn(0x55e336a0e9e0 msgr2=0x55e336a10730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:08:42.926 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.918+0000 7feee57c9740 1 -- 172.21.15.47:0/3802669879 shutdown_connections 2024-04-17T06:08:42.926 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.918+0000 7feee57c9740 1 -- 172.21.15.47:0/3802669879 wait complete. 2024-04-17T06:08:42.926 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.918+0000 7feee57c9740 20 client.16020 trim_cache size 0 max 16384 2024-04-17T06:08:42.933 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.933+0000 7feee57c9740 1 Processor -- start 2024-04-17T06:08:42.933 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.933+0000 7feee57c9740 1 -- start start 2024-04-17T06:08:42.933 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.933+0000 7feee57c9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e336a140c0 0x55e336a1cee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:42.933 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.933+0000 7feee57c9740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e336a149e0 0x55e336a10540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:42.933 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.934+0000 7feee57c9740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e336a1d7b0 0x55e336a32da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:42.933 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.934+0000 7feee57c9740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7feedc1b2510 con 0x55e336a149e0 2024-04-17T06:08:42.933 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.934+0000 7feee57c9740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7feedc1afdd0 con 0x55e336a1d7b0 2024-04-17T06:08:42.933 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.934+0000 7feee57c9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7feedc1adc20 con 0x55e336a140c0 2024-04-17T06:08:42.933 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.934+0000 7feedbfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e336a149e0 0x55e336a10540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:42.933 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.934+0000 7feedbfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e336a149e0 0x55e336a10540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39214/0 (socket says 172.21.15.47:39214) 2024-04-17T06:08:42.933 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.934+0000 7feedbfff640 1 -- 172.21.15.47:0/3413449061 learned_addr learned my addr 172.21.15.47:0/3413449061 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:08:42.934 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.934+0000 7feee08a0640 1 --2- 172.21.15.47:0/3413449061 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e336a140c0 0x55e336a1cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:42.934 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.934+0000 7feedbfff640 1 -- 172.21.15.47:0/3413449061 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e336a140c0 msgr2=0x55e336a1cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:08:42.934 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.934+0000 7feedbfff640 1 --2- 172.21.15.47:0/3413449061 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e336a140c0 0x55e336a1cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:42.934 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.934+0000 7feedbfff640 1 -- 172.21.15.47:0/3413449061 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e336a1d7b0 msgr2=0x55e336a32da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:08:42.934 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.934+0000 7feedbfff640 1 --2- 172.21.15.47:0/3413449061 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e336a1d7b0 0x55e336a32da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:42.934 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.934+0000 7feedbfff640 1 -- 172.21.15.47:0/3413449061 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e336a20460 con 0x55e336a149e0 2024-04-17T06:08:42.934 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.934+0000 7feedbfff640 1 --2- 172.21.15.47:0/3413449061 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e336a149e0 0x55e336a10540 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7feed0099280 tx=0x7feed000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaec4890d171a7c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:08:42.934 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.934+0000 7feec27fc640 1 -- 172.21.15.47:0/3413449061 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed009a9a0 con 0x55e336a149e0 2024-04-17T06:08:42.934 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.934+0000 7feec27fc640 1 -- 172.21.15.47:0/3413449061 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feed0098920 con 0x55e336a149e0 2024-04-17T06:08:42.934 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.934+0000 7feec27fc640 1 -- 172.21.15.47:0/3413449061 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed00ce530 con 0x55e336a149e0 2024-04-17T06:08:42.934 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.935+0000 7feee57c9740 1 -- 172.21.15.47:0/3413449061 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e336a149e0 msgr2=0x55e336a10540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:08:42.934 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.935+0000 7feee57c9740 1 --2- 172.21.15.47:0/3413449061 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e336a149e0 0x55e336a10540 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7feed0099280 tx=0x7feed000cf90 comp rx=0 tx=0).stop 2024-04-17T06:08:42.934 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.935+0000 7feee57c9740 1 -- 172.21.15.47:0/3413449061 shutdown_connections 2024-04-17T06:08:42.934 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.935+0000 7feee57c9740 1 --2- 172.21.15.47:0/3413449061 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e336a1d7b0 0x55e336a32da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.935+0000 7feee57c9740 1 --2- 172.21.15.47:0/3413449061 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e336a149e0 0x55e336a10540 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.935+0000 7feee57c9740 1 --2- 172.21.15.47:0/3413449061 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e336a140c0 0x55e336a1cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.935+0000 7feee57c9740 1 -- 172.21.15.47:0/3413449061 >> 172.21.15.47:0/3413449061 conn(0x55e336a0e9e0 msgr2=0x55e336a0fe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:08:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.935+0000 7feee57c9740 1 -- 172.21.15.47:0/3413449061 shutdown_connections 2024-04-17T06:08:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.935+0000 7feee57c9740 1 -- 172.21.15.47:0/3413449061 wait complete. 2024-04-17T06:08:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.935+0000 7feee57c9740 1 -- start start 2024-04-17T06:08:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.935+0000 7feee57c9740 1 Processor -- start 2024-04-17T06:08:42.938 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.936+0000 7feee57c9740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e336a140c0 0x55e336a30b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:42.938 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.936+0000 7feee57c9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e336a149e0 0x55e336a310d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:42.938 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.936+0000 7feee57c9740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e336a1d7b0 0x55e336a92b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:42.938 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.936+0000 7feee57c9740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7feedc1ae290 con 0x55e336a1d7b0 2024-04-17T06:08:42.938 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.936+0000 7feee57c9740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7feedc1af700 con 0x55e336a140c0 2024-04-17T06:08:42.938 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.936+0000 7feee57c9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7feedc1ae960 con 0x55e336a149e0 2024-04-17T06:08:42.938 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.936+0000 7feee57c9740 10 client.0 _ll_register_callbacks cb 0x7feed40075a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:08:42.938 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.936+0000 7feee10a1640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e336a1d7b0 0x55e336a92b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:42.938 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.936+0000 7feee10a1640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e336a1d7b0 0x55e336a92b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39218/0 (socket says 172.21.15.47:39218) 2024-04-17T06:08:42.938 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.936+0000 7feee08a0640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e336a140c0 0x55e336a30b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:42.938 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.936+0000 7feee10a1640 1 -- 172.21.15.47:0/2719621963 learned_addr learned my addr 172.21.15.47:0/2719621963 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:08:42.938 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.936+0000 7feedbfff640 1 --2- 172.21.15.47:0/2719621963 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e336a149e0 0x55e336a310d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:42.939 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.936+0000 7feee10a1640 1 -- 172.21.15.47:0/2719621963 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e336a149e0 msgr2=0x55e336a310d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:08:42.939 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.936+0000 7feee10a1640 1 --2- 172.21.15.47:0/2719621963 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e336a149e0 0x55e336a310d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:42.939 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.936+0000 7feee10a1640 1 -- 172.21.15.47:0/2719621963 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e336a140c0 msgr2=0x55e336a30b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:08:42.939 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.936+0000 7feee10a1640 1 --2- 172.21.15.47:0/2719621963 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e336a140c0 0x55e336a30b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:42.939 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.936+0000 7feee10a1640 1 -- 172.21.15.47:0/2719621963 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e336a1fba0 con 0x55e336a1d7b0 2024-04-17T06:08:42.939 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.936+0000 7feee10a1640 1 --2- 172.21.15.47:0/2719621963 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e336a1d7b0 0x55e336a92b70 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7feedc102f70 tx=0x7feecc009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7e29e797e9dbb9d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:08:42.939 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.936+0000 7feebbfff640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-17T06:08:42.939 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.937+0000 7feebbfff640 1 -- 172.21.15.47:0/2719621963 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feecc051020 con 0x55e336a1d7b0 2024-04-17T06:08:42.939 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.937+0000 7feee57c9740 1 -- 172.21.15.47:0/2719621963 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feed0098920 con 0x55e336a1d7b0 2024-04-17T06:08:42.939 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.937+0000 7feebbfff640 1 -- 172.21.15.47:0/2719621963 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8469+0+0 (secure 0 0 0) 0x7feecc0048c0 con 0x55e336a1d7b0 2024-04-17T06:08:42.939 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.937+0000 7feebbfff640 1 -- 172.21.15.47:0/2719621963 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7feecc05c050 con 0x55e336a1d7b0 2024-04-17T06:08:42.939 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.937+0000 7feee57c9740 10 client.16026 fetch_fsmap learned FSMap version 32 2024-04-17T06:08:42.939 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.937+0000 7feee57c9740 1 -- 172.21.15.47:0/2719621963 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55e336a93ff0 con 0x55e336a1d7b0 2024-04-17T06:08:42.939 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.937+0000 7feebbfff640 1 -- 172.21.15.47:0/2719621963 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feed0098920 con 0x55e336a1d7b0 2024-04-17T06:08:42.939 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.937+0000 7feebbfff640 1 -- 172.21.15.47:0/2719621963 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) ==== 8339+0+0 (secure 0 0 0) 0x7feecc042440 con 0x55e336a1d7b0 2024-04-17T06:08:42.940 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.937+0000 7feee57c9740 10 client.16026 fetch_fsmap finished waiting for FSMap version 32 2024-04-17T06:08:42.940 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.937+0000 7feee57c9740 10 client.16026 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-17T06:08:42.940 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.937+0000 7feee57c9740 20 client.16026 populate_metadata read hostname 'smithi047' 2024-04-17T06:08:42.940 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.937+0000 7feee57c9740 1 --2- 172.21.15.47:0/2719621963 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55e336a94790 0x55e336a96bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:42.940 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.937+0000 7feee57c9740 4 client.16026 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-17T06:08:42.940 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.937+0000 7feee57c9740 1 -- 172.21.15.47:0/2719621963 --> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e336981060 con 0x55e336a94790 2024-04-17T06:08:42.940 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.937+0000 7feee08a0640 1 --2- 172.21.15.47:0/2719621963 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55e336a94790 0x55e336a96bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:42.940 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.938+0000 7feee08a0640 1 --2- 172.21.15.47:0/2719621963 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55e336a94790 0x55e336a96bb0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9327bfa5e091ba79 server_cookie=805ebf0d17292ffa in_seq=0 out_seq=0 2024-04-17T06:08:42.940 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.938+0000 7feebbfff640 10 client.16026 ms_handle_connect on v2:172.21.15.98:6838/2841287917 2024-04-17T06:08:42.944 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.946+0000 7feebbfff640 1 -- 172.21.15.47:0/2719621963 <== mds.0 v2:172.21.15.98:6838/2841287917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e336981060 con 0x55e336a94790 2024-04-17T06:08:42.945 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.946+0000 7feebbfff640 10 client.16026 handle_command_reply: tid=0 2024-04-17T06:08:42.945 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.946+0000 7feee57c9740 1 client.16026 shutdown 2024-04-17T06:08:42.945 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.946+0000 7feee57c9740 1 -- 172.21.15.47:0/2719621963 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e336a1d7b0 msgr2=0x55e336a92b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:08:42.945 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.946+0000 7feee57c9740 1 --2- 172.21.15.47:0/2719621963 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e336a1d7b0 0x55e336a92b70 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7feedc102f70 tx=0x7feecc009260 comp rx=0 tx=0).stop 2024-04-17T06:08:42.945 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.946+0000 7feee57c9740 1 -- 172.21.15.47:0/2719621963 shutdown_connections 2024-04-17T06:08:42.945 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.946+0000 7feee57c9740 1 --2- 172.21.15.47:0/2719621963 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55e336a94790 0x55e336a96bb0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:42.945 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.946+0000 7feee57c9740 1 --2- 172.21.15.47:0/2719621963 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e336a1d7b0 0x55e336a92b70 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:42.945 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.946+0000 7feee57c9740 1 --2- 172.21.15.47:0/2719621963 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e336a149e0 0x55e336a310d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:42.945 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.946+0000 7feebbfff640 0 client.16026 ms_handle_reset on v2:172.21.15.98:6838/2841287917 2024-04-17T06:08:42.945 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.946+0000 7feee57c9740 1 --2- 172.21.15.47:0/2719621963 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e336a140c0 0x55e336a30b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:42.945 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.946+0000 7feee57c9740 1 -- 172.21.15.47:0/2719621963 >> 172.21.15.47:0/2719621963 conn(0x55e336a0e9e0 msgr2=0x55e336a0db00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:08:42.945 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.947+0000 7feee57c9740 1 -- 172.21.15.47:0/2719621963 shutdown_connections 2024-04-17T06:08:42.945 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.947+0000 7feee57c9740 1 -- 172.21.15.47:0/2719621963 wait complete. 2024-04-17T06:08:42.945 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.947+0000 7feee57c9740 20 client.16026 trim_cache size 0 max 16384 2024-04-17T06:08:42.946 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-04-17T06:08:42.946 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-04-17T06:08:42.946 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-04-17T06:08:42.946 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-04-17T06:08:42.947 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.947+0000 7feee2b04640 1 -- 172.21.15.47:0/2010996717 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7feec40a3dc0 msgr2=0x7feec40a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:08:42.947 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.947+0000 7feee2b04640 1 --2- 172.21.15.47:0/2010996717 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7feec40a3dc0 0x7feec40a6280 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7feecc036040 tx=0x7feecc02cac0 comp rx=0 tx=0).stop 2024-04-17T06:08:42.947 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.947+0000 7feee2b04640 1 -- 172.21.15.47:0/2010996717 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feedc1016b0 msgr2=0x7feedc1a0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:08:42.947 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.947+0000 7feee2b04640 1 --2- 172.21.15.47:0/2010996717 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feedc1016b0 0x7feedc1a0ab0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7feed000c920 tx=0x7feed000cde0 comp rx=0 tx=0).stop 2024-04-17T06:08:42.947 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.947+0000 7feee2b04640 1 -- 172.21.15.47:0/2010996717 shutdown_connections 2024-04-17T06:08:42.947 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.947+0000 7feee2b04640 1 --2- 172.21.15.47:0/2010996717 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feedc10e940 0x7feedc19a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:42.947 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.947+0000 7feee2b04640 1 --2- 172.21.15.47:0/2010996717 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7feec40a3dc0 0x7feec40a6280 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:42.947 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.947+0000 7feee2b04640 1 --2- 172.21.15.47:0/2010996717 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feedc1016b0 0x7feedc1a0ab0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:42.947 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.947+0000 7feee2b04640 1 --2- 172.21.15.47:0/2010996717 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feedc100d90 0x7feedc1a0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:42.947 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.947+0000 7feee2b04640 1 -- 172.21.15.47:0/2010996717 >> 172.21.15.47:0/2010996717 conn(0x7feedc077fe0 msgr2=0x7feedc10fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:08:42.948 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.947+0000 7feee2b04640 1 -- 172.21.15.47:0/2010996717 shutdown_connections 2024-04-17T06:08:42.948 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:42.947+0000 7feee2b04640 1 -- 172.21.15.47:0/2010996717 wait complete. 2024-04-17T06:08:44.382 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:08:44 smithi098 ceph-mon[36516]: pgmap v230: 97 pgs: 97 active+clean; 3.7 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:08:44.388 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:08:44 smithi047 ceph-mon[32223]: pgmap v230: 97 pgs: 97 active+clean; 3.7 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:08:44.533 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:08:44 smithi103 ceph-mon[36243]: pgmap v230: 97 pgs: 97 active+clean; 3.7 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:08:46.382 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:08:46 smithi098 ceph-mon[36516]: pgmap v231: 97 pgs: 97 active+clean; 3.8 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 21 KiB/s wr, 18 op/s 2024-04-17T06:08:46.388 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:08:46 smithi047 ceph-mon[32223]: pgmap v231: 97 pgs: 97 active+clean; 3.8 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 21 KiB/s wr, 18 op/s 2024-04-17T06:08:46.533 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:08:46 smithi103 ceph-mon[36243]: pgmap v231: 97 pgs: 97 active+clean; 3.8 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 21 KiB/s wr, 18 op/s 2024-04-17T06:08:48.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:08:48 smithi098 ceph-mon[36516]: pgmap v232: 97 pgs: 97 active+clean; 3.8 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:08:48.388 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:08:48 smithi047 ceph-mon[32223]: pgmap v232: 97 pgs: 97 active+clean; 3.8 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:08:48.533 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:08:48 smithi103 ceph-mon[36243]: pgmap v232: 97 pgs: 97 active+clean; 3.8 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:08:50.382 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:08:50 smithi098 ceph-mon[36516]: pgmap v233: 97 pgs: 97 active+clean; 3.8 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:08:50.388 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:08:50 smithi047 ceph-mon[32223]: pgmap v233: 97 pgs: 97 active+clean; 3.8 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:08:50.533 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:08:50 smithi103 ceph-mon[36243]: pgmap v233: 97 pgs: 97 active+clean; 3.8 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:08:52.382 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:08:52 smithi098 ceph-mon[36516]: pgmap v234: 97 pgs: 97 active+clean; 3.8 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-04-17T06:08:52.388 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:08:52 smithi047 ceph-mon[32223]: pgmap v234: 97 pgs: 97 active+clean; 3.8 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-04-17T06:08:52.533 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:08:52 smithi103 ceph-mon[36243]: pgmap v234: 97 pgs: 97 active+clean; 3.8 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-04-17T06:08:54.382 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:08:54 smithi098 ceph-mon[36516]: pgmap v235: 97 pgs: 97 active+clean; 3.8 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:08:54.388 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:08:54 smithi047 ceph-mon[32223]: pgmap v235: 97 pgs: 97 active+clean; 3.8 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:08:54.533 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:08:54 smithi103 ceph-mon[36243]: pgmap v235: 97 pgs: 97 active+clean; 3.8 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:08:56.047 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.047+0000 7ff189274640 1 -- 172.21.15.47:0/3805135100 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1841031f0 msgr2=0x7ff1841035d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:08:56.047 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.047+0000 7ff189274640 1 --2- 172.21.15.47:0/3805135100 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1841031f0 0x7ff1841035d0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7ff174009900 tx=0x7ff17402bd60 comp rx=0 tx=0).stop 2024-04-17T06:08:56.047 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.047+0000 7ff189274640 1 -- 172.21.15.47:0/3805135100 shutdown_connections 2024-04-17T06:08:56.047 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.047+0000 7ff189274640 1 --2- 172.21.15.47:0/3805135100 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff18410c530 0x7ff18410e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:56.047 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.047+0000 7ff189274640 1 --2- 172.21.15.47:0/3805135100 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff184103b10 0x7ff18410bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:56.047 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.047+0000 7ff189274640 1 --2- 172.21.15.47:0/3805135100 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1841031f0 0x7ff1841035d0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:56.047 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.047+0000 7ff189274640 1 -- 172.21.15.47:0/3805135100 >> 172.21.15.47:0/3805135100 conn(0x7ff184077fe0 msgr2=0x7ff1840783f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:08:56.048 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.047+0000 7ff189274640 1 -- 172.21.15.47:0/3805135100 shutdown_connections 2024-04-17T06:08:56.048 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.047+0000 7ff189274640 1 -- 172.21.15.47:0/3805135100 wait complete. 2024-04-17T06:08:56.048 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.047+0000 7ff189274640 1 Processor -- start 2024-04-17T06:08:56.048 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.048+0000 7ff189274640 1 -- start start 2024-04-17T06:08:56.048 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.048+0000 7ff189274640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff1841031f0 0x7ff18419e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:56.048 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.048+0000 7ff189274640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff184103b10 0x7ff18419eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:56.048 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.048+0000 7ff189274640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff18410c530 0x7ff184198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:56.048 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.048+0000 7ff189274640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff184112360 con 0x7ff18410c530 2024-04-17T06:08:56.048 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.048+0000 7ff189274640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff1841121e0 con 0x7ff184103b10 2024-04-17T06:08:56.048 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.048+0000 7ff189274640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff1841124e0 con 0x7ff1841031f0 2024-04-17T06:08:56.048 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.048+0000 7ff1837fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff18410c530 0x7ff184198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:56.048 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.048+0000 7ff1837fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff18410c530 0x7ff184198660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57212/0 (socket says 172.21.15.47:57212) 2024-04-17T06:08:56.049 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.048+0000 7ff1837fe640 1 -- 172.21.15.47:0/1449716302 learned_addr learned my addr 172.21.15.47:0/1449716302 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:08:56.049 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.048+0000 7ff1827fc640 1 --2- 172.21.15.47:0/1449716302 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff184103b10 0x7ff18419eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:56.049 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.048+0000 7ff182ffd640 1 --2- 172.21.15.47:0/1449716302 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff1841031f0 0x7ff18419e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:56.049 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.049+0000 7ff1837fe640 1 -- 172.21.15.47:0/1449716302 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff1841031f0 msgr2=0x7ff18419e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:08:56.049 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.049+0000 7ff1837fe640 1 --2- 172.21.15.47:0/1449716302 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff1841031f0 0x7ff18419e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:56.049 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.049+0000 7ff1837fe640 1 -- 172.21.15.47:0/1449716302 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff184103b10 msgr2=0x7ff18419eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:08:56.049 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.049+0000 7ff1837fe640 1 --2- 172.21.15.47:0/1449716302 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff184103b10 0x7ff18419eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:56.049 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.049+0000 7ff1837fe640 1 -- 172.21.15.47:0/1449716302 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff184198f20 con 0x7ff18410c530 2024-04-17T06:08:56.049 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.049+0000 7ff182ffd640 1 --2- 172.21.15.47:0/1449716302 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff1841031f0 0x7ff18419e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:08:56.049 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.049+0000 7ff1837fe640 1 --2- 172.21.15.47:0/1449716302 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff18410c530 0x7ff184198660 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7ff17c00eaa0 tx=0x7ff17c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:08:56.049 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.049+0000 7ff17bfff640 1 -- 172.21.15.47:0/1449716302 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff17c00cdb0 con 0x7ff18410c530 2024-04-17T06:08:56.049 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.049+0000 7ff189274640 1 -- 172.21.15.47:0/1449716302 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff184199210 con 0x7ff18410c530 2024-04-17T06:08:56.049 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.049+0000 7ff17bfff640 1 -- 172.21.15.47:0/1449716302 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff17c005020 con 0x7ff18410c530 2024-04-17T06:08:56.049 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.049+0000 7ff17bfff640 1 -- 172.21.15.47:0/1449716302 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff17c0116b0 con 0x7ff18410c530 2024-04-17T06:08:56.050 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.049+0000 7ff189274640 1 -- 172.21.15.47:0/1449716302 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1841a5520 con 0x7ff18410c530 2024-04-17T06:08:56.057 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.051+0000 7ff17bfff640 1 -- 172.21.15.47:0/1449716302 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff17c005560 con 0x7ff18410c530 2024-04-17T06:08:56.057 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.052+0000 7ff18be67740 1 Processor -- start 2024-04-17T06:08:56.057 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.052+0000 7ff18be67740 1 -- start start 2024-04-17T06:08:56.057 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.052+0000 7ff18be67740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f589166be0 0x55f589166fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:56.057 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.052+0000 7ff18be67740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f589167500 0x55f589174680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:56.057 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.052+0000 7ff18be67740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f589174bc0 0x55f589176fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:56.057 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.052+0000 7ff18be67740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55f5890cc7d0 con 0x55f589166be0 2024-04-17T06:08:56.057 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.052+0000 7ff18be67740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55f589069f70 con 0x55f589174bc0 2024-04-17T06:08:56.057 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.052+0000 7ff18be67740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f5890c8790 con 0x55f589167500 2024-04-17T06:08:56.058 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.052+0000 7ff17bfff640 1 --2- 172.21.15.47:0/1449716302 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7ff1700a3dc0 0x7ff1700a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:56.058 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.052+0000 7ff1827fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f589166be0 0x55f589166fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:56.058 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.052+0000 7ff1827fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f589166be0 0x55f589166fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57228/0 (socket says 172.21.15.47:57228) 2024-04-17T06:08:56.058 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.052+0000 7ff1827fc640 1 -- 172.21.15.47:0/2367745125 learned_addr learned my addr 172.21.15.47:0/2367745125 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:08:56.058 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.052+0000 7ff1837fe640 1 --2- 172.21.15.47:0/1449716302 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7ff1700a3dc0 0x7ff1700a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:56.058 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.052+0000 7ff182ffd640 1 --2- 172.21.15.47:0/2367745125 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f589167500 0x55f589174680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:56.058 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.052+0000 7ff17bfff640 1 -- 172.21.15.47:0/1449716302 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8469+0+0 (secure 0 0 0) 0x7ff17c0d0a00 con 0x7ff18410c530 2024-04-17T06:08:56.058 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.053+0000 7ff1827fc640 1 -- 172.21.15.47:0/2367745125 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f589167500 msgr2=0x55f589174680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:08:56.058 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.053+0000 7ff1827fc640 1 --2- 172.21.15.47:0/2367745125 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f589167500 0x55f589174680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:56.058 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.053+0000 7ff1827fc640 1 -- 172.21.15.47:0/2367745125 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f589174bc0 msgr2=0x55f589176fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:08:56.058 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.053+0000 7ff1827fc640 1 --2- 172.21.15.47:0/2367745125 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f589174bc0 0x55f589176fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:56.058 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.053+0000 7ff1827fc640 1 -- 172.21.15.47:0/2367745125 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f588ddf210 con 0x55f589166be0 2024-04-17T06:08:56.059 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.053+0000 7ff182ffd640 1 --2- 172.21.15.47:0/2367745125 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f589167500 0x55f589174680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:08:56.059 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.053+0000 7ff1827fc640 1 --2- 172.21.15.47:0/2367745125 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f589166be0 0x55f589166fc0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7ff16c00bd50 tx=0x7ff16c0127c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=395d52505619bae9 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:08:56.059 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.053+0000 7ff1837fe640 1 --2- 172.21.15.47:0/1449716302 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7ff1700a3dc0 0x7ff1700a6280 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff17c09c670 tx=0x7ff17c0bd040 comp rx=0 tx=0).ready entity=mgr.24923 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:08:56.059 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.053+0000 7ff1797fa640 1 -- 172.21.15.47:0/2367745125 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff16c0120d0 con 0x55f589166be0 2024-04-17T06:08:56.059 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.053+0000 7ff1797fa640 1 -- 172.21.15.47:0/2367745125 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff16c016070 con 0x55f589166be0 2024-04-17T06:08:56.059 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.053+0000 7ff1797fa640 1 -- 172.21.15.47:0/2367745125 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff16c0107f0 con 0x55f589166be0 2024-04-17T06:08:56.059 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.054+0000 7ff18be67740 1 -- 172.21.15.47:0/2367745125 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f589166be0 msgr2=0x55f589166fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:08:56.059 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.054+0000 7ff18be67740 1 --2- 172.21.15.47:0/2367745125 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f589166be0 0x55f589166fc0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7ff16c00bd50 tx=0x7ff16c0127c0 comp rx=0 tx=0).stop 2024-04-17T06:08:56.059 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.054+0000 7ff18be67740 1 -- 172.21.15.47:0/2367745125 shutdown_connections 2024-04-17T06:08:56.059 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.054+0000 7ff18be67740 1 --2- 172.21.15.47:0/2367745125 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f589174bc0 0x55f589176fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:56.059 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.054+0000 7ff18be67740 1 --2- 172.21.15.47:0/2367745125 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f589167500 0x55f589174680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:56.059 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.054+0000 7ff18be67740 1 --2- 172.21.15.47:0/2367745125 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f589166be0 0x55f589166fc0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:56.059 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.054+0000 7ff18be67740 1 -- 172.21.15.47:0/2367745125 >> 172.21.15.47:0/2367745125 conn(0x55f5891619e0 msgr2=0x55f589161dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:08:56.059 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.054+0000 7ff18be67740 1 -- 172.21.15.47:0/2367745125 shutdown_connections 2024-04-17T06:08:56.060 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.054+0000 7ff18be67740 1 -- 172.21.15.47:0/2367745125 wait complete. 2024-04-17T06:08:56.060 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.054+0000 7ff18be67740 1 -- start start 2024-04-17T06:08:56.060 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.054+0000 7ff18be67740 1 Processor -- start 2024-04-17T06:08:56.060 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.055+0000 7ff18be67740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f589166be0 0x55f589171d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:56.060 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.055+0000 7ff18be67740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f589167500 0x55f589172260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:56.060 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.055+0000 7ff18be67740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f589174bc0 0x55f5891727a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:56.060 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.055+0000 7ff18be67740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55f589177780 con 0x55f589174bc0 2024-04-17T06:08:56.060 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.055+0000 7ff18be67740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55f589054e90 con 0x55f589167500 2024-04-17T06:08:56.060 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.055+0000 7ff18be67740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f589177900 con 0x55f589166be0 2024-04-17T06:08:56.060 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.055+0000 7ff18be67740 10 client.0 _ll_register_callbacks cb 0x55f58906ad40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:08:56.060 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.055+0000 7ff1827fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f589174bc0 0x55f5891727a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:56.060 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.055+0000 7ff1827fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f589174bc0 0x55f5891727a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57230/0 (socket says 172.21.15.47:57230) 2024-04-17T06:08:56.060 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.055+0000 7ff1827fc640 1 -- 172.21.15.47:0/13962179 learned_addr learned my addr 172.21.15.47:0/13962179 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:08:56.060 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.055+0000 7ff182ffd640 1 --2- 172.21.15.47:0/13962179 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f589167500 0x55f589172260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:56.061 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.055+0000 7ff1827fc640 1 --2- 172.21.15.47:0/13962179 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f589166be0 0x55f589171d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:56.061 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.055+0000 7ff1827fc640 1 -- 172.21.15.47:0/13962179 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f589166be0 msgr2=0x55f589171d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:08:56.061 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.055+0000 7ff1827fc640 1 --2- 172.21.15.47:0/13962179 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f589166be0 0x55f589171d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:56.061 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.055+0000 7ff1827fc640 1 -- 172.21.15.47:0/13962179 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f589167500 msgr2=0x55f589172260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:08:56.061 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.055+0000 7ff1827fc640 1 --2- 172.21.15.47:0/13962179 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f589167500 0x55f589172260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:56.061 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.055+0000 7ff1827fc640 1 -- 172.21.15.47:0/13962179 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f588eab620 con 0x55f589174bc0 2024-04-17T06:08:56.061 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.055+0000 7ff1827fc640 1 --2- 172.21.15.47:0/13962179 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f589174bc0 0x55f5891727a0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7ff16c00bff0 tx=0x7ff16c00bb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad7a33152d794234 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:08:56.061 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.055+0000 7ff16a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-17T06:08:56.061 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.056+0000 7ff16a7fc640 1 -- 172.21.15.47:0/13962179 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff16c0048c0 con 0x55f589174bc0 2024-04-17T06:08:56.061 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.056+0000 7ff18be67740 1 -- 172.21.15.47:0/13962179 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff16c016070 con 0x55f589174bc0 2024-04-17T06:08:56.061 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.056+0000 7ff16a7fc640 1 -- 172.21.15.47:0/13962179 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8469+0+0 (secure 0 0 0) 0x7ff16c004a60 con 0x55f589174bc0 2024-04-17T06:08:56.061 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.056+0000 7ff16a7fc640 1 -- 172.21.15.47:0/13962179 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff16c005600 con 0x55f589174bc0 2024-04-17T06:08:56.061 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.056+0000 7ff18be67740 10 client.16038 fetch_fsmap learned FSMap version 32 2024-04-17T06:08:56.061 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.056+0000 7ff18be67740 1 -- 172.21.15.47:0/13962179 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f589183b00 con 0x55f589174bc0 2024-04-17T06:08:56.062 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.056+0000 7ff16a7fc640 1 -- 172.21.15.47:0/13962179 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff16c016070 con 0x55f589174bc0 2024-04-17T06:08:56.062 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.056+0000 7ff16a7fc640 1 -- 172.21.15.47:0/13962179 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) ==== 8339+0+0 (secure 0 0 0) 0x7ff16c010530 con 0x55f589174bc0 2024-04-17T06:08:56.062 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.056+0000 7ff18be67740 10 client.16038 fetch_fsmap finished waiting for FSMap version 32 2024-04-17T06:08:56.062 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.056+0000 7ff18be67740 10 client.16038 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-17T06:08:56.062 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.056+0000 7ff18be67740 20 client.16038 populate_metadata read hostname 'smithi047' 2024-04-17T06:08:56.062 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.056+0000 7ff18be67740 1 --2- 172.21.15.47:0/13962179 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55f589184eb0 0x55f5891872a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:56.062 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.056+0000 7ff18be67740 4 client.16038 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T06:08:56.062 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.056+0000 7ff18be67740 1 -- 172.21.15.47:0/13962179 --> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f5890da1f0 con 0x55f589184eb0 2024-04-17T06:08:56.062 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.056+0000 7ff182ffd640 1 --2- 172.21.15.47:0/13962179 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55f589184eb0 0x55f5891872a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:56.062 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.057+0000 7ff16a7fc640 10 client.16038 ms_handle_connect on v2:172.21.15.98:6838/2841287917 2024-04-17T06:08:56.062 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.057+0000 7ff182ffd640 1 --2- 172.21.15.47:0/13962179 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55f589184eb0 0x55f5891872a0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fcfe791dd98d3251 server_cookie=9e79cf12a42f0298 in_seq=0 out_seq=0 2024-04-17T06:08:56.062 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.058+0000 7ff16a7fc640 1 -- 172.21.15.47:0/13962179 <== mds.0 v2:172.21.15.98:6838/2841287917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f5890da1f0 con 0x55f589184eb0 2024-04-17T06:08:56.062 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.058+0000 7ff16a7fc640 10 client.16038 handle_command_reply: tid=0 2024-04-17T06:08:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.058+0000 7ff18be67740 1 client.16038 shutdown 2024-04-17T06:08:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.058+0000 7ff18be67740 1 -- 172.21.15.47:0/13962179 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f589174bc0 msgr2=0x55f5891727a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:08:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.058+0000 7ff18be67740 1 --2- 172.21.15.47:0/13962179 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f589174bc0 0x55f5891727a0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7ff16c00bff0 tx=0x7ff16c00bb10 comp rx=0 tx=0).stop 2024-04-17T06:08:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.058+0000 7ff18be67740 1 -- 172.21.15.47:0/13962179 shutdown_connections 2024-04-17T06:08:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.058+0000 7ff18be67740 1 --2- 172.21.15.47:0/13962179 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55f589184eb0 0x55f5891872a0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.058+0000 7ff18be67740 1 --2- 172.21.15.47:0/13962179 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f589174bc0 0x55f5891727a0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.058+0000 7ff16a7fc640 0 client.16038 ms_handle_reset on v2:172.21.15.98:6838/2841287917 2024-04-17T06:08:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.058+0000 7ff18be67740 1 --2- 172.21.15.47:0/13962179 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f589167500 0x55f589172260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.058+0000 7ff18be67740 1 --2- 172.21.15.47:0/13962179 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f589166be0 0x55f589171d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.058+0000 7ff18be67740 1 -- 172.21.15.47:0/13962179 >> 172.21.15.47:0/13962179 conn(0x55f5891619e0 msgr2=0x55f589163730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:08:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.058+0000 7ff18be67740 1 -- 172.21.15.47:0/13962179 shutdown_connections 2024-04-17T06:08:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.058+0000 7ff18be67740 1 -- 172.21.15.47:0/13962179 wait complete. 2024-04-17T06:08:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.058+0000 7ff18be67740 20 client.16038 trim_cache size 0 max 16384 2024-04-17T06:08:56.072 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.073+0000 7ff18be67740 1 Processor -- start 2024-04-17T06:08:56.072 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.073+0000 7ff18be67740 1 -- start start 2024-04-17T06:08:56.072 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.074+0000 7ff18be67740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f589166be0 0x55f5891634d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:56.073 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.074+0000 7ff18be67740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f589167500 0x55f58918a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:56.073 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.074+0000 7ff18be67740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f589174bc0 0x55f58918a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:56.073 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.074+0000 7ff18be67740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff1841ae380 con 0x55f589167500 2024-04-17T06:08:56.073 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.074+0000 7ff18be67740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff1841abc40 con 0x55f589174bc0 2024-04-17T06:08:56.073 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.074+0000 7ff18be67740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff1841a9a90 con 0x55f589166be0 2024-04-17T06:08:56.073 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.074+0000 7ff182ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f589167500 0x55f58918a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:56.073 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.074+0000 7ff182ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f589167500 0x55f58918a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57246/0 (socket says 172.21.15.47:57246) 2024-04-17T06:08:56.073 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.074+0000 7ff182ffd640 1 -- 172.21.15.47:0/1006202889 learned_addr learned my addr 172.21.15.47:0/1006202889 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:08:56.073 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.074+0000 7ff1827fc640 1 --2- 172.21.15.47:0/1006202889 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f589174bc0 0x55f58918a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:56.073 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.074+0000 7ff1827fc640 1 --2- 172.21.15.47:0/1006202889 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f589166be0 0x55f5891634d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:56.073 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.074+0000 7ff182ffd640 1 -- 172.21.15.47:0/1006202889 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f589166be0 msgr2=0x55f5891634d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:08:56.073 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.074+0000 7ff182ffd640 1 --2- 172.21.15.47:0/1006202889 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f589166be0 0x55f5891634d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:56.073 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.074+0000 7ff182ffd640 1 -- 172.21.15.47:0/1006202889 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f589174bc0 msgr2=0x55f58918a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:08:56.073 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.074+0000 7ff182ffd640 1 --2- 172.21.15.47:0/1006202889 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f589174bc0 0x55f58918a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:56.073 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.074+0000 7ff182ffd640 1 -- 172.21.15.47:0/1006202889 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f589170270 con 0x55f589167500 2024-04-17T06:08:56.074 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.074+0000 7ff1827fc640 1 --2- 172.21.15.47:0/1006202889 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f589174bc0 0x55f58918a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T06:08:56.074 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.074+0000 7ff182ffd640 1 --2- 172.21.15.47:0/1006202889 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f589167500 0x55f58918a2c0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7ff174036040 tx=0x7ff174000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a311596103c4b77 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:08:56.074 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.074+0000 7ff178ff9640 1 -- 172.21.15.47:0/1006202889 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff174048020 con 0x55f589167500 2024-04-17T06:08:56.074 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.075+0000 7ff178ff9640 1 -- 172.21.15.47:0/1006202889 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff174004480 con 0x55f589167500 2024-04-17T06:08:56.074 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.075+0000 7ff178ff9640 1 -- 172.21.15.47:0/1006202889 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff174044430 con 0x55f589167500 2024-04-17T06:08:56.077 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.075+0000 7ff18be67740 1 -- 172.21.15.47:0/1006202889 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f589167500 msgr2=0x55f58918a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:08:56.077 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.075+0000 7ff18be67740 1 --2- 172.21.15.47:0/1006202889 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f589167500 0x55f58918a2c0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7ff174036040 tx=0x7ff174000bc0 comp rx=0 tx=0).stop 2024-04-17T06:08:56.077 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.075+0000 7ff18be67740 1 -- 172.21.15.47:0/1006202889 shutdown_connections 2024-04-17T06:08:56.077 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.075+0000 7ff18be67740 1 --2- 172.21.15.47:0/1006202889 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f589174bc0 0x55f58918a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:56.077 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.075+0000 7ff18be67740 1 --2- 172.21.15.47:0/1006202889 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f589167500 0x55f58918a2c0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:56.078 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.075+0000 7ff18be67740 1 --2- 172.21.15.47:0/1006202889 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f589166be0 0x55f5891634d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:56.078 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.075+0000 7ff18be67740 1 -- 172.21.15.47:0/1006202889 >> 172.21.15.47:0/1006202889 conn(0x55f5891619e0 msgr2=0x55f589162dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:08:56.078 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.075+0000 7ff18be67740 1 -- 172.21.15.47:0/1006202889 shutdown_connections 2024-04-17T06:08:56.078 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.075+0000 7ff18be67740 1 -- 172.21.15.47:0/1006202889 wait complete. 2024-04-17T06:08:56.078 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.076+0000 7ff18be67740 1 -- start start 2024-04-17T06:08:56.078 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.076+0000 7ff18be67740 1 Processor -- start 2024-04-17T06:08:56.078 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.076+0000 7ff18be67740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f589166be0 0x55f58918a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:56.078 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.076+0000 7ff18be67740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f589167500 0x55f58918a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:56.078 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.076+0000 7ff18be67740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f589174bc0 0x55f589187840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:56.078 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.076+0000 7ff18be67740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff1841aa100 con 0x55f589166be0 2024-04-17T06:08:56.078 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.076+0000 7ff18be67740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff1841ab570 con 0x55f589167500 2024-04-17T06:08:56.078 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.076+0000 7ff18be67740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff1841aa7d0 con 0x55f589174bc0 2024-04-17T06:08:56.078 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.076+0000 7ff18be67740 10 client.0 _ll_register_callbacks cb 0x7ff16c00bc50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:08:56.079 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.076+0000 7ff1827fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f589166be0 0x55f58918a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:56.079 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.076+0000 7ff1827fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f589166be0 0x55f58918a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57258/0 (socket says 172.21.15.47:57258) 2024-04-17T06:08:56.079 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.076+0000 7ff1827fc640 1 -- 172.21.15.47:0/721490853 learned_addr learned my addr 172.21.15.47:0/721490853 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:08:56.079 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.076+0000 7ff1827fc640 1 -- 172.21.15.47:0/721490853 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f589174bc0 msgr2=0x55f589187840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:08:56.079 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.076+0000 7ff1827fc640 1 --2- 172.21.15.47:0/721490853 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f589174bc0 0x55f589187840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:56.079 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.076+0000 7ff1827fc640 1 -- 172.21.15.47:0/721490853 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f589167500 msgr2=0x55f58918a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:08:56.079 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.076+0000 7ff1827fc640 1 --2- 172.21.15.47:0/721490853 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f589167500 0x55f58918a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:56.079 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.076+0000 7ff1827fc640 1 -- 172.21.15.47:0/721490853 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f589188040 con 0x55f589166be0 2024-04-17T06:08:56.079 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.077+0000 7ff1827fc640 1 --2- 172.21.15.47:0/721490853 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f589166be0 0x55f58918a2b0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7ff16c010a50 tx=0x7ff16c0052d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e822d9bfef83f1c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:08:56.079 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.077+0000 7ff16a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-17T06:08:56.079 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.077+0000 7ff16a7fc640 1 -- 172.21.15.47:0/721490853 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff16c036070 con 0x55f589166be0 2024-04-17T06:08:56.079 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.077+0000 7ff16a7fc640 1 -- 172.21.15.47:0/721490853 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8469+0+0 (secure 0 0 0) 0x7ff16c010630 con 0x55f589166be0 2024-04-17T06:08:56.079 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.077+0000 7ff18be67740 1 -- 172.21.15.47:0/721490853 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff174004480 con 0x55f589166be0 2024-04-17T06:08:56.079 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.077+0000 7ff16a7fc640 1 -- 172.21.15.47:0/721490853 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff16c012cc0 con 0x55f589166be0 2024-04-17T06:08:56.080 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.077+0000 7ff18be67740 10 client.16044 fetch_fsmap learned FSMap version 32 2024-04-17T06:08:56.080 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.077+0000 7ff18be67740 1 -- 172.21.15.47:0/721490853 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f5891ead30 con 0x55f589166be0 2024-04-17T06:08:56.080 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.077+0000 7ff16a7fc640 1 -- 172.21.15.47:0/721490853 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff174004480 con 0x55f589166be0 2024-04-17T06:08:56.080 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.077+0000 7ff16a7fc640 1 -- 172.21.15.47:0/721490853 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) ==== 8339+0+0 (secure 0 0 0) 0x7ff16c03c030 con 0x55f589166be0 2024-04-17T06:08:56.080 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.077+0000 7ff18be67740 10 client.16044 fetch_fsmap finished waiting for FSMap version 32 2024-04-17T06:08:56.080 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.077+0000 7ff18be67740 10 client.16044 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-17T06:08:56.080 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.077+0000 7ff18be67740 20 client.16044 populate_metadata read hostname 'smithi047' 2024-04-17T06:08:56.080 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.077+0000 7ff18be67740 1 --2- 172.21.15.47:0/721490853 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55f589184eb0 0x55f5891ed470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:56.080 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.077+0000 7ff18be67740 4 client.16044 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-17T06:08:56.080 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.077+0000 7ff18be67740 1 -- 172.21.15.47:0/721490853 --> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f5890d4060 con 0x55f589184eb0 2024-04-17T06:08:56.080 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.078+0000 7ff182ffd640 1 --2- 172.21.15.47:0/721490853 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55f589184eb0 0x55f5891ed470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:56.080 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.078+0000 7ff182ffd640 1 --2- 172.21.15.47:0/721490853 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55f589184eb0 0x55f5891ed470 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4655aad22b616f3 server_cookie=9d37eb858fb2138c in_seq=0 out_seq=0 2024-04-17T06:08:56.080 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.078+0000 7ff16a7fc640 10 client.16044 ms_handle_connect on v2:172.21.15.98:6838/2841287917 2024-04-17T06:08:56.084 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.086+0000 7ff16a7fc640 1 -- 172.21.15.47:0/721490853 <== mds.0 v2:172.21.15.98:6838/2841287917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f5890d4060 con 0x55f589184eb0 2024-04-17T06:08:56.084 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.086+0000 7ff16a7fc640 10 client.16044 handle_command_reply: tid=0 2024-04-17T06:08:56.084 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.086+0000 7ff18be67740 1 client.16044 shutdown 2024-04-17T06:08:56.085 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.086+0000 7ff18be67740 1 -- 172.21.15.47:0/721490853 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f589166be0 msgr2=0x55f58918a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:08:56.085 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.086+0000 7ff18be67740 1 --2- 172.21.15.47:0/721490853 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f589166be0 0x55f58918a2b0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7ff16c010a50 tx=0x7ff16c0052d0 comp rx=0 tx=0).stop 2024-04-17T06:08:56.085 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.086+0000 7ff18be67740 1 -- 172.21.15.47:0/721490853 shutdown_connections 2024-04-17T06:08:56.085 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.086+0000 7ff18be67740 1 --2- 172.21.15.47:0/721490853 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55f589184eb0 0x55f5891ed470 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:56.085 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.086+0000 7ff18be67740 1 --2- 172.21.15.47:0/721490853 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f589174bc0 0x55f589187840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:56.085 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.086+0000 7ff16a7fc640 0 client.16044 ms_handle_reset on v2:172.21.15.98:6838/2841287917 2024-04-17T06:08:56.085 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.086+0000 7ff18be67740 1 --2- 172.21.15.47:0/721490853 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f589167500 0x55f58918a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:56.085 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.086+0000 7ff18be67740 1 --2- 172.21.15.47:0/721490853 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f589166be0 0x55f58918a2b0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:56.085 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.086+0000 7ff18be67740 1 -- 172.21.15.47:0/721490853 >> 172.21.15.47:0/721490853 conn(0x55f5891619e0 msgr2=0x55f589180420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:08:56.085 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.086+0000 7ff18be67740 1 -- 172.21.15.47:0/721490853 shutdown_connections 2024-04-17T06:08:56.085 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.086+0000 7ff18be67740 1 -- 172.21.15.47:0/721490853 wait complete. 2024-04-17T06:08:56.085 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.087+0000 7ff18be67740 20 client.16044 trim_cache size 0 max 16384 2024-04-17T06:08:56.086 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-04-17T06:08:56.086 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-04-17T06:08:56.086 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-04-17T06:08:56.086 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-04-17T06:08:56.087 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.087+0000 7ff189274640 1 -- 172.21.15.47:0/1449716302 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7ff1700a3dc0 msgr2=0x7ff1700a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:08:56.087 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.087+0000 7ff189274640 1 --2- 172.21.15.47:0/1449716302 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7ff1700a3dc0 0x7ff1700a6280 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff17c09c670 tx=0x7ff17c0bd040 comp rx=0 tx=0).stop 2024-04-17T06:08:56.087 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.087+0000 7ff189274640 1 -- 172.21.15.47:0/1449716302 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff18410c530 msgr2=0x7ff184198660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:08:56.087 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.087+0000 7ff189274640 1 --2- 172.21.15.47:0/1449716302 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff18410c530 0x7ff184198660 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7ff17c00eaa0 tx=0x7ff17c00ef60 comp rx=0 tx=0).stop 2024-04-17T06:08:56.087 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.087+0000 7ff189274640 1 -- 172.21.15.47:0/1449716302 shutdown_connections 2024-04-17T06:08:56.087 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.087+0000 7ff189274640 1 --2- 172.21.15.47:0/1449716302 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7ff1700a3dc0 0x7ff1700a6280 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:56.087 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.087+0000 7ff189274640 1 --2- 172.21.15.47:0/1449716302 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff18410c530 0x7ff184198660 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:56.087 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.087+0000 7ff189274640 1 --2- 172.21.15.47:0/1449716302 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff184103b10 0x7ff18419eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:56.087 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.087+0000 7ff189274640 1 --2- 172.21.15.47:0/1449716302 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff1841031f0 0x7ff18419e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:56.087 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.087+0000 7ff189274640 1 -- 172.21.15.47:0/1449716302 >> 172.21.15.47:0/1449716302 conn(0x7ff184077fe0 msgr2=0x7ff1840fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:08:56.088 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.087+0000 7ff189274640 1 -- 172.21.15.47:0/1449716302 shutdown_connections 2024-04-17T06:08:56.088 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:08:56.087+0000 7ff189274640 1 -- 172.21.15.47:0/1449716302 wait complete. 2024-04-17T06:08:56.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:08:56 smithi098 ceph-mon[36516]: pgmap v236: 97 pgs: 97 active+clean; 3.9 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-04-17T06:08:56.388 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:08:56 smithi047 ceph-mon[32223]: pgmap v236: 97 pgs: 97 active+clean; 3.9 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-04-17T06:08:56.533 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:08:56 smithi103 ceph-mon[36243]: pgmap v236: 97 pgs: 97 active+clean; 3.9 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-04-17T06:08:58.336 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:08:58 smithi103 ceph-mon[36243]: pgmap v237: 97 pgs: 97 active+clean; 3.9 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.6 KiB/s wr, 9 op/s 2024-04-17T06:08:58.382 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:08:58 smithi098 ceph-mon[36516]: pgmap v237: 97 pgs: 97 active+clean; 3.9 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.6 KiB/s wr, 9 op/s 2024-04-17T06:08:58.388 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:08:58 smithi047 ceph-mon[32223]: pgmap v237: 97 pgs: 97 active+clean; 3.9 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.6 KiB/s wr, 9 op/s 2024-04-17T06:08:59.382 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:08:59 smithi098 ceph-mon[36516]: pgmap v238: 97 pgs: 97 active+clean; 3.9 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.6 KiB/s wr, 9 op/s 2024-04-17T06:08:59.388 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:08:59 smithi047 ceph-mon[32223]: pgmap v238: 97 pgs: 97 active+clean; 3.9 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.6 KiB/s wr, 9 op/s 2024-04-17T06:08:59.533 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:08:59 smithi103 ceph-mon[36243]: pgmap v238: 97 pgs: 97 active+clean; 3.9 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.6 KiB/s wr, 9 op/s 2024-04-17T06:09:01.783 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:09:01 smithi103 ceph-mon[36243]: pgmap v239: 97 pgs: 97 active+clean; 3.9 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-04-17T06:09:01.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:09:01 smithi098 ceph-mon[36516]: pgmap v239: 97 pgs: 97 active+clean; 3.9 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-04-17T06:09:01.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:09:01 smithi047 ceph-mon[32223]: pgmap v239: 97 pgs: 97 active+clean; 3.9 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-04-17T06:09:03.783 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:09:03 smithi103 ceph-mon[36243]: pgmap v240: 97 pgs: 97 active+clean; 3.9 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:09:03.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:09:03 smithi098 ceph-mon[36516]: pgmap v240: 97 pgs: 97 active+clean; 3.9 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:09:03.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:09:03 smithi047 ceph-mon[32223]: pgmap v240: 97 pgs: 97 active+clean; 3.9 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:09:05.783 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:09:05 smithi103 ceph-mon[36243]: pgmap v241: 97 pgs: 97 active+clean; 4.0 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 62 KiB/s wr, 33 op/s 2024-04-17T06:09:05.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:09:05 smithi098 ceph-mon[36516]: pgmap v241: 97 pgs: 97 active+clean; 4.0 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 62 KiB/s wr, 33 op/s 2024-04-17T06:09:05.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:09:05 smithi047 ceph-mon[32223]: pgmap v241: 97 pgs: 97 active+clean; 4.0 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 62 KiB/s wr, 33 op/s 2024-04-17T06:09:07.783 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:09:07 smithi103 ceph-mon[36243]: pgmap v242: 97 pgs: 97 active+clean; 4.0 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 57 KiB/s wr, 28 op/s 2024-04-17T06:09:07.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:09:07 smithi098 ceph-mon[36516]: pgmap v242: 97 pgs: 97 active+clean; 4.0 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 57 KiB/s wr, 28 op/s 2024-04-17T06:09:07.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:09:07 smithi047 ceph-mon[32223]: pgmap v242: 97 pgs: 97 active+clean; 4.0 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 57 KiB/s wr, 28 op/s 2024-04-17T06:09:08.633 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:09:08.633 INFO:tasks.workunit.client.0.smithi047.stdout:313/600: open fd = 3 2024-04-17T06:09:08.633 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.633 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.633 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:09:08.633 INFO:tasks.workunit.client.0.smithi047.stdout:314/600: open fd = 3 2024-04-17T06:09:08.633 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.633 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.633 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:09:08.633 INFO:tasks.workunit.client.0.smithi047.stdout:315/600: open fd = 3 2024-04-17T06:09:08.634 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.634 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.634 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:09:08.634 INFO:tasks.workunit.client.0.smithi047.stdout:316/600: open fd = 3 2024-04-17T06:09:08.634 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.634 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.634 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:09:08.634 INFO:tasks.workunit.client.0.smithi047.stdout:317/600: open fd = 3 2024-04-17T06:09:08.634 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.634 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.634 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:09:08.634 INFO:tasks.workunit.client.0.smithi047.stdout:318/600: open fd = 3 2024-04-17T06:09:08.634 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.634 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.634 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:09:08.635 INFO:tasks.workunit.client.0.smithi047.stdout:319/600: open fd = 3 2024-04-17T06:09:08.635 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.635 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.635 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:09:08.635 INFO:tasks.workunit.client.0.smithi047.stdout:320/600: open fd = 3 2024-04-17T06:09:08.635 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.635 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.635 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:09:08.635 INFO:tasks.workunit.client.0.smithi047.stdout:321/600: open fd = 3 2024-04-17T06:09:08.635 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.635 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.635 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:09:08.635 INFO:tasks.workunit.client.0.smithi047.stdout:322/600: open fd = 3 2024-04-17T06:09:08.635 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.636 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.636 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:09:08.636 INFO:tasks.workunit.client.0.smithi047.stdout:323/600: open fd = 3 2024-04-17T06:09:08.636 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.636 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.636 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:09:08.636 INFO:tasks.workunit.client.0.smithi047.stdout:324/600: open fd = 3 2024-04-17T06:09:08.636 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.636 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.636 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:09:08.636 INFO:tasks.workunit.client.0.smithi047.stdout:325/600: open fd = 3 2024-04-17T06:09:08.636 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.636 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.636 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:09:08.636 INFO:tasks.workunit.client.0.smithi047.stdout:326/600: open fd = 3 2024-04-17T06:09:08.637 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.637 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.637 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:09:08.637 INFO:tasks.workunit.client.0.smithi047.stdout:327/600: open fd = 3 2024-04-17T06:09:08.637 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.637 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.637 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:09:08.637 INFO:tasks.workunit.client.0.smithi047.stdout:328/600: open fd = 3 2024-04-17T06:09:08.637 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.637 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.637 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:09:08.637 INFO:tasks.workunit.client.0.smithi047.stdout:329/600: open fd = 3 2024-04-17T06:09:08.637 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.637 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.638 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:09:08.638 INFO:tasks.workunit.client.0.smithi047.stdout:330/600: open fd = 3 2024-04-17T06:09:08.638 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.638 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.638 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:09:08.638 INFO:tasks.workunit.client.0.smithi047.stdout:331/600: open fd = 3 2024-04-17T06:09:08.638 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.638 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.638 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:09:08.638 INFO:tasks.workunit.client.0.smithi047.stdout:332/600: open fd = 3 2024-04-17T06:09:08.638 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.638 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.638 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:09:08.638 INFO:tasks.workunit.client.0.smithi047.stdout:333/600: open fd = 3 2024-04-17T06:09:08.638 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.639 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.639 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:09:08.639 INFO:tasks.workunit.client.0.smithi047.stdout:334/600: open fd = 3 2024-04-17T06:09:08.639 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.639 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.639 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:09:08.639 INFO:tasks.workunit.client.0.smithi047.stdout:335/600: open fd = 3 2024-04-17T06:09:08.639 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.639 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.639 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:09:08.639 INFO:tasks.workunit.client.0.smithi047.stdout:336/600: open fd = 3 2024-04-17T06:09:08.639 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.639 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.639 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:09:08.639 INFO:tasks.workunit.client.0.smithi047.stdout:337/600: open fd = 3 2024-04-17T06:09:08.640 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.640 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.640 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:09:08.640 INFO:tasks.workunit.client.0.smithi047.stdout:338/600: open fd = 3 2024-04-17T06:09:08.640 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.640 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.640 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:09:08.640 INFO:tasks.workunit.client.0.smithi047.stdout:339/600: open fd = 3 2024-04-17T06:09:08.640 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.640 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.640 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:09:08.640 INFO:tasks.workunit.client.0.smithi047.stdout:340/600: open fd = 3 2024-04-17T06:09:08.640 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.640 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.641 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:09:08.641 INFO:tasks.workunit.client.0.smithi047.stdout:341/600: open fd = 3 2024-04-17T06:09:08.641 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.641 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.641 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:09:08.641 INFO:tasks.workunit.client.0.smithi047.stdout:342/600: open fd = 3 2024-04-17T06:09:08.641 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.641 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.641 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:09:08.641 INFO:tasks.workunit.client.0.smithi047.stdout:343/600: open fd = 3 2024-04-17T06:09:08.641 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.641 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.641 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:09:08.641 INFO:tasks.workunit.client.0.smithi047.stdout:344/600: open fd = 3 2024-04-17T06:09:08.641 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.642 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.642 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:09:08.642 INFO:tasks.workunit.client.0.smithi047.stdout:345/600: open fd = 3 2024-04-17T06:09:08.642 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.642 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.642 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:09:08.642 INFO:tasks.workunit.client.0.smithi047.stdout:346/600: open fd = 3 2024-04-17T06:09:08.642 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.642 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.642 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:09:08.642 INFO:tasks.workunit.client.0.smithi047.stdout:347/600: open fd = 3 2024-04-17T06:09:08.642 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.643 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.643 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:09:08.643 INFO:tasks.workunit.client.0.smithi047.stdout:348/600: open fd = 3 2024-04-17T06:09:08.643 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.643 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.643 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:09:08.643 INFO:tasks.workunit.client.0.smithi047.stdout:349/600: open fd = 3 2024-04-17T06:09:08.643 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.643 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.643 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:09:08.643 INFO:tasks.workunit.client.0.smithi047.stdout:350/600: open fd = 3 2024-04-17T06:09:08.643 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.643 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.643 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:09:08.644 INFO:tasks.workunit.client.0.smithi047.stdout:351/600: open fd = 3 2024-04-17T06:09:08.644 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.644 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.644 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:09:08.644 INFO:tasks.workunit.client.0.smithi047.stdout:352/600: open fd = 3 2024-04-17T06:09:08.644 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.644 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.644 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:09:08.644 INFO:tasks.workunit.client.0.smithi047.stdout:353/600: open fd = 3 2024-04-17T06:09:08.644 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.644 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.644 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:09:08.644 INFO:tasks.workunit.client.0.smithi047.stdout:354/600: open fd = 3 2024-04-17T06:09:08.644 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.644 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.645 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:09:08.645 INFO:tasks.workunit.client.0.smithi047.stdout:355/600: open fd = 3 2024-04-17T06:09:08.645 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.645 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.645 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:09:08.645 INFO:tasks.workunit.client.0.smithi047.stdout:356/600: open fd = 3 2024-04-17T06:09:08.645 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.645 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.645 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:09:08.645 INFO:tasks.workunit.client.0.smithi047.stdout:357/600: open fd = 3 2024-04-17T06:09:08.645 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.645 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.645 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:09:08.645 INFO:tasks.workunit.client.0.smithi047.stdout:358/600: open fd = 3 2024-04-17T06:09:08.645 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.646 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.646 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:09:08.646 INFO:tasks.workunit.client.0.smithi047.stdout:359/600: open fd = 3 2024-04-17T06:09:08.646 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.646 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.646 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:09:08.646 INFO:tasks.workunit.client.0.smithi047.stdout:360/600: open fd = 3 2024-04-17T06:09:08.646 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.646 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.646 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:09:08.646 INFO:tasks.workunit.client.0.smithi047.stdout:361/600: open fd = 3 2024-04-17T06:09:08.646 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.646 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.646 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:09:08.646 INFO:tasks.workunit.client.0.smithi047.stdout:362/600: open fd = 3 2024-04-17T06:09:08.647 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.647 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.647 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:09:08.647 INFO:tasks.workunit.client.0.smithi047.stdout:363/600: open fd = 3 2024-04-17T06:09:08.647 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.647 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.647 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:09:08.647 INFO:tasks.workunit.client.0.smithi047.stdout:364/600: open fd = 3 2024-04-17T06:09:08.647 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.647 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.647 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:09:08.647 INFO:tasks.workunit.client.0.smithi047.stdout:365/600: open fd = 3 2024-04-17T06:09:08.647 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.647 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.648 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:09:08.648 INFO:tasks.workunit.client.0.smithi047.stdout:366/600: open fd = 3 2024-04-17T06:09:08.648 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.648 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.648 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:09:08.648 INFO:tasks.workunit.client.0.smithi047.stdout:367/600: open fd = 3 2024-04-17T06:09:08.648 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.648 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.648 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:09:08.648 INFO:tasks.workunit.client.0.smithi047.stdout:368/600: open fd = 3 2024-04-17T06:09:08.648 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.648 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.648 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:09:08.648 INFO:tasks.workunit.client.0.smithi047.stdout:369/600: open fd = 3 2024-04-17T06:09:08.648 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.649 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.649 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:09:08.649 INFO:tasks.workunit.client.0.smithi047.stdout:370/600: open fd = 3 2024-04-17T06:09:08.649 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.649 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.649 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:09:08.649 INFO:tasks.workunit.client.0.smithi047.stdout:371/600: open fd = 3 2024-04-17T06:09:08.649 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.649 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.649 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:09:08.649 INFO:tasks.workunit.client.0.smithi047.stdout:372/600: open fd = 3 2024-04-17T06:09:08.649 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.649 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.649 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:09:08.650 INFO:tasks.workunit.client.0.smithi047.stdout:373/600: open fd = 3 2024-04-17T06:09:08.650 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.650 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.650 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:09:08.650 INFO:tasks.workunit.client.0.smithi047.stdout:374/600: open fd = 3 2024-04-17T06:09:08.650 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.650 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:09:08.650 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:09:09.186 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.186+0000 7f9aa4172640 1 -- 172.21.15.47:0/2905807950 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a9c06acd0 msgr2=0x7f9a9c06b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:09.186 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.186+0000 7f9aa4172640 1 --2- 172.21.15.47:0/2905807950 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a9c06acd0 0x7f9a9c06b150 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c009900 tx=0x7f9a8c02bc30 comp rx=0 tx=0).stop 2024-04-17T06:09:09.186 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.186+0000 7f9aa4172640 1 -- 172.21.15.47:0/2905807950 shutdown_connections 2024-04-17T06:09:09.186 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.186+0000 7f9aa4172640 1 --2- 172.21.15.47:0/2905807950 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9a9c104950 0x7f9a9c104dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:09.186 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.186+0000 7f9aa4172640 1 --2- 172.21.15.47:0/2905807950 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a9c06acd0 0x7f9a9c06b150 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:09.186 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.186+0000 7f9aa4172640 1 --2- 172.21.15.47:0/2905807950 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9a9c106fb0 0x7f9a9c06a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:09.186 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.186+0000 7f9aa4172640 1 -- 172.21.15.47:0/2905807950 >> 172.21.15.47:0/2905807950 conn(0x7f9a9c076c60 msgr2=0x7f9a9c077070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:09.186 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.186+0000 7f9aa4172640 1 -- 172.21.15.47:0/2905807950 shutdown_connections 2024-04-17T06:09:09.186 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.186+0000 7f9aa4172640 1 -- 172.21.15.47:0/2905807950 wait complete. 2024-04-17T06:09:09.186 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.186+0000 7f9aa4172640 1 Processor -- start 2024-04-17T06:09:09.186 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.187+0000 7f9aa4172640 1 -- start start 2024-04-17T06:09:09.187 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.187+0000 7f9aa4172640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a9c06acd0 0x7f9a9c1a4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:09.187 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.187+0000 7f9aa4172640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9a9c104950 0x7f9a9c1a4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:09.187 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.187+0000 7f9aa4172640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9a9c106fb0 0x7f9a9c19ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:09.187 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.187+0000 7f9aa4172640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f9a9c1188f0 con 0x7f9a9c06acd0 2024-04-17T06:09:09.187 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.187+0000 7f9aa4172640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9a9c118770 con 0x7f9a9c104950 2024-04-17T06:09:09.187 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.187+0000 7f9aa4172640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9a9c118a70 con 0x7f9a9c106fb0 2024-04-17T06:09:09.187 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.187+0000 7f9aa1f0e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a9c06acd0 0x7f9a9c1a4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:09.187 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.187+0000 7f9aa1f0e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a9c06acd0 0x7f9a9c1a4880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47350/0 (socket says 172.21.15.47:47350) 2024-04-17T06:09:09.187 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.187+0000 7f9aa1f0e640 1 -- 172.21.15.47:0/3786028865 learned_addr learned my addr 172.21.15.47:0/3786028865 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:09:09.187 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.187+0000 7f9aa170d640 1 --2- 172.21.15.47:0/3786028865 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9a9c104950 0x7f9a9c1a4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:09.187 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.187+0000 7f9aa270f640 1 --2- 172.21.15.47:0/3786028865 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9a9c106fb0 0x7f9a9c19ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:09.187 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.187+0000 7f9aa1f0e640 1 -- 172.21.15.47:0/3786028865 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9a9c106fb0 msgr2=0x7f9a9c19ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:09.187 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.187+0000 7f9aa1f0e640 1 --2- 172.21.15.47:0/3786028865 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9a9c106fb0 0x7f9a9c19ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:09.187 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.187+0000 7f9aa1f0e640 1 -- 172.21.15.47:0/3786028865 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9a9c104950 msgr2=0x7f9a9c1a4dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:09.187 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.187+0000 7f9aa1f0e640 1 --2- 172.21.15.47:0/3786028865 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9a9c104950 0x7f9a9c1a4dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:09.188 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.187+0000 7f9aa1f0e640 1 -- 172.21.15.47:0/3786028865 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a9c19f1b0 con 0x7f9a9c06acd0 2024-04-17T06:09:09.188 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.187+0000 7f9aa270f640 1 --2- 172.21.15.47:0/3786028865 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9a9c106fb0 0x7f9a9c19ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:09:09.188 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.187+0000 7f9aa1f0e640 1 --2- 172.21.15.47:0/3786028865 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a9c06acd0 0x7f9a9c1a4880 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f9a9800d950 tx=0x7f9a9800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:09.188 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.188+0000 7f9a92ffd640 1 -- 172.21.15.47:0/3786028865 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a98017070 con 0x7f9a9c06acd0 2024-04-17T06:09:09.188 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.188+0000 7f9a92ffd640 1 -- 172.21.15.47:0/3786028865 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a98009c30 con 0x7f9a9c06acd0 2024-04-17T06:09:09.188 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.188+0000 7f9aa4172640 1 -- 172.21.15.47:0/3786028865 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a9c19f4a0 con 0x7f9a9c06acd0 2024-04-17T06:09:09.188 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.188+0000 7f9a92ffd640 1 -- 172.21.15.47:0/3786028865 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a9800f6b0 con 0x7f9a9c06acd0 2024-04-17T06:09:09.188 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.188+0000 7f9aa4172640 1 -- 172.21.15.47:0/3786028865 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a9c1ab740 con 0x7f9a9c06acd0 2024-04-17T06:09:09.195 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.190+0000 7f9a92ffd640 1 -- 172.21.15.47:0/3786028865 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9a980109d0 con 0x7f9a9c06acd0 2024-04-17T06:09:09.195 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.190+0000 7f9aa6e36740 1 Processor -- start 2024-04-17T06:09:09.195 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.191+0000 7f9aa6e36740 1 -- start start 2024-04-17T06:09:09.195 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.191+0000 7f9aa6e36740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f2b54520c0 0x55f2b54524a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:09.196 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.191+0000 7f9aa6e36740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2b54529e0 0x55f2b545b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:09.196 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.191+0000 7f9aa6e36740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f2b545b7b0 0x55f2b545dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:09.196 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.191+0000 7f9aa6e36740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55f2b53b77d0 con 0x55f2b545b7b0 2024-04-17T06:09:09.196 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.191+0000 7f9aa6e36740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55f2b5354f70 con 0x55f2b54520c0 2024-04-17T06:09:09.196 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.191+0000 7f9aa6e36740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f2b53b3790 con 0x55f2b54529e0 2024-04-17T06:09:09.196 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.191+0000 7f9aa170d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f2b545b7b0 0x55f2b545dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:09.196 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.191+0000 7f9aa170d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f2b545b7b0 0x55f2b545dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47358/0 (socket says 172.21.15.47:47358) 2024-04-17T06:09:09.196 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.191+0000 7f9aa170d640 1 -- 172.21.15.47:0/2287053274 learned_addr learned my addr 172.21.15.47:0/2287053274 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:09:09.196 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.191+0000 7f9a92ffd640 1 --2- 172.21.15.47:0/3786028865 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f9a880a3dc0 0x7f9a880a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:09.196 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.191+0000 7f9aa1f0e640 1 --2- 172.21.15.47:0/2287053274 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2b54529e0 0x55f2b545b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:09.196 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.191+0000 7f9aa270f640 1 --2- 172.21.15.47:0/2287053274 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f2b54520c0 0x55f2b54524a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:09.196 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.192+0000 7f9aa170d640 1 -- 172.21.15.47:0/2287053274 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2b54529e0 msgr2=0x55f2b545b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:09.196 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.192+0000 7f9aa270f640 1 --2- 172.21.15.47:0/3786028865 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f9a880a3dc0 0x7f9a880a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:09.197 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.192+0000 7f9aa170d640 1 --2- 172.21.15.47:0/2287053274 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2b54529e0 0x55f2b545b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:09.197 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.192+0000 7f9aa170d640 1 -- 172.21.15.47:0/2287053274 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f2b54520c0 msgr2=0x55f2b54524a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:09.197 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.192+0000 7f9aa170d640 1 --2- 172.21.15.47:0/2287053274 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f2b54520c0 0x55f2b54524a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:09.197 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.192+0000 7f9aa170d640 1 -- 172.21.15.47:0/2287053274 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2b50ca210 con 0x55f2b545b7b0 2024-04-17T06:09:09.197 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.192+0000 7f9a92ffd640 1 -- 172.21.15.47:0/3786028865 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8469+0+0 (secure 0 0 0) 0x7f9a980d0420 con 0x7f9a9c06acd0 2024-04-17T06:09:09.197 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.192+0000 7f9aa170d640 1 --2- 172.21.15.47:0/2287053274 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f2b545b7b0 0x55f2b545dba0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c02c170 tx=0x7f9a8c0095e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49cb232c47b90102 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:09.197 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.192+0000 7f9aa270f640 1 --2- 172.21.15.47:0/3786028865 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f9a880a3dc0 0x7f9a880a6280 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9a94005c40 tx=0x7f9a94009500 comp rx=0 tx=0).ready entity=mgr.24923 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:09.197 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.192+0000 7f9a87fff640 1 -- 172.21.15.47:0/2287053274 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a8c042070 con 0x55f2b545b7b0 2024-04-17T06:09:09.197 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.192+0000 7f9a87fff640 1 -- 172.21.15.47:0/2287053274 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a8c02ca80 con 0x55f2b545b7b0 2024-04-17T06:09:09.197 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.192+0000 7f9a87fff640 1 -- 172.21.15.47:0/2287053274 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a8c004ec0 con 0x55f2b545b7b0 2024-04-17T06:09:09.197 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.193+0000 7f9aa6e36740 1 -- 172.21.15.47:0/2287053274 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f2b545b7b0 msgr2=0x55f2b545dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:09.197 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.193+0000 7f9aa6e36740 1 --2- 172.21.15.47:0/2287053274 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f2b545b7b0 0x55f2b545dba0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c02c170 tx=0x7f9a8c0095e0 comp rx=0 tx=0).stop 2024-04-17T06:09:09.197 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.193+0000 7f9aa6e36740 1 -- 172.21.15.47:0/2287053274 shutdown_connections 2024-04-17T06:09:09.197 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.193+0000 7f9aa6e36740 1 --2- 172.21.15.47:0/2287053274 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f2b545b7b0 0x55f2b545dba0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:09.198 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.193+0000 7f9aa6e36740 1 --2- 172.21.15.47:0/2287053274 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2b54529e0 0x55f2b545b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:09.198 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.193+0000 7f9aa6e36740 1 --2- 172.21.15.47:0/2287053274 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f2b54520c0 0x55f2b54524a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:09.198 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.193+0000 7f9aa6e36740 1 -- 172.21.15.47:0/2287053274 >> 172.21.15.47:0/2287053274 conn(0x55f2b544c9e0 msgr2=0x55f2b544cdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:09.198 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.193+0000 7f9aa6e36740 1 -- 172.21.15.47:0/2287053274 shutdown_connections 2024-04-17T06:09:09.198 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.193+0000 7f9aa6e36740 1 -- 172.21.15.47:0/2287053274 wait complete. 2024-04-17T06:09:09.198 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.193+0000 7f9aa6e36740 1 -- start start 2024-04-17T06:09:09.198 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.194+0000 7f9aa6e36740 1 Processor -- start 2024-04-17T06:09:09.198 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.194+0000 7f9aa6e36740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f2b54520c0 0x55f2b5458830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:09.198 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.194+0000 7f9aa6e36740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f2b54529e0 0x55f2b5458d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:09.198 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.194+0000 7f9aa6e36740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2b545b7b0 0x55f2b54592b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:09.198 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.194+0000 7f9aa6e36740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55f2b545e380 con 0x55f2b54529e0 2024-04-17T06:09:09.198 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.194+0000 7f9aa6e36740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55f2b533fe90 con 0x55f2b54520c0 2024-04-17T06:09:09.198 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.194+0000 7f9aa6e36740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f2b545e500 con 0x55f2b545b7b0 2024-04-17T06:09:09.199 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.194+0000 7f9aa6e36740 10 client.0 _ll_register_callbacks cb 0x55f2b5355d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:09:09.199 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.194+0000 7f9aa170d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f2b54529e0 0x55f2b5458d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:09.199 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.194+0000 7f9aa170d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f2b54529e0 0x55f2b5458d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47372/0 (socket says 172.21.15.47:47372) 2024-04-17T06:09:09.199 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.194+0000 7f9aa170d640 1 -- 172.21.15.47:0/1382755651 learned_addr learned my addr 172.21.15.47:0/1382755651 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:09:09.199 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.194+0000 7f9aa1f0e640 1 --2- 172.21.15.47:0/1382755651 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f2b54520c0 0x55f2b5458830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:09.199 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.194+0000 7f9aa170d640 1 -- 172.21.15.47:0/1382755651 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2b545b7b0 msgr2=0x55f2b54592b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:09.199 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.194+0000 7f9aa270f640 1 --2- 172.21.15.47:0/1382755651 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2b545b7b0 0x55f2b54592b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:09.199 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.195+0000 7f9aa170d640 1 --2- 172.21.15.47:0/1382755651 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2b545b7b0 0x55f2b54592b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:09.199 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.195+0000 7f9aa170d640 1 -- 172.21.15.47:0/1382755651 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f2b54520c0 msgr2=0x55f2b5458830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:09.199 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.195+0000 7f9aa170d640 1 --2- 172.21.15.47:0/1382755651 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f2b54520c0 0x55f2b5458830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:09.199 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.195+0000 7f9aa170d640 1 -- 172.21.15.47:0/1382755651 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2b5196620 con 0x55f2b54529e0 2024-04-17T06:09:09.199 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.195+0000 7f9aa1f0e640 1 --2- 172.21.15.47:0/1382755651 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f2b54520c0 0x55f2b5458830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:09:09.199 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.195+0000 7f9aa170d640 1 --2- 172.21.15.47:0/1382755651 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f2b54529e0 0x55f2b5458d70 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c000c00 tx=0x7f9a8c02c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de50b48b8fbac5c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:09.199 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.195+0000 7f9a857fa640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-17T06:09:09.200 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.195+0000 7f9a857fa640 1 -- 172.21.15.47:0/1382755651 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a8c042040 con 0x55f2b54529e0 2024-04-17T06:09:09.200 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.195+0000 7f9a857fa640 1 -- 172.21.15.47:0/1382755651 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8469+0+0 (secure 0 0 0) 0x7f9a8c004b50 con 0x55f2b54529e0 2024-04-17T06:09:09.200 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.195+0000 7f9aa6e36740 1 -- 172.21.15.47:0/1382755651 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a8c02ca80 con 0x55f2b54529e0 2024-04-17T06:09:09.200 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.195+0000 7f9a857fa640 1 -- 172.21.15.47:0/1382755651 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9a8c004ec0 con 0x55f2b54529e0 2024-04-17T06:09:09.200 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.195+0000 7f9aa6e36740 10 client.16056 fetch_fsmap learned FSMap version 32 2024-04-17T06:09:09.200 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.195+0000 7f9aa6e36740 1 -- 172.21.15.47:0/1382755651 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f2b546a670 con 0x55f2b54529e0 2024-04-17T06:09:09.200 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.195+0000 7f9a857fa640 1 -- 172.21.15.47:0/1382755651 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a8c02ca80 con 0x55f2b54529e0 2024-04-17T06:09:09.200 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.196+0000 7f9a857fa640 1 -- 172.21.15.47:0/1382755651 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) ==== 8339+0+0 (secure 0 0 0) 0x7f9a8c003bd0 con 0x55f2b54529e0 2024-04-17T06:09:09.200 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.196+0000 7f9aa6e36740 10 client.16056 fetch_fsmap finished waiting for FSMap version 32 2024-04-17T06:09:09.200 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.196+0000 7f9aa6e36740 10 client.16056 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-17T06:09:09.200 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.196+0000 7f9aa6e36740 20 client.16056 populate_metadata read hostname 'smithi047' 2024-04-17T06:09:09.200 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.196+0000 7f9aa6e36740 1 --2- 172.21.15.47:0/1382755651 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55f2b546b9c0 0x55f2b546ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:09.200 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.196+0000 7f9aa6e36740 4 client.16056 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T06:09:09.200 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.196+0000 7f9aa6e36740 1 -- 172.21.15.47:0/1382755651 --> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f2b53c51f0 con 0x55f2b546b9c0 2024-04-17T06:09:09.200 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.196+0000 7f9aa1f0e640 1 --2- 172.21.15.47:0/1382755651 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55f2b546b9c0 0x55f2b546ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:09.201 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.196+0000 7f9aa1f0e640 1 --2- 172.21.15.47:0/1382755651 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55f2b546b9c0 0x55f2b546ddb0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5583b1289bba4d85 server_cookie=bed92c3ccc6cd196 in_seq=0 out_seq=0 2024-04-17T06:09:09.201 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.196+0000 7f9a857fa640 10 client.16056 ms_handle_connect on v2:172.21.15.98:6838/2841287917 2024-04-17T06:09:09.201 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.197+0000 7f9a857fa640 1 -- 172.21.15.47:0/1382755651 <== mds.0 v2:172.21.15.98:6838/2841287917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f2b53c51f0 con 0x55f2b546b9c0 2024-04-17T06:09:09.201 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.197+0000 7f9a857fa640 10 client.16056 handle_command_reply: tid=0 2024-04-17T06:09:09.201 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.197+0000 7f9aa6e36740 1 client.16056 shutdown 2024-04-17T06:09:09.201 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.197+0000 7f9aa6e36740 1 -- 172.21.15.47:0/1382755651 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f2b54529e0 msgr2=0x55f2b5458d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:09.201 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.197+0000 7f9aa6e36740 1 --2- 172.21.15.47:0/1382755651 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f2b54529e0 0x55f2b5458d70 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c000c00 tx=0x7f9a8c02c3b0 comp rx=0 tx=0).stop 2024-04-17T06:09:09.201 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.197+0000 7f9aa6e36740 1 -- 172.21.15.47:0/1382755651 shutdown_connections 2024-04-17T06:09:09.201 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.197+0000 7f9aa6e36740 1 --2- 172.21.15.47:0/1382755651 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55f2b546b9c0 0x55f2b546ddb0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:09.201 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.197+0000 7f9aa6e36740 1 --2- 172.21.15.47:0/1382755651 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2b545b7b0 0x55f2b54592b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:09.201 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.197+0000 7f9aa6e36740 1 --2- 172.21.15.47:0/1382755651 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f2b54529e0 0x55f2b5458d70 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:09.201 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.197+0000 7f9aa6e36740 1 --2- 172.21.15.47:0/1382755651 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f2b54520c0 0x55f2b5458830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:09.201 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.197+0000 7f9a857fa640 0 client.16056 ms_handle_reset on v2:172.21.15.98:6838/2841287917 2024-04-17T06:09:09.202 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.197+0000 7f9aa6e36740 1 -- 172.21.15.47:0/1382755651 >> 172.21.15.47:0/1382755651 conn(0x55f2b544c9e0 msgr2=0x55f2b544e730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:09.202 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.197+0000 7f9aa6e36740 1 -- 172.21.15.47:0/1382755651 shutdown_connections 2024-04-17T06:09:09.202 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.197+0000 7f9aa6e36740 1 -- 172.21.15.47:0/1382755651 wait complete. 2024-04-17T06:09:09.202 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.197+0000 7f9aa6e36740 20 client.16056 trim_cache size 0 max 16384 2024-04-17T06:09:09.215 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.212+0000 7f9aa6e36740 1 Processor -- start 2024-04-17T06:09:09.215 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.212+0000 7f9aa6e36740 1 -- start start 2024-04-17T06:09:09.215 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.213+0000 7f9aa6e36740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f2b54520c0 0x55f2b545af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:09.215 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.213+0000 7f9aa6e36740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f2b54529e0 0x55f2b544b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:09.215 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.213+0000 7f9aa6e36740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2b545b7b0 0x55f2b5470dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:09.215 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.213+0000 7f9aa6e36740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f9a9c1b68d0 con 0x55f2b54529e0 2024-04-17T06:09:09.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.213+0000 7f9aa6e36740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9a9c1b4190 con 0x55f2b54520c0 2024-04-17T06:09:09.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.213+0000 7f9aa6e36740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9a9c1b1fe0 con 0x55f2b545b7b0 2024-04-17T06:09:09.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.213+0000 7f9aa170d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f2b54529e0 0x55f2b544b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:09.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.213+0000 7f9aa170d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f2b54529e0 0x55f2b544b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47382/0 (socket says 172.21.15.47:47382) 2024-04-17T06:09:09.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.213+0000 7f9aa170d640 1 -- 172.21.15.47:0/1459531413 learned_addr learned my addr 172.21.15.47:0/1459531413 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:09:09.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.213+0000 7f9aa270f640 1 --2- 172.21.15.47:0/1459531413 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2b545b7b0 0x55f2b5470dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:09.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.213+0000 7f9aa170d640 1 -- 172.21.15.47:0/1459531413 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2b545b7b0 msgr2=0x55f2b5470dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:09.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.213+0000 7f9aa170d640 1 --2- 172.21.15.47:0/1459531413 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2b545b7b0 0x55f2b5470dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:09.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.213+0000 7f9aa170d640 1 -- 172.21.15.47:0/1459531413 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f2b54520c0 msgr2=0x55f2b545af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:09.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.213+0000 7f9aa170d640 1 --2- 172.21.15.47:0/1459531413 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f2b54520c0 0x55f2b545af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:09.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.213+0000 7f9aa170d640 1 -- 172.21.15.47:0/1459531413 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2b545dba0 con 0x55f2b54529e0 2024-04-17T06:09:09.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.213+0000 7f9aa270f640 1 --2- 172.21.15.47:0/1459531413 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2b545b7b0 0x55f2b5470dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:09:09.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.213+0000 7f9aa170d640 1 --2- 172.21.15.47:0/1459531413 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f2b54529e0 0x55f2b544b980 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c02bc70 tx=0x7f9a8c005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c2262e93a8f791a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:09.217 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.213+0000 7f9a877fe640 1 -- 172.21.15.47:0/1459531413 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a8c05b030 con 0x55f2b54529e0 2024-04-17T06:09:09.217 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.213+0000 7f9a877fe640 1 -- 172.21.15.47:0/1459531413 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a8c034e20 con 0x55f2b54529e0 2024-04-17T06:09:09.217 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.213+0000 7f9a877fe640 1 -- 172.21.15.47:0/1459531413 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a8c042040 con 0x55f2b54529e0 2024-04-17T06:09:09.217 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.214+0000 7f9aa6e36740 1 -- 172.21.15.47:0/1459531413 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f2b54529e0 msgr2=0x55f2b544b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:09.217 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.214+0000 7f9aa6e36740 1 --2- 172.21.15.47:0/1459531413 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f2b54529e0 0x55f2b544b980 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c02bc70 tx=0x7f9a8c005c00 comp rx=0 tx=0).stop 2024-04-17T06:09:09.217 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.214+0000 7f9aa6e36740 1 -- 172.21.15.47:0/1459531413 shutdown_connections 2024-04-17T06:09:09.217 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.214+0000 7f9aa6e36740 1 --2- 172.21.15.47:0/1459531413 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2b545b7b0 0x55f2b5470dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:09.217 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.214+0000 7f9aa6e36740 1 --2- 172.21.15.47:0/1459531413 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f2b54529e0 0x55f2b544b980 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:09.217 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.214+0000 7f9aa6e36740 1 --2- 172.21.15.47:0/1459531413 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f2b54520c0 0x55f2b545af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:09.217 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.214+0000 7f9aa6e36740 1 -- 172.21.15.47:0/1459531413 >> 172.21.15.47:0/1459531413 conn(0x55f2b544c9e0 msgr2=0x55f2b544b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:09.217 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.214+0000 7f9aa6e36740 1 -- 172.21.15.47:0/1459531413 shutdown_connections 2024-04-17T06:09:09.217 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.214+0000 7f9aa6e36740 1 -- 172.21.15.47:0/1459531413 wait complete. 2024-04-17T06:09:09.217 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.214+0000 7f9aa6e36740 1 -- start start 2024-04-17T06:09:09.217 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.214+0000 7f9aa6e36740 1 Processor -- start 2024-04-17T06:09:09.218 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.215+0000 7f9aa6e36740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2b54520c0 0x55f2b546eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:09.218 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.215+0000 7f9aa6e36740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f2b54529e0 0x55f2b546f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:09.218 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.215+0000 7f9aa6e36740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f2b545b7b0 0x55f2b54d0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:09.218 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.215+0000 7f9aa6e36740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f9a9c1b2650 con 0x55f2b54529e0 2024-04-17T06:09:09.218 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.215+0000 7f9aa6e36740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9a9c1b3ac0 con 0x55f2b545b7b0 2024-04-17T06:09:09.218 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.215+0000 7f9aa6e36740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9a9c1b2d20 con 0x55f2b54520c0 2024-04-17T06:09:09.218 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.215+0000 7f9aa6e36740 10 client.0 _ll_register_callbacks cb 0x7f9a8c005e40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:09:09.218 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.215+0000 7f9aa170d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f2b54529e0 0x55f2b546f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:09.218 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.215+0000 7f9aa170d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f2b54529e0 0x55f2b546f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47398/0 (socket says 172.21.15.47:47398) 2024-04-17T06:09:09.218 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.215+0000 7f9aa170d640 1 -- 172.21.15.47:0/1708560538 learned_addr learned my addr 172.21.15.47:0/1708560538 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:09:09.218 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.215+0000 7f9aa270f640 1 --2- 172.21.15.47:0/1708560538 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f2b545b7b0 0x55f2b54d0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:09.218 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.215+0000 7f9aa1f0e640 1 --2- 172.21.15.47:0/1708560538 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2b54520c0 0x55f2b546eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:09.218 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.215+0000 7f9aa170d640 1 -- 172.21.15.47:0/1708560538 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2b54520c0 msgr2=0x55f2b546eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:09.218 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.215+0000 7f9aa170d640 1 --2- 172.21.15.47:0/1708560538 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2b54520c0 0x55f2b546eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:09.219 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.215+0000 7f9aa170d640 1 -- 172.21.15.47:0/1708560538 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f2b545b7b0 msgr2=0x55f2b54d0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:09.219 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.215+0000 7f9aa170d640 1 --2- 172.21.15.47:0/1708560538 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f2b545b7b0 0x55f2b54d0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:09.219 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.215+0000 7f9aa170d640 1 -- 172.21.15.47:0/1708560538 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2b546f980 con 0x55f2b54529e0 2024-04-17T06:09:09.219 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.215+0000 7f9aa170d640 1 --2- 172.21.15.47:0/1708560538 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f2b54529e0 0x55f2b546f0b0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c02c170 tx=0x7f9a8c02caa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37cde4547d5bfa77 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:09.219 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.215+0000 7f9a857fa640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-17T06:09:09.219 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.216+0000 7f9a857fa640 1 -- 172.21.15.47:0/1708560538 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a8c05b070 con 0x55f2b54529e0 2024-04-17T06:09:09.219 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.216+0000 7f9aa6e36740 1 -- 172.21.15.47:0/1708560538 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a8c034e20 con 0x55f2b54529e0 2024-04-17T06:09:09.219 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.216+0000 7f9a857fa640 1 -- 172.21.15.47:0/1708560538 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8469+0+0 (secure 0 0 0) 0x7f9a8c0348f0 con 0x55f2b54529e0 2024-04-17T06:09:09.219 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.216+0000 7f9a857fa640 1 -- 172.21.15.47:0/1708560538 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9a8c042040 con 0x55f2b54529e0 2024-04-17T06:09:09.219 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.216+0000 7f9aa6e36740 10 client.16062 fetch_fsmap learned FSMap version 32 2024-04-17T06:09:09.219 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.216+0000 7f9aa6e36740 1 -- 172.21.15.47:0/1708560538 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f2b54d2160 con 0x55f2b54529e0 2024-04-17T06:09:09.219 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.216+0000 7f9a857fa640 1 -- 172.21.15.47:0/1708560538 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a8c034e20 con 0x55f2b54529e0 2024-04-17T06:09:09.219 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.216+0000 7f9a857fa640 1 -- 172.21.15.47:0/1708560538 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) ==== 8339+0+0 (secure 0 0 0) 0x7f9a8c003850 con 0x55f2b54529e0 2024-04-17T06:09:09.219 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.216+0000 7f9aa6e36740 10 client.16062 fetch_fsmap finished waiting for FSMap version 32 2024-04-17T06:09:09.219 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.216+0000 7f9aa6e36740 10 client.16062 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-17T06:09:09.220 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.216+0000 7f9aa6e36740 20 client.16062 populate_metadata read hostname 'smithi047' 2024-04-17T06:09:09.220 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.216+0000 7f9aa6e36740 1 --2- 172.21.15.47:0/1708560538 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55f2b54d2580 0x55f2b54d49a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:09.220 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.216+0000 7f9aa6e36740 4 client.16062 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-17T06:09:09.220 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.216+0000 7f9aa6e36740 1 -- 172.21.15.47:0/1708560538 --> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f2b53bf060 con 0x55f2b54d2580 2024-04-17T06:09:09.220 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.217+0000 7f9aa1f0e640 1 --2- 172.21.15.47:0/1708560538 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55f2b54d2580 0x55f2b54d49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:09.220 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.217+0000 7f9aa1f0e640 1 --2- 172.21.15.47:0/1708560538 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55f2b54d2580 0x55f2b54d49a0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8da8b5a3c7b9889c server_cookie=27dec7f286b9deb in_seq=0 out_seq=0 2024-04-17T06:09:09.220 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.217+0000 7f9a857fa640 10 client.16062 ms_handle_connect on v2:172.21.15.98:6838/2841287917 2024-04-17T06:09:09.224 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.225+0000 7f9a857fa640 1 -- 172.21.15.47:0/1708560538 <== mds.0 v2:172.21.15.98:6838/2841287917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f2b53bf060 con 0x55f2b54d2580 2024-04-17T06:09:09.224 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.225+0000 7f9a857fa640 10 client.16062 handle_command_reply: tid=0 2024-04-17T06:09:09.224 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.225+0000 7f9aa6e36740 1 client.16062 shutdown 2024-04-17T06:09:09.224 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.225+0000 7f9aa6e36740 1 -- 172.21.15.47:0/1708560538 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f2b54529e0 msgr2=0x55f2b546f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:09.224 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.225+0000 7f9aa6e36740 1 --2- 172.21.15.47:0/1708560538 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f2b54529e0 0x55f2b546f0b0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c02c170 tx=0x7f9a8c02caa0 comp rx=0 tx=0).stop 2024-04-17T06:09:09.224 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.225+0000 7f9aa6e36740 1 -- 172.21.15.47:0/1708560538 shutdown_connections 2024-04-17T06:09:09.224 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.225+0000 7f9aa6e36740 1 --2- 172.21.15.47:0/1708560538 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55f2b54d2580 0x55f2b54d49a0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:09.224 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.225+0000 7f9aa6e36740 1 --2- 172.21.15.47:0/1708560538 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f2b545b7b0 0x55f2b54d0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:09.224 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.225+0000 7f9aa6e36740 1 --2- 172.21.15.47:0/1708560538 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f2b54529e0 0x55f2b546f0b0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:09.224 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.225+0000 7f9a857fa640 0 client.16062 ms_handle_reset on v2:172.21.15.98:6838/2841287917 2024-04-17T06:09:09.224 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.225+0000 7f9aa6e36740 1 --2- 172.21.15.47:0/1708560538 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2b54520c0 0x55f2b546eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:09.224 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.225+0000 7f9aa6e36740 1 -- 172.21.15.47:0/1708560538 >> 172.21.15.47:0/1708560538 conn(0x55f2b544c9e0 msgr2=0x55f2b5459060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:09.225 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.226+0000 7f9aa6e36740 1 -- 172.21.15.47:0/1708560538 shutdown_connections 2024-04-17T06:09:09.225 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.226+0000 7f9aa6e36740 1 -- 172.21.15.47:0/1708560538 wait complete. 2024-04-17T06:09:09.225 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.226+0000 7f9aa6e36740 20 client.16062 trim_cache size 0 max 16384 2024-04-17T06:09:09.225 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-04-17T06:09:09.225 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-04-17T06:09:09.225 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-04-17T06:09:09.225 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-04-17T06:09:09.226 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.226+0000 7f9aa4172640 1 -- 172.21.15.47:0/3786028865 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f9a880a3dc0 msgr2=0x7f9a880a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:09.226 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.226+0000 7f9aa4172640 1 --2- 172.21.15.47:0/3786028865 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f9a880a3dc0 0x7f9a880a6280 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9a94005c40 tx=0x7f9a94009500 comp rx=0 tx=0).stop 2024-04-17T06:09:09.226 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.226+0000 7f9aa4172640 1 -- 172.21.15.47:0/3786028865 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a9c06acd0 msgr2=0x7f9a9c1a4880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:09.226 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.226+0000 7f9aa4172640 1 --2- 172.21.15.47:0/3786028865 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a9c06acd0 0x7f9a9c1a4880 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f9a9800d950 tx=0x7f9a9800de10 comp rx=0 tx=0).stop 2024-04-17T06:09:09.226 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.226+0000 7f9aa4172640 1 -- 172.21.15.47:0/3786028865 shutdown_connections 2024-04-17T06:09:09.226 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.226+0000 7f9aa4172640 1 --2- 172.21.15.47:0/3786028865 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9a9c106fb0 0x7f9a9c19ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:09.226 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.226+0000 7f9aa4172640 1 --2- 172.21.15.47:0/3786028865 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9a9c104950 0x7f9a9c1a4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:09.227 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.226+0000 7f9aa4172640 1 --2- 172.21.15.47:0/3786028865 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f9a880a3dc0 0x7f9a880a6280 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:09.227 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.226+0000 7f9aa4172640 1 --2- 172.21.15.47:0/3786028865 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a9c06acd0 0x7f9a9c1a4880 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:09.227 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.226+0000 7f9aa4172640 1 -- 172.21.15.47:0/3786028865 >> 172.21.15.47:0/3786028865 conn(0x7f9a9c076c60 msgr2=0x7f9a9c103600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:09.227 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.226+0000 7f9aa4172640 1 -- 172.21.15.47:0/3786028865 shutdown_connections 2024-04-17T06:09:09.227 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:09.226+0000 7f9aa4172640 1 -- 172.21.15.47:0/3786028865 wait complete. 2024-04-17T06:09:09.783 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:09:09 smithi103 ceph-mon[36243]: pgmap v243: 97 pgs: 97 active+clean; 4.0 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 57 KiB/s wr, 28 op/s 2024-04-17T06:09:09.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:09:09 smithi098 ceph-mon[36516]: pgmap v243: 97 pgs: 97 active+clean; 4.0 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 57 KiB/s wr, 28 op/s 2024-04-17T06:09:09.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:09:09 smithi047 ceph-mon[32223]: pgmap v243: 97 pgs: 97 active+clean; 4.0 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 57 KiB/s wr, 28 op/s 2024-04-17T06:09:10.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:09:10 smithi103 ceph-mon[36243]: osdmap e95: 12 total, 12 up, 12 in 2024-04-17T06:09:10.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:09:10 smithi098 ceph-mon[36516]: osdmap e95: 12 total, 12 up, 12 in 2024-04-17T06:09:10.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:09:10 smithi047 ceph-mon[32223]: osdmap e95: 12 total, 12 up, 12 in 2024-04-17T06:09:11.783 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:09:11 smithi103 ceph-mon[36243]: pgmap v245: 97 pgs: 97 active+clean; 4.0 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 68 KiB/s wr, 34 op/s 2024-04-17T06:09:11.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:09:11 smithi098 ceph-mon[36516]: pgmap v245: 97 pgs: 97 active+clean; 4.0 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 68 KiB/s wr, 34 op/s 2024-04-17T06:09:11.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:09:11 smithi047 ceph-mon[32223]: pgmap v245: 97 pgs: 97 active+clean; 4.0 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 68 KiB/s wr, 34 op/s 2024-04-17T06:09:13.783 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:09:13 smithi103 ceph-mon[36243]: pgmap v246: 97 pgs: 97 active+clean; 4.0 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 68 KiB/s wr, 34 op/s 2024-04-17T06:09:13.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:09:13 smithi098 ceph-mon[36516]: pgmap v246: 97 pgs: 97 active+clean; 4.0 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 68 KiB/s wr, 34 op/s 2024-04-17T06:09:13.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:09:13 smithi047 ceph-mon[32223]: pgmap v246: 97 pgs: 97 active+clean; 4.0 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 68 KiB/s wr, 34 op/s 2024-04-17T06:09:16.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:09:15 smithi103 ceph-mon[36243]: pgmap v247: 97 pgs: 97 active+clean; 4.0 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-17T06:09:16.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:09:15 smithi103 ceph-mon[36243]: osdmap e96: 12 total, 12 up, 12 in 2024-04-17T06:09:16.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:09:15 smithi098 ceph-mon[36516]: pgmap v247: 97 pgs: 97 active+clean; 4.0 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-17T06:09:16.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:09:15 smithi098 ceph-mon[36516]: osdmap e96: 12 total, 12 up, 12 in 2024-04-17T06:09:16.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:09:15 smithi047 ceph-mon[32223]: pgmap v247: 97 pgs: 97 active+clean; 4.0 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-17T06:09:16.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:09:15 smithi047 ceph-mon[32223]: osdmap e96: 12 total, 12 up, 12 in 2024-04-17T06:09:18.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:09:17 smithi103 ceph-mon[36243]: pgmap v249: 97 pgs: 97 active+clean; 4.0 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-04-17T06:09:18.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:09:17 smithi098 ceph-mon[36516]: pgmap v249: 97 pgs: 97 active+clean; 4.0 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-04-17T06:09:18.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:09:17 smithi047 ceph-mon[32223]: pgmap v249: 97 pgs: 97 active+clean; 4.0 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-04-17T06:09:20.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:09:19 smithi103 ceph-mon[36243]: pgmap v250: 97 pgs: 97 active+clean; 4.0 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 6.5 KiB/s wr, 6 op/s 2024-04-17T06:09:20.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:09:19 smithi098 ceph-mon[36516]: pgmap v250: 97 pgs: 97 active+clean; 4.0 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 6.5 KiB/s wr, 6 op/s 2024-04-17T06:09:20.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:09:19 smithi047 ceph-mon[32223]: pgmap v250: 97 pgs: 97 active+clean; 4.0 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 6.5 KiB/s wr, 6 op/s 2024-04-17T06:09:22.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:09:21 smithi103 ceph-mon[36243]: pgmap v251: 97 pgs: 97 active+clean; 4.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.1 KiB/s wr, 11 op/s 2024-04-17T06:09:22.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:09:21 smithi098 ceph-mon[36516]: pgmap v251: 97 pgs: 97 active+clean; 4.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.1 KiB/s wr, 11 op/s 2024-04-17T06:09:22.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:09:21 smithi047 ceph-mon[32223]: pgmap v251: 97 pgs: 97 active+clean; 4.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.1 KiB/s wr, 11 op/s 2024-04-17T06:09:22.325 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.324+0000 7f8c54eba640 1 -- 172.21.15.47:0/3327024578 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8c50100e20 msgr2=0x7f8c5010edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:22.326 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.324+0000 7f8c54eba640 1 --2- 172.21.15.47:0/3327024578 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8c50100e20 0x7f8c5010edf0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f8c48009cd0 tx=0x7f8c4802c0f0 comp rx=0 tx=0).stop 2024-04-17T06:09:22.326 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.324+0000 7f8c54eba640 1 -- 172.21.15.47:0/3327024578 shutdown_connections 2024-04-17T06:09:22.326 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.324+0000 7f8c54eba640 1 --2- 172.21.15.47:0/3327024578 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8c50100e20 0x7f8c5010edf0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:22.326 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.324+0000 7f8c54eba640 1 --2- 172.21.15.47:0/3327024578 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8c50100460 0x7f8c501008e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:22.326 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.324+0000 7f8c54eba640 1 --2- 172.21.15.47:0/3327024578 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8c50106f40 0x7f8c50107320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:22.326 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.324+0000 7f8c54eba640 1 -- 172.21.15.47:0/3327024578 >> 172.21.15.47:0/3327024578 conn(0x7f8c500773f0 msgr2=0x7f8c500fe770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:22.326 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.324+0000 7f8c54eba640 1 -- 172.21.15.47:0/3327024578 shutdown_connections 2024-04-17T06:09:22.326 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.324+0000 7f8c54eba640 1 -- 172.21.15.47:0/3327024578 wait complete. 2024-04-17T06:09:22.327 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.325+0000 7f8c54eba640 1 Processor -- start 2024-04-17T06:09:22.327 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.325+0000 7f8c54eba640 1 -- start start 2024-04-17T06:09:22.327 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.325+0000 7f8c54eba640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8c50100460 0x7f8c50101640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:22.327 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.325+0000 7f8c54eba640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8c50100e20 0x7f8c50101b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:22.327 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.325+0000 7f8c54eba640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8c50106f40 0x7f8c501020c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:22.327 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.325+0000 7f8c54eba640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f8c501123d0 con 0x7f8c50106f40 2024-04-17T06:09:22.327 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.325+0000 7f8c54eba640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8c50112250 con 0x7f8c50100e20 2024-04-17T06:09:22.327 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.325+0000 7f8c54eba640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8c50112550 con 0x7f8c50100460 2024-04-17T06:09:22.327 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.325+0000 7f8c4ed9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8c50106f40 0x7f8c501020c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:22.327 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.325+0000 7f8c4ed9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8c50106f40 0x7f8c501020c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47892/0 (socket says 172.21.15.47:47892) 2024-04-17T06:09:22.327 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.325+0000 7f8c4ed9d640 1 -- 172.21.15.47:0/4125846957 learned_addr learned my addr 172.21.15.47:0/4125846957 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:09:22.327 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.325+0000 7f8c4e59c640 1 --2- 172.21.15.47:0/4125846957 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8c50100460 0x7f8c50101640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:22.327 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.326+0000 7f8c47fff640 1 --2- 172.21.15.47:0/4125846957 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8c50100e20 0x7f8c50101b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:22.328 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.326+0000 7f8c4ed9d640 1 -- 172.21.15.47:0/4125846957 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8c50100460 msgr2=0x7f8c50101640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:22.328 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.326+0000 7f8c4ed9d640 1 --2- 172.21.15.47:0/4125846957 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8c50100460 0x7f8c50101640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:22.328 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.326+0000 7f8c4ed9d640 1 -- 172.21.15.47:0/4125846957 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8c50100e20 msgr2=0x7f8c50101b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:22.328 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.326+0000 7f8c4ed9d640 1 --2- 172.21.15.47:0/4125846957 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8c50100e20 0x7f8c50101b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:22.328 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.326+0000 7f8c4ed9d640 1 -- 172.21.15.47:0/4125846957 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c501a6c90 con 0x7f8c50106f40 2024-04-17T06:09:22.328 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.326+0000 7f8c4e59c640 1 --2- 172.21.15.47:0/4125846957 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8c50100460 0x7f8c50101640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:09:22.328 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.326+0000 7f8c47fff640 1 --2- 172.21.15.47:0/4125846957 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8c50100e20 0x7f8c50101b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:09:22.328 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.326+0000 7f8c4ed9d640 1 --2- 172.21.15.47:0/4125846957 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8c50106f40 0x7f8c501020c0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f8c48004af0 tx=0x7f8c48032ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:22.328 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.326+0000 7f8c477fe640 1 -- 172.21.15.47:0/4125846957 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c48041020 con 0x7f8c50106f40 2024-04-17T06:09:22.328 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.326+0000 7f8c477fe640 1 -- 172.21.15.47:0/4125846957 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c48034cd0 con 0x7f8c50106f40 2024-04-17T06:09:22.328 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.326+0000 7f8c477fe640 1 -- 172.21.15.47:0/4125846957 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c48039040 con 0x7f8c50106f40 2024-04-17T06:09:22.328 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.326+0000 7f8c54eba640 1 -- 172.21.15.47:0/4125846957 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c501a6f20 con 0x7f8c50106f40 2024-04-17T06:09:22.328 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.326+0000 7f8c54eba640 1 -- 172.21.15.47:0/4125846957 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c501a73e0 con 0x7f8c50106f40 2024-04-17T06:09:22.335 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.328+0000 7f8c477fe640 1 -- 172.21.15.47:0/4125846957 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8c48044020 con 0x7f8c50106f40 2024-04-17T06:09:22.335 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.329+0000 7f8c57b7d740 1 Processor -- start 2024-04-17T06:09:22.336 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.329+0000 7f8c57b7d740 1 -- start start 2024-04-17T06:09:22.336 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.329+0000 7f8c57b7d740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cf7c91f0c0 0x55cf7c91f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:22.336 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.329+0000 7f8c57b7d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf7c91f9e0 0x55cf7c928270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:22.336 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.329+0000 7f8c57b7d740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cf7c9287b0 0x55cf7c92aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:22.336 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.329+0000 7f8c57b7d740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55cf7c8847d0 con 0x55cf7c9287b0 2024-04-17T06:09:22.336 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.329+0000 7f8c57b7d740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55cf7c821f70 con 0x55cf7c91f0c0 2024-04-17T06:09:22.336 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.329+0000 7f8c57b7d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55cf7c880790 con 0x55cf7c91f9e0 2024-04-17T06:09:22.336 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.329+0000 7f8c477fe640 1 --2- 172.21.15.47:0/4125846957 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f8c3c0a3df0 0x7f8c3c0a62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:22.336 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.330+0000 7f8c4ed9d640 1 --2- 172.21.15.47:0/4125846957 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f8c3c0a3df0 0x7f8c3c0a62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:22.336 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.330+0000 7f8c4e59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf7c91f9e0 0x55cf7c928270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:22.336 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.330+0000 7f8c47fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cf7c9287b0 0x55cf7c92aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:22.336 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.330+0000 7f8c4e59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf7c91f9e0 0x55cf7c928270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.47:59706/0 (socket says 172.21.15.47:59706) 2024-04-17T06:09:22.336 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.330+0000 7f8c4e59c640 1 -- 172.21.15.47:0/1820784147 learned_addr learned my addr 172.21.15.47:0/1820784147 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:09:22.336 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.330+0000 7f8c47fff640 1 --2- 172.21.15.47:0/1820784147 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cf7c91f0c0 0x55cf7c91f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:22.337 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.330+0000 7f8c477fe640 1 -- 172.21.15.47:0/4125846957 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8513+0+0 (secure 0 0 0) 0x7f8c480f2550 con 0x7f8c50106f40 2024-04-17T06:09:22.337 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.330+0000 7f8c4e59c640 1 -- 172.21.15.47:0/1820784147 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cf7c91f0c0 msgr2=0x55cf7c91f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:22.337 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.330+0000 7f8c4e59c640 1 --2- 172.21.15.47:0/1820784147 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cf7c91f0c0 0x55cf7c91f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:22.337 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.330+0000 7f8c4e59c640 1 -- 172.21.15.47:0/1820784147 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cf7c9287b0 msgr2=0x55cf7c92aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:22.337 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.330+0000 7f8c4e59c640 1 --2- 172.21.15.47:0/1820784147 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cf7c9287b0 0x55cf7c92aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:22.337 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.330+0000 7f8c4e59c640 1 -- 172.21.15.47:0/1820784147 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf7c597210 con 0x55cf7c91f9e0 2024-04-17T06:09:22.337 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.330+0000 7f8c47fff640 1 --2- 172.21.15.47:0/1820784147 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cf7c9287b0 0x55cf7c92aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T06:09:22.337 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.330+0000 7f8c4ed9d640 1 --2- 172.21.15.47:0/4125846957 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f8c3c0a3df0 0x7f8c3c0a62b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8c4802c6d0 tx=0x7f8c480a9040 comp rx=0 tx=0).ready entity=mgr.24923 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:22.337 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.330+0000 7f8c4e59c640 1 --2- 172.21.15.47:0/1820784147 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf7c91f9e0 0x55cf7c928270 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8c40009cd0 tx=0x7f8c400117b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3061e4e7946f5658 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:22.337 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.331+0000 7f8c44ff9640 1 -- 172.21.15.47:0/1820784147 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c40012450 con 0x55cf7c91f9e0 2024-04-17T06:09:22.337 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.331+0000 7f8c44ff9640 1 -- 172.21.15.47:0/1820784147 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c40012ad0 con 0x55cf7c91f9e0 2024-04-17T06:09:22.337 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.331+0000 7f8c44ff9640 1 -- 172.21.15.47:0/1820784147 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c4000fbc0 con 0x55cf7c91f9e0 2024-04-17T06:09:22.337 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.331+0000 7f8c57b7d740 1 -- 172.21.15.47:0/1820784147 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf7c91f9e0 msgr2=0x55cf7c928270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:22.337 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.331+0000 7f8c57b7d740 1 --2- 172.21.15.47:0/1820784147 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf7c91f9e0 0x55cf7c928270 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8c40009cd0 tx=0x7f8c400117b0 comp rx=0 tx=0).stop 2024-04-17T06:09:22.338 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.331+0000 7f8c57b7d740 1 -- 172.21.15.47:0/1820784147 shutdown_connections 2024-04-17T06:09:22.338 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.331+0000 7f8c57b7d740 1 --2- 172.21.15.47:0/1820784147 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cf7c9287b0 0x55cf7c92aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:22.338 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.331+0000 7f8c57b7d740 1 --2- 172.21.15.47:0/1820784147 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf7c91f9e0 0x55cf7c928270 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:22.338 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.331+0000 7f8c57b7d740 1 --2- 172.21.15.47:0/1820784147 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cf7c91f0c0 0x55cf7c91f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:22.338 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.331+0000 7f8c57b7d740 1 -- 172.21.15.47:0/1820784147 >> 172.21.15.47:0/1820784147 conn(0x55cf7c9199e0 msgr2=0x55cf7c919dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:22.338 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.331+0000 7f8c57b7d740 1 -- 172.21.15.47:0/1820784147 shutdown_connections 2024-04-17T06:09:22.338 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.332+0000 7f8c57b7d740 1 -- 172.21.15.47:0/1820784147 wait complete. 2024-04-17T06:09:22.338 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.332+0000 7f8c57b7d740 1 -- start start 2024-04-17T06:09:22.338 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.332+0000 7f8c57b7d740 1 Processor -- start 2024-04-17T06:09:22.338 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.332+0000 7f8c57b7d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf7c91f0c0 0x55cf7c9257c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:22.338 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.332+0000 7f8c57b7d740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cf7c91f9e0 0x55cf7c925d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:22.338 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.332+0000 7f8c57b7d740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cf7c9287b0 0x55cf7c926240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:22.338 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.332+0000 7f8c57b7d740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55cf7c92b3e0 con 0x55cf7c9287b0 2024-04-17T06:09:22.338 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.332+0000 7f8c57b7d740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55cf7c80ce90 con 0x55cf7c91f9e0 2024-04-17T06:09:22.338 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.332+0000 7f8c57b7d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55cf7c92b560 con 0x55cf7c91f0c0 2024-04-17T06:09:22.339 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.332+0000 7f8c57b7d740 10 client.0 _ll_register_callbacks cb 0x55cf7c822d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:09:22.339 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.332+0000 7f8c47fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cf7c9287b0 0x55cf7c926240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:22.339 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.332+0000 7f8c47fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cf7c9287b0 0x55cf7c926240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47904/0 (socket says 172.21.15.47:47904) 2024-04-17T06:09:22.339 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.332+0000 7f8c47fff640 1 -- 172.21.15.47:0/1042582967 learned_addr learned my addr 172.21.15.47:0/1042582967 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:09:22.339 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.333+0000 7f8c4e59c640 1 --2- 172.21.15.47:0/1042582967 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cf7c91f9e0 0x55cf7c925d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:22.339 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.333+0000 7f8c47fff640 1 --2- 172.21.15.47:0/1042582967 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf7c91f0c0 0x55cf7c9257c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:22.339 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.333+0000 7f8c47fff640 1 -- 172.21.15.47:0/1042582967 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf7c91f0c0 msgr2=0x55cf7c9257c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:22.339 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.333+0000 7f8c47fff640 1 --2- 172.21.15.47:0/1042582967 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf7c91f0c0 0x55cf7c9257c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:22.339 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.333+0000 7f8c47fff640 1 -- 172.21.15.47:0/1042582967 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cf7c91f9e0 msgr2=0x55cf7c925d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:22.339 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.333+0000 7f8c47fff640 1 --2- 172.21.15.47:0/1042582967 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cf7c91f9e0 0x55cf7c925d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:22.339 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.333+0000 7f8c47fff640 1 -- 172.21.15.47:0/1042582967 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf7c663620 con 0x55cf7c9287b0 2024-04-17T06:09:22.339 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.333+0000 7f8c47fff640 1 --2- 172.21.15.47:0/1042582967 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cf7c9287b0 0x55cf7c926240 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f8c38013f10 tx=0x7f8c3800bac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80fac87d1d183afa server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:22.339 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.333+0000 7f8c35ffb640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-17T06:09:22.339 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.333+0000 7f8c35ffb640 1 -- 172.21.15.47:0/1042582967 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c38004020 con 0x55cf7c9287b0 2024-04-17T06:09:22.339 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.333+0000 7f8c35ffb640 1 -- 172.21.15.47:0/1042582967 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8513+0+0 (secure 0 0 0) 0x7f8c380041c0 con 0x55cf7c9287b0 2024-04-17T06:09:22.339 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.333+0000 7f8c57b7d740 1 -- 172.21.15.47:0/1042582967 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c40012ad0 con 0x55cf7c9287b0 2024-04-17T06:09:22.340 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.333+0000 7f8c35ffb640 1 -- 172.21.15.47:0/1042582967 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8c380043c0 con 0x55cf7c9287b0 2024-04-17T06:09:22.340 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.333+0000 7f8c57b7d740 10 client.16074 fetch_fsmap learned FSMap version 32 2024-04-17T06:09:22.340 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.333+0000 7f8c57b7d740 1 -- 172.21.15.47:0/1042582967 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55cf7c937690 con 0x55cf7c9287b0 2024-04-17T06:09:22.340 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.334+0000 7f8c35ffb640 1 -- 172.21.15.47:0/1042582967 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c40012ad0 con 0x55cf7c9287b0 2024-04-17T06:09:22.340 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.334+0000 7f8c35ffb640 1 -- 172.21.15.47:0/1042582967 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) ==== 8339+0+0 (secure 0 0 0) 0x7f8c3800f040 con 0x55cf7c9287b0 2024-04-17T06:09:22.340 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.334+0000 7f8c57b7d740 10 client.16074 fetch_fsmap finished waiting for FSMap version 32 2024-04-17T06:09:22.340 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.334+0000 7f8c57b7d740 10 client.16074 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-17T06:09:22.340 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.334+0000 7f8c57b7d740 20 client.16074 populate_metadata read hostname 'smithi047' 2024-04-17T06:09:22.340 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.334+0000 7f8c57b7d740 1 --2- 172.21.15.47:0/1042582967 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55cf7c938b50 0x55cf7c93af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:22.340 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.334+0000 7f8c57b7d740 4 client.16074 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T06:09:22.340 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.334+0000 7f8c57b7d740 1 -- 172.21.15.47:0/1042582967 --> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cf7c8921f0 con 0x55cf7c938b50 2024-04-17T06:09:22.340 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.334+0000 7f8c4e59c640 1 --2- 172.21.15.47:0/1042582967 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55cf7c938b50 0x55cf7c93af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:22.340 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.334+0000 7f8c4e59c640 1 --2- 172.21.15.47:0/1042582967 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55cf7c938b50 0x55cf7c93af40 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=720023e286690427 server_cookie=18a9a6d7f16b1d63 in_seq=0 out_seq=0 2024-04-17T06:09:22.340 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.334+0000 7f8c35ffb640 10 client.16074 ms_handle_connect on v2:172.21.15.98:6838/2841287917 2024-04-17T06:09:22.340 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.335+0000 7f8c35ffb640 1 -- 172.21.15.47:0/1042582967 <== mds.0 v2:172.21.15.98:6838/2841287917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cf7c8921f0 con 0x55cf7c938b50 2024-04-17T06:09:22.340 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.335+0000 7f8c35ffb640 10 client.16074 handle_command_reply: tid=0 2024-04-17T06:09:22.341 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.335+0000 7f8c57b7d740 1 client.16074 shutdown 2024-04-17T06:09:22.341 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.335+0000 7f8c57b7d740 1 -- 172.21.15.47:0/1042582967 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cf7c9287b0 msgr2=0x55cf7c926240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:22.341 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.335+0000 7f8c57b7d740 1 --2- 172.21.15.47:0/1042582967 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cf7c9287b0 0x55cf7c926240 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f8c38013f10 tx=0x7f8c3800bac0 comp rx=0 tx=0).stop 2024-04-17T06:09:22.341 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.335+0000 7f8c57b7d740 1 -- 172.21.15.47:0/1042582967 shutdown_connections 2024-04-17T06:09:22.341 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.335+0000 7f8c57b7d740 1 --2- 172.21.15.47:0/1042582967 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55cf7c938b50 0x55cf7c93af40 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:22.341 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.335+0000 7f8c57b7d740 1 --2- 172.21.15.47:0/1042582967 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cf7c9287b0 0x55cf7c926240 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:22.341 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.335+0000 7f8c57b7d740 1 --2- 172.21.15.47:0/1042582967 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cf7c91f9e0 0x55cf7c925d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:22.341 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.335+0000 7f8c35ffb640 0 client.16074 ms_handle_reset on v2:172.21.15.98:6838/2841287917 2024-04-17T06:09:22.341 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.335+0000 7f8c57b7d740 1 --2- 172.21.15.47:0/1042582967 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf7c91f0c0 0x55cf7c9257c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:22.341 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.335+0000 7f8c57b7d740 1 -- 172.21.15.47:0/1042582967 >> 172.21.15.47:0/1042582967 conn(0x55cf7c9199e0 msgr2=0x55cf7c91b730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:22.341 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.335+0000 7f8c57b7d740 1 -- 172.21.15.47:0/1042582967 shutdown_connections 2024-04-17T06:09:22.341 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.335+0000 7f8c57b7d740 1 -- 172.21.15.47:0/1042582967 wait complete. 2024-04-17T06:09:22.341 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.336+0000 7f8c57b7d740 20 client.16074 trim_cache size 0 max 16384 2024-04-17T06:09:22.353 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.350+0000 7f8c57b7d740 1 Processor -- start 2024-04-17T06:09:22.353 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.350+0000 7f8c57b7d740 1 -- start start 2024-04-17T06:09:22.354 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.351+0000 7f8c57b7d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf7c91f0c0 0x55cf7c927ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:22.354 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.351+0000 7f8c57b7d740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cf7c91f9e0 0x55cf7c9342d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:22.354 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.351+0000 7f8c57b7d740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cf7c9287b0 0x55cf7c940830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:22.354 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.351+0000 7f8c57b7d740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f8c501b2710 con 0x55cf7c91f9e0 2024-04-17T06:09:22.354 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.351+0000 7f8c57b7d740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8c501affd0 con 0x55cf7c9287b0 2024-04-17T06:09:22.354 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.351+0000 7f8c57b7d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8c501ade40 con 0x55cf7c91f0c0 2024-04-17T06:09:22.354 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.351+0000 7f8c4e59c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cf7c91f9e0 0x55cf7c9342d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:22.354 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.351+0000 7f8c4e59c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cf7c91f9e0 0x55cf7c9342d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47912/0 (socket says 172.21.15.47:47912) 2024-04-17T06:09:22.354 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.351+0000 7f8c4e59c640 1 -- 172.21.15.47:0/250427058 learned_addr learned my addr 172.21.15.47:0/250427058 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:09:22.354 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.351+0000 7f8c47fff640 1 --2- 172.21.15.47:0/250427058 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf7c91f0c0 0x55cf7c927ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:22.354 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.351+0000 7f8c4e59c640 1 -- 172.21.15.47:0/250427058 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf7c91f0c0 msgr2=0x55cf7c927ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:22.354 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.351+0000 7f8c4e59c640 1 --2- 172.21.15.47:0/250427058 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf7c91f0c0 0x55cf7c927ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:22.355 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.351+0000 7f8c4e59c640 1 -- 172.21.15.47:0/250427058 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cf7c9287b0 msgr2=0x55cf7c940830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:22.355 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.351+0000 7f8c4e59c640 1 --2- 172.21.15.47:0/250427058 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cf7c9287b0 0x55cf7c940830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:22.356 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.351+0000 7f8c4e59c640 1 -- 172.21.15.47:0/250427058 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf7c91ff40 con 0x55cf7c91f9e0 2024-04-17T06:09:22.356 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.351+0000 7f8c47fff640 1 --2- 172.21.15.47:0/250427058 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf7c91f0c0 0x55cf7c927ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:09:22.356 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.351+0000 7f8c4e59c640 1 --2- 172.21.15.47:0/250427058 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cf7c91f9e0 0x55cf7c9342d0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f8c40009cd0 tx=0x7f8c40005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4a3bf7582513ef2 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:22.356 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.351+0000 7f8c37fff640 1 -- 172.21.15.47:0/250427058 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c40009530 con 0x55cf7c91f9e0 2024-04-17T06:09:22.356 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.351+0000 7f8c37fff640 1 -- 172.21.15.47:0/250427058 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c4002f070 con 0x55cf7c91f9e0 2024-04-17T06:09:22.356 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.351+0000 7f8c37fff640 1 -- 172.21.15.47:0/250427058 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c400035f0 con 0x55cf7c91f9e0 2024-04-17T06:09:22.356 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.352+0000 7f8c57b7d740 1 -- 172.21.15.47:0/250427058 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cf7c91f9e0 msgr2=0x55cf7c9342d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:22.356 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.352+0000 7f8c57b7d740 1 --2- 172.21.15.47:0/250427058 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cf7c91f9e0 0x55cf7c9342d0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f8c40009cd0 tx=0x7f8c40005f90 comp rx=0 tx=0).stop 2024-04-17T06:09:22.356 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.352+0000 7f8c57b7d740 1 -- 172.21.15.47:0/250427058 shutdown_connections 2024-04-17T06:09:22.356 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.352+0000 7f8c57b7d740 1 --2- 172.21.15.47:0/250427058 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cf7c9287b0 0x55cf7c940830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:22.356 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.352+0000 7f8c57b7d740 1 --2- 172.21.15.47:0/250427058 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cf7c91f9e0 0x55cf7c9342d0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:22.356 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.352+0000 7f8c57b7d740 1 --2- 172.21.15.47:0/250427058 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf7c91f0c0 0x55cf7c927ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:22.356 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.352+0000 7f8c57b7d740 1 -- 172.21.15.47:0/250427058 >> 172.21.15.47:0/250427058 conn(0x55cf7c9199e0 msgr2=0x55cf7c919dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:22.357 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.352+0000 7f8c57b7d740 1 -- 172.21.15.47:0/250427058 shutdown_connections 2024-04-17T06:09:22.357 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.352+0000 7f8c57b7d740 1 -- 172.21.15.47:0/250427058 wait complete. 2024-04-17T06:09:22.357 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.352+0000 7f8c57b7d740 1 -- start start 2024-04-17T06:09:22.357 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.352+0000 7f8c57b7d740 1 Processor -- start 2024-04-17T06:09:22.357 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.353+0000 7f8c57b7d740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cf7c91f0c0 0x55cf7c922790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:22.357 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.353+0000 7f8c57b7d740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cf7c91f9e0 0x55cf7c922cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:22.357 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.353+0000 7f8c57b7d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf7c9287b0 0x55cf7c99e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:22.357 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.353+0000 7f8c57b7d740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f8c501ae490 con 0x55cf7c91f9e0 2024-04-17T06:09:22.357 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.353+0000 7f8c57b7d740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8c501af900 con 0x55cf7c91f0c0 2024-04-17T06:09:22.357 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.353+0000 7f8c57b7d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8c501aeb60 con 0x55cf7c9287b0 2024-04-17T06:09:22.357 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.353+0000 7f8c57b7d740 10 client.0 _ll_register_callbacks cb 0x7f8c38013b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:09:22.357 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.353+0000 7f8c4e59c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cf7c91f9e0 0x55cf7c922cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:22.357 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.353+0000 7f8c4e59c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cf7c91f9e0 0x55cf7c922cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47916/0 (socket says 172.21.15.47:47916) 2024-04-17T06:09:22.357 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.353+0000 7f8c4e59c640 1 -- 172.21.15.47:0/146925821 learned_addr learned my addr 172.21.15.47:0/146925821 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:09:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.353+0000 7f8c47fff640 1 --2- 172.21.15.47:0/146925821 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cf7c91f0c0 0x55cf7c922790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.353+0000 7f8c47fff640 1 --2- 172.21.15.47:0/146925821 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf7c9287b0 0x55cf7c99e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.353+0000 7f8c4e59c640 1 -- 172.21.15.47:0/146925821 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf7c9287b0 msgr2=0x55cf7c99e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.353+0000 7f8c4e59c640 1 --2- 172.21.15.47:0/146925821 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf7c9287b0 0x55cf7c99e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.353+0000 7f8c4e59c640 1 -- 172.21.15.47:0/146925821 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cf7c91f0c0 msgr2=0x55cf7c922790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.353+0000 7f8c4e59c640 1 --2- 172.21.15.47:0/146925821 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cf7c91f0c0 0x55cf7c922790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.353+0000 7f8c4e59c640 1 -- 172.21.15.47:0/146925821 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf7c919810 con 0x55cf7c91f9e0 2024-04-17T06:09:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.353+0000 7f8c47fff640 1 --2- 172.21.15.47:0/146925821 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf7c9287b0 0x55cf7c99e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:09:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.353+0000 7f8c4e59c640 1 --2- 172.21.15.47:0/146925821 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cf7c91f9e0 0x55cf7c922cd0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f8c40029590 tx=0x7f8c40009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f618c393cf5c007 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.353+0000 7f8c35ffb640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-17T06:09:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.353+0000 7f8c35ffb640 1 -- 172.21.15.47:0/146925821 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c40038060 con 0x55cf7c91f9e0 2024-04-17T06:09:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.353+0000 7f8c35ffb640 1 -- 172.21.15.47:0/146925821 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8513+0+0 (secure 0 0 0) 0x7f8c400050b0 con 0x55cf7c91f9e0 2024-04-17T06:09:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.353+0000 7f8c57b7d740 1 -- 172.21.15.47:0/146925821 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c4002f070 con 0x55cf7c91f9e0 2024-04-17T06:09:22.359 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.354+0000 7f8c35ffb640 1 -- 172.21.15.47:0/146925821 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8c400298f0 con 0x55cf7c91f9e0 2024-04-17T06:09:22.359 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.354+0000 7f8c57b7d740 10 client.16080 fetch_fsmap learned FSMap version 32 2024-04-17T06:09:22.359 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.354+0000 7f8c57b7d740 1 -- 172.21.15.47:0/146925821 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55cf7c99f4e0 con 0x55cf7c91f9e0 2024-04-17T06:09:22.359 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.354+0000 7f8c35ffb640 1 -- 172.21.15.47:0/146925821 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c4002f070 con 0x55cf7c91f9e0 2024-04-17T06:09:22.359 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.354+0000 7f8c35ffb640 1 -- 172.21.15.47:0/146925821 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) ==== 8339+0+0 (secure 0 0 0) 0x7f8c4000f5c0 con 0x55cf7c91f9e0 2024-04-17T06:09:22.359 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.354+0000 7f8c57b7d740 10 client.16080 fetch_fsmap finished waiting for FSMap version 32 2024-04-17T06:09:22.359 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.354+0000 7f8c57b7d740 10 client.16080 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-17T06:09:22.359 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.354+0000 7f8c57b7d740 20 client.16080 populate_metadata read hostname 'smithi047' 2024-04-17T06:09:22.359 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.354+0000 7f8c57b7d740 1 --2- 172.21.15.47:0/146925821 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55cf7c99fa30 0x55cf7c9a1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:22.359 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.354+0000 7f8c57b7d740 4 client.16080 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-17T06:09:22.359 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.354+0000 7f8c57b7d740 1 -- 172.21.15.47:0/146925821 --> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cf7c88c060 con 0x55cf7c99fa30 2024-04-17T06:09:22.359 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.354+0000 7f8c47fff640 1 --2- 172.21.15.47:0/146925821 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55cf7c99fa30 0x55cf7c9a1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:22.359 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.355+0000 7f8c47fff640 1 --2- 172.21.15.47:0/146925821 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55cf7c99fa30 0x55cf7c9a1e50 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f89af5fc28af21a2 server_cookie=eb4e0479f0db7cc4 in_seq=0 out_seq=0 2024-04-17T06:09:22.359 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.355+0000 7f8c35ffb640 10 client.16080 ms_handle_connect on v2:172.21.15.98:6838/2841287917 2024-04-17T06:09:22.362 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.363+0000 7f8c35ffb640 1 -- 172.21.15.47:0/146925821 <== mds.0 v2:172.21.15.98:6838/2841287917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55cf7c88c060 con 0x55cf7c99fa30 2024-04-17T06:09:22.362 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.363+0000 7f8c35ffb640 10 client.16080 handle_command_reply: tid=0 2024-04-17T06:09:22.362 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.363+0000 7f8c57b7d740 1 client.16080 shutdown 2024-04-17T06:09:22.362 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.363+0000 7f8c57b7d740 1 -- 172.21.15.47:0/146925821 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cf7c91f9e0 msgr2=0x55cf7c922cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:22.362 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.363+0000 7f8c57b7d740 1 --2- 172.21.15.47:0/146925821 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cf7c91f9e0 0x55cf7c922cd0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f8c40029590 tx=0x7f8c40009e30 comp rx=0 tx=0).stop 2024-04-17T06:09:22.362 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.363+0000 7f8c57b7d740 1 -- 172.21.15.47:0/146925821 shutdown_connections 2024-04-17T06:09:22.362 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.363+0000 7f8c57b7d740 1 --2- 172.21.15.47:0/146925821 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55cf7c99fa30 0x55cf7c9a1e50 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:22.363 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.363+0000 7f8c57b7d740 1 --2- 172.21.15.47:0/146925821 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf7c9287b0 0x55cf7c99e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:22.363 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.363+0000 7f8c57b7d740 1 --2- 172.21.15.47:0/146925821 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cf7c91f9e0 0x55cf7c922cd0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:22.363 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.363+0000 7f8c35ffb640 0 client.16080 ms_handle_reset on v2:172.21.15.98:6838/2841287917 2024-04-17T06:09:22.363 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.363+0000 7f8c57b7d740 1 --2- 172.21.15.47:0/146925821 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cf7c91f0c0 0x55cf7c922790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:22.363 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.363+0000 7f8c57b7d740 1 -- 172.21.15.47:0/146925821 >> 172.21.15.47:0/146925821 conn(0x55cf7c9199e0 msgr2=0x55cf7c925780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:22.363 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.363+0000 7f8c57b7d740 1 -- 172.21.15.47:0/146925821 shutdown_connections 2024-04-17T06:09:22.363 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.363+0000 7f8c57b7d740 1 -- 172.21.15.47:0/146925821 wait complete. 2024-04-17T06:09:22.363 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.363+0000 7f8c57b7d740 20 client.16080 trim_cache size 0 max 16384 2024-04-17T06:09:22.363 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-04-17T06:09:22.363 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-04-17T06:09:22.363 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-04-17T06:09:22.363 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-04-17T06:09:22.364 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.364+0000 7f8c54eba640 1 -- 172.21.15.47:0/4125846957 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f8c3c0a3df0 msgr2=0x7f8c3c0a62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:22.364 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.364+0000 7f8c54eba640 1 --2- 172.21.15.47:0/4125846957 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f8c3c0a3df0 0x7f8c3c0a62b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8c4802c6d0 tx=0x7f8c480a9040 comp rx=0 tx=0).stop 2024-04-17T06:09:22.364 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.364+0000 7f8c54eba640 1 -- 172.21.15.47:0/4125846957 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8c50106f40 msgr2=0x7f8c501020c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:22.364 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.364+0000 7f8c54eba640 1 --2- 172.21.15.47:0/4125846957 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8c50106f40 0x7f8c501020c0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f8c48004af0 tx=0x7f8c48032ec0 comp rx=0 tx=0).stop 2024-04-17T06:09:22.365 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.364+0000 7f8c54eba640 1 -- 172.21.15.47:0/4125846957 shutdown_connections 2024-04-17T06:09:22.365 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.364+0000 7f8c54eba640 1 --2- 172.21.15.47:0/4125846957 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f8c3c0a3df0 0x7f8c3c0a62b0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:22.365 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.364+0000 7f8c54eba640 1 --2- 172.21.15.47:0/4125846957 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8c50106f40 0x7f8c501020c0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:22.365 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.364+0000 7f8c54eba640 1 --2- 172.21.15.47:0/4125846957 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8c50100e20 0x7f8c50101b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:22.365 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.364+0000 7f8c54eba640 1 --2- 172.21.15.47:0/4125846957 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8c50100460 0x7f8c50101640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:22.365 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.364+0000 7f8c54eba640 1 -- 172.21.15.47:0/4125846957 >> 172.21.15.47:0/4125846957 conn(0x7f8c500773f0 msgr2=0x7f8c50104f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:22.365 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.364+0000 7f8c54eba640 1 -- 172.21.15.47:0/4125846957 shutdown_connections 2024-04-17T06:09:22.365 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:22.364+0000 7f8c54eba640 1 -- 172.21.15.47:0/4125846957 wait complete. 2024-04-17T06:09:24.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:09:23 smithi103 ceph-mon[36243]: pgmap v252: 97 pgs: 97 active+clean; 4.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.1 KiB/s wr, 11 op/s 2024-04-17T06:09:24.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:09:23 smithi098 ceph-mon[36516]: pgmap v252: 97 pgs: 97 active+clean; 4.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.1 KiB/s wr, 11 op/s 2024-04-17T06:09:24.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:09:23 smithi047 ceph-mon[32223]: pgmap v252: 97 pgs: 97 active+clean; 4.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.1 KiB/s wr, 11 op/s 2024-04-17T06:09:26.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:09:25 smithi103 ceph-mon[36243]: pgmap v253: 97 pgs: 97 active+clean; 4.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 15 KiB/s wr, 13 op/s 2024-04-17T06:09:26.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:09:25 smithi098 ceph-mon[36516]: pgmap v253: 97 pgs: 97 active+clean; 4.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 15 KiB/s wr, 13 op/s 2024-04-17T06:09:26.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:09:25 smithi047 ceph-mon[32223]: pgmap v253: 97 pgs: 97 active+clean; 4.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 15 KiB/s wr, 13 op/s 2024-04-17T06:09:28.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:09:27 smithi103 ceph-mon[36243]: pgmap v254: 97 pgs: 97 active+clean; 4.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-04-17T06:09:28.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:09:27 smithi098 ceph-mon[36516]: pgmap v254: 97 pgs: 97 active+clean; 4.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-04-17T06:09:28.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:09:27 smithi047 ceph-mon[32223]: pgmap v254: 97 pgs: 97 active+clean; 4.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-04-17T06:09:30.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:09:29 smithi103 ceph-mon[36243]: pgmap v255: 97 pgs: 97 active+clean; 4.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:09:30.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:09:29 smithi098 ceph-mon[36516]: pgmap v255: 97 pgs: 97 active+clean; 4.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:09:30.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:09:29 smithi047 ceph-mon[32223]: pgmap v255: 97 pgs: 97 active+clean; 4.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:09:31.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:09:30 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T06:09:31.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:09:30 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T06:09:31.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:09:30 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T06:09:32.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:09:31 smithi103 ceph-mon[36243]: pgmap v256: 97 pgs: 97 active+clean; 4.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-04-17T06:09:32.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:09:31 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T06:09:32.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:09:31 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T06:09:32.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:09:31 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:09:32.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:09:31 smithi098 ceph-mon[36516]: pgmap v256: 97 pgs: 97 active+clean; 4.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-04-17T06:09:32.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:09:31 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T06:09:32.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:09:31 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T06:09:32.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:09:31 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:09:32.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:09:31 smithi047 ceph-mon[32223]: pgmap v256: 97 pgs: 97 active+clean; 4.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-04-17T06:09:32.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:09:31 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T06:09:32.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:09:31 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T06:09:32.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:09:31 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:09:34.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:09:33 smithi103 ceph-mon[36243]: pgmap v257: 97 pgs: 97 active+clean; 4.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:09:34.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:09:33 smithi098 ceph-mon[36516]: pgmap v257: 97 pgs: 97 active+clean; 4.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:09:34.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:09:33 smithi047 ceph-mon[32223]: pgmap v257: 97 pgs: 97 active+clean; 4.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:09:35.465 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.464+0000 7f135b450640 1 -- 172.21.15.47:0/437765761 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13541061c0 msgr2=0x7f135410dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:35.465 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.464+0000 7f135b450640 1 --2- 172.21.15.47:0/437765761 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13541061c0 0x7f135410dd90 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f134c009ca0 tx=0x7f134c02c0f0 comp rx=0 tx=0).stop 2024-04-17T06:09:35.465 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.464+0000 7f135b450640 1 -- 172.21.15.47:0/437765761 shutdown_connections 2024-04-17T06:09:35.465 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.464+0000 7f135b450640 1 --2- 172.21.15.47:0/437765761 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13541061c0 0x7f135410dd90 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:35.465 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.464+0000 7f135b450640 1 --2- 172.21.15.47:0/437765761 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1354101aa0 0x7f1354105ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:35.465 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.464+0000 7f135b450640 1 --2- 172.21.15.47:0/437765761 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f13541010f0 0x7f13541014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:35.465 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.464+0000 7f135b450640 1 -- 172.21.15.47:0/437765761 >> 172.21.15.47:0/437765761 conn(0x7f13540771e0 msgr2=0x7f13540775f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:35.465 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.465+0000 7f135b450640 1 -- 172.21.15.47:0/437765761 shutdown_connections 2024-04-17T06:09:35.466 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.465+0000 7f135b450640 1 -- 172.21.15.47:0/437765761 wait complete. 2024-04-17T06:09:35.466 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.465+0000 7f135b450640 1 Processor -- start 2024-04-17T06:09:35.466 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.465+0000 7f135b450640 1 -- start start 2024-04-17T06:09:35.466 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.465+0000 7f135b450640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f13541010f0 0x7f1354198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:35.466 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.465+0000 7f135b450640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13541061c0 0x7f1354198cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:35.466 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.465+0000 7f135b450640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f135419cfe0 0x7f135419d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:35.466 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.465+0000 7f135b450640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f1354111280 con 0x7f13541061c0 2024-04-17T06:09:35.466 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.465+0000 7f135b450640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1354111100 con 0x7f135419cfe0 2024-04-17T06:09:35.466 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.465+0000 7f135b450640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1354111400 con 0x7f13541010f0 2024-04-17T06:09:35.466 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.466+0000 7f13589eb640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13541061c0 0x7f1354198cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:35.466 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.466+0000 7f13589eb640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13541061c0 0x7f1354198cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45648/0 (socket says 172.21.15.47:45648) 2024-04-17T06:09:35.466 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.466+0000 7f13589eb640 1 -- 172.21.15.47:0/232641857 learned_addr learned my addr 172.21.15.47:0/232641857 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:09:35.467 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.466+0000 7f13591ec640 1 --2- 172.21.15.47:0/232641857 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f13541010f0 0x7f1354198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:35.467 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.466+0000 7f13599ed640 1 --2- 172.21.15.47:0/232641857 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f135419cfe0 0x7f135419d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:35.467 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.466+0000 7f13589eb640 1 -- 172.21.15.47:0/232641857 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f13541010f0 msgr2=0x7f1354198770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:35.467 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.466+0000 7f13589eb640 1 --2- 172.21.15.47:0/232641857 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f13541010f0 0x7f1354198770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:35.467 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.466+0000 7f13589eb640 1 -- 172.21.15.47:0/232641857 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f135419cfe0 msgr2=0x7f135419d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:35.467 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.466+0000 7f13589eb640 1 --2- 172.21.15.47:0/232641857 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f135419cfe0 0x7f135419d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:35.468 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.466+0000 7f13589eb640 1 -- 172.21.15.47:0/232641857 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f135419dad0 con 0x7f13541061c0 2024-04-17T06:09:35.468 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.466+0000 7f13589eb640 1 --2- 172.21.15.47:0/232641857 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13541061c0 0x7f1354198cb0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f1344005fd0 tx=0x7f134400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:35.468 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.466+0000 7f134a7fc640 1 -- 172.21.15.47:0/232641857 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1344017070 con 0x7f13541061c0 2024-04-17T06:09:35.468 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.466+0000 7f134a7fc640 1 -- 172.21.15.47:0/232641857 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1344009c30 con 0x7f13541061c0 2024-04-17T06:09:35.468 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.466+0000 7f135b450640 1 -- 172.21.15.47:0/232641857 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f135419ddc0 con 0x7f13541061c0 2024-04-17T06:09:35.468 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.466+0000 7f134a7fc640 1 -- 172.21.15.47:0/232641857 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1344010460 con 0x7f13541061c0 2024-04-17T06:09:35.468 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.466+0000 7f135b450640 1 -- 172.21.15.47:0/232641857 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13541a55b0 con 0x7f13541061c0 2024-04-17T06:09:35.476 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.469+0000 7f134a7fc640 1 -- 172.21.15.47:0/232641857 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1344003e70 con 0x7f13541061c0 2024-04-17T06:09:35.476 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.469+0000 7f135e030740 1 Processor -- start 2024-04-17T06:09:35.476 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.469+0000 7f135e030740 1 -- start start 2024-04-17T06:09:35.476 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.469+0000 7f135e030740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c42899b0c0 0x55c42899b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:35.476 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.469+0000 7f135e030740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c42899b9e0 0x55c4289a4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:35.477 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.469+0000 7f135e030740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c4289a47b0 0x55c4289a6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:35.477 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.469+0000 7f135e030740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55c4289007d0 con 0x55c42899b0c0 2024-04-17T06:09:35.477 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.469+0000 7f135e030740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55c42889df70 con 0x55c4289a47b0 2024-04-17T06:09:35.477 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.469+0000 7f135e030740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c4288fc790 con 0x55c42899b9e0 2024-04-17T06:09:35.477 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.470+0000 7f134a7fc640 1 --2- 172.21.15.47:0/232641857 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f13400a3dc0 0x7f13400a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:35.477 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.470+0000 7f13599ed640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c42899b0c0 0x55c42899b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:35.477 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.470+0000 7f13599ed640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c42899b0c0 0x55c42899b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45662/0 (socket says 172.21.15.47:45662) 2024-04-17T06:09:35.477 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.470+0000 7f13599ed640 1 -- 172.21.15.47:0/3177327072 learned_addr learned my addr 172.21.15.47:0/3177327072 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:09:35.477 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.470+0000 7f13591ec640 1 --2- 172.21.15.47:0/3177327072 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c42899b9e0 0x55c4289a4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:35.477 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.470+0000 7f134a7fc640 1 -- 172.21.15.47:0/232641857 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8513+0+0 (secure 0 0 0) 0x7f13440d0340 con 0x7f13541061c0 2024-04-17T06:09:35.477 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.470+0000 7f13589eb640 1 --2- 172.21.15.47:0/3177327072 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c4289a47b0 0x55c4289a6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:35.477 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.470+0000 7f13599ed640 1 --2- 172.21.15.47:0/232641857 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f13400a3dc0 0x7f13400a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:35.477 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.470+0000 7f13591ec640 1 -- 172.21.15.47:0/3177327072 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c4289a47b0 msgr2=0x55c4289a6ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:35.477 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.470+0000 7f13591ec640 1 --2- 172.21.15.47:0/3177327072 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c4289a47b0 0x55c4289a6ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:35.478 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.470+0000 7f13591ec640 1 -- 172.21.15.47:0/3177327072 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c42899b0c0 msgr2=0x55c42899b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:35.478 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.470+0000 7f13591ec640 1 --2- 172.21.15.47:0/3177327072 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c42899b0c0 0x55c42899b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:35.478 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.470+0000 7f13591ec640 1 -- 172.21.15.47:0/3177327072 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c428613210 con 0x55c42899b9e0 2024-04-17T06:09:35.478 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.470+0000 7f13599ed640 1 --2- 172.21.15.47:0/3177327072 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c42899b0c0 0x55c42899b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T06:09:35.478 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.470+0000 7f13591ec640 1 --2- 172.21.15.47:0/3177327072 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c42899b9e0 0x55c4289a4270 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1350004630 tx=0x7f135000bb20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0b1201736a2d4ac server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:35.478 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.470+0000 7f13599ed640 1 --2- 172.21.15.47:0/232641857 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f13400a3dc0 0x7f13400a6280 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f134c009430 tx=0x7f134c02c0f0 comp rx=0 tx=0).ready entity=mgr.24923 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:35.478 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.470+0000 7f133f7fe640 1 -- 172.21.15.47:0/3177327072 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1350003bd0 con 0x55c42899b9e0 2024-04-17T06:09:35.478 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.471+0000 7f133f7fe640 1 -- 172.21.15.47:0/3177327072 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1350015070 con 0x55c42899b9e0 2024-04-17T06:09:35.478 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.471+0000 7f133f7fe640 1 -- 172.21.15.47:0/3177327072 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1350004d90 con 0x55c42899b9e0 2024-04-17T06:09:35.478 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.471+0000 7f135e030740 1 -- 172.21.15.47:0/3177327072 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c42899b9e0 msgr2=0x55c4289a4270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:35.478 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.471+0000 7f135e030740 1 --2- 172.21.15.47:0/3177327072 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c42899b9e0 0x55c4289a4270 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1350004630 tx=0x7f135000bb20 comp rx=0 tx=0).stop 2024-04-17T06:09:35.478 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.471+0000 7f135e030740 1 -- 172.21.15.47:0/3177327072 shutdown_connections 2024-04-17T06:09:35.478 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.471+0000 7f135e030740 1 --2- 172.21.15.47:0/3177327072 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c4289a47b0 0x55c4289a6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:35.478 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.471+0000 7f135e030740 1 --2- 172.21.15.47:0/3177327072 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c42899b9e0 0x55c4289a4270 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:35.479 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.471+0000 7f135e030740 1 --2- 172.21.15.47:0/3177327072 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c42899b0c0 0x55c42899b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:35.479 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.471+0000 7f135e030740 1 -- 172.21.15.47:0/3177327072 >> 172.21.15.47:0/3177327072 conn(0x55c4289959e0 msgr2=0x55c428995dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:35.479 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.471+0000 7f135e030740 1 -- 172.21.15.47:0/3177327072 shutdown_connections 2024-04-17T06:09:35.479 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.471+0000 7f135e030740 1 -- 172.21.15.47:0/3177327072 wait complete. 2024-04-17T06:09:35.479 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.472+0000 7f135e030740 1 -- start start 2024-04-17T06:09:35.479 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.472+0000 7f135e030740 1 Processor -- start 2024-04-17T06:09:35.479 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.472+0000 7f135e030740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c42899b0c0 0x55c4289a18d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:35.479 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.472+0000 7f135e030740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c42899b9e0 0x55c4289a1e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:35.479 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.472+0000 7f135e030740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4289a47b0 0x55c4289a2350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:35.479 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.472+0000 7f135e030740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55c4289a73e0 con 0x55c4289a47b0 2024-04-17T06:09:35.479 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.472+0000 7f135e030740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55c428888e90 con 0x55c42899b0c0 2024-04-17T06:09:35.479 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.472+0000 7f135e030740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c4289a7560 con 0x55c42899b9e0 2024-04-17T06:09:35.479 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.472+0000 7f135e030740 10 client.0 _ll_register_callbacks cb 0x55c42889ed40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:09:35.479 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.472+0000 7f13599ed640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4289a47b0 0x55c4289a2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:35.480 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.472+0000 7f13599ed640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4289a47b0 0x55c4289a2350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45674/0 (socket says 172.21.15.47:45674) 2024-04-17T06:09:35.480 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.472+0000 7f13599ed640 1 -- 172.21.15.47:0/1387066125 learned_addr learned my addr 172.21.15.47:0/1387066125 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:09:35.480 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.472+0000 7f13591ec640 1 --2- 172.21.15.47:0/1387066125 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c42899b0c0 0x55c4289a18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:35.480 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.472+0000 7f13589eb640 1 --2- 172.21.15.47:0/1387066125 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c42899b9e0 0x55c4289a1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:35.480 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.473+0000 7f13599ed640 1 -- 172.21.15.47:0/1387066125 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c42899b9e0 msgr2=0x55c4289a1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:35.480 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.473+0000 7f13599ed640 1 --2- 172.21.15.47:0/1387066125 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c42899b9e0 0x55c4289a1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:35.480 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.473+0000 7f13599ed640 1 -- 172.21.15.47:0/1387066125 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c42899b0c0 msgr2=0x55c4289a18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:35.480 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.473+0000 7f13599ed640 1 --2- 172.21.15.47:0/1387066125 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c42899b0c0 0x55c4289a18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:35.480 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.473+0000 7f13599ed640 1 -- 172.21.15.47:0/1387066125 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4286df620 con 0x55c4289a47b0 2024-04-17T06:09:35.480 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.473+0000 7f13591ec640 1 --2- 172.21.15.47:0/1387066125 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c42899b0c0 0x55c4289a18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T06:09:35.480 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.473+0000 7f13599ed640 1 --2- 172.21.15.47:0/1387066125 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4289a47b0 0x55c4289a2350 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f134c02c630 tx=0x7f134c0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9fce92bae6fb6b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:35.480 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.473+0000 7f133cff9640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-17T06:09:35.481 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.473+0000 7f133cff9640 1 -- 172.21.15.47:0/1387066125 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f134c052060 con 0x55c4289a47b0 2024-04-17T06:09:35.481 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.473+0000 7f133cff9640 1 -- 172.21.15.47:0/1387066125 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8513+0+0 (secure 0 0 0) 0x7f134c004040 con 0x55c4289a47b0 2024-04-17T06:09:35.481 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.473+0000 7f135e030740 1 -- 172.21.15.47:0/1387066125 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1350015070 con 0x55c4289a47b0 2024-04-17T06:09:35.481 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.473+0000 7f133cff9640 1 -- 172.21.15.47:0/1387066125 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f134c039040 con 0x55c4289a47b0 2024-04-17T06:09:35.481 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.473+0000 7f135e030740 10 client.16092 fetch_fsmap learned FSMap version 32 2024-04-17T06:09:35.481 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.473+0000 7f135e030740 1 -- 172.21.15.47:0/1387066125 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c4289b36f0 con 0x55c4289a47b0 2024-04-17T06:09:35.481 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.474+0000 7f133cff9640 1 -- 172.21.15.47:0/1387066125 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1350015070 con 0x55c4289a47b0 2024-04-17T06:09:35.481 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.474+0000 7f133cff9640 1 -- 172.21.15.47:0/1387066125 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) ==== 8339+0+0 (secure 0 0 0) 0x7f134c03f070 con 0x55c4289a47b0 2024-04-17T06:09:35.481 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.474+0000 7f135e030740 10 client.16092 fetch_fsmap finished waiting for FSMap version 32 2024-04-17T06:09:35.481 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.474+0000 7f135e030740 10 client.16092 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-17T06:09:35.481 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.474+0000 7f135e030740 20 client.16092 populate_metadata read hostname 'smithi047' 2024-04-17T06:09:35.481 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.474+0000 7f135e030740 1 --2- 172.21.15.47:0/1387066125 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55c4289b4bb0 0x55c4289b6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:35.481 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.474+0000 7f135e030740 4 client.16092 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T06:09:35.482 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.474+0000 7f135e030740 1 -- 172.21.15.47:0/1387066125 --> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c42890e1f0 con 0x55c4289b4bb0 2024-04-17T06:09:35.482 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.474+0000 7f13591ec640 1 --2- 172.21.15.47:0/1387066125 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55c4289b4bb0 0x55c4289b6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:35.482 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.475+0000 7f13591ec640 1 --2- 172.21.15.47:0/1387066125 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55c4289b4bb0 0x55c4289b6fa0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9c30e018290a343 server_cookie=191824bd9bc83451 in_seq=0 out_seq=0 2024-04-17T06:09:35.482 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.475+0000 7f133cff9640 10 client.16092 ms_handle_connect on v2:172.21.15.98:6838/2841287917 2024-04-17T06:09:35.482 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.475+0000 7f133cff9640 1 -- 172.21.15.47:0/1387066125 <== mds.0 v2:172.21.15.98:6838/2841287917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c42890e1f0 con 0x55c4289b4bb0 2024-04-17T06:09:35.482 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.475+0000 7f133cff9640 10 client.16092 handle_command_reply: tid=0 2024-04-17T06:09:35.482 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.475+0000 7f135e030740 1 client.16092 shutdown 2024-04-17T06:09:35.482 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.476+0000 7f135e030740 1 -- 172.21.15.47:0/1387066125 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4289a47b0 msgr2=0x55c4289a2350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:35.482 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.476+0000 7f135e030740 1 --2- 172.21.15.47:0/1387066125 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4289a47b0 0x55c4289a2350 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f134c02c630 tx=0x7f134c0095c0 comp rx=0 tx=0).stop 2024-04-17T06:09:35.482 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.476+0000 7f135e030740 1 -- 172.21.15.47:0/1387066125 shutdown_connections 2024-04-17T06:09:35.482 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.476+0000 7f135e030740 1 --2- 172.21.15.47:0/1387066125 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55c4289b4bb0 0x55c4289b6fa0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:35.482 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.476+0000 7f135e030740 1 --2- 172.21.15.47:0/1387066125 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4289a47b0 0x55c4289a2350 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:35.482 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.476+0000 7f135e030740 1 --2- 172.21.15.47:0/1387066125 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c42899b9e0 0x55c4289a1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:35.482 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.476+0000 7f135e030740 1 --2- 172.21.15.47:0/1387066125 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c42899b0c0 0x55c4289a18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:35.483 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.476+0000 7f135e030740 1 -- 172.21.15.47:0/1387066125 >> 172.21.15.47:0/1387066125 conn(0x55c4289959e0 msgr2=0x55c428997730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:35.483 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.476+0000 7f133cff9640 0 client.16092 ms_handle_reset on v2:172.21.15.98:6838/2841287917 2024-04-17T06:09:35.483 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.476+0000 7f135e030740 1 -- 172.21.15.47:0/1387066125 shutdown_connections 2024-04-17T06:09:35.483 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.476+0000 7f135e030740 1 -- 172.21.15.47:0/1387066125 wait complete. 2024-04-17T06:09:35.483 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.476+0000 7f135e030740 20 client.16092 trim_cache size 0 max 16384 2024-04-17T06:09:35.494 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.491+0000 7f135e030740 1 Processor -- start 2024-04-17T06:09:35.494 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.491+0000 7f135e030740 1 -- start start 2024-04-17T06:09:35.494 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.491+0000 7f135e030740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c42899b0c0 0x55c4289a4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:35.494 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.491+0000 7f135e030740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c42899b9e0 0x55c4289b02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:35.494 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.491+0000 7f135e030740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4289a47b0 0x55c428994980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:35.494 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.491+0000 7f135e030740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f13541b0460 con 0x55c42899b9e0 2024-04-17T06:09:35.494 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.491+0000 7f135e030740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f13541add20 con 0x55c42899b0c0 2024-04-17T06:09:35.494 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.491+0000 7f135e030740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f13541abb90 con 0x55c4289a47b0 2024-04-17T06:09:35.494 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.491+0000 7f13589eb640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c42899b9e0 0x55c4289b02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:35.495 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.491+0000 7f13589eb640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c42899b9e0 0x55c4289b02a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45680/0 (socket says 172.21.15.47:45680) 2024-04-17T06:09:35.495 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.491+0000 7f13589eb640 1 -- 172.21.15.47:0/2093329898 learned_addr learned my addr 172.21.15.47:0/2093329898 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:09:35.495 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.492+0000 7f13589eb640 1 -- 172.21.15.47:0/2093329898 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4289a47b0 msgr2=0x55c428994980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:35.495 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.492+0000 7f13589eb640 1 --2- 172.21.15.47:0/2093329898 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4289a47b0 0x55c428994980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:35.495 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.492+0000 7f13589eb640 1 -- 172.21.15.47:0/2093329898 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c42899b0c0 msgr2=0x55c4289a4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:35.495 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.492+0000 7f13589eb640 1 --2- 172.21.15.47:0/2093329898 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c42899b0c0 0x55c4289a4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:35.495 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.492+0000 7f13589eb640 1 -- 172.21.15.47:0/2093329898 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c428991080 con 0x55c42899b9e0 2024-04-17T06:09:35.495 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.492+0000 7f13589eb640 1 --2- 172.21.15.47:0/2093329898 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c42899b9e0 0x55c4289b02a0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f134409bfe0 tx=0x7f134409caa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=babd8213745a60e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:35.495 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.492+0000 7f133effd640 1 -- 172.21.15.47:0/2093329898 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13440a0690 con 0x55c42899b9e0 2024-04-17T06:09:35.495 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.492+0000 7f133effd640 1 -- 172.21.15.47:0/2093329898 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13440b5070 con 0x55c42899b9e0 2024-04-17T06:09:35.495 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.492+0000 7f133effd640 1 -- 172.21.15.47:0/2093329898 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1344098950 con 0x55c42899b9e0 2024-04-17T06:09:35.495 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.492+0000 7f135e030740 1 -- 172.21.15.47:0/2093329898 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c42899b9e0 msgr2=0x55c4289b02a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:35.495 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.492+0000 7f135e030740 1 --2- 172.21.15.47:0/2093329898 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c42899b9e0 0x55c4289b02a0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f134409bfe0 tx=0x7f134409caa0 comp rx=0 tx=0).stop 2024-04-17T06:09:35.496 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.493+0000 7f135e030740 1 -- 172.21.15.47:0/2093329898 shutdown_connections 2024-04-17T06:09:35.496 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.493+0000 7f135e030740 1 --2- 172.21.15.47:0/2093329898 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4289a47b0 0x55c428994980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:35.496 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.493+0000 7f135e030740 1 --2- 172.21.15.47:0/2093329898 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c42899b9e0 0x55c4289b02a0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:35.496 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.493+0000 7f135e030740 1 --2- 172.21.15.47:0/2093329898 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c42899b0c0 0x55c4289a4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:35.496 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.493+0000 7f135e030740 1 -- 172.21.15.47:0/2093329898 >> 172.21.15.47:0/2093329898 conn(0x55c4289959e0 msgr2=0x55c428994270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:35.496 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.493+0000 7f135e030740 1 -- 172.21.15.47:0/2093329898 shutdown_connections 2024-04-17T06:09:35.496 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.493+0000 7f135e030740 1 -- 172.21.15.47:0/2093329898 wait complete. 2024-04-17T06:09:35.496 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.493+0000 7f135e030740 1 -- start start 2024-04-17T06:09:35.496 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.493+0000 7f135e030740 1 Processor -- start 2024-04-17T06:09:35.496 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.493+0000 7f135e030740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c42899b0c0 0x55c42899e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:35.496 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.493+0000 7f135e030740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c42899b9e0 0x55c42899e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:35.496 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.493+0000 7f135e030740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4289a47b0 0x55c4289ab7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:35.496 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.493+0000 7f135e030740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f13541ac1e0 con 0x55c4289a47b0 2024-04-17T06:09:35.496 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.493+0000 7f135e030740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f13541ad650 con 0x55c42899b0c0 2024-04-17T06:09:35.497 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.493+0000 7f135e030740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f13541ac8b0 con 0x55c42899b9e0 2024-04-17T06:09:35.497 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.493+0000 7f135e030740 10 client.0 _ll_register_callbacks cb 0x7f134c032590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:09:35.497 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.493+0000 7f13599ed640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4289a47b0 0x55c4289ab7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:35.497 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.493+0000 7f13599ed640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4289a47b0 0x55c4289ab7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45692/0 (socket says 172.21.15.47:45692) 2024-04-17T06:09:35.497 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.493+0000 7f13599ed640 1 -- 172.21.15.47:0/1513345062 learned_addr learned my addr 172.21.15.47:0/1513345062 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:09:35.497 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.494+0000 7f13591ec640 1 --2- 172.21.15.47:0/1513345062 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c42899b0c0 0x55c42899e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:35.497 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.494+0000 7f13589eb640 1 --2- 172.21.15.47:0/1513345062 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c42899b9e0 0x55c42899e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:35.497 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.494+0000 7f13599ed640 1 -- 172.21.15.47:0/1513345062 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c42899b9e0 msgr2=0x55c42899e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:35.497 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.494+0000 7f13599ed640 1 --2- 172.21.15.47:0/1513345062 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c42899b9e0 0x55c42899e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:35.497 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.494+0000 7f13599ed640 1 -- 172.21.15.47:0/1513345062 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c42899b0c0 msgr2=0x55c42899e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:35.497 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.494+0000 7f13599ed640 1 --2- 172.21.15.47:0/1513345062 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c42899b0c0 0x55c42899e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:35.497 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.494+0000 7f13599ed640 1 -- 172.21.15.47:0/1513345062 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4289abce0 con 0x55c4289a47b0 2024-04-17T06:09:35.497 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.494+0000 7f13589eb640 1 --2- 172.21.15.47:0/1513345062 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c42899b9e0 0x55c42899e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:09:35.497 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.494+0000 7f13599ed640 1 --2- 172.21.15.47:0/1513345062 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4289a47b0 0x55c4289ab7a0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f134c009bf0 tx=0x7f134c04d670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c025642aad144b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:35.497 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.494+0000 7f133cff9640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-17T06:09:35.498 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.494+0000 7f133cff9640 1 -- 172.21.15.47:0/1513345062 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f134c052070 con 0x55c4289a47b0 2024-04-17T06:09:35.498 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.494+0000 7f133cff9640 1 -- 172.21.15.47:0/1513345062 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8513+0+0 (secure 0 0 0) 0x7f134c034910 con 0x55c4289a47b0 2024-04-17T06:09:35.498 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.494+0000 7f135e030740 1 -- 172.21.15.47:0/1513345062 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13440b5070 con 0x55c4289a47b0 2024-04-17T06:09:35.498 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.494+0000 7f133cff9640 1 -- 172.21.15.47:0/1513345062 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f134c04d860 con 0x55c4289a47b0 2024-04-17T06:09:35.498 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.494+0000 7f135e030740 10 client.16098 fetch_fsmap learned FSMap version 32 2024-04-17T06:09:35.498 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.494+0000 7f135e030740 1 -- 172.21.15.47:0/1513345062 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c4289acff0 con 0x55c4289a47b0 2024-04-17T06:09:35.498 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.494+0000 7f133cff9640 1 -- 172.21.15.47:0/1513345062 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13440b5070 con 0x55c4289a47b0 2024-04-17T06:09:35.498 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.495+0000 7f133cff9640 1 -- 172.21.15.47:0/1513345062 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) ==== 8339+0+0 (secure 0 0 0) 0x7f134c03f040 con 0x55c4289a47b0 2024-04-17T06:09:35.498 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.495+0000 7f135e030740 10 client.16098 fetch_fsmap finished waiting for FSMap version 32 2024-04-17T06:09:35.498 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.495+0000 7f135e030740 10 client.16098 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-17T06:09:35.498 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.495+0000 7f135e030740 20 client.16098 populate_metadata read hostname 'smithi047' 2024-04-17T06:09:35.498 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.495+0000 7f135e030740 1 --2- 172.21.15.47:0/1513345062 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55c428a24070 0x55c428a26490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:35.498 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.495+0000 7f135e030740 4 client.16098 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-17T06:09:35.498 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.495+0000 7f135e030740 1 -- 172.21.15.47:0/1513345062 --> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c428908060 con 0x55c428a24070 2024-04-17T06:09:35.499 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.495+0000 7f13591ec640 1 --2- 172.21.15.47:0/1513345062 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55c428a24070 0x55c428a26490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:35.499 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.495+0000 7f13591ec640 1 --2- 172.21.15.47:0/1513345062 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55c428a24070 0x55c428a26490 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=492ef692afc18f83 server_cookie=f05be8f302c977ed in_seq=0 out_seq=0 2024-04-17T06:09:35.499 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.495+0000 7f133cff9640 10 client.16098 ms_handle_connect on v2:172.21.15.98:6838/2841287917 2024-04-17T06:09:35.502 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.504+0000 7f133cff9640 1 -- 172.21.15.47:0/1513345062 <== mds.0 v2:172.21.15.98:6838/2841287917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c428908060 con 0x55c428a24070 2024-04-17T06:09:35.502 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.504+0000 7f133cff9640 10 client.16098 handle_command_reply: tid=0 2024-04-17T06:09:35.502 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.504+0000 7f135e030740 1 client.16098 shutdown 2024-04-17T06:09:35.502 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.504+0000 7f135e030740 1 -- 172.21.15.47:0/1513345062 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4289a47b0 msgr2=0x55c4289ab7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:35.502 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.504+0000 7f135e030740 1 --2- 172.21.15.47:0/1513345062 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4289a47b0 0x55c4289ab7a0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f134c009bf0 tx=0x7f134c04d670 comp rx=0 tx=0).stop 2024-04-17T06:09:35.502 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.504+0000 7f135e030740 1 -- 172.21.15.47:0/1513345062 shutdown_connections 2024-04-17T06:09:35.502 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.504+0000 7f135e030740 1 --2- 172.21.15.47:0/1513345062 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55c428a24070 0x55c428a26490 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:35.503 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.504+0000 7f135e030740 1 --2- 172.21.15.47:0/1513345062 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4289a47b0 0x55c4289ab7a0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:35.503 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.504+0000 7f135e030740 1 --2- 172.21.15.47:0/1513345062 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c42899b9e0 0x55c42899e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:35.503 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.504+0000 7f133cff9640 0 client.16098 ms_handle_reset on v2:172.21.15.98:6838/2841287917 2024-04-17T06:09:35.503 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.504+0000 7f135e030740 1 --2- 172.21.15.47:0/1513345062 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c42899b0c0 0x55c42899e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:35.503 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.504+0000 7f135e030740 1 -- 172.21.15.47:0/1513345062 >> 172.21.15.47:0/1513345062 conn(0x55c4289959e0 msgr2=0x55c428994b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:35.503 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.504+0000 7f135e030740 1 -- 172.21.15.47:0/1513345062 shutdown_connections 2024-04-17T06:09:35.503 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.504+0000 7f135e030740 1 -- 172.21.15.47:0/1513345062 wait complete. 2024-04-17T06:09:35.503 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.504+0000 7f135e030740 20 client.16098 trim_cache size 0 max 16384 2024-04-17T06:09:35.503 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-04-17T06:09:35.503 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-04-17T06:09:35.503 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-04-17T06:09:35.504 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-04-17T06:09:35.504 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.505+0000 7f135b450640 1 -- 172.21.15.47:0/232641857 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f13400a3dc0 msgr2=0x7f13400a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:35.504 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.505+0000 7f135b450640 1 --2- 172.21.15.47:0/232641857 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f13400a3dc0 0x7f13400a6280 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f134c009430 tx=0x7f134c02c0f0 comp rx=0 tx=0).stop 2024-04-17T06:09:35.504 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.505+0000 7f135b450640 1 -- 172.21.15.47:0/232641857 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13541061c0 msgr2=0x7f1354198cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:35.504 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.505+0000 7f135b450640 1 --2- 172.21.15.47:0/232641857 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13541061c0 0x7f1354198cb0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f1344005fd0 tx=0x7f134400dde0 comp rx=0 tx=0).stop 2024-04-17T06:09:35.505 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.505+0000 7f135b450640 1 -- 172.21.15.47:0/232641857 shutdown_connections 2024-04-17T06:09:35.505 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.505+0000 7f135b450640 1 --2- 172.21.15.47:0/232641857 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f135419cfe0 0x7f135419d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:35.505 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.505+0000 7f135b450640 1 --2- 172.21.15.47:0/232641857 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f13400a3dc0 0x7f13400a6280 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:35.505 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.505+0000 7f135b450640 1 --2- 172.21.15.47:0/232641857 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13541061c0 0x7f1354198cb0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:35.505 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.505+0000 7f135b450640 1 --2- 172.21.15.47:0/232641857 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f13541010f0 0x7f1354198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:35.505 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.505+0000 7f135b450640 1 -- 172.21.15.47:0/232641857 >> 172.21.15.47:0/232641857 conn(0x7f13540771e0 msgr2=0x7f135406a970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:35.505 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.505+0000 7f135b450640 1 -- 172.21.15.47:0/232641857 shutdown_connections 2024-04-17T06:09:35.505 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:35.505+0000 7f135b450640 1 -- 172.21.15.47:0/232641857 wait complete. 2024-04-17T06:09:36.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:09:35 smithi103 ceph-mon[36243]: pgmap v258: 97 pgs: 97 active+clean; 4.2 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-04-17T06:09:36.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:09:35 smithi098 ceph-mon[36516]: pgmap v258: 97 pgs: 97 active+clean; 4.2 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-04-17T06:09:36.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:09:35 smithi047 ceph-mon[32223]: pgmap v258: 97 pgs: 97 active+clean; 4.2 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-04-17T06:09:38.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:09:37 smithi103 ceph-mon[36243]: pgmap v259: 97 pgs: 97 active+clean; 4.2 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.2 KiB/s wr, 9 op/s 2024-04-17T06:09:38.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:09:37 smithi098 ceph-mon[36516]: pgmap v259: 97 pgs: 97 active+clean; 4.2 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.2 KiB/s wr, 9 op/s 2024-04-17T06:09:38.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:09:37 smithi047 ceph-mon[32223]: pgmap v259: 97 pgs: 97 active+clean; 4.2 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.2 KiB/s wr, 9 op/s 2024-04-17T06:09:40.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:09:39 smithi103 ceph-mon[36243]: pgmap v260: 97 pgs: 97 active+clean; 4.2 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.2 KiB/s wr, 9 op/s 2024-04-17T06:09:40.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:09:39 smithi098 ceph-mon[36516]: pgmap v260: 97 pgs: 97 active+clean; 4.2 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.2 KiB/s wr, 9 op/s 2024-04-17T06:09:40.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:09:39 smithi047 ceph-mon[32223]: pgmap v260: 97 pgs: 97 active+clean; 4.2 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.2 KiB/s wr, 9 op/s 2024-04-17T06:09:42.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:09:41 smithi103 ceph-mon[36243]: pgmap v261: 97 pgs: 97 active+clean; 4.2 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-04-17T06:09:42.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:09:41 smithi098 ceph-mon[36516]: pgmap v261: 97 pgs: 97 active+clean; 4.2 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-04-17T06:09:42.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:09:41 smithi047 ceph-mon[32223]: pgmap v261: 97 pgs: 97 active+clean; 4.2 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-04-17T06:09:44.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:09:43 smithi103 ceph-mon[36243]: pgmap v262: 97 pgs: 97 active+clean; 4.2 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:09:44.134 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:09:43 smithi098 ceph-mon[36516]: pgmap v262: 97 pgs: 97 active+clean; 4.2 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:09:44.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:09:43 smithi047 ceph-mon[32223]: pgmap v262: 97 pgs: 97 active+clean; 4.2 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:09:46.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:09:45 smithi103 ceph-mon[36243]: pgmap v263: 97 pgs: 97 active+clean; 4.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-04-17T06:09:46.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:09:45 smithi098 ceph-mon[36516]: pgmap v263: 97 pgs: 97 active+clean; 4.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-04-17T06:09:46.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:09:45 smithi047 ceph-mon[32223]: pgmap v263: 97 pgs: 97 active+clean; 4.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-04-17T06:09:48.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:09:47 smithi103 ceph-mon[36243]: pgmap v264: 97 pgs: 97 active+clean; 4.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:09:48.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:09:47 smithi098 ceph-mon[36516]: pgmap v264: 97 pgs: 97 active+clean; 4.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:09:48.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:09:47 smithi047 ceph-mon[32223]: pgmap v264: 97 pgs: 97 active+clean; 4.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:09:48.604 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.604+0000 7f4457b1c640 1 -- 172.21.15.47:0/522344654 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4450100f90 msgr2=0x7f445010d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:48.604 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.604+0000 7f4457b1c640 1 --2- 172.21.15.47:0/522344654 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4450100f90 0x7f445010d4a0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f4448009cd0 tx=0x7f444802c0f0 comp rx=0 tx=0).stop 2024-04-17T06:09:48.604 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.604+0000 7f4457b1c640 1 -- 172.21.15.47:0/522344654 shutdown_connections 2024-04-17T06:09:48.604 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.604+0000 7f4457b1c640 1 --2- 172.21.15.47:0/522344654 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4450100f90 0x7f445010d4a0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:48.604 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.604+0000 7f4457b1c640 1 --2- 172.21.15.47:0/522344654 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f44501005d0 0x7f4450100a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:48.604 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.604+0000 7f4457b1c640 1 --2- 172.21.15.47:0/522344654 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4450106670 0x7f4450106a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:48.604 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.604+0000 7f4457b1c640 1 -- 172.21.15.47:0/522344654 >> 172.21.15.47:0/522344654 conn(0x7f44500773a0 msgr2=0x7f44500fe770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:48.604 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.604+0000 7f4457b1c640 1 -- 172.21.15.47:0/522344654 shutdown_connections 2024-04-17T06:09:48.604 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.604+0000 7f4457b1c640 1 -- 172.21.15.47:0/522344654 wait complete. 2024-04-17T06:09:48.604 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.604+0000 7f4457b1c640 1 Processor -- start 2024-04-17T06:09:48.604 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.604+0000 7f4457b1c640 1 -- start start 2024-04-17T06:09:48.605 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.605+0000 7f4457b1c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44501005d0 0x7f4450102270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:48.605 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.605+0000 7f4457b1c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4450100f90 0x7f44501027b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:48.605 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.605+0000 7f4457b1c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4450106670 0x7f4450102cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:48.605 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.605+0000 7f4457b1c640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f4450111220 con 0x7f44501005d0 2024-04-17T06:09:48.605 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.605+0000 7f4457b1c640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f44501110a0 con 0x7f4450106670 2024-04-17T06:09:48.605 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.605+0000 7f4457b1c640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f44501113a0 con 0x7f4450100f90 2024-04-17T06:09:48.605 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.605+0000 7f44558b8640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44501005d0 0x7f4450102270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:48.605 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.605+0000 7f44558b8640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44501005d0 0x7f4450102270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33628/0 (socket says 172.21.15.47:33628) 2024-04-17T06:09:48.605 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.605+0000 7f44558b8640 1 -- 172.21.15.47:0/629826671 learned_addr learned my addr 172.21.15.47:0/629826671 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:09:48.605 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.605+0000 7f44560b9640 1 --2- 172.21.15.47:0/629826671 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4450106670 0x7f4450102cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:48.605 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.605+0000 7f44550b7640 1 --2- 172.21.15.47:0/629826671 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4450100f90 0x7f44501027b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:48.605 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.605+0000 7f44558b8640 1 -- 172.21.15.47:0/629826671 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4450100f90 msgr2=0x7f44501027b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:48.605 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.605+0000 7f44558b8640 1 --2- 172.21.15.47:0/629826671 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4450100f90 0x7f44501027b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:48.606 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.605+0000 7f44558b8640 1 -- 172.21.15.47:0/629826671 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4450106670 msgr2=0x7f4450102cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:48.606 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.605+0000 7f44558b8640 1 --2- 172.21.15.47:0/629826671 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4450106670 0x7f4450102cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:48.606 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.605+0000 7f44558b8640 1 -- 172.21.15.47:0/629826671 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44501a6bd0 con 0x7f44501005d0 2024-04-17T06:09:48.606 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.605+0000 7f44550b7640 1 --2- 172.21.15.47:0/629826671 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4450100f90 0x7f44501027b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:09:48.606 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.605+0000 7f44558b8640 1 --2- 172.21.15.47:0/629826671 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44501005d0 0x7f4450102270 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f444c00e9c0 tx=0x7f444c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:48.606 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.605+0000 7f4446ffd640 1 -- 172.21.15.47:0/629826671 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f444c00cd80 con 0x7f44501005d0 2024-04-17T06:09:48.606 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.606+0000 7f4446ffd640 1 -- 172.21.15.47:0/629826671 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f444c0035f0 con 0x7f44501005d0 2024-04-17T06:09:48.606 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.606+0000 7f4457b1c640 1 -- 172.21.15.47:0/629826671 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44501a6ec0 con 0x7f44501005d0 2024-04-17T06:09:48.606 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.606+0000 7f4446ffd640 1 -- 172.21.15.47:0/629826671 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f444c010690 con 0x7f44501005d0 2024-04-17T06:09:48.606 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.606+0000 7f4457b1c640 1 -- 172.21.15.47:0/629826671 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44501a7380 con 0x7f44501005d0 2024-04-17T06:09:48.608 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.609+0000 7f4446ffd640 1 -- 172.21.15.47:0/629826671 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f444c0037a0 con 0x7f44501005d0 2024-04-17T06:09:48.608 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.609+0000 7f445a7e3740 1 Processor -- start 2024-04-17T06:09:48.608 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.609+0000 7f445a7e3740 1 -- start start 2024-04-17T06:09:48.609 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.609+0000 7f445a7e3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d3a35c50c0 0x55d3a35c54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:48.609 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.609+0000 7f445a7e3740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d3a35c59e0 0x55d3a35d6a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:48.609 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.609+0000 7f445a7e3740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3a35d6fc0 0x55d3a35d93b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:48.609 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.609+0000 7f445a7e3740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55d3a352a7d0 con 0x55d3a35d6fc0 2024-04-17T06:09:48.609 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.609+0000 7f445a7e3740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55d3a34c7f70 con 0x55d3a35c59e0 2024-04-17T06:09:48.609 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.609+0000 7f445a7e3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d3a3526790 con 0x55d3a35c50c0 2024-04-17T06:09:48.609 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.610+0000 7f44550b7640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3a35d6fc0 0x55d3a35d93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:48.609 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.610+0000 7f44550b7640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3a35d6fc0 0x55d3a35d93b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33634/0 (socket says 172.21.15.47:33634) 2024-04-17T06:09:48.609 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.610+0000 7f44550b7640 1 -- 172.21.15.47:0/1021945349 learned_addr learned my addr 172.21.15.47:0/1021945349 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:09:48.609 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.610+0000 7f44560b9640 1 --2- 172.21.15.47:0/1021945349 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d3a35c50c0 0x55d3a35c54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:48.609 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.610+0000 7f4446ffd640 1 --2- 172.21.15.47:0/629826671 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f443c0a3e70 0x7f443c0a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:48.609 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.610+0000 7f44558b8640 1 --2- 172.21.15.47:0/1021945349 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d3a35c59e0 0x55d3a35d6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:48.609 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.610+0000 7f44560b9640 1 --2- 172.21.15.47:0/629826671 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f443c0a3e70 0x7f443c0a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:48.609 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.610+0000 7f44550b7640 1 -- 172.21.15.47:0/1021945349 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d3a35c50c0 msgr2=0x55d3a35c54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:48.609 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.610+0000 7f44550b7640 1 --2- 172.21.15.47:0/1021945349 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d3a35c50c0 0x55d3a35c54a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:48.610 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.610+0000 7f44550b7640 1 -- 172.21.15.47:0/1021945349 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d3a35c59e0 msgr2=0x55d3a35d6a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:48.610 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.610+0000 7f44550b7640 1 --2- 172.21.15.47:0/1021945349 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d3a35c59e0 0x55d3a35d6a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:48.610 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.610+0000 7f4446ffd640 1 -- 172.21.15.47:0/629826671 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8513+0+0 (secure 0 0 0) 0x7f444c0d0ad0 con 0x7f44501005d0 2024-04-17T06:09:48.610 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.610+0000 7f44550b7640 1 -- 172.21.15.47:0/1021945349 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3a323d210 con 0x55d3a35d6fc0 2024-04-17T06:09:48.610 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.610+0000 7f44560b9640 1 --2- 172.21.15.47:0/1021945349 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d3a35c50c0 0x55d3a35c54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T06:09:48.610 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.610+0000 7f44550b7640 1 --2- 172.21.15.47:0/1021945349 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3a35d6fc0 0x55d3a35d93b0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f4440009c00 tx=0x7f4440010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6ebea0dd8fa3efe server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:48.610 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.611+0000 7f44560b9640 1 --2- 172.21.15.47:0/629826671 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f443c0a3e70 0x7f443c0a6330 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4448005cc0 tx=0x7f444802c0f0 comp rx=0 tx=0).ready entity=mgr.24923 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:48.610 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.611+0000 7f443bfff640 1 -- 172.21.15.47:0/1021945349 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4440004030 con 0x55d3a35d6fc0 2024-04-17T06:09:48.610 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.611+0000 7f443bfff640 1 -- 172.21.15.47:0/1021945349 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44400041d0 con 0x55d3a35d6fc0 2024-04-17T06:09:48.610 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.611+0000 7f443bfff640 1 -- 172.21.15.47:0/1021945349 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44400126b0 con 0x55d3a35d6fc0 2024-04-17T06:09:48.615 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.611+0000 7f445a7e3740 1 -- 172.21.15.47:0/1021945349 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3a35d6fc0 msgr2=0x55d3a35d93b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:48.615 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.611+0000 7f445a7e3740 1 --2- 172.21.15.47:0/1021945349 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3a35d6fc0 0x55d3a35d93b0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f4440009c00 tx=0x7f4440010740 comp rx=0 tx=0).stop 2024-04-17T06:09:48.615 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.611+0000 7f445a7e3740 1 -- 172.21.15.47:0/1021945349 shutdown_connections 2024-04-17T06:09:48.615 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.611+0000 7f445a7e3740 1 --2- 172.21.15.47:0/1021945349 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3a35d6fc0 0x55d3a35d93b0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:48.615 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.611+0000 7f445a7e3740 1 --2- 172.21.15.47:0/1021945349 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d3a35c59e0 0x55d3a35d6a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:48.615 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.611+0000 7f445a7e3740 1 --2- 172.21.15.47:0/1021945349 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d3a35c50c0 0x55d3a35c54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:48.615 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.611+0000 7f445a7e3740 1 -- 172.21.15.47:0/1021945349 >> 172.21.15.47:0/1021945349 conn(0x55d3a35bf9e0 msgr2=0x55d3a35bfdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:48.616 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.612+0000 7f445a7e3740 1 -- 172.21.15.47:0/1021945349 shutdown_connections 2024-04-17T06:09:48.616 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.612+0000 7f445a7e3740 1 -- 172.21.15.47:0/1021945349 wait complete. 2024-04-17T06:09:48.616 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.612+0000 7f445a7e3740 1 -- start start 2024-04-17T06:09:48.616 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.612+0000 7f445a7e3740 1 Processor -- start 2024-04-17T06:09:48.616 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.612+0000 7f445a7e3740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d3a35c50c0 0x55d3a35d4010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:48.616 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.612+0000 7f445a7e3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d3a35c59e0 0x55d3a35d4550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:48.616 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.612+0000 7f445a7e3740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3a35d6fc0 0x55d3a35d4a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:48.616 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.612+0000 7f445a7e3740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55d3a35d9bf0 con 0x55d3a35d6fc0 2024-04-17T06:09:48.616 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.612+0000 7f445a7e3740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55d3a34b2e90 con 0x55d3a35c50c0 2024-04-17T06:09:48.616 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.612+0000 7f445a7e3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d3a35d9d70 con 0x55d3a35c59e0 2024-04-17T06:09:48.616 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.612+0000 7f445a7e3740 10 client.0 _ll_register_callbacks cb 0x55d3a34c8d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:09:48.616 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.613+0000 7f44560b9640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3a35d6fc0 0x55d3a35d4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:48.616 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.613+0000 7f44560b9640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3a35d6fc0 0x55d3a35d4a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33636/0 (socket says 172.21.15.47:33636) 2024-04-17T06:09:48.617 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.613+0000 7f44560b9640 1 -- 172.21.15.47:0/398680423 learned_addr learned my addr 172.21.15.47:0/398680423 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:09:48.617 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.613+0000 7f44550b7640 1 --2- 172.21.15.47:0/398680423 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d3a35c59e0 0x55d3a35d4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:48.617 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.613+0000 7f44558b8640 1 --2- 172.21.15.47:0/398680423 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d3a35c50c0 0x55d3a35d4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:48.617 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.613+0000 7f44560b9640 1 -- 172.21.15.47:0/398680423 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d3a35c59e0 msgr2=0x55d3a35d4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:48.617 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.613+0000 7f44560b9640 1 --2- 172.21.15.47:0/398680423 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d3a35c59e0 0x55d3a35d4550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:48.617 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.613+0000 7f44560b9640 1 -- 172.21.15.47:0/398680423 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d3a35c50c0 msgr2=0x55d3a35d4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:48.617 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.613+0000 7f44560b9640 1 --2- 172.21.15.47:0/398680423 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d3a35c50c0 0x55d3a35d4010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:48.617 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.613+0000 7f44560b9640 1 -- 172.21.15.47:0/398680423 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3a3309620 con 0x55d3a35d6fc0 2024-04-17T06:09:48.617 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.613+0000 7f44550b7640 1 --2- 172.21.15.47:0/398680423 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d3a35c59e0 0x55d3a35d4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T06:09:48.617 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.613+0000 7f44558b8640 1 --2- 172.21.15.47:0/398680423 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d3a35c50c0 0x55d3a35d4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:09:48.617 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.613+0000 7f44560b9640 1 --2- 172.21.15.47:0/398680423 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3a35d6fc0 0x55d3a35d4a90 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f44480329e0 tx=0x7f44480095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7706af70d3f2058 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:48.617 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.613+0000 7f44397fa640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-17T06:09:48.617 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.613+0000 7f44397fa640 1 -- 172.21.15.47:0/398680423 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4448052060 con 0x55d3a35d6fc0 2024-04-17T06:09:48.617 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.613+0000 7f44397fa640 1 -- 172.21.15.47:0/398680423 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8513+0+0 (secure 0 0 0) 0x7f4448004240 con 0x55d3a35d6fc0 2024-04-17T06:09:48.617 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.613+0000 7f445a7e3740 1 -- 172.21.15.47:0/398680423 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44400041d0 con 0x55d3a35d6fc0 2024-04-17T06:09:48.618 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.614+0000 7f44397fa640 1 -- 172.21.15.47:0/398680423 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f4448039040 con 0x55d3a35d6fc0 2024-04-17T06:09:48.618 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.614+0000 7f445a7e3740 10 client.16110 fetch_fsmap learned FSMap version 32 2024-04-17T06:09:48.618 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.614+0000 7f445a7e3740 1 -- 172.21.15.47:0/398680423 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d3a35e5e70 con 0x55d3a35d6fc0 2024-04-17T06:09:48.618 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.614+0000 7f44397fa640 1 -- 172.21.15.47:0/398680423 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44400041d0 con 0x55d3a35d6fc0 2024-04-17T06:09:48.618 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.614+0000 7f44397fa640 1 -- 172.21.15.47:0/398680423 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) ==== 8339+0+0 (secure 0 0 0) 0x7f444803f070 con 0x55d3a35d6fc0 2024-04-17T06:09:48.618 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.614+0000 7f445a7e3740 10 client.16110 fetch_fsmap finished waiting for FSMap version 32 2024-04-17T06:09:48.618 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.614+0000 7f445a7e3740 10 client.16110 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-17T06:09:48.618 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.614+0000 7f445a7e3740 20 client.16110 populate_metadata read hostname 'smithi047' 2024-04-17T06:09:48.618 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.614+0000 7f445a7e3740 1 --2- 172.21.15.47:0/398680423 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55d3a35e7330 0x55d3a35e9720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:48.618 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.614+0000 7f445a7e3740 4 client.16110 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T06:09:48.618 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.614+0000 7f445a7e3740 1 -- 172.21.15.47:0/398680423 --> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d3a35381f0 con 0x55d3a35e7330 2024-04-17T06:09:48.618 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.614+0000 7f44558b8640 1 --2- 172.21.15.47:0/398680423 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55d3a35e7330 0x55d3a35e9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:48.618 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.615+0000 7f44558b8640 1 --2- 172.21.15.47:0/398680423 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55d3a35e7330 0x55d3a35e9720 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32f26062c1633d2d server_cookie=2db0c7592b1cdce4 in_seq=0 out_seq=0 2024-04-17T06:09:48.618 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.615+0000 7f44397fa640 10 client.16110 ms_handle_connect on v2:172.21.15.98:6838/2841287917 2024-04-17T06:09:48.619 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.616+0000 7f44397fa640 1 -- 172.21.15.47:0/398680423 <== mds.0 v2:172.21.15.98:6838/2841287917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d3a35381f0 con 0x55d3a35e7330 2024-04-17T06:09:48.619 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.616+0000 7f44397fa640 10 client.16110 handle_command_reply: tid=0 2024-04-17T06:09:48.619 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.616+0000 7f445a7e3740 1 client.16110 shutdown 2024-04-17T06:09:48.619 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.616+0000 7f445a7e3740 1 -- 172.21.15.47:0/398680423 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3a35d6fc0 msgr2=0x55d3a35d4a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:48.619 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.616+0000 7f445a7e3740 1 --2- 172.21.15.47:0/398680423 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3a35d6fc0 0x55d3a35d4a90 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f44480329e0 tx=0x7f44480095c0 comp rx=0 tx=0).stop 2024-04-17T06:09:48.619 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.616+0000 7f445a7e3740 1 -- 172.21.15.47:0/398680423 shutdown_connections 2024-04-17T06:09:48.619 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.616+0000 7f445a7e3740 1 --2- 172.21.15.47:0/398680423 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55d3a35e7330 0x55d3a35e9720 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:48.619 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.616+0000 7f445a7e3740 1 --2- 172.21.15.47:0/398680423 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3a35d6fc0 0x55d3a35d4a90 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:48.619 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.616+0000 7f445a7e3740 1 --2- 172.21.15.47:0/398680423 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d3a35c59e0 0x55d3a35d4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:48.619 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.616+0000 7f44397fa640 0 client.16110 ms_handle_reset on v2:172.21.15.98:6838/2841287917 2024-04-17T06:09:48.619 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.616+0000 7f445a7e3740 1 --2- 172.21.15.47:0/398680423 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d3a35c50c0 0x55d3a35d4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:48.619 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.616+0000 7f445a7e3740 1 -- 172.21.15.47:0/398680423 >> 172.21.15.47:0/398680423 conn(0x55d3a35bf9e0 msgr2=0x55d3a35c1730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:48.619 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.616+0000 7f445a7e3740 1 -- 172.21.15.47:0/398680423 shutdown_connections 2024-04-17T06:09:48.619 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.616+0000 7f445a7e3740 1 -- 172.21.15.47:0/398680423 wait complete. 2024-04-17T06:09:48.620 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.616+0000 7f445a7e3740 20 client.16110 trim_cache size 0 max 16384 2024-04-17T06:09:48.631 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.631+0000 7f445a7e3740 1 Processor -- start 2024-04-17T06:09:48.631 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.631+0000 7f445a7e3740 1 -- start start 2024-04-17T06:09:48.631 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.631+0000 7f445a7e3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d3a35c50c0 0x55d3a35d6740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:48.631 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.631+0000 7f445a7e3740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3a35c59e0 0x55d3a35e2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:48.631 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.631+0000 7f445a7e3740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d3a35d6fc0 0x55d3a35ef030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:48.631 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.631+0000 7f445a7e3740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f44501ae380 con 0x55d3a35c59e0 2024-04-17T06:09:48.631 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.631+0000 7f445a7e3740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f44501abc40 con 0x55d3a35d6fc0 2024-04-17T06:09:48.631 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.631+0000 7f445a7e3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f44501a9af0 con 0x55d3a35c50c0 2024-04-17T06:09:48.631 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.631+0000 7f44550b7640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3a35c59e0 0x55d3a35e2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:48.631 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.631+0000 7f44550b7640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3a35c59e0 0x55d3a35e2ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33642/0 (socket says 172.21.15.47:33642) 2024-04-17T06:09:48.631 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.632+0000 7f44550b7640 1 -- 172.21.15.47:0/3545276360 learned_addr learned my addr 172.21.15.47:0/3545276360 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:09:48.632 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.632+0000 7f44558b8640 1 --2- 172.21.15.47:0/3545276360 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d3a35c50c0 0x55d3a35d6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:48.632 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.632+0000 7f44550b7640 1 -- 172.21.15.47:0/3545276360 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d3a35c50c0 msgr2=0x55d3a35d6740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:48.632 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.632+0000 7f44550b7640 1 --2- 172.21.15.47:0/3545276360 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d3a35c50c0 0x55d3a35d6740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:48.632 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.632+0000 7f44550b7640 1 -- 172.21.15.47:0/3545276360 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d3a35d6fc0 msgr2=0x55d3a35ef030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:48.632 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.632+0000 7f44560b9640 1 --2- 172.21.15.47:0/3545276360 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d3a35d6fc0 0x55d3a35ef030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:48.632 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.632+0000 7f44550b7640 1 --2- 172.21.15.47:0/3545276360 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d3a35d6fc0 0x55d3a35ef030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:48.632 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.632+0000 7f44550b7640 1 -- 172.21.15.47:0/3545276360 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3a35c0ba0 con 0x55d3a35c59e0 2024-04-17T06:09:48.632 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.632+0000 7f44550b7640 1 --2- 172.21.15.47:0/3545276360 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3a35c59e0 0x55d3a35e2ab0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x55d3a35c0b70 tx=0x7f4440009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c21c83a1339cc723 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:48.632 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.632+0000 7f443b7fe640 1 -- 172.21.15.47:0/3545276360 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4440005850 con 0x55d3a35c59e0 2024-04-17T06:09:48.632 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.632+0000 7f443b7fe640 1 -- 172.21.15.47:0/3545276360 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44400055a0 con 0x55d3a35c59e0 2024-04-17T06:09:48.632 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.632+0000 7f443b7fe640 1 -- 172.21.15.47:0/3545276360 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44400129b0 con 0x55d3a35c59e0 2024-04-17T06:09:48.632 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.633+0000 7f445a7e3740 1 -- 172.21.15.47:0/3545276360 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3a35c59e0 msgr2=0x55d3a35e2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:48.632 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.633+0000 7f445a7e3740 1 --2- 172.21.15.47:0/3545276360 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3a35c59e0 0x55d3a35e2ab0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x55d3a35c0b70 tx=0x7f4440009b60 comp rx=0 tx=0).stop 2024-04-17T06:09:48.632 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.633+0000 7f445a7e3740 1 -- 172.21.15.47:0/3545276360 shutdown_connections 2024-04-17T06:09:48.632 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.633+0000 7f445a7e3740 1 --2- 172.21.15.47:0/3545276360 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d3a35d6fc0 0x55d3a35ef030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:48.633 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.633+0000 7f445a7e3740 1 --2- 172.21.15.47:0/3545276360 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3a35c59e0 0x55d3a35e2ab0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:48.633 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.633+0000 7f445a7e3740 1 --2- 172.21.15.47:0/3545276360 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d3a35c50c0 0x55d3a35d6740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:48.633 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.633+0000 7f445a7e3740 1 -- 172.21.15.47:0/3545276360 >> 172.21.15.47:0/3545276360 conn(0x55d3a35bf9e0 msgr2=0x55d3a35cf340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:48.633 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.633+0000 7f445a7e3740 1 -- 172.21.15.47:0/3545276360 shutdown_connections 2024-04-17T06:09:48.633 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.633+0000 7f445a7e3740 1 -- 172.21.15.47:0/3545276360 wait complete. 2024-04-17T06:09:48.633 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.633+0000 7f445a7e3740 1 -- start start 2024-04-17T06:09:48.633 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.633+0000 7f445a7e3740 1 Processor -- start 2024-04-17T06:09:48.636 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.634+0000 7f445a7e3740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3a35c50c0 0x55d3a35df1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:48.636 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.634+0000 7f445a7e3740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d3a35c59e0 0x55d3a35d0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:48.636 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.634+0000 7f445a7e3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d3a35d6fc0 0x55d3a35d0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:48.636 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.634+0000 7f445a7e3740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f44501aa140 con 0x55d3a35c50c0 2024-04-17T06:09:48.636 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.634+0000 7f445a7e3740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f44501ab570 con 0x55d3a35c59e0 2024-04-17T06:09:48.636 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.634+0000 7f445a7e3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f44501aa7d0 con 0x55d3a35d6fc0 2024-04-17T06:09:48.636 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.634+0000 7f445a7e3740 10 client.0 _ll_register_callbacks cb 0x7f444c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:09:48.637 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.634+0000 7f44558b8640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3a35c50c0 0x55d3a35df1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:48.637 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.634+0000 7f44558b8640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3a35c50c0 0x55d3a35df1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33656/0 (socket says 172.21.15.47:33656) 2024-04-17T06:09:48.637 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.634+0000 7f44558b8640 1 -- 172.21.15.47:0/383432039 learned_addr learned my addr 172.21.15.47:0/383432039 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:09:48.637 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.634+0000 7f44550b7640 1 --2- 172.21.15.47:0/383432039 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d3a35c59e0 0x55d3a35d0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:48.637 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.634+0000 7f44560b9640 1 --2- 172.21.15.47:0/383432039 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d3a35d6fc0 0x55d3a35d0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:48.637 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.634+0000 7f44558b8640 1 -- 172.21.15.47:0/383432039 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d3a35d6fc0 msgr2=0x55d3a35d0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:48.637 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.634+0000 7f44558b8640 1 --2- 172.21.15.47:0/383432039 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d3a35d6fc0 0x55d3a35d0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:48.637 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.634+0000 7f44558b8640 1 -- 172.21.15.47:0/383432039 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d3a35c59e0 msgr2=0x55d3a35d0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:48.637 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.634+0000 7f44558b8640 1 --2- 172.21.15.47:0/383432039 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d3a35c59e0 0x55d3a35d0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:48.637 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.634+0000 7f44558b8640 1 -- 172.21.15.47:0/383432039 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3a35d1380 con 0x55d3a35c50c0 2024-04-17T06:09:48.637 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.634+0000 7f44558b8640 1 --2- 172.21.15.47:0/383432039 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3a35c50c0 0x55d3a35df1b0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f444c09c770 tx=0x7f444c098e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=294662a9f2d630c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:48.637 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.634+0000 7f44397fa640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-17T06:09:48.637 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.634+0000 7f44397fa640 1 -- 172.21.15.47:0/383432039 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f444c0a0e20 con 0x55d3a35c50c0 2024-04-17T06:09:48.637 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.634+0000 7f44397fa640 1 -- 172.21.15.47:0/383432039 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8513+0+0 (secure 0 0 0) 0x7f444c0ce4a0 con 0x55d3a35c50c0 2024-04-17T06:09:48.638 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.635+0000 7f445a7e3740 1 -- 172.21.15.47:0/383432039 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44400055a0 con 0x55d3a35c50c0 2024-04-17T06:09:48.638 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.635+0000 7f44397fa640 1 -- 172.21.15.47:0/383432039 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f444c0a56e0 con 0x55d3a35c50c0 2024-04-17T06:09:48.638 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.635+0000 7f445a7e3740 10 client.16116 fetch_fsmap learned FSMap version 32 2024-04-17T06:09:48.638 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.635+0000 7f445a7e3740 1 -- 172.21.15.47:0/383432039 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d3a364d6a0 con 0x55d3a35c50c0 2024-04-17T06:09:48.638 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.635+0000 7f44397fa640 1 -- 172.21.15.47:0/383432039 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44400055a0 con 0x55d3a35c50c0 2024-04-17T06:09:48.638 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.635+0000 7f44397fa640 1 -- 172.21.15.47:0/383432039 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) ==== 8339+0+0 (secure 0 0 0) 0x7f444c0a5250 con 0x55d3a35c50c0 2024-04-17T06:09:48.638 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.635+0000 7f445a7e3740 10 client.16116 fetch_fsmap finished waiting for FSMap version 32 2024-04-17T06:09:48.638 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.635+0000 7f445a7e3740 10 client.16116 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-17T06:09:48.638 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.635+0000 7f445a7e3740 20 client.16116 populate_metadata read hostname 'smithi047' 2024-04-17T06:09:48.638 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.635+0000 7f445a7e3740 1 --2- 172.21.15.47:0/383432039 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55d3a364ddd0 0x55d3a36501f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:48.638 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.635+0000 7f445a7e3740 4 client.16116 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-17T06:09:48.638 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.635+0000 7f445a7e3740 1 -- 172.21.15.47:0/383432039 --> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d3a3532060 con 0x55d3a364ddd0 2024-04-17T06:09:48.638 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.635+0000 7f44550b7640 1 --2- 172.21.15.47:0/383432039 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55d3a364ddd0 0x55d3a36501f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:48.638 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.636+0000 7f44397fa640 10 client.16116 ms_handle_connect on v2:172.21.15.98:6838/2841287917 2024-04-17T06:09:48.639 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.636+0000 7f44550b7640 1 --2- 172.21.15.47:0/383432039 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55d3a364ddd0 0x55d3a36501f0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a696e6175f61904 server_cookie=a4a59938d548e9e8 in_seq=0 out_seq=0 2024-04-17T06:09:48.642 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.644+0000 7f44397fa640 1 -- 172.21.15.47:0/383432039 <== mds.0 v2:172.21.15.98:6838/2841287917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d3a3532060 con 0x55d3a364ddd0 2024-04-17T06:09:48.642 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.644+0000 7f44397fa640 10 client.16116 handle_command_reply: tid=0 2024-04-17T06:09:48.642 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.644+0000 7f445a7e3740 1 client.16116 shutdown 2024-04-17T06:09:48.642 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.644+0000 7f445a7e3740 1 -- 172.21.15.47:0/383432039 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3a35c50c0 msgr2=0x55d3a35df1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:48.643 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.644+0000 7f445a7e3740 1 --2- 172.21.15.47:0/383432039 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3a35c50c0 0x55d3a35df1b0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f444c09c770 tx=0x7f444c098e30 comp rx=0 tx=0).stop 2024-04-17T06:09:48.643 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.644+0000 7f445a7e3740 1 -- 172.21.15.47:0/383432039 shutdown_connections 2024-04-17T06:09:48.643 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.644+0000 7f445a7e3740 1 --2- 172.21.15.47:0/383432039 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55d3a364ddd0 0x55d3a36501f0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:48.643 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.644+0000 7f445a7e3740 1 --2- 172.21.15.47:0/383432039 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d3a35d6fc0 0x55d3a35d0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:48.643 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.644+0000 7f445a7e3740 1 --2- 172.21.15.47:0/383432039 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d3a35c59e0 0x55d3a35d0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:48.643 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.644+0000 7f445a7e3740 1 --2- 172.21.15.47:0/383432039 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3a35c50c0 0x55d3a35df1b0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:48.643 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.644+0000 7f44397fa640 0 client.16116 ms_handle_reset on v2:172.21.15.98:6838/2841287917 2024-04-17T06:09:48.643 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.644+0000 7f445a7e3740 1 -- 172.21.15.47:0/383432039 >> 172.21.15.47:0/383432039 conn(0x55d3a35bf9e0 msgr2=0x55d3a35c0de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:48.643 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.644+0000 7f445a7e3740 1 -- 172.21.15.47:0/383432039 shutdown_connections 2024-04-17T06:09:48.643 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.644+0000 7f445a7e3740 1 -- 172.21.15.47:0/383432039 wait complete. 2024-04-17T06:09:48.643 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.644+0000 7f445a7e3740 20 client.16116 trim_cache size 0 max 16384 2024-04-17T06:09:48.644 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-04-17T06:09:48.644 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-04-17T06:09:48.644 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-04-17T06:09:48.644 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-04-17T06:09:48.644 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.645+0000 7f4457b1c640 1 -- 172.21.15.47:0/629826671 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f443c0a3e70 msgr2=0x7f443c0a6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:48.645 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.645+0000 7f4457b1c640 1 --2- 172.21.15.47:0/629826671 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f443c0a3e70 0x7f443c0a6330 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4448005cc0 tx=0x7f444802c0f0 comp rx=0 tx=0).stop 2024-04-17T06:09:48.645 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.645+0000 7f4457b1c640 1 -- 172.21.15.47:0/629826671 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44501005d0 msgr2=0x7f4450102270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:48.645 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.645+0000 7f4457b1c640 1 --2- 172.21.15.47:0/629826671 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44501005d0 0x7f4450102270 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f444c00e9c0 tx=0x7f444c00ee80 comp rx=0 tx=0).stop 2024-04-17T06:09:48.645 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.645+0000 7f4457b1c640 1 -- 172.21.15.47:0/629826671 shutdown_connections 2024-04-17T06:09:48.645 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.645+0000 7f4457b1c640 1 --2- 172.21.15.47:0/629826671 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4450106670 0x7f4450102cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:48.645 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.645+0000 7f4457b1c640 1 --2- 172.21.15.47:0/629826671 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4450100f90 0x7f44501027b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:48.645 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.645+0000 7f4457b1c640 1 --2- 172.21.15.47:0/629826671 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f443c0a3e70 0x7f443c0a6330 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:48.645 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.645+0000 7f4457b1c640 1 --2- 172.21.15.47:0/629826671 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44501005d0 0x7f4450102270 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:48.645 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.645+0000 7f4457b1c640 1 -- 172.21.15.47:0/629826671 >> 172.21.15.47:0/629826671 conn(0x7f44500773a0 msgr2=0x7f44500ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:48.645 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.645+0000 7f4457b1c640 1 -- 172.21.15.47:0/629826671 shutdown_connections 2024-04-17T06:09:48.645 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:09:48.645+0000 7f4457b1c640 1 -- 172.21.15.47:0/629826671 wait complete. 2024-04-17T06:09:50.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:09:49 smithi098 ceph-mon[36516]: pgmap v265: 97 pgs: 97 active+clean; 4.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:09:50.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:09:49 smithi047 ceph-mon[32223]: pgmap v265: 97 pgs: 97 active+clean; 4.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:09:50.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:09:49 smithi103 ceph-mon[36243]: pgmap v265: 97 pgs: 97 active+clean; 4.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:09:52.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:09:51 smithi098 ceph-mon[36516]: pgmap v266: 97 pgs: 97 active+clean; 4.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 19 KiB/s wr, 18 op/s 2024-04-17T06:09:52.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:09:51 smithi047 ceph-mon[32223]: pgmap v266: 97 pgs: 97 active+clean; 4.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 19 KiB/s wr, 18 op/s 2024-04-17T06:09:52.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:09:51 smithi103 ceph-mon[36243]: pgmap v266: 97 pgs: 97 active+clean; 4.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 19 KiB/s wr, 18 op/s 2024-04-17T06:09:54.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:09:53 smithi098 ceph-mon[36516]: pgmap v267: 97 pgs: 97 active+clean; 4.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:09:54.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:09:53 smithi047 ceph-mon[32223]: pgmap v267: 97 pgs: 97 active+clean; 4.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:09:54.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:09:53 smithi103 ceph-mon[36243]: pgmap v267: 97 pgs: 97 active+clean; 4.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:09:56.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:09:55 smithi098 ceph-mon[36516]: pgmap v268: 97 pgs: 97 active+clean; 4.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-04-17T06:09:56.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:09:55 smithi047 ceph-mon[32223]: pgmap v268: 97 pgs: 97 active+clean; 4.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-04-17T06:09:56.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:09:55 smithi103 ceph-mon[36243]: pgmap v268: 97 pgs: 97 active+clean; 4.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-04-17T06:09:58.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:09:57 smithi098 ceph-mon[36516]: pgmap v269: 97 pgs: 97 active+clean; 4.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:09:58.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:09:57 smithi047 ceph-mon[32223]: pgmap v269: 97 pgs: 97 active+clean; 4.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:09:58.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:09:57 smithi103 ceph-mon[36243]: pgmap v269: 97 pgs: 97 active+clean; 4.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:10:00.086 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:09:59 smithi103 ceph-mon[36243]: pgmap v270: 97 pgs: 97 active+clean; 4.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:10:00.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:09:59 smithi098 ceph-mon[36516]: pgmap v270: 97 pgs: 97 active+clean; 4.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:10:00.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:09:59 smithi047 ceph-mon[32223]: pgmap v270: 97 pgs: 97 active+clean; 4.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:10:01.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:10:00 smithi098 ceph-mon[36516]: overall HEALTH_OK 2024-04-17T06:10:01.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:10:00 smithi047 ceph-mon[32223]: overall HEALTH_OK 2024-04-17T06:10:01.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:10:00 smithi103 ceph-mon[36243]: overall HEALTH_OK 2024-04-17T06:10:01.745 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.744+0000 7f2ce850e640 1 -- 172.21.15.47:0/2968238235 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2ce006b4f0 msgr2=0x7f2ce0106c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:01.745 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.744+0000 7f2ce850e640 1 --2- 172.21.15.47:0/2968238235 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2ce006b4f0 0x7f2ce0106c20 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f2cd4009900 tx=0x7f2cd402bc30 comp rx=0 tx=0).stop 2024-04-17T06:10:01.745 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.744+0000 7f2ce850e640 1 -- 172.21.15.47:0/2968238235 shutdown_connections 2024-04-17T06:10:01.745 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.744+0000 7f2ce850e640 1 --2- 172.21.15.47:0/2968238235 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2ce0107300 0x7f2ce010eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:01.745 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.744+0000 7f2ce850e640 1 --2- 172.21.15.47:0/2968238235 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2ce006b4f0 0x7f2ce0106c20 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:01.745 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.744+0000 7f2ce850e640 1 --2- 172.21.15.47:0/2968238235 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2ce006ab40 0x7f2ce006af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:01.746 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.744+0000 7f2ce850e640 1 -- 172.21.15.47:0/2968238235 >> 172.21.15.47:0/2968238235 conn(0x7f2ce00771a0 msgr2=0x7f2ce00775b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:01.746 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.744+0000 7f2ce850e640 1 -- 172.21.15.47:0/2968238235 shutdown_connections 2024-04-17T06:10:01.746 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.744+0000 7f2ce850e640 1 -- 172.21.15.47:0/2968238235 wait complete. 2024-04-17T06:10:01.746 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.745+0000 7f2ce850e640 1 Processor -- start 2024-04-17T06:10:01.746 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.745+0000 7f2ce850e640 1 -- start start 2024-04-17T06:10:01.746 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.745+0000 7f2ce850e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2ce006ab40 0x7f2ce0198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:01.746 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.745+0000 7f2ce850e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2ce0107300 0x7f2ce0198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:01.746 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.745+0000 7f2ce850e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2ce019d010 0x7f2ce019d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:01.746 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.745+0000 7f2ce850e640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce0112310 con 0x7f2ce0107300 2024-04-17T06:10:01.746 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.745+0000 7f2ce850e640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce0112190 con 0x7f2ce006ab40 2024-04-17T06:10:01.746 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.745+0000 7f2ce850e640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce0112490 con 0x7f2ce019d010 2024-04-17T06:10:01.746 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.745+0000 7f2ce5aa9640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2ce0107300 0x7f2ce0198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:01.747 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.745+0000 7f2ce5aa9640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2ce0107300 0x7f2ce0198c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49296/0 (socket says 172.21.15.47:49296) 2024-04-17T06:10:01.747 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.745+0000 7f2ce5aa9640 1 -- 172.21.15.47:0/3954651026 learned_addr learned my addr 172.21.15.47:0/3954651026 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:10:01.747 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.745+0000 7f2ce6aab640 1 --2- 172.21.15.47:0/3954651026 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2ce019d010 0x7f2ce019d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:01.747 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.745+0000 7f2ce62aa640 1 --2- 172.21.15.47:0/3954651026 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2ce006ab40 0x7f2ce0198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:01.747 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.745+0000 7f2ce5aa9640 1 -- 172.21.15.47:0/3954651026 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2ce019d010 msgr2=0x7f2ce019d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:01.747 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.745+0000 7f2ce5aa9640 1 --2- 172.21.15.47:0/3954651026 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2ce019d010 0x7f2ce019d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:01.747 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.745+0000 7f2ce5aa9640 1 -- 172.21.15.47:0/3954651026 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2ce006ab40 msgr2=0x7f2ce0198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:01.747 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.745+0000 7f2ce5aa9640 1 --2- 172.21.15.47:0/3954651026 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2ce006ab40 0x7f2ce0198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:01.747 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.745+0000 7f2ce5aa9640 1 -- 172.21.15.47:0/3954651026 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ce019db00 con 0x7f2ce0107300 2024-04-17T06:10:01.747 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.745+0000 7f2ce62aa640 1 --2- 172.21.15.47:0/3954651026 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2ce006ab40 0x7f2ce0198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:10:01.747 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.746+0000 7f2ce5aa9640 1 --2- 172.21.15.47:0/3954651026 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2ce0107300 0x7f2ce0198c90 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f2cd4036040 tx=0x7f2cd402cb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:01.747 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.746+0000 7f2cdf7fe640 1 -- 172.21.15.47:0/3954651026 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd4041070 con 0x7f2ce0107300 2024-04-17T06:10:01.747 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.746+0000 7f2cdf7fe640 1 -- 172.21.15.47:0/3954651026 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2cd4004450 con 0x7f2ce0107300 2024-04-17T06:10:01.747 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.746+0000 7f2ce850e640 1 -- 172.21.15.47:0/3954651026 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ce019dd90 con 0x7f2ce0107300 2024-04-17T06:10:01.748 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.746+0000 7f2cdf7fe640 1 -- 172.21.15.47:0/3954651026 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd4003d90 con 0x7f2ce0107300 2024-04-17T06:10:01.748 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.746+0000 7f2ce850e640 1 -- 172.21.15.47:0/3954651026 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ce01a5550 con 0x7f2ce0107300 2024-04-17T06:10:01.753 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.748+0000 7f2cdf7fe640 1 -- 172.21.15.47:0/3954651026 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2cd4044070 con 0x7f2ce0107300 2024-04-17T06:10:01.753 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.749+0000 7f2ceb1d1740 1 Processor -- start 2024-04-17T06:10:01.753 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.749+0000 7f2ceb1d1740 1 -- start start 2024-04-17T06:10:01.753 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.749+0000 7f2ceb1d1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5642b3167ba0 0x5642b3167f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:01.753 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.749+0000 7f2ceb1d1740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5642b31684c0 0x5642b3175680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:01.753 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.749+0000 7f2ceb1d1740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5642b3175bc0 0x5642b3177fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:01.754 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.749+0000 7f2ceb1d1740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5642b30cd7d0 con 0x5642b31684c0 2024-04-17T06:10:01.754 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.749+0000 7f2ceb1d1740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5642b306af70 con 0x5642b3175bc0 2024-04-17T06:10:01.754 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.749+0000 7f2ceb1d1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5642b30c9790 con 0x5642b3167ba0 2024-04-17T06:10:01.754 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.749+0000 7f2cdf7fe640 1 --2- 172.21.15.47:0/3954651026 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f2cc80a3dc0 0x7f2cc80a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:01.754 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.750+0000 7f2ce62aa640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5642b31684c0 0x5642b3175680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:01.754 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.750+0000 7f2ce62aa640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5642b31684c0 0x5642b3175680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49298/0 (socket says 172.21.15.47:49298) 2024-04-17T06:10:01.754 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.750+0000 7f2ce62aa640 1 -- 172.21.15.47:0/968470784 learned_addr learned my addr 172.21.15.47:0/968470784 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:10:01.754 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.750+0000 7f2ce6aab640 1 --2- 172.21.15.47:0/968470784 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5642b3167ba0 0x5642b3167f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:01.754 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.750+0000 7f2ce6aab640 1 --2- 172.21.15.47:0/3954651026 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f2cc80a3dc0 0x7f2cc80a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:01.754 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.750+0000 7f2cdf7fe640 1 -- 172.21.15.47:0/3954651026 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8513+0+0 (secure 0 0 0) 0x7f2cd40f21f0 con 0x7f2ce0107300 2024-04-17T06:10:01.754 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.750+0000 7f2ce5aa9640 1 --2- 172.21.15.47:0/968470784 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5642b3175bc0 0x5642b3177fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:01.754 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.750+0000 7f2ce62aa640 1 -- 172.21.15.47:0/968470784 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5642b3167ba0 msgr2=0x5642b3167f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:01.755 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.750+0000 7f2ce62aa640 1 --2- 172.21.15.47:0/968470784 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5642b3167ba0 0x5642b3167f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:01.755 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.750+0000 7f2ce62aa640 1 -- 172.21.15.47:0/968470784 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5642b3175bc0 msgr2=0x5642b3177fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:01.755 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.750+0000 7f2ce62aa640 1 --2- 172.21.15.47:0/968470784 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5642b3175bc0 0x5642b3177fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:01.755 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.750+0000 7f2ce62aa640 1 -- 172.21.15.47:0/968470784 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642b2de0210 con 0x5642b31684c0 2024-04-17T06:10:01.755 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.750+0000 7f2ce6aab640 1 --2- 172.21.15.47:0/3954651026 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f2cc80a3dc0 0x7f2cc80a6280 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2cd0005fd0 tx=0x7f2cd0009500 comp rx=0 tx=0).ready entity=mgr.24923 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:01.755 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.750+0000 7f2ce62aa640 1 --2- 172.21.15.47:0/968470784 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5642b31684c0 0x5642b3175680 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f2cd8009d50 tx=0x7f2cd80117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7456d548107a7b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:01.755 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.750+0000 7f2cdcbfe640 1 -- 172.21.15.47:0/968470784 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd8012450 con 0x5642b31684c0 2024-04-17T06:10:01.755 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.750+0000 7f2cdcbfe640 1 -- 172.21.15.47:0/968470784 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2cd8012ad0 con 0x5642b31684c0 2024-04-17T06:10:01.755 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.751+0000 7f2cdcbfe640 1 -- 172.21.15.47:0/968470784 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd800fc00 con 0x5642b31684c0 2024-04-17T06:10:01.755 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.751+0000 7f2ceb1d1740 1 -- 172.21.15.47:0/968470784 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5642b31684c0 msgr2=0x5642b3175680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:01.755 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.751+0000 7f2ceb1d1740 1 --2- 172.21.15.47:0/968470784 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5642b31684c0 0x5642b3175680 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f2cd8009d50 tx=0x7f2cd80117b0 comp rx=0 tx=0).stop 2024-04-17T06:10:01.755 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.751+0000 7f2ceb1d1740 1 -- 172.21.15.47:0/968470784 shutdown_connections 2024-04-17T06:10:01.755 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.751+0000 7f2ceb1d1740 1 --2- 172.21.15.47:0/968470784 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5642b3175bc0 0x5642b3177fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:01.755 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.751+0000 7f2ceb1d1740 1 --2- 172.21.15.47:0/968470784 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5642b31684c0 0x5642b3175680 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:01.756 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.751+0000 7f2ceb1d1740 1 --2- 172.21.15.47:0/968470784 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5642b3167ba0 0x5642b3167f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:01.756 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.751+0000 7f2ceb1d1740 1 -- 172.21.15.47:0/968470784 >> 172.21.15.47:0/968470784 conn(0x5642b31629e0 msgr2=0x5642b3162dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:01.756 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.751+0000 7f2ceb1d1740 1 -- 172.21.15.47:0/968470784 shutdown_connections 2024-04-17T06:10:01.756 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.751+0000 7f2ceb1d1740 1 -- 172.21.15.47:0/968470784 wait complete. 2024-04-17T06:10:01.756 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.752+0000 7f2ceb1d1740 1 -- start start 2024-04-17T06:10:01.756 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.752+0000 7f2ceb1d1740 1 Processor -- start 2024-04-17T06:10:01.756 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.752+0000 7f2ceb1d1740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5642b3167ba0 0x5642b3172bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:01.756 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.752+0000 7f2ceb1d1740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5642b31684c0 0x5642b31730f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:01.756 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.752+0000 7f2ceb1d1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5642b3175bc0 0x5642b3173630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:01.756 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.752+0000 7f2ceb1d1740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5642b3178790 con 0x5642b3167ba0 2024-04-17T06:10:01.756 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.752+0000 7f2ceb1d1740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5642b3055e90 con 0x5642b31684c0 2024-04-17T06:10:01.756 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.752+0000 7f2ceb1d1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5642b3178910 con 0x5642b3175bc0 2024-04-17T06:10:01.757 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.752+0000 7f2ceb1d1740 10 client.0 _ll_register_callbacks cb 0x5642b306bd40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:10:01.757 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.752+0000 7f2ce62aa640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5642b3167ba0 0x5642b3172bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:01.757 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.752+0000 7f2ce62aa640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5642b3167ba0 0x5642b3172bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49306/0 (socket says 172.21.15.47:49306) 2024-04-17T06:10:01.757 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.752+0000 7f2ce62aa640 1 -- 172.21.15.47:0/87065822 learned_addr learned my addr 172.21.15.47:0/87065822 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:10:01.757 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.752+0000 7f2ce5aa9640 1 --2- 172.21.15.47:0/87065822 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5642b31684c0 0x5642b31730f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:01.757 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.752+0000 7f2ce6aab640 1 --2- 172.21.15.47:0/87065822 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5642b3175bc0 0x5642b3173630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:01.757 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.752+0000 7f2ce62aa640 1 -- 172.21.15.47:0/87065822 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5642b3175bc0 msgr2=0x5642b3173630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:01.757 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.753+0000 7f2ce62aa640 1 --2- 172.21.15.47:0/87065822 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5642b3175bc0 0x5642b3173630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:01.757 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.753+0000 7f2ce62aa640 1 -- 172.21.15.47:0/87065822 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5642b31684c0 msgr2=0x5642b31730f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:01.757 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.753+0000 7f2ce62aa640 1 --2- 172.21.15.47:0/87065822 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5642b31684c0 0x5642b31730f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:01.757 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.753+0000 7f2ce62aa640 1 -- 172.21.15.47:0/87065822 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642b2eac620 con 0x5642b3167ba0 2024-04-17T06:10:01.757 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.753+0000 7f2ce5aa9640 1 --2- 172.21.15.47:0/87065822 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5642b31684c0 0x5642b31730f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:10:01.757 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.753+0000 7f2ce6aab640 1 --2- 172.21.15.47:0/87065822 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5642b3175bc0 0x5642b3173630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:10:01.757 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.753+0000 7f2ce62aa640 1 --2- 172.21.15.47:0/87065822 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5642b3167ba0 0x5642b3172bb0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f2cd8011cb0 tx=0x7f2cd8009ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b9c8020b0ce5aed server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:01.757 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.753+0000 7f2cc5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-17T06:10:01.758 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.753+0000 7f2cc5ffb640 1 -- 172.21.15.47:0/87065822 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd8028030 con 0x5642b3167ba0 2024-04-17T06:10:01.758 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.753+0000 7f2cc5ffb640 1 -- 172.21.15.47:0/87065822 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8513+0+0 (secure 0 0 0) 0x7f2cd8004250 con 0x5642b3167ba0 2024-04-17T06:10:01.758 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.753+0000 7f2ceb1d1740 1 -- 172.21.15.47:0/87065822 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2cd8012ad0 con 0x5642b3167ba0 2024-04-17T06:10:01.758 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.753+0000 7f2cc5ffb640 1 -- 172.21.15.47:0/87065822 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f2cd8032050 con 0x5642b3167ba0 2024-04-17T06:10:01.758 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.753+0000 7f2ceb1d1740 10 client.16128 fetch_fsmap learned FSMap version 32 2024-04-17T06:10:01.758 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.753+0000 7f2ceb1d1740 1 -- 172.21.15.47:0/87065822 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5642b3184a20 con 0x5642b3167ba0 2024-04-17T06:10:01.758 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.753+0000 7f2cc5ffb640 1 -- 172.21.15.47:0/87065822 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2cd8012ad0 con 0x5642b3167ba0 2024-04-17T06:10:01.758 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.753+0000 7f2cc5ffb640 1 -- 172.21.15.47:0/87065822 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) ==== 8339+0+0 (secure 0 0 0) 0x7f2cd80035f0 con 0x5642b3167ba0 2024-04-17T06:10:01.758 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.754+0000 7f2ceb1d1740 10 client.16128 fetch_fsmap finished waiting for FSMap version 32 2024-04-17T06:10:01.758 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.754+0000 7f2ceb1d1740 10 client.16128 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-17T06:10:01.758 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.754+0000 7f2ceb1d1740 20 client.16128 populate_metadata read hostname 'smithi047' 2024-04-17T06:10:01.758 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.754+0000 7f2ceb1d1740 1 --2- 172.21.15.47:0/87065822 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x5642b3185ee0 0x5642b31882d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:01.758 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.754+0000 7f2ceb1d1740 4 client.16128 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T06:10:01.759 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.754+0000 7f2ceb1d1740 1 -- 172.21.15.47:0/87065822 --> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5642b30db1f0 con 0x5642b3185ee0 2024-04-17T06:10:01.759 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.754+0000 7f2ce5aa9640 1 --2- 172.21.15.47:0/87065822 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x5642b3185ee0 0x5642b31882d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:01.759 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.754+0000 7f2cc5ffb640 10 client.16128 ms_handle_connect on v2:172.21.15.98:6838/2841287917 2024-04-17T06:10:01.759 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.754+0000 7f2ce5aa9640 1 --2- 172.21.15.47:0/87065822 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x5642b3185ee0 0x5642b31882d0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=861e59754c8f0fef server_cookie=baee7d0d12f099b1 in_seq=0 out_seq=0 2024-04-17T06:10:01.760 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.755+0000 7f2cc5ffb640 1 -- 172.21.15.47:0/87065822 <== mds.0 v2:172.21.15.98:6838/2841287917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5642b30db1f0 con 0x5642b3185ee0 2024-04-17T06:10:01.760 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.755+0000 7f2cc5ffb640 10 client.16128 handle_command_reply: tid=0 2024-04-17T06:10:01.760 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.755+0000 7f2ceb1d1740 1 client.16128 shutdown 2024-04-17T06:10:01.760 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.755+0000 7f2ceb1d1740 1 -- 172.21.15.47:0/87065822 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5642b3167ba0 msgr2=0x5642b3172bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:01.760 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.755+0000 7f2ceb1d1740 1 --2- 172.21.15.47:0/87065822 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5642b3167ba0 0x5642b3172bb0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f2cd8011cb0 tx=0x7f2cd8009ed0 comp rx=0 tx=0).stop 2024-04-17T06:10:01.760 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.755+0000 7f2ceb1d1740 1 -- 172.21.15.47:0/87065822 shutdown_connections 2024-04-17T06:10:01.760 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.755+0000 7f2ceb1d1740 1 --2- 172.21.15.47:0/87065822 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x5642b3185ee0 0x5642b31882d0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:01.760 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.755+0000 7f2ceb1d1740 1 --2- 172.21.15.47:0/87065822 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5642b3175bc0 0x5642b3173630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:01.760 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.755+0000 7f2ceb1d1740 1 --2- 172.21.15.47:0/87065822 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5642b31684c0 0x5642b31730f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:01.761 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.755+0000 7f2cc5ffb640 0 client.16128 ms_handle_reset on v2:172.21.15.98:6838/2841287917 2024-04-17T06:10:01.761 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.755+0000 7f2ceb1d1740 1 --2- 172.21.15.47:0/87065822 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5642b3167ba0 0x5642b3172bb0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:01.761 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.755+0000 7f2ceb1d1740 1 -- 172.21.15.47:0/87065822 >> 172.21.15.47:0/87065822 conn(0x5642b31629e0 msgr2=0x5642b3164730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:01.761 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.755+0000 7f2ceb1d1740 1 -- 172.21.15.47:0/87065822 shutdown_connections 2024-04-17T06:10:01.761 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.755+0000 7f2ceb1d1740 1 -- 172.21.15.47:0/87065822 wait complete. 2024-04-17T06:10:01.761 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.755+0000 7f2ceb1d1740 20 client.16128 trim_cache size 0 max 16384 2024-04-17T06:10:01.774 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.770+0000 7f2ceb1d1740 1 Processor -- start 2024-04-17T06:10:01.774 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.771+0000 7f2ceb1d1740 1 -- start start 2024-04-17T06:10:01.774 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.771+0000 7f2ceb1d1740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5642b3167ba0 0x5642b31752e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:01.774 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.771+0000 7f2ceb1d1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5642b31684c0 0x5642b3181690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:01.774 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.771+0000 7f2ceb1d1740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5642b3175bc0 0x5642b31644d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:01.774 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.771+0000 7f2ceb1d1740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce01ae2b0 con 0x5642b3167ba0 2024-04-17T06:10:01.774 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.771+0000 7f2ceb1d1740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce01abb70 con 0x5642b3175bc0 2024-04-17T06:10:01.774 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.771+0000 7f2ceb1d1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce01a9a00 con 0x5642b31684c0 2024-04-17T06:10:01.774 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.771+0000 7f2ce62aa640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5642b3167ba0 0x5642b31752e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:01.775 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.771+0000 7f2ce62aa640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5642b3167ba0 0x5642b31752e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49310/0 (socket says 172.21.15.47:49310) 2024-04-17T06:10:01.775 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.771+0000 7f2ce62aa640 1 -- 172.21.15.47:0/467441584 learned_addr learned my addr 172.21.15.47:0/467441584 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:10:01.775 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.771+0000 7f2ce6aab640 1 --2- 172.21.15.47:0/467441584 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5642b3175bc0 0x5642b31644d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:01.775 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.771+0000 7f2ce62aa640 1 -- 172.21.15.47:0/467441584 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5642b31684c0 msgr2=0x5642b3181690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:01.775 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.771+0000 7f2ce62aa640 1 --2- 172.21.15.47:0/467441584 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5642b31684c0 0x5642b3181690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:01.775 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.771+0000 7f2ce62aa640 1 -- 172.21.15.47:0/467441584 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5642b3175bc0 msgr2=0x5642b31644d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:01.775 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.771+0000 7f2ce62aa640 1 --2- 172.21.15.47:0/467441584 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5642b3175bc0 0x5642b31644d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:01.775 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.771+0000 7f2ce62aa640 1 -- 172.21.15.47:0/467441584 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642b31682f0 con 0x5642b3167ba0 2024-04-17T06:10:01.775 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.771+0000 7f2ce6aab640 1 --2- 172.21.15.47:0/467441584 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5642b3175bc0 0x5642b31644d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:10:01.775 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.771+0000 7f2ce62aa640 1 --2- 172.21.15.47:0/467441584 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5642b3167ba0 0x5642b31752e0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f2cd8012fd0 tx=0x7f2cd8011d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c25afe323e64684d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:01.775 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.771+0000 7f2cc7fff640 1 -- 172.21.15.47:0/467441584 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd8028070 con 0x5642b3167ba0 2024-04-17T06:10:01.775 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.772+0000 7f2cc7fff640 1 -- 172.21.15.47:0/467441584 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2cd8029040 con 0x5642b3167ba0 2024-04-17T06:10:01.775 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.772+0000 7f2cc7fff640 1 -- 172.21.15.47:0/467441584 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd8005530 con 0x5642b3167ba0 2024-04-17T06:10:01.775 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.772+0000 7f2ceb1d1740 1 -- 172.21.15.47:0/467441584 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5642b3167ba0 msgr2=0x5642b31752e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:01.775 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.772+0000 7f2ceb1d1740 1 --2- 172.21.15.47:0/467441584 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5642b3167ba0 0x5642b31752e0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f2cd8012fd0 tx=0x7f2cd8011d60 comp rx=0 tx=0).stop 2024-04-17T06:10:01.776 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.772+0000 7f2ceb1d1740 1 -- 172.21.15.47:0/467441584 shutdown_connections 2024-04-17T06:10:01.776 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.772+0000 7f2ceb1d1740 1 --2- 172.21.15.47:0/467441584 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5642b3175bc0 0x5642b31644d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:01.776 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.772+0000 7f2ceb1d1740 1 --2- 172.21.15.47:0/467441584 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5642b31684c0 0x5642b3181690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:01.776 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.772+0000 7f2ceb1d1740 1 --2- 172.21.15.47:0/467441584 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5642b3167ba0 0x5642b31752e0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:01.776 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.772+0000 7f2ceb1d1740 1 -- 172.21.15.47:0/467441584 >> 172.21.15.47:0/467441584 conn(0x5642b31629e0 msgr2=0x5642b3163dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:01.776 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.772+0000 7f2ceb1d1740 1 -- 172.21.15.47:0/467441584 shutdown_connections 2024-04-17T06:10:01.776 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.772+0000 7f2ceb1d1740 1 -- 172.21.15.47:0/467441584 wait complete. 2024-04-17T06:10:01.776 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.772+0000 7f2ceb1d1740 1 -- start start 2024-04-17T06:10:01.776 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.772+0000 7f2ceb1d1740 1 Processor -- start 2024-04-17T06:10:01.776 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.773+0000 7f2ceb1d1740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5642b3167ba0 0x5642b3179660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:01.776 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.773+0000 7f2ceb1d1740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5642b31684c0 0x5642b3179ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:01.776 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.773+0000 7f2ceb1d1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5642b3175bc0 0x5642b317a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:01.776 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.773+0000 7f2ceb1d1740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce01aa050 con 0x5642b31684c0 2024-04-17T06:10:01.776 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.773+0000 7f2ceb1d1740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce01ab4a0 con 0x5642b3167ba0 2024-04-17T06:10:01.776 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.773+0000 7f2ceb1d1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce01aa700 con 0x5642b3175bc0 2024-04-17T06:10:01.776 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.773+0000 7f2ceb1d1740 10 client.0 _ll_register_callbacks cb 0x7f2cd0010670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:10:01.777 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.773+0000 7f2ce5aa9640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5642b31684c0 0x5642b3179ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:01.777 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.773+0000 7f2ce5aa9640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5642b31684c0 0x5642b3179ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49326/0 (socket says 172.21.15.47:49326) 2024-04-17T06:10:01.777 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.773+0000 7f2ce5aa9640 1 -- 172.21.15.47:0/1901014354 learned_addr learned my addr 172.21.15.47:0/1901014354 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:10:01.777 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.773+0000 7f2ce62aa640 1 --2- 172.21.15.47:0/1901014354 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5642b3167ba0 0x5642b3179660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:01.777 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.773+0000 7f2ce6aab640 1 --2- 172.21.15.47:0/1901014354 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5642b3175bc0 0x5642b317a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:01.777 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.773+0000 7f2ce5aa9640 1 -- 172.21.15.47:0/1901014354 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5642b3175bc0 msgr2=0x5642b317a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:01.777 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.773+0000 7f2ce5aa9640 1 --2- 172.21.15.47:0/1901014354 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5642b3175bc0 0x5642b317a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:01.777 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.773+0000 7f2ce5aa9640 1 -- 172.21.15.47:0/1901014354 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5642b3167ba0 msgr2=0x5642b3179660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:01.777 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.773+0000 7f2ce5aa9640 1 --2- 172.21.15.47:0/1901014354 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5642b3167ba0 0x5642b3179660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:01.777 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.773+0000 7f2ce5aa9640 1 -- 172.21.15.47:0/1901014354 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642b31eb220 con 0x5642b31684c0 2024-04-17T06:10:01.777 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.773+0000 7f2ce6aab640 1 --2- 172.21.15.47:0/1901014354 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5642b3175bc0 0x5642b317a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:10:01.777 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.773+0000 7f2ce5aa9640 1 --2- 172.21.15.47:0/1901014354 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5642b31684c0 0x5642b3179ba0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f2cd402cfd0 tx=0x7f2cd40bea60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4218e2eed752eed6 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:01.777 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.773+0000 7f2cc5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-17T06:10:01.778 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.774+0000 7f2cc5ffb640 1 -- 172.21.15.47:0/1901014354 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd40a7020 con 0x5642b31684c0 2024-04-17T06:10:01.778 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.774+0000 7f2cc5ffb640 1 -- 172.21.15.47:0/1901014354 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8513+0+0 (secure 0 0 0) 0x7f2cd40035f0 con 0x5642b31684c0 2024-04-17T06:10:01.778 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.774+0000 7f2ceb1d1740 1 -- 172.21.15.47:0/1901014354 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2cd8029040 con 0x5642b31684c0 2024-04-17T06:10:01.778 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.774+0000 7f2cc5ffb640 1 -- 172.21.15.47:0/1901014354 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f2cd404b040 con 0x5642b31684c0 2024-04-17T06:10:01.778 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.774+0000 7f2ceb1d1740 10 client.16134 fetch_fsmap learned FSMap version 32 2024-04-17T06:10:01.778 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.774+0000 7f2ceb1d1740 1 -- 172.21.15.47:0/1901014354 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5642b31ec4e0 con 0x5642b31684c0 2024-04-17T06:10:01.778 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.774+0000 7f2cc5ffb640 1 -- 172.21.15.47:0/1901014354 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2cd8029040 con 0x5642b31684c0 2024-04-17T06:10:01.778 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.774+0000 7f2cc5ffb640 1 -- 172.21.15.47:0/1901014354 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) ==== 8339+0+0 (secure 0 0 0) 0x7f2cd40ba910 con 0x5642b31684c0 2024-04-17T06:10:01.778 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.774+0000 7f2ceb1d1740 10 client.16134 fetch_fsmap finished waiting for FSMap version 32 2024-04-17T06:10:01.778 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.774+0000 7f2ceb1d1740 10 client.16134 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-17T06:10:01.778 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.774+0000 7f2ceb1d1740 20 client.16134 populate_metadata read hostname 'smithi047' 2024-04-17T06:10:01.778 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.774+0000 7f2ceb1d1740 1 --2- 172.21.15.47:0/1901014354 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x5642b31ecc90 0x5642b31ef0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:01.778 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.774+0000 7f2ceb1d1740 4 client.16134 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-17T06:10:01.779 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.774+0000 7f2ceb1d1740 1 -- 172.21.15.47:0/1901014354 --> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5642b30d5060 con 0x5642b31ecc90 2024-04-17T06:10:01.779 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.774+0000 7f2ce62aa640 1 --2- 172.21.15.47:0/1901014354 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x5642b31ecc90 0x5642b31ef0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:01.779 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.775+0000 7f2cc5ffb640 10 client.16134 ms_handle_connect on v2:172.21.15.98:6838/2841287917 2024-04-17T06:10:01.779 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:01.775+0000 7f2ce62aa640 1 --2- 172.21.15.47:0/1901014354 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x5642b31ecc90 0x5642b31ef0b0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f769c103225a656f server_cookie=62b9401ed55dd1d3 in_seq=0 out_seq=0 2024-04-17T06:10:02.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:10:01 smithi098 ceph-mon[36516]: pgmap v271: 97 pgs: 97 active+clean; 4.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-04-17T06:10:02.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:10:01 smithi047 ceph-mon[32223]: pgmap v271: 97 pgs: 97 active+clean; 4.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-04-17T06:10:02.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:10:01 smithi103 ceph-mon[36243]: pgmap v271: 97 pgs: 97 active+clean; 4.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-04-17T06:10:03.536 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:03.536+0000 7f2cc5ffb640 1 -- 172.21.15.47:0/1901014354 <== mds.0 v2:172.21.15.98:6838/2841287917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5642b30d5060 con 0x5642b31ecc90 2024-04-17T06:10:03.536 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:03.536+0000 7f2cc5ffb640 10 client.16134 handle_command_reply: tid=0 2024-04-17T06:10:03.536 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:03.536+0000 7f2ceb1d1740 1 client.16134 shutdown 2024-04-17T06:10:03.536 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:03.537+0000 7f2ceb1d1740 1 -- 172.21.15.47:0/1901014354 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5642b31684c0 msgr2=0x5642b3179ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:03.536 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:03.537+0000 7f2ceb1d1740 1 --2- 172.21.15.47:0/1901014354 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5642b31684c0 0x5642b3179ba0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f2cd402cfd0 tx=0x7f2cd40bea60 comp rx=0 tx=0).stop 2024-04-17T06:10:03.536 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:03.537+0000 7f2ceb1d1740 1 -- 172.21.15.47:0/1901014354 shutdown_connections 2024-04-17T06:10:03.536 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:03.537+0000 7f2ceb1d1740 1 --2- 172.21.15.47:0/1901014354 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x5642b31ecc90 0x5642b31ef0b0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:03.536 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:03.537+0000 7f2ceb1d1740 1 --2- 172.21.15.47:0/1901014354 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5642b3175bc0 0x5642b317a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:03.537 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:03.537+0000 7f2ceb1d1740 1 --2- 172.21.15.47:0/1901014354 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5642b31684c0 0x5642b3179ba0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:03.537 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:03.537+0000 7f2ceb1d1740 1 --2- 172.21.15.47:0/1901014354 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5642b3167ba0 0x5642b3179660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:03.537 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:03.537+0000 7f2cc5ffb640 0 client.16134 ms_handle_reset on v2:172.21.15.98:6838/2841287917 2024-04-17T06:10:03.537 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:03.537+0000 7f2ceb1d1740 1 -- 172.21.15.47:0/1901014354 >> 172.21.15.47:0/1901014354 conn(0x5642b31629e0 msgr2=0x5642b3161b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:03.537 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:03.537+0000 7f2ceb1d1740 1 -- 172.21.15.47:0/1901014354 shutdown_connections 2024-04-17T06:10:03.537 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:03.537+0000 7f2ceb1d1740 1 -- 172.21.15.47:0/1901014354 wait complete. 2024-04-17T06:10:03.537 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:03.537+0000 7f2ceb1d1740 20 client.16134 trim_cache size 0 max 16384 2024-04-17T06:10:03.537 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-04-17T06:10:03.537 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-04-17T06:10:03.537 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-04-17T06:10:03.537 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-04-17T06:10:03.538 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:03.538+0000 7f2ce850e640 1 -- 172.21.15.47:0/3954651026 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f2cc80a3dc0 msgr2=0x7f2cc80a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:03.538 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:03.538+0000 7f2ce850e640 1 --2- 172.21.15.47:0/3954651026 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f2cc80a3dc0 0x7f2cc80a6280 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2cd0005fd0 tx=0x7f2cd0009500 comp rx=0 tx=0).stop 2024-04-17T06:10:03.538 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:03.538+0000 7f2ce850e640 1 -- 172.21.15.47:0/3954651026 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2ce0107300 msgr2=0x7f2ce0198c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:03.538 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:03.538+0000 7f2ce850e640 1 --2- 172.21.15.47:0/3954651026 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2ce0107300 0x7f2ce0198c90 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f2cd4036040 tx=0x7f2cd402cb20 comp rx=0 tx=0).stop 2024-04-17T06:10:03.539 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:03.538+0000 7f2ce850e640 1 -- 172.21.15.47:0/3954651026 shutdown_connections 2024-04-17T06:10:03.539 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:03.538+0000 7f2ce850e640 1 --2- 172.21.15.47:0/3954651026 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2ce019d010 0x7f2ce019d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:03.539 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:03.538+0000 7f2ce850e640 1 --2- 172.21.15.47:0/3954651026 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f2cc80a3dc0 0x7f2cc80a6280 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:03.539 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:03.538+0000 7f2ce850e640 1 --2- 172.21.15.47:0/3954651026 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2ce0107300 0x7f2ce0198c90 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:03.539 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:03.538+0000 7f2ce850e640 1 --2- 172.21.15.47:0/3954651026 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2ce006ab40 0x7f2ce0198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:03.539 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:03.538+0000 7f2ce850e640 1 -- 172.21.15.47:0/3954651026 >> 172.21.15.47:0/3954651026 conn(0x7f2ce00771a0 msgr2=0x7f2ce00fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:03.539 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:03.538+0000 7f2ce850e640 1 -- 172.21.15.47:0/3954651026 shutdown_connections 2024-04-17T06:10:03.539 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:03.538+0000 7f2ce850e640 1 -- 172.21.15.47:0/3954651026 wait complete. 2024-04-17T06:10:04.080 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:10:03 smithi098 ceph-mon[36516]: pgmap v272: 97 pgs: 97 active+clean; 4.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.3 KiB/s wr, 9 op/s 2024-04-17T06:10:04.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:10:03 smithi047 ceph-mon[32223]: pgmap v272: 97 pgs: 97 active+clean; 4.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.3 KiB/s wr, 9 op/s 2024-04-17T06:10:04.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:10:03 smithi103 ceph-mon[36243]: pgmap v272: 97 pgs: 97 active+clean; 4.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.3 KiB/s wr, 9 op/s 2024-04-17T06:10:04.365 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:10:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[71775]: *** Caught signal (Segmentation fault) ** 2024-04-17T06:10:04.365 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:10:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[71775]: in thread 7f0d8fd01700 thread_name:md_log_replay 2024-04-17T06:10:04.365 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:10:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[71775]: ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev) 2024-04-17T06:10:04.365 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:10:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[71775]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f0da10c2d20] 2024-04-17T06:10:04.365 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:10:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[71775]: 2: /usr/bin/ceph-mds(+0x549612) [0x557e1a00b612] 2024-04-17T06:10:04.365 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:10:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[71775]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x557e1a00edc3] 2024-04-17T06:10:04.365 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:10:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[71775]: 4: (MDLog::_replay_thread()+0x617) [0x557e1a0698e7] 2024-04-17T06:10:04.365 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:10:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[71775]: 5: (MDLog::ReplayThread::entry()+0x11) [0x557e1a072fd1] 2024-04-17T06:10:04.365 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:10:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[71775]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f0da10b81ca] 2024-04-17T06:10:04.365 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:10:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[71775]: 7: clone() 2024-04-17T06:10:04.365 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:10:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[71775]: 2024-04-17T06:10:04.082+0000 7f0d8fd01700 -1 *** Caught signal (Segmentation fault) ** 2024-04-17T06:10:04.366 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:10:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[71775]: in thread 7f0d8fd01700 thread_name:md_log_replay 2024-04-17T06:10:04.366 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:10:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[71775]: 2024-04-17T06:10:04.366 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:10:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[71775]: ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev) 2024-04-17T06:10:04.366 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:10:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[71775]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f0da10c2d20] 2024-04-17T06:10:04.366 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:10:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[71775]: 2: /usr/bin/ceph-mds(+0x549612) [0x557e1a00b612] 2024-04-17T06:10:04.366 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:10:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[71775]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x557e1a00edc3] 2024-04-17T06:10:04.366 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:10:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[71775]: 4: (MDLog::_replay_thread()+0x617) [0x557e1a0698e7] 2024-04-17T06:10:04.366 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:10:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[71775]: 5: (MDLog::ReplayThread::entry()+0x11) [0x557e1a072fd1] 2024-04-17T06:10:04.366 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:10:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[71775]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f0da10b81ca] 2024-04-17T06:10:04.366 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:10:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[71775]: 7: clone() 2024-04-17T06:10:04.366 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:10:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[71775]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-17T06:10:04.366 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:10:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[71775]: 2024-04-17T06:10:04.366 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:10:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[71775]: 0> 2024-04-17T06:10:04.082+0000 7f0d8fd01700 -1 *** Caught signal (Segmentation fault) ** 2024-04-17T06:10:04.366 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:10:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[71775]: in thread 7f0d8fd01700 thread_name:md_log_replay 2024-04-17T06:10:04.367 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:10:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[71775]: 2024-04-17T06:10:04.367 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:10:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[71775]: ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev) 2024-04-17T06:10:04.367 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:10:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[71775]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f0da10c2d20] 2024-04-17T06:10:04.367 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:10:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[71775]: 2: /usr/bin/ceph-mds(+0x549612) [0x557e1a00b612] 2024-04-17T06:10:04.367 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:10:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[71775]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x557e1a00edc3] 2024-04-17T06:10:04.367 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:10:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[71775]: 4: (MDLog::_replay_thread()+0x617) [0x557e1a0698e7] 2024-04-17T06:10:04.367 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:10:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[71775]: 5: (MDLog::ReplayThread::entry()+0x11) [0x557e1a072fd1] 2024-04-17T06:10:04.367 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:10:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[71775]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f0da10b81ca] 2024-04-17T06:10:04.367 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:10:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[71775]: 7: clone() 2024-04-17T06:10:04.367 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:10:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[71775]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-17T06:10:04.367 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:10:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[71775]: 2024-04-17T06:10:04.367 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:10:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[71775]: 0> 2024-04-17T06:10:04.082+0000 7f0d8fd01700 -1 *** Caught signal (Segmentation fault) ** 2024-04-17T06:10:04.367 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:10:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[71775]: in thread 7f0d8fd01700 thread_name:md_log_replay 2024-04-17T06:10:04.367 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:10:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[71775]: 2024-04-17T06:10:04.367 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:10:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[71775]: ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev) 2024-04-17T06:10:04.368 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:10:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[71775]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f0da10c2d20] 2024-04-17T06:10:04.368 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:10:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[71775]: 2: /usr/bin/ceph-mds(+0x549612) [0x557e1a00b612] 2024-04-17T06:10:04.368 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:10:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[71775]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x557e1a00edc3] 2024-04-17T06:10:04.368 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:10:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[71775]: 4: (MDLog::_replay_thread()+0x617) [0x557e1a0698e7] 2024-04-17T06:10:04.368 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:10:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[71775]: 5: (MDLog::ReplayThread::entry()+0x11) [0x557e1a072fd1] 2024-04-17T06:10:04.368 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:10:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[71775]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f0da10b81ca] 2024-04-17T06:10:04.368 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:10:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[71775]: 7: clone() 2024-04-17T06:10:04.368 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:10:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[71775]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-17T06:10:04.368 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:10:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[71775]: 2024-04-17T06:10:04.368 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:10:04 smithi098 podman[72108]: 2024-04-17 06:10:04.13955099 +0000 UTC m=+0.021757913 container died 87698d00c9e06b8a3ba01b7441d686fa32afaca37d49cbb752c0977710c3b739 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS) 2024-04-17T06:10:05.089 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:10:04 smithi098 ceph-mon[36516]: osdmap e97: 12 total, 12 up, 12 in 2024-04-17T06:10:05.089 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:10:04 smithi098 podman[72108]: 2024-04-17 06:10:04.839704538 +0000 UTC m=+0.721911491 container remove 87698d00c9e06b8a3ba01b7441d686fa32afaca37d49cbb752c0977710c3b739 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True) 2024-04-17T06:10:05.089 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:10:04 smithi098 systemd[1]: ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@mds.k.service: Main process exited, code=exited, status=139/n/a 2024-04-17T06:10:05.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:10:04 smithi047 ceph-mon[32223]: osdmap e97: 12 total, 12 up, 12 in 2024-04-17T06:10:05.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:10:04 smithi103 ceph-mon[36243]: osdmap e97: 12 total, 12 up, 12 in 2024-04-17T06:10:05.382 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:10:05 smithi098 systemd[1]: ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@mds.k.service: Failed with result 'exit-code'. 2024-04-17T06:10:06.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:10:05 smithi098 ceph-mon[36516]: pgmap v274: 97 pgs: 5 active+clean+snaptrim_wait, 90 active+clean, 2 active+clean+snaptrim; 4.5 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 68 KiB/s wr, 34 op/s 2024-04-17T06:10:06.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:10:05 smithi047 ceph-mon[32223]: pgmap v274: 97 pgs: 5 active+clean+snaptrim_wait, 90 active+clean, 2 active+clean+snaptrim; 4.5 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 68 KiB/s wr, 34 op/s 2024-04-17T06:10:06.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:10:05 smithi103 ceph-mon[36243]: pgmap v274: 97 pgs: 5 active+clean+snaptrim_wait, 90 active+clean, 2 active+clean+snaptrim; 4.5 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 68 KiB/s wr, 34 op/s 2024-04-17T06:10:08.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:10:07 smithi098 ceph-mon[36516]: pgmap v275: 97 pgs: 5 active+clean+snaptrim_wait, 90 active+clean, 2 active+clean+snaptrim; 4.5 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 68 KiB/s wr, 34 op/s 2024-04-17T06:10:08.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:10:07 smithi047 ceph-mon[32223]: pgmap v275: 97 pgs: 5 active+clean+snaptrim_wait, 90 active+clean, 2 active+clean+snaptrim; 4.5 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 68 KiB/s wr, 34 op/s 2024-04-17T06:10:08.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:10:07 smithi103 ceph-mon[36243]: pgmap v275: 97 pgs: 5 active+clean+snaptrim_wait, 90 active+clean, 2 active+clean+snaptrim; 4.5 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 68 KiB/s wr, 34 op/s 2024-04-17T06:10:10.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:10:09 smithi098 ceph-mon[36516]: pgmap v276: 97 pgs: 5 active+clean+snaptrim_wait, 90 active+clean, 2 active+clean+snaptrim; 4.5 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 68 KiB/s wr, 34 op/s 2024-04-17T06:10:10.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:10:09 smithi047 ceph-mon[32223]: pgmap v276: 97 pgs: 5 active+clean+snaptrim_wait, 90 active+clean, 2 active+clean+snaptrim; 4.5 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 68 KiB/s wr, 34 op/s 2024-04-17T06:10:10.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:10:09 smithi103 ceph-mon[36243]: pgmap v276: 97 pgs: 5 active+clean+snaptrim_wait, 90 active+clean, 2 active+clean+snaptrim; 4.5 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 68 KiB/s wr, 34 op/s 2024-04-17T06:10:11.075 INFO:tasks.workunit.client.0.smithi047.stdout:375/600: open fd = 3 2024-04-17T06:10:11.075 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.075 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.075 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:10:11.076 INFO:tasks.workunit.client.0.smithi047.stdout:376/600: open fd = 3 2024-04-17T06:10:11.076 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.076 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.076 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:10:11.076 INFO:tasks.workunit.client.0.smithi047.stdout:377/600: open fd = 3 2024-04-17T06:10:11.076 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.076 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.076 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:10:11.076 INFO:tasks.workunit.client.0.smithi047.stdout:378/600: open fd = 3 2024-04-17T06:10:11.076 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.076 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.076 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:10:11.076 INFO:tasks.workunit.client.0.smithi047.stdout:379/600: open fd = 3 2024-04-17T06:10:11.076 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.076 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.077 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:10:11.077 INFO:tasks.workunit.client.0.smithi047.stdout:380/600: open fd = 3 2024-04-17T06:10:11.077 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.077 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.077 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:10:11.077 INFO:tasks.workunit.client.0.smithi047.stdout:381/600: open fd = 3 2024-04-17T06:10:11.077 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.077 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.077 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:10:11.077 INFO:tasks.workunit.client.0.smithi047.stdout:382/600: open fd = 3 2024-04-17T06:10:11.077 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.077 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.077 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:10:11.077 INFO:tasks.workunit.client.0.smithi047.stdout:383/600: open fd = 3 2024-04-17T06:10:11.078 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.078 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.078 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:10:11.078 INFO:tasks.workunit.client.0.smithi047.stdout:384/600: open fd = 3 2024-04-17T06:10:11.078 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.078 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.078 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:10:11.078 INFO:tasks.workunit.client.0.smithi047.stdout:385/600: open fd = 3 2024-04-17T06:10:11.078 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.078 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.078 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:10:11.078 INFO:tasks.workunit.client.0.smithi047.stdout:386/600: open fd = 3 2024-04-17T06:10:11.078 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.078 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.079 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:10:11.079 INFO:tasks.workunit.client.0.smithi047.stdout:387/600: open fd = 3 2024-04-17T06:10:11.079 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.079 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.079 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:10:11.079 INFO:tasks.workunit.client.0.smithi047.stdout:388/600: open fd = 3 2024-04-17T06:10:11.079 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.079 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.079 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:10:11.079 INFO:tasks.workunit.client.0.smithi047.stdout:389/600: open fd = 3 2024-04-17T06:10:11.079 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.079 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.079 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:10:11.079 INFO:tasks.workunit.client.0.smithi047.stdout:390/600: open fd = 3 2024-04-17T06:10:11.079 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.080 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.080 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:10:11.080 INFO:tasks.workunit.client.0.smithi047.stdout:391/600: open fd = 3 2024-04-17T06:10:11.080 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.080 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.080 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:10:11.080 INFO:tasks.workunit.client.0.smithi047.stdout:392/600: open fd = 3 2024-04-17T06:10:11.080 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.080 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.080 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:10:11.080 INFO:tasks.workunit.client.0.smithi047.stdout:393/600: open fd = 3 2024-04-17T06:10:11.080 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.080 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.080 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:10:11.080 INFO:tasks.workunit.client.0.smithi047.stdout:394/600: open fd = 3 2024-04-17T06:10:11.081 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.081 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.081 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:10:11.081 INFO:tasks.workunit.client.0.smithi047.stdout:395/600: open fd = 3 2024-04-17T06:10:11.081 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.081 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.081 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:10:11.081 INFO:tasks.workunit.client.0.smithi047.stdout:396/600: open fd = 3 2024-04-17T06:10:11.081 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.081 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.081 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:10:11.081 INFO:tasks.workunit.client.0.smithi047.stdout:397/600: open fd = 3 2024-04-17T06:10:11.081 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.081 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.082 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:10:11.082 INFO:tasks.workunit.client.0.smithi047.stdout:398/600: open fd = 3 2024-04-17T06:10:11.082 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.082 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.082 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:10:11.082 INFO:tasks.workunit.client.0.smithi047.stdout:399/600: open fd = 3 2024-04-17T06:10:11.082 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.082 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.082 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:10:11.082 INFO:tasks.workunit.client.0.smithi047.stdout:400/600: open fd = 3 2024-04-17T06:10:11.082 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.082 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.082 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:10:11.082 INFO:tasks.workunit.client.0.smithi047.stdout:401/600: open fd = 3 2024-04-17T06:10:11.083 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.083 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.083 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:10:11.083 INFO:tasks.workunit.client.0.smithi047.stdout:402/600: open fd = 3 2024-04-17T06:10:11.083 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.083 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.083 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:10:11.083 INFO:tasks.workunit.client.0.smithi047.stdout:403/600: open fd = 3 2024-04-17T06:10:11.083 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.083 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.083 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:10:11.083 INFO:tasks.workunit.client.0.smithi047.stdout:404/600: open fd = 3 2024-04-17T06:10:11.083 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.083 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.083 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:10:11.084 INFO:tasks.workunit.client.0.smithi047.stdout:405/600: open fd = 3 2024-04-17T06:10:11.084 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.084 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.084 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:10:11.084 INFO:tasks.workunit.client.0.smithi047.stdout:406/600: open fd = 3 2024-04-17T06:10:11.084 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.084 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.084 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:10:11.084 INFO:tasks.workunit.client.0.smithi047.stdout:407/600: open fd = 3 2024-04-17T06:10:11.084 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.084 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.084 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:10:11.085 INFO:tasks.workunit.client.0.smithi047.stdout:408/600: open fd = 3 2024-04-17T06:10:11.085 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.085 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.085 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:10:11.085 INFO:tasks.workunit.client.0.smithi047.stdout:409/600: open fd = 3 2024-04-17T06:10:11.085 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.085 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.085 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:10:11.085 INFO:tasks.workunit.client.0.smithi047.stdout:410/600: open fd = 3 2024-04-17T06:10:11.085 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.085 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.085 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:10:11.085 INFO:tasks.workunit.client.0.smithi047.stdout:411/600: open fd = 3 2024-04-17T06:10:11.085 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.086 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.086 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:10:11.086 INFO:tasks.workunit.client.0.smithi047.stdout:412/600: open fd = 3 2024-04-17T06:10:11.086 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.086 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.086 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:10:11.086 INFO:tasks.workunit.client.0.smithi047.stdout:413/600: open fd = 3 2024-04-17T06:10:11.086 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.086 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.086 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:10:11.086 INFO:tasks.workunit.client.0.smithi047.stdout:414/600: open fd = 3 2024-04-17T06:10:11.086 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.086 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.086 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:10:11.086 INFO:tasks.workunit.client.0.smithi047.stdout:415/600: open fd = 3 2024-04-17T06:10:11.087 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.087 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.087 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:10:11.087 INFO:tasks.workunit.client.0.smithi047.stdout:416/600: open fd = 3 2024-04-17T06:10:11.087 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.087 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.087 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:10:11.087 INFO:tasks.workunit.client.0.smithi047.stdout:417/600: open fd = 3 2024-04-17T06:10:11.087 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.087 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.087 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:10:11.087 INFO:tasks.workunit.client.0.smithi047.stdout:418/600: open fd = 3 2024-04-17T06:10:11.087 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.087 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.087 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:10:11.088 INFO:tasks.workunit.client.0.smithi047.stdout:419/600: open fd = 3 2024-04-17T06:10:11.088 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.088 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.088 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:10:11.088 INFO:tasks.workunit.client.0.smithi047.stdout:420/600: open fd = 3 2024-04-17T06:10:11.088 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.088 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.088 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:10:11.088 INFO:tasks.workunit.client.0.smithi047.stdout:421/600: open fd = 3 2024-04-17T06:10:11.088 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.088 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.088 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:10:11.088 INFO:tasks.workunit.client.0.smithi047.stdout:422/600: open fd = 3 2024-04-17T06:10:11.088 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.088 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.089 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:10:11.089 INFO:tasks.workunit.client.0.smithi047.stdout:423/600: open fd = 3 2024-04-17T06:10:11.089 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.089 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.089 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:10:11.089 INFO:tasks.workunit.client.0.smithi047.stdout:424/600: open fd = 3 2024-04-17T06:10:11.089 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.089 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.089 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:10:11.089 INFO:tasks.workunit.client.0.smithi047.stdout:425/600: open fd = 3 2024-04-17T06:10:11.089 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.089 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.089 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:10:11.089 INFO:tasks.workunit.client.0.smithi047.stdout:426/600: open fd = 3 2024-04-17T06:10:11.089 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.089 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.090 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:10:11.090 INFO:tasks.workunit.client.0.smithi047.stdout:427/600: open fd = 3 2024-04-17T06:10:11.090 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.090 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.090 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:10:11.090 INFO:tasks.workunit.client.0.smithi047.stdout:428/600: open fd = 3 2024-04-17T06:10:11.090 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.090 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.090 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:10:11.090 INFO:tasks.workunit.client.0.smithi047.stdout:429/600: open fd = 3 2024-04-17T06:10:11.090 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.090 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.090 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:10:11.090 INFO:tasks.workunit.client.0.smithi047.stdout:430/600: open fd = 3 2024-04-17T06:10:11.090 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.091 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.091 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:10:11.091 INFO:tasks.workunit.client.0.smithi047.stdout:431/600: open fd = 3 2024-04-17T06:10:11.091 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.091 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.091 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:10:11.091 INFO:tasks.workunit.client.0.smithi047.stdout:432/600: open fd = 3 2024-04-17T06:10:11.091 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.091 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.091 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:10:11.091 INFO:tasks.workunit.client.0.smithi047.stdout:433/600: open fd = 3 2024-04-17T06:10:11.091 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.091 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.091 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:10:11.091 INFO:tasks.workunit.client.0.smithi047.stdout:434/600: open fd = 3 2024-04-17T06:10:11.092 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.092 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.092 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:10:11.092 INFO:tasks.workunit.client.0.smithi047.stdout:435/600: open fd = 3 2024-04-17T06:10:11.092 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.092 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.092 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:10:11.092 INFO:tasks.workunit.client.0.smithi047.stdout:436/600: open fd = 3 2024-04-17T06:10:11.092 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.092 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:10:11.092 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:10:12.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:10:11 smithi098 ceph-mon[36516]: pgmap v277: 97 pgs: 97 active+clean; 4.5 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 68 KiB/s wr, 33 op/s 2024-04-17T06:10:12.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:10:11 smithi047 ceph-mon[32223]: pgmap v277: 97 pgs: 97 active+clean; 4.5 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 68 KiB/s wr, 33 op/s 2024-04-17T06:10:12.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:10:11 smithi103 ceph-mon[36243]: pgmap v277: 97 pgs: 97 active+clean; 4.5 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 68 KiB/s wr, 33 op/s 2024-04-17T06:10:14.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:10:13 smithi098 ceph-mon[36516]: pgmap v278: 97 pgs: 97 active+clean; 4.5 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 68 KiB/s wr, 33 op/s 2024-04-17T06:10:14.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:10:13 smithi047 ceph-mon[32223]: pgmap v278: 97 pgs: 97 active+clean; 4.5 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 68 KiB/s wr, 33 op/s 2024-04-17T06:10:14.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:10:13 smithi103 ceph-mon[36243]: pgmap v278: 97 pgs: 97 active+clean; 4.5 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 68 KiB/s wr, 33 op/s 2024-04-17T06:10:15.474 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:10:15 smithi098 systemd[1]: ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@mds.k.service: Scheduled restart job, restart counter is at 2. 2024-04-17T06:10:15.474 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:10:15 smithi098 systemd[1]: Stopped Ceph mds.k for 565db2d8-fc7e-11ee-bc91-c7b262605968. 2024-04-17T06:10:15.474 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:10:15 smithi098 systemd[1]: Starting Ceph mds.k for 565db2d8-fc7e-11ee-bc91-c7b262605968... 2024-04-17T06:10:15.474 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:10:15 smithi098 podman[72208]: 2024-04-17 06:10:15.367111402 +0000 UTC m=+0.108687795 container create b4076f002011592a4842588b1e92ed80dbabe30471f18396a3ad1e5e3aa7912e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-17T06:10:15.475 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:10:15 smithi098 podman[72208]: 2024-04-17 06:10:15.276654722 +0000 UTC m=+0.018231142 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-17T06:10:15.747 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:10:15 smithi098 podman[72208]: 2024-04-17 06:10:15.595354929 +0000 UTC m=+0.336931346 container init b4076f002011592a4842588b1e92ed80dbabe30471f18396a3ad1e5e3aa7912e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-04-17T06:10:15.747 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:10:15 smithi098 podman[72208]: 2024-04-17 06:10:15.598068853 +0000 UTC m=+0.339645257 container start b4076f002011592a4842588b1e92ed80dbabe30471f18396a3ad1e5e3aa7912e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-04-17T06:10:15.747 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:10:15 smithi098 ceph-mds[72228]: -- 172.21.15.98:0/595750332 <== mon.2 v2:172.21.15.98:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c946765a40 con 0x55c946739000 2024-04-17T06:10:15.747 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:10:15 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[72223]: starting mds.k at 2024-04-17T06:10:15.747 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:10:15 smithi098 bash[72208]: b4076f002011592a4842588b1e92ed80dbabe30471f18396a3ad1e5e3aa7912e 2024-04-17T06:10:15.747 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:10:15 smithi098 systemd[1]: Started Ceph mds.k for 565db2d8-fc7e-11ee-bc91-c7b262605968. 2024-04-17T06:10:16.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:10:15 smithi103 ceph-mon[36243]: pgmap v279: 97 pgs: 97 active+clean; 4.5 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 70 KiB/s wr, 36 op/s 2024-04-17T06:10:16.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:10:15 smithi103 ceph-mon[36243]: osdmap e98: 12 total, 12 up, 12 in 2024-04-17T06:10:16.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:10:15 smithi103 ceph-mon[36243]: daemon mds.k restarted 2024-04-17T06:10:16.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:10:15 smithi098 ceph-mon[36516]: pgmap v279: 97 pgs: 97 active+clean; 4.5 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 70 KiB/s wr, 36 op/s 2024-04-17T06:10:16.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:10:15 smithi098 ceph-mon[36516]: osdmap e98: 12 total, 12 up, 12 in 2024-04-17T06:10:16.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:10:15 smithi098 ceph-mon[36516]: daemon mds.k restarted 2024-04-17T06:10:16.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:10:15 smithi047 ceph-mon[32223]: pgmap v279: 97 pgs: 97 active+clean; 4.5 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 70 KiB/s wr, 36 op/s 2024-04-17T06:10:16.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:10:15 smithi047 ceph-mon[32223]: osdmap e98: 12 total, 12 up, 12 in 2024-04-17T06:10:16.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:10:15 smithi047 ceph-mon[32223]: daemon mds.k restarted 2024-04-17T06:10:16.640 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.642+0000 7f5355a21640 1 -- 172.21.15.47:0/3261653766 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5350101ab0 msgr2=0x7f5350105aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:16.640 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.642+0000 7f5355a21640 1 --2- 172.21.15.47:0/3261653766 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5350101ab0 0x7f5350105aa0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f5338009900 tx=0x7f533802bc30 comp rx=0 tx=0).stop 2024-04-17T06:10:16.640 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.642+0000 7f5355a21640 1 -- 172.21.15.47:0/3261653766 shutdown_connections 2024-04-17T06:10:16.640 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.642+0000 7f5355a21640 1 --2- 172.21.15.47:0/3261653766 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5350106180 0x7f535010dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:16.640 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.642+0000 7f5355a21640 1 --2- 172.21.15.47:0/3261653766 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5350101ab0 0x7f5350105aa0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:16.640 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.642+0000 7f5355a21640 1 --2- 172.21.15.47:0/3261653766 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5350101100 0x7f53501014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:16.641 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.642+0000 7f5355a21640 1 -- 172.21.15.47:0/3261653766 >> 172.21.15.47:0/3261653766 conn(0x7f5350077fe0 msgr2=0x7f53500783f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:16.641 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.642+0000 7f5355a21640 1 -- 172.21.15.47:0/3261653766 shutdown_connections 2024-04-17T06:10:16.641 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.642+0000 7f5355a21640 1 -- 172.21.15.47:0/3261653766 wait complete. 2024-04-17T06:10:16.641 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.642+0000 7f5355a21640 1 Processor -- start 2024-04-17T06:10:16.641 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.643+0000 7f5355a21640 1 -- start start 2024-04-17T06:10:16.643 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.643+0000 7f5355a21640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5350101100 0x7f535019e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:16.643 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.643+0000 7f5355a21640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5350106180 0x7f535019eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:16.643 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.643+0000 7f5355a21640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5350198660 0x7f5350198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:16.643 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.643+0000 7f5355a21640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f5350111210 con 0x7f5350198660 2024-04-17T06:10:16.643 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.643+0000 7f5355a21640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5350111090 con 0x7f5350101100 2024-04-17T06:10:16.643 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.643+0000 7f5355a21640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5350111390 con 0x7f5350106180 2024-04-17T06:10:16.643 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.643+0000 7f534f7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5350198660 0x7f5350198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:16.644 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.643+0000 7f534f7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5350198660 0x7f5350198ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57910/0 (socket says 172.21.15.47:57910) 2024-04-17T06:10:16.644 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.643+0000 7f534f7fe640 1 -- 172.21.15.47:0/506511664 learned_addr learned my addr 172.21.15.47:0/506511664 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:10:16.644 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.643+0000 7f534effd640 1 --2- 172.21.15.47:0/506511664 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5350101100 0x7f535019e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:16.644 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.643+0000 7f53467fc640 1 --2- 172.21.15.47:0/506511664 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5350106180 0x7f535019eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:16.644 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.643+0000 7f534f7fe640 1 -- 172.21.15.47:0/506511664 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5350106180 msgr2=0x7f535019eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:16.644 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.643+0000 7f534f7fe640 1 --2- 172.21.15.47:0/506511664 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5350106180 0x7f535019eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:16.644 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.643+0000 7f534f7fe640 1 -- 172.21.15.47:0/506511664 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5350101100 msgr2=0x7f535019e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:16.644 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.643+0000 7f534f7fe640 1 --2- 172.21.15.47:0/506511664 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5350101100 0x7f535019e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:16.644 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.643+0000 7f534f7fe640 1 -- 172.21.15.47:0/506511664 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5350199260 con 0x7f5350198660 2024-04-17T06:10:16.644 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.643+0000 7f534effd640 1 --2- 172.21.15.47:0/506511664 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5350101100 0x7f535019e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T06:10:16.644 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.644+0000 7f534f7fe640 1 --2- 172.21.15.47:0/506511664 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5350198660 0x7f5350198ae0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f534800e970 tx=0x7f534800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:16.644 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.644+0000 7f5354a1f640 1 -- 172.21.15.47:0/506511664 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f534800ccc0 con 0x7f5350198660 2024-04-17T06:10:16.644 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.644+0000 7f5355a21640 1 -- 172.21.15.47:0/506511664 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53501a5340 con 0x7f5350198660 2024-04-17T06:10:16.644 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.644+0000 7f5354a1f640 1 -- 172.21.15.47:0/506511664 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f534800ce60 con 0x7f5350198660 2024-04-17T06:10:16.645 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.644+0000 7f5354a1f640 1 -- 172.21.15.47:0/506511664 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5348010490 con 0x7f5350198660 2024-04-17T06:10:16.645 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.644+0000 7f5355a21640 1 -- 172.21.15.47:0/506511664 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53501a5740 con 0x7f5350198660 2024-04-17T06:10:16.652 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.647+0000 7f5354a1f640 1 -- 172.21.15.47:0/506511664 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5348005020 con 0x7f5350198660 2024-04-17T06:10:16.652 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.647+0000 7f5358605740 1 Processor -- start 2024-04-17T06:10:16.652 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.647+0000 7f5358605740 1 -- start start 2024-04-17T06:10:16.652 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.647+0000 7f5358605740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561cf2d79b60 0x561cf2d79f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:16.652 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.647+0000 7f5358605740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561cf2d7a480 0x561cf2d87680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:16.652 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.647+0000 7f5358605740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561cf2d87bc0 0x561cf2d89fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:16.652 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.647+0000 7f5358605740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x561cf2cdf7d0 con 0x561cf2d7a480 2024-04-17T06:10:16.652 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.647+0000 7f5358605740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x561cf2c7cf70 con 0x561cf2d87bc0 2024-04-17T06:10:16.652 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.647+0000 7f5358605740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561cf2cdb790 con 0x561cf2d79b60 2024-04-17T06:10:16.652 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.648+0000 7f534effd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561cf2d7a480 0x561cf2d87680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:16.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.648+0000 7f534effd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561cf2d7a480 0x561cf2d87680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57912/0 (socket says 172.21.15.47:57912) 2024-04-17T06:10:16.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.648+0000 7f534effd640 1 -- 172.21.15.47:0/4158325883 learned_addr learned my addr 172.21.15.47:0/4158325883 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:10:16.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.648+0000 7f53467fc640 1 --2- 172.21.15.47:0/4158325883 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561cf2d87bc0 0x561cf2d89fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:16.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.648+0000 7f53467fc640 1 --2- 172.21.15.47:0/4158325883 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561cf2d79b60 0x561cf2d79f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:16.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.648+0000 7f5354a1f640 1 --2- 172.21.15.47:0/506511664 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f533c0a3dc0 0x7f533c0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:16.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.648+0000 7f534effd640 1 -- 172.21.15.47:0/4158325883 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561cf2d79b60 msgr2=0x561cf2d79f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:16.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.648+0000 7f534effd640 1 --2- 172.21.15.47:0/4158325883 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561cf2d79b60 0x561cf2d79f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:16.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.648+0000 7f534effd640 1 -- 172.21.15.47:0/4158325883 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561cf2d87bc0 msgr2=0x561cf2d89fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:16.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.648+0000 7f534effd640 1 --2- 172.21.15.47:0/4158325883 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561cf2d87bc0 0x561cf2d89fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:16.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.648+0000 7f534effd640 1 -- 172.21.15.47:0/4158325883 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561cf29f2210 con 0x561cf2d7a480 2024-04-17T06:10:16.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.648+0000 7f53467fc640 1 --2- 172.21.15.47:0/4158325883 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561cf2d79b60 0x561cf2d79f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:10:16.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.648+0000 7f534f7fe640 1 --2- 172.21.15.47:0/506511664 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f533c0a3dc0 0x7f533c0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:16.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.648+0000 7f5354a1f640 1 -- 172.21.15.47:0/506511664 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8469+0+0 (secure 0 0 0) 0x7f53480d0cb0 con 0x7f5350198660 2024-04-17T06:10:16.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.648+0000 7f534effd640 1 --2- 172.21.15.47:0/4158325883 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561cf2d7a480 0x561cf2d87680 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f5340010800 tx=0x7f5340010cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28dc2f6bfbeaa66e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:16.654 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.648+0000 7f5345ffb640 1 -- 172.21.15.47:0/4158325883 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5340016070 con 0x561cf2d7a480 2024-04-17T06:10:16.654 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.649+0000 7f5345ffb640 1 -- 172.21.15.47:0/4158325883 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f534001b460 con 0x561cf2d7a480 2024-04-17T06:10:16.654 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.649+0000 7f5345ffb640 1 -- 172.21.15.47:0/4158325883 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5340012670 con 0x561cf2d7a480 2024-04-17T06:10:16.654 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.649+0000 7f5358605740 1 -- 172.21.15.47:0/4158325883 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561cf2d7a480 msgr2=0x561cf2d87680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:16.654 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.649+0000 7f534f7fe640 1 --2- 172.21.15.47:0/506511664 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f533c0a3dc0 0x7f533c0a6280 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f534809c920 tx=0x7f534806c500 comp rx=0 tx=0).ready entity=mgr.24923 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:16.654 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.649+0000 7f5358605740 1 --2- 172.21.15.47:0/4158325883 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561cf2d7a480 0x561cf2d87680 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f5340010800 tx=0x7f5340010cc0 comp rx=0 tx=0).stop 2024-04-17T06:10:16.654 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.649+0000 7f5358605740 1 -- 172.21.15.47:0/4158325883 shutdown_connections 2024-04-17T06:10:16.654 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.649+0000 7f5358605740 1 --2- 172.21.15.47:0/4158325883 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561cf2d87bc0 0x561cf2d89fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:16.654 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.649+0000 7f5358605740 1 --2- 172.21.15.47:0/4158325883 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561cf2d7a480 0x561cf2d87680 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:16.654 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.649+0000 7f5358605740 1 --2- 172.21.15.47:0/4158325883 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561cf2d79b60 0x561cf2d79f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:16.654 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.649+0000 7f5358605740 1 -- 172.21.15.47:0/4158325883 >> 172.21.15.47:0/4158325883 conn(0x561cf2d749e0 msgr2=0x561cf2d74dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:16.654 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.649+0000 7f5358605740 1 -- 172.21.15.47:0/4158325883 shutdown_connections 2024-04-17T06:10:16.654 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.649+0000 7f5358605740 1 -- 172.21.15.47:0/4158325883 wait complete. 2024-04-17T06:10:16.654 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.650+0000 7f5358605740 1 -- start start 2024-04-17T06:10:16.655 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.650+0000 7f5358605740 1 Processor -- start 2024-04-17T06:10:16.655 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.650+0000 7f5358605740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561cf2d79b60 0x561cf2d84bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:16.655 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.650+0000 7f5358605740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561cf2d7a480 0x561cf2d850f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:16.655 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.650+0000 7f5358605740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561cf2d87bc0 0x561cf2d85630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:16.655 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.650+0000 7f5358605740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x561cf2d8a7f0 con 0x561cf2d87bc0 2024-04-17T06:10:16.655 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.650+0000 7f5358605740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x561cf2c67e90 con 0x561cf2d79b60 2024-04-17T06:10:16.655 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.650+0000 7f5358605740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561cf2d8a970 con 0x561cf2d7a480 2024-04-17T06:10:16.655 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.650+0000 7f5358605740 10 client.0 _ll_register_callbacks cb 0x561cf2c7dd40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:10:16.655 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.650+0000 7f53467fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561cf2d87bc0 0x561cf2d85630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:16.656 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.650+0000 7f53467fc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561cf2d79b60 0x561cf2d84bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:16.656 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.650+0000 7f53467fc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561cf2d79b60 0x561cf2d84bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.47:55256/0 (socket says 172.21.15.47:55256) 2024-04-17T06:10:16.656 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.650+0000 7f53467fc640 1 -- 172.21.15.47:0/2753248442 learned_addr learned my addr 172.21.15.47:0/2753248442 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:10:16.656 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.651+0000 7f534effd640 1 --2- 172.21.15.47:0/2753248442 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561cf2d7a480 0x561cf2d850f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:16.656 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.651+0000 7f53467fc640 1 -- 172.21.15.47:0/2753248442 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561cf2d7a480 msgr2=0x561cf2d850f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:16.656 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.651+0000 7f53467fc640 1 --2- 172.21.15.47:0/2753248442 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561cf2d7a480 0x561cf2d850f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:16.656 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.651+0000 7f53467fc640 1 -- 172.21.15.47:0/2753248442 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561cf2d87bc0 msgr2=0x561cf2d85630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:16.656 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.651+0000 7f53467fc640 1 --2- 172.21.15.47:0/2753248442 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561cf2d87bc0 0x561cf2d85630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:16.656 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.651+0000 7f53467fc640 1 -- 172.21.15.47:0/2753248442 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561cf2abe620 con 0x561cf2d79b60 2024-04-17T06:10:16.657 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.651+0000 7f534effd640 1 --2- 172.21.15.47:0/2753248442 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561cf2d7a480 0x561cf2d850f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:10:16.657 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.651+0000 7f53467fc640 1 --2- 172.21.15.47:0/2753248442 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561cf2d79b60 0x561cf2d84bb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5338005ec0 tx=0x7f5338000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a26aa86f8721527 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:16.657 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.651+0000 7f532effd640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T06:10:16.657 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.651+0000 7f532effd640 1 -- 172.21.15.47:0/2753248442 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f533802cce0 con 0x561cf2d79b60 2024-04-17T06:10:16.657 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.651+0000 7f5358605740 1 -- 172.21.15.47:0/2753248442 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f534001b460 con 0x561cf2d79b60 2024-04-17T06:10:16.657 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.651+0000 7f532effd640 1 -- 172.21.15.47:0/2753248442 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8469+0+0 (secure 0 0 0) 0x7f5338004480 con 0x561cf2d79b60 2024-04-17T06:10:16.657 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.651+0000 7f532effd640 1 -- 172.21.15.47:0/2753248442 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f5338005020 con 0x561cf2d79b60 2024-04-17T06:10:16.657 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.651+0000 7f5358605740 10 client.25574 fetch_fsmap learned FSMap version 34 2024-04-17T06:10:16.657 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.651+0000 7f5358605740 1 -- 172.21.15.47:0/2753248442 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x561cf2d96b00 con 0x561cf2d79b60 2024-04-17T06:10:16.657 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.651+0000 7f532effd640 1 -- 172.21.15.47:0/2753248442 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f534001b460 con 0x561cf2d79b60 2024-04-17T06:10:16.657 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.652+0000 7f532effd640 1 -- 172.21.15.47:0/2753248442 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 34) ==== 8339+0+0 (secure 0 0 0) 0x7f53380035f0 con 0x561cf2d79b60 2024-04-17T06:10:16.657 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.652+0000 7f5358605740 10 client.25574 fetch_fsmap finished waiting for FSMap version 34 2024-04-17T06:10:16.657 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.652+0000 7f5358605740 10 client.25574 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-17T06:10:16.657 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.652+0000 7f5358605740 20 client.25574 populate_metadata read hostname 'smithi047' 2024-04-17T06:10:16.658 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.652+0000 7f5358605740 1 --2- 172.21.15.47:0/2753248442 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x561cf2d97f70 0x561cf2d9a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:16.658 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.652+0000 7f5358605740 4 client.25574 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T06:10:16.658 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.652+0000 7f5358605740 1 -- 172.21.15.47:0/2753248442 --> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561cf2ced1f0 con 0x561cf2d97f70 2024-04-17T06:10:16.658 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.652+0000 7f534effd640 1 --2- 172.21.15.47:0/2753248442 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x561cf2d97f70 0x561cf2d9a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:16.658 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.652+0000 7f532effd640 10 client.25574 ms_handle_connect on v2:172.21.15.98:6838/2841287917 2024-04-17T06:10:16.658 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.652+0000 7f534effd640 1 --2- 172.21.15.47:0/2753248442 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x561cf2d97f70 0x561cf2d9a360 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=784c34ec451e1581 server_cookie=a84d22371ed0c2b9 in_seq=0 out_seq=0 2024-04-17T06:10:16.658 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.653+0000 7f532effd640 1 -- 172.21.15.47:0/2753248442 <== mds.0 v2:172.21.15.98:6838/2841287917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561cf2ced1f0 con 0x561cf2d97f70 2024-04-17T06:10:16.658 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.653+0000 7f532effd640 10 client.25574 handle_command_reply: tid=0 2024-04-17T06:10:16.658 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.653+0000 7f5358605740 1 client.25574 shutdown 2024-04-17T06:10:16.658 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.653+0000 7f5358605740 1 -- 172.21.15.47:0/2753248442 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561cf2d79b60 msgr2=0x561cf2d84bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:16.658 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.653+0000 7f5358605740 1 --2- 172.21.15.47:0/2753248442 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561cf2d79b60 0x561cf2d84bb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5338005ec0 tx=0x7f5338000bc0 comp rx=0 tx=0).stop 2024-04-17T06:10:16.658 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.653+0000 7f5358605740 1 -- 172.21.15.47:0/2753248442 shutdown_connections 2024-04-17T06:10:16.658 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.653+0000 7f5358605740 1 --2- 172.21.15.47:0/2753248442 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x561cf2d97f70 0x561cf2d9a360 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:16.659 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.653+0000 7f5358605740 1 --2- 172.21.15.47:0/2753248442 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561cf2d87bc0 0x561cf2d85630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:16.659 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.653+0000 7f5358605740 1 --2- 172.21.15.47:0/2753248442 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561cf2d7a480 0x561cf2d850f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:16.659 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.653+0000 7f5358605740 1 --2- 172.21.15.47:0/2753248442 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561cf2d79b60 0x561cf2d84bb0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:16.659 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.653+0000 7f532effd640 0 client.25574 ms_handle_reset on v2:172.21.15.98:6838/2841287917 2024-04-17T06:10:16.659 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.653+0000 7f5358605740 1 -- 172.21.15.47:0/2753248442 >> 172.21.15.47:0/2753248442 conn(0x561cf2d749e0 msgr2=0x561cf2d76730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:16.659 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.653+0000 7f5358605740 1 -- 172.21.15.47:0/2753248442 shutdown_connections 2024-04-17T06:10:16.659 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.653+0000 7f5358605740 1 -- 172.21.15.47:0/2753248442 wait complete. 2024-04-17T06:10:16.659 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.654+0000 7f5358605740 20 client.25574 trim_cache size 0 max 16384 2024-04-17T06:10:16.671 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.668+0000 7f5358605740 1 Processor -- start 2024-04-17T06:10:16.672 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.668+0000 7f5358605740 1 -- start start 2024-04-17T06:10:16.672 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.669+0000 7f5358605740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561cf2d79b60 0x561cf2d872e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:16.672 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.669+0000 7f5358605740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561cf2d7a480 0x561cf2d93720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:16.672 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.669+0000 7f5358605740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561cf2d87bc0 0x561cf2d76540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:16.672 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.669+0000 7f5358605740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f53501ae260 con 0x561cf2d87bc0 2024-04-17T06:10:16.672 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.669+0000 7f5358605740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f53501abb20 con 0x561cf2d7a480 2024-04-17T06:10:16.672 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.669+0000 7f5358605740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f53501a9970 con 0x561cf2d79b60 2024-04-17T06:10:16.672 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.669+0000 7f53467fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561cf2d87bc0 0x561cf2d76540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:16.673 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.669+0000 7f53467fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561cf2d87bc0 0x561cf2d76540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57940/0 (socket says 172.21.15.47:57940) 2024-04-17T06:10:16.673 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.669+0000 7f53467fc640 1 -- 172.21.15.47:0/1168939841 learned_addr learned my addr 172.21.15.47:0/1168939841 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:10:16.673 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.669+0000 7f534effd640 1 --2- 172.21.15.47:0/1168939841 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561cf2d7a480 0x561cf2d93720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:16.673 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.669+0000 7f53467fc640 1 -- 172.21.15.47:0/1168939841 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561cf2d79b60 msgr2=0x561cf2d872e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:16.673 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.669+0000 7f53467fc640 1 --2- 172.21.15.47:0/1168939841 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561cf2d79b60 0x561cf2d872e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:16.673 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.669+0000 7f53467fc640 1 -- 172.21.15.47:0/1168939841 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561cf2d7a480 msgr2=0x561cf2d93720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:16.673 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.669+0000 7f53467fc640 1 --2- 172.21.15.47:0/1168939841 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561cf2d7a480 0x561cf2d93720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:16.673 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.669+0000 7f53467fc640 1 -- 172.21.15.47:0/1168939841 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561cf2d6ef10 con 0x561cf2d87bc0 2024-04-17T06:10:16.673 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.669+0000 7f53467fc640 1 --2- 172.21.15.47:0/1168939841 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561cf2d87bc0 0x561cf2d76540 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f5338003a90 tx=0x7f533802c690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbe576f23c364e1e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:16.673 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.669+0000 7f53457fa640 1 -- 172.21.15.47:0/1168939841 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f533802cb80 con 0x561cf2d87bc0 2024-04-17T06:10:16.673 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.669+0000 7f53457fa640 1 -- 172.21.15.47:0/1168939841 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f533802cd20 con 0x561cf2d87bc0 2024-04-17T06:10:16.673 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.669+0000 7f53457fa640 1 -- 172.21.15.47:0/1168939841 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5338004920 con 0x561cf2d87bc0 2024-04-17T06:10:16.673 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.670+0000 7f5358605740 1 -- 172.21.15.47:0/1168939841 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561cf2d87bc0 msgr2=0x561cf2d76540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:16.674 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.670+0000 7f5358605740 1 --2- 172.21.15.47:0/1168939841 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561cf2d87bc0 0x561cf2d76540 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f5338003a90 tx=0x7f533802c690 comp rx=0 tx=0).stop 2024-04-17T06:10:16.674 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.670+0000 7f5358605740 1 -- 172.21.15.47:0/1168939841 shutdown_connections 2024-04-17T06:10:16.674 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.670+0000 7f5358605740 1 --2- 172.21.15.47:0/1168939841 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561cf2d87bc0 0x561cf2d76540 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:16.674 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.670+0000 7f5358605740 1 --2- 172.21.15.47:0/1168939841 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561cf2d7a480 0x561cf2d93720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:16.674 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.670+0000 7f5358605740 1 --2- 172.21.15.47:0/1168939841 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561cf2d79b60 0x561cf2d872e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:16.674 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.670+0000 7f5358605740 1 -- 172.21.15.47:0/1168939841 >> 172.21.15.47:0/1168939841 conn(0x561cf2d749e0 msgr2=0x561cf2d75e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:16.674 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.670+0000 7f5358605740 1 -- 172.21.15.47:0/1168939841 shutdown_connections 2024-04-17T06:10:16.674 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.670+0000 7f5358605740 1 -- 172.21.15.47:0/1168939841 wait complete. 2024-04-17T06:10:16.674 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.670+0000 7f5358605740 1 -- start start 2024-04-17T06:10:16.674 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.671+0000 7f5358605740 1 Processor -- start 2024-04-17T06:10:16.674 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.671+0000 7f5358605740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561cf2d79b60 0x561cf2d901c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:16.674 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.671+0000 7f5358605740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561cf2d7a480 0x561cf2d8b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:16.674 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.671+0000 7f5358605740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561cf2d87bc0 0x561cf2d8bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:16.675 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.671+0000 7f5358605740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f53501a9fe0 con 0x561cf2d87bc0 2024-04-17T06:10:16.675 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.671+0000 7f5358605740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f53501ab450 con 0x561cf2d79b60 2024-04-17T06:10:16.675 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.671+0000 7f5358605740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f53501aa6b0 con 0x561cf2d7a480 2024-04-17T06:10:16.675 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.671+0000 7f5358605740 10 client.0 _ll_register_callbacks cb 0x7f5338009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:10:16.675 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.671+0000 7f53467fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561cf2d87bc0 0x561cf2d8bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:16.675 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.671+0000 7f53467fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561cf2d87bc0 0x561cf2d8bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57948/0 (socket says 172.21.15.47:57948) 2024-04-17T06:10:16.675 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.671+0000 7f53467fc640 1 -- 172.21.15.47:0/923011226 learned_addr learned my addr 172.21.15.47:0/923011226 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:10:16.675 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.671+0000 7f53467fc640 1 --2- 172.21.15.47:0/923011226 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561cf2d79b60 0x561cf2d901c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:16.675 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.671+0000 7f534effd640 1 --2- 172.21.15.47:0/923011226 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561cf2d7a480 0x561cf2d8b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:16.675 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.671+0000 7f53467fc640 1 -- 172.21.15.47:0/923011226 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561cf2d7a480 msgr2=0x561cf2d8b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:16.675 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.671+0000 7f53467fc640 1 --2- 172.21.15.47:0/923011226 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561cf2d7a480 0x561cf2d8b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:16.675 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.671+0000 7f53467fc640 1 -- 172.21.15.47:0/923011226 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561cf2d79b60 msgr2=0x561cf2d901c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:16.675 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.671+0000 7f53467fc640 1 --2- 172.21.15.47:0/923011226 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561cf2d79b60 0x561cf2d901c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:16.675 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.671+0000 7f53467fc640 1 -- 172.21.15.47:0/923011226 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561cf2dfd4a0 con 0x561cf2d87bc0 2024-04-17T06:10:16.676 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.672+0000 7f53467fc640 1 --2- 172.21.15.47:0/923011226 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561cf2d87bc0 0x561cf2d8bc70 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f533802c420 tx=0x7f5338003690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20cc3286f8e21351 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:16.676 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.672+0000 7f532effd640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-17T06:10:16.676 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.672+0000 7f532effd640 1 -- 172.21.15.47:0/923011226 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5338005d50 con 0x561cf2d87bc0 2024-04-17T06:10:16.676 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.672+0000 7f532effd640 1 -- 172.21.15.47:0/923011226 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8469+0+0 (secure 0 0 0) 0x7f5338004480 con 0x561cf2d87bc0 2024-04-17T06:10:16.676 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.672+0000 7f5358605740 1 -- 172.21.15.47:0/923011226 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f533802cd20 con 0x561cf2d87bc0 2024-04-17T06:10:16.676 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.672+0000 7f532effd640 1 -- 172.21.15.47:0/923011226 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f5338003e10 con 0x561cf2d87bc0 2024-04-17T06:10:16.676 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.672+0000 7f5358605740 10 client.16155 fetch_fsmap learned FSMap version 34 2024-04-17T06:10:16.676 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.672+0000 7f5358605740 1 -- 172.21.15.47:0/923011226 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x561cf2dfe670 con 0x561cf2d87bc0 2024-04-17T06:10:16.676 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.672+0000 7f532effd640 1 -- 172.21.15.47:0/923011226 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f533802cd20 con 0x561cf2d87bc0 2024-04-17T06:10:16.676 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.672+0000 7f532effd640 1 -- 172.21.15.47:0/923011226 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 34) ==== 8339+0+0 (secure 0 0 0) 0x7f5338058440 con 0x561cf2d87bc0 2024-04-17T06:10:16.676 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.672+0000 7f5358605740 10 client.16155 fetch_fsmap finished waiting for FSMap version 34 2024-04-17T06:10:16.676 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.672+0000 7f5358605740 10 client.16155 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-17T06:10:16.676 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.672+0000 7f5358605740 20 client.16155 populate_metadata read hostname 'smithi047' 2024-04-17T06:10:16.676 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.672+0000 7f5358605740 1 --2- 172.21.15.47:0/923011226 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x561cf2dfed30 0x561cf2e01150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:16.677 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.672+0000 7f5358605740 4 client.16155 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-17T06:10:16.677 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.672+0000 7f5358605740 1 -- 172.21.15.47:0/923011226 --> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561cf2ce7060 con 0x561cf2dfed30 2024-04-17T06:10:16.677 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.673+0000 7f534effd640 1 --2- 172.21.15.47:0/923011226 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x561cf2dfed30 0x561cf2e01150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:16.677 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.673+0000 7f534effd640 1 --2- 172.21.15.47:0/923011226 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x561cf2dfed30 0x561cf2e01150 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc8dfd75e9c218d5 server_cookie=53ba0339636cb79a in_seq=0 out_seq=0 2024-04-17T06:10:16.677 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.673+0000 7f532effd640 10 client.16155 ms_handle_connect on v2:172.21.15.98:6838/2841287917 2024-04-17T06:10:16.679 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.680+0000 7f532effd640 1 -- 172.21.15.47:0/923011226 <== mds.0 v2:172.21.15.98:6838/2841287917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561cf2ce7060 con 0x561cf2dfed30 2024-04-17T06:10:16.679 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.680+0000 7f532effd640 10 client.16155 handle_command_reply: tid=0 2024-04-17T06:10:16.679 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.681+0000 7f5358605740 1 client.16155 shutdown 2024-04-17T06:10:16.679 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.681+0000 7f5358605740 1 -- 172.21.15.47:0/923011226 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561cf2d87bc0 msgr2=0x561cf2d8bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:16.679 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.681+0000 7f5358605740 1 --2- 172.21.15.47:0/923011226 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561cf2d87bc0 0x561cf2d8bc70 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f533802c420 tx=0x7f5338003690 comp rx=0 tx=0).stop 2024-04-17T06:10:16.679 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.681+0000 7f5358605740 1 -- 172.21.15.47:0/923011226 shutdown_connections 2024-04-17T06:10:16.679 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.681+0000 7f5358605740 1 --2- 172.21.15.47:0/923011226 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x561cf2dfed30 0x561cf2e01150 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:16.679 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.681+0000 7f5358605740 1 --2- 172.21.15.47:0/923011226 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561cf2d87bc0 0x561cf2d8bc70 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:16.679 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.681+0000 7f532effd640 0 client.16155 ms_handle_reset on v2:172.21.15.98:6838/2841287917 2024-04-17T06:10:16.680 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.681+0000 7f5358605740 1 --2- 172.21.15.47:0/923011226 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561cf2d7a480 0x561cf2d8b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:16.680 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.681+0000 7f5358605740 1 --2- 172.21.15.47:0/923011226 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561cf2d79b60 0x561cf2d901c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:16.680 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.681+0000 7f5358605740 1 -- 172.21.15.47:0/923011226 >> 172.21.15.47:0/923011226 conn(0x561cf2d749e0 msgr2=0x561cf2d766c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:16.680 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.681+0000 7f5358605740 1 -- 172.21.15.47:0/923011226 shutdown_connections 2024-04-17T06:10:16.680 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.681+0000 7f5358605740 1 -- 172.21.15.47:0/923011226 wait complete. 2024-04-17T06:10:16.680 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.681+0000 7f5358605740 20 client.16155 trim_cache size 0 max 16384 2024-04-17T06:10:16.680 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-04-17T06:10:16.680 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-04-17T06:10:16.680 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-04-17T06:10:16.680 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-04-17T06:10:16.681 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.681+0000 7f5355a21640 1 -- 172.21.15.47:0/506511664 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f533c0a3dc0 msgr2=0x7f533c0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:16.681 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.682+0000 7f5355a21640 1 --2- 172.21.15.47:0/506511664 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f533c0a3dc0 0x7f533c0a6280 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f534809c920 tx=0x7f534806c500 comp rx=0 tx=0).stop 2024-04-17T06:10:16.681 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.682+0000 7f5355a21640 1 -- 172.21.15.47:0/506511664 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5350198660 msgr2=0x7f5350198ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:16.682 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.682+0000 7f5355a21640 1 --2- 172.21.15.47:0/506511664 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5350198660 0x7f5350198ae0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f534800e970 tx=0x7f534800ed70 comp rx=0 tx=0).stop 2024-04-17T06:10:16.682 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.682+0000 7f5355a21640 1 -- 172.21.15.47:0/506511664 shutdown_connections 2024-04-17T06:10:16.682 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.682+0000 7f5355a21640 1 --2- 172.21.15.47:0/506511664 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f533c0a3dc0 0x7f533c0a6280 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:16.682 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.682+0000 7f5355a21640 1 --2- 172.21.15.47:0/506511664 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5350198660 0x7f5350198ae0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:16.682 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.682+0000 7f5355a21640 1 --2- 172.21.15.47:0/506511664 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5350106180 0x7f535019eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:16.682 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.682+0000 7f5355a21640 1 --2- 172.21.15.47:0/506511664 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5350101100 0x7f535019e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:16.682 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.682+0000 7f5355a21640 1 -- 172.21.15.47:0/506511664 >> 172.21.15.47:0/506511664 conn(0x7f5350077fe0 msgr2=0x7f5350104dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:16.682 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.682+0000 7f5355a21640 1 -- 172.21.15.47:0/506511664 shutdown_connections 2024-04-17T06:10:16.682 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:16.682+0000 7f5355a21640 1 -- 172.21.15.47:0/506511664 wait complete. 2024-04-17T06:10:17.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:10:16 smithi103 ceph-mon[36243]: osdmap e99: 12 total, 12 up, 12 in 2024-04-17T06:10:17.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:10:16 smithi103 ceph-mon[36243]: mds.? [v2:172.21.15.98:6832/2380016002,v1:172.21.15.98:6833/2380016002] up:boot 2024-04-17T06:10:17.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:10:16 smithi103 ceph-mon[36243]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-04-17T06:10:17.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:10:16 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-17T06:10:17.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:10:16 smithi103 ceph-mon[36243]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-17T06:10:17.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:10:16 smithi098 ceph-mon[36516]: osdmap e99: 12 total, 12 up, 12 in 2024-04-17T06:10:17.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:10:16 smithi098 ceph-mon[36516]: mds.? [v2:172.21.15.98:6832/2380016002,v1:172.21.15.98:6833/2380016002] up:boot 2024-04-17T06:10:17.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:10:16 smithi098 ceph-mon[36516]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-04-17T06:10:17.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:10:16 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-17T06:10:17.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:10:16 smithi098 ceph-mon[36516]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-17T06:10:17.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:10:16 smithi047 ceph-mon[32223]: osdmap e99: 12 total, 12 up, 12 in 2024-04-17T06:10:17.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:10:16 smithi047 ceph-mon[32223]: mds.? [v2:172.21.15.98:6832/2380016002,v1:172.21.15.98:6833/2380016002] up:boot 2024-04-17T06:10:17.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:10:16 smithi047 ceph-mon[32223]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-04-17T06:10:17.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:10:16 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-17T06:10:17.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:10:16 smithi047 ceph-mon[32223]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-17T06:10:18.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:10:17 smithi098 ceph-mon[36516]: pgmap v282: 97 pgs: 97 active+clean; 4.5 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-04-17T06:10:18.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:10:17 smithi047 ceph-mon[32223]: pgmap v282: 97 pgs: 97 active+clean; 4.5 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-04-17T06:10:18.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:10:17 smithi103 ceph-mon[36243]: pgmap v282: 97 pgs: 97 active+clean; 4.5 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-04-17T06:10:20.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:10:19 smithi098 ceph-mon[36516]: pgmap v283: 97 pgs: 97 active+clean; 4.5 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7.1 KiB/s wr, 5 op/s 2024-04-17T06:10:20.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:10:19 smithi047 ceph-mon[32223]: pgmap v283: 97 pgs: 97 active+clean; 4.5 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7.1 KiB/s wr, 5 op/s 2024-04-17T06:10:20.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:10:19 smithi103 ceph-mon[36243]: pgmap v283: 97 pgs: 97 active+clean; 4.5 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7.1 KiB/s wr, 5 op/s 2024-04-17T06:10:22.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:10:21 smithi098 ceph-mon[36516]: pgmap v284: 97 pgs: 97 active+clean; 4.6 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 16 op/s 2024-04-17T06:10:22.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:10:21 smithi047 ceph-mon[32223]: pgmap v284: 97 pgs: 97 active+clean; 4.6 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 16 op/s 2024-04-17T06:10:22.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:10:21 smithi103 ceph-mon[36243]: pgmap v284: 97 pgs: 97 active+clean; 4.6 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 16 op/s 2024-04-17T06:10:24.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:10:23 smithi098 ceph-mon[36516]: pgmap v285: 97 pgs: 97 active+clean; 4.6 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-04-17T06:10:24.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:10:23 smithi047 ceph-mon[32223]: pgmap v285: 97 pgs: 97 active+clean; 4.6 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-04-17T06:10:24.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:10:23 smithi103 ceph-mon[36243]: pgmap v285: 97 pgs: 97 active+clean; 4.6 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-04-17T06:10:26.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:10:25 smithi098 ceph-mon[36516]: pgmap v286: 97 pgs: 97 active+clean; 4.6 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 15 op/s 2024-04-17T06:10:26.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:10:25 smithi047 ceph-mon[32223]: pgmap v286: 97 pgs: 97 active+clean; 4.6 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 15 op/s 2024-04-17T06:10:26.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:10:25 smithi103 ceph-mon[36243]: pgmap v286: 97 pgs: 97 active+clean; 4.6 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 15 op/s 2024-04-17T06:10:28.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:10:27 smithi098 ceph-mon[36516]: pgmap v287: 97 pgs: 97 active+clean; 4.6 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-17T06:10:28.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:10:27 smithi047 ceph-mon[32223]: pgmap v287: 97 pgs: 97 active+clean; 4.6 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-17T06:10:28.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:10:27 smithi103 ceph-mon[36243]: pgmap v287: 97 pgs: 97 active+clean; 4.6 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-17T06:10:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.780+0000 7f703e6ba640 1 -- 172.21.15.47:0/2572266638 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f70381011b0 msgr2=0x7f7038101630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.780+0000 7f703e6ba640 1 --2- 172.21.15.47:0/2572266638 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f70381011b0 0x7f7038101630 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f702c009900 tx=0x7f702c02bc80 comp rx=0 tx=0).stop 2024-04-17T06:10:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.781+0000 7f703e6ba640 1 -- 172.21.15.47:0/2572266638 shutdown_connections 2024-04-17T06:10:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.781+0000 7f703e6ba640 1 --2- 172.21.15.47:0/2572266638 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7038101b70 0x7f703810dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.781+0000 7f703e6ba640 1 --2- 172.21.15.47:0/2572266638 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f70381011b0 0x7f7038101630 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.781+0000 7f703e6ba640 1 --2- 172.21.15.47:0/2572266638 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7038106fc0 0x7f70381073a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.781+0000 7f703e6ba640 1 -- 172.21.15.47:0/2572266638 >> 172.21.15.47:0/2572266638 conn(0x7f7038076c60 msgr2=0x7f7038077070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.781+0000 7f703e6ba640 1 -- 172.21.15.47:0/2572266638 shutdown_connections 2024-04-17T06:10:29.782 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.781+0000 7f703e6ba640 1 -- 172.21.15.47:0/2572266638 wait complete. 2024-04-17T06:10:29.782 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.781+0000 7f703e6ba640 1 Processor -- start 2024-04-17T06:10:29.782 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.781+0000 7f703e6ba640 1 -- start start 2024-04-17T06:10:29.782 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.782+0000 7f703e6ba640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f70381011b0 0x7f7038102b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:29.782 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.782+0000 7f703e6ba640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7038101b70 0x7f7038103060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:29.782 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.782+0000 7f703e6ba640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7038106fc0 0x7f70381035a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:29.782 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.782+0000 7f703e6ba640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f70381111e0 con 0x7f7038106fc0 2024-04-17T06:10:29.782 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.782+0000 7f703e6ba640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f7038111060 con 0x7f70381011b0 2024-04-17T06:10:29.782 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.782+0000 7f703e6ba640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7038111360 con 0x7f7038101b70 2024-04-17T06:10:29.782 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.782+0000 7f703cc57640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7038106fc0 0x7f70381035a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:29.782 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.782+0000 7f703cc57640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7038106fc0 0x7f70381035a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37972/0 (socket says 172.21.15.47:37972) 2024-04-17T06:10:29.782 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.782+0000 7f703cc57640 1 -- 172.21.15.47:0/1096739087 learned_addr learned my addr 172.21.15.47:0/1096739087 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:10:29.782 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.782+0000 7f7037fff640 1 --2- 172.21.15.47:0/1096739087 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f70381011b0 0x7f7038102b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:29.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.782+0000 7f70377fe640 1 --2- 172.21.15.47:0/1096739087 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7038101b70 0x7f7038103060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:29.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.782+0000 7f703cc57640 1 -- 172.21.15.47:0/1096739087 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7038101b70 msgr2=0x7f7038103060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:29.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.782+0000 7f703cc57640 1 --2- 172.21.15.47:0/1096739087 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7038101b70 0x7f7038103060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:29.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.782+0000 7f703cc57640 1 -- 172.21.15.47:0/1096739087 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f70381011b0 msgr2=0x7f7038102b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:29.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.782+0000 7f703cc57640 1 --2- 172.21.15.47:0/1096739087 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f70381011b0 0x7f7038102b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:29.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.782+0000 7f703cc57640 1 -- 172.21.15.47:0/1096739087 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70381a6c30 con 0x7f7038106fc0 2024-04-17T06:10:29.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.782+0000 7f703cc57640 1 --2- 172.21.15.47:0/1096739087 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7038106fc0 0x7f70381035a0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f703000cad0 tx=0x7f703000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:29.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.783+0000 7f70357fa640 1 -- 172.21.15.47:0/1096739087 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f703000ae20 con 0x7f7038106fc0 2024-04-17T06:10:29.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.783+0000 7f70357fa640 1 -- 172.21.15.47:0/1096739087 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7030005020 con 0x7f7038106fc0 2024-04-17T06:10:29.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.783+0000 7f703e6ba640 1 -- 172.21.15.47:0/1096739087 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70381a6f20 con 0x7f7038106fc0 2024-04-17T06:10:29.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.783+0000 7f70357fa640 1 -- 172.21.15.47:0/1096739087 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f703000f6f0 con 0x7f7038106fc0 2024-04-17T06:10:29.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.783+0000 7f703e6ba640 1 -- 172.21.15.47:0/1096739087 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70381a73f0 con 0x7f7038106fc0 2024-04-17T06:10:29.789 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.785+0000 7f70357fa640 1 -- 172.21.15.47:0/1096739087 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f70300051d0 con 0x7f7038106fc0 2024-04-17T06:10:29.789 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.785+0000 7f7041376740 1 Processor -- start 2024-04-17T06:10:29.789 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.785+0000 7f7041376740 1 -- start start 2024-04-17T06:10:29.790 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.785+0000 7f7041376740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a4f54b30c0 0x55a4f54b34a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:29.790 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.785+0000 7f7041376740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a4f54b39e0 0x55a4f54bc270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:29.790 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.785+0000 7f7041376740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a4f54bc7b0 0x55a4f54beba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:29.790 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.786+0000 7f7041376740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55a4f54187d0 con 0x55a4f54b30c0 2024-04-17T06:10:29.790 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.786+0000 7f7041376740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55a4f53b5f70 con 0x55a4f54b39e0 2024-04-17T06:10:29.790 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.786+0000 7f7041376740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55a4f5414790 con 0x55a4f54bc7b0 2024-04-17T06:10:29.790 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.786+0000 7f70377fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a4f54b30c0 0x55a4f54b34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:29.790 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.786+0000 7f70377fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a4f54b30c0 0x55a4f54b34a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37984/0 (socket says 172.21.15.47:37984) 2024-04-17T06:10:29.790 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.786+0000 7f70377fe640 1 -- 172.21.15.47:0/3881216616 learned_addr learned my addr 172.21.15.47:0/3881216616 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:10:29.790 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.786+0000 7f7037fff640 1 --2- 172.21.15.47:0/3881216616 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a4f54b39e0 0x55a4f54bc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:29.790 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.786+0000 7f70377fe640 1 --2- 172.21.15.47:0/3881216616 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a4f54bc7b0 0x55a4f54beba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:29.790 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.786+0000 7f70357fa640 1 --2- 172.21.15.47:0/1096739087 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f70200a3dc0 0x7f70200a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:29.790 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.786+0000 7f70377fe640 1 -- 172.21.15.47:0/3881216616 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a4f54bc7b0 msgr2=0x55a4f54beba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:29.791 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.786+0000 7f70377fe640 1 --2- 172.21.15.47:0/3881216616 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a4f54bc7b0 0x55a4f54beba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:29.791 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.786+0000 7f70377fe640 1 -- 172.21.15.47:0/3881216616 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a4f54b39e0 msgr2=0x55a4f54bc270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:29.791 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.786+0000 7f70377fe640 1 --2- 172.21.15.47:0/3881216616 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a4f54b39e0 0x55a4f54bc270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:29.791 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.786+0000 7f70377fe640 1 -- 172.21.15.47:0/3881216616 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4f512b210 con 0x55a4f54b30c0 2024-04-17T06:10:29.791 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.786+0000 7f703cc57640 1 --2- 172.21.15.47:0/1096739087 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f70200a3dc0 0x7f70200a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:29.791 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.786+0000 7f70357fa640 1 -- 172.21.15.47:0/1096739087 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8469+0+0 (secure 0 0 0) 0x7f70300d0860 con 0x7f7038106fc0 2024-04-17T06:10:29.791 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.786+0000 7f70377fe640 1 --2- 172.21.15.47:0/3881216616 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a4f54b30c0 0x55a4f54b34a0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f702c005ec0 tx=0x7f702c0058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0b413c3ba2d8374 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:29.791 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.786+0000 7f701e7fc640 1 -- 172.21.15.47:0/3881216616 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f702c004020 con 0x55a4f54b30c0 2024-04-17T06:10:29.791 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.787+0000 7f701e7fc640 1 -- 172.21.15.47:0/3881216616 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f702c004ba0 con 0x55a4f54b30c0 2024-04-17T06:10:29.791 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.787+0000 7f701e7fc640 1 -- 172.21.15.47:0/3881216616 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f702c005450 con 0x55a4f54b30c0 2024-04-17T06:10:29.791 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.787+0000 7f7041376740 1 -- 172.21.15.47:0/3881216616 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a4f54b30c0 msgr2=0x55a4f54b34a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:29.791 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.787+0000 7f703cc57640 1 --2- 172.21.15.47:0/1096739087 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f70200a3dc0 0x7f70200a6280 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f703009c530 tx=0x7f7030087040 comp rx=0 tx=0).ready entity=mgr.24923 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:29.791 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.787+0000 7f7041376740 1 --2- 172.21.15.47:0/3881216616 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a4f54b30c0 0x55a4f54b34a0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f702c005ec0 tx=0x7f702c0058b0 comp rx=0 tx=0).stop 2024-04-17T06:10:29.791 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.787+0000 7f7041376740 1 -- 172.21.15.47:0/3881216616 shutdown_connections 2024-04-17T06:10:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.787+0000 7f7041376740 1 --2- 172.21.15.47:0/3881216616 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a4f54bc7b0 0x55a4f54beba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.787+0000 7f7041376740 1 --2- 172.21.15.47:0/3881216616 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a4f54b39e0 0x55a4f54bc270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.787+0000 7f7041376740 1 --2- 172.21.15.47:0/3881216616 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a4f54b30c0 0x55a4f54b34a0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.787+0000 7f7041376740 1 -- 172.21.15.47:0/3881216616 >> 172.21.15.47:0/3881216616 conn(0x55a4f54ad9e0 msgr2=0x55a4f54addc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.787+0000 7f7041376740 1 -- 172.21.15.47:0/3881216616 shutdown_connections 2024-04-17T06:10:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.787+0000 7f7041376740 1 -- 172.21.15.47:0/3881216616 wait complete. 2024-04-17T06:10:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.788+0000 7f7041376740 1 -- start start 2024-04-17T06:10:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.788+0000 7f7041376740 1 Processor -- start 2024-04-17T06:10:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.788+0000 7f7041376740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a4f54b30c0 0x55a4f54b9870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.788+0000 7f7041376740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a4f54b39e0 0x55a4f54b9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.788+0000 7f7041376740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a4f54bc7b0 0x55a4f54ba2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.788+0000 7f7041376740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55a4f54bf320 con 0x55a4f54b39e0 2024-04-17T06:10:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.788+0000 7f7041376740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55a4f53a0e90 con 0x55a4f54bc7b0 2024-04-17T06:10:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.788+0000 7f7041376740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55a4f54bf4a0 con 0x55a4f54b30c0 2024-04-17T06:10:29.793 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.788+0000 7f7041376740 10 client.0 _ll_register_callbacks cb 0x55a4f53b6d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:10:29.793 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.788+0000 7f7037fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a4f54b39e0 0x55a4f54b9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:29.793 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.788+0000 7f7037fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a4f54b39e0 0x55a4f54b9db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38000/0 (socket says 172.21.15.47:38000) 2024-04-17T06:10:29.793 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.788+0000 7f7037fff640 1 -- 172.21.15.47:0/3428573558 learned_addr learned my addr 172.21.15.47:0/3428573558 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:10:29.793 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.788+0000 7f70377fe640 1 --2- 172.21.15.47:0/3428573558 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a4f54bc7b0 0x55a4f54ba2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:29.793 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.788+0000 7f70377fe640 1 --2- 172.21.15.47:0/3428573558 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a4f54b30c0 0x55a4f54b9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:29.793 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.789+0000 7f7037fff640 1 -- 172.21.15.47:0/3428573558 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a4f54b30c0 msgr2=0x55a4f54b9870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:29.793 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.789+0000 7f7037fff640 1 --2- 172.21.15.47:0/3428573558 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a4f54b30c0 0x55a4f54b9870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:29.793 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.789+0000 7f7037fff640 1 -- 172.21.15.47:0/3428573558 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a4f54bc7b0 msgr2=0x55a4f54ba2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:29.793 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.789+0000 7f7037fff640 1 --2- 172.21.15.47:0/3428573558 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a4f54bc7b0 0x55a4f54ba2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:29.793 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.789+0000 7f7037fff640 1 -- 172.21.15.47:0/3428573558 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4f51f7620 con 0x55a4f54b39e0 2024-04-17T06:10:29.793 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.789+0000 7f7037fff640 1 --2- 172.21.15.47:0/3428573558 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a4f54b39e0 0x55a4f54b9db0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f7028010820 tx=0x7f7028010ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaeeacc98b6bcb45 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:29.793 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.789+0000 7f70177fe640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-17T06:10:29.793 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.789+0000 7f70177fe640 1 -- 172.21.15.47:0/3428573558 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f702800fb10 con 0x55a4f54b39e0 2024-04-17T06:10:29.793 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.789+0000 7f70177fe640 1 -- 172.21.15.47:0/3428573558 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8469+0+0 (secure 0 0 0) 0x7f702800fcb0 con 0x55a4f54b39e0 2024-04-17T06:10:29.794 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.789+0000 7f7041376740 1 -- 172.21.15.47:0/3428573558 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f702c004ba0 con 0x55a4f54b39e0 2024-04-17T06:10:29.794 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.789+0000 7f70177fe640 1 -- 172.21.15.47:0/3428573558 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f7028004020 con 0x55a4f54b39e0 2024-04-17T06:10:29.794 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.789+0000 7f7041376740 10 client.16167 fetch_fsmap learned FSMap version 34 2024-04-17T06:10:29.794 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.789+0000 7f7041376740 1 -- 172.21.15.47:0/3428573558 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55a4f54cb670 con 0x55a4f54b39e0 2024-04-17T06:10:29.794 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.789+0000 7f70177fe640 1 -- 172.21.15.47:0/3428573558 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f702c004ba0 con 0x55a4f54b39e0 2024-04-17T06:10:29.794 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.790+0000 7f70177fe640 1 -- 172.21.15.47:0/3428573558 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 34) ==== 8339+0+0 (secure 0 0 0) 0x7f7028019410 con 0x55a4f54b39e0 2024-04-17T06:10:29.794 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.790+0000 7f7041376740 10 client.16167 fetch_fsmap finished waiting for FSMap version 34 2024-04-17T06:10:29.794 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.790+0000 7f7041376740 10 client.16167 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-17T06:10:29.794 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.790+0000 7f7041376740 20 client.16167 populate_metadata read hostname 'smithi047' 2024-04-17T06:10:29.794 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.790+0000 7f7041376740 1 --2- 172.21.15.47:0/3428573558 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55a4f54ccbd0 0x55a4f54cefc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:29.794 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.790+0000 7f7041376740 4 client.16167 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T06:10:29.794 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.790+0000 7f7041376740 1 -- 172.21.15.47:0/3428573558 --> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a4f54261f0 con 0x55a4f54ccbd0 2024-04-17T06:10:29.794 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.790+0000 7f70377fe640 1 --2- 172.21.15.47:0/3428573558 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55a4f54ccbd0 0x55a4f54cefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:29.795 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.790+0000 7f70377fe640 1 --2- 172.21.15.47:0/3428573558 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55a4f54ccbd0 0x55a4f54cefc0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55c850afcd32c177 server_cookie=d1a4ded677b39509 in_seq=0 out_seq=0 2024-04-17T06:10:29.795 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.790+0000 7f70177fe640 10 client.16167 ms_handle_connect on v2:172.21.15.98:6838/2841287917 2024-04-17T06:10:29.795 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.791+0000 7f70177fe640 1 -- 172.21.15.47:0/3428573558 <== mds.0 v2:172.21.15.98:6838/2841287917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a4f54261f0 con 0x55a4f54ccbd0 2024-04-17T06:10:29.795 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.791+0000 7f70177fe640 10 client.16167 handle_command_reply: tid=0 2024-04-17T06:10:29.795 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.791+0000 7f7041376740 1 client.16167 shutdown 2024-04-17T06:10:29.795 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.791+0000 7f7041376740 1 -- 172.21.15.47:0/3428573558 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a4f54b39e0 msgr2=0x55a4f54b9db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:29.795 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.791+0000 7f7041376740 1 --2- 172.21.15.47:0/3428573558 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a4f54b39e0 0x55a4f54b9db0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f7028010820 tx=0x7f7028010ce0 comp rx=0 tx=0).stop 2024-04-17T06:10:29.795 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.791+0000 7f7041376740 1 -- 172.21.15.47:0/3428573558 shutdown_connections 2024-04-17T06:10:29.795 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.791+0000 7f7041376740 1 --2- 172.21.15.47:0/3428573558 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55a4f54ccbd0 0x55a4f54cefc0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:29.795 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.792+0000 7f7041376740 1 --2- 172.21.15.47:0/3428573558 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a4f54bc7b0 0x55a4f54ba2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:29.796 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.792+0000 7f70177fe640 0 client.16167 ms_handle_reset on v2:172.21.15.98:6838/2841287917 2024-04-17T06:10:29.796 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.792+0000 7f7041376740 1 --2- 172.21.15.47:0/3428573558 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a4f54b39e0 0x55a4f54b9db0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:29.796 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.792+0000 7f7041376740 1 --2- 172.21.15.47:0/3428573558 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a4f54b30c0 0x55a4f54b9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:29.796 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.792+0000 7f7041376740 1 -- 172.21.15.47:0/3428573558 >> 172.21.15.47:0/3428573558 conn(0x55a4f54ad9e0 msgr2=0x55a4f54af730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:29.796 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.792+0000 7f7041376740 1 -- 172.21.15.47:0/3428573558 shutdown_connections 2024-04-17T06:10:29.796 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.792+0000 7f7041376740 1 -- 172.21.15.47:0/3428573558 wait complete. 2024-04-17T06:10:29.796 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.792+0000 7f7041376740 20 client.16167 trim_cache size 0 max 16384 2024-04-17T06:10:29.809 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.807+0000 7f7041376740 1 Processor -- start 2024-04-17T06:10:29.809 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.807+0000 7f7041376740 1 -- start start 2024-04-17T06:10:29.809 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.807+0000 7f7041376740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a4f54b30c0 0x55a4f54c81e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:29.809 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.807+0000 7f7041376740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a4f54b39e0 0x55a4f54ba110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:29.809 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.807+0000 7f7041376740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a4f54bc7b0 0x55a4f54af4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:29.809 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.807+0000 7f7041376740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f70381b25b0 con 0x55a4f54b39e0 2024-04-17T06:10:29.810 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.807+0000 7f7041376740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f70381afe70 con 0x55a4f54b30c0 2024-04-17T06:10:29.810 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.807+0000 7f7041376740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f70381adcc0 con 0x55a4f54bc7b0 2024-04-17T06:10:29.810 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.807+0000 7f7037fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a4f54b39e0 0x55a4f54ba110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:29.810 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.807+0000 7f7037fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a4f54b39e0 0x55a4f54ba110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38002/0 (socket says 172.21.15.47:38002) 2024-04-17T06:10:29.810 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.807+0000 7f7037fff640 1 -- 172.21.15.47:0/967979972 learned_addr learned my addr 172.21.15.47:0/967979972 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:10:29.810 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.807+0000 7f70377fe640 1 --2- 172.21.15.47:0/967979972 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a4f54b30c0 0x55a4f54c81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:29.810 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.807+0000 7f7037fff640 1 -- 172.21.15.47:0/967979972 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a4f54bc7b0 msgr2=0x55a4f54af4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:29.810 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.807+0000 7f7037fff640 1 --2- 172.21.15.47:0/967979972 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a4f54bc7b0 0x55a4f54af4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:29.810 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.807+0000 7f7037fff640 1 -- 172.21.15.47:0/967979972 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a4f54b30c0 msgr2=0x55a4f54c81e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:29.810 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.807+0000 7f7037fff640 1 --2- 172.21.15.47:0/967979972 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a4f54b30c0 0x55a4f54c81e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:29.810 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.807+0000 7f7037fff640 1 -- 172.21.15.47:0/967979972 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4f54b7dc0 con 0x55a4f54b39e0 2024-04-17T06:10:29.810 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.807+0000 7f70377fe640 1 --2- 172.21.15.47:0/967979972 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a4f54b30c0 0x55a4f54c81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:10:29.810 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.808+0000 7f7037fff640 1 --2- 172.21.15.47:0/967979972 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a4f54b39e0 0x55a4f54ba110 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f702800acb0 tx=0x7f702801afa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5dc5cab55225a67 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:29.810 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.808+0000 7f701dffb640 1 -- 172.21.15.47:0/967979972 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7028030070 con 0x55a4f54b39e0 2024-04-17T06:10:29.811 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.808+0000 7f701dffb640 1 -- 172.21.15.47:0/967979972 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f702800a2c0 con 0x55a4f54b39e0 2024-04-17T06:10:29.811 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.808+0000 7f701dffb640 1 -- 172.21.15.47:0/967979972 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7028004490 con 0x55a4f54b39e0 2024-04-17T06:10:29.811 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.808+0000 7f7041376740 1 -- 172.21.15.47:0/967979972 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a4f54b39e0 msgr2=0x55a4f54ba110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:29.811 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.808+0000 7f7041376740 1 --2- 172.21.15.47:0/967979972 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a4f54b39e0 0x55a4f54ba110 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f702800acb0 tx=0x7f702801afa0 comp rx=0 tx=0).stop 2024-04-17T06:10:29.811 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.808+0000 7f7041376740 1 -- 172.21.15.47:0/967979972 shutdown_connections 2024-04-17T06:10:29.811 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.808+0000 7f7041376740 1 --2- 172.21.15.47:0/967979972 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a4f54bc7b0 0x55a4f54af4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:29.811 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.808+0000 7f7041376740 1 --2- 172.21.15.47:0/967979972 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a4f54b39e0 0x55a4f54ba110 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:29.811 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.808+0000 7f7041376740 1 --2- 172.21.15.47:0/967979972 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a4f54b30c0 0x55a4f54c81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:29.811 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.808+0000 7f7041376740 1 -- 172.21.15.47:0/967979972 >> 172.21.15.47:0/967979972 conn(0x55a4f54ad9e0 msgr2=0x55a4f54aedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:29.811 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.808+0000 7f7041376740 1 -- 172.21.15.47:0/967979972 shutdown_connections 2024-04-17T06:10:29.811 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.808+0000 7f7041376740 1 -- 172.21.15.47:0/967979972 wait complete. 2024-04-17T06:10:29.811 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.809+0000 7f7041376740 1 -- start start 2024-04-17T06:10:29.811 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.809+0000 7f7041376740 1 Processor -- start 2024-04-17T06:10:29.811 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.809+0000 7f7041376740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a4f54b30c0 0x55a4f54b6790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:29.811 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.809+0000 7f7041376740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a4f54b39e0 0x55a4f54b6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:29.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.809+0000 7f7041376740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a4f54bc7b0 0x55a4f54b7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:29.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.809+0000 7f7041376740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f70381ae330 con 0x55a4f54b30c0 2024-04-17T06:10:29.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.809+0000 7f7041376740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f70381af7a0 con 0x55a4f54bc7b0 2024-04-17T06:10:29.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.809+0000 7f7041376740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f70381aea00 con 0x55a4f54b39e0 2024-04-17T06:10:29.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.809+0000 7f7041376740 10 client.0 _ll_register_callbacks cb 0x7f702800a720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:10:29.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.809+0000 7f70377fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a4f54b30c0 0x55a4f54b6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:29.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.809+0000 7f70377fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a4f54b30c0 0x55a4f54b6790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38004/0 (socket says 172.21.15.47:38004) 2024-04-17T06:10:29.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.809+0000 7f70377fe640 1 -- 172.21.15.47:0/2335962523 learned_addr learned my addr 172.21.15.47:0/2335962523 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:10:29.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.810+0000 7f7037fff640 1 --2- 172.21.15.47:0/2335962523 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a4f54b39e0 0x55a4f54b6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:29.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.810+0000 7f70377fe640 1 -- 172.21.15.47:0/2335962523 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a4f54b39e0 msgr2=0x55a4f54b6cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:29.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.810+0000 7f70377fe640 1 --2- 172.21.15.47:0/2335962523 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a4f54b39e0 0x55a4f54b6cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:29.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.810+0000 7f70377fe640 1 -- 172.21.15.47:0/2335962523 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a4f54bc7b0 msgr2=0x55a4f54b7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:29.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.810+0000 7f70377fe640 1 --2- 172.21.15.47:0/2335962523 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a4f54bc7b0 0x55a4f54b7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:29.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.810+0000 7f70377fe640 1 -- 172.21.15.47:0/2335962523 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4f5532340 con 0x55a4f54b30c0 2024-04-17T06:10:29.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.810+0000 7f70377fe640 1 --2- 172.21.15.47:0/2335962523 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a4f54b30c0 0x55a4f54b6790 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f702c02c1c0 tx=0x7f702c005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c82b08b361e7bc33 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:29.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.810+0000 7f70177fe640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-17T06:10:29.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.810+0000 7f70177fe640 1 -- 172.21.15.47:0/2335962523 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f702c004250 con 0x55a4f54b30c0 2024-04-17T06:10:29.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.810+0000 7f70177fe640 1 -- 172.21.15.47:0/2335962523 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8469+0+0 (secure 0 0 0) 0x7f702c0043f0 con 0x55a4f54b30c0 2024-04-17T06:10:29.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.810+0000 7f7041376740 1 -- 172.21.15.47:0/2335962523 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f702800a2c0 con 0x55a4f54b30c0 2024-04-17T06:10:29.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.810+0000 7f70177fe640 1 -- 172.21.15.47:0/2335962523 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f702c005190 con 0x55a4f54b30c0 2024-04-17T06:10:29.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.810+0000 7f7041376740 10 client.16173 fetch_fsmap learned FSMap version 34 2024-04-17T06:10:29.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.810+0000 7f7041376740 1 -- 172.21.15.47:0/2335962523 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55a4f55335a0 con 0x55a4f54b30c0 2024-04-17T06:10:29.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.810+0000 7f70177fe640 1 -- 172.21.15.47:0/2335962523 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f702800a2c0 con 0x55a4f54b30c0 2024-04-17T06:10:29.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.811+0000 7f70177fe640 1 -- 172.21.15.47:0/2335962523 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 34) ==== 8339+0+0 (secure 0 0 0) 0x7f702c053250 con 0x55a4f54b30c0 2024-04-17T06:10:29.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.811+0000 7f7041376740 10 client.16173 fetch_fsmap finished waiting for FSMap version 34 2024-04-17T06:10:29.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.811+0000 7f7041376740 10 client.16173 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-17T06:10:29.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.811+0000 7f7041376740 20 client.16173 populate_metadata read hostname 'smithi047' 2024-04-17T06:10:29.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.811+0000 7f7041376740 1 --2- 172.21.15.47:0/2335962523 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55a4f5533820 0x55a4f5535c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:29.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.811+0000 7f7041376740 4 client.16173 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-17T06:10:29.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.811+0000 7f7041376740 1 -- 172.21.15.47:0/2335962523 --> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a4f5420060 con 0x55a4f5533820 2024-04-17T06:10:29.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.811+0000 7f7037fff640 1 --2- 172.21.15.47:0/2335962523 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55a4f5533820 0x55a4f5535c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:29.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.811+0000 7f7037fff640 1 --2- 172.21.15.47:0/2335962523 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55a4f5533820 0x55a4f5535c10 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=525e909a3355f59a server_cookie=77cd5e013879b411 in_seq=0 out_seq=0 2024-04-17T06:10:29.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.811+0000 7f70177fe640 10 client.16173 ms_handle_connect on v2:172.21.15.98:6838/2841287917 2024-04-17T06:10:29.818 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.819+0000 7f70177fe640 1 -- 172.21.15.47:0/2335962523 <== mds.0 v2:172.21.15.98:6838/2841287917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a4f5420060 con 0x55a4f5533820 2024-04-17T06:10:29.819 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.819+0000 7f70177fe640 10 client.16173 handle_command_reply: tid=0 2024-04-17T06:10:29.819 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.819+0000 7f7041376740 1 client.16173 shutdown 2024-04-17T06:10:29.819 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.820+0000 7f7041376740 1 -- 172.21.15.47:0/2335962523 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a4f54b30c0 msgr2=0x55a4f54b6790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:29.819 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.820+0000 7f7041376740 1 --2- 172.21.15.47:0/2335962523 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a4f54b30c0 0x55a4f54b6790 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f702c02c1c0 tx=0x7f702c005030 comp rx=0 tx=0).stop 2024-04-17T06:10:29.819 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.820+0000 7f7041376740 1 -- 172.21.15.47:0/2335962523 shutdown_connections 2024-04-17T06:10:29.819 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.820+0000 7f7041376740 1 --2- 172.21.15.47:0/2335962523 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55a4f5533820 0x55a4f5535c10 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:29.819 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.820+0000 7f7041376740 1 --2- 172.21.15.47:0/2335962523 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a4f54bc7b0 0x55a4f54b7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:29.819 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.820+0000 7f7041376740 1 --2- 172.21.15.47:0/2335962523 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a4f54b39e0 0x55a4f54b6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:29.819 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.820+0000 7f70177fe640 0 client.16173 ms_handle_reset on v2:172.21.15.98:6838/2841287917 2024-04-17T06:10:29.819 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.820+0000 7f7041376740 1 --2- 172.21.15.47:0/2335962523 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a4f54b30c0 0x55a4f54b6790 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:29.819 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.820+0000 7f7041376740 1 -- 172.21.15.47:0/2335962523 >> 172.21.15.47:0/2335962523 conn(0x55a4f54ad9e0 msgr2=0x55a4f54acb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:29.819 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.820+0000 7f7041376740 1 -- 172.21.15.47:0/2335962523 shutdown_connections 2024-04-17T06:10:29.819 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.820+0000 7f7041376740 1 -- 172.21.15.47:0/2335962523 wait complete. 2024-04-17T06:10:29.819 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.820+0000 7f7041376740 20 client.16173 trim_cache size 0 max 16384 2024-04-17T06:10:29.820 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-04-17T06:10:29.820 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-04-17T06:10:29.820 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-04-17T06:10:29.820 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-04-17T06:10:29.821 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.820+0000 7f703e6ba640 1 -- 172.21.15.47:0/1096739087 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f70200a3dc0 msgr2=0x7f70200a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:29.821 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.820+0000 7f703e6ba640 1 --2- 172.21.15.47:0/1096739087 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f70200a3dc0 0x7f70200a6280 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f703009c530 tx=0x7f7030087040 comp rx=0 tx=0).stop 2024-04-17T06:10:29.821 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.820+0000 7f703e6ba640 1 -- 172.21.15.47:0/1096739087 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7038106fc0 msgr2=0x7f70381035a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:29.821 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.820+0000 7f703e6ba640 1 --2- 172.21.15.47:0/1096739087 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7038106fc0 0x7f70381035a0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f703000cad0 tx=0x7f703000cf90 comp rx=0 tx=0).stop 2024-04-17T06:10:29.821 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.821+0000 7f703e6ba640 1 -- 172.21.15.47:0/1096739087 shutdown_connections 2024-04-17T06:10:29.821 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.821+0000 7f703e6ba640 1 --2- 172.21.15.47:0/1096739087 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f70200a3dc0 0x7f70200a6280 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:29.821 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.821+0000 7f703e6ba640 1 --2- 172.21.15.47:0/1096739087 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7038106fc0 0x7f70381035a0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:29.821 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.821+0000 7f703e6ba640 1 --2- 172.21.15.47:0/1096739087 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7038101b70 0x7f7038103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:29.821 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.821+0000 7f703e6ba640 1 --2- 172.21.15.47:0/1096739087 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f70381011b0 0x7f7038102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:29.821 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.821+0000 7f703e6ba640 1 -- 172.21.15.47:0/1096739087 >> 172.21.15.47:0/1096739087 conn(0x7f7038076c60 msgr2=0x7f70381004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:29.822 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.821+0000 7f703e6ba640 1 -- 172.21.15.47:0/1096739087 shutdown_connections 2024-04-17T06:10:29.822 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:29.821+0000 7f703e6ba640 1 -- 172.21.15.47:0/1096739087 wait complete. 2024-04-17T06:10:30.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:10:29 smithi098 ceph-mon[36516]: pgmap v288: 97 pgs: 97 active+clean; 4.6 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-04-17T06:10:30.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:10:29 smithi047 ceph-mon[32223]: pgmap v288: 97 pgs: 97 active+clean; 4.6 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-04-17T06:10:30.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:10:29 smithi103 ceph-mon[36243]: pgmap v288: 97 pgs: 97 active+clean; 4.6 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-04-17T06:10:32.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:10:31 smithi098 ceph-mon[36516]: pgmap v289: 97 pgs: 97 active+clean; 4.6 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-04-17T06:10:32.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:10:31 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T06:10:32.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:10:31 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T06:10:32.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:10:31 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T06:10:32.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:10:31 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:10:32.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:10:31 smithi047 ceph-mon[32223]: pgmap v289: 97 pgs: 97 active+clean; 4.6 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-04-17T06:10:32.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:10:31 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T06:10:32.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:10:31 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T06:10:32.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:10:31 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T06:10:32.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:10:31 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:10:32.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:10:31 smithi103 ceph-mon[36243]: pgmap v289: 97 pgs: 97 active+clean; 4.6 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-04-17T06:10:32.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:10:31 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T06:10:32.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:10:31 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T06:10:32.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:10:31 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T06:10:32.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:10:31 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:10:34.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:10:33 smithi098 ceph-mon[36516]: pgmap v290: 97 pgs: 97 active+clean; 4.6 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.6 KiB/s wr, 9 op/s 2024-04-17T06:10:34.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:10:33 smithi047 ceph-mon[32223]: pgmap v290: 97 pgs: 97 active+clean; 4.6 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.6 KiB/s wr, 9 op/s 2024-04-17T06:10:34.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:10:33 smithi103 ceph-mon[36243]: pgmap v290: 97 pgs: 97 active+clean; 4.6 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.6 KiB/s wr, 9 op/s 2024-04-17T06:10:36.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:10:35 smithi098 ceph-mon[36516]: pgmap v291: 97 pgs: 97 active+clean; 4.7 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-04-17T06:10:36.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:10:35 smithi047 ceph-mon[32223]: pgmap v291: 97 pgs: 97 active+clean; 4.7 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-04-17T06:10:36.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:10:35 smithi103 ceph-mon[36243]: pgmap v291: 97 pgs: 97 active+clean; 4.7 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-04-17T06:10:38.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:10:37 smithi098 ceph-mon[36516]: pgmap v292: 97 pgs: 97 active+clean; 4.7 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:10:38.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:10:37 smithi047 ceph-mon[32223]: pgmap v292: 97 pgs: 97 active+clean; 4.7 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:10:38.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:10:37 smithi103 ceph-mon[36243]: pgmap v292: 97 pgs: 97 active+clean; 4.7 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:10:40.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:10:39 smithi047 ceph-mon[32223]: pgmap v293: 97 pgs: 97 active+clean; 4.7 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:10:40.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:10:39 smithi103 ceph-mon[36243]: pgmap v293: 97 pgs: 97 active+clean; 4.7 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:10:40.382 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:10:39 smithi098 ceph-mon[36516]: pgmap v293: 97 pgs: 97 active+clean; 4.7 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:10:42.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:10:41 smithi047 ceph-mon[32223]: pgmap v294: 97 pgs: 97 active+clean; 4.7 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-04-17T06:10:42.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:10:41 smithi103 ceph-mon[36243]: pgmap v294: 97 pgs: 97 active+clean; 4.7 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-04-17T06:10:42.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:10:41 smithi098 ceph-mon[36516]: pgmap v294: 97 pgs: 97 active+clean; 4.7 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-04-17T06:10:42.922 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.922+0000 7fc3debe8640 1 -- 172.21.15.47:0/2698560194 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc3d806b130 msgr2=0x7fc3d810ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:42.922 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.922+0000 7fc3debe8640 1 --2- 172.21.15.47:0/2698560194 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc3d806b130 0x7fc3d810ee00 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fc3d0009ca0 tx=0x7fc3d002c180 comp rx=0 tx=0).stop 2024-04-17T06:10:42.922 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.922+0000 7fc3debe8640 1 -- 172.21.15.47:0/2698560194 shutdown_connections 2024-04-17T06:10:42.922 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.922+0000 7fc3debe8640 1 --2- 172.21.15.47:0/2698560194 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc3d806b130 0x7fc3d810ee00 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:42.922 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.922+0000 7fc3debe8640 1 --2- 172.21.15.47:0/2698560194 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc3d806a790 0x7fc3d806abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:42.922 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.922+0000 7fc3debe8640 1 --2- 172.21.15.47:0/2698560194 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc3d81080f0 0x7fc3d81084d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:42.922 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.922+0000 7fc3debe8640 1 -- 172.21.15.47:0/2698560194 >> 172.21.15.47:0/2698560194 conn(0x7fc3d8076c60 msgr2=0x7fc3d8077070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:42.922 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.922+0000 7fc3debe8640 1 -- 172.21.15.47:0/2698560194 shutdown_connections 2024-04-17T06:10:42.923 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.922+0000 7fc3debe8640 1 -- 172.21.15.47:0/2698560194 wait complete. 2024-04-17T06:10:42.923 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.922+0000 7fc3debe8640 1 Processor -- start 2024-04-17T06:10:42.923 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.922+0000 7fc3debe8640 1 -- start start 2024-04-17T06:10:42.923 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.922+0000 7fc3debe8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc3d806a790 0x7fc3d8101280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:42.923 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.922+0000 7fc3debe8640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc3d806b130 0x7fc3d81017c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:42.923 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.923+0000 7fc3debe8640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc3d81080f0 0x7fc3d8101d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:42.923 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.923+0000 7fc3debe8640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fc3d81122f0 con 0x7fc3d806b130 2024-04-17T06:10:42.923 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.923+0000 7fc3debe8640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc3d8112170 con 0x7fc3d81080f0 2024-04-17T06:10:42.923 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.923+0000 7fc3debe8640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc3d8112470 con 0x7fc3d806a790 2024-04-17T06:10:42.923 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.923+0000 7fc3d7fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc3d806b130 0x7fc3d81017c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:42.923 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.923+0000 7fc3d7fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc3d806b130 0x7fc3d81017c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:36444/0 (socket says 172.21.15.47:36444) 2024-04-17T06:10:42.923 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.923+0000 7fc3d7fff640 1 -- 172.21.15.47:0/1204360628 learned_addr learned my addr 172.21.15.47:0/1204360628 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:10:42.923 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.923+0000 7fc3dc984640 1 --2- 172.21.15.47:0/1204360628 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc3d806a790 0x7fc3d8101280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:42.923 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.923+0000 7fc3dd185640 1 --2- 172.21.15.47:0/1204360628 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc3d81080f0 0x7fc3d8101d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.923+0000 7fc3d7fff640 1 -- 172.21.15.47:0/1204360628 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc3d806a790 msgr2=0x7fc3d8101280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.923+0000 7fc3d7fff640 1 --2- 172.21.15.47:0/1204360628 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc3d806a790 0x7fc3d8101280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.923+0000 7fc3d7fff640 1 -- 172.21.15.47:0/1204360628 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc3d81080f0 msgr2=0x7fc3d8101d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.923+0000 7fc3d7fff640 1 --2- 172.21.15.47:0/1204360628 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc3d81080f0 0x7fc3d8101d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.923+0000 7fc3d7fff640 1 -- 172.21.15.47:0/1204360628 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3d81a6b70 con 0x7fc3d806b130 2024-04-17T06:10:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.923+0000 7fc3dd185640 1 --2- 172.21.15.47:0/1204360628 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc3d81080f0 0x7fc3d8101d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:10:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.923+0000 7fc3d7fff640 1 --2- 172.21.15.47:0/1204360628 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc3d806b130 0x7fc3d81017c0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fc3c400d950 tx=0x7fc3c400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.923+0000 7fc3d5ffb640 1 -- 172.21.15.47:0/1204360628 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3c4017070 con 0x7fc3d806b130 2024-04-17T06:10:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.923+0000 7fc3d5ffb640 1 -- 172.21.15.47:0/1204360628 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3c4009c30 con 0x7fc3d806b130 2024-04-17T06:10:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.923+0000 7fc3debe8640 1 -- 172.21.15.47:0/1204360628 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3d81a6e60 con 0x7fc3d806b130 2024-04-17T06:10:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.923+0000 7fc3d5ffb640 1 -- 172.21.15.47:0/1204360628 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3c400f6b0 con 0x7fc3d806b130 2024-04-17T06:10:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.924+0000 7fc3debe8640 1 -- 172.21.15.47:0/1204360628 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3d81a7320 con 0x7fc3d806b130 2024-04-17T06:10:42.932 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.926+0000 7fc3d5ffb640 1 -- 172.21.15.47:0/1204360628 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc3c40109d0 con 0x7fc3d806b130 2024-04-17T06:10:42.932 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.926+0000 7fc3e17cc740 1 Processor -- start 2024-04-17T06:10:42.932 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.926+0000 7fc3e17cc740 1 -- start start 2024-04-17T06:10:42.932 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.926+0000 7fc3d5ffb640 1 --2- 172.21.15.47:0/1204360628 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fc3c80a3dc0 0x7fc3c80a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:42.932 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.926+0000 7fc3e17cc740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a134b090c0 0x55a134b094a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:42.932 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.926+0000 7fc3e17cc740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a134b099e0 0x55a134b12270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:42.932 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.926+0000 7fc3e17cc740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a134b127b0 0x55a134b14ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:42.932 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.926+0000 7fc3e17cc740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55a134a6e7d0 con 0x55a134b127b0 2024-04-17T06:10:42.932 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.926+0000 7fc3e17cc740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55a134a0bf70 con 0x55a134b099e0 2024-04-17T06:10:42.933 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.926+0000 7fc3e17cc740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55a134a6a790 con 0x55a134b090c0 2024-04-17T06:10:42.933 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.927+0000 7fc3d5ffb640 1 -- 172.21.15.47:0/1204360628 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8469+0+0 (secure 0 0 0) 0x7fc3c40d0490 con 0x7fc3d806b130 2024-04-17T06:10:42.933 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.927+0000 7fc3dd185640 1 --2- 172.21.15.47:0/1204360628 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fc3c80a3dc0 0x7fc3c80a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:42.933 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.927+0000 7fc3dc984640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a134b090c0 0x55a134b094a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:42.933 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.927+0000 7fc3dd185640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a134b127b0 0x55a134b14ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:42.933 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.927+0000 7fc3dc984640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a134b090c0 0x55a134b094a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.47:47428/0 (socket says 172.21.15.47:47428) 2024-04-17T06:10:42.933 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.927+0000 7fc3dc984640 1 -- 172.21.15.47:0/4122348083 learned_addr learned my addr 172.21.15.47:0/4122348083 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:10:42.933 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.927+0000 7fc3d7fff640 1 --2- 172.21.15.47:0/4122348083 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a134b099e0 0x55a134b12270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:42.933 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.927+0000 7fc3dc984640 1 -- 172.21.15.47:0/4122348083 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a134b099e0 msgr2=0x55a134b12270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:42.933 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.927+0000 7fc3dc984640 1 --2- 172.21.15.47:0/4122348083 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a134b099e0 0x55a134b12270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:42.933 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.927+0000 7fc3dc984640 1 -- 172.21.15.47:0/4122348083 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a134b127b0 msgr2=0x55a134b14ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:42.933 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.927+0000 7fc3dc984640 1 --2- 172.21.15.47:0/4122348083 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a134b127b0 0x55a134b14ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:42.933 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.927+0000 7fc3dc984640 1 -- 172.21.15.47:0/4122348083 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a134781210 con 0x55a134b090c0 2024-04-17T06:10:42.933 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.927+0000 7fc3dd185640 1 --2- 172.21.15.47:0/1204360628 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fc3c80a3dc0 0x7fc3c80a6280 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc3d0009a60 tx=0x7fc3d00323c0 comp rx=0 tx=0).ready entity=mgr.24923 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:42.934 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.927+0000 7fc3dc984640 1 --2- 172.21.15.47:0/4122348083 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a134b090c0 0x55a134b094a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc3cc009c00 tx=0x7fc3cc0117b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc613ab035271d89 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:42.934 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.928+0000 7fc3c2ffd640 1 -- 172.21.15.47:0/4122348083 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3cc012450 con 0x55a134b090c0 2024-04-17T06:10:42.934 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.928+0000 7fc3c2ffd640 1 -- 172.21.15.47:0/4122348083 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3cc012ad0 con 0x55a134b090c0 2024-04-17T06:10:42.934 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.928+0000 7fc3c2ffd640 1 -- 172.21.15.47:0/4122348083 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3cc00fb10 con 0x55a134b090c0 2024-04-17T06:10:42.934 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.928+0000 7fc3e17cc740 1 -- 172.21.15.47:0/4122348083 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a134b090c0 msgr2=0x55a134b094a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:42.934 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.928+0000 7fc3e17cc740 1 --2- 172.21.15.47:0/4122348083 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a134b090c0 0x55a134b094a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc3cc009c00 tx=0x7fc3cc0117b0 comp rx=0 tx=0).stop 2024-04-17T06:10:42.934 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.928+0000 7fc3e17cc740 1 -- 172.21.15.47:0/4122348083 shutdown_connections 2024-04-17T06:10:42.934 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.928+0000 7fc3e17cc740 1 --2- 172.21.15.47:0/4122348083 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a134b127b0 0x55a134b14ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:42.934 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.928+0000 7fc3e17cc740 1 --2- 172.21.15.47:0/4122348083 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a134b099e0 0x55a134b12270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:42.934 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.928+0000 7fc3e17cc740 1 --2- 172.21.15.47:0/4122348083 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a134b090c0 0x55a134b094a0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:42.934 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.929+0000 7fc3e17cc740 1 -- 172.21.15.47:0/4122348083 >> 172.21.15.47:0/4122348083 conn(0x55a134b039e0 msgr2=0x55a134b03dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:42.934 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.929+0000 7fc3e17cc740 1 -- 172.21.15.47:0/4122348083 shutdown_connections 2024-04-17T06:10:42.934 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.929+0000 7fc3e17cc740 1 -- 172.21.15.47:0/4122348083 wait complete. 2024-04-17T06:10:42.934 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.929+0000 7fc3e17cc740 1 -- start start 2024-04-17T06:10:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.929+0000 7fc3e17cc740 1 Processor -- start 2024-04-17T06:10:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.930+0000 7fc3e17cc740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a134b090c0 0x55a134b0f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.930+0000 7fc3e17cc740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a134b099e0 0x55a134b0fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.930+0000 7fc3e17cc740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a134b127b0 0x55a134b10230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.930+0000 7fc3e17cc740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55a134b153e0 con 0x55a134b090c0 2024-04-17T06:10:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.930+0000 7fc3e17cc740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55a1349f6e90 con 0x55a134b127b0 2024-04-17T06:10:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.930+0000 7fc3e17cc740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55a134b15560 con 0x55a134b099e0 2024-04-17T06:10:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.930+0000 7fc3dc984640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a134b090c0 0x55a134b0f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.930+0000 7fc3e17cc740 10 client.0 _ll_register_callbacks cb 0x55a134a0cd40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:10:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.930+0000 7fc3dc984640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a134b090c0 0x55a134b0f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:36456/0 (socket says 172.21.15.47:36456) 2024-04-17T06:10:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.930+0000 7fc3dc984640 1 -- 172.21.15.47:0/2552234099 learned_addr learned my addr 172.21.15.47:0/2552234099 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:10:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.930+0000 7fc3dd185640 1 --2- 172.21.15.47:0/2552234099 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a134b127b0 0x55a134b10230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.930+0000 7fc3d7fff640 1 --2- 172.21.15.47:0/2552234099 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a134b099e0 0x55a134b0fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.930+0000 7fc3dc984640 1 -- 172.21.15.47:0/2552234099 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a134b099e0 msgr2=0x55a134b0fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:42.936 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.930+0000 7fc3dc984640 1 --2- 172.21.15.47:0/2552234099 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a134b099e0 0x55a134b0fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:42.936 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.930+0000 7fc3dc984640 1 -- 172.21.15.47:0/2552234099 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a134b127b0 msgr2=0x55a134b10230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:42.936 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.930+0000 7fc3dc984640 1 --2- 172.21.15.47:0/2552234099 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a134b127b0 0x55a134b10230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:42.936 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.930+0000 7fc3dc984640 1 -- 172.21.15.47:0/2552234099 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a13484d620 con 0x55a134b090c0 2024-04-17T06:10:42.936 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.930+0000 7fc3dc984640 1 --2- 172.21.15.47:0/2552234099 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a134b090c0 0x55a134b0f7b0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fc3cc01c040 tx=0x7fc3cc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67ab93ab442ae303 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:42.936 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.930+0000 7fc3b7fff640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-17T06:10:42.936 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.931+0000 7fc3b7fff640 1 -- 172.21.15.47:0/2552234099 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3cc029020 con 0x55a134b090c0 2024-04-17T06:10:42.936 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.931+0000 7fc3b7fff640 1 -- 172.21.15.47:0/2552234099 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8469+0+0 (secure 0 0 0) 0x7fc3cc012450 con 0x55a134b090c0 2024-04-17T06:10:42.936 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.931+0000 7fc3e17cc740 1 -- 172.21.15.47:0/2552234099 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc3cc012ad0 con 0x55a134b090c0 2024-04-17T06:10:42.936 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.931+0000 7fc3b7fff640 1 -- 172.21.15.47:0/2552234099 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fc3cc034050 con 0x55a134b090c0 2024-04-17T06:10:42.936 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.931+0000 7fc3e17cc740 10 client.16185 fetch_fsmap learned FSMap version 34 2024-04-17T06:10:42.936 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.931+0000 7fc3e17cc740 1 -- 172.21.15.47:0/2552234099 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55a134b21600 con 0x55a134b090c0 2024-04-17T06:10:42.936 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.931+0000 7fc3b7fff640 1 -- 172.21.15.47:0/2552234099 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3cc012ad0 con 0x55a134b090c0 2024-04-17T06:10:42.936 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.931+0000 7fc3b7fff640 1 -- 172.21.15.47:0/2552234099 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 34) ==== 8339+0+0 (secure 0 0 0) 0x7fc3cc0035f0 con 0x55a134b090c0 2024-04-17T06:10:42.937 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.931+0000 7fc3e17cc740 10 client.16185 fetch_fsmap finished waiting for FSMap version 34 2024-04-17T06:10:42.937 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.931+0000 7fc3e17cc740 10 client.16185 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-17T06:10:42.937 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.931+0000 7fc3e17cc740 20 client.16185 populate_metadata read hostname 'smithi047' 2024-04-17T06:10:42.937 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.931+0000 7fc3e17cc740 1 --2- 172.21.15.47:0/2552234099 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55a134b22ac0 0x55a134b24eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:42.937 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.931+0000 7fc3e17cc740 4 client.16185 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T06:10:42.937 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.931+0000 7fc3e17cc740 1 -- 172.21.15.47:0/2552234099 --> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a134a7c1f0 con 0x55a134b22ac0 2024-04-17T06:10:42.937 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.932+0000 7fc3d7fff640 1 --2- 172.21.15.47:0/2552234099 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55a134b22ac0 0x55a134b24eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:42.937 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.932+0000 7fc3d7fff640 1 --2- 172.21.15.47:0/2552234099 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55a134b22ac0 0x55a134b24eb0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ab974eb8071a1d2 server_cookie=b3163c54a54f72d0 in_seq=0 out_seq=0 2024-04-17T06:10:42.937 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.932+0000 7fc3b7fff640 10 client.16185 ms_handle_connect on v2:172.21.15.98:6838/2841287917 2024-04-17T06:10:42.937 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.933+0000 7fc3b7fff640 1 -- 172.21.15.47:0/2552234099 <== mds.0 v2:172.21.15.98:6838/2841287917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a134a7c1f0 con 0x55a134b22ac0 2024-04-17T06:10:42.937 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.933+0000 7fc3b7fff640 10 client.16185 handle_command_reply: tid=0 2024-04-17T06:10:42.937 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.933+0000 7fc3e17cc740 1 client.16185 shutdown 2024-04-17T06:10:42.937 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.933+0000 7fc3e17cc740 1 -- 172.21.15.47:0/2552234099 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a134b090c0 msgr2=0x55a134b0f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:42.937 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.933+0000 7fc3e17cc740 1 --2- 172.21.15.47:0/2552234099 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a134b090c0 0x55a134b0f7b0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fc3cc01c040 tx=0x7fc3cc004270 comp rx=0 tx=0).stop 2024-04-17T06:10:42.938 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.933+0000 7fc3e17cc740 1 -- 172.21.15.47:0/2552234099 shutdown_connections 2024-04-17T06:10:42.938 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.933+0000 7fc3e17cc740 1 --2- 172.21.15.47:0/2552234099 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55a134b22ac0 0x55a134b24eb0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:42.938 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.933+0000 7fc3e17cc740 1 --2- 172.21.15.47:0/2552234099 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a134b127b0 0x55a134b10230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:42.938 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.933+0000 7fc3e17cc740 1 --2- 172.21.15.47:0/2552234099 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a134b099e0 0x55a134b0fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:42.938 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.933+0000 7fc3e17cc740 1 --2- 172.21.15.47:0/2552234099 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a134b090c0 0x55a134b0f7b0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:42.938 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.933+0000 7fc3b7fff640 0 client.16185 ms_handle_reset on v2:172.21.15.98:6838/2841287917 2024-04-17T06:10:42.938 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.933+0000 7fc3e17cc740 1 -- 172.21.15.47:0/2552234099 >> 172.21.15.47:0/2552234099 conn(0x55a134b039e0 msgr2=0x55a134b05730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:42.938 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.933+0000 7fc3e17cc740 1 -- 172.21.15.47:0/2552234099 shutdown_connections 2024-04-17T06:10:42.938 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.933+0000 7fc3e17cc740 1 -- 172.21.15.47:0/2552234099 wait complete. 2024-04-17T06:10:42.938 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.933+0000 7fc3e17cc740 20 client.16185 trim_cache size 0 max 16384 2024-04-17T06:10:42.951 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.948+0000 7fc3e17cc740 1 Processor -- start 2024-04-17T06:10:42.951 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.948+0000 7fc3e17cc740 1 -- start start 2024-04-17T06:10:42.951 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.948+0000 7fc3e17cc740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a134b090c0 0x55a134b11ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:42.951 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.948+0000 7fc3e17cc740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a134b099e0 0x55a134b1e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:42.951 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.948+0000 7fc3e17cc740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a134b127b0 0x55a134b0a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:42.951 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.948+0000 7fc3e17cc740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fc3d81b2560 con 0x55a134b090c0 2024-04-17T06:10:42.951 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.948+0000 7fc3e17cc740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc3d81afe20 con 0x55a134b099e0 2024-04-17T06:10:42.951 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.948+0000 7fc3e17cc740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc3d81adc70 con 0x55a134b127b0 2024-04-17T06:10:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.949+0000 7fc3dc984640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a134b090c0 0x55a134b11ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.949+0000 7fc3dc984640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a134b090c0 0x55a134b11ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:36466/0 (socket says 172.21.15.47:36466) 2024-04-17T06:10:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.949+0000 7fc3dc984640 1 -- 172.21.15.47:0/2916208715 learned_addr learned my addr 172.21.15.47:0/2916208715 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:10:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.949+0000 7fc3dd185640 1 --2- 172.21.15.47:0/2916208715 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a134b127b0 0x55a134b0a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.949+0000 7fc3dc984640 1 -- 172.21.15.47:0/2916208715 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a134b127b0 msgr2=0x55a134b0a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.949+0000 7fc3dc984640 1 --2- 172.21.15.47:0/2916208715 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a134b127b0 0x55a134b0a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.949+0000 7fc3dc984640 1 -- 172.21.15.47:0/2916208715 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a134b099e0 msgr2=0x55a134b1e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.949+0000 7fc3dc984640 1 --2- 172.21.15.47:0/2916208715 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a134b099e0 0x55a134b1e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.949+0000 7fc3dc984640 1 -- 172.21.15.47:0/2916208715 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a134b14ba0 con 0x55a134b090c0 2024-04-17T06:10:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.949+0000 7fc3dc984640 1 --2- 172.21.15.47:0/2916208715 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a134b090c0 0x55a134b11ee0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fc3cc0039c0 tx=0x7fc3cc005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a091341237a0f518 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.949+0000 7fc3c27fc640 1 -- 172.21.15.47:0/2916208715 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3cc03d070 con 0x55a134b090c0 2024-04-17T06:10:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.949+0000 7fc3c27fc640 1 -- 172.21.15.47:0/2916208715 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3cc011cb0 con 0x55a134b090c0 2024-04-17T06:10:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.949+0000 7fc3c27fc640 1 -- 172.21.15.47:0/2916208715 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3cc029040 con 0x55a134b090c0 2024-04-17T06:10:42.953 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.950+0000 7fc3e17cc740 1 -- 172.21.15.47:0/2916208715 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a134b090c0 msgr2=0x55a134b11ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:42.953 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.950+0000 7fc3e17cc740 1 --2- 172.21.15.47:0/2916208715 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a134b090c0 0x55a134b11ee0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fc3cc0039c0 tx=0x7fc3cc005f90 comp rx=0 tx=0).stop 2024-04-17T06:10:42.953 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.950+0000 7fc3e17cc740 1 -- 172.21.15.47:0/2916208715 shutdown_connections 2024-04-17T06:10:42.953 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.950+0000 7fc3e17cc740 1 --2- 172.21.15.47:0/2916208715 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a134b127b0 0x55a134b0a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:42.953 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.950+0000 7fc3e17cc740 1 --2- 172.21.15.47:0/2916208715 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a134b099e0 0x55a134b1e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:42.953 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.950+0000 7fc3e17cc740 1 --2- 172.21.15.47:0/2916208715 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a134b090c0 0x55a134b11ee0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:42.953 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.950+0000 7fc3e17cc740 1 -- 172.21.15.47:0/2916208715 >> 172.21.15.47:0/2916208715 conn(0x55a134b039e0 msgr2=0x55a134b0f770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:42.953 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.950+0000 7fc3e17cc740 1 -- 172.21.15.47:0/2916208715 shutdown_connections 2024-04-17T06:10:42.953 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.950+0000 7fc3e17cc740 1 -- 172.21.15.47:0/2916208715 wait complete. 2024-04-17T06:10:42.953 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.950+0000 7fc3e17cc740 1 -- start start 2024-04-17T06:10:42.953 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.950+0000 7fc3e17cc740 1 Processor -- start 2024-04-17T06:10:42.953 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.951+0000 7fc3e17cc740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a134b090c0 0x55a134b1ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:42.953 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.951+0000 7fc3e17cc740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a134b099e0 0x55a134b0c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:42.953 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.951+0000 7fc3e17cc740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a134b127b0 0x55a134b0c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:42.954 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.951+0000 7fc3e17cc740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fc3d81ae2e0 con 0x55a134b099e0 2024-04-17T06:10:42.954 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.951+0000 7fc3e17cc740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc3d81af750 con 0x55a134b127b0 2024-04-17T06:10:42.954 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.951+0000 7fc3e17cc740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc3d81ae9b0 con 0x55a134b090c0 2024-04-17T06:10:42.954 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.951+0000 7fc3e17cc740 10 client.0 _ll_register_callbacks cb 0x7fc3d0037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:10:42.954 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.951+0000 7fc3d7fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a134b099e0 0x55a134b0c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:42.954 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.951+0000 7fc3d7fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a134b099e0 0x55a134b0c180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:36472/0 (socket says 172.21.15.47:36472) 2024-04-17T06:10:42.954 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.951+0000 7fc3d7fff640 1 -- 172.21.15.47:0/3900546210 learned_addr learned my addr 172.21.15.47:0/3900546210 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:10:42.954 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.951+0000 7fc3dc984640 1 --2- 172.21.15.47:0/3900546210 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a134b090c0 0x55a134b1ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:42.954 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.951+0000 7fc3dd185640 1 --2- 172.21.15.47:0/3900546210 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a134b127b0 0x55a134b0c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:42.954 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.951+0000 7fc3d7fff640 1 -- 172.21.15.47:0/3900546210 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a134b090c0 msgr2=0x55a134b1ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:42.954 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.951+0000 7fc3d7fff640 1 --2- 172.21.15.47:0/3900546210 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a134b090c0 0x55a134b1ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:42.954 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.951+0000 7fc3d7fff640 1 -- 172.21.15.47:0/3900546210 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a134b127b0 msgr2=0x55a134b0c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:42.954 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.951+0000 7fc3d7fff640 1 --2- 172.21.15.47:0/3900546210 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a134b127b0 0x55a134b0c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:42.955 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.951+0000 7fc3d7fff640 1 -- 172.21.15.47:0/3900546210 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a134b0ee70 con 0x55a134b099e0 2024-04-17T06:10:42.955 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.951+0000 7fc3dd185640 1 --2- 172.21.15.47:0/3900546210 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a134b127b0 0x55a134b0c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:10:42.955 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.951+0000 7fc3dc984640 1 --2- 172.21.15.47:0/3900546210 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a134b090c0 0x55a134b1ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:10:42.955 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.951+0000 7fc3d7fff640 1 --2- 172.21.15.47:0/3900546210 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a134b099e0 0x55a134b0c180 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fc3c4098e30 tx=0x7fc3c40980d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d4c2d53b337d875 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:42.955 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.951+0000 7fc3b7fff640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-17T06:10:42.955 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.951+0000 7fc3b7fff640 1 -- 172.21.15.47:0/3900546210 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3c4020040 con 0x55a134b099e0 2024-04-17T06:10:42.955 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.951+0000 7fc3b7fff640 1 -- 172.21.15.47:0/3900546210 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8469+0+0 (secure 0 0 0) 0x7fc3c40cde90 con 0x55a134b099e0 2024-04-17T06:10:42.955 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.951+0000 7fc3e17cc740 1 -- 172.21.15.47:0/3900546210 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc3cc011cb0 con 0x55a134b099e0 2024-04-17T06:10:42.955 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.952+0000 7fc3b7fff640 1 -- 172.21.15.47:0/3900546210 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fc3c406da40 con 0x55a134b099e0 2024-04-17T06:10:42.955 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.952+0000 7fc3e17cc740 10 client.16191 fetch_fsmap learned FSMap version 34 2024-04-17T06:10:42.955 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.952+0000 7fc3e17cc740 1 -- 172.21.15.47:0/3900546210 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55a134b88b30 con 0x55a134b099e0 2024-04-17T06:10:42.955 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.952+0000 7fc3b7fff640 1 -- 172.21.15.47:0/3900546210 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3cc011cb0 con 0x55a134b099e0 2024-04-17T06:10:42.955 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.952+0000 7fc3b7fff640 1 -- 172.21.15.47:0/3900546210 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 34) ==== 8339+0+0 (secure 0 0 0) 0x7fc3c4098ad0 con 0x55a134b099e0 2024-04-17T06:10:42.955 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.952+0000 7fc3e17cc740 10 client.16191 fetch_fsmap finished waiting for FSMap version 34 2024-04-17T06:10:42.955 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.952+0000 7fc3e17cc740 10 client.16191 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-17T06:10:42.956 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.952+0000 7fc3e17cc740 20 client.16191 populate_metadata read hostname 'smithi047' 2024-04-17T06:10:42.956 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.952+0000 7fc3e17cc740 1 --2- 172.21.15.47:0/3900546210 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55a134b89210 0x55a134b8b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:42.956 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.952+0000 7fc3e17cc740 4 client.16191 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-17T06:10:42.956 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.952+0000 7fc3e17cc740 1 -- 172.21.15.47:0/3900546210 --> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a134a76060 con 0x55a134b89210 2024-04-17T06:10:42.956 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.952+0000 7fc3dc984640 1 --2- 172.21.15.47:0/3900546210 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55a134b89210 0x55a134b8b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:42.956 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.953+0000 7fc3b7fff640 10 client.16191 ms_handle_connect on v2:172.21.15.98:6838/2841287917 2024-04-17T06:10:42.956 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.953+0000 7fc3dc984640 1 --2- 172.21.15.47:0/3900546210 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55a134b89210 0x55a134b8b630 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f213e208267ff12 server_cookie=d8f9c5e10878c517 in_seq=0 out_seq=0 2024-04-17T06:10:42.959 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.961+0000 7fc3b7fff640 1 -- 172.21.15.47:0/3900546210 <== mds.0 v2:172.21.15.98:6838/2841287917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a134a76060 con 0x55a134b89210 2024-04-17T06:10:42.959 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.961+0000 7fc3b7fff640 10 client.16191 handle_command_reply: tid=0 2024-04-17T06:10:42.959 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.961+0000 7fc3e17cc740 1 client.16191 shutdown 2024-04-17T06:10:42.959 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.961+0000 7fc3e17cc740 1 -- 172.21.15.47:0/3900546210 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a134b099e0 msgr2=0x55a134b0c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:42.960 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.961+0000 7fc3e17cc740 1 --2- 172.21.15.47:0/3900546210 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a134b099e0 0x55a134b0c180 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fc3c4098e30 tx=0x7fc3c40980d0 comp rx=0 tx=0).stop 2024-04-17T06:10:42.960 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.961+0000 7fc3e17cc740 1 -- 172.21.15.47:0/3900546210 shutdown_connections 2024-04-17T06:10:42.960 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.961+0000 7fc3e17cc740 1 --2- 172.21.15.47:0/3900546210 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55a134b89210 0x55a134b8b630 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:42.960 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.961+0000 7fc3e17cc740 1 --2- 172.21.15.47:0/3900546210 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a134b127b0 0x55a134b0c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:42.960 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.961+0000 7fc3e17cc740 1 --2- 172.21.15.47:0/3900546210 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a134b099e0 0x55a134b0c180 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:42.960 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.961+0000 7fc3e17cc740 1 --2- 172.21.15.47:0/3900546210 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a134b090c0 0x55a134b1ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:42.960 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.961+0000 7fc3b7fff640 0 client.16191 ms_handle_reset on v2:172.21.15.98:6838/2841287917 2024-04-17T06:10:42.960 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.961+0000 7fc3e17cc740 1 -- 172.21.15.47:0/3900546210 >> 172.21.15.47:0/3900546210 conn(0x55a134b039e0 msgr2=0x55a134b02b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:42.960 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.961+0000 7fc3e17cc740 1 -- 172.21.15.47:0/3900546210 shutdown_connections 2024-04-17T06:10:42.960 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.961+0000 7fc3e17cc740 1 -- 172.21.15.47:0/3900546210 wait complete. 2024-04-17T06:10:42.960 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.961+0000 7fc3e17cc740 20 client.16191 trim_cache size 0 max 16384 2024-04-17T06:10:42.960 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-04-17T06:10:42.961 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-04-17T06:10:42.961 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-04-17T06:10:42.961 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-04-17T06:10:42.961 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.962+0000 7fc3debe8640 1 -- 172.21.15.47:0/1204360628 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fc3c80a3dc0 msgr2=0x7fc3c80a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:42.962 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.962+0000 7fc3debe8640 1 --2- 172.21.15.47:0/1204360628 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fc3c80a3dc0 0x7fc3c80a6280 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc3d0009a60 tx=0x7fc3d00323c0 comp rx=0 tx=0).stop 2024-04-17T06:10:42.962 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.962+0000 7fc3debe8640 1 -- 172.21.15.47:0/1204360628 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc3d806b130 msgr2=0x7fc3d81017c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:42.962 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.962+0000 7fc3debe8640 1 --2- 172.21.15.47:0/1204360628 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc3d806b130 0x7fc3d81017c0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fc3c400d950 tx=0x7fc3c400de10 comp rx=0 tx=0).stop 2024-04-17T06:10:42.962 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.962+0000 7fc3debe8640 1 -- 172.21.15.47:0/1204360628 shutdown_connections 2024-04-17T06:10:42.962 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.962+0000 7fc3debe8640 1 --2- 172.21.15.47:0/1204360628 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc3d81080f0 0x7fc3d8101d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:42.962 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.962+0000 7fc3debe8640 1 --2- 172.21.15.47:0/1204360628 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fc3c80a3dc0 0x7fc3c80a6280 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:42.962 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.962+0000 7fc3debe8640 1 --2- 172.21.15.47:0/1204360628 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc3d806b130 0x7fc3d81017c0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:42.962 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.962+0000 7fc3debe8640 1 --2- 172.21.15.47:0/1204360628 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc3d806a790 0x7fc3d8101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:42.962 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.962+0000 7fc3debe8640 1 -- 172.21.15.47:0/1204360628 >> 172.21.15.47:0/1204360628 conn(0x7fc3d8076c60 msgr2=0x7fc3d80ffb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:42.962 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.962+0000 7fc3debe8640 1 -- 172.21.15.47:0/1204360628 shutdown_connections 2024-04-17T06:10:42.962 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:42.962+0000 7fc3debe8640 1 -- 172.21.15.47:0/1204360628 wait complete. 2024-04-17T06:10:44.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:10:43 smithi103 ceph-mon[36243]: pgmap v295: 97 pgs: 97 active+clean; 4.7 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:10:44.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:10:43 smithi098 ceph-mon[36516]: pgmap v295: 97 pgs: 97 active+clean; 4.7 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:10:44.388 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:10:43 smithi047 ceph-mon[32223]: pgmap v295: 97 pgs: 97 active+clean; 4.7 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:10:46.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:10:45 smithi103 ceph-mon[36243]: pgmap v296: 97 pgs: 97 active+clean; 4.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 21 KiB/s wr, 18 op/s 2024-04-17T06:10:46.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:10:45 smithi098 ceph-mon[36516]: pgmap v296: 97 pgs: 97 active+clean; 4.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 21 KiB/s wr, 18 op/s 2024-04-17T06:10:46.390 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:10:45 smithi047 ceph-mon[32223]: pgmap v296: 97 pgs: 97 active+clean; 4.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 21 KiB/s wr, 18 op/s 2024-04-17T06:10:48.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:10:47 smithi103 ceph-mon[36243]: pgmap v297: 97 pgs: 97 active+clean; 4.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:10:48.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:10:47 smithi098 ceph-mon[36516]: pgmap v297: 97 pgs: 97 active+clean; 4.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:10:48.388 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:10:47 smithi047 ceph-mon[32223]: pgmap v297: 97 pgs: 97 active+clean; 4.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:10:50.191 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:10:49 smithi047 ceph-mon[32223]: pgmap v298: 97 pgs: 97 active+clean; 4.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:10:50.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:10:49 smithi103 ceph-mon[36243]: pgmap v298: 97 pgs: 97 active+clean; 4.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:10:50.382 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:10:49 smithi098 ceph-mon[36516]: pgmap v298: 97 pgs: 97 active+clean; 4.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:10:52.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:10:51 smithi103 ceph-mon[36243]: pgmap v299: 97 pgs: 97 active+clean; 4.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-04-17T06:10:52.382 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:10:51 smithi098 ceph-mon[36516]: pgmap v299: 97 pgs: 97 active+clean; 4.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-04-17T06:10:52.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:10:51 smithi047 ceph-mon[32223]: pgmap v299: 97 pgs: 97 active+clean; 4.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-04-17T06:10:54.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:10:53 smithi103 ceph-mon[36243]: pgmap v300: 97 pgs: 97 active+clean; 4.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-04-17T06:10:54.382 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:10:53 smithi098 ceph-mon[36516]: pgmap v300: 97 pgs: 97 active+clean; 4.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-04-17T06:10:54.388 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:10:53 smithi047 ceph-mon[32223]: pgmap v300: 97 pgs: 97 active+clean; 4.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-04-17T06:10:56.062 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.061+0000 7f3669ba8640 1 -- 172.21.15.47:0/4252742437 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36641061a0 msgr2=0x7f366410dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:56.062 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.061+0000 7f3669ba8640 1 --2- 172.21.15.47:0/4252742437 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36641061a0 0x7f366410dd70 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f365c009ca0 tx=0x7f365c02c0f0 comp rx=0 tx=0).stop 2024-04-17T06:10:56.062 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.061+0000 7f3669ba8640 1 -- 172.21.15.47:0/4252742437 shutdown_connections 2024-04-17T06:10:56.062 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.061+0000 7f3669ba8640 1 --2- 172.21.15.47:0/4252742437 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36641061a0 0x7f366410dd70 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:56.062 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.061+0000 7f3669ba8640 1 --2- 172.21.15.47:0/4252742437 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3664101a80 0x7f3664105ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:56.062 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.061+0000 7f3669ba8640 1 --2- 172.21.15.47:0/4252742437 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f36641010d0 0x7f36641014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.061+0000 7f3669ba8640 1 -- 172.21.15.47:0/4252742437 >> 172.21.15.47:0/4252742437 conn(0x7f3664077fe0 msgr2=0x7f36640783f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.061+0000 7f3669ba8640 1 -- 172.21.15.47:0/4252742437 shutdown_connections 2024-04-17T06:10:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.061+0000 7f3669ba8640 1 -- 172.21.15.47:0/4252742437 wait complete. 2024-04-17T06:10:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.062+0000 7f3669ba8640 1 Processor -- start 2024-04-17T06:10:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.062+0000 7f3669ba8640 1 -- start start 2024-04-17T06:10:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.062+0000 7f3669ba8640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36641010d0 0x7f3664198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.062+0000 7f3669ba8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f36641061a0 0x7f3664198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.062+0000 7f3669ba8640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f366419cff0 0x7f366419d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.062+0000 7f3669ba8640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f3664111260 con 0x7f36641010d0 2024-04-17T06:10:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.062+0000 7f3669ba8640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f36641110e0 con 0x7f366419cff0 2024-04-17T06:10:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.062+0000 7f3669ba8640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f36641113e0 con 0x7f36641061a0 2024-04-17T06:10:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.062+0000 7f36637fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36641010d0 0x7f3664198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.062+0000 7f36637fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36641010d0 0x7f3664198730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54600/0 (socket says 172.21.15.47:54600) 2024-04-17T06:10:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.062+0000 7f36637fe640 1 -- 172.21.15.47:0/21941387 learned_addr learned my addr 172.21.15.47:0/21941387 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:10:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.062+0000 7f3663fff640 1 --2- 172.21.15.47:0/21941387 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f366419cff0 0x7f366419d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.062+0000 7f365bfff640 1 --2- 172.21.15.47:0/21941387 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f36641061a0 0x7f3664198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.063+0000 7f36637fe640 1 -- 172.21.15.47:0/21941387 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f36641061a0 msgr2=0x7f3664198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.063+0000 7f36637fe640 1 --2- 172.21.15.47:0/21941387 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f36641061a0 0x7f3664198c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.063+0000 7f36637fe640 1 -- 172.21.15.47:0/21941387 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f366419cff0 msgr2=0x7f366419d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.063+0000 7f36637fe640 1 --2- 172.21.15.47:0/21941387 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f366419cff0 0x7f366419d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.063+0000 7f36637fe640 1 -- 172.21.15.47:0/21941387 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f366419dae0 con 0x7f36641010d0 2024-04-17T06:10:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.063+0000 7f36637fe640 1 --2- 172.21.15.47:0/21941387 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36641010d0 0x7f3664198730 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f365400d770 tx=0x7f365400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.063+0000 7f36617fa640 1 -- 172.21.15.47:0/21941387 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3654017070 con 0x7f36641010d0 2024-04-17T06:10:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.063+0000 7f36617fa640 1 -- 172.21.15.47:0/21941387 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3654005500 con 0x7f36641010d0 2024-04-17T06:10:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.063+0000 7f3669ba8640 1 -- 172.21.15.47:0/21941387 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f366419ddd0 con 0x7f36641010d0 2024-04-17T06:10:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.063+0000 7f36617fa640 1 -- 172.21.15.47:0/21941387 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3654004e00 con 0x7f36641010d0 2024-04-17T06:10:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.063+0000 7f3669ba8640 1 -- 172.21.15.47:0/21941387 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36641a5570 con 0x7f36641010d0 2024-04-17T06:10:56.071 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.066+0000 7f36617fa640 1 -- 172.21.15.47:0/21941387 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3654004fa0 con 0x7f36641010d0 2024-04-17T06:10:56.072 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.066+0000 7f366c78d740 1 Processor -- start 2024-04-17T06:10:56.072 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.066+0000 7f366c78d740 1 -- start start 2024-04-17T06:10:56.072 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.067+0000 7f366c78d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557e72748be0 0x557e72748fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:56.072 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.067+0000 7f366c78d740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557e72749500 0x557e72756680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:56.072 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.067+0000 7f366c78d740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557e72756bc0 0x557e72758fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:56.072 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.067+0000 7f366c78d740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x557e726ae7d0 con 0x557e72749500 2024-04-17T06:10:56.072 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.067+0000 7f366c78d740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x557e7264bf70 con 0x557e72756bc0 2024-04-17T06:10:56.072 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.067+0000 7f366c78d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x557e726aa790 con 0x557e72748be0 2024-04-17T06:10:56.072 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.067+0000 7f36637fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557e72749500 0x557e72756680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:56.072 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.067+0000 7f36637fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557e72749500 0x557e72756680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54616/0 (socket says 172.21.15.47:54616) 2024-04-17T06:10:56.072 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.067+0000 7f36637fe640 1 -- 172.21.15.47:0/4238939138 learned_addr learned my addr 172.21.15.47:0/4238939138 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:10:56.072 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.067+0000 7f365bfff640 1 --2- 172.21.15.47:0/4238939138 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557e72756bc0 0x557e72758fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:56.072 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.067+0000 7f3663fff640 1 --2- 172.21.15.47:0/4238939138 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557e72748be0 0x557e72748fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:56.072 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.067+0000 7f36637fe640 1 -- 172.21.15.47:0/4238939138 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557e72748be0 msgr2=0x557e72748fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:56.073 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.067+0000 7f36637fe640 1 --2- 172.21.15.47:0/4238939138 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557e72748be0 0x557e72748fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:56.073 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.067+0000 7f36637fe640 1 -- 172.21.15.47:0/4238939138 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557e72756bc0 msgr2=0x557e72758fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:56.073 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.067+0000 7f36637fe640 1 --2- 172.21.15.47:0/4238939138 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557e72756bc0 0x557e72758fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:56.073 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.067+0000 7f36637fe640 1 -- 172.21.15.47:0/4238939138 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e723c1210 con 0x557e72749500 2024-04-17T06:10:56.073 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.067+0000 7f36617fa640 1 --2- 172.21.15.47:0/21941387 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f36500a3dc0 0x7f36500a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:56.073 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.067+0000 7f36637fe640 1 --2- 172.21.15.47:0/4238939138 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557e72749500 0x557e72756680 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f36540cd9c0 tx=0x7f36540d8a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d57c3db0498601d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:56.073 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.067+0000 7f3663fff640 1 --2- 172.21.15.47:0/21941387 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f36500a3dc0 0x7f36500a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:56.073 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.067+0000 7f36617fa640 1 -- 172.21.15.47:0/21941387 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8469+0+0 (secure 0 0 0) 0x7f36540d0360 con 0x7f36641010d0 2024-04-17T06:10:56.073 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.068+0000 7f3659ffb640 1 -- 172.21.15.47:0/4238939138 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3654098570 con 0x557e72749500 2024-04-17T06:10:56.073 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.068+0000 7f3659ffb640 1 -- 172.21.15.47:0/4238939138 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36540046c0 con 0x557e72749500 2024-04-17T06:10:56.073 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.068+0000 7f3659ffb640 1 -- 172.21.15.47:0/4238939138 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f365409bc30 con 0x557e72749500 2024-04-17T06:10:56.073 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.068+0000 7f366c78d740 1 -- 172.21.15.47:0/4238939138 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557e72749500 msgr2=0x557e72756680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:56.073 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.068+0000 7f3663fff640 1 --2- 172.21.15.47:0/21941387 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f36500a3dc0 0x7f36500a6280 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f365c02c700 tx=0x7f365c005f80 comp rx=0 tx=0).ready entity=mgr.24923 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:56.074 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.068+0000 7f366c78d740 1 --2- 172.21.15.47:0/4238939138 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557e72749500 0x557e72756680 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f36540cd9c0 tx=0x7f36540d8a30 comp rx=0 tx=0).stop 2024-04-17T06:10:56.074 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.068+0000 7f366c78d740 1 -- 172.21.15.47:0/4238939138 shutdown_connections 2024-04-17T06:10:56.074 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.068+0000 7f366c78d740 1 --2- 172.21.15.47:0/4238939138 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557e72756bc0 0x557e72758fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:56.074 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.068+0000 7f366c78d740 1 --2- 172.21.15.47:0/4238939138 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557e72749500 0x557e72756680 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:56.074 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.068+0000 7f366c78d740 1 --2- 172.21.15.47:0/4238939138 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557e72748be0 0x557e72748fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:56.074 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.068+0000 7f366c78d740 1 -- 172.21.15.47:0/4238939138 >> 172.21.15.47:0/4238939138 conn(0x557e727439e0 msgr2=0x557e72743dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:56.074 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.068+0000 7f366c78d740 1 -- 172.21.15.47:0/4238939138 shutdown_connections 2024-04-17T06:10:56.074 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.068+0000 7f366c78d740 1 -- 172.21.15.47:0/4238939138 wait complete. 2024-04-17T06:10:56.074 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.069+0000 7f366c78d740 1 -- start start 2024-04-17T06:10:56.074 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.069+0000 7f366c78d740 1 Processor -- start 2024-04-17T06:10:56.074 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.069+0000 7f366c78d740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557e72748be0 0x557e72753c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:56.074 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.069+0000 7f366c78d740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557e72749500 0x557e72754160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:56.074 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.069+0000 7f366c78d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557e72756bc0 0x557e727546a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:56.074 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.069+0000 7f366c78d740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x557e72759780 con 0x557e72749500 2024-04-17T06:10:56.075 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.069+0000 7f366c78d740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x557e72636e90 con 0x557e72748be0 2024-04-17T06:10:56.075 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.069+0000 7f366c78d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x557e72759900 con 0x557e72756bc0 2024-04-17T06:10:56.075 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.069+0000 7f366c78d740 10 client.0 _ll_register_callbacks cb 0x557e7264cd40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:10:56.075 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.069+0000 7f365bfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557e72749500 0x557e72754160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:56.075 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.069+0000 7f365bfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557e72749500 0x557e72754160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54628/0 (socket says 172.21.15.47:54628) 2024-04-17T06:10:56.075 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.069+0000 7f365bfff640 1 -- 172.21.15.47:0/3864496862 learned_addr learned my addr 172.21.15.47:0/3864496862 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:10:56.075 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.069+0000 7f3663fff640 1 --2- 172.21.15.47:0/3864496862 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557e72756bc0 0x557e727546a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:56.075 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.070+0000 7f36637fe640 1 --2- 172.21.15.47:0/3864496862 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557e72748be0 0x557e72753c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:56.075 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.070+0000 7f365bfff640 1 -- 172.21.15.47:0/3864496862 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557e72756bc0 msgr2=0x557e727546a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:56.075 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.070+0000 7f365bfff640 1 --2- 172.21.15.47:0/3864496862 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557e72756bc0 0x557e727546a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:56.075 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.070+0000 7f365bfff640 1 -- 172.21.15.47:0/3864496862 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557e72748be0 msgr2=0x557e72753c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:56.075 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.070+0000 7f365bfff640 1 --2- 172.21.15.47:0/3864496862 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557e72748be0 0x557e72753c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:56.075 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.070+0000 7f365bfff640 1 -- 172.21.15.47:0/3864496862 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e7248d620 con 0x557e72749500 2024-04-17T06:10:56.075 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.070+0000 7f365bfff640 1 --2- 172.21.15.47:0/3864496862 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557e72749500 0x557e72754160 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f364c010420 tx=0x7f364c0108e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93043853a80d64fb server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:56.076 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.070+0000 7f3642ffd640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-17T06:10:56.076 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.070+0000 7f3642ffd640 1 -- 172.21.15.47:0/3864496862 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f364c012920 con 0x557e72749500 2024-04-17T06:10:56.076 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.070+0000 7f3642ffd640 1 -- 172.21.15.47:0/3864496862 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8469+0+0 (secure 0 0 0) 0x7f364c012ac0 con 0x557e72749500 2024-04-17T06:10:56.076 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.070+0000 7f366c78d740 1 -- 172.21.15.47:0/3864496862 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557e726bc1f0 con 0x557e72749500 2024-04-17T06:10:56.076 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.070+0000 7f3642ffd640 1 -- 172.21.15.47:0/3864496862 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f364c012cc0 con 0x557e72749500 2024-04-17T06:10:56.076 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.070+0000 7f366c78d740 10 client.16203 fetch_fsmap learned FSMap version 34 2024-04-17T06:10:56.076 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.070+0000 7f366c78d740 1 -- 172.21.15.47:0/3864496862 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x557e72765af0 con 0x557e72749500 2024-04-17T06:10:56.076 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.070+0000 7f3642ffd640 1 -- 172.21.15.47:0/3864496862 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x557e726bc1f0 con 0x557e72749500 2024-04-17T06:10:56.077 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.071+0000 7f3642ffd640 1 -- 172.21.15.47:0/3864496862 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 34) ==== 8339+0+0 (secure 0 0 0) 0x7f364c00f040 con 0x557e72749500 2024-04-17T06:10:56.077 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.071+0000 7f366c78d740 10 client.16203 fetch_fsmap finished waiting for FSMap version 34 2024-04-17T06:10:56.077 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.071+0000 7f366c78d740 10 client.16203 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-17T06:10:56.077 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.071+0000 7f366c78d740 20 client.16203 populate_metadata read hostname 'smithi047' 2024-04-17T06:10:56.077 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.071+0000 7f366c78d740 1 --2- 172.21.15.47:0/3864496862 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x557e72766f60 0x557e72769370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:56.077 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.071+0000 7f366c78d740 4 client.16203 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T06:10:56.077 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.071+0000 7f366c78d740 1 -- 172.21.15.47:0/3864496862 --> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557e726b6060 con 0x557e72766f60 2024-04-17T06:10:56.077 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.071+0000 7f36637fe640 1 --2- 172.21.15.47:0/3864496862 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x557e72766f60 0x557e72769370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:56.077 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.071+0000 7f3642ffd640 10 client.16203 ms_handle_connect on v2:172.21.15.98:6838/2841287917 2024-04-17T06:10:56.077 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.071+0000 7f36637fe640 1 --2- 172.21.15.47:0/3864496862 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x557e72766f60 0x557e72769370 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b83b913bbc9ce31 server_cookie=180bf67b9296cc84 in_seq=0 out_seq=0 2024-04-17T06:10:56.077 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.072+0000 7f3642ffd640 1 -- 172.21.15.47:0/3864496862 <== mds.0 v2:172.21.15.98:6838/2841287917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557e726b6060 con 0x557e72766f60 2024-04-17T06:10:56.077 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.072+0000 7f3642ffd640 10 client.16203 handle_command_reply: tid=0 2024-04-17T06:10:56.077 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.072+0000 7f366c78d740 1 client.16203 shutdown 2024-04-17T06:10:56.078 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.072+0000 7f366c78d740 1 -- 172.21.15.47:0/3864496862 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557e72749500 msgr2=0x557e72754160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:56.078 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.072+0000 7f366c78d740 1 --2- 172.21.15.47:0/3864496862 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557e72749500 0x557e72754160 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f364c010420 tx=0x7f364c0108e0 comp rx=0 tx=0).stop 2024-04-17T06:10:56.078 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.072+0000 7f366c78d740 1 -- 172.21.15.47:0/3864496862 shutdown_connections 2024-04-17T06:10:56.078 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.072+0000 7f366c78d740 1 --2- 172.21.15.47:0/3864496862 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x557e72766f60 0x557e72769370 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:56.078 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.072+0000 7f366c78d740 1 --2- 172.21.15.47:0/3864496862 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557e72756bc0 0x557e727546a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:56.078 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.072+0000 7f366c78d740 1 --2- 172.21.15.47:0/3864496862 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557e72749500 0x557e72754160 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:56.078 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.072+0000 7f366c78d740 1 --2- 172.21.15.47:0/3864496862 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557e72748be0 0x557e72753c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:56.078 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.072+0000 7f3642ffd640 0 client.16203 ms_handle_reset on v2:172.21.15.98:6838/2841287917 2024-04-17T06:10:56.078 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.072+0000 7f366c78d740 1 -- 172.21.15.47:0/3864496862 >> 172.21.15.47:0/3864496862 conn(0x557e727439e0 msgr2=0x557e72745730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:56.078 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.073+0000 7f366c78d740 1 -- 172.21.15.47:0/3864496862 shutdown_connections 2024-04-17T06:10:56.078 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.073+0000 7f366c78d740 1 -- 172.21.15.47:0/3864496862 wait complete. 2024-04-17T06:10:56.078 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.073+0000 7f366c78d740 20 client.16203 trim_cache size 0 max 16384 2024-04-17T06:10:56.088 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.088+0000 7f366c78d740 1 Processor -- start 2024-04-17T06:10:56.088 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.088+0000 7f366c78d740 1 -- start start 2024-04-17T06:10:56.088 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.088+0000 7f366c78d740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557e72748be0 0x557e72756350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:56.088 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.088+0000 7f366c78d740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557e72749500 0x557e72762680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:56.089 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.088+0000 7f366c78d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557e72756bc0 0x557e727542f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:56.089 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.088+0000 7f366c78d740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f36641ae250 con 0x557e72748be0 2024-04-17T06:10:56.089 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.088+0000 7f366c78d740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f36641abb10 con 0x557e72749500 2024-04-17T06:10:56.089 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.088+0000 7f366c78d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f36641a9960 con 0x557e72756bc0 2024-04-17T06:10:56.089 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.088+0000 7f36637fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557e72748be0 0x557e72756350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:56.089 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.088+0000 7f36637fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557e72748be0 0x557e72756350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54632/0 (socket says 172.21.15.47:54632) 2024-04-17T06:10:56.089 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.088+0000 7f36637fe640 1 -- 172.21.15.47:0/572065795 learned_addr learned my addr 172.21.15.47:0/572065795 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:10:56.089 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.088+0000 7f365bfff640 1 --2- 172.21.15.47:0/572065795 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557e72749500 0x557e72762680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:56.089 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.088+0000 7f36637fe640 1 -- 172.21.15.47:0/572065795 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557e72756bc0 msgr2=0x557e727542f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:56.089 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.088+0000 7f3663fff640 1 --2- 172.21.15.47:0/572065795 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557e72756bc0 0x557e727542f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:56.089 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.088+0000 7f36637fe640 1 --2- 172.21.15.47:0/572065795 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557e72756bc0 0x557e727542f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:56.089 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.088+0000 7f36637fe640 1 -- 172.21.15.47:0/572065795 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557e72749500 msgr2=0x557e72762680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:56.089 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.088+0000 7f36637fe640 1 --2- 172.21.15.47:0/572065795 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557e72749500 0x557e72762680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:56.089 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.088+0000 7f36637fe640 1 -- 172.21.15.47:0/572065795 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e72749b90 con 0x557e72748be0 2024-04-17T06:10:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.088+0000 7f3663fff640 1 --2- 172.21.15.47:0/572065795 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557e72756bc0 0x557e727542f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:10:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.088+0000 7f365bfff640 1 --2- 172.21.15.47:0/572065795 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557e72749500 0x557e72762680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:10:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.088+0000 7f36637fe640 1 --2- 172.21.15.47:0/572065795 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557e72748be0 0x557e72756350 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f3654009610 tx=0x7f36540d8150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b8ec384a553085f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.089+0000 7f36597fa640 1 -- 172.21.15.47:0/572065795 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3654020040 con 0x557e72748be0 2024-04-17T06:10:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.089+0000 7f36597fa640 1 -- 172.21.15.47:0/572065795 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36540d81c0 con 0x557e72748be0 2024-04-17T06:10:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.089+0000 7f36597fa640 1 -- 172.21.15.47:0/572065795 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f365409be10 con 0x557e72748be0 2024-04-17T06:10:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.089+0000 7f366c78d740 1 -- 172.21.15.47:0/572065795 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557e72748be0 msgr2=0x557e72756350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.089+0000 7f366c78d740 1 --2- 172.21.15.47:0/572065795 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557e72748be0 0x557e72756350 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f3654009610 tx=0x7f36540d8150 comp rx=0 tx=0).stop 2024-04-17T06:10:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.089+0000 7f366c78d740 1 -- 172.21.15.47:0/572065795 shutdown_connections 2024-04-17T06:10:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.089+0000 7f366c78d740 1 --2- 172.21.15.47:0/572065795 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557e72756bc0 0x557e727542f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.089+0000 7f366c78d740 1 --2- 172.21.15.47:0/572065795 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557e72749500 0x557e72762680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.089+0000 7f366c78d740 1 --2- 172.21.15.47:0/572065795 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557e72748be0 0x557e72756350 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.089+0000 7f366c78d740 1 -- 172.21.15.47:0/572065795 >> 172.21.15.47:0/572065795 conn(0x557e727439e0 msgr2=0x557e72753be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:56.091 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.089+0000 7f366c78d740 1 -- 172.21.15.47:0/572065795 shutdown_connections 2024-04-17T06:10:56.091 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.089+0000 7f366c78d740 1 -- 172.21.15.47:0/572065795 wait complete. 2024-04-17T06:10:56.091 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.090+0000 7f366c78d740 1 -- start start 2024-04-17T06:10:56.091 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.090+0000 7f366c78d740 1 Processor -- start 2024-04-17T06:10:56.093 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.090+0000 7f366c78d740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557e72748be0 0x557e7275a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:56.093 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.090+0000 7f366c78d740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557e72749500 0x557e7275abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:56.094 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.090+0000 7f366c78d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557e72756bc0 0x557e7275b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:56.094 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.090+0000 7f366c78d740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f36641a9fd0 con 0x557e72749500 2024-04-17T06:10:56.094 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.090+0000 7f366c78d740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f36641ab440 con 0x557e72748be0 2024-04-17T06:10:56.094 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.090+0000 7f366c78d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f36641aa6a0 con 0x557e72756bc0 2024-04-17T06:10:56.094 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.090+0000 7f366c78d740 10 client.0 _ll_register_callbacks cb 0x7f364c00e2f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:10:56.094 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.090+0000 7f365bfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557e72749500 0x557e7275abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:56.094 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.090+0000 7f365bfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557e72749500 0x557e7275abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54648/0 (socket says 172.21.15.47:54648) 2024-04-17T06:10:56.094 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.090+0000 7f365bfff640 1 -- 172.21.15.47:0/1748268362 learned_addr learned my addr 172.21.15.47:0/1748268362 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:10:56.094 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.090+0000 7f36637fe640 1 --2- 172.21.15.47:0/1748268362 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557e72748be0 0x557e7275a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:56.094 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.090+0000 7f365bfff640 1 -- 172.21.15.47:0/1748268362 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557e72756bc0 msgr2=0x557e7275b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:56.094 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.090+0000 7f365bfff640 1 --2- 172.21.15.47:0/1748268362 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557e72756bc0 0x557e7275b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:56.094 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.090+0000 7f365bfff640 1 -- 172.21.15.47:0/1748268362 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557e72748be0 msgr2=0x557e7275a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:56.095 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.090+0000 7f365bfff640 1 --2- 172.21.15.47:0/1748268362 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557e72748be0 0x557e7275a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:56.095 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.090+0000 7f365bfff640 1 -- 172.21.15.47:0/1748268362 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e727cc570 con 0x557e72749500 2024-04-17T06:10:56.095 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.091+0000 7f365bfff640 1 --2- 172.21.15.47:0/1748268362 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557e72749500 0x557e7275abc0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f364c000c00 tx=0x7f364c005ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47cbdc31929c4b0d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:56.095 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.091+0000 7f3642ffd640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-17T06:10:56.095 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.091+0000 7f3642ffd640 1 -- 172.21.15.47:0/1748268362 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f364c02b070 con 0x557e72749500 2024-04-17T06:10:56.095 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.091+0000 7f3642ffd640 1 -- 172.21.15.47:0/1748268362 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8469+0+0 (secure 0 0 0) 0x7f364c012720 con 0x557e72749500 2024-04-17T06:10:56.095 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.091+0000 7f366c78d740 1 -- 172.21.15.47:0/1748268362 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36540d81c0 con 0x557e72749500 2024-04-17T06:10:56.095 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.091+0000 7f3642ffd640 1 -- 172.21.15.47:0/1748268362 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f364c012e60 con 0x557e72749500 2024-04-17T06:10:56.095 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.091+0000 7f366c78d740 10 client.16209 fetch_fsmap learned FSMap version 34 2024-04-17T06:10:56.095 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.091+0000 7f366c78d740 1 -- 172.21.15.47:0/1748268362 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x557e727cd800 con 0x557e72749500 2024-04-17T06:10:56.095 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.091+0000 7f3642ffd640 1 -- 172.21.15.47:0/1748268362 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36540d81c0 con 0x557e72749500 2024-04-17T06:10:56.095 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.091+0000 7f3642ffd640 1 -- 172.21.15.47:0/1748268362 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 34) ==== 8339+0+0 (secure 0 0 0) 0x7f364c011780 con 0x557e72749500 2024-04-17T06:10:56.095 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.091+0000 7f366c78d740 10 client.16209 fetch_fsmap finished waiting for FSMap version 34 2024-04-17T06:10:56.096 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.091+0000 7f366c78d740 10 client.16209 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-17T06:10:56.096 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.091+0000 7f366c78d740 20 client.16209 populate_metadata read hostname 'smithi047' 2024-04-17T06:10:56.096 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.091+0000 7f366c78d740 1 --2- 172.21.15.47:0/1748268362 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x557e727ce000 0x557e727d0420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:56.096 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.092+0000 7f366c78d740 4 client.16209 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-17T06:10:56.096 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.092+0000 7f366c78d740 1 -- 172.21.15.47:0/1748268362 --> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557e72632670 con 0x557e727ce000 2024-04-17T06:10:56.096 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.092+0000 7f36637fe640 1 --2- 172.21.15.47:0/1748268362 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x557e727ce000 0x557e727d0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:56.096 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.092+0000 7f36637fe640 1 --2- 172.21.15.47:0/1748268362 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x557e727ce000 0x557e727d0420 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e28db6e001907f9 server_cookie=f08eb98512347855 in_seq=0 out_seq=0 2024-04-17T06:10:56.096 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.092+0000 7f3642ffd640 10 client.16209 ms_handle_connect on v2:172.21.15.98:6838/2841287917 2024-04-17T06:10:56.099 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.100+0000 7f3642ffd640 1 -- 172.21.15.47:0/1748268362 <== mds.0 v2:172.21.15.98:6838/2841287917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557e72632670 con 0x557e727ce000 2024-04-17T06:10:56.100 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.100+0000 7f3642ffd640 10 client.16209 handle_command_reply: tid=0 2024-04-17T06:10:56.100 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.100+0000 7f366c78d740 1 client.16209 shutdown 2024-04-17T06:10:56.100 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.100+0000 7f366c78d740 1 -- 172.21.15.47:0/1748268362 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557e72749500 msgr2=0x557e7275abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:56.100 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.100+0000 7f366c78d740 1 --2- 172.21.15.47:0/1748268362 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557e72749500 0x557e7275abc0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f364c000c00 tx=0x7f364c005ed0 comp rx=0 tx=0).stop 2024-04-17T06:10:56.100 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.100+0000 7f366c78d740 1 -- 172.21.15.47:0/1748268362 shutdown_connections 2024-04-17T06:10:56.100 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.100+0000 7f366c78d740 1 --2- 172.21.15.47:0/1748268362 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x557e727ce000 0x557e727d0420 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:56.100 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.100+0000 7f366c78d740 1 --2- 172.21.15.47:0/1748268362 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557e72756bc0 0x557e7275b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:56.100 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.100+0000 7f366c78d740 1 --2- 172.21.15.47:0/1748268362 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557e72749500 0x557e7275abc0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:56.100 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.100+0000 7f366c78d740 1 --2- 172.21.15.47:0/1748268362 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557e72748be0 0x557e7275a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:56.100 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.100+0000 7f3642ffd640 0 client.16209 ms_handle_reset on v2:172.21.15.98:6838/2841287917 2024-04-17T06:10:56.100 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.101+0000 7f366c78d740 1 -- 172.21.15.47:0/1748268362 >> 172.21.15.47:0/1748268362 conn(0x557e727439e0 msgr2=0x557e72742b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:56.100 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.101+0000 7f366c78d740 1 -- 172.21.15.47:0/1748268362 shutdown_connections 2024-04-17T06:10:56.101 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.101+0000 7f366c78d740 1 -- 172.21.15.47:0/1748268362 wait complete. 2024-04-17T06:10:56.101 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.101+0000 7f366c78d740 20 client.16209 trim_cache size 0 max 16384 2024-04-17T06:10:56.101 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-04-17T06:10:56.101 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-04-17T06:10:56.101 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-04-17T06:10:56.101 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-04-17T06:10:56.102 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.101+0000 7f3669ba8640 1 -- 172.21.15.47:0/21941387 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f36500a3dc0 msgr2=0x7f36500a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:56.102 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.101+0000 7f3669ba8640 1 --2- 172.21.15.47:0/21941387 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f36500a3dc0 0x7f36500a6280 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f365c02c700 tx=0x7f365c005f80 comp rx=0 tx=0).stop 2024-04-17T06:10:56.102 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.101+0000 7f3669ba8640 1 -- 172.21.15.47:0/21941387 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36641010d0 msgr2=0x7f3664198730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:56.102 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.101+0000 7f3669ba8640 1 --2- 172.21.15.47:0/21941387 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36641010d0 0x7f3664198730 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f365400d770 tx=0x7f365400dc30 comp rx=0 tx=0).stop 2024-04-17T06:10:56.102 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.101+0000 7f3669ba8640 1 -- 172.21.15.47:0/21941387 shutdown_connections 2024-04-17T06:10:56.102 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.101+0000 7f3669ba8640 1 --2- 172.21.15.47:0/21941387 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f366419cff0 0x7f366419d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:56.102 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.101+0000 7f3669ba8640 1 --2- 172.21.15.47:0/21941387 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f36641061a0 0x7f3664198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:56.102 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.101+0000 7f3669ba8640 1 --2- 172.21.15.47:0/21941387 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f36500a3dc0 0x7f36500a6280 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:56.103 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.101+0000 7f3669ba8640 1 --2- 172.21.15.47:0/21941387 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36641010d0 0x7f3664198730 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:56.103 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.101+0000 7f3669ba8640 1 -- 172.21.15.47:0/21941387 >> 172.21.15.47:0/21941387 conn(0x7f3664077fe0 msgr2=0x7f366406a9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:56.103 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.101+0000 7f3669ba8640 1 -- 172.21.15.47:0/21941387 shutdown_connections 2024-04-17T06:10:56.103 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:10:56.101+0000 7f3669ba8640 1 -- 172.21.15.47:0/21941387 wait complete. 2024-04-17T06:10:56.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:10:55 smithi103 ceph-mon[36243]: pgmap v301: 97 pgs: 97 active+clean; 4.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 16 KiB/s wr, 15 op/s 2024-04-17T06:10:56.382 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:10:55 smithi098 ceph-mon[36516]: pgmap v301: 97 pgs: 97 active+clean; 4.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 16 KiB/s wr, 15 op/s 2024-04-17T06:10:56.388 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:10:55 smithi047 ceph-mon[32223]: pgmap v301: 97 pgs: 97 active+clean; 4.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 16 KiB/s wr, 15 op/s 2024-04-17T06:10:58.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:10:57 smithi103 ceph-mon[36243]: pgmap v302: 97 pgs: 97 active+clean; 4.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.1 KiB/s wr, 9 op/s 2024-04-17T06:10:58.382 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:10:57 smithi098 ceph-mon[36516]: pgmap v302: 97 pgs: 97 active+clean; 4.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.1 KiB/s wr, 9 op/s 2024-04-17T06:10:58.388 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:10:57 smithi047 ceph-mon[32223]: pgmap v302: 97 pgs: 97 active+clean; 4.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.1 KiB/s wr, 9 op/s 2024-04-17T06:11:00.185 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:10:59 smithi098 ceph-mon[36516]: pgmap v303: 97 pgs: 97 active+clean; 4.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.1 KiB/s wr, 9 op/s 2024-04-17T06:11:00.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:10:59 smithi103 ceph-mon[36243]: pgmap v303: 97 pgs: 97 active+clean; 4.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.1 KiB/s wr, 9 op/s 2024-04-17T06:11:00.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:10:59 smithi047 ceph-mon[32223]: pgmap v303: 97 pgs: 97 active+clean; 4.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.1 KiB/s wr, 9 op/s 2024-04-17T06:11:02.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:11:01 smithi103 ceph-mon[36243]: pgmap v304: 97 pgs: 97 active+clean; 4.9 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-04-17T06:11:02.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:11:01 smithi098 ceph-mon[36516]: pgmap v304: 97 pgs: 97 active+clean; 4.9 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-04-17T06:11:02.388 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:11:01 smithi047 ceph-mon[32223]: pgmap v304: 97 pgs: 97 active+clean; 4.9 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-04-17T06:11:04.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:11:03 smithi103 ceph-mon[36243]: pgmap v305: 97 pgs: 97 active+clean; 4.9 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:11:04.382 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:11:03 smithi098 ceph-mon[36516]: pgmap v305: 97 pgs: 97 active+clean; 4.9 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:11:04.388 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:11:03 smithi047 ceph-mon[32223]: pgmap v305: 97 pgs: 97 active+clean; 4.9 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:11:06.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:11:05 smithi103 ceph-mon[36243]: pgmap v306: 97 pgs: 97 active+clean; 4.9 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 62 KiB/s wr, 33 op/s 2024-04-17T06:11:06.382 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:11:05 smithi098 ceph-mon[36516]: pgmap v306: 97 pgs: 97 active+clean; 4.9 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 62 KiB/s wr, 33 op/s 2024-04-17T06:11:06.388 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:11:05 smithi047 ceph-mon[32223]: pgmap v306: 97 pgs: 97 active+clean; 4.9 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 62 KiB/s wr, 33 op/s 2024-04-17T06:11:08.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:11:07 smithi103 ceph-mon[36243]: pgmap v307: 97 pgs: 97 active+clean; 4.9 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 57 KiB/s wr, 28 op/s 2024-04-17T06:11:08.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:11:07 smithi098 ceph-mon[36516]: pgmap v307: 97 pgs: 97 active+clean; 4.9 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 57 KiB/s wr, 28 op/s 2024-04-17T06:11:08.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:11:07 smithi047 ceph-mon[32223]: pgmap v307: 97 pgs: 97 active+clean; 4.9 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 57 KiB/s wr, 28 op/s 2024-04-17T06:11:09.202 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.201+0000 7f6406eba640 1 -- 172.21.15.47:0/1221615244 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64001021e0 msgr2=0x7f64001025c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:11:09.202 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.201+0000 7f6406eba640 1 --2- 172.21.15.47:0/1221615244 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64001021e0 0x7f64001025c0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f63f4009900 tx=0x7f63f402bc30 comp rx=0 tx=0).stop 2024-04-17T06:11:09.202 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.201+0000 7f6406eba640 1 -- 172.21.15.47:0/1221615244 shutdown_connections 2024-04-17T06:11:09.202 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.201+0000 7f6406eba640 1 --2- 172.21.15.47:0/1221615244 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f64001072b0 0x7f640010ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:09.202 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.201+0000 7f6406eba640 1 --2- 172.21.15.47:0/1221615244 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6400102b90 0x7f6400106bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:09.202 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.201+0000 7f6406eba640 1 --2- 172.21.15.47:0/1221615244 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64001021e0 0x7f64001025c0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:09.202 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.201+0000 7f6406eba640 1 -- 172.21.15.47:0/1221615244 >> 172.21.15.47:0/1221615244 conn(0x7f64000793f0 msgr2=0x7f6400079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:11:09.203 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.201+0000 7f6406eba640 1 -- 172.21.15.47:0/1221615244 shutdown_connections 2024-04-17T06:11:09.203 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.201+0000 7f6406eba640 1 -- 172.21.15.47:0/1221615244 wait complete. 2024-04-17T06:11:09.203 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.202+0000 7f6406eba640 1 Processor -- start 2024-04-17T06:11:09.203 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.202+0000 7f6406eba640 1 -- start start 2024-04-17T06:11:09.203 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.202+0000 7f6406eba640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f64001021e0 0x7f640019e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:09.203 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.202+0000 7f6406eba640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64001072b0 0x7f640019e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:09.203 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.202+0000 7f6406eba640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6400198600 0x7f6400198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:09.203 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.202+0000 7f6406eba640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f64001123c0 con 0x7f64001072b0 2024-04-17T06:11:09.203 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.202+0000 7f6406eba640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6400112240 con 0x7f6400198600 2024-04-17T06:11:09.203 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.202+0000 7f6406eba640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6400112540 con 0x7f64001021e0 2024-04-17T06:11:09.203 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.202+0000 7f63fffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64001072b0 0x7f640019e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:09.203 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.202+0000 7f63fffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64001072b0 0x7f640019e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:42984/0 (socket says 172.21.15.47:42984) 2024-04-17T06:11:09.204 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.202+0000 7f63fffff640 1 -- 172.21.15.47:0/1518065430 learned_addr learned my addr 172.21.15.47:0/1518065430 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:11:09.204 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.203+0000 7f6405457640 1 --2- 172.21.15.47:0/1518065430 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6400198600 0x7f6400198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:09.204 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.203+0000 7f6404c56640 1 --2- 172.21.15.47:0/1518065430 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f64001021e0 0x7f640019e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:09.204 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.203+0000 7f63fffff640 1 -- 172.21.15.47:0/1518065430 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f64001021e0 msgr2=0x7f640019e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:11:09.204 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.203+0000 7f63fffff640 1 --2- 172.21.15.47:0/1518065430 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f64001021e0 0x7f640019e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:09.204 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.203+0000 7f63fffff640 1 -- 172.21.15.47:0/1518065430 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6400198600 msgr2=0x7f6400198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:11:09.204 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.203+0000 7f63fffff640 1 --2- 172.21.15.47:0/1518065430 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6400198600 0x7f6400198ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:09.204 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.203+0000 7f63fffff640 1 -- 172.21.15.47:0/1518065430 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6400199120 con 0x7f64001072b0 2024-04-17T06:11:09.204 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.203+0000 7f63fffff640 1 --2- 172.21.15.47:0/1518065430 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64001072b0 0x7f640019e990 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f63ec00b450 tx=0x7f63ec00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:11:09.204 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.203+0000 7f63fdffb640 1 -- 172.21.15.47:0/1518065430 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63ec015020 con 0x7f64001072b0 2024-04-17T06:11:09.205 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.203+0000 7f63fdffb640 1 -- 172.21.15.47:0/1518065430 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63ec011d70 con 0x7f64001072b0 2024-04-17T06:11:09.205 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.203+0000 7f6406eba640 1 -- 172.21.15.47:0/1518065430 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6400199410 con 0x7f64001072b0 2024-04-17T06:11:09.205 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.203+0000 7f63fdffb640 1 -- 172.21.15.47:0/1518065430 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63ec00f340 con 0x7f64001072b0 2024-04-17T06:11:09.205 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.203+0000 7f6406eba640 1 -- 172.21.15.47:0/1518065430 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64001a5660 con 0x7f64001072b0 2024-04-17T06:11:09.211 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.206+0000 7f63fdffb640 1 -- 172.21.15.47:0/1518065430 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f63ec018020 con 0x7f64001072b0 2024-04-17T06:11:09.212 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.206+0000 7f6409b7f740 1 Processor -- start 2024-04-17T06:11:09.212 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.206+0000 7f6409b7f740 1 -- start start 2024-04-17T06:11:09.212 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.206+0000 7f63fdffb640 1 --2- 172.21.15.47:0/1518065430 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f63f00a3dc0 0x7f63f00a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:09.212 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.206+0000 7f6409b7f740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e539e500c0 0x55e539e504a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:09.212 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.206+0000 7f6409b7f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e539e509e0 0x55e539e59270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:09.212 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.206+0000 7f6409b7f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e539e597b0 0x55e539e5bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:09.212 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.206+0000 7f6409b7f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55e539db57d0 con 0x55e539e597b0 2024-04-17T06:11:09.212 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.206+0000 7f6409b7f740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55e539d52f70 con 0x55e539e500c0 2024-04-17T06:11:09.212 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.206+0000 7f6409b7f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55e539db1790 con 0x55e539e509e0 2024-04-17T06:11:09.212 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.207+0000 7f6405457640 1 --2- 172.21.15.47:0/1518065430 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f63f00a3dc0 0x7f63f00a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:09.212 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.207+0000 7f63fdffb640 1 -- 172.21.15.47:0/1518065430 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8557+0+0 (secure 0 0 0) 0x7f63ec0d0110 con 0x7f64001072b0 2024-04-17T06:11:09.212 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.207+0000 7f6405457640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e539e597b0 0x55e539e5bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:09.212 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.207+0000 7f63fffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e539e509e0 0x55e539e59270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:09.213 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.207+0000 7f6405457640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e539e597b0 0x55e539e5bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43000/0 (socket says 172.21.15.47:43000) 2024-04-17T06:11:09.213 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.207+0000 7f6405457640 1 -- 172.21.15.47:0/3041828648 learned_addr learned my addr 172.21.15.47:0/3041828648 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:11:09.213 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.207+0000 7f6404c56640 1 --2- 172.21.15.47:0/3041828648 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e539e500c0 0x55e539e504a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:09.213 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.207+0000 7f63fffff640 1 -- 172.21.15.47:0/3041828648 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e539e500c0 msgr2=0x55e539e504a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:11:09.213 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.207+0000 7f63fffff640 1 --2- 172.21.15.47:0/3041828648 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e539e500c0 0x55e539e504a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:09.213 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.207+0000 7f63fffff640 1 -- 172.21.15.47:0/3041828648 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e539e597b0 msgr2=0x55e539e5bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:11:09.213 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.207+0000 7f63fffff640 1 --2- 172.21.15.47:0/3041828648 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e539e597b0 0x55e539e5bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:09.213 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.207+0000 7f63fffff640 1 -- 172.21.15.47:0/3041828648 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e539ac8210 con 0x55e539e509e0 2024-04-17T06:11:09.213 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.207+0000 7f6405457640 1 --2- 172.21.15.47:0/3041828648 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e539e597b0 0x55e539e5bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T06:11:09.213 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.207+0000 7f6404c56640 1 --2- 172.21.15.47:0/3041828648 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e539e500c0 0x55e539e504a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T06:11:09.213 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.207+0000 7f6405457640 1 --2- 172.21.15.47:0/1518065430 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f63f00a3dc0 0x7f63f00a6280 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f63f8014c40 tx=0x7f63f800ae50 comp rx=0 tx=0).ready entity=mgr.24923 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:11:09.213 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.207+0000 7f63fffff640 1 --2- 172.21.15.47:0/3041828648 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e539e509e0 0x55e539e59270 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f63ec0cd9f0 tx=0x7f63ec09c140 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c854bd7f459e088 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:11:09.213 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.207+0000 7f63eaffd640 1 -- 172.21.15.47:0/3041828648 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63ec09c940 con 0x55e539e509e0 2024-04-17T06:11:09.214 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.208+0000 7f63eaffd640 1 -- 172.21.15.47:0/3041828648 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63ec0700f0 con 0x55e539e509e0 2024-04-17T06:11:09.214 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.208+0000 7f63eaffd640 1 -- 172.21.15.47:0/3041828648 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63ec0988b0 con 0x55e539e509e0 2024-04-17T06:11:09.214 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.208+0000 7f6409b7f740 1 -- 172.21.15.47:0/3041828648 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e539e509e0 msgr2=0x55e539e59270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:11:09.214 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.208+0000 7f6409b7f740 1 --2- 172.21.15.47:0/3041828648 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e539e509e0 0x55e539e59270 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f63ec0cd9f0 tx=0x7f63ec09c140 comp rx=0 tx=0).stop 2024-04-17T06:11:09.214 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.208+0000 7f6409b7f740 1 -- 172.21.15.47:0/3041828648 shutdown_connections 2024-04-17T06:11:09.214 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.208+0000 7f6409b7f740 1 --2- 172.21.15.47:0/3041828648 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e539e597b0 0x55e539e5bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:09.214 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.208+0000 7f6409b7f740 1 --2- 172.21.15.47:0/3041828648 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e539e509e0 0x55e539e59270 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:09.214 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.208+0000 7f6409b7f740 1 --2- 172.21.15.47:0/3041828648 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e539e500c0 0x55e539e504a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:09.214 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.208+0000 7f6409b7f740 1 -- 172.21.15.47:0/3041828648 >> 172.21.15.47:0/3041828648 conn(0x55e539e4a9e0 msgr2=0x55e539e4adc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:11:09.214 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.208+0000 7f6409b7f740 1 -- 172.21.15.47:0/3041828648 shutdown_connections 2024-04-17T06:11:09.214 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.208+0000 7f6409b7f740 1 -- 172.21.15.47:0/3041828648 wait complete. 2024-04-17T06:11:09.214 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.209+0000 7f6409b7f740 1 -- start start 2024-04-17T06:11:09.215 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.209+0000 7f6409b7f740 1 Processor -- start 2024-04-17T06:11:09.215 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.209+0000 7f6409b7f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e539e500c0 0x55e539e568d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:09.215 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.209+0000 7f6409b7f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e539e509e0 0x55e539e56e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:09.215 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.209+0000 7f6409b7f740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e539e597b0 0x55e539e57350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:09.215 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.209+0000 7f6409b7f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55e539e5c3e0 con 0x55e539e509e0 2024-04-17T06:11:09.215 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.209+0000 7f6409b7f740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55e539d3de90 con 0x55e539e597b0 2024-04-17T06:11:09.215 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.209+0000 7f6409b7f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55e539e5c560 con 0x55e539e500c0 2024-04-17T06:11:09.215 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.209+0000 7f63fffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e539e509e0 0x55e539e56e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:09.215 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.209+0000 7f6409b7f740 10 client.0 _ll_register_callbacks cb 0x55e539d53d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:11:09.215 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.209+0000 7f63fffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e539e509e0 0x55e539e56e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43002/0 (socket says 172.21.15.47:43002) 2024-04-17T06:11:09.215 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.209+0000 7f6404c56640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e539e500c0 0x55e539e568d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:09.215 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.209+0000 7f63fffff640 1 -- 172.21.15.47:0/119502749 learned_addr learned my addr 172.21.15.47:0/119502749 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:11:09.215 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.210+0000 7f6405457640 1 --2- 172.21.15.47:0/119502749 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e539e597b0 0x55e539e57350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:09.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.210+0000 7f63fffff640 1 -- 172.21.15.47:0/119502749 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e539e500c0 msgr2=0x55e539e568d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:11:09.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.210+0000 7f63fffff640 1 --2- 172.21.15.47:0/119502749 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e539e500c0 0x55e539e568d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:09.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.210+0000 7f63fffff640 1 -- 172.21.15.47:0/119502749 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e539e597b0 msgr2=0x55e539e57350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:11:09.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.210+0000 7f63fffff640 1 --2- 172.21.15.47:0/119502749 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e539e597b0 0x55e539e57350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:09.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.210+0000 7f63fffff640 1 -- 172.21.15.47:0/119502749 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e539b94620 con 0x55e539e509e0 2024-04-17T06:11:09.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.210+0000 7f6405457640 1 --2- 172.21.15.47:0/119502749 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e539e597b0 0x55e539e57350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:11:09.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.210+0000 7f63fffff640 1 --2- 172.21.15.47:0/119502749 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e539e509e0 0x55e539e56e10 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f63ec009fd0 tx=0x7f63ec098ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f626b8d3f7c9b7e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:11:09.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.210+0000 7f63dffff640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-17T06:11:09.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.210+0000 7f63dffff640 1 -- 172.21.15.47:0/119502749 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63ec09c940 con 0x55e539e509e0 2024-04-17T06:11:09.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.210+0000 7f63dffff640 1 -- 172.21.15.47:0/119502749 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8557+0+0 (secure 0 0 0) 0x7f63ec076ad0 con 0x55e539e509e0 2024-04-17T06:11:09.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.210+0000 7f6409b7f740 1 -- 172.21.15.47:0/119502749 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63ec0700f0 con 0x55e539e509e0 2024-04-17T06:11:09.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.210+0000 7f63dffff640 1 -- 172.21.15.47:0/119502749 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f63ec0b2e60 con 0x55e539e509e0 2024-04-17T06:11:09.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.210+0000 7f6409b7f740 10 client.16221 fetch_fsmap learned FSMap version 34 2024-04-17T06:11:09.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.210+0000 7f6409b7f740 1 -- 172.21.15.47:0/119502749 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55e539e68730 con 0x55e539e509e0 2024-04-17T06:11:09.217 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.210+0000 7f63dffff640 1 -- 172.21.15.47:0/119502749 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63ec0700f0 con 0x55e539e509e0 2024-04-17T06:11:09.217 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.211+0000 7f63dffff640 1 -- 172.21.15.47:0/119502749 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 34) ==== 8339+0+0 (secure 0 0 0) 0x7f63ec0b0ce0 con 0x55e539e509e0 2024-04-17T06:11:09.217 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.211+0000 7f6409b7f740 10 client.16221 fetch_fsmap finished waiting for FSMap version 34 2024-04-17T06:11:09.217 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.211+0000 7f6409b7f740 10 client.16221 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-17T06:11:09.217 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.211+0000 7f6409b7f740 20 client.16221 populate_metadata read hostname 'smithi047' 2024-04-17T06:11:09.217 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.211+0000 7f6409b7f740 1 --2- 172.21.15.47:0/119502749 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55e539e69a80 0x55e539e6be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:09.217 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.211+0000 7f6409b7f740 4 client.16221 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T06:11:09.217 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.211+0000 7f6409b7f740 1 -- 172.21.15.47:0/119502749 --> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e539dc31f0 con 0x55e539e69a80 2024-04-17T06:11:09.217 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.211+0000 7f6404c56640 1 --2- 172.21.15.47:0/119502749 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55e539e69a80 0x55e539e6be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:09.217 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.211+0000 7f6404c56640 1 --2- 172.21.15.47:0/119502749 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55e539e69a80 0x55e539e6be70 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e0659f34ddff81d server_cookie=d56a106df7147532 in_seq=0 out_seq=0 2024-04-17T06:11:09.217 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.211+0000 7f63dffff640 10 client.16221 ms_handle_connect on v2:172.21.15.98:6838/2841287917 2024-04-17T06:11:09.217 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.212+0000 7f63dffff640 1 -- 172.21.15.47:0/119502749 <== mds.0 v2:172.21.15.98:6838/2841287917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e539dc31f0 con 0x55e539e69a80 2024-04-17T06:11:09.217 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.212+0000 7f63dffff640 10 client.16221 handle_command_reply: tid=0 2024-04-17T06:11:09.218 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.212+0000 7f6409b7f740 1 client.16221 shutdown 2024-04-17T06:11:09.218 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.212+0000 7f6409b7f740 1 -- 172.21.15.47:0/119502749 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e539e509e0 msgr2=0x55e539e56e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:11:09.218 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.212+0000 7f6409b7f740 1 --2- 172.21.15.47:0/119502749 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e539e509e0 0x55e539e56e10 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f63ec009fd0 tx=0x7f63ec098ae0 comp rx=0 tx=0).stop 2024-04-17T06:11:09.218 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.212+0000 7f6409b7f740 1 -- 172.21.15.47:0/119502749 shutdown_connections 2024-04-17T06:11:09.218 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.212+0000 7f6409b7f740 1 --2- 172.21.15.47:0/119502749 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55e539e69a80 0x55e539e6be70 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:09.218 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.212+0000 7f6409b7f740 1 --2- 172.21.15.47:0/119502749 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e539e597b0 0x55e539e57350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:09.218 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.212+0000 7f6409b7f740 1 --2- 172.21.15.47:0/119502749 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e539e509e0 0x55e539e56e10 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:09.218 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.212+0000 7f6409b7f740 1 --2- 172.21.15.47:0/119502749 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e539e500c0 0x55e539e568d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:09.218 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.212+0000 7f63dffff640 0 client.16221 ms_handle_reset on v2:172.21.15.98:6838/2841287917 2024-04-17T06:11:09.218 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.212+0000 7f6409b7f740 1 -- 172.21.15.47:0/119502749 >> 172.21.15.47:0/119502749 conn(0x55e539e4a9e0 msgr2=0x55e539e4c730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:11:09.218 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.213+0000 7f6409b7f740 1 -- 172.21.15.47:0/119502749 shutdown_connections 2024-04-17T06:11:09.218 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.213+0000 7f6409b7f740 1 -- 172.21.15.47:0/119502749 wait complete. 2024-04-17T06:11:09.218 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.213+0000 7f6409b7f740 20 client.16221 trim_cache size 0 max 16384 2024-04-17T06:11:09.228 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.228+0000 7f6409b7f740 1 Processor -- start 2024-04-17T06:11:09.228 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.228+0000 7f6409b7f740 1 -- start start 2024-04-17T06:11:09.228 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.228+0000 7f6409b7f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e539e4a9e0 0x55e539e59000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:09.228 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.228+0000 7f6409b7f740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e539e500c0 0x55e539e57170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:09.228 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.228+0000 7f6409b7f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e539e509e0 0x55e539e53b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:09.228 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.228+0000 7f6409b7f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f64001b0600 con 0x55e539e4a9e0 2024-04-17T06:11:09.228 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.228+0000 7f6409b7f740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f64001adec0 con 0x55e539e500c0 2024-04-17T06:11:09.228 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.228+0000 7f6409b7f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f64001abd30 con 0x55e539e509e0 2024-04-17T06:11:09.229 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.228+0000 7f6404c56640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e539e4a9e0 0x55e539e59000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:09.229 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.228+0000 7f6404c56640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e539e4a9e0 0x55e539e59000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43008/0 (socket says 172.21.15.47:43008) 2024-04-17T06:11:09.229 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.228+0000 7f6404c56640 1 -- 172.21.15.47:0/3543619776 learned_addr learned my addr 172.21.15.47:0/3543619776 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:11:09.229 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.229+0000 7f6405457640 1 --2- 172.21.15.47:0/3543619776 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e539e509e0 0x55e539e53b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:09.229 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.229+0000 7f63fffff640 1 --2- 172.21.15.47:0/3543619776 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e539e500c0 0x55e539e57170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:09.229 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.229+0000 7f6404c56640 1 -- 172.21.15.47:0/3543619776 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e539e509e0 msgr2=0x55e539e53b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:11:09.229 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.229+0000 7f6404c56640 1 --2- 172.21.15.47:0/3543619776 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e539e509e0 0x55e539e53b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:09.229 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.229+0000 7f6404c56640 1 -- 172.21.15.47:0/3543619776 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e539e500c0 msgr2=0x55e539e57170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:11:09.229 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.229+0000 7f6404c56640 1 --2- 172.21.15.47:0/3543619776 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e539e500c0 0x55e539e57170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:09.229 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.229+0000 7f6404c56640 1 -- 172.21.15.47:0/3543619776 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e539e662d0 con 0x55e539e4a9e0 2024-04-17T06:11:09.229 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.229+0000 7f6405457640 1 --2- 172.21.15.47:0/3543619776 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e539e509e0 0x55e539e53b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:11:09.229 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.229+0000 7f6404c56640 1 --2- 172.21.15.47:0/3543619776 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e539e4a9e0 0x55e539e59000 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f63f402cfd0 tx=0x7f63f4005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c759db269b24bf1 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:11:09.229 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.229+0000 7f63ea7fc640 1 -- 172.21.15.47:0/3543619776 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f4047bc0 con 0x55e539e4a9e0 2024-04-17T06:11:09.229 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.229+0000 7f63ea7fc640 1 -- 172.21.15.47:0/3543619776 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63f4047d60 con 0x55e539e4a9e0 2024-04-17T06:11:09.230 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.229+0000 7f63ea7fc640 1 -- 172.21.15.47:0/3543619776 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f4004920 con 0x55e539e4a9e0 2024-04-17T06:11:09.230 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.230+0000 7f6409b7f740 1 -- 172.21.15.47:0/3543619776 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e539e4a9e0 msgr2=0x55e539e59000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:11:09.230 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.230+0000 7f6409b7f740 1 --2- 172.21.15.47:0/3543619776 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e539e4a9e0 0x55e539e59000 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f63f402cfd0 tx=0x7f63f4005a20 comp rx=0 tx=0).stop 2024-04-17T06:11:09.230 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.230+0000 7f6409b7f740 1 -- 172.21.15.47:0/3543619776 shutdown_connections 2024-04-17T06:11:09.230 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.230+0000 7f6409b7f740 1 --2- 172.21.15.47:0/3543619776 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e539e509e0 0x55e539e53b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:09.230 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.230+0000 7f6409b7f740 1 --2- 172.21.15.47:0/3543619776 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e539e500c0 0x55e539e57170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:09.230 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.230+0000 7f6409b7f740 1 --2- 172.21.15.47:0/3543619776 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e539e4a9e0 0x55e539e59000 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:09.230 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.230+0000 7f6409b7f740 1 -- 172.21.15.47:0/3543619776 >> 172.21.15.47:0/3543619776 conn(0x55e539e45e70 msgr2=0x55e539e4adc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:11:09.230 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.230+0000 7f6409b7f740 1 -- 172.21.15.47:0/3543619776 shutdown_connections 2024-04-17T06:11:09.230 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.230+0000 7f6409b7f740 1 -- 172.21.15.47:0/3543619776 wait complete. 2024-04-17T06:11:09.230 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.230+0000 7f6409b7f740 1 -- start start 2024-04-17T06:11:09.230 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.230+0000 7f6409b7f740 1 Processor -- start 2024-04-17T06:11:09.233 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.231+0000 7f6409b7f740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e539e4a9e0 0x55e539e6f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:09.233 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.231+0000 7f6409b7f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e539e500c0 0x55e539e6c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:09.233 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.231+0000 7f6409b7f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e539e509e0 0x55e539e6ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:09.233 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.231+0000 7f6409b7f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f64001ac380 con 0x55e539e500c0 2024-04-17T06:11:09.234 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.231+0000 7f6409b7f740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f64001ad7f0 con 0x55e539e4a9e0 2024-04-17T06:11:09.234 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.231+0000 7f6409b7f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f64001aca50 con 0x55e539e509e0 2024-04-17T06:11:09.234 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.231+0000 7f6409b7f740 10 client.0 _ll_register_callbacks cb 0x7f63f8014790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:11:09.234 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.231+0000 7f63fffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e539e500c0 0x55e539e6c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:09.234 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.231+0000 7f63fffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e539e500c0 0x55e539e6c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43010/0 (socket says 172.21.15.47:43010) 2024-04-17T06:11:09.234 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.231+0000 7f63fffff640 1 -- 172.21.15.47:0/2678108914 learned_addr learned my addr 172.21.15.47:0/2678108914 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:11:09.234 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.231+0000 7f6405457640 1 --2- 172.21.15.47:0/2678108914 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e539e509e0 0x55e539e6ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:09.234 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.231+0000 7f6404c56640 1 --2- 172.21.15.47:0/2678108914 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e539e4a9e0 0x55e539e6f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:09.234 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.231+0000 7f63fffff640 1 -- 172.21.15.47:0/2678108914 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e539e509e0 msgr2=0x55e539e6ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:11:09.234 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.231+0000 7f63fffff640 1 --2- 172.21.15.47:0/2678108914 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e539e509e0 0x55e539e6ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:09.234 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.231+0000 7f63fffff640 1 -- 172.21.15.47:0/2678108914 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e539e4a9e0 msgr2=0x55e539e6f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:11:09.234 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.231+0000 7f63fffff640 1 --2- 172.21.15.47:0/2678108914 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e539e4a9e0 0x55e539e6f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:09.234 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.231+0000 7f63fffff640 1 -- 172.21.15.47:0/2678108914 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e539ed3070 con 0x55e539e500c0 2024-04-17T06:11:09.235 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.231+0000 7f63fffff640 1 --2- 172.21.15.47:0/2678108914 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e539e500c0 0x55e539e6c760 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f63ec098c10 tx=0x7f63ec09bc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e99505f74724e021 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:11:09.235 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.231+0000 7f63dffff640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-17T06:11:09.235 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.231+0000 7f63dffff640 1 -- 172.21.15.47:0/2678108914 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63ec09c940 con 0x55e539e500c0 2024-04-17T06:11:09.235 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.232+0000 7f63dffff640 1 -- 172.21.15.47:0/2678108914 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8557+0+0 (secure 0 0 0) 0x7f63ec09b8a0 con 0x55e539e500c0 2024-04-17T06:11:09.235 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.232+0000 7f6409b7f740 1 -- 172.21.15.47:0/2678108914 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63f4047d60 con 0x55e539e500c0 2024-04-17T06:11:09.235 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.232+0000 7f63dffff640 1 -- 172.21.15.47:0/2678108914 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f63ec0a05f0 con 0x55e539e500c0 2024-04-17T06:11:09.235 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.232+0000 7f6409b7f740 10 client.16227 fetch_fsmap learned FSMap version 34 2024-04-17T06:11:09.235 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.232+0000 7f6409b7f740 1 -- 172.21.15.47:0/2678108914 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55e539ed3070 con 0x55e539e500c0 2024-04-17T06:11:09.235 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.232+0000 7f63dffff640 1 -- 172.21.15.47:0/2678108914 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63f4047d60 con 0x55e539e500c0 2024-04-17T06:11:09.235 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.232+0000 7f63dffff640 1 -- 172.21.15.47:0/2678108914 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 34) ==== 8339+0+0 (secure 0 0 0) 0x7f63ec0b0ce0 con 0x55e539e500c0 2024-04-17T06:11:09.235 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.232+0000 7f6409b7f740 10 client.16227 fetch_fsmap finished waiting for FSMap version 34 2024-04-17T06:11:09.235 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.232+0000 7f6409b7f740 10 client.16227 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-17T06:11:09.235 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.232+0000 7f6409b7f740 20 client.16227 populate_metadata read hostname 'smithi047' 2024-04-17T06:11:09.235 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.232+0000 7f6409b7f740 1 --2- 172.21.15.47:0/2678108914 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55e539e597b0 0x55e539ed6530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:09.236 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.232+0000 7f6409b7f740 4 client.16227 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-17T06:11:09.236 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.232+0000 7f6409b7f740 1 -- 172.21.15.47:0/2678108914 --> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e539dbd060 con 0x55e539e597b0 2024-04-17T06:11:09.236 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.232+0000 7f6404c56640 1 --2- 172.21.15.47:0/2678108914 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55e539e597b0 0x55e539ed6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:09.236 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.233+0000 7f6404c56640 1 --2- 172.21.15.47:0/2678108914 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55e539e597b0 0x55e539ed6530 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11e1938019af76fe server_cookie=5554ccb20f439f94 in_seq=0 out_seq=0 2024-04-17T06:11:09.236 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.233+0000 7f63dffff640 10 client.16227 ms_handle_connect on v2:172.21.15.98:6838/2841287917 2024-04-17T06:11:09.247 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.248+0000 7f63dffff640 1 -- 172.21.15.47:0/2678108914 <== mds.0 v2:172.21.15.98:6838/2841287917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e539dbd060 con 0x55e539e597b0 2024-04-17T06:11:09.247 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.248+0000 7f63dffff640 10 client.16227 handle_command_reply: tid=0 2024-04-17T06:11:09.247 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.248+0000 7f6409b7f740 1 client.16227 shutdown 2024-04-17T06:11:09.247 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.248+0000 7f6409b7f740 1 -- 172.21.15.47:0/2678108914 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e539e500c0 msgr2=0x55e539e6c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:11:09.247 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.248+0000 7f6409b7f740 1 --2- 172.21.15.47:0/2678108914 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e539e500c0 0x55e539e6c760 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f63ec098c10 tx=0x7f63ec09bc70 comp rx=0 tx=0).stop 2024-04-17T06:11:09.247 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.248+0000 7f6409b7f740 1 -- 172.21.15.47:0/2678108914 shutdown_connections 2024-04-17T06:11:09.247 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.248+0000 7f6409b7f740 1 --2- 172.21.15.47:0/2678108914 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55e539e597b0 0x55e539ed6530 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:09.247 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.248+0000 7f6409b7f740 1 --2- 172.21.15.47:0/2678108914 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e539e509e0 0x55e539e6ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:09.247 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.248+0000 7f63dffff640 0 client.16227 ms_handle_reset on v2:172.21.15.98:6838/2841287917 2024-04-17T06:11:09.247 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.248+0000 7f6409b7f740 1 --2- 172.21.15.47:0/2678108914 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e539e500c0 0x55e539e6c760 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:09.247 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.248+0000 7f6409b7f740 1 --2- 172.21.15.47:0/2678108914 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e539e4a9e0 0x55e539e6f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:09.248 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.248+0000 7f6409b7f740 1 -- 172.21.15.47:0/2678108914 >> 172.21.15.47:0/2678108914 conn(0x55e539e45e70 msgr2=0x55e539e4be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:11:09.248 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.249+0000 7f6409b7f740 1 -- 172.21.15.47:0/2678108914 shutdown_connections 2024-04-17T06:11:09.248 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.249+0000 7f6409b7f740 1 -- 172.21.15.47:0/2678108914 wait complete. 2024-04-17T06:11:09.248 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.249+0000 7f6409b7f740 20 client.16227 trim_cache size 0 max 16384 2024-04-17T06:11:09.248 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-04-17T06:11:09.248 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-04-17T06:11:09.248 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-04-17T06:11:09.248 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-04-17T06:11:09.249 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.249+0000 7f6406eba640 1 -- 172.21.15.47:0/1518065430 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f63f00a3dc0 msgr2=0x7f63f00a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:11:09.249 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.249+0000 7f6406eba640 1 --2- 172.21.15.47:0/1518065430 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f63f00a3dc0 0x7f63f00a6280 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f63f8014c40 tx=0x7f63f800ae50 comp rx=0 tx=0).stop 2024-04-17T06:11:09.249 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.249+0000 7f6406eba640 1 -- 172.21.15.47:0/1518065430 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64001072b0 msgr2=0x7f640019e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:11:09.249 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.249+0000 7f6406eba640 1 --2- 172.21.15.47:0/1518065430 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64001072b0 0x7f640019e990 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f63ec00b450 tx=0x7f63ec00b910 comp rx=0 tx=0).stop 2024-04-17T06:11:09.249 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.249+0000 7f6406eba640 1 -- 172.21.15.47:0/1518065430 shutdown_connections 2024-04-17T06:11:09.249 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.249+0000 7f6406eba640 1 --2- 172.21.15.47:0/1518065430 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6400198600 0x7f6400198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:09.249 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.249+0000 7f6406eba640 1 --2- 172.21.15.47:0/1518065430 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f63f00a3dc0 0x7f63f00a6280 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:09.249 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.249+0000 7f6406eba640 1 --2- 172.21.15.47:0/1518065430 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64001072b0 0x7f640019e990 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:09.250 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.249+0000 7f6406eba640 1 --2- 172.21.15.47:0/1518065430 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f64001021e0 0x7f640019e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:09.250 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.249+0000 7f6406eba640 1 -- 172.21.15.47:0/1518065430 >> 172.21.15.47:0/1518065430 conn(0x7f64000793f0 msgr2=0x7f64000ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:11:09.250 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.250+0000 7f6406eba640 1 -- 172.21.15.47:0/1518065430 shutdown_connections 2024-04-17T06:11:09.250 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:09.250+0000 7f6406eba640 1 -- 172.21.15.47:0/1518065430 wait complete. 2024-04-17T06:11:10.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:11:09 smithi103 ceph-mon[36243]: pgmap v308: 97 pgs: 97 active+clean; 4.9 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 57 KiB/s wr, 28 op/s 2024-04-17T06:11:10.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:11:09 smithi103 ceph-mon[36243]: osdmap e100: 12 total, 12 up, 12 in 2024-04-17T06:11:10.382 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:11:09 smithi098 ceph-mon[36516]: pgmap v308: 97 pgs: 97 active+clean; 4.9 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 57 KiB/s wr, 28 op/s 2024-04-17T06:11:10.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:11:09 smithi098 ceph-mon[36516]: osdmap e100: 12 total, 12 up, 12 in 2024-04-17T06:11:10.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:11:09 smithi047 ceph-mon[32223]: pgmap v308: 97 pgs: 97 active+clean; 4.9 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 57 KiB/s wr, 28 op/s 2024-04-17T06:11:10.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:11:09 smithi047 ceph-mon[32223]: osdmap e100: 12 total, 12 up, 12 in 2024-04-17T06:11:12.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:11:11 smithi103 ceph-mon[36243]: pgmap v310: 97 pgs: 97 active+clean; 5.0 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 68 KiB/s wr, 34 op/s 2024-04-17T06:11:12.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:11:11 smithi098 ceph-mon[36516]: pgmap v310: 97 pgs: 97 active+clean; 5.0 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 68 KiB/s wr, 34 op/s 2024-04-17T06:11:12.388 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:11:11 smithi047 ceph-mon[32223]: pgmap v310: 97 pgs: 97 active+clean; 5.0 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 68 KiB/s wr, 34 op/s 2024-04-17T06:11:13.522 INFO:tasks.workunit.client.0.smithi047.stdout:437/600: open fd = 3 2024-04-17T06:11:13.522 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.522 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.522 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:11:13.522 INFO:tasks.workunit.client.0.smithi047.stdout:438/600: open fd = 3 2024-04-17T06:11:13.522 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.522 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.522 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:11:13.522 INFO:tasks.workunit.client.0.smithi047.stdout:439/600: open fd = 3 2024-04-17T06:11:13.523 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.523 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.523 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:11:13.523 INFO:tasks.workunit.client.0.smithi047.stdout:440/600: open fd = 3 2024-04-17T06:11:13.523 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.523 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.523 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:11:13.523 INFO:tasks.workunit.client.0.smithi047.stdout:441/600: open fd = 3 2024-04-17T06:11:13.523 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.523 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.523 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:11:13.523 INFO:tasks.workunit.client.0.smithi047.stdout:442/600: open fd = 3 2024-04-17T06:11:13.523 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.524 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.524 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:11:13.524 INFO:tasks.workunit.client.0.smithi047.stdout:443/600: open fd = 3 2024-04-17T06:11:13.524 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.524 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.524 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:11:13.524 INFO:tasks.workunit.client.0.smithi047.stdout:444/600: open fd = 3 2024-04-17T06:11:13.524 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.524 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.524 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:11:13.524 INFO:tasks.workunit.client.0.smithi047.stdout:445/600: open fd = 3 2024-04-17T06:11:13.524 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.524 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.524 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:11:13.524 INFO:tasks.workunit.client.0.smithi047.stdout:446/600: open fd = 3 2024-04-17T06:11:13.525 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.525 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.525 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:11:13.525 INFO:tasks.workunit.client.0.smithi047.stdout:447/600: open fd = 3 2024-04-17T06:11:13.525 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.525 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.525 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:11:13.525 INFO:tasks.workunit.client.0.smithi047.stdout:448/600: open fd = 3 2024-04-17T06:11:13.525 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.525 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.525 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:11:13.525 INFO:tasks.workunit.client.0.smithi047.stdout:449/600: open fd = 3 2024-04-17T06:11:13.525 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.525 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.525 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:11:13.526 INFO:tasks.workunit.client.0.smithi047.stdout:450/600: open fd = 3 2024-04-17T06:11:13.526 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.526 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.526 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:11:13.526 INFO:tasks.workunit.client.0.smithi047.stdout:451/600: open fd = 3 2024-04-17T06:11:13.526 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.526 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.526 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:11:13.526 INFO:tasks.workunit.client.0.smithi047.stdout:452/600: open fd = 3 2024-04-17T06:11:13.526 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.526 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.526 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:11:13.526 INFO:tasks.workunit.client.0.smithi047.stdout:453/600: open fd = 3 2024-04-17T06:11:13.526 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.527 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.527 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:11:13.527 INFO:tasks.workunit.client.0.smithi047.stdout:454/600: open fd = 3 2024-04-17T06:11:13.527 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.527 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.527 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:11:13.527 INFO:tasks.workunit.client.0.smithi047.stdout:455/600: open fd = 3 2024-04-17T06:11:13.527 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.527 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.527 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:11:13.527 INFO:tasks.workunit.client.0.smithi047.stdout:456/600: open fd = 3 2024-04-17T06:11:13.527 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.527 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.527 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:11:13.527 INFO:tasks.workunit.client.0.smithi047.stdout:457/600: open fd = 3 2024-04-17T06:11:13.527 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.528 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.528 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:11:13.528 INFO:tasks.workunit.client.0.smithi047.stdout:458/600: open fd = 3 2024-04-17T06:11:13.528 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.528 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.528 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:11:13.528 INFO:tasks.workunit.client.0.smithi047.stdout:459/600: open fd = 3 2024-04-17T06:11:13.528 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.528 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.528 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:11:13.528 INFO:tasks.workunit.client.0.smithi047.stdout:460/600: open fd = 3 2024-04-17T06:11:13.528 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.528 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.528 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:11:13.528 INFO:tasks.workunit.client.0.smithi047.stdout:461/600: open fd = 3 2024-04-17T06:11:13.529 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.529 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.529 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:11:13.529 INFO:tasks.workunit.client.0.smithi047.stdout:462/600: open fd = 3 2024-04-17T06:11:13.529 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.529 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.529 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:11:13.529 INFO:tasks.workunit.client.0.smithi047.stdout:463/600: open fd = 3 2024-04-17T06:11:13.529 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.529 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.529 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:11:13.529 INFO:tasks.workunit.client.0.smithi047.stdout:464/600: open fd = 3 2024-04-17T06:11:13.529 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.529 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.529 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:11:13.529 INFO:tasks.workunit.client.0.smithi047.stdout:465/600: open fd = 3 2024-04-17T06:11:13.530 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.530 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.530 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:11:13.530 INFO:tasks.workunit.client.0.smithi047.stdout:466/600: open fd = 3 2024-04-17T06:11:13.530 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.530 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.530 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:11:13.530 INFO:tasks.workunit.client.0.smithi047.stdout:467/600: open fd = 3 2024-04-17T06:11:13.530 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.530 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.530 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:11:13.530 INFO:tasks.workunit.client.0.smithi047.stdout:468/600: open fd = 3 2024-04-17T06:11:13.530 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.530 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.531 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:11:13.531 INFO:tasks.workunit.client.0.smithi047.stdout:469/600: open fd = 3 2024-04-17T06:11:13.531 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.531 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.531 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:11:13.531 INFO:tasks.workunit.client.0.smithi047.stdout:470/600: open fd = 3 2024-04-17T06:11:13.531 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.531 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.531 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:11:13.531 INFO:tasks.workunit.client.0.smithi047.stdout:471/600: open fd = 3 2024-04-17T06:11:13.531 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.531 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.531 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:11:13.531 INFO:tasks.workunit.client.0.smithi047.stdout:472/600: open fd = 3 2024-04-17T06:11:13.531 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.531 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.532 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:11:13.532 INFO:tasks.workunit.client.0.smithi047.stdout:473/600: open fd = 3 2024-04-17T06:11:13.532 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.532 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.532 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:11:13.532 INFO:tasks.workunit.client.0.smithi047.stdout:474/600: open fd = 3 2024-04-17T06:11:13.532 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.532 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.532 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:11:13.532 INFO:tasks.workunit.client.0.smithi047.stdout:475/600: open fd = 3 2024-04-17T06:11:13.532 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.532 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.532 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:11:13.532 INFO:tasks.workunit.client.0.smithi047.stdout:476/600: open fd = 3 2024-04-17T06:11:13.532 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.532 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.533 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:11:13.533 INFO:tasks.workunit.client.0.smithi047.stdout:477/600: open fd = 3 2024-04-17T06:11:13.533 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.533 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.533 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:11:13.533 INFO:tasks.workunit.client.0.smithi047.stdout:478/600: open fd = 3 2024-04-17T06:11:13.533 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.533 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.533 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:11:13.533 INFO:tasks.workunit.client.0.smithi047.stdout:479/600: open fd = 3 2024-04-17T06:11:13.533 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.533 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.533 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:11:13.533 INFO:tasks.workunit.client.0.smithi047.stdout:480/600: open fd = 3 2024-04-17T06:11:13.533 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.533 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.533 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:11:13.534 INFO:tasks.workunit.client.0.smithi047.stdout:481/600: open fd = 3 2024-04-17T06:11:13.534 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.534 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.534 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:11:13.534 INFO:tasks.workunit.client.0.smithi047.stdout:482/600: open fd = 3 2024-04-17T06:11:13.534 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.534 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.534 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:11:13.534 INFO:tasks.workunit.client.0.smithi047.stdout:483/600: open fd = 3 2024-04-17T06:11:13.534 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.534 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.534 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:11:13.534 INFO:tasks.workunit.client.0.smithi047.stdout:484/600: open fd = 3 2024-04-17T06:11:13.534 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.535 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.535 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:11:13.535 INFO:tasks.workunit.client.0.smithi047.stdout:485/600: open fd = 3 2024-04-17T06:11:13.535 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.535 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.535 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:11:13.535 INFO:tasks.workunit.client.0.smithi047.stdout:486/600: open fd = 3 2024-04-17T06:11:13.535 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.535 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.535 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:11:13.535 INFO:tasks.workunit.client.0.smithi047.stdout:487/600: open fd = 3 2024-04-17T06:11:13.535 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.535 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.535 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:11:13.535 INFO:tasks.workunit.client.0.smithi047.stdout:488/600: open fd = 3 2024-04-17T06:11:13.536 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.536 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.536 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:11:13.536 INFO:tasks.workunit.client.0.smithi047.stdout:489/600: open fd = 3 2024-04-17T06:11:13.536 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.536 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.536 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:11:13.536 INFO:tasks.workunit.client.0.smithi047.stdout:490/600: open fd = 3 2024-04-17T06:11:13.536 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.536 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.536 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:11:13.536 INFO:tasks.workunit.client.0.smithi047.stdout:491/600: open fd = 3 2024-04-17T06:11:13.536 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.536 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.536 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:11:13.537 INFO:tasks.workunit.client.0.smithi047.stdout:492/600: open fd = 3 2024-04-17T06:11:13.537 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.537 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.537 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:11:13.537 INFO:tasks.workunit.client.0.smithi047.stdout:493/600: open fd = 3 2024-04-17T06:11:13.537 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.537 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.537 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:11:13.537 INFO:tasks.workunit.client.0.smithi047.stdout:494/600: open fd = 3 2024-04-17T06:11:13.537 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.537 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.537 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:11:13.537 INFO:tasks.workunit.client.0.smithi047.stdout:495/600: open fd = 3 2024-04-17T06:11:13.538 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.538 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.538 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:11:13.538 INFO:tasks.workunit.client.0.smithi047.stdout:496/600: open fd = 3 2024-04-17T06:11:13.538 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.538 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.538 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:11:13.538 INFO:tasks.workunit.client.0.smithi047.stdout:497/600: open fd = 3 2024-04-17T06:11:13.538 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.538 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.538 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:11:13.538 INFO:tasks.workunit.client.0.smithi047.stdout:498/600: open fd = 3 2024-04-17T06:11:13.538 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.538 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:11:13.538 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:11:14.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:11:13 smithi103 ceph-mon[36243]: pgmap v311: 97 pgs: 97 active+clean; 5.0 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 68 KiB/s wr, 34 op/s 2024-04-17T06:11:14.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:11:13 smithi098 ceph-mon[36516]: pgmap v311: 97 pgs: 97 active+clean; 5.0 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 68 KiB/s wr, 34 op/s 2024-04-17T06:11:14.388 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:11:13 smithi047 ceph-mon[32223]: pgmap v311: 97 pgs: 97 active+clean; 5.0 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 68 KiB/s wr, 34 op/s 2024-04-17T06:11:16.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:11:15 smithi103 ceph-mon[36243]: pgmap v312: 97 pgs: 97 active+clean; 5.0 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-17T06:11:16.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:11:15 smithi103 ceph-mon[36243]: osdmap e101: 12 total, 12 up, 12 in 2024-04-17T06:11:16.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:11:15 smithi098 ceph-mon[36516]: pgmap v312: 97 pgs: 97 active+clean; 5.0 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-17T06:11:16.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:11:15 smithi098 ceph-mon[36516]: osdmap e101: 12 total, 12 up, 12 in 2024-04-17T06:11:16.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:11:15 smithi047 ceph-mon[32223]: pgmap v312: 97 pgs: 97 active+clean; 5.0 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-17T06:11:16.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:11:15 smithi047 ceph-mon[32223]: osdmap e101: 12 total, 12 up, 12 in 2024-04-17T06:11:18.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:11:17 smithi103 ceph-mon[36243]: pgmap v314: 97 pgs: 97 active+clean; 5.0 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-04-17T06:11:18.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:11:17 smithi098 ceph-mon[36516]: pgmap v314: 97 pgs: 97 active+clean; 5.0 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-04-17T06:11:18.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:11:17 smithi047 ceph-mon[32223]: pgmap v314: 97 pgs: 97 active+clean; 5.0 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-04-17T06:11:20.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:11:19 smithi103 ceph-mon[36243]: pgmap v315: 97 pgs: 97 active+clean; 5.0 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 KiB/s wr, 14 op/s 2024-04-17T06:11:20.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:11:19 smithi098 ceph-mon[36516]: pgmap v315: 97 pgs: 97 active+clean; 5.0 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 KiB/s wr, 14 op/s 2024-04-17T06:11:20.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:11:19 smithi047 ceph-mon[32223]: pgmap v315: 97 pgs: 97 active+clean; 5.0 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 KiB/s wr, 14 op/s 2024-04-17T06:11:22.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:11:21 smithi103 ceph-mon[36243]: pgmap v316: 97 pgs: 97 active+clean; 5.0 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.7 KiB/s wr, 11 op/s 2024-04-17T06:11:22.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:11:21 smithi098 ceph-mon[36516]: pgmap v316: 97 pgs: 97 active+clean; 5.0 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.7 KiB/s wr, 11 op/s 2024-04-17T06:11:22.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:11:21 smithi047 ceph-mon[32223]: pgmap v316: 97 pgs: 97 active+clean; 5.0 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.7 KiB/s wr, 11 op/s 2024-04-17T06:11:22.347 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.348+0000 7f95bec88640 1 -- 172.21.15.47:0/505698667 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95b8101100 msgr2=0x7f95b81014e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:11:22.347 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.348+0000 7f95bec88640 1 --2- 172.21.15.47:0/505698667 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95b8101100 0x7f95b81014e0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f95b4009900 tx=0x7f95b402bc30 comp rx=0 tx=0).stop 2024-04-17T06:11:22.347 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.348+0000 7f95bec88640 1 -- 172.21.15.47:0/505698667 shutdown_connections 2024-04-17T06:11:22.347 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.348+0000 7f95bec88640 1 --2- 172.21.15.47:0/505698667 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f95b8106180 0x7f95b810dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:22.347 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.348+0000 7f95bec88640 1 --2- 172.21.15.47:0/505698667 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f95b8101ab0 0x7f95b8105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:22.347 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.348+0000 7f95bec88640 1 --2- 172.21.15.47:0/505698667 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95b8101100 0x7f95b81014e0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:22.347 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.348+0000 7f95bec88640 1 -- 172.21.15.47:0/505698667 >> 172.21.15.47:0/505698667 conn(0x7f95b8077fe0 msgr2=0x7f95b80783f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:11:22.347 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.348+0000 7f95bec88640 1 -- 172.21.15.47:0/505698667 shutdown_connections 2024-04-17T06:11:22.347 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.348+0000 7f95bec88640 1 -- 172.21.15.47:0/505698667 wait complete. 2024-04-17T06:11:22.347 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.349+0000 7f95bec88640 1 Processor -- start 2024-04-17T06:11:22.347 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.349+0000 7f95bec88640 1 -- start start 2024-04-17T06:11:22.349 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.349+0000 7f95bec88640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95b8101100 0x7f95b819e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:22.349 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.349+0000 7f95bec88640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f95b8106180 0x7f95b819e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:22.349 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.349+0000 7f95bec88640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f95b81985e0 0x7f95b8198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:22.349 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.349+0000 7f95bec88640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f95b8111210 con 0x7f95b8101100 2024-04-17T06:11:22.350 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.349+0000 7f95bec88640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f95b8111090 con 0x7f95b8106180 2024-04-17T06:11:22.350 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.349+0000 7f95bec88640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f95b8111390 con 0x7f95b81985e0 2024-04-17T06:11:22.350 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.350+0000 7f95bca24640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95b8101100 0x7f95b819e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:22.350 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.350+0000 7f95bca24640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95b8101100 0x7f95b819e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55450/0 (socket says 172.21.15.47:55450) 2024-04-17T06:11:22.350 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.350+0000 7f95bca24640 1 -- 172.21.15.47:0/3556406008 learned_addr learned my addr 172.21.15.47:0/3556406008 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:11:22.350 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.350+0000 7f95bd225640 1 --2- 172.21.15.47:0/3556406008 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f95b81985e0 0x7f95b8198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:22.350 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.350+0000 7f95affff640 1 --2- 172.21.15.47:0/3556406008 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f95b8106180 0x7f95b819e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:22.350 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.350+0000 7f95bca24640 1 -- 172.21.15.47:0/3556406008 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f95b81985e0 msgr2=0x7f95b8198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:11:22.350 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.350+0000 7f95bca24640 1 --2- 172.21.15.47:0/3556406008 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f95b81985e0 0x7f95b8198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:22.350 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.350+0000 7f95bca24640 1 -- 172.21.15.47:0/3556406008 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f95b8106180 msgr2=0x7f95b819e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:11:22.350 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.350+0000 7f95bca24640 1 --2- 172.21.15.47:0/3556406008 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f95b8106180 0x7f95b819e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:22.350 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.350+0000 7f95bca24640 1 -- 172.21.15.47:0/3556406008 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95b8199100 con 0x7f95b8101100 2024-04-17T06:11:22.350 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.350+0000 7f95bca24640 1 --2- 172.21.15.47:0/3556406008 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95b8101100 0x7f95b819e430 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f95b4006010 tx=0x7f95b402cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:11:22.350 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.350+0000 7f95adffb640 1 -- 172.21.15.47:0/3556406008 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95b4041070 con 0x7f95b8101100 2024-04-17T06:11:22.351 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.350+0000 7f95adffb640 1 -- 172.21.15.47:0/3556406008 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95b4004450 con 0x7f95b8101100 2024-04-17T06:11:22.351 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.350+0000 7f95adffb640 1 -- 172.21.15.47:0/3556406008 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95b4003e00 con 0x7f95b8101100 2024-04-17T06:11:22.351 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.350+0000 7f95bec88640 1 -- 172.21.15.47:0/3556406008 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95b8199390 con 0x7f95b8101100 2024-04-17T06:11:22.351 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.351+0000 7f95bec88640 1 -- 172.21.15.47:0/3556406008 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95b81a55e0 con 0x7f95b8101100 2024-04-17T06:11:22.353 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.353+0000 7f95adffb640 1 -- 172.21.15.47:0/3556406008 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f95b4044070 con 0x7f95b8101100 2024-04-17T06:11:22.353 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.354+0000 7f95c1879740 1 Processor -- start 2024-04-17T06:11:22.353 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.354+0000 7f95c1879740 1 -- start start 2024-04-17T06:11:22.353 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.354+0000 7f95adffb640 1 --2- 172.21.15.47:0/3556406008 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f95a00a3e40 0x7f95a00a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:22.353 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.354+0000 7f95c1879740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55692823a0c0 0x55692823a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:22.354 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.354+0000 7f95c1879740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55692823a9e0 0x556928243270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:22.354 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.354+0000 7f95c1879740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5569282437b0 0x556928245ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:22.354 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.354+0000 7f95c1879740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55692819f7d0 con 0x55692823a0c0 2024-04-17T06:11:22.354 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.354+0000 7f95c1879740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55692813cf70 con 0x5569282437b0 2024-04-17T06:11:22.354 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.354+0000 7f95c1879740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55692819b790 con 0x55692823a9e0 2024-04-17T06:11:22.354 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.354+0000 7f95bca24640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55692823a0c0 0x55692823a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:22.354 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.354+0000 7f95bd225640 1 --2- 172.21.15.47:0/3556406008 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f95a00a3e40 0x7f95a00a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:22.354 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.354+0000 7f95bca24640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55692823a0c0 0x55692823a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55452/0 (socket says 172.21.15.47:55452) 2024-04-17T06:11:22.354 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.354+0000 7f95bca24640 1 -- 172.21.15.47:0/573480057 learned_addr learned my addr 172.21.15.47:0/573480057 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:11:22.354 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.354+0000 7f95affff640 1 --2- 172.21.15.47:0/573480057 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55692823a9e0 0x556928243270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:22.354 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.354+0000 7f95adffb640 1 -- 172.21.15.47:0/3556406008 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8513+0+0 (secure 0 0 0) 0x7f95b40f2120 con 0x7f95b8101100 2024-04-17T06:11:22.354 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.354+0000 7f95bd225640 1 --2- 172.21.15.47:0/573480057 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5569282437b0 0x556928245ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:22.355 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.355+0000 7f95affff640 1 -- 172.21.15.47:0/573480057 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5569282437b0 msgr2=0x556928245ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:11:22.355 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.355+0000 7f95affff640 1 --2- 172.21.15.47:0/573480057 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5569282437b0 0x556928245ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:22.355 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.355+0000 7f95affff640 1 -- 172.21.15.47:0/573480057 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55692823a0c0 msgr2=0x55692823a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:11:22.355 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.355+0000 7f95affff640 1 --2- 172.21.15.47:0/573480057 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55692823a0c0 0x55692823a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:22.355 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.355+0000 7f95affff640 1 -- 172.21.15.47:0/573480057 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556927eb2210 con 0x55692823a9e0 2024-04-17T06:11:22.355 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.355+0000 7f95bca24640 1 --2- 172.21.15.47:0/573480057 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55692823a0c0 0x55692823a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T06:11:22.355 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.355+0000 7f95bd225640 1 --2- 172.21.15.47:0/3556406008 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f95a00a3e40 0x7f95a00a6300 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f95b000a780 tx=0x7f95b0013a50 comp rx=0 tx=0).ready entity=mgr.24923 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:11:22.355 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.355+0000 7f95affff640 1 --2- 172.21.15.47:0/573480057 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55692823a9e0 0x556928243270 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f95b8199720 tx=0x7f95a8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d68670714e6a01d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:11:22.355 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.355+0000 7f959effd640 1 -- 172.21.15.47:0/573480057 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95a8004050 con 0x55692823a9e0 2024-04-17T06:11:22.355 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.355+0000 7f959effd640 1 -- 172.21.15.47:0/573480057 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95a8005500 con 0x55692823a9e0 2024-04-17T06:11:22.355 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.355+0000 7f959effd640 1 -- 172.21.15.47:0/573480057 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95a8004e00 con 0x55692823a9e0 2024-04-17T06:11:22.359 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.356+0000 7f95c1879740 1 -- 172.21.15.47:0/573480057 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55692823a9e0 msgr2=0x556928243270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:11:22.359 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.356+0000 7f95c1879740 1 --2- 172.21.15.47:0/573480057 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55692823a9e0 0x556928243270 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f95b8199720 tx=0x7f95a8011810 comp rx=0 tx=0).stop 2024-04-17T06:11:22.359 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.356+0000 7f95c1879740 1 -- 172.21.15.47:0/573480057 shutdown_connections 2024-04-17T06:11:22.360 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.356+0000 7f95c1879740 1 --2- 172.21.15.47:0/573480057 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5569282437b0 0x556928245ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:22.360 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.356+0000 7f95c1879740 1 --2- 172.21.15.47:0/573480057 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55692823a9e0 0x556928243270 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:22.360 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.356+0000 7f95c1879740 1 --2- 172.21.15.47:0/573480057 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55692823a0c0 0x55692823a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:22.360 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.356+0000 7f95c1879740 1 -- 172.21.15.47:0/573480057 >> 172.21.15.47:0/573480057 conn(0x5569282349e0 msgr2=0x556928234dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:11:22.360 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.356+0000 7f95c1879740 1 -- 172.21.15.47:0/573480057 shutdown_connections 2024-04-17T06:11:22.360 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.356+0000 7f95c1879740 1 -- 172.21.15.47:0/573480057 wait complete. 2024-04-17T06:11:22.360 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.356+0000 7f95c1879740 1 -- start start 2024-04-17T06:11:22.360 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.356+0000 7f95c1879740 1 Processor -- start 2024-04-17T06:11:22.360 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.357+0000 7f95c1879740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55692823a0c0 0x556928240740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:22.360 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.357+0000 7f95c1879740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55692823a9e0 0x556928240c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:22.360 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.357+0000 7f95c1879740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5569282437b0 0x5569282411c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:22.360 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.357+0000 7f95c1879740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5569282463e0 con 0x5569282437b0 2024-04-17T06:11:22.361 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.357+0000 7f95c1879740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x556928127e90 con 0x55692823a9e0 2024-04-17T06:11:22.361 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.357+0000 7f95c1879740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556928246560 con 0x55692823a0c0 2024-04-17T06:11:22.361 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.357+0000 7f95c1879740 10 client.0 _ll_register_callbacks cb 0x55692813dd40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:11:22.361 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.357+0000 7f95bd225640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5569282437b0 0x5569282411c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:22.361 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.357+0000 7f95bca24640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55692823a0c0 0x556928240740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:22.361 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.357+0000 7f95bd225640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5569282437b0 0x5569282411c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55468/0 (socket says 172.21.15.47:55468) 2024-04-17T06:11:22.361 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.357+0000 7f95affff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55692823a9e0 0x556928240c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:22.361 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.357+0000 7f95bd225640 1 -- 172.21.15.47:0/3143667846 learned_addr learned my addr 172.21.15.47:0/3143667846 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:11:22.361 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.357+0000 7f95bd225640 1 -- 172.21.15.47:0/3143667846 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55692823a0c0 msgr2=0x556928240740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:11:22.361 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.357+0000 7f95bd225640 1 --2- 172.21.15.47:0/3143667846 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55692823a0c0 0x556928240740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:22.361 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.357+0000 7f95bd225640 1 -- 172.21.15.47:0/3143667846 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55692823a9e0 msgr2=0x556928240c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:11:22.361 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.357+0000 7f95bd225640 1 --2- 172.21.15.47:0/3143667846 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55692823a9e0 0x556928240c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:22.361 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.357+0000 7f95bd225640 1 -- 172.21.15.47:0/3143667846 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556927f7e620 con 0x5569282437b0 2024-04-17T06:11:22.362 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.358+0000 7f95bd225640 1 --2- 172.21.15.47:0/3143667846 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5569282437b0 0x5569282411c0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f95b0011e70 tx=0x7f95b0019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c9659abb35a554a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:11:22.362 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.358+0000 7f9597fff640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-17T06:11:22.362 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.358+0000 7f9597fff640 1 -- 172.21.15.47:0/3143667846 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95b0004d20 con 0x5569282437b0 2024-04-17T06:11:22.362 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.358+0000 7f9597fff640 1 -- 172.21.15.47:0/3143667846 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8513+0+0 (secure 0 0 0) 0x7f95b0005020 con 0x5569282437b0 2024-04-17T06:11:22.362 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.358+0000 7f95c1879740 1 -- 172.21.15.47:0/3143667846 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95a8005500 con 0x5569282437b0 2024-04-17T06:11:22.362 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.358+0000 7f9597fff640 1 -- 172.21.15.47:0/3143667846 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f95b0003970 con 0x5569282437b0 2024-04-17T06:11:22.362 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.358+0000 7f95c1879740 10 client.16239 fetch_fsmap learned FSMap version 34 2024-04-17T06:11:22.362 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.358+0000 7f95c1879740 1 -- 172.21.15.47:0/3143667846 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5569282525a0 con 0x5569282437b0 2024-04-17T06:11:22.362 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.358+0000 7f9597fff640 1 -- 172.21.15.47:0/3143667846 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95a8005500 con 0x5569282437b0 2024-04-17T06:11:22.362 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.358+0000 7f9597fff640 1 -- 172.21.15.47:0/3143667846 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 34) ==== 8339+0+0 (secure 0 0 0) 0x7f95b0005840 con 0x5569282437b0 2024-04-17T06:11:22.362 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.358+0000 7f95c1879740 10 client.16239 fetch_fsmap finished waiting for FSMap version 34 2024-04-17T06:11:22.362 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.358+0000 7f95c1879740 10 client.16239 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-17T06:11:22.362 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.358+0000 7f95c1879740 20 client.16239 populate_metadata read hostname 'smithi047' 2024-04-17T06:11:22.362 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.358+0000 7f95c1879740 1 --2- 172.21.15.47:0/3143667846 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x556928253a60 0x556928255e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:22.363 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.359+0000 7f95c1879740 4 client.16239 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T06:11:22.363 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.359+0000 7f95c1879740 1 -- 172.21.15.47:0/3143667846 --> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5569281ad1f0 con 0x556928253a60 2024-04-17T06:11:22.363 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.359+0000 7f95bca24640 1 --2- 172.21.15.47:0/3143667846 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x556928253a60 0x556928255e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:22.363 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.359+0000 7f95bca24640 1 --2- 172.21.15.47:0/3143667846 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x556928253a60 0x556928255e50 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6864d9c4c89833a0 server_cookie=9f4ed0ef61b562af in_seq=0 out_seq=0 2024-04-17T06:11:22.363 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.359+0000 7f9597fff640 10 client.16239 ms_handle_connect on v2:172.21.15.98:6838/2841287917 2024-04-17T06:11:22.363 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.360+0000 7f9597fff640 1 -- 172.21.15.47:0/3143667846 <== mds.0 v2:172.21.15.98:6838/2841287917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5569281ad1f0 con 0x556928253a60 2024-04-17T06:11:22.363 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.360+0000 7f9597fff640 10 client.16239 handle_command_reply: tid=0 2024-04-17T06:11:22.363 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.360+0000 7f95c1879740 1 client.16239 shutdown 2024-04-17T06:11:22.363 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.360+0000 7f95c1879740 1 -- 172.21.15.47:0/3143667846 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5569282437b0 msgr2=0x5569282411c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:11:22.363 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.360+0000 7f95c1879740 1 --2- 172.21.15.47:0/3143667846 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5569282437b0 0x5569282411c0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f95b0011e70 tx=0x7f95b0019000 comp rx=0 tx=0).stop 2024-04-17T06:11:22.363 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.360+0000 7f95c1879740 1 -- 172.21.15.47:0/3143667846 shutdown_connections 2024-04-17T06:11:22.363 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.360+0000 7f95c1879740 1 --2- 172.21.15.47:0/3143667846 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x556928253a60 0x556928255e50 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:22.363 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.360+0000 7f95c1879740 1 --2- 172.21.15.47:0/3143667846 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5569282437b0 0x5569282411c0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:22.363 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.360+0000 7f95c1879740 1 --2- 172.21.15.47:0/3143667846 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55692823a9e0 0x556928240c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:22.364 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.360+0000 7f95c1879740 1 --2- 172.21.15.47:0/3143667846 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55692823a0c0 0x556928240740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:22.364 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.360+0000 7f9597fff640 0 client.16239 ms_handle_reset on v2:172.21.15.98:6838/2841287917 2024-04-17T06:11:22.364 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.360+0000 7f95c1879740 1 -- 172.21.15.47:0/3143667846 >> 172.21.15.47:0/3143667846 conn(0x5569282349e0 msgr2=0x556928236730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:11:22.364 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.361+0000 7f95c1879740 1 -- 172.21.15.47:0/3143667846 shutdown_connections 2024-04-17T06:11:22.364 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.361+0000 7f95c1879740 1 -- 172.21.15.47:0/3143667846 wait complete. 2024-04-17T06:11:22.364 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.361+0000 7f95c1879740 20 client.16239 trim_cache size 0 max 16384 2024-04-17T06:11:22.375 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.376+0000 7f95c1879740 1 Processor -- start 2024-04-17T06:11:22.375 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.376+0000 7f95c1879740 1 -- start start 2024-04-17T06:11:22.375 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.376+0000 7f95c1879740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55692823a0c0 0x556928242e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:22.375 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.376+0000 7f95c1879740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55692823a9e0 0x55692824f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:22.376 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.376+0000 7f95c1879740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5569282437b0 0x556928240f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:22.376 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.376+0000 7f95c1879740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f95b81b04c0 con 0x5569282437b0 2024-04-17T06:11:22.376 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.376+0000 7f95c1879740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f95b81add80 con 0x55692823a9e0 2024-04-17T06:11:22.376 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.376+0000 7f95c1879740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f95b81abbf0 con 0x55692823a0c0 2024-04-17T06:11:22.376 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.376+0000 7f95bd225640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5569282437b0 0x556928240f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:22.376 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.376+0000 7f95bd225640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5569282437b0 0x556928240f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55478/0 (socket says 172.21.15.47:55478) 2024-04-17T06:11:22.376 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.376+0000 7f95bd225640 1 -- 172.21.15.47:0/1220149567 learned_addr learned my addr 172.21.15.47:0/1220149567 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:11:22.376 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.376+0000 7f95bca24640 1 --2- 172.21.15.47:0/1220149567 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55692823a0c0 0x556928242e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:22.376 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.376+0000 7f95bd225640 1 -- 172.21.15.47:0/1220149567 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55692823a0c0 msgr2=0x556928242e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:11:22.376 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.376+0000 7f95bd225640 1 --2- 172.21.15.47:0/1220149567 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55692823a0c0 0x556928242e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:22.376 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.376+0000 7f95bd225640 1 -- 172.21.15.47:0/1220149567 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55692823a9e0 msgr2=0x55692824f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:11:22.376 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.376+0000 7f95bd225640 1 --2- 172.21.15.47:0/1220149567 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55692823a9e0 0x55692824f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:22.376 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.376+0000 7f95bd225640 1 -- 172.21.15.47:0/1220149567 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55692823f6f0 con 0x5569282437b0 2024-04-17T06:11:22.376 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.376+0000 7f95bd225640 1 --2- 172.21.15.47:0/1220149567 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5569282437b0 0x556928240f90 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f95b0012000 tx=0x7f95b000ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=542e14d869fe2d07 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:11:22.377 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.377+0000 7f959e7fc640 1 -- 172.21.15.47:0/1220149567 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95b0038070 con 0x5569282437b0 2024-04-17T06:11:22.377 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.377+0000 7f959e7fc640 1 -- 172.21.15.47:0/1220149567 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95b0003d10 con 0x5569282437b0 2024-04-17T06:11:22.377 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.377+0000 7f959e7fc640 1 -- 172.21.15.47:0/1220149567 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95b00052a0 con 0x5569282437b0 2024-04-17T06:11:22.377 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.377+0000 7f95c1879740 1 -- 172.21.15.47:0/1220149567 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5569282437b0 msgr2=0x556928240f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:11:22.377 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.377+0000 7f95c1879740 1 --2- 172.21.15.47:0/1220149567 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5569282437b0 0x556928240f90 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f95b0012000 tx=0x7f95b000ef10 comp rx=0 tx=0).stop 2024-04-17T06:11:22.377 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.377+0000 7f95c1879740 1 -- 172.21.15.47:0/1220149567 shutdown_connections 2024-04-17T06:11:22.377 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.377+0000 7f95c1879740 1 --2- 172.21.15.47:0/1220149567 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5569282437b0 0x556928240f90 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:22.377 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.377+0000 7f95c1879740 1 --2- 172.21.15.47:0/1220149567 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55692823a9e0 0x55692824f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:22.377 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.377+0000 7f95c1879740 1 --2- 172.21.15.47:0/1220149567 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55692823a0c0 0x556928242e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:22.377 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.377+0000 7f95c1879740 1 -- 172.21.15.47:0/1220149567 >> 172.21.15.47:0/1220149567 conn(0x5569282349e0 msgr2=0x556928234dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:11:22.377 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.377+0000 7f95c1879740 1 -- 172.21.15.47:0/1220149567 shutdown_connections 2024-04-17T06:11:22.377 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.377+0000 7f95c1879740 1 -- 172.21.15.47:0/1220149567 wait complete. 2024-04-17T06:11:22.377 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.378+0000 7f95c1879740 1 -- start start 2024-04-17T06:11:22.377 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.378+0000 7f95c1879740 1 Processor -- start 2024-04-17T06:11:22.380 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.378+0000 7f95c1879740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55692823a0c0 0x556928247170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:22.380 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.378+0000 7f95c1879740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55692823a9e0 0x5569282476b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:22.380 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.378+0000 7f95c1879740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5569282437b0 0x556928247bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:22.380 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.378+0000 7f95c1879740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f95b81ac240 con 0x5569282437b0 2024-04-17T06:11:22.380 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.378+0000 7f95c1879740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f95b81ad6b0 con 0x55692823a9e0 2024-04-17T06:11:22.381 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.378+0000 7f95c1879740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f95b81ac910 con 0x55692823a0c0 2024-04-17T06:11:22.381 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.378+0000 7f95c1879740 10 client.0 _ll_register_callbacks cb 0x7f95b0011b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:11:22.381 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.378+0000 7f95bd225640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5569282437b0 0x556928247bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:22.381 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.378+0000 7f95bd225640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5569282437b0 0x556928247bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55482/0 (socket says 172.21.15.47:55482) 2024-04-17T06:11:22.381 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.378+0000 7f95bd225640 1 -- 172.21.15.47:0/4077464536 learned_addr learned my addr 172.21.15.47:0/4077464536 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:11:22.381 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.378+0000 7f95bca24640 1 --2- 172.21.15.47:0/4077464536 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55692823a0c0 0x556928247170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:22.381 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.378+0000 7f95affff640 1 --2- 172.21.15.47:0/4077464536 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55692823a9e0 0x5569282476b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:22.381 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.379+0000 7f95bd225640 1 -- 172.21.15.47:0/4077464536 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55692823a0c0 msgr2=0x556928247170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:11:22.381 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.379+0000 7f95bd225640 1 --2- 172.21.15.47:0/4077464536 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55692823a0c0 0x556928247170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:22.381 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.379+0000 7f95bd225640 1 -- 172.21.15.47:0/4077464536 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55692823a9e0 msgr2=0x5569282476b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:11:22.381 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.379+0000 7f95bd225640 1 --2- 172.21.15.47:0/4077464536 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55692823a9e0 0x5569282476b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:22.381 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.379+0000 7f95bd225640 1 -- 172.21.15.47:0/4077464536 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569282b8fd0 con 0x5569282437b0 2024-04-17T06:11:22.381 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.379+0000 7f95affff640 1 --2- 172.21.15.47:0/4077464536 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55692823a9e0 0x5569282476b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:11:22.381 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.379+0000 7f95bd225640 1 --2- 172.21.15.47:0/4077464536 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5569282437b0 0x556928247bf0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f95b0004f50 tx=0x7f95b0004f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b74ff249661a6ef8 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:11:22.381 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.379+0000 7f9597fff640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-17T06:11:22.381 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.379+0000 7f9597fff640 1 -- 172.21.15.47:0/4077464536 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95b0041030 con 0x5569282437b0 2024-04-17T06:11:22.382 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.379+0000 7f9597fff640 1 -- 172.21.15.47:0/4077464536 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8513+0+0 (secure 0 0 0) 0x7f95b0003b10 con 0x5569282437b0 2024-04-17T06:11:22.382 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.379+0000 7f95c1879740 1 -- 172.21.15.47:0/4077464536 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95b0003d10 con 0x5569282437b0 2024-04-17T06:11:22.382 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.379+0000 7f9597fff640 1 -- 172.21.15.47:0/4077464536 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f95b0038050 con 0x5569282437b0 2024-04-17T06:11:22.382 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.379+0000 7f95c1879740 10 client.16245 fetch_fsmap learned FSMap version 34 2024-04-17T06:11:22.382 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.379+0000 7f95c1879740 1 -- 172.21.15.47:0/4077464536 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5569282ba280 con 0x5569282437b0 2024-04-17T06:11:22.382 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.379+0000 7f9597fff640 1 -- 172.21.15.47:0/4077464536 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95b0003d10 con 0x5569282437b0 2024-04-17T06:11:22.382 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.379+0000 7f9597fff640 1 -- 172.21.15.47:0/4077464536 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 34) ==== 8339+0+0 (secure 0 0 0) 0x7f95b00484c0 con 0x5569282437b0 2024-04-17T06:11:22.382 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.380+0000 7f95c1879740 10 client.16245 fetch_fsmap finished waiting for FSMap version 34 2024-04-17T06:11:22.382 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.380+0000 7f95c1879740 10 client.16245 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-17T06:11:22.382 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.380+0000 7f95c1879740 20 client.16245 populate_metadata read hostname 'smithi047' 2024-04-17T06:11:22.382 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.380+0000 7f95c1879740 1 --2- 172.21.15.47:0/4077464536 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x5569282ba8f0 0x5569282bcd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:22.382 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.380+0000 7f95c1879740 4 client.16245 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-17T06:11:22.382 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.380+0000 7f95c1879740 1 -- 172.21.15.47:0/4077464536 --> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5569281a7060 con 0x5569282ba8f0 2024-04-17T06:11:22.383 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.380+0000 7f95bca24640 1 --2- 172.21.15.47:0/4077464536 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x5569282ba8f0 0x5569282bcd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:22.383 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.380+0000 7f95bca24640 1 --2- 172.21.15.47:0/4077464536 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x5569282ba8f0 0x5569282bcd10 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dbf0be25c4e7db14 server_cookie=3447187f7e170c47 in_seq=0 out_seq=0 2024-04-17T06:11:22.383 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.380+0000 7f9597fff640 10 client.16245 ms_handle_connect on v2:172.21.15.98:6838/2841287917 2024-04-17T06:11:22.386 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.388+0000 7f9597fff640 1 -- 172.21.15.47:0/4077464536 <== mds.0 v2:172.21.15.98:6838/2841287917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5569281a7060 con 0x5569282ba8f0 2024-04-17T06:11:22.386 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.388+0000 7f9597fff640 10 client.16245 handle_command_reply: tid=0 2024-04-17T06:11:22.387 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.388+0000 7f95c1879740 1 client.16245 shutdown 2024-04-17T06:11:22.387 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.388+0000 7f95c1879740 1 -- 172.21.15.47:0/4077464536 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5569282437b0 msgr2=0x556928247bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:11:22.387 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.388+0000 7f95c1879740 1 --2- 172.21.15.47:0/4077464536 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5569282437b0 0x556928247bf0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f95b0004f50 tx=0x7f95b0004f80 comp rx=0 tx=0).stop 2024-04-17T06:11:22.387 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.388+0000 7f95c1879740 1 -- 172.21.15.47:0/4077464536 shutdown_connections 2024-04-17T06:11:22.387 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.388+0000 7f95c1879740 1 --2- 172.21.15.47:0/4077464536 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x5569282ba8f0 0x5569282bcd10 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:22.387 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.388+0000 7f95c1879740 1 --2- 172.21.15.47:0/4077464536 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5569282437b0 0x556928247bf0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:22.387 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.388+0000 7f95c1879740 1 --2- 172.21.15.47:0/4077464536 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55692823a9e0 0x5569282476b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:22.388 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.388+0000 7f95c1879740 1 --2- 172.21.15.47:0/4077464536 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55692823a0c0 0x556928247170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:22.388 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.388+0000 7f9597fff640 0 client.16245 ms_handle_reset on v2:172.21.15.98:6838/2841287917 2024-04-17T06:11:22.388 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.388+0000 7f95c1879740 1 -- 172.21.15.47:0/4077464536 >> 172.21.15.47:0/4077464536 conn(0x5569282349e0 msgr2=0x556928233b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:11:22.388 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.388+0000 7f95c1879740 1 -- 172.21.15.47:0/4077464536 shutdown_connections 2024-04-17T06:11:22.388 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.388+0000 7f95c1879740 1 -- 172.21.15.47:0/4077464536 wait complete. 2024-04-17T06:11:22.388 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.389+0000 7f95c1879740 20 client.16245 trim_cache size 0 max 16384 2024-04-17T06:11:22.388 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-04-17T06:11:22.388 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-04-17T06:11:22.388 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-04-17T06:11:22.388 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-04-17T06:11:22.389 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.389+0000 7f95bec88640 1 -- 172.21.15.47:0/3556406008 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f95a00a3e40 msgr2=0x7f95a00a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:11:22.389 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.389+0000 7f95bec88640 1 --2- 172.21.15.47:0/3556406008 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f95a00a3e40 0x7f95a00a6300 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f95b000a780 tx=0x7f95b0013a50 comp rx=0 tx=0).stop 2024-04-17T06:11:22.389 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.389+0000 7f95bec88640 1 -- 172.21.15.47:0/3556406008 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95b8101100 msgr2=0x7f95b819e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:11:22.389 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.389+0000 7f95bec88640 1 --2- 172.21.15.47:0/3556406008 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95b8101100 0x7f95b819e430 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f95b4006010 tx=0x7f95b402cb30 comp rx=0 tx=0).stop 2024-04-17T06:11:22.389 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.389+0000 7f95bec88640 1 -- 172.21.15.47:0/3556406008 shutdown_connections 2024-04-17T06:11:22.389 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.389+0000 7f95bec88640 1 --2- 172.21.15.47:0/3556406008 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f95b81985e0 0x7f95b8198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:22.389 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.389+0000 7f95bec88640 1 --2- 172.21.15.47:0/3556406008 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f95b8106180 0x7f95b819e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:22.389 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.389+0000 7f95bec88640 1 --2- 172.21.15.47:0/3556406008 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f95a00a3e40 0x7f95a00a6300 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:22.389 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.389+0000 7f95bec88640 1 --2- 172.21.15.47:0/3556406008 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95b8101100 0x7f95b819e430 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:22.390 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.389+0000 7f95bec88640 1 -- 172.21.15.47:0/3556406008 >> 172.21.15.47:0/3556406008 conn(0x7f95b8077fe0 msgr2=0x7f95b8069930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:11:22.390 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.389+0000 7f95bec88640 1 -- 172.21.15.47:0/3556406008 shutdown_connections 2024-04-17T06:11:22.390 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:22.389+0000 7f95bec88640 1 -- 172.21.15.47:0/3556406008 wait complete. 2024-04-17T06:11:24.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:11:23 smithi103 ceph-mon[36243]: pgmap v317: 97 pgs: 97 active+clean; 5.0 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.7 KiB/s wr, 11 op/s 2024-04-17T06:11:24.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:11:23 smithi098 ceph-mon[36516]: pgmap v317: 97 pgs: 97 active+clean; 5.0 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.7 KiB/s wr, 11 op/s 2024-04-17T06:11:24.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:11:23 smithi047 ceph-mon[32223]: pgmap v317: 97 pgs: 97 active+clean; 5.0 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.7 KiB/s wr, 11 op/s 2024-04-17T06:11:26.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:11:25 smithi103 ceph-mon[36243]: pgmap v318: 97 pgs: 97 active+clean; 5.1 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-17T06:11:26.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:11:25 smithi098 ceph-mon[36516]: pgmap v318: 97 pgs: 97 active+clean; 5.1 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-17T06:11:26.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:11:25 smithi047 ceph-mon[32223]: pgmap v318: 97 pgs: 97 active+clean; 5.1 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-17T06:11:28.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:11:27 smithi103 ceph-mon[36243]: pgmap v319: 97 pgs: 97 active+clean; 5.1 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:11:28.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:11:27 smithi098 ceph-mon[36516]: pgmap v319: 97 pgs: 97 active+clean; 5.1 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:11:28.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:11:27 smithi047 ceph-mon[32223]: pgmap v319: 97 pgs: 97 active+clean; 5.1 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:11:30.034 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:11:29 smithi103 ceph-mon[36243]: pgmap v320: 97 pgs: 97 active+clean; 5.1 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:11:30.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:11:29 smithi098 ceph-mon[36516]: pgmap v320: 97 pgs: 97 active+clean; 5.1 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:11:30.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:11:29 smithi047 ceph-mon[32223]: pgmap v320: 97 pgs: 97 active+clean; 5.1 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:11:32.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:11:31 smithi103 ceph-mon[36243]: pgmap v321: 97 pgs: 97 active+clean; 5.1 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-04-17T06:11:32.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:11:31 smithi098 ceph-mon[36516]: pgmap v321: 97 pgs: 97 active+clean; 5.1 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-04-17T06:11:32.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:11:31 smithi047 ceph-mon[32223]: pgmap v321: 97 pgs: 97 active+clean; 5.1 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-04-17T06:11:32.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:11:32 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T06:11:32.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:11:32 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T06:11:33.033 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:11:32 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T06:11:34.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:11:33 smithi098 ceph-mon[36516]: pgmap v322: 97 pgs: 97 active+clean; 5.1 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:11:34.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:11:33 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:11:34.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:11:33 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:11:34.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:11:33 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:11:34.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:11:33 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:11:34.134 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:11:33 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:11:34.134 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:11:33 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:11:34.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:11:33 smithi047 ceph-mon[32223]: pgmap v322: 97 pgs: 97 active+clean; 5.1 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:11:34.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:11:33 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:11:34.140 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:11:33 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:11:34.140 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:11:33 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:11:34.140 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:11:33 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:11:34.140 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:11:33 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:11:34.141 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:11:33 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:11:34.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:11:33 smithi103 ceph-mon[36243]: pgmap v322: 97 pgs: 97 active+clean; 5.1 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:11:34.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:11:33 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:11:34.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:11:33 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:11:34.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:11:33 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:11:34.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:11:33 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:11:34.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:11:33 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:11:34.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:11:33 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:11:35.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:11:34 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-17T06:11:35.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:11:34 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-17T06:11:35.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:11:34 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-17T06:11:35.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:11:34 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-17T06:11:35.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:11:34 smithi098 ceph-mon[36516]: Adjusting osd_memory_target on smithi103 to 174.1M 2024-04-17T06:11:35.134 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:11:34 smithi098 ceph-mon[36516]: Unable to set osd_memory_target on smithi103 to 182637056: error parsing value: Value '182637056' is below minimum 939524096 2024-04-17T06:11:35.134 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:11:34 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-17T06:11:35.134 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:11:34 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-17T06:11:35.134 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:11:34 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-17T06:11:35.134 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:11:34 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-17T06:11:35.134 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:11:34 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-17T06:11:35.134 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:11:34 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-17T06:11:35.134 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:11:34 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-17T06:11:35.134 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:11:34 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-17T06:11:35.134 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:11:34 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T06:11:35.134 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:11:34 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T06:11:35.134 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:11:34 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:11:35.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:11:34 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-17T06:11:35.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:11:34 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-17T06:11:35.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:11:34 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-17T06:11:35.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:11:34 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-17T06:11:35.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:11:34 smithi047 ceph-mon[32223]: Adjusting osd_memory_target on smithi103 to 174.1M 2024-04-17T06:11:35.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:11:34 smithi047 ceph-mon[32223]: Unable to set osd_memory_target on smithi103 to 182637056: error parsing value: Value '182637056' is below minimum 939524096 2024-04-17T06:11:35.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:11:34 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-17T06:11:35.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:11:34 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-17T06:11:35.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:11:34 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-17T06:11:35.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:11:34 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-17T06:11:35.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:11:34 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-17T06:11:35.140 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:11:34 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-17T06:11:35.140 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:11:34 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-17T06:11:35.140 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:11:34 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-17T06:11:35.140 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:11:34 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T06:11:35.140 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:11:34 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T06:11:35.140 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:11:34 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:11:35.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:11:34 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-17T06:11:35.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:11:34 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-17T06:11:35.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:11:34 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-17T06:11:35.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:11:34 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-17T06:11:35.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:11:34 smithi103 ceph-mon[36243]: Adjusting osd_memory_target on smithi103 to 174.1M 2024-04-17T06:11:35.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:11:34 smithi103 ceph-mon[36243]: Unable to set osd_memory_target on smithi103 to 182637056: error parsing value: Value '182637056' is below minimum 939524096 2024-04-17T06:11:35.285 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:11:34 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-17T06:11:35.285 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:11:34 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-17T06:11:35.285 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:11:34 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-17T06:11:35.285 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:11:34 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-17T06:11:35.285 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:11:34 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-17T06:11:35.285 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:11:34 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-17T06:11:35.285 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:11:34 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-17T06:11:35.285 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:11:34 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-17T06:11:35.285 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:11:34 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T06:11:35.285 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:11:34 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T06:11:35.285 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:11:34 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:11:35.488 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.488+0000 7f77f3463640 1 -- 172.21.15.47:0/1714082329 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f77ec101ab0 msgr2=0x7f77ec105aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:11:35.488 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.488+0000 7f77f3463640 1 --2- 172.21.15.47:0/1714082329 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f77ec101ab0 0x7f77ec105aa0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f77e0009900 tx=0x7f77e002bc30 comp rx=0 tx=0).stop 2024-04-17T06:11:35.488 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.489+0000 7f77f3463640 1 -- 172.21.15.47:0/1714082329 shutdown_connections 2024-04-17T06:11:35.488 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.489+0000 7f77f3463640 1 --2- 172.21.15.47:0/1714082329 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f77ec106180 0x7f77ec10dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:35.488 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.489+0000 7f77f3463640 1 --2- 172.21.15.47:0/1714082329 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f77ec101ab0 0x7f77ec105aa0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:35.489 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.489+0000 7f77f3463640 1 --2- 172.21.15.47:0/1714082329 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f77ec101100 0x7f77ec1014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:35.489 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.489+0000 7f77f3463640 1 -- 172.21.15.47:0/1714082329 >> 172.21.15.47:0/1714082329 conn(0x7f77ec077fe0 msgr2=0x7f77ec0783f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:11:35.489 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.489+0000 7f77f3463640 1 -- 172.21.15.47:0/1714082329 shutdown_connections 2024-04-17T06:11:35.489 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.489+0000 7f77f3463640 1 -- 172.21.15.47:0/1714082329 wait complete. 2024-04-17T06:11:35.489 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.489+0000 7f77f3463640 1 Processor -- start 2024-04-17T06:11:35.489 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.489+0000 7f77f3463640 1 -- start start 2024-04-17T06:11:35.490 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.490+0000 7f77f3463640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f77ec101100 0x7f77ec19e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:35.491 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.490+0000 7f77f3463640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f77ec106180 0x7f77ec19eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:35.491 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.490+0000 7f77f3463640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f77ec198630 0x7f77ec198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:35.491 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.490+0000 7f77f3463640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f77ec1111f0 con 0x7f77ec101100 2024-04-17T06:11:35.491 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.490+0000 7f77f3463640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f77ec111070 con 0x7f77ec198630 2024-04-17T06:11:35.491 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.490+0000 7f77f3463640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f77ec111370 con 0x7f77ec106180 2024-04-17T06:11:35.491 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.490+0000 7f77f11ff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f77ec101100 0x7f77ec19e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:35.491 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.490+0000 7f77f11ff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f77ec101100 0x7f77ec19e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38374/0 (socket says 172.21.15.47:38374) 2024-04-17T06:11:35.491 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.490+0000 7f77f11ff640 1 -- 172.21.15.47:0/2589955431 learned_addr learned my addr 172.21.15.47:0/2589955431 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:11:35.491 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.490+0000 7f77f1a00640 1 --2- 172.21.15.47:0/2589955431 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f77ec198630 0x7f77ec198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:35.491 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.490+0000 7f77f09fe640 1 --2- 172.21.15.47:0/2589955431 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f77ec106180 0x7f77ec19eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:35.491 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.490+0000 7f77f11ff640 1 -- 172.21.15.47:0/2589955431 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f77ec106180 msgr2=0x7f77ec19eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:11:35.491 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.490+0000 7f77f11ff640 1 --2- 172.21.15.47:0/2589955431 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f77ec106180 0x7f77ec19eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:35.492 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.490+0000 7f77f11ff640 1 -- 172.21.15.47:0/2589955431 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f77ec198630 msgr2=0x7f77ec198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:11:35.492 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.490+0000 7f77f11ff640 1 --2- 172.21.15.47:0/2589955431 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f77ec198630 0x7f77ec198ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:35.492 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.490+0000 7f77f11ff640 1 -- 172.21.15.47:0/2589955431 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77ec199230 con 0x7f77ec101100 2024-04-17T06:11:35.492 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.490+0000 7f77f11ff640 1 --2- 172.21.15.47:0/2589955431 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f77ec101100 0x7f77ec19e570 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f77dc00eab0 tx=0x7f77dc00ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:11:35.492 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.491+0000 7f77ea7fc640 1 -- 172.21.15.47:0/2589955431 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77dc00ce40 con 0x7f77ec101100 2024-04-17T06:11:35.492 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.491+0000 7f77f3463640 1 -- 172.21.15.47:0/2589955431 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77ec1a5310 con 0x7f77ec101100 2024-04-17T06:11:35.492 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.491+0000 7f77ea7fc640 1 -- 172.21.15.47:0/2589955431 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77dc011950 con 0x7f77ec101100 2024-04-17T06:11:35.492 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.491+0000 7f77f3463640 1 -- 172.21.15.47:0/2589955431 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77ec1a56c0 con 0x7f77ec101100 2024-04-17T06:11:35.492 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.491+0000 7f77ea7fc640 1 -- 172.21.15.47:0/2589955431 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77dc010a80 con 0x7f77ec101100 2024-04-17T06:11:35.498 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.493+0000 7f77ea7fc640 1 -- 172.21.15.47:0/2589955431 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f77dc011b00 con 0x7f77ec101100 2024-04-17T06:11:35.498 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.494+0000 7f77f6050740 1 Processor -- start 2024-04-17T06:11:35.498 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.494+0000 7f77f6050740 1 -- start start 2024-04-17T06:11:35.498 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.494+0000 7f77f6050740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55656401a0c0 0x55656401a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:35.498 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.494+0000 7f77f6050740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55656401a9e0 0x556564023270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:35.498 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.494+0000 7f77f6050740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5565640237b0 0x556564025ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:35.498 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.494+0000 7f77f6050740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x556563f7f7d0 con 0x5565640237b0 2024-04-17T06:11:35.499 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.494+0000 7f77f6050740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x556563f1cf70 con 0x55656401a9e0 2024-04-17T06:11:35.499 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.494+0000 7f77f6050740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556563f7b790 con 0x55656401a0c0 2024-04-17T06:11:35.499 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.494+0000 7f77ea7fc640 1 --2- 172.21.15.47:0/2589955431 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f77d40a3dc0 0x7f77d40a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:35.499 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.494+0000 7f77f09fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5565640237b0 0x556564025ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:35.499 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.494+0000 7f77f09fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5565640237b0 0x556564025ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38388/0 (socket says 172.21.15.47:38388) 2024-04-17T06:11:35.499 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.494+0000 7f77f09fe640 1 -- 172.21.15.47:0/2899815356 learned_addr learned my addr 172.21.15.47:0/2899815356 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:11:35.499 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.494+0000 7f77f11ff640 1 --2- 172.21.15.47:0/2899815356 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55656401a9e0 0x556564023270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:35.499 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.494+0000 7f77f1a00640 1 --2- 172.21.15.47:0/2589955431 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f77d40a3dc0 0x7f77d40a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:35.499 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.494+0000 7f77f1a00640 1 --2- 172.21.15.47:0/2899815356 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55656401a0c0 0x55656401a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:35.499 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.494+0000 7f77ea7fc640 1 -- 172.21.15.47:0/2589955431 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8513+0+0 (secure 0 0 0) 0x7f77dc0d0f00 con 0x7f77ec101100 2024-04-17T06:11:35.499 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.494+0000 7f77f09fe640 1 -- 172.21.15.47:0/2899815356 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55656401a0c0 msgr2=0x55656401a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:11:35.499 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.494+0000 7f77f09fe640 1 --2- 172.21.15.47:0/2899815356 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55656401a0c0 0x55656401a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:35.499 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.494+0000 7f77f09fe640 1 -- 172.21.15.47:0/2899815356 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55656401a9e0 msgr2=0x556564023270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:11:35.500 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.494+0000 7f77f09fe640 1 --2- 172.21.15.47:0/2899815356 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55656401a9e0 0x556564023270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:35.500 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.494+0000 7f77f09fe640 1 -- 172.21.15.47:0/2899815356 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556563c92210 con 0x5565640237b0 2024-04-17T06:11:35.500 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.494+0000 7f77f11ff640 1 --2- 172.21.15.47:0/2899815356 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55656401a9e0 0x556564023270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:11:35.500 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.495+0000 7f77f09fe640 1 --2- 172.21.15.47:0/2899815356 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5565640237b0 0x556564025ba0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f77ec06b740 tx=0x7f77e0005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=332bb5a628972bc5 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:11:35.500 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.495+0000 7f77f1a00640 1 --2- 172.21.15.47:0/2589955431 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f77d40a3dc0 0x7f77d40a6280 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f77e40114a0 tx=0x7f77e4009890 comp rx=0 tx=0).ready entity=mgr.24923 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:11:35.500 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.495+0000 7f77d37fe640 1 -- 172.21.15.47:0/2899815356 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77e002ca80 con 0x5565640237b0 2024-04-17T06:11:35.500 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.495+0000 7f77d37fe640 1 -- 172.21.15.47:0/2899815356 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77e002cc20 con 0x5565640237b0 2024-04-17T06:11:35.500 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.495+0000 7f77d37fe640 1 -- 172.21.15.47:0/2899815356 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77e00046c0 con 0x5565640237b0 2024-04-17T06:11:35.500 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.496+0000 7f77f6050740 1 -- 172.21.15.47:0/2899815356 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5565640237b0 msgr2=0x556564025ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:11:35.500 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.496+0000 7f77f6050740 1 --2- 172.21.15.47:0/2899815356 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5565640237b0 0x556564025ba0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f77ec06b740 tx=0x7f77e0005d50 comp rx=0 tx=0).stop 2024-04-17T06:11:35.500 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.496+0000 7f77f6050740 1 -- 172.21.15.47:0/2899815356 shutdown_connections 2024-04-17T06:11:35.500 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.496+0000 7f77f6050740 1 --2- 172.21.15.47:0/2899815356 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5565640237b0 0x556564025ba0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:35.500 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.496+0000 7f77f6050740 1 --2- 172.21.15.47:0/2899815356 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55656401a9e0 0x556564023270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:35.500 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.496+0000 7f77f6050740 1 --2- 172.21.15.47:0/2899815356 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55656401a0c0 0x55656401a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:35.501 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.496+0000 7f77f6050740 1 -- 172.21.15.47:0/2899815356 >> 172.21.15.47:0/2899815356 conn(0x5565640149e0 msgr2=0x556564014dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:11:35.501 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.496+0000 7f77f6050740 1 -- 172.21.15.47:0/2899815356 shutdown_connections 2024-04-17T06:11:35.501 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.496+0000 7f77f6050740 1 -- 172.21.15.47:0/2899815356 wait complete. 2024-04-17T06:11:35.501 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.496+0000 7f77f6050740 1 -- start start 2024-04-17T06:11:35.501 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.496+0000 7f77f6050740 1 Processor -- start 2024-04-17T06:11:35.501 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.497+0000 7f77f6050740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55656401a0c0 0x556564020770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:35.501 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.497+0000 7f77f6050740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55656401a9e0 0x556564020cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:35.501 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.497+0000 7f77f6050740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565640237b0 0x5565640211f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:35.501 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.497+0000 7f77f6050740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x556564026380 con 0x55656401a0c0 2024-04-17T06:11:35.501 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.497+0000 7f77f6050740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x556563f07e90 con 0x55656401a9e0 2024-04-17T06:11:35.501 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.497+0000 7f77f6050740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556564026500 con 0x5565640237b0 2024-04-17T06:11:35.501 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.497+0000 7f77f6050740 10 client.0 _ll_register_callbacks cb 0x556563f1dd40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:11:35.501 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.497+0000 7f77f11ff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55656401a0c0 0x556564020770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:35.502 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.497+0000 7f77f11ff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55656401a0c0 0x556564020770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38392/0 (socket says 172.21.15.47:38392) 2024-04-17T06:11:35.502 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.497+0000 7f77f11ff640 1 -- 172.21.15.47:0/2238548019 learned_addr learned my addr 172.21.15.47:0/2238548019 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:11:35.502 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.497+0000 7f77f09fe640 1 --2- 172.21.15.47:0/2238548019 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55656401a9e0 0x556564020cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:35.502 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.497+0000 7f77f1a00640 1 --2- 172.21.15.47:0/2238548019 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565640237b0 0x5565640211f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:35.502 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.497+0000 7f77f11ff640 1 -- 172.21.15.47:0/2238548019 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565640237b0 msgr2=0x5565640211f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:11:35.502 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.497+0000 7f77f11ff640 1 --2- 172.21.15.47:0/2238548019 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565640237b0 0x5565640211f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:35.502 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.497+0000 7f77f11ff640 1 -- 172.21.15.47:0/2238548019 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55656401a9e0 msgr2=0x556564020cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:11:35.502 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.497+0000 7f77f11ff640 1 --2- 172.21.15.47:0/2238548019 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55656401a9e0 0x556564020cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:35.502 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.497+0000 7f77f11ff640 1 -- 172.21.15.47:0/2238548019 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556563d5e620 con 0x55656401a0c0 2024-04-17T06:11:35.502 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.497+0000 7f77f1a00640 1 --2- 172.21.15.47:0/2238548019 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565640237b0 0x5565640211f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:11:35.502 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.498+0000 7f77f11ff640 1 --2- 172.21.15.47:0/2238548019 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55656401a0c0 0x556564020770 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f77dc00ea80 tx=0x7f77dc01ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f626d55b06867c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:11:35.502 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.498+0000 7f77d0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-17T06:11:35.502 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.498+0000 7f77d0ff9640 1 -- 172.21.15.47:0/2238548019 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77dc09f070 con 0x55656401a0c0 2024-04-17T06:11:35.503 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.498+0000 7f77d0ff9640 1 -- 172.21.15.47:0/2238548019 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8513+0+0 (secure 0 0 0) 0x7f77dc0151b0 con 0x55656401a0c0 2024-04-17T06:11:35.503 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.498+0000 7f77f6050740 1 -- 172.21.15.47:0/2238548019 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77e002cc20 con 0x55656401a0c0 2024-04-17T06:11:35.503 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.498+0000 7f77d0ff9640 1 -- 172.21.15.47:0/2238548019 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f77dc09d540 con 0x55656401a0c0 2024-04-17T06:11:35.503 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.498+0000 7f77f6050740 10 client.16257 fetch_fsmap learned FSMap version 34 2024-04-17T06:11:35.503 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.498+0000 7f77f6050740 1 -- 172.21.15.47:0/2238548019 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x556564032650 con 0x55656401a0c0 2024-04-17T06:11:35.503 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.498+0000 7f77d0ff9640 1 -- 172.21.15.47:0/2238548019 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77e002cc20 con 0x55656401a0c0 2024-04-17T06:11:35.503 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.498+0000 7f77d0ff9640 1 -- 172.21.15.47:0/2238548019 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 34) ==== 8339+0+0 (secure 0 0 0) 0x7f77dc00f040 con 0x55656401a0c0 2024-04-17T06:11:35.503 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.498+0000 7f77f6050740 10 client.16257 fetch_fsmap finished waiting for FSMap version 34 2024-04-17T06:11:35.503 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.498+0000 7f77f6050740 10 client.16257 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-17T06:11:35.503 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.498+0000 7f77f6050740 20 client.16257 populate_metadata read hostname 'smithi047' 2024-04-17T06:11:35.503 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.498+0000 7f77f6050740 1 --2- 172.21.15.47:0/2238548019 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x556564033b10 0x556564035f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:35.503 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.498+0000 7f77f6050740 4 client.16257 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T06:11:35.503 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.498+0000 7f77f6050740 1 -- 172.21.15.47:0/2238548019 --> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556563f8d1f0 con 0x556564033b10 2024-04-17T06:11:35.503 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.499+0000 7f77f09fe640 1 --2- 172.21.15.47:0/2238548019 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x556564033b10 0x556564035f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:35.504 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.499+0000 7f77d0ff9640 10 client.16257 ms_handle_connect on v2:172.21.15.98:6838/2841287917 2024-04-17T06:11:35.504 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.499+0000 7f77f09fe640 1 --2- 172.21.15.47:0/2238548019 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x556564033b10 0x556564035f20 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89e108eb829f6f8c server_cookie=b015f18520f1a141 in_seq=0 out_seq=0 2024-04-17T06:11:35.505 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.500+0000 7f77d0ff9640 1 -- 172.21.15.47:0/2238548019 <== mds.0 v2:172.21.15.98:6838/2841287917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556563f8d1f0 con 0x556564033b10 2024-04-17T06:11:35.505 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.500+0000 7f77d0ff9640 10 client.16257 handle_command_reply: tid=0 2024-04-17T06:11:35.505 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.500+0000 7f77f6050740 1 client.16257 shutdown 2024-04-17T06:11:35.505 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.500+0000 7f77f6050740 1 -- 172.21.15.47:0/2238548019 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55656401a0c0 msgr2=0x556564020770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:11:35.505 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.500+0000 7f77f6050740 1 --2- 172.21.15.47:0/2238548019 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55656401a0c0 0x556564020770 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f77dc00ea80 tx=0x7f77dc01ef90 comp rx=0 tx=0).stop 2024-04-17T06:11:35.505 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.500+0000 7f77f6050740 1 -- 172.21.15.47:0/2238548019 shutdown_connections 2024-04-17T06:11:35.505 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.500+0000 7f77f6050740 1 --2- 172.21.15.47:0/2238548019 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x556564033b10 0x556564035f20 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:35.505 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.500+0000 7f77f6050740 1 --2- 172.21.15.47:0/2238548019 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565640237b0 0x5565640211f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:35.505 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.500+0000 7f77f6050740 1 --2- 172.21.15.47:0/2238548019 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55656401a9e0 0x556564020cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:35.506 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.500+0000 7f77d0ff9640 0 client.16257 ms_handle_reset on v2:172.21.15.98:6838/2841287917 2024-04-17T06:11:35.506 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.500+0000 7f77f6050740 1 --2- 172.21.15.47:0/2238548019 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55656401a0c0 0x556564020770 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:35.506 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.500+0000 7f77f6050740 1 -- 172.21.15.47:0/2238548019 >> 172.21.15.47:0/2238548019 conn(0x5565640149e0 msgr2=0x556564016730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:11:35.506 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.501+0000 7f77f6050740 1 -- 172.21.15.47:0/2238548019 shutdown_connections 2024-04-17T06:11:35.506 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.501+0000 7f77f6050740 1 -- 172.21.15.47:0/2238548019 wait complete. 2024-04-17T06:11:35.506 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.501+0000 7f77f6050740 20 client.16257 trim_cache size 0 max 16384 2024-04-17T06:11:35.521 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.516+0000 7f77f6050740 1 Processor -- start 2024-04-17T06:11:35.522 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.516+0000 7f77f6050740 1 -- start start 2024-04-17T06:11:35.522 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.516+0000 7f77f6050740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55656401a0c0 0x556564022ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:35.522 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.516+0000 7f77f6050740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55656401a9e0 0x55656402f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:35.522 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.516+0000 7f77f6050740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5565640237b0 0x556564016540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:35.522 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.516+0000 7f77f6050740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f77ec1b04e0 con 0x5565640237b0 2024-04-17T06:11:35.522 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.516+0000 7f77f6050740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f77ec1adda0 con 0x55656401a0c0 2024-04-17T06:11:35.522 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.516+0000 7f77f6050740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f77ec1abc10 con 0x55656401a9e0 2024-04-17T06:11:35.522 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.516+0000 7f77f1a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5565640237b0 0x556564016540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:35.522 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.516+0000 7f77f1a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5565640237b0 0x556564016540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38396/0 (socket says 172.21.15.47:38396) 2024-04-17T06:11:35.522 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.516+0000 7f77f1a00640 1 -- 172.21.15.47:0/1217781832 learned_addr learned my addr 172.21.15.47:0/1217781832 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:11:35.522 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.516+0000 7f77f11ff640 1 --2- 172.21.15.47:0/1217781832 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55656401a0c0 0x556564022ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:35.523 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.516+0000 7f77f09fe640 1 --2- 172.21.15.47:0/1217781832 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55656401a9e0 0x55656402f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:35.523 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.516+0000 7f77f1a00640 1 -- 172.21.15.47:0/1217781832 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55656401a9e0 msgr2=0x55656402f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:11:35.523 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.516+0000 7f77f1a00640 1 --2- 172.21.15.47:0/1217781832 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55656401a9e0 0x55656402f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:35.523 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.516+0000 7f77f1a00640 1 -- 172.21.15.47:0/1217781832 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55656401a0c0 msgr2=0x556564022ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:11:35.523 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.516+0000 7f77f1a00640 1 --2- 172.21.15.47:0/1217781832 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55656401a0c0 0x556564022ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:35.523 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.516+0000 7f77f1a00640 1 -- 172.21.15.47:0/1217781832 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556564014810 con 0x5565640237b0 2024-04-17T06:11:35.523 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.516+0000 7f77f1a00640 1 --2- 172.21.15.47:0/1217781832 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5565640237b0 0x556564016540 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f77e4012760 tx=0x7f77e4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c27b9be17889b83 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:11:35.523 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.517+0000 7f77d2ffd640 1 -- 172.21.15.47:0/1217781832 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77e402e070 con 0x5565640237b0 2024-04-17T06:11:35.524 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.517+0000 7f77d2ffd640 1 -- 172.21.15.47:0/1217781832 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77e4021c70 con 0x5565640237b0 2024-04-17T06:11:35.524 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.517+0000 7f77d2ffd640 1 -- 172.21.15.47:0/1217781832 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77e4029d30 con 0x5565640237b0 2024-04-17T06:11:35.524 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.517+0000 7f77f6050740 1 -- 172.21.15.47:0/1217781832 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5565640237b0 msgr2=0x556564016540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:11:35.524 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.517+0000 7f77f6050740 1 --2- 172.21.15.47:0/1217781832 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5565640237b0 0x556564016540 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f77e4012760 tx=0x7f77e4004730 comp rx=0 tx=0).stop 2024-04-17T06:11:35.524 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.517+0000 7f77f6050740 1 -- 172.21.15.47:0/1217781832 shutdown_connections 2024-04-17T06:11:35.524 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.517+0000 7f77f6050740 1 --2- 172.21.15.47:0/1217781832 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5565640237b0 0x556564016540 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:35.524 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.517+0000 7f77f6050740 1 --2- 172.21.15.47:0/1217781832 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55656401a9e0 0x55656402f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:35.524 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.517+0000 7f77f6050740 1 --2- 172.21.15.47:0/1217781832 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55656401a0c0 0x556564022ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:35.524 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.517+0000 7f77f6050740 1 -- 172.21.15.47:0/1217781832 >> 172.21.15.47:0/1217781832 conn(0x5565640149e0 msgr2=0x556564015e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:11:35.525 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.517+0000 7f77f6050740 1 -- 172.21.15.47:0/1217781832 shutdown_connections 2024-04-17T06:11:35.525 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.517+0000 7f77f6050740 1 -- 172.21.15.47:0/1217781832 wait complete. 2024-04-17T06:11:35.525 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.518+0000 7f77f6050740 1 -- start start 2024-04-17T06:11:35.525 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.518+0000 7f77f6050740 1 Processor -- start 2024-04-17T06:11:35.525 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.518+0000 7f77f6050740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55656401a0c0 0x55656402b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:35.525 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.518+0000 7f77f6050740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55656401a9e0 0x55656402bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:35.525 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.518+0000 7f77f6050740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5565640237b0 0x556564028750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:35.525 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.518+0000 7f77f6050740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f77ec1ac260 con 0x5565640237b0 2024-04-17T06:11:35.525 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.518+0000 7f77f6050740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f77ec1ad6d0 con 0x55656401a9e0 2024-04-17T06:11:35.525 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.518+0000 7f77f6050740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f77ec1ac930 con 0x55656401a0c0 2024-04-17T06:11:35.525 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.518+0000 7f77f6050740 10 client.0 _ll_register_callbacks cb 0x7f77dc0103d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:11:35.525 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.518+0000 7f77f1a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5565640237b0 0x556564028750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:35.525 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.518+0000 7f77f1a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5565640237b0 0x556564028750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38408/0 (socket says 172.21.15.47:38408) 2024-04-17T06:11:35.525 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.518+0000 7f77f1a00640 1 -- 172.21.15.47:0/3717659111 learned_addr learned my addr 172.21.15.47:0/3717659111 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:11:35.526 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.518+0000 7f77f09fe640 1 --2- 172.21.15.47:0/3717659111 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55656401a9e0 0x55656402bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:35.526 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.518+0000 7f77f11ff640 1 --2- 172.21.15.47:0/3717659111 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55656401a0c0 0x55656402b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:35.526 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.519+0000 7f77f1a00640 1 -- 172.21.15.47:0/3717659111 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55656401a0c0 msgr2=0x55656402b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:11:35.526 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.519+0000 7f77f1a00640 1 --2- 172.21.15.47:0/3717659111 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55656401a0c0 0x55656402b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:35.526 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.519+0000 7f77f1a00640 1 -- 172.21.15.47:0/3717659111 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55656401a9e0 msgr2=0x55656402bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:11:35.526 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.519+0000 7f77f1a00640 1 --2- 172.21.15.47:0/3717659111 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55656401a9e0 0x55656402bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:35.526 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.519+0000 7f77f1a00640 1 -- 172.21.15.47:0/3717659111 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556564028f50 con 0x5565640237b0 2024-04-17T06:11:35.526 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.519+0000 7f77f1a00640 1 --2- 172.21.15.47:0/3717659111 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5565640237b0 0x556564028750 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f77e4016040 tx=0x7f77e4029740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db5ad128614ba0b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:11:35.526 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.519+0000 7f77d0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-17T06:11:35.526 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.519+0000 7f77d0ff9640 1 -- 172.21.15.47:0/3717659111 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77e4038070 con 0x5565640237b0 2024-04-17T06:11:35.526 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.519+0000 7f77d0ff9640 1 -- 172.21.15.47:0/3717659111 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8513+0+0 (secure 0 0 0) 0x7f77e4021590 con 0x5565640237b0 2024-04-17T06:11:35.527 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.519+0000 7f77f6050740 1 -- 172.21.15.47:0/3717659111 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77e4021c70 con 0x5565640237b0 2024-04-17T06:11:35.527 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.519+0000 7f77d0ff9640 1 -- 172.21.15.47:0/3717659111 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f77e402e040 con 0x5565640237b0 2024-04-17T06:11:35.527 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.519+0000 7f77f6050740 10 client.16263 fetch_fsmap learned FSMap version 34 2024-04-17T06:11:35.527 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.519+0000 7f77f6050740 1 -- 172.21.15.47:0/3717659111 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55656409ab00 con 0x5565640237b0 2024-04-17T06:11:35.527 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.519+0000 7f77d0ff9640 1 -- 172.21.15.47:0/3717659111 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77e4021c70 con 0x5565640237b0 2024-04-17T06:11:35.527 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.519+0000 7f77d0ff9640 1 -- 172.21.15.47:0/3717659111 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 34) ==== 8339+0+0 (secure 0 0 0) 0x7f77e403b020 con 0x5565640237b0 2024-04-17T06:11:35.527 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.520+0000 7f77f6050740 10 client.16263 fetch_fsmap finished waiting for FSMap version 34 2024-04-17T06:11:35.527 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.520+0000 7f77f6050740 10 client.16263 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-17T06:11:35.528 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.520+0000 7f77f6050740 20 client.16263 populate_metadata read hostname 'smithi047' 2024-04-17T06:11:35.528 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.520+0000 7f77f6050740 1 --2- 172.21.15.47:0/3717659111 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55656409b190 0x55656409d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:35.528 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.520+0000 7f77f6050740 4 client.16263 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-17T06:11:35.528 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.520+0000 7f77f6050740 1 -- 172.21.15.47:0/3717659111 --> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556563f87060 con 0x55656409b190 2024-04-17T06:11:35.528 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.520+0000 7f77f11ff640 1 --2- 172.21.15.47:0/3717659111 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55656409b190 0x55656409d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:35.528 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.520+0000 7f77f11ff640 1 --2- 172.21.15.47:0/3717659111 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55656409b190 0x55656409d5b0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bff40ed0e0240d80 server_cookie=4eddeb6d5ba06da6 in_seq=0 out_seq=0 2024-04-17T06:11:35.529 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.520+0000 7f77d0ff9640 10 client.16263 ms_handle_connect on v2:172.21.15.98:6838/2841287917 2024-04-17T06:11:35.531 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.529+0000 7f77d0ff9640 1 -- 172.21.15.47:0/3717659111 <== mds.0 v2:172.21.15.98:6838/2841287917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556563f87060 con 0x55656409b190 2024-04-17T06:11:35.531 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.529+0000 7f77d0ff9640 10 client.16263 handle_command_reply: tid=0 2024-04-17T06:11:35.531 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.529+0000 7f77f6050740 1 client.16263 shutdown 2024-04-17T06:11:35.531 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.529+0000 7f77f6050740 1 -- 172.21.15.47:0/3717659111 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5565640237b0 msgr2=0x556564028750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:11:35.531 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.529+0000 7f77f6050740 1 --2- 172.21.15.47:0/3717659111 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5565640237b0 0x556564028750 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f77e4016040 tx=0x7f77e4029740 comp rx=0 tx=0).stop 2024-04-17T06:11:35.531 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.529+0000 7f77f6050740 1 -- 172.21.15.47:0/3717659111 shutdown_connections 2024-04-17T06:11:35.531 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.529+0000 7f77f6050740 1 --2- 172.21.15.47:0/3717659111 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x55656409b190 0x55656409d5b0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:35.531 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.529+0000 7f77f6050740 1 --2- 172.21.15.47:0/3717659111 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5565640237b0 0x556564028750 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:35.531 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.529+0000 7f77f6050740 1 --2- 172.21.15.47:0/3717659111 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55656401a9e0 0x55656402bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:35.531 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.529+0000 7f77d0ff9640 0 client.16263 ms_handle_reset on v2:172.21.15.98:6838/2841287917 2024-04-17T06:11:35.531 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.529+0000 7f77f6050740 1 --2- 172.21.15.47:0/3717659111 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55656401a0c0 0x55656402b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:35.532 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.529+0000 7f77f6050740 1 -- 172.21.15.47:0/3717659111 >> 172.21.15.47:0/3717659111 conn(0x5565640149e0 msgr2=0x556564013b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:11:35.532 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.529+0000 7f77f6050740 1 -- 172.21.15.47:0/3717659111 shutdown_connections 2024-04-17T06:11:35.532 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.529+0000 7f77f6050740 1 -- 172.21.15.47:0/3717659111 wait complete. 2024-04-17T06:11:35.532 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.529+0000 7f77f6050740 20 client.16263 trim_cache size 0 max 16384 2024-04-17T06:11:35.532 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-04-17T06:11:35.532 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-04-17T06:11:35.532 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-04-17T06:11:35.532 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-04-17T06:11:35.533 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.530+0000 7f77f3463640 1 -- 172.21.15.47:0/2589955431 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f77d40a3dc0 msgr2=0x7f77d40a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:11:35.533 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.530+0000 7f77f3463640 1 --2- 172.21.15.47:0/2589955431 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f77d40a3dc0 0x7f77d40a6280 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f77e40114a0 tx=0x7f77e4009890 comp rx=0 tx=0).stop 2024-04-17T06:11:35.533 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.530+0000 7f77f3463640 1 -- 172.21.15.47:0/2589955431 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f77ec101100 msgr2=0x7f77ec19e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:11:35.533 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.530+0000 7f77f3463640 1 --2- 172.21.15.47:0/2589955431 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f77ec101100 0x7f77ec19e570 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f77dc00eab0 tx=0x7f77dc00ef70 comp rx=0 tx=0).stop 2024-04-17T06:11:35.533 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.530+0000 7f77f3463640 1 -- 172.21.15.47:0/2589955431 shutdown_connections 2024-04-17T06:11:35.533 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.530+0000 7f77f3463640 1 --2- 172.21.15.47:0/2589955431 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f77ec198630 0x7f77ec198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:35.533 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.530+0000 7f77f3463640 1 --2- 172.21.15.47:0/2589955431 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f77ec106180 0x7f77ec19eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:35.534 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.530+0000 7f77f3463640 1 --2- 172.21.15.47:0/2589955431 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f77d40a3dc0 0x7f77d40a6280 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:35.534 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.530+0000 7f77f3463640 1 --2- 172.21.15.47:0/2589955431 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f77ec101100 0x7f77ec19e570 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:35.534 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.530+0000 7f77f3463640 1 -- 172.21.15.47:0/2589955431 >> 172.21.15.47:0/2589955431 conn(0x7f77ec077fe0 msgr2=0x7f77ec069970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:11:35.534 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.530+0000 7f77f3463640 1 -- 172.21.15.47:0/2589955431 shutdown_connections 2024-04-17T06:11:35.534 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:35.530+0000 7f77f3463640 1 -- 172.21.15.47:0/2589955431 wait complete. 2024-04-17T06:11:36.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:11:35 smithi098 ceph-mon[36516]: Adjusting osd_memory_target on smithi047 to 177.2M 2024-04-17T06:11:36.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:11:35 smithi098 ceph-mon[36516]: Unable to set osd_memory_target on smithi047 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-04-17T06:11:36.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:11:35 smithi098 ceph-mon[36516]: Adjusting osd_memory_target on smithi098 to 174.1M 2024-04-17T06:11:36.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:11:35 smithi098 ceph-mon[36516]: Unable to set osd_memory_target on smithi098 to 182632755: error parsing value: Value '182632755' is below minimum 939524096 2024-04-17T06:11:36.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:11:35 smithi098 ceph-mon[36516]: pgmap v323: 97 pgs: 97 active+clean; 5.1 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-04-17T06:11:36.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:11:35 smithi047 ceph-mon[32223]: Adjusting osd_memory_target on smithi047 to 177.2M 2024-04-17T06:11:36.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:11:35 smithi047 ceph-mon[32223]: Unable to set osd_memory_target on smithi047 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-04-17T06:11:36.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:11:35 smithi047 ceph-mon[32223]: Adjusting osd_memory_target on smithi098 to 174.1M 2024-04-17T06:11:36.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:11:35 smithi047 ceph-mon[32223]: Unable to set osd_memory_target on smithi098 to 182632755: error parsing value: Value '182632755' is below minimum 939524096 2024-04-17T06:11:36.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:11:35 smithi047 ceph-mon[32223]: pgmap v323: 97 pgs: 97 active+clean; 5.1 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-04-17T06:11:36.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:11:35 smithi103 ceph-mon[36243]: Adjusting osd_memory_target on smithi047 to 177.2M 2024-04-17T06:11:36.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:11:35 smithi103 ceph-mon[36243]: Unable to set osd_memory_target on smithi047 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-04-17T06:11:36.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:11:35 smithi103 ceph-mon[36243]: Adjusting osd_memory_target on smithi098 to 174.1M 2024-04-17T06:11:36.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:11:35 smithi103 ceph-mon[36243]: Unable to set osd_memory_target on smithi098 to 182632755: error parsing value: Value '182632755' is below minimum 939524096 2024-04-17T06:11:36.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:11:35 smithi103 ceph-mon[36243]: pgmap v323: 97 pgs: 97 active+clean; 5.1 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-04-17T06:11:38.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:11:37 smithi098 ceph-mon[36516]: pgmap v324: 97 pgs: 97 active+clean; 5.1 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.2 KiB/s wr, 9 op/s 2024-04-17T06:11:38.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:11:37 smithi047 ceph-mon[32223]: pgmap v324: 97 pgs: 97 active+clean; 5.1 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.2 KiB/s wr, 9 op/s 2024-04-17T06:11:38.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:11:37 smithi103 ceph-mon[36243]: pgmap v324: 97 pgs: 97 active+clean; 5.1 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.2 KiB/s wr, 9 op/s 2024-04-17T06:11:40.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:11:39 smithi098 ceph-mon[36516]: pgmap v325: 97 pgs: 97 active+clean; 5.1 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.2 KiB/s wr, 9 op/s 2024-04-17T06:11:40.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:11:39 smithi047 ceph-mon[32223]: pgmap v325: 97 pgs: 97 active+clean; 5.1 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.2 KiB/s wr, 9 op/s 2024-04-17T06:11:40.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:11:39 smithi103 ceph-mon[36243]: pgmap v325: 97 pgs: 97 active+clean; 5.1 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.2 KiB/s wr, 9 op/s 2024-04-17T06:11:42.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:11:41 smithi098 ceph-mon[36516]: pgmap v326: 97 pgs: 97 active+clean; 5.2 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-04-17T06:11:42.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:11:41 smithi047 ceph-mon[32223]: pgmap v326: 97 pgs: 97 active+clean; 5.2 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-04-17T06:11:42.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:11:41 smithi103 ceph-mon[36243]: pgmap v326: 97 pgs: 97 active+clean; 5.2 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-04-17T06:11:44.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:11:43 smithi098 ceph-mon[36516]: pgmap v327: 97 pgs: 97 active+clean; 5.2 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:11:44.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:11:43 smithi047 ceph-mon[32223]: pgmap v327: 97 pgs: 97 active+clean; 5.2 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:11:44.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:11:43 smithi103 ceph-mon[36243]: pgmap v327: 97 pgs: 97 active+clean; 5.2 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:11:46.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:11:45 smithi098 ceph-mon[36516]: pgmap v328: 97 pgs: 97 active+clean; 5.2 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-04-17T06:11:46.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:11:45 smithi047 ceph-mon[32223]: pgmap v328: 97 pgs: 97 active+clean; 5.2 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-04-17T06:11:46.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:11:45 smithi103 ceph-mon[36243]: pgmap v328: 97 pgs: 97 active+clean; 5.2 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-04-17T06:11:48.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:11:47 smithi098 ceph-mon[36516]: pgmap v329: 97 pgs: 97 active+clean; 5.2 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:11:48.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:11:47 smithi047 ceph-mon[32223]: pgmap v329: 97 pgs: 97 active+clean; 5.2 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:11:48.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:11:47 smithi103 ceph-mon[36243]: pgmap v329: 97 pgs: 97 active+clean; 5.2 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:11:48.629 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.630+0000 7fae61635640 1 -- 172.21.15.47:0/3363812457 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae5c1021e0 msgr2=0x7fae5c1025c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:11:48.629 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.630+0000 7fae61635640 1 --2- 172.21.15.47:0/3363812457 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae5c1021e0 0x7fae5c1025c0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fae4c009900 tx=0x7fae4c02bc30 comp rx=0 tx=0).stop 2024-04-17T06:11:48.629 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.630+0000 7fae61635640 1 -- 172.21.15.47:0/3363812457 shutdown_connections 2024-04-17T06:11:48.629 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.630+0000 7fae61635640 1 --2- 172.21.15.47:0/3363812457 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae5c1072b0 0x7fae5c10ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:48.629 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.630+0000 7fae61635640 1 --2- 172.21.15.47:0/3363812457 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fae5c102b90 0x7fae5c106bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:48.629 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.630+0000 7fae61635640 1 --2- 172.21.15.47:0/3363812457 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae5c1021e0 0x7fae5c1025c0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:48.629 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.630+0000 7fae61635640 1 -- 172.21.15.47:0/3363812457 >> 172.21.15.47:0/3363812457 conn(0x7fae5c0793f0 msgr2=0x7fae5c079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:11:48.629 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.630+0000 7fae61635640 1 -- 172.21.15.47:0/3363812457 shutdown_connections 2024-04-17T06:11:48.629 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.631+0000 7fae61635640 1 -- 172.21.15.47:0/3363812457 wait complete. 2024-04-17T06:11:48.629 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.631+0000 7fae61635640 1 Processor -- start 2024-04-17T06:11:48.631 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.631+0000 7fae61635640 1 -- start start 2024-04-17T06:11:48.631 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.632+0000 7fae61635640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fae5c1021e0 0x7fae5c074260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:48.631 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.632+0000 7fae61635640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae5c1072b0 0x7fae5c0747a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:48.631 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.632+0000 7fae61635640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae5c074ce0 0x7fae5c1a6960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:48.632 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.632+0000 7fae61635640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fae5c1123b0 con 0x7fae5c074ce0 2024-04-17T06:11:48.632 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.632+0000 7fae61635640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fae5c112230 con 0x7fae5c1072b0 2024-04-17T06:11:48.632 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.632+0000 7fae61635640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fae5c112530 con 0x7fae5c1021e0 2024-04-17T06:11:48.632 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.632+0000 7fae5b7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae5c074ce0 0x7fae5c1a6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:48.632 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.632+0000 7fae5b7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae5c074ce0 0x7fae5c1a6960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39704/0 (socket says 172.21.15.47:39704) 2024-04-17T06:11:48.632 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.632+0000 7fae5b7fe640 1 -- 172.21.15.47:0/232648477 learned_addr learned my addr 172.21.15.47:0/232648477 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:11:48.632 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.632+0000 7fae5affd640 1 --2- 172.21.15.47:0/232648477 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fae5c1021e0 0x7fae5c074260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:48.632 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.632+0000 7fae5a7fc640 1 --2- 172.21.15.47:0/232648477 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae5c1072b0 0x7fae5c0747a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:48.632 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.632+0000 7fae5b7fe640 1 -- 172.21.15.47:0/232648477 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fae5c1021e0 msgr2=0x7fae5c074260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:11:48.632 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.632+0000 7fae5b7fe640 1 --2- 172.21.15.47:0/232648477 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fae5c1021e0 0x7fae5c074260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:48.632 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.632+0000 7fae5b7fe640 1 -- 172.21.15.47:0/232648477 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae5c1072b0 msgr2=0x7fae5c0747a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:11:48.632 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.632+0000 7fae5b7fe640 1 --2- 172.21.15.47:0/232648477 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae5c1072b0 0x7fae5c0747a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:48.632 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.632+0000 7fae5b7fe640 1 -- 172.21.15.47:0/232648477 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae5c1a7060 con 0x7fae5c074ce0 2024-04-17T06:11:48.632 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.632+0000 7fae5b7fe640 1 --2- 172.21.15.47:0/232648477 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae5c074ce0 0x7fae5c1a6960 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fae5400e960 tx=0x7fae5400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:11:48.632 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.632+0000 7fae43fff640 1 -- 172.21.15.47:0/232648477 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae54004d20 con 0x7fae5c074ce0 2024-04-17T06:11:48.633 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.632+0000 7fae61635640 1 -- 172.21.15.47:0/232648477 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae5c1a7350 con 0x7fae5c074ce0 2024-04-17T06:11:48.633 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.632+0000 7fae43fff640 1 -- 172.21.15.47:0/232648477 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae54005020 con 0x7fae5c074ce0 2024-04-17T06:11:48.633 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.632+0000 7fae43fff640 1 -- 172.21.15.47:0/232648477 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae540116d0 con 0x7fae5c074ce0 2024-04-17T06:11:48.633 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.633+0000 7fae61635640 1 -- 172.21.15.47:0/232648477 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae5c1a7850 con 0x7fae5c074ce0 2024-04-17T06:11:48.639 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.635+0000 7fae43fff640 1 -- 172.21.15.47:0/232648477 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fae54005560 con 0x7fae5c074ce0 2024-04-17T06:11:48.639 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.635+0000 7fae64224740 1 Processor -- start 2024-04-17T06:11:48.639 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.635+0000 7fae64224740 1 -- start start 2024-04-17T06:11:48.640 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.636+0000 7fae64224740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5605195450c0 0x5605195454a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:48.640 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.636+0000 7fae64224740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5605195459e0 0x56051954e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:48.640 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.636+0000 7fae64224740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56051954e7b0 0x560519550ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:48.640 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.636+0000 7fae64224740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5605194aa7d0 con 0x5605195459e0 2024-04-17T06:11:48.640 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.636+0000 7fae64224740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x560519447f70 con 0x56051954e7b0 2024-04-17T06:11:48.640 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.636+0000 7fae64224740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5605194a6790 con 0x5605195450c0 2024-04-17T06:11:48.640 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.636+0000 7fae5affd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5605195459e0 0x56051954e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:48.640 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.636+0000 7fae5affd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5605195459e0 0x56051954e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39720/0 (socket says 172.21.15.47:39720) 2024-04-17T06:11:48.640 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.636+0000 7fae5affd640 1 -- 172.21.15.47:0/2046534751 learned_addr learned my addr 172.21.15.47:0/2046534751 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:11:48.640 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.636+0000 7fae5a7fc640 1 --2- 172.21.15.47:0/2046534751 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56051954e7b0 0x560519550ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:48.640 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.636+0000 7fae43fff640 1 --2- 172.21.15.47:0/232648477 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fae440a3df0 0x7fae440a62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:48.640 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.636+0000 7fae5a7fc640 1 --2- 172.21.15.47:0/2046534751 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5605195450c0 0x5605195454a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:48.640 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.636+0000 7fae5affd640 1 -- 172.21.15.47:0/2046534751 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5605195450c0 msgr2=0x5605195454a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:11:48.640 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.636+0000 7fae5affd640 1 --2- 172.21.15.47:0/2046534751 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5605195450c0 0x5605195454a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:48.641 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.636+0000 7fae5affd640 1 -- 172.21.15.47:0/2046534751 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56051954e7b0 msgr2=0x560519550ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:11:48.641 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.636+0000 7fae5affd640 1 --2- 172.21.15.47:0/2046534751 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56051954e7b0 0x560519550ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:48.641 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.636+0000 7fae5b7fe640 1 --2- 172.21.15.47:0/232648477 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fae440a3df0 0x7fae440a62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:48.641 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.636+0000 7fae5affd640 1 -- 172.21.15.47:0/2046534751 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605191bd210 con 0x5605195459e0 2024-04-17T06:11:48.641 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.636+0000 7fae5a7fc640 1 --2- 172.21.15.47:0/2046534751 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5605195450c0 0x5605195454a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:11:48.641 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.636+0000 7fae43fff640 1 -- 172.21.15.47:0/232648477 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8513+0+0 (secure 0 0 0) 0x7fae540d0820 con 0x7fae5c074ce0 2024-04-17T06:11:48.641 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.636+0000 7fae5affd640 1 --2- 172.21.15.47:0/2046534751 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5605195459e0 0x56051954e270 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fae4c009900 tx=0x7fae4c02c5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a363695c19a93d61 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:11:48.641 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.636+0000 7fae5b7fe640 1 --2- 172.21.15.47:0/232648477 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fae440a3df0 0x7fae440a62b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fae540ce020 tx=0x7fae5409f040 comp rx=0 tx=0).ready entity=mgr.24923 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:11:48.641 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.637+0000 7fae417fa640 1 -- 172.21.15.47:0/2046534751 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae4c045070 con 0x5605195459e0 2024-04-17T06:11:48.641 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.637+0000 7fae417fa640 1 -- 172.21.15.47:0/2046534751 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae4c02c880 con 0x5605195459e0 2024-04-17T06:11:48.641 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.637+0000 7fae417fa640 1 -- 172.21.15.47:0/2046534751 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae4c004f20 con 0x5605195459e0 2024-04-17T06:11:48.641 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.637+0000 7fae64224740 1 -- 172.21.15.47:0/2046534751 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5605195459e0 msgr2=0x56051954e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:11:48.641 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.637+0000 7fae64224740 1 --2- 172.21.15.47:0/2046534751 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5605195459e0 0x56051954e270 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fae4c009900 tx=0x7fae4c02c5d0 comp rx=0 tx=0).stop 2024-04-17T06:11:48.641 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.637+0000 7fae64224740 1 -- 172.21.15.47:0/2046534751 shutdown_connections 2024-04-17T06:11:48.641 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.637+0000 7fae64224740 1 --2- 172.21.15.47:0/2046534751 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56051954e7b0 0x560519550ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:48.642 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.637+0000 7fae64224740 1 --2- 172.21.15.47:0/2046534751 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5605195459e0 0x56051954e270 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:48.642 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.637+0000 7fae64224740 1 --2- 172.21.15.47:0/2046534751 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5605195450c0 0x5605195454a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:48.642 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.637+0000 7fae64224740 1 -- 172.21.15.47:0/2046534751 >> 172.21.15.47:0/2046534751 conn(0x56051953f9e0 msgr2=0x56051953fdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:11:48.642 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.637+0000 7fae64224740 1 -- 172.21.15.47:0/2046534751 shutdown_connections 2024-04-17T06:11:48.642 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.637+0000 7fae64224740 1 -- 172.21.15.47:0/2046534751 wait complete. 2024-04-17T06:11:48.642 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.638+0000 7fae64224740 1 -- start start 2024-04-17T06:11:48.642 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.638+0000 7fae64224740 1 Processor -- start 2024-04-17T06:11:48.642 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.638+0000 7fae64224740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5605195450c0 0x56051954b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:48.642 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.638+0000 7fae64224740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5605195459e0 0x56051954bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:48.642 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.638+0000 7fae64224740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56051954e7b0 0x56051954c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:48.642 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.638+0000 7fae64224740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x560519551380 con 0x5605195459e0 2024-04-17T06:11:48.642 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.638+0000 7fae64224740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x560519432e90 con 0x5605195450c0 2024-04-17T06:11:48.642 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.638+0000 7fae64224740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560519551500 con 0x56051954e7b0 2024-04-17T06:11:48.642 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.638+0000 7fae64224740 10 client.0 _ll_register_callbacks cb 0x560519448d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:11:48.643 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.638+0000 7fae5affd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5605195459e0 0x56051954bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:48.643 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.638+0000 7fae5affd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5605195459e0 0x56051954bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39726/0 (socket says 172.21.15.47:39726) 2024-04-17T06:11:48.643 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.638+0000 7fae5affd640 1 -- 172.21.15.47:0/2924160190 learned_addr learned my addr 172.21.15.47:0/2924160190 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:11:48.643 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.638+0000 7fae5a7fc640 1 --2- 172.21.15.47:0/2924160190 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5605195450c0 0x56051954b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:48.643 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.638+0000 7fae5affd640 1 -- 172.21.15.47:0/2924160190 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56051954e7b0 msgr2=0x56051954c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:11:48.643 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.638+0000 7fae5affd640 1 --2- 172.21.15.47:0/2924160190 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56051954e7b0 0x56051954c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:48.643 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.638+0000 7fae5affd640 1 -- 172.21.15.47:0/2924160190 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5605195450c0 msgr2=0x56051954b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:11:48.643 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.638+0000 7fae5affd640 1 --2- 172.21.15.47:0/2924160190 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5605195450c0 0x56051954b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:48.643 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.638+0000 7fae5affd640 1 -- 172.21.15.47:0/2924160190 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560519289620 con 0x5605195459e0 2024-04-17T06:11:48.643 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.639+0000 7fae5affd640 1 --2- 172.21.15.47:0/2924160190 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5605195459e0 0x56051954bcb0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fae4c036040 tx=0x7fae4c02ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=296c91f988158c69 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:11:48.643 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.639+0000 7fae3eff5640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-17T06:11:48.643 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.639+0000 7fae3eff5640 1 -- 172.21.15.47:0/2924160190 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae4c04e070 con 0x5605195459e0 2024-04-17T06:11:48.643 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.639+0000 7fae3eff5640 1 -- 172.21.15.47:0/2924160190 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8513+0+0 (secure 0 0 0) 0x7fae4c004ec0 con 0x5605195459e0 2024-04-17T06:11:48.643 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.639+0000 7fae64224740 1 -- 172.21.15.47:0/2924160190 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae4c02c880 con 0x5605195459e0 2024-04-17T06:11:48.643 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.639+0000 7fae3eff5640 1 -- 172.21.15.47:0/2924160190 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fae4c05a050 con 0x5605195459e0 2024-04-17T06:11:48.643 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.639+0000 7fae64224740 10 client.16275 fetch_fsmap learned FSMap version 34 2024-04-17T06:11:48.644 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.639+0000 7fae64224740 1 -- 172.21.15.47:0/2924160190 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56051955d580 con 0x5605195459e0 2024-04-17T06:11:48.644 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.639+0000 7fae3eff5640 1 -- 172.21.15.47:0/2924160190 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae4c02c880 con 0x5605195459e0 2024-04-17T06:11:48.644 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.639+0000 7fae3eff5640 1 -- 172.21.15.47:0/2924160190 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 34) ==== 8339+0+0 (secure 0 0 0) 0x7fae4c003b50 con 0x5605195459e0 2024-04-17T06:11:48.644 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.639+0000 7fae64224740 10 client.16275 fetch_fsmap finished waiting for FSMap version 34 2024-04-17T06:11:48.644 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.640+0000 7fae64224740 10 client.16275 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-17T06:11:48.644 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.640+0000 7fae64224740 20 client.16275 populate_metadata read hostname 'smithi047' 2024-04-17T06:11:48.644 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.640+0000 7fae64224740 1 --2- 172.21.15.47:0/2924160190 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x56051955e8d0 0x560519560cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:48.644 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.640+0000 7fae64224740 4 client.16275 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T06:11:48.644 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.640+0000 7fae64224740 1 -- 172.21.15.47:0/2924160190 --> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5605194b81f0 con 0x56051955e8d0 2024-04-17T06:11:48.644 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.640+0000 7fae5a7fc640 1 --2- 172.21.15.47:0/2924160190 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x56051955e8d0 0x560519560cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:48.644 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.640+0000 7fae3eff5640 10 client.16275 ms_handle_connect on v2:172.21.15.98:6838/2841287917 2024-04-17T06:11:48.644 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.640+0000 7fae5a7fc640 1 --2- 172.21.15.47:0/2924160190 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x56051955e8d0 0x560519560cc0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac4b48a753f374d7 server_cookie=4c9797dbb6888d1c in_seq=0 out_seq=0 2024-04-17T06:11:48.644 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.641+0000 7fae3eff5640 1 -- 172.21.15.47:0/2924160190 <== mds.0 v2:172.21.15.98:6838/2841287917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5605194b81f0 con 0x56051955e8d0 2024-04-17T06:11:48.644 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.641+0000 7fae3eff5640 10 client.16275 handle_command_reply: tid=0 2024-04-17T06:11:48.644 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.641+0000 7fae64224740 1 client.16275 shutdown 2024-04-17T06:11:48.644 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.641+0000 7fae64224740 1 -- 172.21.15.47:0/2924160190 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5605195459e0 msgr2=0x56051954bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:11:48.645 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.641+0000 7fae64224740 1 --2- 172.21.15.47:0/2924160190 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5605195459e0 0x56051954bcb0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fae4c036040 tx=0x7fae4c02ca30 comp rx=0 tx=0).stop 2024-04-17T06:11:48.645 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.641+0000 7fae64224740 1 -- 172.21.15.47:0/2924160190 shutdown_connections 2024-04-17T06:11:48.645 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.641+0000 7fae64224740 1 --2- 172.21.15.47:0/2924160190 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x56051955e8d0 0x560519560cc0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:48.645 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.641+0000 7fae64224740 1 --2- 172.21.15.47:0/2924160190 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56051954e7b0 0x56051954c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:48.645 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.641+0000 7fae64224740 1 --2- 172.21.15.47:0/2924160190 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5605195459e0 0x56051954bcb0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:48.645 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.641+0000 7fae3eff5640 0 client.16275 ms_handle_reset on v2:172.21.15.98:6838/2841287917 2024-04-17T06:11:48.645 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.641+0000 7fae64224740 1 --2- 172.21.15.47:0/2924160190 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5605195450c0 0x56051954b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:48.645 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.641+0000 7fae64224740 1 -- 172.21.15.47:0/2924160190 >> 172.21.15.47:0/2924160190 conn(0x56051953f9e0 msgr2=0x560519541730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:11:48.645 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.641+0000 7fae64224740 1 -- 172.21.15.47:0/2924160190 shutdown_connections 2024-04-17T06:11:48.645 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.641+0000 7fae64224740 1 -- 172.21.15.47:0/2924160190 wait complete. 2024-04-17T06:11:48.645 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.641+0000 7fae64224740 20 client.16275 trim_cache size 0 max 16384 2024-04-17T06:11:48.659 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.657+0000 7fae64224740 1 Processor -- start 2024-04-17T06:11:48.659 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.657+0000 7fae64224740 1 -- start start 2024-04-17T06:11:48.659 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.657+0000 7fae64224740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5605195450c0 0x56051954dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:48.659 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.657+0000 7fae64224740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5605195459e0 0x560519563ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:48.659 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.657+0000 7fae64224740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56051954e7b0 0x560519564220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:48.659 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.657+0000 7fae64224740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fae5c1b2690 con 0x5605195459e0 2024-04-17T06:11:48.659 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.657+0000 7fae64224740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fae5c1aff50 con 0x56051954e7b0 2024-04-17T06:11:48.659 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.657+0000 7fae64224740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fae5c1addc0 con 0x5605195450c0 2024-04-17T06:11:48.659 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.657+0000 7fae5affd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5605195459e0 0x560519563ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:48.659 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.657+0000 7fae5affd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5605195459e0 0x560519563ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39730/0 (socket says 172.21.15.47:39730) 2024-04-17T06:11:48.659 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.657+0000 7fae5affd640 1 -- 172.21.15.47:0/961156122 learned_addr learned my addr 172.21.15.47:0/961156122 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:11:48.659 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.657+0000 7fae5a7fc640 1 --2- 172.21.15.47:0/961156122 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56051954e7b0 0x560519564220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:48.659 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.657+0000 7fae5a7fc640 1 --2- 172.21.15.47:0/961156122 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5605195450c0 0x56051954dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:48.659 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.657+0000 7fae5affd640 1 -- 172.21.15.47:0/961156122 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5605195450c0 msgr2=0x56051954dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:11:48.660 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.657+0000 7fae5affd640 1 --2- 172.21.15.47:0/961156122 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5605195450c0 0x56051954dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:48.660 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.657+0000 7fae5affd640 1 -- 172.21.15.47:0/961156122 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56051954e7b0 msgr2=0x560519564220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:11:48.660 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.657+0000 7fae5affd640 1 --2- 172.21.15.47:0/961156122 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56051954e7b0 0x560519564220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:48.660 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.657+0000 7fae5affd640 1 -- 172.21.15.47:0/961156122 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56051953f810 con 0x5605195459e0 2024-04-17T06:11:48.660 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.657+0000 7fae5a7fc640 1 --2- 172.21.15.47:0/961156122 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5605195450c0 0x56051954dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:11:48.660 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.657+0000 7fae5affd640 1 --2- 172.21.15.47:0/961156122 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5605195459e0 0x560519563ce0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fae4c0048f0 tx=0x7fae4c02c5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c30e3692280bd4c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:11:48.660 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.657+0000 7fae40ff9640 1 -- 172.21.15.47:0/961156122 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae4c04e040 con 0x5605195459e0 2024-04-17T06:11:48.660 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.658+0000 7fae40ff9640 1 -- 172.21.15.47:0/961156122 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae4c02caa0 con 0x5605195459e0 2024-04-17T06:11:48.660 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.658+0000 7fae40ff9640 1 -- 172.21.15.47:0/961156122 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae4c045040 con 0x5605195459e0 2024-04-17T06:11:48.660 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.658+0000 7fae64224740 1 -- 172.21.15.47:0/961156122 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5605195459e0 msgr2=0x560519563ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:11:48.660 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.658+0000 7fae64224740 1 --2- 172.21.15.47:0/961156122 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5605195459e0 0x560519563ce0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fae4c0048f0 tx=0x7fae4c02c5d0 comp rx=0 tx=0).stop 2024-04-17T06:11:48.660 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.658+0000 7fae64224740 1 -- 172.21.15.47:0/961156122 shutdown_connections 2024-04-17T06:11:48.660 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.658+0000 7fae64224740 1 --2- 172.21.15.47:0/961156122 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56051954e7b0 0x560519564220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:48.660 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.658+0000 7fae64224740 1 --2- 172.21.15.47:0/961156122 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5605195459e0 0x560519563ce0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:48.660 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.658+0000 7fae64224740 1 --2- 172.21.15.47:0/961156122 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5605195450c0 0x56051954dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:48.660 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.658+0000 7fae64224740 1 -- 172.21.15.47:0/961156122 >> 172.21.15.47:0/961156122 conn(0x56051953f9e0 msgr2=0x560519559fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:11:48.661 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.658+0000 7fae64224740 1 -- 172.21.15.47:0/961156122 shutdown_connections 2024-04-17T06:11:48.661 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.658+0000 7fae64224740 1 -- 172.21.15.47:0/961156122 wait complete. 2024-04-17T06:11:48.661 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.658+0000 7fae64224740 1 -- start start 2024-04-17T06:11:48.661 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.659+0000 7fae64224740 1 Processor -- start 2024-04-17T06:11:48.661 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.659+0000 7fae64224740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5605195450c0 0x5605195617a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:48.661 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.659+0000 7fae64224740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5605195459e0 0x560519561ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:48.661 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.659+0000 7fae64224740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56051954e7b0 0x560519562220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:48.661 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.659+0000 7fae64224740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fae5c1ae410 con 0x56051954e7b0 2024-04-17T06:11:48.661 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.659+0000 7fae64224740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fae5c1af880 con 0x5605195450c0 2024-04-17T06:11:48.661 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.659+0000 7fae64224740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fae5c1aeae0 con 0x5605195459e0 2024-04-17T06:11:48.661 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.659+0000 7fae64224740 10 client.0 _ll_register_callbacks cb 0x7fae5000ba80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:11:48.661 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.659+0000 7fae5a7fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56051954e7b0 0x560519562220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:48.661 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.659+0000 7fae5a7fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56051954e7b0 0x560519562220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39740/0 (socket says 172.21.15.47:39740) 2024-04-17T06:11:48.661 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.659+0000 7fae5a7fc640 1 -- 172.21.15.47:0/1524513106 learned_addr learned my addr 172.21.15.47:0/1524513106 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:11:48.661 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.659+0000 7fae5a7fc640 1 --2- 172.21.15.47:0/1524513106 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5605195450c0 0x5605195617a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:48.662 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.659+0000 7fae5affd640 1 --2- 172.21.15.47:0/1524513106 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5605195459e0 0x560519561ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:48.662 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.659+0000 7fae5a7fc640 1 -- 172.21.15.47:0/1524513106 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5605195459e0 msgr2=0x560519561ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:11:48.662 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.659+0000 7fae5a7fc640 1 --2- 172.21.15.47:0/1524513106 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5605195459e0 0x560519561ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:48.662 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.659+0000 7fae5a7fc640 1 -- 172.21.15.47:0/1524513106 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5605195450c0 msgr2=0x5605195617a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:11:48.662 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.659+0000 7fae5a7fc640 1 --2- 172.21.15.47:0/1524513106 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5605195450c0 0x5605195617a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:48.662 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.659+0000 7fae5a7fc640 1 -- 172.21.15.47:0/1524513106 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560519544aa0 con 0x56051954e7b0 2024-04-17T06:11:48.662 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.659+0000 7fae5a7fc640 1 --2- 172.21.15.47:0/1524513106 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56051954e7b0 0x560519562220 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fae500041e0 tx=0x7fae50014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6dfecb8f2e0e7d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:11:48.662 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.659+0000 7fae3eff5640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-17T06:11:48.662 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.660+0000 7fae3eff5640 1 -- 172.21.15.47:0/1524513106 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae50003f10 con 0x56051954e7b0 2024-04-17T06:11:48.662 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.660+0000 7fae3eff5640 1 -- 172.21.15.47:0/1524513106 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8513+0+0 (secure 0 0 0) 0x7fae50021780 con 0x56051954e7b0 2024-04-17T06:11:48.662 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.660+0000 7fae64224740 1 -- 172.21.15.47:0/1524513106 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae4c02caa0 con 0x56051954e7b0 2024-04-17T06:11:48.662 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.660+0000 7fae3eff5640 1 -- 172.21.15.47:0/1524513106 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fae50034050 con 0x56051954e7b0 2024-04-17T06:11:48.662 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.660+0000 7fae64224740 10 client.16281 fetch_fsmap learned FSMap version 34 2024-04-17T06:11:48.663 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.660+0000 7fae64224740 1 -- 172.21.15.47:0/1524513106 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5605195c4bc0 con 0x56051954e7b0 2024-04-17T06:11:48.663 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.660+0000 7fae3eff5640 1 -- 172.21.15.47:0/1524513106 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae4c02caa0 con 0x56051954e7b0 2024-04-17T06:11:48.663 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.660+0000 7fae3eff5640 1 -- 172.21.15.47:0/1524513106 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 34) ==== 8339+0+0 (secure 0 0 0) 0x7fae50020420 con 0x56051954e7b0 2024-04-17T06:11:48.663 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.660+0000 7fae64224740 10 client.16281 fetch_fsmap finished waiting for FSMap version 34 2024-04-17T06:11:48.663 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.660+0000 7fae64224740 10 client.16281 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-17T06:11:48.663 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.660+0000 7fae64224740 20 client.16281 populate_metadata read hostname 'smithi047' 2024-04-17T06:11:48.663 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.660+0000 7fae64224740 1 --2- 172.21.15.47:0/1524513106 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x5605195c5340 0x5605195c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:48.663 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.660+0000 7fae64224740 4 client.16281 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-17T06:11:48.663 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.660+0000 7fae64224740 1 -- 172.21.15.47:0/1524513106 --> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5605194b2060 con 0x5605195c5340 2024-04-17T06:11:48.663 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.660+0000 7fae5affd640 1 --2- 172.21.15.47:0/1524513106 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x5605195c5340 0x5605195c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:48.663 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.661+0000 7fae5affd640 1 --2- 172.21.15.47:0/1524513106 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x5605195c5340 0x5605195c7760 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3d6c9f16570c090 server_cookie=1fbea6cfbc790a57 in_seq=0 out_seq=0 2024-04-17T06:11:48.663 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.661+0000 7fae3eff5640 10 client.16281 ms_handle_connect on v2:172.21.15.98:6838/2841287917 2024-04-17T06:11:48.667 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.669+0000 7fae3eff5640 1 -- 172.21.15.47:0/1524513106 <== mds.0 v2:172.21.15.98:6838/2841287917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5605194b2060 con 0x5605195c5340 2024-04-17T06:11:48.667 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.669+0000 7fae3eff5640 10 client.16281 handle_command_reply: tid=0 2024-04-17T06:11:48.667 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.669+0000 7fae64224740 1 client.16281 shutdown 2024-04-17T06:11:48.668 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.669+0000 7fae64224740 1 -- 172.21.15.47:0/1524513106 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56051954e7b0 msgr2=0x560519562220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:11:48.668 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.669+0000 7fae64224740 1 --2- 172.21.15.47:0/1524513106 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56051954e7b0 0x560519562220 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fae500041e0 tx=0x7fae50014040 comp rx=0 tx=0).stop 2024-04-17T06:11:48.668 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.669+0000 7fae64224740 1 -- 172.21.15.47:0/1524513106 shutdown_connections 2024-04-17T06:11:48.668 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.669+0000 7fae64224740 1 --2- 172.21.15.47:0/1524513106 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x5605195c5340 0x5605195c7760 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:48.668 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.669+0000 7fae64224740 1 --2- 172.21.15.47:0/1524513106 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56051954e7b0 0x560519562220 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:48.668 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.669+0000 7fae64224740 1 --2- 172.21.15.47:0/1524513106 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5605195459e0 0x560519561ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:48.668 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.670+0000 7fae64224740 1 --2- 172.21.15.47:0/1524513106 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5605195450c0 0x5605195617a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:48.668 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.670+0000 7fae64224740 1 -- 172.21.15.47:0/1524513106 >> 172.21.15.47:0/1524513106 conn(0x56051953f9e0 msgr2=0x56051953e270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:11:48.668 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.669+0000 7fae3eff5640 0 client.16281 ms_handle_reset on v2:172.21.15.98:6838/2841287917 2024-04-17T06:11:48.669 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.670+0000 7fae64224740 1 -- 172.21.15.47:0/1524513106 shutdown_connections 2024-04-17T06:11:48.669 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.670+0000 7fae64224740 1 -- 172.21.15.47:0/1524513106 wait complete. 2024-04-17T06:11:48.669 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.670+0000 7fae64224740 20 client.16281 trim_cache size 0 max 16384 2024-04-17T06:11:48.669 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-04-17T06:11:48.669 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-04-17T06:11:48.669 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-04-17T06:11:48.669 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-04-17T06:11:48.670 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.670+0000 7fae61635640 1 -- 172.21.15.47:0/232648477 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fae440a3df0 msgr2=0x7fae440a62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:11:48.670 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.670+0000 7fae61635640 1 --2- 172.21.15.47:0/232648477 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fae440a3df0 0x7fae440a62b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fae540ce020 tx=0x7fae5409f040 comp rx=0 tx=0).stop 2024-04-17T06:11:48.670 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.670+0000 7fae61635640 1 -- 172.21.15.47:0/232648477 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae5c074ce0 msgr2=0x7fae5c1a6960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:11:48.670 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.670+0000 7fae61635640 1 --2- 172.21.15.47:0/232648477 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae5c074ce0 0x7fae5c1a6960 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fae5400e960 tx=0x7fae5400ee20 comp rx=0 tx=0).stop 2024-04-17T06:11:48.670 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.670+0000 7fae61635640 1 -- 172.21.15.47:0/232648477 shutdown_connections 2024-04-17T06:11:48.670 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.670+0000 7fae61635640 1 --2- 172.21.15.47:0/232648477 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fae440a3df0 0x7fae440a62b0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:48.670 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.670+0000 7fae61635640 1 --2- 172.21.15.47:0/232648477 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae5c074ce0 0x7fae5c1a6960 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:48.671 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.670+0000 7fae61635640 1 --2- 172.21.15.47:0/232648477 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae5c1072b0 0x7fae5c0747a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:48.671 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.670+0000 7fae61635640 1 --2- 172.21.15.47:0/232648477 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fae5c1021e0 0x7fae5c074260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:48.671 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.670+0000 7fae61635640 1 -- 172.21.15.47:0/232648477 >> 172.21.15.47:0/232648477 conn(0x7fae5c0793f0 msgr2=0x7fae5c0ffc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:11:48.671 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.670+0000 7fae61635640 1 -- 172.21.15.47:0/232648477 shutdown_connections 2024-04-17T06:11:48.671 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:11:48.671+0000 7fae61635640 1 -- 172.21.15.47:0/232648477 wait complete. 2024-04-17T06:11:50.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:11:49 smithi098 ceph-mon[36516]: pgmap v330: 97 pgs: 97 active+clean; 5.2 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:11:50.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:11:49 smithi047 ceph-mon[32223]: pgmap v330: 97 pgs: 97 active+clean; 5.2 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:11:50.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:11:49 smithi103 ceph-mon[36243]: pgmap v330: 97 pgs: 97 active+clean; 5.2 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:11:52.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:11:51 smithi098 ceph-mon[36516]: pgmap v331: 97 pgs: 97 active+clean; 5.3 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 19 KiB/s wr, 17 op/s 2024-04-17T06:11:52.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:11:51 smithi047 ceph-mon[32223]: pgmap v331: 97 pgs: 97 active+clean; 5.3 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 19 KiB/s wr, 17 op/s 2024-04-17T06:11:52.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:11:51 smithi103 ceph-mon[36243]: pgmap v331: 97 pgs: 97 active+clean; 5.3 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 19 KiB/s wr, 17 op/s 2024-04-17T06:11:54.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:11:53 smithi098 ceph-mon[36516]: pgmap v332: 97 pgs: 97 active+clean; 5.3 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:11:54.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:11:53 smithi047 ceph-mon[32223]: pgmap v332: 97 pgs: 97 active+clean; 5.3 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:11:54.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:11:53 smithi103 ceph-mon[36243]: pgmap v332: 97 pgs: 97 active+clean; 5.3 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:11:56.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:11:55 smithi098 ceph-mon[36516]: pgmap v333: 97 pgs: 97 active+clean; 5.3 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-04-17T06:11:56.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:11:55 smithi047 ceph-mon[32223]: pgmap v333: 97 pgs: 97 active+clean; 5.3 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-04-17T06:11:56.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:11:55 smithi103 ceph-mon[36243]: pgmap v333: 97 pgs: 97 active+clean; 5.3 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-04-17T06:11:58.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:11:57 smithi098 ceph-mon[36516]: pgmap v334: 97 pgs: 97 active+clean; 5.3 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:11:58.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:11:57 smithi047 ceph-mon[32223]: pgmap v334: 97 pgs: 97 active+clean; 5.3 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:11:58.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:11:57 smithi103 ceph-mon[36243]: pgmap v334: 97 pgs: 97 active+clean; 5.3 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:12:00.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:11:59 smithi098 ceph-mon[36516]: pgmap v335: 97 pgs: 97 active+clean; 5.3 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:12:00.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:11:59 smithi047 ceph-mon[32223]: pgmap v335: 97 pgs: 97 active+clean; 5.3 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:12:00.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:11:59 smithi103 ceph-mon[36243]: pgmap v335: 97 pgs: 97 active+clean; 5.3 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:12:01.771 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.771+0000 7f8267635640 1 -- 172.21.15.47:0/2913746584 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8260074260 msgr2=0x7f8260074640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:12:01.772 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.771+0000 7f8267635640 1 --2- 172.21.15.47:0/2913746584 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8260074260 0x7f8260074640 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f8258009900 tx=0x7f825802bcd0 comp rx=0 tx=0).stop 2024-04-17T06:12:01.772 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.771+0000 7f8267635640 1 -- 172.21.15.47:0/2913746584 shutdown_connections 2024-04-17T06:12:01.772 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.771+0000 7f8267635640 1 --2- 172.21.15.47:0/2913746584 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8260113020 0x7f8260115410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:01.772 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.771+0000 7f8267635640 1 --2- 172.21.15.47:0/2913746584 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8260074b80 0x7f8260112ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:01.772 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.771+0000 7f8267635640 1 --2- 172.21.15.47:0/2913746584 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8260074260 0x7f8260074640 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:01.772 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.771+0000 7f8267635640 1 -- 172.21.15.47:0/2913746584 >> 172.21.15.47:0/2913746584 conn(0x7f82600793f0 msgr2=0x7f8260079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:12:01.772 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.771+0000 7f8267635640 1 -- 172.21.15.47:0/2913746584 shutdown_connections 2024-04-17T06:12:01.772 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.771+0000 7f8267635640 1 -- 172.21.15.47:0/2913746584 wait complete. 2024-04-17T06:12:01.772 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.772+0000 7f8267635640 1 Processor -- start 2024-04-17T06:12:01.772 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.772+0000 7f8267635640 1 -- start start 2024-04-17T06:12:01.772 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.772+0000 7f8267635640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8260074260 0x7f826019ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:01.772 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.772+0000 7f8267635640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8260074b80 0x7f826019f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:01.773 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.772+0000 7f8267635640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8260113020 0x7f82601a3620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:01.773 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.772+0000 7f8267635640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f8260118960 con 0x7f8260074260 2024-04-17T06:12:01.773 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.772+0000 7f8267635640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f82601187e0 con 0x7f8260074b80 2024-04-17T06:12:01.773 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.772+0000 7f8267635640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8260118ae0 con 0x7f8260113020 2024-04-17T06:12:01.773 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.772+0000 7f82653d1640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8260074260 0x7f826019ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:12:01.773 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.772+0000 7f82653d1640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8260074260 0x7f826019ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44172/0 (socket says 172.21.15.47:44172) 2024-04-17T06:12:01.773 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.772+0000 7f82653d1640 1 -- 172.21.15.47:0/233560735 learned_addr learned my addr 172.21.15.47:0/233560735 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:12:01.773 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.772+0000 7f8264bd0640 1 --2- 172.21.15.47:0/233560735 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8260074b80 0x7f826019f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:12:01.773 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.772+0000 7f8265bd2640 1 --2- 172.21.15.47:0/233560735 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8260113020 0x7f82601a3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:12:01.773 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.772+0000 7f82653d1640 1 -- 172.21.15.47:0/233560735 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8260113020 msgr2=0x7f82601a3620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:12:01.773 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.772+0000 7f82653d1640 1 --2- 172.21.15.47:0/233560735 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8260113020 0x7f82601a3620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:01.773 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.772+0000 7f82653d1640 1 -- 172.21.15.47:0/233560735 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8260074b80 msgr2=0x7f826019f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:12:01.773 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.772+0000 7f82653d1640 1 --2- 172.21.15.47:0/233560735 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8260074b80 0x7f826019f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:01.773 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.772+0000 7f82653d1640 1 -- 172.21.15.47:0/233560735 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82601a3d50 con 0x7f8260074260 2024-04-17T06:12:01.773 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.772+0000 7f8265bd2640 1 --2- 172.21.15.47:0/233560735 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8260113020 0x7f82601a3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:12:01.774 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.772+0000 7f8264bd0640 1 --2- 172.21.15.47:0/233560735 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8260074b80 0x7f826019f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:12:01.774 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.773+0000 7f82653d1640 1 --2- 172.21.15.47:0/233560735 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8260074260 0x7f826019ed60 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f8258006010 tx=0x7f82580043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:12:01.774 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.773+0000 7f825e7fc640 1 -- 172.21.15.47:0/233560735 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8258042020 con 0x7f8260074260 2024-04-17T06:12:01.774 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.773+0000 7f825e7fc640 1 -- 172.21.15.47:0/233560735 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82580054e0 con 0x7f8260074260 2024-04-17T06:12:01.774 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.773+0000 7f8267635640 1 -- 172.21.15.47:0/233560735 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82601a3fe0 con 0x7f8260074260 2024-04-17T06:12:01.774 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.773+0000 7f825e7fc640 1 -- 172.21.15.47:0/233560735 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8258003be0 con 0x7f8260074260 2024-04-17T06:12:01.774 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.773+0000 7f8267635640 1 -- 172.21.15.47:0/233560735 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82601ab8d0 con 0x7f8260074260 2024-04-17T06:12:01.781 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.776+0000 7f825e7fc640 1 -- 172.21.15.47:0/233560735 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8258045020 con 0x7f8260074260 2024-04-17T06:12:01.781 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.776+0000 7f826a228740 1 Processor -- start 2024-04-17T06:12:01.781 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.776+0000 7f826a228740 1 -- start start 2024-04-17T06:12:01.781 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.776+0000 7f826a228740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5609eda440c0 0x5609eda444a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:01.782 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.776+0000 7f826a228740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5609eda449e0 0x5609eda4d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:01.782 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.776+0000 7f826a228740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5609eda4d7b0 0x5609eda4fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:01.782 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.777+0000 7f826a228740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5609ed9a97d0 con 0x5609eda449e0 2024-04-17T06:12:01.782 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.777+0000 7f826a228740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5609ed946f70 con 0x5609eda440c0 2024-04-17T06:12:01.782 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.777+0000 7f826a228740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5609ed9a5790 con 0x5609eda4d7b0 2024-04-17T06:12:01.782 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.777+0000 7f82653d1640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5609eda449e0 0x5609eda4d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:12:01.782 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.777+0000 7f82653d1640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5609eda449e0 0x5609eda4d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44180/0 (socket says 172.21.15.47:44180) 2024-04-17T06:12:01.782 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.777+0000 7f82653d1640 1 -- 172.21.15.47:0/2667109905 learned_addr learned my addr 172.21.15.47:0/2667109905 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:12:01.782 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.777+0000 7f8264bd0640 1 --2- 172.21.15.47:0/2667109905 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5609eda4d7b0 0x5609eda4fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:12:01.782 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.777+0000 7f8265bd2640 1 --2- 172.21.15.47:0/2667109905 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5609eda440c0 0x5609eda444a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:12:01.782 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.777+0000 7f825e7fc640 1 --2- 172.21.15.47:0/233560735 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f82480a3dc0 0x7f82480a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:01.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.777+0000 7f82653d1640 1 -- 172.21.15.47:0/2667109905 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5609eda4d7b0 msgr2=0x5609eda4fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:12:01.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.777+0000 7f82653d1640 1 --2- 172.21.15.47:0/2667109905 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5609eda4d7b0 0x5609eda4fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:01.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.777+0000 7f82653d1640 1 -- 172.21.15.47:0/2667109905 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5609eda440c0 msgr2=0x5609eda444a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:12:01.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.777+0000 7f82653d1640 1 --2- 172.21.15.47:0/2667109905 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5609eda440c0 0x5609eda444a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:01.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.777+0000 7f82653d1640 1 -- 172.21.15.47:0/2667109905 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609ed6bc210 con 0x5609eda449e0 2024-04-17T06:12:01.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.777+0000 7f8265bd2640 1 --2- 172.21.15.47:0/233560735 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f82480a3dc0 0x7f82480a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:12:01.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.777+0000 7f825e7fc640 1 -- 172.21.15.47:0/233560735 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8513+0+0 (secure 0 0 0) 0x7f82580f25d0 con 0x7f8260074260 2024-04-17T06:12:01.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.777+0000 7f82653d1640 1 --2- 172.21.15.47:0/2667109905 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5609eda449e0 0x5609eda4d270 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f82580efd10 tx=0x7f82580e9730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=120a68728a751cc1 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:12:01.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.778+0000 7f82477fe640 1 -- 172.21.15.47:0/2667109905 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82580be160 con 0x5609eda449e0 2024-04-17T06:12:01.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.778+0000 7f82477fe640 1 -- 172.21.15.47:0/2667109905 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8258004c80 con 0x5609eda449e0 2024-04-17T06:12:01.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.778+0000 7f82477fe640 1 -- 172.21.15.47:0/2667109905 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82580be620 con 0x5609eda449e0 2024-04-17T06:12:01.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.778+0000 7f826a228740 1 -- 172.21.15.47:0/2667109905 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5609eda449e0 msgr2=0x5609eda4d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:12:01.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.778+0000 7f8265bd2640 1 --2- 172.21.15.47:0/233560735 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f82480a3dc0 0x7f82480a6280 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8250005cc0 tx=0x7f8250004a40 comp rx=0 tx=0).ready entity=mgr.24923 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:12:01.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.778+0000 7f826a228740 1 --2- 172.21.15.47:0/2667109905 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5609eda449e0 0x5609eda4d270 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f82580efd10 tx=0x7f82580e9730 comp rx=0 tx=0).stop 2024-04-17T06:12:01.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.778+0000 7f826a228740 1 -- 172.21.15.47:0/2667109905 shutdown_connections 2024-04-17T06:12:01.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.778+0000 7f826a228740 1 --2- 172.21.15.47:0/2667109905 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5609eda4d7b0 0x5609eda4fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:01.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.778+0000 7f826a228740 1 --2- 172.21.15.47:0/2667109905 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5609eda449e0 0x5609eda4d270 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:01.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.778+0000 7f826a228740 1 --2- 172.21.15.47:0/2667109905 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5609eda440c0 0x5609eda444a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:01.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.778+0000 7f826a228740 1 -- 172.21.15.47:0/2667109905 >> 172.21.15.47:0/2667109905 conn(0x5609eda3e9e0 msgr2=0x5609eda3edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:12:01.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.779+0000 7f826a228740 1 -- 172.21.15.47:0/2667109905 shutdown_connections 2024-04-17T06:12:01.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.779+0000 7f826a228740 1 -- 172.21.15.47:0/2667109905 wait complete. 2024-04-17T06:12:01.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.779+0000 7f826a228740 1 -- start start 2024-04-17T06:12:01.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.779+0000 7f826a228740 1 Processor -- start 2024-04-17T06:12:01.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.779+0000 7f826a228740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5609eda440c0 0x5609eda4a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:01.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.779+0000 7f826a228740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5609eda449e0 0x5609eda4acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:01.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.779+0000 7f826a228740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5609eda4d7b0 0x5609eda4b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:01.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.779+0000 7f826a228740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5609ed9a97d0 con 0x5609eda440c0 2024-04-17T06:12:01.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.779+0000 7f826a228740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5609ed931e90 con 0x5609eda449e0 2024-04-17T06:12:01.785 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.779+0000 7f826a228740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5609eda501a0 con 0x5609eda4d7b0 2024-04-17T06:12:01.785 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.780+0000 7f826a228740 10 client.0 _ll_register_callbacks cb 0x5609ed947d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:12:01.785 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.780+0000 7f82653d1640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5609eda440c0 0x5609eda4a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:12:01.785 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.780+0000 7f82653d1640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5609eda440c0 0x5609eda4a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44186/0 (socket says 172.21.15.47:44186) 2024-04-17T06:12:01.785 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.780+0000 7f82653d1640 1 -- 172.21.15.47:0/3357499478 learned_addr learned my addr 172.21.15.47:0/3357499478 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:12:01.785 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.780+0000 7f8265bd2640 1 --2- 172.21.15.47:0/3357499478 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5609eda4d7b0 0x5609eda4b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:12:01.785 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.780+0000 7f8264bd0640 1 --2- 172.21.15.47:0/3357499478 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5609eda449e0 0x5609eda4acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:12:01.785 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.780+0000 7f82653d1640 1 -- 172.21.15.47:0/3357499478 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5609eda4d7b0 msgr2=0x5609eda4b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:12:01.785 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.780+0000 7f82653d1640 1 --2- 172.21.15.47:0/3357499478 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5609eda4d7b0 0x5609eda4b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:01.785 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.780+0000 7f82653d1640 1 -- 172.21.15.47:0/3357499478 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5609eda449e0 msgr2=0x5609eda4acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:12:01.785 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.780+0000 7f82653d1640 1 --2- 172.21.15.47:0/3357499478 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5609eda449e0 0x5609eda4acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:01.785 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.780+0000 7f82653d1640 1 -- 172.21.15.47:0/3357499478 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609ed788620 con 0x5609eda440c0 2024-04-17T06:12:01.785 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.780+0000 7f8264bd0640 1 --2- 172.21.15.47:0/3357499478 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5609eda449e0 0x5609eda4acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:12:01.785 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.780+0000 7f82653d1640 1 --2- 172.21.15.47:0/3357499478 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5609eda440c0 0x5609eda4a780 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f82580baa80 tx=0x7f82580057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffb8c5fc78bd48ee server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:12:01.786 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.780+0000 7f8244ff9640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-17T06:12:01.786 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.780+0000 7f8244ff9640 1 -- 172.21.15.47:0/3357499478 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82580be880 con 0x5609eda440c0 2024-04-17T06:12:01.786 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.780+0000 7f8244ff9640 1 -- 172.21.15.47:0/3357499478 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8513+0+0 (secure 0 0 0) 0x7f82580d2d80 con 0x5609eda440c0 2024-04-17T06:12:01.786 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.780+0000 7f826a228740 1 -- 172.21.15.47:0/3357499478 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8258004c80 con 0x5609eda440c0 2024-04-17T06:12:01.786 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.780+0000 7f8244ff9640 1 -- 172.21.15.47:0/3357499478 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f82580bf950 con 0x5609eda440c0 2024-04-17T06:12:01.786 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.780+0000 7f826a228740 10 client.16293 fetch_fsmap learned FSMap version 34 2024-04-17T06:12:01.786 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.781+0000 7f826a228740 1 -- 172.21.15.47:0/3357499478 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5609eda5c220 con 0x5609eda440c0 2024-04-17T06:12:01.786 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.781+0000 7f8244ff9640 1 -- 172.21.15.47:0/3357499478 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8258004c80 con 0x5609eda440c0 2024-04-17T06:12:01.786 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.781+0000 7f8244ff9640 1 -- 172.21.15.47:0/3357499478 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 34) ==== 8339+0+0 (secure 0 0 0) 0x7f82580bac90 con 0x5609eda440c0 2024-04-17T06:12:01.786 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.781+0000 7f826a228740 10 client.16293 fetch_fsmap finished waiting for FSMap version 34 2024-04-17T06:12:01.786 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.781+0000 7f826a228740 10 client.16293 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-17T06:12:01.786 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.781+0000 7f826a228740 20 client.16293 populate_metadata read hostname 'smithi047' 2024-04-17T06:12:01.786 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.781+0000 7f826a228740 1 --2- 172.21.15.47:0/3357499478 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x5609eda5d570 0x5609eda5f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:01.786 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.781+0000 7f826a228740 4 client.16293 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T06:12:01.787 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.781+0000 7f826a228740 1 -- 172.21.15.47:0/3357499478 --> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5609ed9b71f0 con 0x5609eda5d570 2024-04-17T06:12:01.787 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.781+0000 7f8264bd0640 1 --2- 172.21.15.47:0/3357499478 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x5609eda5d570 0x5609eda5f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:12:01.787 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.782+0000 7f8264bd0640 1 --2- 172.21.15.47:0/3357499478 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x5609eda5d570 0x5609eda5f960 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a69ecf976e2db6d server_cookie=47b787472547bfaf in_seq=0 out_seq=0 2024-04-17T06:12:01.787 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.782+0000 7f8244ff9640 10 client.16293 ms_handle_connect on v2:172.21.15.98:6838/2841287917 2024-04-17T06:12:01.787 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.782+0000 7f8244ff9640 1 -- 172.21.15.47:0/3357499478 <== mds.0 v2:172.21.15.98:6838/2841287917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5609ed9b71f0 con 0x5609eda5d570 2024-04-17T06:12:01.787 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.782+0000 7f8244ff9640 10 client.16293 handle_command_reply: tid=0 2024-04-17T06:12:01.787 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.782+0000 7f826a228740 1 client.16293 shutdown 2024-04-17T06:12:01.787 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.783+0000 7f826a228740 1 -- 172.21.15.47:0/3357499478 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5609eda440c0 msgr2=0x5609eda4a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:12:01.787 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.783+0000 7f826a228740 1 --2- 172.21.15.47:0/3357499478 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5609eda440c0 0x5609eda4a780 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f82580baa80 tx=0x7f82580057b0 comp rx=0 tx=0).stop 2024-04-17T06:12:01.787 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.783+0000 7f826a228740 1 -- 172.21.15.47:0/3357499478 shutdown_connections 2024-04-17T06:12:01.787 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.783+0000 7f826a228740 1 --2- 172.21.15.47:0/3357499478 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x5609eda5d570 0x5609eda5f960 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:01.787 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.783+0000 7f826a228740 1 --2- 172.21.15.47:0/3357499478 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5609eda4d7b0 0x5609eda4b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:01.787 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.783+0000 7f826a228740 1 --2- 172.21.15.47:0/3357499478 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5609eda449e0 0x5609eda4acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:01.787 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.783+0000 7f8244ff9640 0 client.16293 ms_handle_reset on v2:172.21.15.98:6838/2841287917 2024-04-17T06:12:01.787 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.783+0000 7f826a228740 1 --2- 172.21.15.47:0/3357499478 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5609eda440c0 0x5609eda4a780 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:01.787 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.783+0000 7f826a228740 1 -- 172.21.15.47:0/3357499478 >> 172.21.15.47:0/3357499478 conn(0x5609eda3e9e0 msgr2=0x5609eda40730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:12:01.788 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.783+0000 7f826a228740 1 -- 172.21.15.47:0/3357499478 shutdown_connections 2024-04-17T06:12:01.788 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.783+0000 7f826a228740 1 -- 172.21.15.47:0/3357499478 wait complete. 2024-04-17T06:12:01.788 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.783+0000 7f826a228740 20 client.16293 trim_cache size 0 max 16384 2024-04-17T06:12:01.801 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.798+0000 7f826a228740 1 Processor -- start 2024-04-17T06:12:01.801 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.798+0000 7f826a228740 1 -- start start 2024-04-17T06:12:01.801 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.798+0000 7f826a228740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5609eda440c0 0x5609eda4ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:01.801 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.798+0000 7f826a228740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5609eda449e0 0x5609eda58ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:01.801 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.798+0000 7f826a228740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5609eda4d7b0 0x5609eda40540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:01.801 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.798+0000 7f826a228740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f82601b6b10 con 0x5609eda4d7b0 2024-04-17T06:12:01.801 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.798+0000 7f826a228740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f82601b43d0 con 0x5609eda440c0 2024-04-17T06:12:01.802 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.798+0000 7f826a228740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f82601b2240 con 0x5609eda449e0 2024-04-17T06:12:01.802 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.798+0000 7f8265bd2640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5609eda4d7b0 0x5609eda40540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:12:01.802 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.798+0000 7f8265bd2640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5609eda4d7b0 0x5609eda40540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44196/0 (socket says 172.21.15.47:44196) 2024-04-17T06:12:01.802 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.798+0000 7f8265bd2640 1 -- 172.21.15.47:0/2861812371 learned_addr learned my addr 172.21.15.47:0/2861812371 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:12:01.802 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.799+0000 7f82653d1640 1 --2- 172.21.15.47:0/2861812371 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5609eda440c0 0x5609eda4ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:12:01.802 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.799+0000 7f8264bd0640 1 --2- 172.21.15.47:0/2861812371 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5609eda449e0 0x5609eda58ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:12:01.802 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.799+0000 7f8265bd2640 1 -- 172.21.15.47:0/2861812371 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5609eda449e0 msgr2=0x5609eda58ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:12:01.802 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.799+0000 7f8265bd2640 1 --2- 172.21.15.47:0/2861812371 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5609eda449e0 0x5609eda58ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:01.802 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.799+0000 7f8265bd2640 1 -- 172.21.15.47:0/2861812371 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5609eda440c0 msgr2=0x5609eda4ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:12:01.802 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.799+0000 7f8265bd2640 1 --2- 172.21.15.47:0/2861812371 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5609eda440c0 0x5609eda4ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:01.802 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.799+0000 7f8265bd2640 1 -- 172.21.15.47:0/2861812371 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609eda44810 con 0x5609eda4d7b0 2024-04-17T06:12:01.803 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.799+0000 7f8265bd2640 1 --2- 172.21.15.47:0/2861812371 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5609eda4d7b0 0x5609eda40540 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f825001c610 tx=0x7f8250015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a5b87db34fc5db4 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:12:01.803 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.799+0000 7f8246ffd640 1 -- 172.21.15.47:0/2861812371 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8250011a90 con 0x5609eda4d7b0 2024-04-17T06:12:01.803 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.799+0000 7f8246ffd640 1 -- 172.21.15.47:0/2861812371 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8250011c30 con 0x5609eda4d7b0 2024-04-17T06:12:01.803 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.799+0000 7f8246ffd640 1 -- 172.21.15.47:0/2861812371 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8250019790 con 0x5609eda4d7b0 2024-04-17T06:12:01.803 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.800+0000 7f826a228740 1 -- 172.21.15.47:0/2861812371 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5609eda4d7b0 msgr2=0x5609eda40540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:12:01.803 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.800+0000 7f826a228740 1 --2- 172.21.15.47:0/2861812371 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5609eda4d7b0 0x5609eda40540 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f825001c610 tx=0x7f8250015000 comp rx=0 tx=0).stop 2024-04-17T06:12:01.803 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.800+0000 7f826a228740 1 -- 172.21.15.47:0/2861812371 shutdown_connections 2024-04-17T06:12:01.803 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.800+0000 7f826a228740 1 --2- 172.21.15.47:0/2861812371 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5609eda4d7b0 0x5609eda40540 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:01.803 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.800+0000 7f826a228740 1 --2- 172.21.15.47:0/2861812371 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5609eda449e0 0x5609eda58ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:01.803 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.800+0000 7f826a228740 1 --2- 172.21.15.47:0/2861812371 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5609eda440c0 0x5609eda4ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:01.803 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.800+0000 7f826a228740 1 -- 172.21.15.47:0/2861812371 >> 172.21.15.47:0/2861812371 conn(0x5609eda3e9e0 msgr2=0x5609eda3fe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:12:01.803 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.800+0000 7f826a228740 1 -- 172.21.15.47:0/2861812371 shutdown_connections 2024-04-17T06:12:01.803 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.800+0000 7f826a228740 1 -- 172.21.15.47:0/2861812371 wait complete. 2024-04-17T06:12:01.803 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.800+0000 7f826a228740 1 -- start start 2024-04-17T06:12:01.804 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.800+0000 7f826a228740 1 Processor -- start 2024-04-17T06:12:01.804 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.800+0000 7f826a228740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5609eda440c0 0x5609eda55890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:01.804 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.801+0000 7f826a228740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5609eda449e0 0x5609eda50c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:01.804 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.801+0000 7f826a228740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5609eda4d7b0 0x5609eda51210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:01.804 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.801+0000 7f826a228740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f82601b2890 con 0x5609eda440c0 2024-04-17T06:12:01.804 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.801+0000 7f826a228740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f82601b3d00 con 0x5609eda449e0 2024-04-17T06:12:01.804 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.801+0000 7f826a228740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f82601b2f60 con 0x5609eda4d7b0 2024-04-17T06:12:01.804 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.801+0000 7f826a228740 10 client.0 _ll_register_callbacks cb 0x7f826006b7e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:12:01.804 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.801+0000 7f82653d1640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5609eda440c0 0x5609eda55890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:12:01.804 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.801+0000 7f82653d1640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5609eda440c0 0x5609eda55890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44200/0 (socket says 172.21.15.47:44200) 2024-04-17T06:12:01.804 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.801+0000 7f82653d1640 1 -- 172.21.15.47:0/2924764739 learned_addr learned my addr 172.21.15.47:0/2924764739 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:12:01.804 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.801+0000 7f8264bd0640 1 --2- 172.21.15.47:0/2924764739 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5609eda449e0 0x5609eda50c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:12:01.804 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.801+0000 7f8265bd2640 1 --2- 172.21.15.47:0/2924764739 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5609eda4d7b0 0x5609eda51210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:12:01.805 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.801+0000 7f82653d1640 1 -- 172.21.15.47:0/2924764739 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5609eda4d7b0 msgr2=0x5609eda51210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:12:01.805 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.801+0000 7f82653d1640 1 --2- 172.21.15.47:0/2924764739 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5609eda4d7b0 0x5609eda51210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:01.805 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.801+0000 7f82653d1640 1 -- 172.21.15.47:0/2924764739 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5609eda449e0 msgr2=0x5609eda50c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:12:01.805 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.801+0000 7f82653d1640 1 --2- 172.21.15.47:0/2924764739 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5609eda449e0 0x5609eda50c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:01.805 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.801+0000 7f82653d1640 1 -- 172.21.15.47:0/2924764739 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609eda519e0 con 0x5609eda440c0 2024-04-17T06:12:01.805 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.801+0000 7f82653d1640 1 --2- 172.21.15.47:0/2924764739 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5609eda440c0 0x5609eda55890 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f82580bec80 tx=0x7f82580035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f50282991b91d8ae server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:12:01.805 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.801+0000 7f8244ff9640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-17T06:12:01.805 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.801+0000 7f8244ff9640 1 -- 172.21.15.47:0/2924764739 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82580bdf80 con 0x5609eda440c0 2024-04-17T06:12:01.805 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.801+0000 7f8244ff9640 1 -- 172.21.15.47:0/2924764739 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8513+0+0 (secure 0 0 0) 0x7f82580c6e00 con 0x5609eda440c0 2024-04-17T06:12:01.805 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.801+0000 7f826a228740 1 -- 172.21.15.47:0/2924764739 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8250011c30 con 0x5609eda440c0 2024-04-17T06:12:01.805 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.801+0000 7f8244ff9640 1 -- 172.21.15.47:0/2924764739 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f82580be160 con 0x5609eda440c0 2024-04-17T06:12:01.805 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.801+0000 7f826a228740 10 client.16299 fetch_fsmap learned FSMap version 34 2024-04-17T06:12:01.805 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.801+0000 7f826a228740 1 -- 172.21.15.47:0/2924764739 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5609eda519e0 con 0x5609eda440c0 2024-04-17T06:12:01.805 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.802+0000 7f8244ff9640 1 -- 172.21.15.47:0/2924764739 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8250011c30 con 0x5609eda440c0 2024-04-17T06:12:01.806 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.802+0000 7f8244ff9640 1 -- 172.21.15.47:0/2924764739 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 34) ==== 8339+0+0 (secure 0 0 0) 0x7f82580efeb0 con 0x5609eda440c0 2024-04-17T06:12:01.806 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.802+0000 7f826a228740 10 client.16299 fetch_fsmap finished waiting for FSMap version 34 2024-04-17T06:12:01.806 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.802+0000 7f826a228740 10 client.16299 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-17T06:12:01.806 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.802+0000 7f826a228740 20 client.16299 populate_metadata read hostname 'smithi047' 2024-04-17T06:12:01.806 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.802+0000 7f826a228740 1 --2- 172.21.15.47:0/2924764739 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x5609edac3eb0 0x5609edac62a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:01.806 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.802+0000 7f826a228740 4 client.16299 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-17T06:12:01.806 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.802+0000 7f826a228740 1 -- 172.21.15.47:0/2924764739 --> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5609ed9b1060 con 0x5609edac3eb0 2024-04-17T06:12:01.806 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.802+0000 7f8264bd0640 1 --2- 172.21.15.47:0/2924764739 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x5609edac3eb0 0x5609edac62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:12:01.806 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.802+0000 7f8264bd0640 1 --2- 172.21.15.47:0/2924764739 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x5609edac3eb0 0x5609edac62a0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4d7f144a6e89c0 server_cookie=24d64792cc9150ee in_seq=0 out_seq=0 2024-04-17T06:12:01.806 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:01.802+0000 7f8244ff9640 10 client.16299 ms_handle_connect on v2:172.21.15.98:6838/2841287917 2024-04-17T06:12:02.085 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:12:01 smithi103 ceph-mon[36243]: pgmap v336: 97 pgs: 97 active+clean; 5.3 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-04-17T06:12:02.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:12:01 smithi098 ceph-mon[36516]: pgmap v336: 97 pgs: 97 active+clean; 5.3 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-04-17T06:12:02.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:12:01 smithi047 ceph-mon[32223]: pgmap v336: 97 pgs: 97 active+clean; 5.3 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-04-17T06:12:03.538 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:03.539+0000 7f8244ff9640 1 -- 172.21.15.47:0/2924764739 <== mds.0 v2:172.21.15.98:6838/2841287917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5609ed9b1060 con 0x5609edac3eb0 2024-04-17T06:12:03.538 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:03.539+0000 7f8244ff9640 10 client.16299 handle_command_reply: tid=0 2024-04-17T06:12:03.538 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:03.539+0000 7f826a228740 1 client.16299 shutdown 2024-04-17T06:12:03.538 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:03.539+0000 7f826a228740 1 -- 172.21.15.47:0/2924764739 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5609eda440c0 msgr2=0x5609eda55890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:12:03.538 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:03.539+0000 7f826a228740 1 --2- 172.21.15.47:0/2924764739 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5609eda440c0 0x5609eda55890 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f82580bec80 tx=0x7f82580035f0 comp rx=0 tx=0).stop 2024-04-17T06:12:03.538 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:03.539+0000 7f826a228740 1 -- 172.21.15.47:0/2924764739 shutdown_connections 2024-04-17T06:12:03.538 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:03.539+0000 7f826a228740 1 --2- 172.21.15.47:0/2924764739 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x5609edac3eb0 0x5609edac62a0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:03.538 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:03.539+0000 7f826a228740 1 --2- 172.21.15.47:0/2924764739 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5609eda4d7b0 0x5609eda51210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:03.538 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:03.539+0000 7f826a228740 1 --2- 172.21.15.47:0/2924764739 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5609eda449e0 0x5609eda50c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:03.538 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:03.539+0000 7f826a228740 1 --2- 172.21.15.47:0/2924764739 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5609eda440c0 0x5609eda55890 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:03.539 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:03.539+0000 7f8244ff9640 0 client.16299 ms_handle_reset on v2:172.21.15.98:6838/2841287917 2024-04-17T06:12:03.539 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:03.539+0000 7f826a228740 1 -- 172.21.15.47:0/2924764739 >> 172.21.15.47:0/2924764739 conn(0x5609eda3e9e0 msgr2=0x5609eda406c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:12:03.539 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:03.539+0000 7f826a228740 1 -- 172.21.15.47:0/2924764739 shutdown_connections 2024-04-17T06:12:03.539 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:03.539+0000 7f826a228740 1 -- 172.21.15.47:0/2924764739 wait complete. 2024-04-17T06:12:03.539 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:03.539+0000 7f826a228740 20 client.16299 trim_cache size 0 max 16384 2024-04-17T06:12:03.539 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-04-17T06:12:03.539 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-04-17T06:12:03.539 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-04-17T06:12:03.539 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-04-17T06:12:03.540 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:03.540+0000 7f8267635640 1 -- 172.21.15.47:0/233560735 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f82480a3dc0 msgr2=0x7f82480a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:12:03.540 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:03.540+0000 7f8267635640 1 --2- 172.21.15.47:0/233560735 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f82480a3dc0 0x7f82480a6280 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8250005cc0 tx=0x7f8250004a40 comp rx=0 tx=0).stop 2024-04-17T06:12:03.540 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:03.540+0000 7f8267635640 1 -- 172.21.15.47:0/233560735 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8260074260 msgr2=0x7f826019ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:12:03.540 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:03.540+0000 7f8267635640 1 --2- 172.21.15.47:0/233560735 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8260074260 0x7f826019ed60 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f8258006010 tx=0x7f82580043b0 comp rx=0 tx=0).stop 2024-04-17T06:12:03.540 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:03.540+0000 7f8267635640 1 -- 172.21.15.47:0/233560735 shutdown_connections 2024-04-17T06:12:03.541 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:03.540+0000 7f8267635640 1 --2- 172.21.15.47:0/233560735 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8260113020 0x7f82601a3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:03.541 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:03.540+0000 7f8267635640 1 --2- 172.21.15.47:0/233560735 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8260074b80 0x7f826019f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:03.541 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:03.540+0000 7f8267635640 1 --2- 172.21.15.47:0/233560735 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f82480a3dc0 0x7f82480a6280 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:03.541 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:03.540+0000 7f8267635640 1 --2- 172.21.15.47:0/233560735 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8260074260 0x7f826019ed60 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:03.541 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:03.540+0000 7f8267635640 1 -- 172.21.15.47:0/233560735 >> 172.21.15.47:0/233560735 conn(0x7f82600793f0 msgr2=0x7f82600fed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:12:03.541 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:03.540+0000 7f8267635640 1 -- 172.21.15.47:0/233560735 shutdown_connections 2024-04-17T06:12:03.541 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:03.540+0000 7f8267635640 1 -- 172.21.15.47:0/233560735 wait complete. 2024-04-17T06:12:04.086 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:12:03 smithi098 ceph-mon[36516]: pgmap v337: 97 pgs: 97 active+clean; 5.3 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.6 KiB/s wr, 9 op/s 2024-04-17T06:12:04.086 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:12:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[72223]: *** Caught signal (Segmentation fault) ** 2024-04-17T06:12:04.087 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:12:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[72223]: in thread 7f2771657700 thread_name:md_log_replay 2024-04-17T06:12:04.087 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:12:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[72223]: ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev) 2024-04-17T06:12:04.087 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:12:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[72223]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f2782a18d20] 2024-04-17T06:12:04.087 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:12:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[72223]: 2: /usr/bin/ceph-mds(+0x54962f) [0x55c94416962f] 2024-04-17T06:12:04.087 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:12:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[72223]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55c94416cdc3] 2024-04-17T06:12:04.087 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:12:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[72223]: 4: (MDLog::_replay_thread()+0x617) [0x55c9441c78e7] 2024-04-17T06:12:04.087 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:12:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[72223]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55c9441d0fd1] 2024-04-17T06:12:04.087 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:12:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[72223]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f2782a0e1ca] 2024-04-17T06:12:04.087 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:12:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[72223]: 7: clone() 2024-04-17T06:12:04.087 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:12:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[72223]: 2024-04-17T06:12:04.035+0000 7f2771657700 -1 *** Caught signal (Segmentation fault) ** 2024-04-17T06:12:04.088 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:12:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[72223]: in thread 7f2771657700 thread_name:md_log_replay 2024-04-17T06:12:04.088 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:12:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[72223]: 2024-04-17T06:12:04.088 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:12:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[72223]: ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev) 2024-04-17T06:12:04.088 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:12:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[72223]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f2782a18d20] 2024-04-17T06:12:04.088 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:12:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[72223]: 2: /usr/bin/ceph-mds(+0x54962f) [0x55c94416962f] 2024-04-17T06:12:04.088 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:12:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[72223]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55c94416cdc3] 2024-04-17T06:12:04.088 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:12:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[72223]: 4: (MDLog::_replay_thread()+0x617) [0x55c9441c78e7] 2024-04-17T06:12:04.088 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:12:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[72223]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55c9441d0fd1] 2024-04-17T06:12:04.088 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:12:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[72223]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f2782a0e1ca] 2024-04-17T06:12:04.088 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:12:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[72223]: 7: clone() 2024-04-17T06:12:04.088 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:12:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[72223]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-17T06:12:04.088 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:12:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[72223]: 2024-04-17T06:12:04.088 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:12:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[72223]: 0> 2024-04-17T06:12:04.035+0000 7f2771657700 -1 *** Caught signal (Segmentation fault) ** 2024-04-17T06:12:04.088 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:12:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[72223]: in thread 7f2771657700 thread_name:md_log_replay 2024-04-17T06:12:04.089 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:12:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[72223]: 2024-04-17T06:12:04.089 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:12:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[72223]: ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev) 2024-04-17T06:12:04.089 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:12:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[72223]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f2782a18d20] 2024-04-17T06:12:04.089 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:12:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[72223]: 2: /usr/bin/ceph-mds(+0x54962f) [0x55c94416962f] 2024-04-17T06:12:04.089 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:12:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[72223]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55c94416cdc3] 2024-04-17T06:12:04.089 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:12:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[72223]: 4: (MDLog::_replay_thread()+0x617) [0x55c9441c78e7] 2024-04-17T06:12:04.089 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:12:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[72223]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55c9441d0fd1] 2024-04-17T06:12:04.089 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:12:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[72223]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f2782a0e1ca] 2024-04-17T06:12:04.089 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:12:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[72223]: 7: clone() 2024-04-17T06:12:04.089 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:12:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[72223]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-17T06:12:04.089 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:12:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[72223]: 2024-04-17T06:12:04.089 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:12:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[72223]: 0> 2024-04-17T06:12:04.035+0000 7f2771657700 -1 *** Caught signal (Segmentation fault) ** 2024-04-17T06:12:04.089 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:12:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[72223]: in thread 7f2771657700 thread_name:md_log_replay 2024-04-17T06:12:04.090 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:12:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[72223]: 2024-04-17T06:12:04.090 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:12:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[72223]: ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev) 2024-04-17T06:12:04.090 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:12:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[72223]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f2782a18d20] 2024-04-17T06:12:04.090 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:12:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[72223]: 2: /usr/bin/ceph-mds(+0x54962f) [0x55c94416962f] 2024-04-17T06:12:04.090 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:12:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[72223]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55c94416cdc3] 2024-04-17T06:12:04.090 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:12:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[72223]: 4: (MDLog::_replay_thread()+0x617) [0x55c9441c78e7] 2024-04-17T06:12:04.090 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:12:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[72223]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55c9441d0fd1] 2024-04-17T06:12:04.090 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:12:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[72223]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f2782a0e1ca] 2024-04-17T06:12:04.090 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:12:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[72223]: 7: clone() 2024-04-17T06:12:04.090 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:12:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[72223]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-17T06:12:04.090 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:12:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[72223]: 2024-04-17T06:12:04.090 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:12:04 smithi098 podman[72903]: 2024-04-17 06:12:04.08800638 +0000 UTC m=+0.024389712 container died b4076f002011592a4842588b1e92ed80dbabe30471f18396a3ad1e5e3aa7912e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-17T06:12:04.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:12:03 smithi047 ceph-mon[32223]: pgmap v337: 97 pgs: 97 active+clean; 5.3 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.6 KiB/s wr, 9 op/s 2024-04-17T06:12:04.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:12:03 smithi103 ceph-mon[36243]: pgmap v337: 97 pgs: 97 active+clean; 5.3 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.6 KiB/s wr, 9 op/s 2024-04-17T06:12:05.035 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:12:04 smithi098 ceph-mon[36516]: osdmap e102: 12 total, 12 up, 12 in 2024-04-17T06:12:05.036 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:12:04 smithi098 podman[72903]: 2024-04-17 06:12:04.787553653 +0000 UTC m=+0.723937010 container remove b4076f002011592a4842588b1e92ed80dbabe30471f18396a3ad1e5e3aa7912e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-04-17T06:12:05.036 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:12:04 smithi098 systemd[1]: ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@mds.k.service: Main process exited, code=exited, status=139/n/a 2024-04-17T06:12:05.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:12:04 smithi047 ceph-mon[32223]: osdmap e102: 12 total, 12 up, 12 in 2024-04-17T06:12:05.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:12:04 smithi103 ceph-mon[36243]: osdmap e102: 12 total, 12 up, 12 in 2024-04-17T06:12:05.383 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:12:05 smithi098 systemd[1]: ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@mds.k.service: Failed with result 'exit-code'. 2024-04-17T06:12:06.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:12:05 smithi098 ceph-mon[36516]: pgmap v339: 97 pgs: 97 active+clean; 5.4 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 68 KiB/s wr, 34 op/s 2024-04-17T06:12:06.142 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:12:05 smithi047 ceph-mon[32223]: pgmap v339: 97 pgs: 97 active+clean; 5.4 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 68 KiB/s wr, 34 op/s 2024-04-17T06:12:06.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:12:05 smithi103 ceph-mon[36243]: pgmap v339: 97 pgs: 97 active+clean; 5.4 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 68 KiB/s wr, 34 op/s 2024-04-17T06:12:08.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:12:07 smithi098 ceph-mon[36516]: pgmap v340: 97 pgs: 97 active+clean; 5.4 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 68 KiB/s wr, 34 op/s 2024-04-17T06:12:08.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:12:07 smithi047 ceph-mon[32223]: pgmap v340: 97 pgs: 97 active+clean; 5.4 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 68 KiB/s wr, 34 op/s 2024-04-17T06:12:08.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:12:07 smithi103 ceph-mon[36243]: pgmap v340: 97 pgs: 97 active+clean; 5.4 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 68 KiB/s wr, 34 op/s 2024-04-17T06:12:10.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:12:09 smithi098 ceph-mon[36516]: pgmap v341: 97 pgs: 97 active+clean; 5.4 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 68 KiB/s wr, 34 op/s 2024-04-17T06:12:10.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:12:09 smithi098 ceph-mon[36516]: osdmap e103: 12 total, 12 up, 12 in 2024-04-17T06:12:10.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:12:09 smithi047 ceph-mon[32223]: pgmap v341: 97 pgs: 97 active+clean; 5.4 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 68 KiB/s wr, 34 op/s 2024-04-17T06:12:10.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:12:09 smithi047 ceph-mon[32223]: osdmap e103: 12 total, 12 up, 12 in 2024-04-17T06:12:10.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:12:09 smithi103 ceph-mon[36243]: pgmap v341: 97 pgs: 97 active+clean; 5.4 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 68 KiB/s wr, 34 op/s 2024-04-17T06:12:10.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:12:09 smithi103 ceph-mon[36243]: osdmap e103: 12 total, 12 up, 12 in 2024-04-17T06:12:12.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:12:11 smithi098 ceph-mon[36516]: pgmap v343: 97 pgs: 97 active+clean; 5.4 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 85 KiB/s wr, 42 op/s 2024-04-17T06:12:12.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:12:11 smithi047 ceph-mon[32223]: pgmap v343: 97 pgs: 97 active+clean; 5.4 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 85 KiB/s wr, 42 op/s 2024-04-17T06:12:12.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:12:11 smithi103 ceph-mon[36243]: pgmap v343: 97 pgs: 97 active+clean; 5.4 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 85 KiB/s wr, 42 op/s 2024-04-17T06:12:14.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:12:13 smithi098 ceph-mon[36516]: pgmap v344: 97 pgs: 97 active+clean; 5.4 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 79 KiB/s wr, 39 op/s 2024-04-17T06:12:14.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:12:13 smithi047 ceph-mon[32223]: pgmap v344: 97 pgs: 97 active+clean; 5.4 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 79 KiB/s wr, 39 op/s 2024-04-17T06:12:14.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:12:13 smithi103 ceph-mon[36243]: pgmap v344: 97 pgs: 97 active+clean; 5.4 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 79 KiB/s wr, 39 op/s 2024-04-17T06:12:15.428 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:12:15 smithi098 systemd[1]: ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@mds.k.service: Scheduled restart job, restart counter is at 3. 2024-04-17T06:12:15.428 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:12:15 smithi098 systemd[1]: Stopped Ceph mds.k for 565db2d8-fc7e-11ee-bc91-c7b262605968. 2024-04-17T06:12:15.428 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:12:15 smithi098 systemd[1]: Starting Ceph mds.k for 565db2d8-fc7e-11ee-bc91-c7b262605968... 2024-04-17T06:12:15.428 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:12:15 smithi098 podman[73005]: 2024-04-17 06:12:15.271668858 +0000 UTC m=+0.018063452 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-17T06:12:15.681 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:12:15 smithi098 podman[73005]: 2024-04-17 06:12:15.430431786 +0000 UTC m=+0.176826355 container create 49fcadeeae1a3608b497973c8cfdf03c4da9def26e9c8779af7839fb67df4f9f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-04-17T06:12:15.681 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:12:15 smithi098 podman[73005]: 2024-04-17 06:12:15.658466478 +0000 UTC m=+0.404861052 container init 49fcadeeae1a3608b497973c8cfdf03c4da9def26e9c8779af7839fb67df4f9f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-17T06:12:15.681 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:12:15 smithi098 podman[73005]: 2024-04-17 06:12:15.661077447 +0000 UTC m=+0.407472035 container start 49fcadeeae1a3608b497973c8cfdf03c4da9def26e9c8779af7839fb67df4f9f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, ceph=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-17T06:12:15.996 INFO:tasks.workunit.client.0.smithi047.stdout:499/600: open fd = 3 2024-04-17T06:12:15.996 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:15.996 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:15.996 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:12:15.996 INFO:tasks.workunit.client.0.smithi047.stdout:500/600: open fd = 3 2024-04-17T06:12:15.996 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:15.996 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:15.996 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:12:15.996 INFO:tasks.workunit.client.0.smithi047.stdout:501/600: open fd = 3 2024-04-17T06:12:15.996 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:15.996 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:15.997 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:12:15.997 INFO:tasks.workunit.client.0.smithi047.stdout:502/600: open fd = 3 2024-04-17T06:12:15.997 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:15.997 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:15.997 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:12:15.997 INFO:tasks.workunit.client.0.smithi047.stdout:503/600: open fd = 3 2024-04-17T06:12:15.997 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:15.997 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:15.997 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:12:15.997 INFO:tasks.workunit.client.0.smithi047.stdout:504/600: open fd = 3 2024-04-17T06:12:15.997 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:15.997 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:15.997 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:12:15.997 INFO:tasks.workunit.client.0.smithi047.stdout:505/600: open fd = 3 2024-04-17T06:12:15.997 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:15.998 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:15.998 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:12:15.998 INFO:tasks.workunit.client.0.smithi047.stdout:506/600: open fd = 3 2024-04-17T06:12:15.998 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:15.998 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:15.998 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:12:15.998 INFO:tasks.workunit.client.0.smithi047.stdout:507/600: open fd = 3 2024-04-17T06:12:15.998 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:15.998 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:15.998 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:12:15.998 INFO:tasks.workunit.client.0.smithi047.stdout:508/600: open fd = 3 2024-04-17T06:12:15.998 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:15.998 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:15.998 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:12:15.999 INFO:tasks.workunit.client.0.smithi047.stdout:509/600: open fd = 3 2024-04-17T06:12:15.999 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:15.999 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:15.999 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:12:15.999 INFO:tasks.workunit.client.0.smithi047.stdout:510/600: open fd = 3 2024-04-17T06:12:15.999 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:15.999 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:15.999 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:12:15.999 INFO:tasks.workunit.client.0.smithi047.stdout:511/600: open fd = 3 2024-04-17T06:12:15.999 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:15.999 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:15.999 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:12:15.999 INFO:tasks.workunit.client.0.smithi047.stdout:512/600: open fd = 3 2024-04-17T06:12:15.999 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:16.000 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:16.000 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:12:16.000 INFO:tasks.workunit.client.0.smithi047.stdout:513/600: open fd = 3 2024-04-17T06:12:16.000 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:16.000 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:16.000 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:12:16.000 INFO:tasks.workunit.client.0.smithi047.stdout:514/600: open fd = 3 2024-04-17T06:12:16.000 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:16.000 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:16.000 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:12:16.000 INFO:tasks.workunit.client.0.smithi047.stdout:515/600: open fd = 3 2024-04-17T06:12:16.000 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:16.000 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:16.000 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:12:16.000 INFO:tasks.workunit.client.0.smithi047.stdout:516/600: open fd = 3 2024-04-17T06:12:16.001 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:16.001 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:16.001 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:12:16.001 INFO:tasks.workunit.client.0.smithi047.stdout:517/600: open fd = 3 2024-04-17T06:12:16.001 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:16.001 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:16.001 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:12:16.001 INFO:tasks.workunit.client.0.smithi047.stdout:518/600: open fd = 3 2024-04-17T06:12:16.001 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:16.001 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:16.001 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:12:16.001 INFO:tasks.workunit.client.0.smithi047.stdout:519/600: open fd = 3 2024-04-17T06:12:16.001 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:16.001 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:16.001 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:12:16.001 INFO:tasks.workunit.client.0.smithi047.stdout:520/600: open fd = 3 2024-04-17T06:12:16.002 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:16.002 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:16.002 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:12:16.002 INFO:tasks.workunit.client.0.smithi047.stdout:521/600: open fd = 3 2024-04-17T06:12:16.002 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:16.002 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:16.002 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:12:16.002 INFO:tasks.workunit.client.0.smithi047.stdout:522/600: open fd = 3 2024-04-17T06:12:16.002 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:16.002 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:16.002 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:12:16.002 INFO:tasks.workunit.client.0.smithi047.stdout:523/600: open fd = 3 2024-04-17T06:12:16.002 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:16.002 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:16.002 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:12:16.003 INFO:tasks.workunit.client.0.smithi047.stdout:524/600: open fd = 3 2024-04-17T06:12:16.003 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:16.003 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:16.003 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:12:16.003 INFO:tasks.workunit.client.0.smithi047.stdout:525/600: open fd = 3 2024-04-17T06:12:16.003 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:16.003 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:16.003 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:12:16.003 INFO:tasks.workunit.client.0.smithi047.stdout:526/600: open fd = 3 2024-04-17T06:12:16.003 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:16.003 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:16.003 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:12:16.003 INFO:tasks.workunit.client.0.smithi047.stdout:527/600: open fd = 3 2024-04-17T06:12:16.003 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:16.004 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:16.004 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:12:16.004 INFO:tasks.workunit.client.0.smithi047.stdout:528/600: open fd = 3 2024-04-17T06:12:16.004 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:16.004 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:16.004 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:12:16.004 INFO:tasks.workunit.client.0.smithi047.stdout:529/600: open fd = 3 2024-04-17T06:12:16.004 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:16.004 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:16.004 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:12:16.004 INFO:tasks.workunit.client.0.smithi047.stdout:530/600: open fd = 3 2024-04-17T06:12:16.004 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:16.004 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:16.004 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:12:16.005 INFO:tasks.workunit.client.0.smithi047.stdout:531/600: open fd = 3 2024-04-17T06:12:16.005 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:16.005 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:16.005 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:12:16.005 INFO:tasks.workunit.client.0.smithi047.stdout:532/600: open fd = 3 2024-04-17T06:12:16.005 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:16.005 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:16.005 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:12:16.005 INFO:tasks.workunit.client.0.smithi047.stdout:533/600: open fd = 3 2024-04-17T06:12:16.005 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:16.005 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:16.005 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:12:16.005 INFO:tasks.workunit.client.0.smithi047.stdout:534/600: open fd = 3 2024-04-17T06:12:16.005 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:16.005 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:16.005 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:12:16.006 INFO:tasks.workunit.client.0.smithi047.stdout:535/600: open fd = 3 2024-04-17T06:12:16.006 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:16.006 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:16.006 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:12:16.006 INFO:tasks.workunit.client.0.smithi047.stdout:536/600: open fd = 3 2024-04-17T06:12:16.006 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:16.006 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:16.006 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:12:16.006 INFO:tasks.workunit.client.0.smithi047.stdout:537/600: open fd = 3 2024-04-17T06:12:16.006 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:16.006 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:16.006 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:12:16.006 INFO:tasks.workunit.client.0.smithi047.stdout:538/600: open fd = 3 2024-04-17T06:12:16.006 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:16.007 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:16.007 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:12:16.007 INFO:tasks.workunit.client.0.smithi047.stdout:539/600: open fd = 3 2024-04-17T06:12:16.007 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:16.007 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:16.007 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:12:16.007 INFO:tasks.workunit.client.0.smithi047.stdout:540/600: open fd = 3 2024-04-17T06:12:16.007 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:16.007 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:16.007 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:12:16.007 INFO:tasks.workunit.client.0.smithi047.stdout:541/600: open fd = 3 2024-04-17T06:12:16.007 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:16.007 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:16.007 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:12:16.008 INFO:tasks.workunit.client.0.smithi047.stdout:542/600: open fd = 3 2024-04-17T06:12:16.008 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:16.008 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:16.008 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:12:16.008 INFO:tasks.workunit.client.0.smithi047.stdout:543/600: open fd = 3 2024-04-17T06:12:16.008 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:16.008 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:16.008 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:12:16.008 INFO:tasks.workunit.client.0.smithi047.stdout:544/600: open fd = 3 2024-04-17T06:12:16.008 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:16.008 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:16.008 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:12:16.008 INFO:tasks.workunit.client.0.smithi047.stdout:545/600: open fd = 3 2024-04-17T06:12:16.008 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:16.008 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:16.009 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:12:16.009 INFO:tasks.workunit.client.0.smithi047.stdout:546/600: open fd = 3 2024-04-17T06:12:16.009 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:16.009 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:16.009 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:12:16.009 INFO:tasks.workunit.client.0.smithi047.stdout:547/600: open fd = 3 2024-04-17T06:12:16.009 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:16.009 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:16.009 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:12:16.009 INFO:tasks.workunit.client.0.smithi047.stdout:548/600: open fd = 3 2024-04-17T06:12:16.009 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:16.009 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:16.009 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:12:16.009 INFO:tasks.workunit.client.0.smithi047.stdout:549/600: open fd = 3 2024-04-17T06:12:16.009 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:16.010 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:16.010 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:12:16.010 INFO:tasks.workunit.client.0.smithi047.stdout:550/600: open fd = 3 2024-04-17T06:12:16.010 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:16.010 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:16.010 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:12:16.010 INFO:tasks.workunit.client.0.smithi047.stdout:551/600: open fd = 3 2024-04-17T06:12:16.010 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:16.010 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:16.010 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:12:16.010 INFO:tasks.workunit.client.0.smithi047.stdout:552/600: open fd = 3 2024-04-17T06:12:16.010 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:16.010 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:16.010 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:12:16.011 INFO:tasks.workunit.client.0.smithi047.stdout:553/600: open fd = 3 2024-04-17T06:12:16.011 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:16.011 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:16.011 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:12:16.011 INFO:tasks.workunit.client.0.smithi047.stdout:554/600: open fd = 3 2024-04-17T06:12:16.011 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:16.011 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:16.011 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:12:16.011 INFO:tasks.workunit.client.0.smithi047.stdout:555/600: open fd = 3 2024-04-17T06:12:16.011 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:16.011 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:16.011 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:12:16.011 INFO:tasks.workunit.client.0.smithi047.stdout:556/600: open fd = 3 2024-04-17T06:12:16.011 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:16.011 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:16.011 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:12:16.012 INFO:tasks.workunit.client.0.smithi047.stdout:557/600: open fd = 3 2024-04-17T06:12:16.012 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:16.012 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:16.012 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:12:16.012 INFO:tasks.workunit.client.0.smithi047.stdout:558/600: open fd = 3 2024-04-17T06:12:16.012 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:16.012 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:16.012 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:12:16.012 INFO:tasks.workunit.client.0.smithi047.stdout:559/600: open fd = 3 2024-04-17T06:12:16.012 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:16.012 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:16.012 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:12:16.012 INFO:tasks.workunit.client.0.smithi047.stdout:560/600: open fd = 3 2024-04-17T06:12:16.012 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:16.012 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:16.013 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:12:16.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:12:15 smithi098 ceph-mon[36516]: pgmap v345: 97 pgs: 97 active+clean; 5.5 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 9.0 KiB/s wr, 9 op/s 2024-04-17T06:12:16.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:12:15 smithi098 ceph-mon[36516]: daemon mds.k restarted 2024-04-17T06:12:16.133 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:12:15 smithi098 ceph-mds[73025]: -- 172.21.15.98:0/409463795 <== mon.2 v2:172.21.15.98:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559e8467da40 con 0x559e84650800 2024-04-17T06:12:16.133 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:12:15 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[73021]: starting mds.k at 2024-04-17T06:12:16.133 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:12:15 smithi098 bash[73005]: 49fcadeeae1a3608b497973c8cfdf03c4da9def26e9c8779af7839fb67df4f9f 2024-04-17T06:12:16.133 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:12:15 smithi098 systemd[1]: Started Ceph mds.k for 565db2d8-fc7e-11ee-bc91-c7b262605968. 2024-04-17T06:12:16.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:12:15 smithi047 ceph-mon[32223]: pgmap v345: 97 pgs: 97 active+clean; 5.5 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 9.0 KiB/s wr, 9 op/s 2024-04-17T06:12:16.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:12:15 smithi047 ceph-mon[32223]: daemon mds.k restarted 2024-04-17T06:12:16.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:12:15 smithi103 ceph-mon[36243]: pgmap v345: 97 pgs: 97 active+clean; 5.5 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 9.0 KiB/s wr, 9 op/s 2024-04-17T06:12:16.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:12:15 smithi103 ceph-mon[36243]: daemon mds.k restarted 2024-04-17T06:12:16.640 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.642+0000 7fe663677640 1 -- 172.21.15.47:0/1225459569 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe65c101ab0 msgr2=0x7fe65c105aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:12:16.640 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.642+0000 7fe663677640 1 --2- 172.21.15.47:0/1225459569 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe65c101ab0 0x7fe65c105aa0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fe64c009900 tx=0x7fe64c02bc30 comp rx=0 tx=0).stop 2024-04-17T06:12:16.641 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.642+0000 7fe663677640 1 -- 172.21.15.47:0/1225459569 shutdown_connections 2024-04-17T06:12:16.641 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.642+0000 7fe663677640 1 --2- 172.21.15.47:0/1225459569 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe65c106180 0x7fe65c10dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:16.641 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.642+0000 7fe663677640 1 --2- 172.21.15.47:0/1225459569 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe65c101ab0 0x7fe65c105aa0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:16.641 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.642+0000 7fe663677640 1 --2- 172.21.15.47:0/1225459569 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe65c101100 0x7fe65c1014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:16.641 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.642+0000 7fe663677640 1 -- 172.21.15.47:0/1225459569 >> 172.21.15.47:0/1225459569 conn(0x7fe65c077fe0 msgr2=0x7fe65c0783f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:12:16.641 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.642+0000 7fe663677640 1 -- 172.21.15.47:0/1225459569 shutdown_connections 2024-04-17T06:12:16.641 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.642+0000 7fe663677640 1 -- 172.21.15.47:0/1225459569 wait complete. 2024-04-17T06:12:16.641 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.643+0000 7fe663677640 1 Processor -- start 2024-04-17T06:12:16.641 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.643+0000 7fe663677640 1 -- start start 2024-04-17T06:12:16.643 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.643+0000 7fe663677640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe65c101100 0x7fe65c102b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:16.643 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.643+0000 7fe663677640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe65c106180 0x7fe65c103080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:16.643 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.643+0000 7fe663677640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe65c1035c0 0x7fe65c1a69c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:16.643 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.643+0000 7fe663677640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fe65c111210 con 0x7fe65c1035c0 2024-04-17T06:12:16.643 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.643+0000 7fe663677640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe65c111090 con 0x7fe65c106180 2024-04-17T06:12:16.643 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.643+0000 7fe663677640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe65c111390 con 0x7fe65c101100 2024-04-17T06:12:16.643 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.643+0000 7fe661c14640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe65c1035c0 0x7fe65c1a69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:12:16.644 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.643+0000 7fe661c14640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe65c1035c0 0x7fe65c1a69c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44016/0 (socket says 172.21.15.47:44016) 2024-04-17T06:12:16.644 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.643+0000 7fe661c14640 1 -- 172.21.15.47:0/4064664806 learned_addr learned my addr 172.21.15.47:0/4064664806 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:12:16.644 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.643+0000 7fe660c12640 1 --2- 172.21.15.47:0/4064664806 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe65c106180 0x7fe65c103080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:12:16.644 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.643+0000 7fe661413640 1 --2- 172.21.15.47:0/4064664806 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe65c101100 0x7fe65c102b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:12:16.644 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.644+0000 7fe660c12640 1 -- 172.21.15.47:0/4064664806 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe65c101100 msgr2=0x7fe65c102b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:12:16.644 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.644+0000 7fe660c12640 1 --2- 172.21.15.47:0/4064664806 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe65c101100 0x7fe65c102b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:16.644 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.644+0000 7fe660c12640 1 -- 172.21.15.47:0/4064664806 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe65c1035c0 msgr2=0x7fe65c1a69c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:12:16.644 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.644+0000 7fe660c12640 1 --2- 172.21.15.47:0/4064664806 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe65c1035c0 0x7fe65c1a69c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:16.644 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.644+0000 7fe660c12640 1 -- 172.21.15.47:0/4064664806 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe65c1a7060 con 0x7fe65c106180 2024-04-17T06:12:16.644 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.644+0000 7fe661413640 1 --2- 172.21.15.47:0/4064664806 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe65c101100 0x7fe65c102b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:12:16.644 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.644+0000 7fe661c14640 1 --2- 172.21.15.47:0/4064664806 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe65c1035c0 0x7fe65c1a69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T06:12:16.644 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.644+0000 7fe660c12640 1 --2- 172.21.15.47:0/4064664806 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe65c106180 0x7fe65c103080 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe64c036040 tx=0x7fe64c02cbf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:12:16.644 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.644+0000 7fe6527fc640 1 -- 172.21.15.47:0/4064664806 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe64c041070 con 0x7fe65c106180 2024-04-17T06:12:16.644 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.644+0000 7fe663677640 1 -- 172.21.15.47:0/4064664806 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe65c1a72f0 con 0x7fe65c106180 2024-04-17T06:12:16.645 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.644+0000 7fe6527fc640 1 -- 172.21.15.47:0/4064664806 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe64c004450 con 0x7fe65c106180 2024-04-17T06:12:16.645 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.644+0000 7fe6527fc640 1 -- 172.21.15.47:0/4064664806 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe64c003e30 con 0x7fe65c106180 2024-04-17T06:12:16.645 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.644+0000 7fe663677640 1 -- 172.21.15.47:0/4064664806 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe65c1a77e0 con 0x7fe65c106180 2024-04-17T06:12:16.650 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.647+0000 7fe6527fc640 1 -- 172.21.15.47:0/4064664806 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe64c044070 con 0x7fe65c106180 2024-04-17T06:12:16.650 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.647+0000 7fe66625b740 1 Processor -- start 2024-04-17T06:12:16.650 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.647+0000 7fe66625b740 1 -- start start 2024-04-17T06:12:16.650 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.647+0000 7fe66625b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56307f0fc0c0 0x56307f0fc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:16.650 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.647+0000 7fe66625b740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56307f0fc9e0 0x56307f105270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:16.650 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.647+0000 7fe66625b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56307f1057b0 0x56307f107ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:16.650 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.647+0000 7fe66625b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x56307f0617d0 con 0x56307f1057b0 2024-04-17T06:12:16.650 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.647+0000 7fe66625b740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x56307effef70 con 0x56307f0fc9e0 2024-04-17T06:12:16.650 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.647+0000 7fe66625b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56307f05d790 con 0x56307f0fc0c0 2024-04-17T06:12:16.651 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.647+0000 7fe660c12640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56307f1057b0 0x56307f107ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:12:16.651 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.647+0000 7fe661413640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56307f0fc9e0 0x56307f105270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:12:16.651 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.647+0000 7fe660c12640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56307f1057b0 0x56307f107ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44024/0 (socket says 172.21.15.47:44024) 2024-04-17T06:12:16.651 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.647+0000 7fe660c12640 1 -- 172.21.15.47:0/2481918727 learned_addr learned my addr 172.21.15.47:0/2481918727 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:12:16.651 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.647+0000 7fe661c14640 1 --2- 172.21.15.47:0/2481918727 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56307f0fc0c0 0x56307f0fc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:12:16.651 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.647+0000 7fe6527fc640 1 --2- 172.21.15.47:0/4064664806 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fe6480a3dc0 0x7fe6480a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:16.651 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.647+0000 7fe661413640 1 -- 172.21.15.47:0/2481918727 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56307f0fc0c0 msgr2=0x56307f0fc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:12:16.651 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.648+0000 7fe661413640 1 --2- 172.21.15.47:0/2481918727 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56307f0fc0c0 0x56307f0fc4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:16.651 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.648+0000 7fe661413640 1 -- 172.21.15.47:0/2481918727 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56307f1057b0 msgr2=0x56307f107ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:12:16.651 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.648+0000 7fe661413640 1 --2- 172.21.15.47:0/2481918727 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56307f1057b0 0x56307f107ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:16.651 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.648+0000 7fe661413640 1 -- 172.21.15.47:0/2481918727 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56307ed74210 con 0x56307f0fc9e0 2024-04-17T06:12:16.651 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.648+0000 7fe660c12640 1 --2- 172.21.15.47:0/2481918727 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56307f1057b0 0x56307f107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T06:12:16.651 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.648+0000 7fe661c14640 1 --2- 172.21.15.47:0/4064664806 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fe6480a3dc0 0x7fe6480a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:12:16.651 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.648+0000 7fe6527fc640 1 -- 172.21.15.47:0/4064664806 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8469+0+0 (secure 0 0 0) 0x7fe64c0f2570 con 0x7fe65c106180 2024-04-17T06:12:16.651 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.648+0000 7fe661413640 1 --2- 172.21.15.47:0/2481918727 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56307f0fc9e0 0x56307f105270 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe658009c30 tx=0x7fe6580117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e561a43cc5482c69 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:12:16.652 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.648+0000 7fe6477fe640 1 -- 172.21.15.47:0/2481918727 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe658012450 con 0x56307f0fc9e0 2024-04-17T06:12:16.652 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.648+0000 7fe6477fe640 1 -- 172.21.15.47:0/2481918727 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe658012ad0 con 0x56307f0fc9e0 2024-04-17T06:12:16.652 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.648+0000 7fe6477fe640 1 -- 172.21.15.47:0/2481918727 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe65800fbb0 con 0x56307f0fc9e0 2024-04-17T06:12:16.652 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.649+0000 7fe66625b740 1 -- 172.21.15.47:0/2481918727 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56307f0fc9e0 msgr2=0x56307f105270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:12:16.652 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.649+0000 7fe661c14640 1 --2- 172.21.15.47:0/4064664806 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fe6480a3dc0 0x7fe6480a6280 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe654009970 tx=0x7fe654004a40 comp rx=0 tx=0).ready entity=mgr.24923 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:12:16.652 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.649+0000 7fe66625b740 1 --2- 172.21.15.47:0/2481918727 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56307f0fc9e0 0x56307f105270 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe658009c30 tx=0x7fe6580117b0 comp rx=0 tx=0).stop 2024-04-17T06:12:16.652 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.649+0000 7fe66625b740 1 -- 172.21.15.47:0/2481918727 shutdown_connections 2024-04-17T06:12:16.652 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.649+0000 7fe66625b740 1 --2- 172.21.15.47:0/2481918727 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56307f1057b0 0x56307f107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:16.652 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.649+0000 7fe66625b740 1 --2- 172.21.15.47:0/2481918727 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56307f0fc9e0 0x56307f105270 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:16.652 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.649+0000 7fe66625b740 1 --2- 172.21.15.47:0/2481918727 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56307f0fc0c0 0x56307f0fc4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:16.652 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.649+0000 7fe66625b740 1 -- 172.21.15.47:0/2481918727 >> 172.21.15.47:0/2481918727 conn(0x56307f0f69e0 msgr2=0x56307f0f6dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:12:16.652 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.649+0000 7fe66625b740 1 -- 172.21.15.47:0/2481918727 shutdown_connections 2024-04-17T06:12:16.652 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.649+0000 7fe66625b740 1 -- 172.21.15.47:0/2481918727 wait complete. 2024-04-17T06:12:16.652 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.649+0000 7fe66625b740 1 -- start start 2024-04-17T06:12:16.652 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.649+0000 7fe66625b740 1 Processor -- start 2024-04-17T06:12:16.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.650+0000 7fe66625b740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56307f0fc0c0 0x56307f102790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:16.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.650+0000 7fe66625b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56307f0fc9e0 0x56307f102cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:16.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.650+0000 7fe66625b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56307f1057b0 0x56307f103210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:16.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.650+0000 7fe66625b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x56307f1083e0 con 0x56307f1057b0 2024-04-17T06:12:16.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.650+0000 7fe66625b740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x56307efe9e90 con 0x56307f0fc0c0 2024-04-17T06:12:16.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.650+0000 7fe66625b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56307f108560 con 0x56307f0fc9e0 2024-04-17T06:12:16.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.650+0000 7fe66625b740 10 client.0 _ll_register_callbacks cb 0x56307efffd40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:12:16.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.650+0000 7fe661c14640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56307f1057b0 0x56307f103210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:12:16.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.650+0000 7fe661413640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56307f0fc0c0 0x56307f102790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:12:16.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.650+0000 7fe661413640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56307f0fc0c0 0x56307f102790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.47:45086/0 (socket says 172.21.15.47:45086) 2024-04-17T06:12:16.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.650+0000 7fe661413640 1 -- 172.21.15.47:0/1762372174 learned_addr learned my addr 172.21.15.47:0/1762372174 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:12:16.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.650+0000 7fe660c12640 1 --2- 172.21.15.47:0/1762372174 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56307f0fc9e0 0x56307f102cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:12:16.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.650+0000 7fe661c14640 1 -- 172.21.15.47:0/1762372174 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56307f0fc9e0 msgr2=0x56307f102cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:12:16.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.650+0000 7fe661c14640 1 --2- 172.21.15.47:0/1762372174 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56307f0fc9e0 0x56307f102cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:16.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.650+0000 7fe661c14640 1 -- 172.21.15.47:0/1762372174 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56307f0fc0c0 msgr2=0x56307f102790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:12:16.654 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.650+0000 7fe661c14640 1 --2- 172.21.15.47:0/1762372174 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56307f0fc0c0 0x56307f102790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:16.654 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.650+0000 7fe661c14640 1 -- 172.21.15.47:0/1762372174 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56307ee40620 con 0x56307f1057b0 2024-04-17T06:12:16.654 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.650+0000 7fe661413640 1 --2- 172.21.15.47:0/1762372174 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56307f0fc0c0 0x56307f102790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T06:12:16.654 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.650+0000 7fe661c14640 1 --2- 172.21.15.47:0/1762372174 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56307f1057b0 0x56307f103210 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fe654009270 tx=0x7fe654004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fb8258ee674711e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:12:16.654 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.650+0000 7fe644ff9640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-17T06:12:16.654 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.651+0000 7fe644ff9640 1 -- 172.21.15.47:0/1762372174 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe654028640 con 0x56307f1057b0 2024-04-17T06:12:16.654 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.651+0000 7fe644ff9640 1 -- 172.21.15.47:0/1762372174 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8469+0+0 (secure 0 0 0) 0x7fe6540287e0 con 0x56307f1057b0 2024-04-17T06:12:16.654 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.651+0000 7fe66625b740 1 -- 172.21.15.47:0/1762372174 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe658012ad0 con 0x56307f1057b0 2024-04-17T06:12:16.654 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.651+0000 7fe644ff9640 1 -- 172.21.15.47:0/1762372174 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fe6540289e0 con 0x56307f1057b0 2024-04-17T06:12:16.654 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.651+0000 7fe66625b740 10 client.16311 fetch_fsmap learned FSMap version 36 2024-04-17T06:12:16.654 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.651+0000 7fe66625b740 1 -- 172.21.15.47:0/1762372174 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x56307f114650 con 0x56307f1057b0 2024-04-17T06:12:16.654 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.651+0000 7fe644ff9640 1 -- 172.21.15.47:0/1762372174 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe658012ad0 con 0x56307f1057b0 2024-04-17T06:12:16.654 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.651+0000 7fe644ff9640 1 -- 172.21.15.47:0/1762372174 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 36) ==== 8339+0+0 (secure 0 0 0) 0x7fe654026020 con 0x56307f1057b0 2024-04-17T06:12:16.654 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.651+0000 7fe66625b740 10 client.16311 fetch_fsmap finished waiting for FSMap version 36 2024-04-17T06:12:16.655 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.651+0000 7fe66625b740 10 client.16311 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-17T06:12:16.655 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.651+0000 7fe66625b740 20 client.16311 populate_metadata read hostname 'smithi047' 2024-04-17T06:12:16.655 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.651+0000 7fe66625b740 1 --2- 172.21.15.47:0/1762372174 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x56307f115ac0 0x56307f117ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:16.655 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.651+0000 7fe66625b740 4 client.16311 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T06:12:16.655 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.651+0000 7fe66625b740 1 -- 172.21.15.47:0/1762372174 --> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56307f06f1f0 con 0x56307f115ac0 2024-04-17T06:12:16.655 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.651+0000 7fe661413640 1 --2- 172.21.15.47:0/1762372174 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x56307f115ac0 0x56307f117ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:12:16.655 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.652+0000 7fe661413640 1 --2- 172.21.15.47:0/1762372174 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x56307f115ac0 0x56307f117ed0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c39e2359f7d09f29 server_cookie=7ffd7cebb63570c in_seq=0 out_seq=0 2024-04-17T06:12:16.655 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.652+0000 7fe644ff9640 10 client.16311 ms_handle_connect on v2:172.21.15.98:6838/2841287917 2024-04-17T06:12:16.656 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.652+0000 7fe644ff9640 1 -- 172.21.15.47:0/1762372174 <== mds.0 v2:172.21.15.98:6838/2841287917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56307f06f1f0 con 0x56307f115ac0 2024-04-17T06:12:16.656 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.652+0000 7fe644ff9640 10 client.16311 handle_command_reply: tid=0 2024-04-17T06:12:16.656 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.653+0000 7fe66625b740 1 client.16311 shutdown 2024-04-17T06:12:16.656 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.653+0000 7fe66625b740 1 -- 172.21.15.47:0/1762372174 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56307f1057b0 msgr2=0x56307f103210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:12:16.656 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.653+0000 7fe66625b740 1 --2- 172.21.15.47:0/1762372174 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56307f1057b0 0x56307f103210 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fe654009270 tx=0x7fe654004ab0 comp rx=0 tx=0).stop 2024-04-17T06:12:16.656 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.653+0000 7fe66625b740 1 -- 172.21.15.47:0/1762372174 shutdown_connections 2024-04-17T06:12:16.656 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.653+0000 7fe66625b740 1 --2- 172.21.15.47:0/1762372174 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x56307f115ac0 0x56307f117ed0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:16.656 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.653+0000 7fe66625b740 1 --2- 172.21.15.47:0/1762372174 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56307f1057b0 0x56307f103210 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:16.656 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.653+0000 7fe66625b740 1 --2- 172.21.15.47:0/1762372174 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56307f0fc9e0 0x56307f102cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:16.657 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.653+0000 7fe66625b740 1 --2- 172.21.15.47:0/1762372174 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56307f0fc0c0 0x56307f102790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:16.657 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.653+0000 7fe644ff9640 0 client.16311 ms_handle_reset on v2:172.21.15.98:6838/2841287917 2024-04-17T06:12:16.657 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.653+0000 7fe66625b740 1 -- 172.21.15.47:0/1762372174 >> 172.21.15.47:0/1762372174 conn(0x56307f0f69e0 msgr2=0x56307f0f8730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:12:16.657 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.653+0000 7fe66625b740 1 -- 172.21.15.47:0/1762372174 shutdown_connections 2024-04-17T06:12:16.657 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.653+0000 7fe66625b740 1 -- 172.21.15.47:0/1762372174 wait complete. 2024-04-17T06:12:16.657 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.653+0000 7fe66625b740 20 client.16311 trim_cache size 0 max 16384 2024-04-17T06:12:16.668 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.667+0000 7fe66625b740 1 Processor -- start 2024-04-17T06:12:16.668 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.667+0000 7fe66625b740 1 -- start start 2024-04-17T06:12:16.668 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.667+0000 7fe66625b740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56307f0fc0c0 0x56307f104ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:16.668 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.667+0000 7fe66625b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56307f0fc9e0 0x56307f1112a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:16.668 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.667+0000 7fe66625b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56307f1057b0 0x56307f11d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:16.668 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.667+0000 7fe66625b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fe65c1b2610 con 0x56307f0fc9e0 2024-04-17T06:12:16.668 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.667+0000 7fe66625b740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe65c1afed0 con 0x56307f0fc0c0 2024-04-17T06:12:16.668 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.667+0000 7fe66625b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe65c1add20 con 0x56307f1057b0 2024-04-17T06:12:16.668 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.667+0000 7fe660c12640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56307f0fc9e0 0x56307f1112a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:12:16.669 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.667+0000 7fe660c12640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56307f0fc9e0 0x56307f1112a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44042/0 (socket says 172.21.15.47:44042) 2024-04-17T06:12:16.669 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.667+0000 7fe660c12640 1 -- 172.21.15.47:0/1732542445 learned_addr learned my addr 172.21.15.47:0/1732542445 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:12:16.669 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.667+0000 7fe661413640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56307f0fc0c0 0x56307f104ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:12:16.669 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.668+0000 7fe660c12640 1 -- 172.21.15.47:0/1732542445 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56307f1057b0 msgr2=0x56307f11d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:12:16.669 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.668+0000 7fe661c14640 1 --2- 172.21.15.47:0/1732542445 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56307f1057b0 0x56307f11d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:12:16.669 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.668+0000 7fe660c12640 1 --2- 172.21.15.47:0/1732542445 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56307f1057b0 0x56307f11d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:16.669 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.668+0000 7fe660c12640 1 -- 172.21.15.47:0/1732542445 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56307f0fc0c0 msgr2=0x56307f104ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:12:16.669 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.668+0000 7fe660c12640 1 --2- 172.21.15.47:0/1732542445 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56307f0fc0c0 0x56307f104ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:16.669 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.668+0000 7fe660c12640 1 -- 172.21.15.47:0/1732542445 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56307f107ba0 con 0x56307f0fc9e0 2024-04-17T06:12:16.669 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.668+0000 7fe661413640 1 --2- 172.21.15.47:0/1732542445 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56307f0fc0c0 0x56307f104ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T06:12:16.669 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.668+0000 7fe660c12640 1 --2- 172.21.15.47:0/1732542445 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56307f0fc9e0 0x56307f1112a0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fe64c006010 tx=0x7fe64c0f8110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c454e438b1563bb5 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:12:16.669 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.668+0000 7fe646ffd640 1 -- 172.21.15.47:0/1732542445 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe64c04b070 con 0x56307f0fc9e0 2024-04-17T06:12:16.669 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.668+0000 7fe646ffd640 1 -- 172.21.15.47:0/1732542445 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe64c0bed50 con 0x56307f0fc9e0 2024-04-17T06:12:16.669 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.668+0000 7fe646ffd640 1 -- 172.21.15.47:0/1732542445 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe64c0bdfb0 con 0x56307f0fc9e0 2024-04-17T06:12:16.669 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.669+0000 7fe66625b740 1 -- 172.21.15.47:0/1732542445 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56307f0fc9e0 msgr2=0x56307f1112a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:12:16.670 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.669+0000 7fe66625b740 1 --2- 172.21.15.47:0/1732542445 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56307f0fc9e0 0x56307f1112a0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fe64c006010 tx=0x7fe64c0f8110 comp rx=0 tx=0).stop 2024-04-17T06:12:16.670 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.669+0000 7fe66625b740 1 -- 172.21.15.47:0/1732542445 shutdown_connections 2024-04-17T06:12:16.670 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.669+0000 7fe66625b740 1 --2- 172.21.15.47:0/1732542445 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56307f1057b0 0x56307f11d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:16.670 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.669+0000 7fe66625b740 1 --2- 172.21.15.47:0/1732542445 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56307f0fc9e0 0x56307f1112a0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:16.670 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.669+0000 7fe66625b740 1 --2- 172.21.15.47:0/1732542445 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56307f0fc0c0 0x56307f104ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:16.670 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.669+0000 7fe66625b740 1 -- 172.21.15.47:0/1732542445 >> 172.21.15.47:0/1732542445 conn(0x56307f0f69e0 msgr2=0x56307f0f86c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:12:16.670 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.669+0000 7fe66625b740 1 -- 172.21.15.47:0/1732542445 shutdown_connections 2024-04-17T06:12:16.670 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.669+0000 7fe66625b740 1 -- 172.21.15.47:0/1732542445 wait complete. 2024-04-17T06:12:16.670 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.669+0000 7fe66625b740 1 -- start start 2024-04-17T06:12:16.670 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.669+0000 7fe66625b740 1 Processor -- start 2024-04-17T06:12:16.672 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.670+0000 7fe66625b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56307f0fc0c0 0x56307f0ff4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:16.673 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.670+0000 7fe66625b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56307f0fc9e0 0x56307f0ff9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:16.673 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.670+0000 7fe66625b740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56307f1057b0 0x56307f0fff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:16.673 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.670+0000 7fe66625b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fe65c1ae390 con 0x56307f0fc0c0 2024-04-17T06:12:16.673 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.670+0000 7fe66625b740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe65c1af800 con 0x56307f1057b0 2024-04-17T06:12:16.673 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.670+0000 7fe66625b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe65c1aea60 con 0x56307f0fc9e0 2024-04-17T06:12:16.673 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.670+0000 7fe66625b740 10 client.0 _ll_register_callbacks cb 0x7fe654009e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:12:16.673 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.670+0000 7fe661413640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56307f0fc0c0 0x56307f0ff4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:12:16.673 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.670+0000 7fe661413640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56307f0fc0c0 0x56307f0ff4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44056/0 (socket says 172.21.15.47:44056) 2024-04-17T06:12:16.673 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.670+0000 7fe661413640 1 -- 172.21.15.47:0/78846597 learned_addr learned my addr 172.21.15.47:0/78846597 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:12:16.673 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.670+0000 7fe661c14640 1 --2- 172.21.15.47:0/78846597 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56307f1057b0 0x56307f0fff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:12:16.673 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.670+0000 7fe660c12640 1 --2- 172.21.15.47:0/78846597 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56307f0fc9e0 0x56307f0ff9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:12:16.673 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.670+0000 7fe661c14640 1 -- 172.21.15.47:0/78846597 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56307f0fc9e0 msgr2=0x56307f0ff9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:12:16.673 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.670+0000 7fe661c14640 1 --2- 172.21.15.47:0/78846597 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56307f0fc9e0 0x56307f0ff9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:16.673 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.670+0000 7fe661c14640 1 -- 172.21.15.47:0/78846597 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56307f0fc0c0 msgr2=0x56307f0ff4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:12:16.673 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.670+0000 7fe661c14640 1 --2- 172.21.15.47:0/78846597 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56307f0fc0c0 0x56307f0ff4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:16.673 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.670+0000 7fe661c14640 1 -- 172.21.15.47:0/78846597 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56307f17b0d0 con 0x56307f1057b0 2024-04-17T06:12:16.674 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.670+0000 7fe661413640 1 --2- 172.21.15.47:0/78846597 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56307f0fc0c0 0x56307f0ff4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T06:12:16.674 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.670+0000 7fe661c14640 1 --2- 172.21.15.47:0/78846597 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56307f1057b0 0x56307f0fff20 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe654031f20 tx=0x7fe65401fd40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa363bb7743ee554 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:12:16.674 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.670+0000 7fe644ff9640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T06:12:16.674 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.670+0000 7fe644ff9640 1 -- 172.21.15.47:0/78846597 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe65403f060 con 0x56307f1057b0 2024-04-17T06:12:16.674 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.670+0000 7fe66625b740 1 -- 172.21.15.47:0/78846597 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe64c0bed50 con 0x56307f1057b0 2024-04-17T06:12:16.674 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.670+0000 7fe644ff9640 1 -- 172.21.15.47:0/78846597 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8469+0+0 (secure 0 0 0) 0x7fe654028b80 con 0x56307f1057b0 2024-04-17T06:12:16.674 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.671+0000 7fe644ff9640 1 -- 172.21.15.47:0/78846597 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fe654013450 con 0x56307f1057b0 2024-04-17T06:12:16.674 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.671+0000 7fe66625b740 10 client.25706 fetch_fsmap learned FSMap version 36 2024-04-17T06:12:16.674 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.671+0000 7fe66625b740 1 -- 172.21.15.47:0/78846597 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x56307f17c3c0 con 0x56307f1057b0 2024-04-17T06:12:16.674 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.671+0000 7fe644ff9640 1 -- 172.21.15.47:0/78846597 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe64c0bed50 con 0x56307f1057b0 2024-04-17T06:12:16.674 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.671+0000 7fe644ff9640 1 -- 172.21.15.47:0/78846597 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 36) ==== 8339+0+0 (secure 0 0 0) 0x7fe654026050 con 0x56307f1057b0 2024-04-17T06:12:16.674 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.671+0000 7fe66625b740 10 client.25706 fetch_fsmap finished waiting for FSMap version 36 2024-04-17T06:12:16.674 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.671+0000 7fe66625b740 10 client.25706 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-17T06:12:16.674 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.671+0000 7fe66625b740 20 client.25706 populate_metadata read hostname 'smithi047' 2024-04-17T06:12:16.675 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.671+0000 7fe66625b740 1 --2- 172.21.15.47:0/78846597 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x56307f17ca20 0x56307f17ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:16.675 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.671+0000 7fe66625b740 4 client.25706 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-17T06:12:16.675 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.671+0000 7fe66625b740 1 -- 172.21.15.47:0/78846597 --> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56307f069060 con 0x56307f17ca20 2024-04-17T06:12:16.675 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.671+0000 7fe661413640 1 --2- 172.21.15.47:0/78846597 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x56307f17ca20 0x56307f17ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:12:16.675 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.672+0000 7fe661413640 1 --2- 172.21.15.47:0/78846597 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x56307f17ca20 0x56307f17ee10 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f147a525684eb10 server_cookie=57e15ee33ab53901 in_seq=0 out_seq=0 2024-04-17T06:12:16.675 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.672+0000 7fe644ff9640 10 client.25706 ms_handle_connect on v2:172.21.15.98:6838/2841287917 2024-04-17T06:12:16.678 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.679+0000 7fe644ff9640 1 -- 172.21.15.47:0/78846597 <== mds.0 v2:172.21.15.98:6838/2841287917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56307f069060 con 0x56307f17ca20 2024-04-17T06:12:16.679 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.679+0000 7fe644ff9640 10 client.25706 handle_command_reply: tid=0 2024-04-17T06:12:16.679 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.679+0000 7fe66625b740 1 client.25706 shutdown 2024-04-17T06:12:16.679 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.679+0000 7fe66625b740 1 -- 172.21.15.47:0/78846597 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56307f1057b0 msgr2=0x56307f0fff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:12:16.679 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.679+0000 7fe66625b740 1 --2- 172.21.15.47:0/78846597 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56307f1057b0 0x56307f0fff20 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe654031f20 tx=0x7fe65401fd40 comp rx=0 tx=0).stop 2024-04-17T06:12:16.679 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.679+0000 7fe66625b740 1 -- 172.21.15.47:0/78846597 shutdown_connections 2024-04-17T06:12:16.679 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.679+0000 7fe66625b740 1 --2- 172.21.15.47:0/78846597 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x56307f17ca20 0x56307f17ee10 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:16.679 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.679+0000 7fe66625b740 1 --2- 172.21.15.47:0/78846597 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56307f1057b0 0x56307f0fff20 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:16.679 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.679+0000 7fe66625b740 1 --2- 172.21.15.47:0/78846597 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56307f0fc9e0 0x56307f0ff9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:16.679 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.679+0000 7fe66625b740 1 --2- 172.21.15.47:0/78846597 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56307f0fc0c0 0x56307f0ff4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:16.679 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.679+0000 7fe644ff9640 0 client.25706 ms_handle_reset on v2:172.21.15.98:6838/2841287917 2024-04-17T06:12:16.679 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.679+0000 7fe66625b740 1 -- 172.21.15.47:0/78846597 >> 172.21.15.47:0/78846597 conn(0x56307f0f69e0 msgr2=0x56307f0f5270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:12:16.679 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.679+0000 7fe66625b740 1 -- 172.21.15.47:0/78846597 shutdown_connections 2024-04-17T06:12:16.679 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.679+0000 7fe66625b740 1 -- 172.21.15.47:0/78846597 wait complete. 2024-04-17T06:12:16.679 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.680+0000 7fe66625b740 20 client.25706 trim_cache size 0 max 16384 2024-04-17T06:12:16.680 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-04-17T06:12:16.680 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-04-17T06:12:16.680 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-04-17T06:12:16.680 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-04-17T06:12:16.681 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.680+0000 7fe663677640 1 -- 172.21.15.47:0/4064664806 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fe6480a3dc0 msgr2=0x7fe6480a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:12:16.681 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.680+0000 7fe663677640 1 --2- 172.21.15.47:0/4064664806 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fe6480a3dc0 0x7fe6480a6280 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe654009970 tx=0x7fe654004a40 comp rx=0 tx=0).stop 2024-04-17T06:12:16.681 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.680+0000 7fe663677640 1 -- 172.21.15.47:0/4064664806 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe65c106180 msgr2=0x7fe65c103080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:12:16.681 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.680+0000 7fe663677640 1 --2- 172.21.15.47:0/4064664806 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe65c106180 0x7fe65c103080 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe64c036040 tx=0x7fe64c02cbf0 comp rx=0 tx=0).stop 2024-04-17T06:12:16.681 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.680+0000 7fe663677640 1 -- 172.21.15.47:0/4064664806 shutdown_connections 2024-04-17T06:12:16.681 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.680+0000 7fe663677640 1 --2- 172.21.15.47:0/4064664806 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fe6480a3dc0 0x7fe6480a6280 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:16.681 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.680+0000 7fe663677640 1 --2- 172.21.15.47:0/4064664806 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe65c1035c0 0x7fe65c1a69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:16.681 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.680+0000 7fe663677640 1 --2- 172.21.15.47:0/4064664806 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe65c106180 0x7fe65c103080 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:16.681 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.680+0000 7fe663677640 1 --2- 172.21.15.47:0/4064664806 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe65c101100 0x7fe65c102b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:16.681 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.680+0000 7fe663677640 1 -- 172.21.15.47:0/4064664806 >> 172.21.15.47:0/4064664806 conn(0x7fe65c077fe0 msgr2=0x7fe65c0698f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:12:16.681 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.680+0000 7fe663677640 1 -- 172.21.15.47:0/4064664806 shutdown_connections 2024-04-17T06:12:16.681 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:16.680+0000 7fe663677640 1 -- 172.21.15.47:0/4064664806 wait complete. 2024-04-17T06:12:17.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:12:16 smithi098 ceph-mon[36516]: osdmap e104: 12 total, 12 up, 12 in 2024-04-17T06:12:17.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:12:16 smithi098 ceph-mon[36516]: mds.? [v2:172.21.15.98:6832/4230863674,v1:172.21.15.98:6833/4230863674] up:boot 2024-04-17T06:12:17.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:12:16 smithi098 ceph-mon[36516]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-04-17T06:12:17.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:12:16 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-17T06:12:17.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:12:16 smithi098 ceph-mon[36516]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-17T06:12:17.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:12:16 smithi047 ceph-mon[32223]: osdmap e104: 12 total, 12 up, 12 in 2024-04-17T06:12:17.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:12:16 smithi047 ceph-mon[32223]: mds.? [v2:172.21.15.98:6832/4230863674,v1:172.21.15.98:6833/4230863674] up:boot 2024-04-17T06:12:17.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:12:16 smithi047 ceph-mon[32223]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-04-17T06:12:17.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:12:16 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-17T06:12:17.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:12:16 smithi047 ceph-mon[32223]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-17T06:12:17.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:12:16 smithi103 ceph-mon[36243]: osdmap e104: 12 total, 12 up, 12 in 2024-04-17T06:12:17.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:12:16 smithi103 ceph-mon[36243]: mds.? [v2:172.21.15.98:6832/4230863674,v1:172.21.15.98:6833/4230863674] up:boot 2024-04-17T06:12:17.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:12:16 smithi103 ceph-mon[36243]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-04-17T06:12:17.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:12:16 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-17T06:12:17.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:12:16 smithi103 ceph-mon[36243]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-17T06:12:18.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:12:17 smithi098 ceph-mon[36516]: pgmap v347: 97 pgs: 97 active+clean; 5.5 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-04-17T06:12:18.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:12:17 smithi047 ceph-mon[32223]: pgmap v347: 97 pgs: 97 active+clean; 5.5 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-04-17T06:12:18.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:12:17 smithi103 ceph-mon[36243]: pgmap v347: 97 pgs: 97 active+clean; 5.5 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-04-17T06:12:20.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:12:19 smithi098 ceph-mon[36516]: pgmap v348: 97 pgs: 97 active+clean; 5.5 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6.7 KiB/s wr, 5 op/s 2024-04-17T06:12:20.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:12:19 smithi047 ceph-mon[32223]: pgmap v348: 97 pgs: 97 active+clean; 5.5 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6.7 KiB/s wr, 5 op/s 2024-04-17T06:12:20.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:12:19 smithi103 ceph-mon[36243]: pgmap v348: 97 pgs: 97 active+clean; 5.5 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6.7 KiB/s wr, 5 op/s 2024-04-17T06:12:22.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:12:21 smithi098 ceph-mon[36516]: pgmap v349: 97 pgs: 97 active+clean; 5.5 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-17T06:12:22.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:12:21 smithi047 ceph-mon[32223]: pgmap v349: 97 pgs: 97 active+clean; 5.5 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-17T06:12:22.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:12:21 smithi103 ceph-mon[36243]: pgmap v349: 97 pgs: 97 active+clean; 5.5 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-17T06:12:24.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:12:23 smithi098 ceph-mon[36516]: pgmap v350: 97 pgs: 97 active+clean; 5.5 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-17T06:12:24.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:12:23 smithi047 ceph-mon[32223]: pgmap v350: 97 pgs: 97 active+clean; 5.5 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-17T06:12:24.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:12:23 smithi103 ceph-mon[36243]: pgmap v350: 97 pgs: 97 active+clean; 5.5 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-17T06:12:26.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:12:25 smithi098 ceph-mon[36516]: pgmap v351: 97 pgs: 97 active+clean; 5.6 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-17T06:12:26.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:12:25 smithi047 ceph-mon[32223]: pgmap v351: 97 pgs: 97 active+clean; 5.6 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-17T06:12:26.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:12:25 smithi103 ceph-mon[36243]: pgmap v351: 97 pgs: 97 active+clean; 5.6 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-17T06:12:28.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:12:27 smithi047 ceph-mon[32223]: pgmap v352: 97 pgs: 97 active+clean; 5.6 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-17T06:12:28.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:12:27 smithi103 ceph-mon[36243]: pgmap v352: 97 pgs: 97 active+clean; 5.6 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-17T06:12:28.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:12:27 smithi098 ceph-mon[36516]: pgmap v352: 97 pgs: 97 active+clean; 5.6 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-17T06:12:29.782 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.780+0000 7f906abcd640 1 -- 172.21.15.47:0/3623161173 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9064101100 msgr2=0x7f90641014e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:12:29.782 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.780+0000 7f906abcd640 1 --2- 172.21.15.47:0/3623161173 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9064101100 0x7f90641014e0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f9060009900 tx=0x7f906002bc30 comp rx=0 tx=0).stop 2024-04-17T06:12:29.782 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.780+0000 7f906abcd640 1 -- 172.21.15.47:0/3623161173 shutdown_connections 2024-04-17T06:12:29.782 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.780+0000 7f906abcd640 1 --2- 172.21.15.47:0/3623161173 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9064106180 0x7f906410dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:29.782 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.780+0000 7f906abcd640 1 --2- 172.21.15.47:0/3623161173 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9064101ab0 0x7f9064105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:29.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.780+0000 7f906abcd640 1 --2- 172.21.15.47:0/3623161173 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9064101100 0x7f90641014e0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:29.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.781+0000 7f906abcd640 1 -- 172.21.15.47:0/3623161173 >> 172.21.15.47:0/3623161173 conn(0x7f9064077fe0 msgr2=0x7f90640783f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:12:29.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.781+0000 7f906abcd640 1 -- 172.21.15.47:0/3623161173 shutdown_connections 2024-04-17T06:12:29.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.781+0000 7f906abcd640 1 -- 172.21.15.47:0/3623161173 wait complete. 2024-04-17T06:12:29.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.781+0000 7f906abcd640 1 Processor -- start 2024-04-17T06:12:29.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.781+0000 7f906abcd640 1 -- start start 2024-04-17T06:12:29.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.781+0000 7f906abcd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9064101100 0x7f906419e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:29.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.781+0000 7f906abcd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9064106180 0x7f906419e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:29.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.781+0000 7f906abcd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f90641985e0 0x7f9064198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:29.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.781+0000 7f906abcd640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f9064111230 con 0x7f9064101100 2024-04-17T06:12:29.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.781+0000 7f906abcd640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f90641110b0 con 0x7f9064106180 2024-04-17T06:12:29.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.781+0000 7f906abcd640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f90641113b0 con 0x7f90641985e0 2024-04-17T06:12:29.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.782+0000 7f9068969640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9064101100 0x7f906419e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:12:29.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.782+0000 7f9068969640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9064101100 0x7f906419e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39638/0 (socket says 172.21.15.47:39638) 2024-04-17T06:12:29.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.782+0000 7f9068969640 1 -- 172.21.15.47:0/814976913 learned_addr learned my addr 172.21.15.47:0/814976913 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:12:29.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.782+0000 7f906916a640 1 --2- 172.21.15.47:0/814976913 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f90641985e0 0x7f9064198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:12:29.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.782+0000 7f905bfff640 1 --2- 172.21.15.47:0/814976913 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9064106180 0x7f906419e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:12:29.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.782+0000 7f9068969640 1 -- 172.21.15.47:0/814976913 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f90641985e0 msgr2=0x7f9064198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:12:29.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.782+0000 7f9068969640 1 --2- 172.21.15.47:0/814976913 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f90641985e0 0x7f9064198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:29.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.782+0000 7f9068969640 1 -- 172.21.15.47:0/814976913 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9064106180 msgr2=0x7f906419e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:12:29.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.782+0000 7f9068969640 1 --2- 172.21.15.47:0/814976913 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9064106180 0x7f906419e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:29.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.782+0000 7f9068969640 1 -- 172.21.15.47:0/814976913 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9064199100 con 0x7f9064101100 2024-04-17T06:12:29.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.782+0000 7f9068969640 1 --2- 172.21.15.47:0/814976913 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9064101100 0x7f906419e430 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f9060006010 tx=0x7f906002cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:12:29.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.782+0000 7f9059ffb640 1 -- 172.21.15.47:0/814976913 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9060041070 con 0x7f9064101100 2024-04-17T06:12:29.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.782+0000 7f9059ffb640 1 -- 172.21.15.47:0/814976913 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9060004450 con 0x7f9064101100 2024-04-17T06:12:29.785 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.782+0000 7f906abcd640 1 -- 172.21.15.47:0/814976913 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9064199390 con 0x7f9064101100 2024-04-17T06:12:29.785 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.782+0000 7f9059ffb640 1 -- 172.21.15.47:0/814976913 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9060003e00 con 0x7f9064101100 2024-04-17T06:12:29.785 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.783+0000 7f906abcd640 1 -- 172.21.15.47:0/814976913 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90641a5580 con 0x7f9064101100 2024-04-17T06:12:29.791 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.785+0000 7f9059ffb640 1 -- 172.21.15.47:0/814976913 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9060044070 con 0x7f9064101100 2024-04-17T06:12:29.791 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.786+0000 7f906d7bc740 1 Processor -- start 2024-04-17T06:12:29.791 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.786+0000 7f906d7bc740 1 -- start start 2024-04-17T06:12:29.791 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.786+0000 7f906d7bc740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564361b9f0c0 0x564361b9f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.786+0000 7f906d7bc740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564361b9f9e0 0x564361ba8270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.786+0000 7f906d7bc740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564361ba87b0 0x564361baaba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.786+0000 7f906d7bc740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x564361b047d0 con 0x564361ba87b0 2024-04-17T06:12:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.786+0000 7f906d7bc740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x564361aa1f70 con 0x564361b9f0c0 2024-04-17T06:12:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.786+0000 7f906d7bc740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x564361b00790 con 0x564361b9f9e0 2024-04-17T06:12:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.786+0000 7f9059ffb640 1 --2- 172.21.15.47:0/814976913 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f904c0a3dc0 0x7f904c0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.786+0000 7f905bfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564361ba87b0 0x564361baaba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:12:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.786+0000 7f906916a640 1 --2- 172.21.15.47:0/814976913 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f904c0a3dc0 0x7f904c0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:12:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.786+0000 7f905bfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564361ba87b0 0x564361baaba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39640/0 (socket says 172.21.15.47:39640) 2024-04-17T06:12:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.786+0000 7f905bfff640 1 -- 172.21.15.47:0/2188865572 learned_addr learned my addr 172.21.15.47:0/2188865572 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:12:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.786+0000 7f9059ffb640 1 -- 172.21.15.47:0/814976913 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8469+0+0 (secure 0 0 0) 0x7f90600f2100 con 0x7f9064101100 2024-04-17T06:12:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.786+0000 7f9068969640 1 --2- 172.21.15.47:0/2188865572 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564361b9f9e0 0x564361ba8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:12:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.786+0000 7f906916a640 1 --2- 172.21.15.47:0/2188865572 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564361b9f0c0 0x564361b9f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:12:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.787+0000 7f905bfff640 1 -- 172.21.15.47:0/2188865572 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564361b9f9e0 msgr2=0x564361ba8270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:12:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.787+0000 7f905bfff640 1 --2- 172.21.15.47:0/2188865572 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564361b9f9e0 0x564361ba8270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:29.793 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.787+0000 7f905bfff640 1 -- 172.21.15.47:0/2188865572 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564361b9f0c0 msgr2=0x564361b9f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:12:29.793 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.787+0000 7f905bfff640 1 --2- 172.21.15.47:0/2188865572 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564361b9f0c0 0x564361b9f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:29.793 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.787+0000 7f905bfff640 1 -- 172.21.15.47:0/2188865572 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564361817210 con 0x564361ba87b0 2024-04-17T06:12:29.793 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.787+0000 7f9068969640 1 --2- 172.21.15.47:0/2188865572 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564361b9f9e0 0x564361ba8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T06:12:29.793 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.787+0000 7f906916a640 1 --2- 172.21.15.47:0/814976913 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f904c0a3dc0 0x7f904c0a6280 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f905c005fd0 tx=0x7f905c009e80 comp rx=0 tx=0).ready entity=mgr.24923 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:12:29.793 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.787+0000 7f905bfff640 1 --2- 172.21.15.47:0/2188865572 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564361ba87b0 0x564361baaba0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f9054009c20 tx=0x7f9054011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=698ab518fcc5d8bb server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:12:29.793 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.787+0000 7f904affd640 1 -- 172.21.15.47:0/2188865572 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9054004230 con 0x564361ba87b0 2024-04-17T06:12:29.793 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.787+0000 7f904affd640 1 -- 172.21.15.47:0/2188865572 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90540043d0 con 0x564361ba87b0 2024-04-17T06:12:29.793 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.787+0000 7f904affd640 1 -- 172.21.15.47:0/2188865572 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9054004d40 con 0x564361ba87b0 2024-04-17T06:12:29.793 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.788+0000 7f906d7bc740 1 -- 172.21.15.47:0/2188865572 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564361ba87b0 msgr2=0x564361baaba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:12:29.793 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.788+0000 7f906d7bc740 1 --2- 172.21.15.47:0/2188865572 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564361ba87b0 0x564361baaba0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f9054009c20 tx=0x7f9054011810 comp rx=0 tx=0).stop 2024-04-17T06:12:29.793 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.788+0000 7f906d7bc740 1 -- 172.21.15.47:0/2188865572 shutdown_connections 2024-04-17T06:12:29.793 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.788+0000 7f906d7bc740 1 --2- 172.21.15.47:0/2188865572 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564361ba87b0 0x564361baaba0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:29.794 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.788+0000 7f906d7bc740 1 --2- 172.21.15.47:0/2188865572 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564361b9f9e0 0x564361ba8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:29.794 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.788+0000 7f906d7bc740 1 --2- 172.21.15.47:0/2188865572 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564361b9f0c0 0x564361b9f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:29.794 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.788+0000 7f906d7bc740 1 -- 172.21.15.47:0/2188865572 >> 172.21.15.47:0/2188865572 conn(0x564361b999e0 msgr2=0x564361b99dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:12:29.794 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.788+0000 7f906d7bc740 1 -- 172.21.15.47:0/2188865572 shutdown_connections 2024-04-17T06:12:29.794 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.788+0000 7f906d7bc740 1 -- 172.21.15.47:0/2188865572 wait complete. 2024-04-17T06:12:29.794 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.788+0000 7f906d7bc740 1 -- start start 2024-04-17T06:12:29.794 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.788+0000 7f906d7bc740 1 Processor -- start 2024-04-17T06:12:29.794 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.789+0000 7f906d7bc740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564361b9f0c0 0x564361ba57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:29.794 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.789+0000 7f906d7bc740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564361b9f9e0 0x564361ba5d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:29.794 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.789+0000 7f906d7bc740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564361ba87b0 0x564361ba6250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:29.794 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.789+0000 7f906d7bc740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x564361bab3e0 con 0x564361ba87b0 2024-04-17T06:12:29.794 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.789+0000 7f906d7bc740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x564361a8ce90 con 0x564361b9f0c0 2024-04-17T06:12:29.794 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.789+0000 7f906d7bc740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x564361bab560 con 0x564361b9f9e0 2024-04-17T06:12:29.795 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.789+0000 7f906d7bc740 10 client.0 _ll_register_callbacks cb 0x564361aa2d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:12:29.795 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.789+0000 7f906916a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564361ba87b0 0x564361ba6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:12:29.795 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.789+0000 7f906916a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564361ba87b0 0x564361ba6250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39650/0 (socket says 172.21.15.47:39650) 2024-04-17T06:12:29.795 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.789+0000 7f906916a640 1 -- 172.21.15.47:0/941877467 learned_addr learned my addr 172.21.15.47:0/941877467 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:12:29.795 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.789+0000 7f9068969640 1 --2- 172.21.15.47:0/941877467 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564361b9f0c0 0x564361ba57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:12:29.795 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.789+0000 7f905bfff640 1 --2- 172.21.15.47:0/941877467 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564361b9f9e0 0x564361ba5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:12:29.795 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.789+0000 7f906916a640 1 -- 172.21.15.47:0/941877467 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564361b9f9e0 msgr2=0x564361ba5d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:12:29.795 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.789+0000 7f906916a640 1 --2- 172.21.15.47:0/941877467 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564361b9f9e0 0x564361ba5d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:29.795 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.789+0000 7f906916a640 1 -- 172.21.15.47:0/941877467 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564361b9f0c0 msgr2=0x564361ba57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:12:29.795 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.789+0000 7f906916a640 1 --2- 172.21.15.47:0/941877467 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564361b9f0c0 0x564361ba57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:29.795 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.789+0000 7f906916a640 1 -- 172.21.15.47:0/941877467 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643618e3620 con 0x564361ba87b0 2024-04-17T06:12:29.795 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.789+0000 7f9068969640 1 --2- 172.21.15.47:0/941877467 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564361b9f0c0 0x564361ba57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:12:29.795 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.789+0000 7f906916a640 1 --2- 172.21.15.47:0/941877467 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564361ba87b0 0x564361ba6250 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f905c009dc0 tx=0x7f905c01d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e47d2f721b7ac976 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:12:29.795 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.790+0000 7f9043fff640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-17T06:12:29.795 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.790+0000 7f9043fff640 1 -- 172.21.15.47:0/941877467 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f905c024070 con 0x564361ba87b0 2024-04-17T06:12:29.796 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.790+0000 7f9043fff640 1 -- 172.21.15.47:0/941877467 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8469+0+0 (secure 0 0 0) 0x7f905c0047c0 con 0x564361ba87b0 2024-04-17T06:12:29.796 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.790+0000 7f906d7bc740 1 -- 172.21.15.47:0/941877467 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90540043d0 con 0x564361ba87b0 2024-04-17T06:12:29.796 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.790+0000 7f9043fff640 1 -- 172.21.15.47:0/941877467 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f905c01f450 con 0x564361ba87b0 2024-04-17T06:12:29.796 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.790+0000 7f906d7bc740 10 client.16329 fetch_fsmap learned FSMap version 36 2024-04-17T06:12:29.796 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.790+0000 7f906d7bc740 1 -- 172.21.15.47:0/941877467 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x564361bb7620 con 0x564361ba87b0 2024-04-17T06:12:29.796 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.790+0000 7f9043fff640 1 -- 172.21.15.47:0/941877467 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90540043d0 con 0x564361ba87b0 2024-04-17T06:12:29.796 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.790+0000 7f9043fff640 1 -- 172.21.15.47:0/941877467 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 36) ==== 8339+0+0 (secure 0 0 0) 0x7f905c028520 con 0x564361ba87b0 2024-04-17T06:12:29.797 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.790+0000 7f906d7bc740 10 client.16329 fetch_fsmap finished waiting for FSMap version 36 2024-04-17T06:12:29.797 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.790+0000 7f906d7bc740 10 client.16329 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-17T06:12:29.797 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.790+0000 7f906d7bc740 20 client.16329 populate_metadata read hostname 'smithi047' 2024-04-17T06:12:29.797 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.790+0000 7f906d7bc740 1 --2- 172.21.15.47:0/941877467 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x564361bb8ba0 0x564361bbafd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:29.797 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.790+0000 7f906d7bc740 4 client.16329 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T06:12:29.797 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.790+0000 7f906d7bc740 1 -- 172.21.15.47:0/941877467 --> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564361b121f0 con 0x564361bb8ba0 2024-04-17T06:12:29.798 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.791+0000 7f9068969640 1 --2- 172.21.15.47:0/941877467 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x564361bb8ba0 0x564361bbafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:12:29.798 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.791+0000 7f9043fff640 10 client.16329 ms_handle_connect on v2:172.21.15.98:6838/2841287917 2024-04-17T06:12:29.798 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.791+0000 7f9068969640 1 --2- 172.21.15.47:0/941877467 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x564361bb8ba0 0x564361bbafd0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=292bee0be183a100 server_cookie=124144972cf76d06 in_seq=0 out_seq=0 2024-04-17T06:12:29.798 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.792+0000 7f9043fff640 1 -- 172.21.15.47:0/941877467 <== mds.0 v2:172.21.15.98:6838/2841287917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564361b121f0 con 0x564361bb8ba0 2024-04-17T06:12:29.798 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.792+0000 7f9043fff640 10 client.16329 handle_command_reply: tid=0 2024-04-17T06:12:29.798 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.792+0000 7f906d7bc740 1 client.16329 shutdown 2024-04-17T06:12:29.798 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.792+0000 7f906d7bc740 1 -- 172.21.15.47:0/941877467 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564361ba87b0 msgr2=0x564361ba6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:12:29.798 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.792+0000 7f906d7bc740 1 --2- 172.21.15.47:0/941877467 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564361ba87b0 0x564361ba6250 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f905c009dc0 tx=0x7f905c01d000 comp rx=0 tx=0).stop 2024-04-17T06:12:29.798 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.792+0000 7f906d7bc740 1 -- 172.21.15.47:0/941877467 shutdown_connections 2024-04-17T06:12:29.798 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.792+0000 7f906d7bc740 1 --2- 172.21.15.47:0/941877467 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x564361bb8ba0 0x564361bbafd0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:29.798 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.792+0000 7f906d7bc740 1 --2- 172.21.15.47:0/941877467 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564361ba87b0 0x564361ba6250 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:29.798 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.792+0000 7f906d7bc740 1 --2- 172.21.15.47:0/941877467 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564361b9f9e0 0x564361ba5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:29.798 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.792+0000 7f906d7bc740 1 --2- 172.21.15.47:0/941877467 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564361b9f0c0 0x564361ba57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:29.798 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.792+0000 7f9043fff640 0 client.16329 ms_handle_reset on v2:172.21.15.98:6838/2841287917 2024-04-17T06:12:29.799 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.792+0000 7f906d7bc740 1 -- 172.21.15.47:0/941877467 >> 172.21.15.47:0/941877467 conn(0x564361b999e0 msgr2=0x564361b9b730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:12:29.799 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.792+0000 7f906d7bc740 1 -- 172.21.15.47:0/941877467 shutdown_connections 2024-04-17T06:12:29.799 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.792+0000 7f906d7bc740 1 -- 172.21.15.47:0/941877467 wait complete. 2024-04-17T06:12:29.799 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.792+0000 7f906d7bc740 20 client.16329 trim_cache size 0 max 16384 2024-04-17T06:12:29.810 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.807+0000 7f906d7bc740 1 Processor -- start 2024-04-17T06:12:29.810 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.807+0000 7f906d7bc740 1 -- start start 2024-04-17T06:12:29.810 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.807+0000 7f906d7bc740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564361b9f0c0 0x564361ba7f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:29.810 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.807+0000 7f906d7bc740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564361b9f9e0 0x564361bb42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:29.810 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.807+0000 7f906d7bc740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564361ba87b0 0x564361b94a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:29.810 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.807+0000 7f906d7bc740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f90641b03b0 con 0x564361ba87b0 2024-04-17T06:12:29.810 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.807+0000 7f906d7bc740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f90641adc70 con 0x564361b9f0c0 2024-04-17T06:12:29.810 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.807+0000 7f906d7bc740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f90641abac0 con 0x564361b9f9e0 2024-04-17T06:12:29.811 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.807+0000 7f906916a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564361ba87b0 0x564361b94a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:12:29.811 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.807+0000 7f906916a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564361ba87b0 0x564361b94a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39664/0 (socket says 172.21.15.47:39664) 2024-04-17T06:12:29.811 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.807+0000 7f906916a640 1 -- 172.21.15.47:0/735088520 learned_addr learned my addr 172.21.15.47:0/735088520 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:12:29.811 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.808+0000 7f906916a640 1 -- 172.21.15.47:0/735088520 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564361b9f9e0 msgr2=0x564361bb42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:12:29.811 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.808+0000 7f906916a640 1 --2- 172.21.15.47:0/735088520 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564361b9f9e0 0x564361bb42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:29.811 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.808+0000 7f906916a640 1 -- 172.21.15.47:0/735088520 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564361b9f0c0 msgr2=0x564361ba7f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:12:29.811 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.808+0000 7f9068969640 1 --2- 172.21.15.47:0/735088520 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564361b9f0c0 0x564361ba7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:12:29.811 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.808+0000 7f906916a640 1 --2- 172.21.15.47:0/735088520 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564361b9f0c0 0x564361ba7f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:29.811 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.808+0000 7f906916a640 1 -- 172.21.15.47:0/735088520 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564361ba4780 con 0x564361ba87b0 2024-04-17T06:12:29.811 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.808+0000 7f906916a640 1 --2- 172.21.15.47:0/735088520 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564361ba87b0 0x564361b94a60 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f905c011a50 tx=0x7f905c009ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79044f22afbf8d7f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:12:29.811 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.808+0000 7f904a7fc640 1 -- 172.21.15.47:0/735088520 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f905c024040 con 0x564361ba87b0 2024-04-17T06:12:29.811 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.808+0000 7f904a7fc640 1 -- 172.21.15.47:0/735088520 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f905c004400 con 0x564361ba87b0 2024-04-17T06:12:29.811 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.808+0000 7f904a7fc640 1 -- 172.21.15.47:0/735088520 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f905c0286f0 con 0x564361ba87b0 2024-04-17T06:12:29.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.808+0000 7f906d7bc740 1 -- 172.21.15.47:0/735088520 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564361ba87b0 msgr2=0x564361b94a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:12:29.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.808+0000 7f906d7bc740 1 --2- 172.21.15.47:0/735088520 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564361ba87b0 0x564361b94a60 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f905c011a50 tx=0x7f905c009ef0 comp rx=0 tx=0).stop 2024-04-17T06:12:29.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.809+0000 7f906d7bc740 1 -- 172.21.15.47:0/735088520 shutdown_connections 2024-04-17T06:12:29.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.809+0000 7f906d7bc740 1 --2- 172.21.15.47:0/735088520 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564361ba87b0 0x564361b94a60 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:29.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.809+0000 7f906d7bc740 1 --2- 172.21.15.47:0/735088520 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564361b9f9e0 0x564361bb42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:29.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.809+0000 7f906d7bc740 1 --2- 172.21.15.47:0/735088520 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564361b9f0c0 0x564361ba7f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:29.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.809+0000 7f906d7bc740 1 -- 172.21.15.47:0/735088520 >> 172.21.15.47:0/735088520 conn(0x564361b999e0 msgr2=0x564361b98b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:12:29.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.809+0000 7f906d7bc740 1 -- 172.21.15.47:0/735088520 shutdown_connections 2024-04-17T06:12:29.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.809+0000 7f906d7bc740 1 -- 172.21.15.47:0/735088520 wait complete. 2024-04-17T06:12:29.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.809+0000 7f906d7bc740 1 -- start start 2024-04-17T06:12:29.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.809+0000 7f906d7bc740 1 Processor -- start 2024-04-17T06:12:29.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.809+0000 7f906d7bc740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564361b9f0c0 0x564361ba2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:29.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.809+0000 7f906d7bc740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564361b9f9e0 0x564361ba30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:29.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.809+0000 7f906d7bc740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564361ba87b0 0x564361c1e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:29.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.809+0000 7f906d7bc740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f90641ac130 con 0x564361ba87b0 2024-04-17T06:12:29.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.809+0000 7f906d7bc740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f90641ad5a0 con 0x564361b9f9e0 2024-04-17T06:12:29.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.809+0000 7f906d7bc740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f90641ac800 con 0x564361b9f0c0 2024-04-17T06:12:29.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.809+0000 7f906d7bc740 10 client.0 _ll_register_callbacks cb 0x7f905c009d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:12:29.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.810+0000 7f906916a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564361ba87b0 0x564361c1e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:12:29.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.810+0000 7f906916a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564361ba87b0 0x564361c1e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39674/0 (socket says 172.21.15.47:39674) 2024-04-17T06:12:29.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.810+0000 7f906916a640 1 -- 172.21.15.47:0/1566920391 learned_addr learned my addr 172.21.15.47:0/1566920391 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:12:29.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.810+0000 7f9068969640 1 --2- 172.21.15.47:0/1566920391 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564361b9f0c0 0x564361ba2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:12:29.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.810+0000 7f905bfff640 1 --2- 172.21.15.47:0/1566920391 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564361b9f9e0 0x564361ba30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:12:29.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.810+0000 7f906916a640 1 -- 172.21.15.47:0/1566920391 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564361b9f0c0 msgr2=0x564361ba2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:12:29.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.810+0000 7f906916a640 1 --2- 172.21.15.47:0/1566920391 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564361b9f0c0 0x564361ba2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:29.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.810+0000 7f906916a640 1 -- 172.21.15.47:0/1566920391 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564361b9f9e0 msgr2=0x564361ba30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:12:29.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.810+0000 7f906916a640 1 --2- 172.21.15.47:0/1566920391 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564361b9f9e0 0x564361ba30d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:29.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.810+0000 7f906916a640 1 -- 172.21.15.47:0/1566920391 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564361c1e690 con 0x564361ba87b0 2024-04-17T06:12:29.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.810+0000 7f906916a640 1 --2- 172.21.15.47:0/1566920391 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564361ba87b0 0x564361c1e150 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f905c009dc0 tx=0x7f905c028bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2b7772f8b0ad5a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:12:29.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.810+0000 7f9043fff640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-17T06:12:29.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.810+0000 7f9043fff640 1 -- 172.21.15.47:0/1566920391 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f905c004b20 con 0x564361ba87b0 2024-04-17T06:12:29.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.810+0000 7f9043fff640 1 -- 172.21.15.47:0/1566920391 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8469+0+0 (secure 0 0 0) 0x7f905c004cc0 con 0x564361ba87b0 2024-04-17T06:12:29.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.810+0000 7f906d7bc740 1 -- 172.21.15.47:0/1566920391 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f905c004400 con 0x564361ba87b0 2024-04-17T06:12:29.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.810+0000 7f9043fff640 1 -- 172.21.15.47:0/1566920391 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f905c01f5f0 con 0x564361ba87b0 2024-04-17T06:12:29.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.810+0000 7f906d7bc740 10 client.16335 fetch_fsmap learned FSMap version 36 2024-04-17T06:12:29.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.810+0000 7f906d7bc740 1 -- 172.21.15.47:0/1566920391 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x564361c1f940 con 0x564361ba87b0 2024-04-17T06:12:29.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.811+0000 7f9043fff640 1 -- 172.21.15.47:0/1566920391 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f905c004400 con 0x564361ba87b0 2024-04-17T06:12:29.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.811+0000 7f9043fff640 1 -- 172.21.15.47:0/1566920391 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 36) ==== 8339+0+0 (secure 0 0 0) 0x7f905c031660 con 0x564361ba87b0 2024-04-17T06:12:29.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.811+0000 7f906d7bc740 10 client.16335 fetch_fsmap finished waiting for FSMap version 36 2024-04-17T06:12:29.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.811+0000 7f906d7bc740 10 client.16335 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-17T06:12:29.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.811+0000 7f906d7bc740 20 client.16335 populate_metadata read hostname 'smithi047' 2024-04-17T06:12:29.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.811+0000 7f906d7bc740 1 --2- 172.21.15.47:0/1566920391 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x564361c1ffa0 0x564361c223c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:29.815 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.811+0000 7f906d7bc740 4 client.16335 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-17T06:12:29.815 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.811+0000 7f906d7bc740 1 -- 172.21.15.47:0/1566920391 --> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564361b0c060 con 0x564361c1ffa0 2024-04-17T06:12:29.815 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.811+0000 7f9068969640 1 --2- 172.21.15.47:0/1566920391 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x564361c1ffa0 0x564361c223c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:12:29.815 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.812+0000 7f9068969640 1 --2- 172.21.15.47:0/1566920391 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x564361c1ffa0 0x564361c223c0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc5adca2312e60fe server_cookie=398093d26ad030a1 in_seq=0 out_seq=0 2024-04-17T06:12:29.815 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.812+0000 7f9043fff640 10 client.16335 ms_handle_connect on v2:172.21.15.98:6838/2841287917 2024-04-17T06:12:29.820 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.820+0000 7f9043fff640 1 -- 172.21.15.47:0/1566920391 <== mds.0 v2:172.21.15.98:6838/2841287917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564361b0c060 con 0x564361c1ffa0 2024-04-17T06:12:29.820 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.820+0000 7f9043fff640 10 client.16335 handle_command_reply: tid=0 2024-04-17T06:12:29.820 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.820+0000 7f906d7bc740 1 client.16335 shutdown 2024-04-17T06:12:29.820 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.820+0000 7f906d7bc740 1 -- 172.21.15.47:0/1566920391 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564361ba87b0 msgr2=0x564361c1e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:12:29.820 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.820+0000 7f906d7bc740 1 --2- 172.21.15.47:0/1566920391 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564361ba87b0 0x564361c1e150 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f905c009dc0 tx=0x7f905c028bb0 comp rx=0 tx=0).stop 2024-04-17T06:12:29.820 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.820+0000 7f906d7bc740 1 -- 172.21.15.47:0/1566920391 shutdown_connections 2024-04-17T06:12:29.820 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.820+0000 7f906d7bc740 1 --2- 172.21.15.47:0/1566920391 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x564361c1ffa0 0x564361c223c0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:29.820 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.820+0000 7f906d7bc740 1 --2- 172.21.15.47:0/1566920391 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564361ba87b0 0x564361c1e150 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:29.820 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.820+0000 7f906d7bc740 1 --2- 172.21.15.47:0/1566920391 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564361b9f9e0 0x564361ba30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:29.820 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.820+0000 7f906d7bc740 1 --2- 172.21.15.47:0/1566920391 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564361b9f0c0 0x564361ba2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:29.820 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.820+0000 7f9043fff640 0 client.16335 ms_handle_reset on v2:172.21.15.98:6838/2841287917 2024-04-17T06:12:29.821 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.820+0000 7f906d7bc740 1 -- 172.21.15.47:0/1566920391 >> 172.21.15.47:0/1566920391 conn(0x564361b999e0 msgr2=0x564361ba5790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:12:29.821 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.821+0000 7f906d7bc740 1 -- 172.21.15.47:0/1566920391 shutdown_connections 2024-04-17T06:12:29.821 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.821+0000 7f906d7bc740 1 -- 172.21.15.47:0/1566920391 wait complete. 2024-04-17T06:12:29.821 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.821+0000 7f906d7bc740 20 client.16335 trim_cache size 0 max 16384 2024-04-17T06:12:29.821 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-04-17T06:12:29.821 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-04-17T06:12:29.821 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-04-17T06:12:29.821 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-04-17T06:12:29.822 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.821+0000 7f906abcd640 1 -- 172.21.15.47:0/814976913 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f904c0a3dc0 msgr2=0x7f904c0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:12:29.822 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.821+0000 7f906abcd640 1 --2- 172.21.15.47:0/814976913 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f904c0a3dc0 0x7f904c0a6280 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f905c005fd0 tx=0x7f905c009e80 comp rx=0 tx=0).stop 2024-04-17T06:12:29.822 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.821+0000 7f906abcd640 1 -- 172.21.15.47:0/814976913 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9064101100 msgr2=0x7f906419e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:12:29.822 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.821+0000 7f906abcd640 1 --2- 172.21.15.47:0/814976913 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9064101100 0x7f906419e430 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f9060006010 tx=0x7f906002cb30 comp rx=0 tx=0).stop 2024-04-17T06:12:29.822 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.821+0000 7f906abcd640 1 -- 172.21.15.47:0/814976913 shutdown_connections 2024-04-17T06:12:29.823 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.821+0000 7f906abcd640 1 --2- 172.21.15.47:0/814976913 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f90641985e0 0x7f9064198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:29.823 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.821+0000 7f906abcd640 1 --2- 172.21.15.47:0/814976913 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9064106180 0x7f906419e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:29.823 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.821+0000 7f906abcd640 1 --2- 172.21.15.47:0/814976913 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f904c0a3dc0 0x7f904c0a6280 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:29.823 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.821+0000 7f906abcd640 1 --2- 172.21.15.47:0/814976913 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9064101100 0x7f906419e430 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:29.823 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.821+0000 7f906abcd640 1 -- 172.21.15.47:0/814976913 >> 172.21.15.47:0/814976913 conn(0x7f9064077fe0 msgr2=0x7f9064069920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:12:29.823 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.821+0000 7f906abcd640 1 -- 172.21.15.47:0/814976913 shutdown_connections 2024-04-17T06:12:29.823 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:29.821+0000 7f906abcd640 1 -- 172.21.15.47:0/814976913 wait complete. 2024-04-17T06:12:30.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:12:29 smithi047 ceph-mon[32223]: pgmap v353: 97 pgs: 97 active+clean; 5.6 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:12:30.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:12:29 smithi103 ceph-mon[36243]: pgmap v353: 97 pgs: 97 active+clean; 5.6 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:12:30.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:12:29 smithi098 ceph-mon[36516]: pgmap v353: 97 pgs: 97 active+clean; 5.6 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:12:32.185 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:12:31 smithi098 ceph-mon[36516]: pgmap v354: 97 pgs: 97 active+clean; 5.6 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-04-17T06:12:32.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:12:31 smithi103 ceph-mon[36243]: pgmap v354: 97 pgs: 97 active+clean; 5.6 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-04-17T06:12:32.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:12:31 smithi047 ceph-mon[32223]: pgmap v354: 97 pgs: 97 active+clean; 5.6 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-04-17T06:12:34.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:12:33 smithi103 ceph-mon[36243]: pgmap v355: 97 pgs: 97 active+clean; 5.6 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.6 KiB/s wr, 9 op/s 2024-04-17T06:12:34.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:12:33 smithi098 ceph-mon[36516]: pgmap v355: 97 pgs: 97 active+clean; 5.6 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.6 KiB/s wr, 9 op/s 2024-04-17T06:12:34.388 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:12:33 smithi047 ceph-mon[32223]: pgmap v355: 97 pgs: 97 active+clean; 5.6 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.6 KiB/s wr, 9 op/s 2024-04-17T06:12:35.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:12:34 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T06:12:35.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:12:34 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:12:35.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:12:34 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:12:35.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:12:34 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T06:12:35.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:12:34 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:12:35.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:12:34 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:12:35.388 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:12:34 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T06:12:35.388 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:12:34 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:12:35.388 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:12:34 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:12:36.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:12:35 smithi103 ceph-mon[36243]: pgmap v356: 97 pgs: 97 active+clean; 5.6 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-04-17T06:12:36.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:12:35 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T06:12:36.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:12:35 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T06:12:36.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:12:35 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:12:36.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:12:35 smithi098 ceph-mon[36516]: pgmap v356: 97 pgs: 97 active+clean; 5.6 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-04-17T06:12:36.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:12:35 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T06:12:36.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:12:35 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T06:12:36.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:12:35 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:12:36.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:12:35 smithi047 ceph-mon[32223]: pgmap v356: 97 pgs: 97 active+clean; 5.6 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-04-17T06:12:36.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:12:35 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T06:12:36.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:12:35 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T06:12:36.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:12:35 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:12:38.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:12:37 smithi103 ceph-mon[36243]: pgmap v357: 97 pgs: 97 active+clean; 5.6 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:12:38.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:12:37 smithi098 ceph-mon[36516]: pgmap v357: 97 pgs: 97 active+clean; 5.6 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:12:38.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:12:37 smithi047 ceph-mon[32223]: pgmap v357: 97 pgs: 97 active+clean; 5.6 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:12:40.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:12:39 smithi103 ceph-mon[36243]: pgmap v358: 97 pgs: 97 active+clean; 5.6 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:12:40.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:12:39 smithi098 ceph-mon[36516]: pgmap v358: 97 pgs: 97 active+clean; 5.6 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:12:40.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:12:39 smithi047 ceph-mon[32223]: pgmap v358: 97 pgs: 97 active+clean; 5.6 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:12:42.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:12:41 smithi103 ceph-mon[36243]: pgmap v359: 97 pgs: 97 active+clean; 5.7 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-04-17T06:12:42.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:12:41 smithi098 ceph-mon[36516]: pgmap v359: 97 pgs: 97 active+clean; 5.7 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-04-17T06:12:42.388 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:12:41 smithi047 ceph-mon[32223]: pgmap v359: 97 pgs: 97 active+clean; 5.7 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-04-17T06:12:42.922 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.922+0000 7f37167a8640 1 -- 172.21.15.47:0/976842458 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3710107340 msgr2=0x7f371010eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:12:42.922 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.922+0000 7f37167a8640 1 --2- 172.21.15.47:0/976842458 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3710107340 0x7f371010eea0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f3708009cd0 tx=0x7f370802c140 comp rx=0 tx=0).stop 2024-04-17T06:12:42.922 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.922+0000 7f37167a8640 1 -- 172.21.15.47:0/976842458 shutdown_connections 2024-04-17T06:12:42.922 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.922+0000 7f37167a8640 1 --2- 172.21.15.47:0/976842458 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3710107340 0x7f371010eea0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:42.923 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.922+0000 7f37167a8640 1 --2- 172.21.15.47:0/976842458 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f37101010e0 0x7f3710106c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:42.923 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.922+0000 7f37167a8640 1 --2- 172.21.15.47:0/976842458 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3710100730 0x7f3710100b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:42.923 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.922+0000 7f37167a8640 1 -- 172.21.15.47:0/976842458 >> 172.21.15.47:0/976842458 conn(0x7f37100773a0 msgr2=0x7f37100fe770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:12:42.923 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.922+0000 7f37167a8640 1 -- 172.21.15.47:0/976842458 shutdown_connections 2024-04-17T06:12:42.923 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.922+0000 7f37167a8640 1 -- 172.21.15.47:0/976842458 wait complete. 2024-04-17T06:12:42.923 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.922+0000 7f37167a8640 1 Processor -- start 2024-04-17T06:12:42.923 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.923+0000 7f37167a8640 1 -- start start 2024-04-17T06:12:42.923 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.923+0000 7f37167a8640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3710100730 0x7f37101987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:42.923 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.923+0000 7f37167a8640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3710107340 0x7f3710198ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:42.923 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.923+0000 7f37167a8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f371019d010 0x7f371019d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:42.923 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.923+0000 7f37167a8640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f3710112360 con 0x7f3710107340 2024-04-17T06:12:42.923 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.923+0000 7f37167a8640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f37101121e0 con 0x7f3710100730 2024-04-17T06:12:42.923 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.923+0000 7f37167a8640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f37101124e0 con 0x7f371019d010 2024-04-17T06:12:42.923 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.923+0000 7f3707fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3710107340 0x7f3710198ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:12:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.923+0000 7f3707fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3710107340 0x7f3710198ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58842/0 (socket says 172.21.15.47:58842) 2024-04-17T06:12:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.923+0000 7f3707fff640 1 -- 172.21.15.47:0/3650008317 learned_addr learned my addr 172.21.15.47:0/3650008317 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:12:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.923+0000 7f370ffff640 1 --2- 172.21.15.47:0/3650008317 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3710100730 0x7f37101987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:12:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.923+0000 7f3714d45640 1 --2- 172.21.15.47:0/3650008317 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f371019d010 0x7f371019d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:12:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.923+0000 7f3707fff640 1 -- 172.21.15.47:0/3650008317 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f371019d010 msgr2=0x7f371019d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:12:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.923+0000 7f3707fff640 1 --2- 172.21.15.47:0/3650008317 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f371019d010 0x7f371019d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.923+0000 7f3707fff640 1 -- 172.21.15.47:0/3650008317 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3710100730 msgr2=0x7f37101987a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:12:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.923+0000 7f3707fff640 1 --2- 172.21.15.47:0/3650008317 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3710100730 0x7f37101987a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.923+0000 7f3707fff640 1 -- 172.21.15.47:0/3650008317 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f371019db00 con 0x7f3710107340 2024-04-17T06:12:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.923+0000 7f3707fff640 1 --2- 172.21.15.47:0/3650008317 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3710107340 0x7f3710198ce0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f36f800c970 tx=0x7f36f800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:12:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.924+0000 7f370dffb640 1 -- 172.21.15.47:0/3650008317 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f8007bb0 con 0x7f3710107340 2024-04-17T06:12:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.924+0000 7f370dffb640 1 -- 172.21.15.47:0/3650008317 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36f8007d50 con 0x7f3710107340 2024-04-17T06:12:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.924+0000 7f37167a8640 1 -- 172.21.15.47:0/3650008317 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f371019ddf0 con 0x7f3710107340 2024-04-17T06:12:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.924+0000 7f370dffb640 1 -- 172.21.15.47:0/3650008317 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f80187a0 con 0x7f3710107340 2024-04-17T06:12:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.924+0000 7f37167a8640 1 -- 172.21.15.47:0/3650008317 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37101a5590 con 0x7f3710107340 2024-04-17T06:12:42.932 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.926+0000 7f370dffb640 1 -- 172.21.15.47:0/3650008317 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f36f800f450 con 0x7f3710107340 2024-04-17T06:12:42.933 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.927+0000 7f371938c740 1 Processor -- start 2024-04-17T06:12:42.933 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.927+0000 7f371938c740 1 -- start start 2024-04-17T06:12:42.933 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.927+0000 7f371938c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5577e6a940c0 0x5577e6a944a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:42.933 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.927+0000 7f371938c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5577e6a949e0 0x5577e6a9d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:42.933 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.927+0000 7f371938c740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5577e6a9d7b0 0x5577e6a9fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:42.933 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.927+0000 7f371938c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5577e69f97d0 con 0x5577e6a940c0 2024-04-17T06:12:42.933 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.927+0000 7f371938c740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5577e6996f70 con 0x5577e6a9d7b0 2024-04-17T06:12:42.933 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.927+0000 7f371938c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5577e69f5790 con 0x5577e6a949e0 2024-04-17T06:12:42.933 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.927+0000 7f3714d45640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5577e6a940c0 0x5577e6a944a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:12:42.933 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.927+0000 7f3714d45640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5577e6a940c0 0x5577e6a944a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58850/0 (socket says 172.21.15.47:58850) 2024-04-17T06:12:42.933 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.927+0000 7f3714d45640 1 -- 172.21.15.47:0/1850979257 learned_addr learned my addr 172.21.15.47:0/1850979257 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:12:42.933 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.927+0000 7f3707fff640 1 --2- 172.21.15.47:0/1850979257 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5577e6a9d7b0 0x5577e6a9fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:12:42.933 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.927+0000 7f370ffff640 1 --2- 172.21.15.47:0/1850979257 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5577e6a949e0 0x5577e6a9d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:12:42.933 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.927+0000 7f370dffb640 1 --2- 172.21.15.47:0/3650008317 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f36fc0a3dc0 0x7f36fc0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:42.934 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.928+0000 7f3714d45640 1 -- 172.21.15.47:0/1850979257 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5577e6a949e0 msgr2=0x5577e6a9d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:12:42.934 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.928+0000 7f3714d45640 1 --2- 172.21.15.47:0/1850979257 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5577e6a949e0 0x5577e6a9d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:42.934 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.928+0000 7f3714d45640 1 -- 172.21.15.47:0/1850979257 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5577e6a9d7b0 msgr2=0x5577e6a9fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:12:42.934 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.928+0000 7f3714d45640 1 --2- 172.21.15.47:0/1850979257 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5577e6a9d7b0 0x5577e6a9fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:42.934 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.928+0000 7f3714d45640 1 -- 172.21.15.47:0/1850979257 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577e670c210 con 0x5577e6a940c0 2024-04-17T06:12:42.934 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.928+0000 7f370dffb640 1 -- 172.21.15.47:0/3650008317 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8469+0+0 (secure 0 0 0) 0x7f36f80d0790 con 0x7f3710107340 2024-04-17T06:12:42.934 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.928+0000 7f3714d45640 1 --2- 172.21.15.47:0/3650008317 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f36fc0a3dc0 0x7f36fc0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:12:42.934 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.928+0000 7f3714d45640 1 --2- 172.21.15.47:0/1850979257 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5577e6a940c0 0x5577e6a944a0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f370802c750 tx=0x7f3708009a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88f06e641dff6cea server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:12:42.934 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.928+0000 7f37067fc640 1 -- 172.21.15.47:0/1850979257 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3708039040 con 0x5577e6a940c0 2024-04-17T06:12:42.934 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.928+0000 7f37067fc640 1 -- 172.21.15.47:0/1850979257 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3708003c30 con 0x5577e6a940c0 2024-04-17T06:12:42.934 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.928+0000 7f37067fc640 1 -- 172.21.15.47:0/1850979257 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37080346a0 con 0x5577e6a940c0 2024-04-17T06:12:42.934 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.929+0000 7f371938c740 1 -- 172.21.15.47:0/1850979257 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5577e6a940c0 msgr2=0x5577e6a944a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:12:42.934 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.929+0000 7f371938c740 1 --2- 172.21.15.47:0/1850979257 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5577e6a940c0 0x5577e6a944a0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f370802c750 tx=0x7f3708009a10 comp rx=0 tx=0).stop 2024-04-17T06:12:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.929+0000 7f3714d45640 1 --2- 172.21.15.47:0/3650008317 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f36fc0a3dc0 0x7f36fc0a6280 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f370804c000 tx=0x7f3708045000 comp rx=0 tx=0).ready entity=mgr.24923 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:12:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.929+0000 7f371938c740 1 -- 172.21.15.47:0/1850979257 shutdown_connections 2024-04-17T06:12:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.929+0000 7f371938c740 1 --2- 172.21.15.47:0/1850979257 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5577e6a9d7b0 0x5577e6a9fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.929+0000 7f371938c740 1 --2- 172.21.15.47:0/1850979257 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5577e6a949e0 0x5577e6a9d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.929+0000 7f371938c740 1 --2- 172.21.15.47:0/1850979257 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5577e6a940c0 0x5577e6a944a0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.929+0000 7f371938c740 1 -- 172.21.15.47:0/1850979257 >> 172.21.15.47:0/1850979257 conn(0x5577e6a8e9e0 msgr2=0x5577e6a8edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:12:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.929+0000 7f371938c740 1 -- 172.21.15.47:0/1850979257 shutdown_connections 2024-04-17T06:12:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.929+0000 7f371938c740 1 -- 172.21.15.47:0/1850979257 wait complete. 2024-04-17T06:12:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.929+0000 7f371938c740 1 -- start start 2024-04-17T06:12:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.930+0000 7f371938c740 1 Processor -- start 2024-04-17T06:12:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.930+0000 7f371938c740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5577e6a940c0 0x5577e6a9a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.930+0000 7f371938c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5577e6a949e0 0x5577e6a9ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.930+0000 7f371938c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5577e6a9d7b0 0x5577e6a9b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.930+0000 7f371938c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5577e6aa03e0 con 0x5577e6a9d7b0 2024-04-17T06:12:42.936 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.930+0000 7f371938c740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5577e6981e90 con 0x5577e6a940c0 2024-04-17T06:12:42.936 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.930+0000 7f371938c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5577e6aa0560 con 0x5577e6a949e0 2024-04-17T06:12:42.936 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.930+0000 7f371938c740 10 client.0 _ll_register_callbacks cb 0x5577e6997d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:12:42.936 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.930+0000 7f3714d45640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5577e6a9d7b0 0x5577e6a9b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:12:42.936 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.930+0000 7f3714d45640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5577e6a9d7b0 0x5577e6a9b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58858/0 (socket says 172.21.15.47:58858) 2024-04-17T06:12:42.936 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.930+0000 7f3714d45640 1 -- 172.21.15.47:0/2804087502 learned_addr learned my addr 172.21.15.47:0/2804087502 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:12:42.936 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.930+0000 7f370ffff640 1 --2- 172.21.15.47:0/2804087502 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5577e6a940c0 0x5577e6a9a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:12:42.936 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.930+0000 7f3707fff640 1 --2- 172.21.15.47:0/2804087502 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5577e6a949e0 0x5577e6a9ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:12:42.936 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.930+0000 7f3714d45640 1 -- 172.21.15.47:0/2804087502 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5577e6a949e0 msgr2=0x5577e6a9ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:12:42.936 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.930+0000 7f3714d45640 1 --2- 172.21.15.47:0/2804087502 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5577e6a949e0 0x5577e6a9ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:42.936 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.930+0000 7f3714d45640 1 -- 172.21.15.47:0/2804087502 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5577e6a940c0 msgr2=0x5577e6a9a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:12:42.936 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.930+0000 7f3714d45640 1 --2- 172.21.15.47:0/2804087502 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5577e6a940c0 0x5577e6a9a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:42.936 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.930+0000 7f3714d45640 1 -- 172.21.15.47:0/2804087502 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577e67d8620 con 0x5577e6a9d7b0 2024-04-17T06:12:42.936 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.930+0000 7f3707fff640 1 --2- 172.21.15.47:0/2804087502 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5577e6a949e0 0x5577e6a9ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:12:42.936 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.931+0000 7f3714d45640 1 --2- 172.21.15.47:0/2804087502 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5577e6a9d7b0 0x5577e6a9b2b0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f3708005c60 tx=0x7f3708009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8d7d2677f1126ec server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:12:42.937 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.931+0000 7f36ef7fe640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-17T06:12:42.937 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.931+0000 7f36ef7fe640 1 -- 172.21.15.47:0/2804087502 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3708057070 con 0x5577e6a9d7b0 2024-04-17T06:12:42.937 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.931+0000 7f36ef7fe640 1 -- 172.21.15.47:0/2804087502 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8469+0+0 (secure 0 0 0) 0x7f3708003de0 con 0x5577e6a9d7b0 2024-04-17T06:12:42.937 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.931+0000 7f371938c740 1 -- 172.21.15.47:0/2804087502 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3708003c30 con 0x5577e6a9d7b0 2024-04-17T06:12:42.937 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.931+0000 7f36ef7fe640 1 -- 172.21.15.47:0/2804087502 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f3708063050 con 0x5577e6a9d7b0 2024-04-17T06:12:42.937 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.931+0000 7f371938c740 10 client.16347 fetch_fsmap learned FSMap version 36 2024-04-17T06:12:42.937 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.931+0000 7f371938c740 1 -- 172.21.15.47:0/2804087502 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5577e6aac690 con 0x5577e6a9d7b0 2024-04-17T06:12:42.937 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.931+0000 7f36ef7fe640 1 -- 172.21.15.47:0/2804087502 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3708003c30 con 0x5577e6a9d7b0 2024-04-17T06:12:42.937 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.931+0000 7f36ef7fe640 1 -- 172.21.15.47:0/2804087502 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 36) ==== 8339+0+0 (secure 0 0 0) 0x7f370803f040 con 0x5577e6a9d7b0 2024-04-17T06:12:42.937 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.932+0000 7f371938c740 10 client.16347 fetch_fsmap finished waiting for FSMap version 36 2024-04-17T06:12:42.937 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.932+0000 7f371938c740 10 client.16347 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-17T06:12:42.937 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.932+0000 7f371938c740 20 client.16347 populate_metadata read hostname 'smithi047' 2024-04-17T06:12:42.937 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.932+0000 7f371938c740 1 --2- 172.21.15.47:0/2804087502 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x5577e6aad9e0 0x5577e6aafdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:42.937 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.932+0000 7f371938c740 4 client.16347 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T06:12:42.937 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.932+0000 7f371938c740 1 -- 172.21.15.47:0/2804087502 --> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5577e6a071f0 con 0x5577e6aad9e0 2024-04-17T06:12:42.938 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.932+0000 7f370ffff640 1 --2- 172.21.15.47:0/2804087502 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x5577e6aad9e0 0x5577e6aafdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:12:42.938 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.932+0000 7f370ffff640 1 --2- 172.21.15.47:0/2804087502 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x5577e6aad9e0 0x5577e6aafdd0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9446583a5b7dd40c server_cookie=48f2da9ddc131b73 in_seq=0 out_seq=0 2024-04-17T06:12:42.938 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.932+0000 7f36ef7fe640 10 client.16347 ms_handle_connect on v2:172.21.15.98:6838/2841287917 2024-04-17T06:12:42.938 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.933+0000 7f36ef7fe640 1 -- 172.21.15.47:0/2804087502 <== mds.0 v2:172.21.15.98:6838/2841287917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5577e6a071f0 con 0x5577e6aad9e0 2024-04-17T06:12:42.938 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.933+0000 7f36ef7fe640 10 client.16347 handle_command_reply: tid=0 2024-04-17T06:12:42.938 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.933+0000 7f371938c740 1 client.16347 shutdown 2024-04-17T06:12:42.938 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.933+0000 7f371938c740 1 -- 172.21.15.47:0/2804087502 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5577e6a9d7b0 msgr2=0x5577e6a9b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:12:42.938 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.933+0000 7f371938c740 1 --2- 172.21.15.47:0/2804087502 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5577e6a9d7b0 0x5577e6a9b2b0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f3708005c60 tx=0x7f3708009c20 comp rx=0 tx=0).stop 2024-04-17T06:12:42.938 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.933+0000 7f371938c740 1 -- 172.21.15.47:0/2804087502 shutdown_connections 2024-04-17T06:12:42.938 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.933+0000 7f371938c740 1 --2- 172.21.15.47:0/2804087502 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x5577e6aad9e0 0x5577e6aafdd0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:42.938 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.933+0000 7f371938c740 1 --2- 172.21.15.47:0/2804087502 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5577e6a9d7b0 0x5577e6a9b2b0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:42.938 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.933+0000 7f371938c740 1 --2- 172.21.15.47:0/2804087502 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5577e6a949e0 0x5577e6a9ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:42.938 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.933+0000 7f36ef7fe640 0 client.16347 ms_handle_reset on v2:172.21.15.98:6838/2841287917 2024-04-17T06:12:42.939 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.933+0000 7f371938c740 1 --2- 172.21.15.47:0/2804087502 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5577e6a940c0 0x5577e6a9a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:42.939 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.933+0000 7f371938c740 1 -- 172.21.15.47:0/2804087502 >> 172.21.15.47:0/2804087502 conn(0x5577e6a8e9e0 msgr2=0x5577e6a90730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:12:42.939 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.933+0000 7f371938c740 1 -- 172.21.15.47:0/2804087502 shutdown_connections 2024-04-17T06:12:42.939 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.933+0000 7f371938c740 1 -- 172.21.15.47:0/2804087502 wait complete. 2024-04-17T06:12:42.939 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.933+0000 7f371938c740 20 client.16347 trim_cache size 0 max 16384 2024-04-17T06:12:42.956 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.948+0000 7f371938c740 1 Processor -- start 2024-04-17T06:12:42.956 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.948+0000 7f371938c740 1 -- start start 2024-04-17T06:12:42.956 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.949+0000 7f371938c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5577e6a940c0 0x5577e6a9cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:42.956 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.949+0000 7f371938c740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5577e6a949e0 0x5577e6aa92a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:42.956 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.949+0000 7f371938c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5577e6a9d7b0 0x5577e6ab5400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:42.956 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.949+0000 7f371938c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f37101b0290 con 0x5577e6a940c0 2024-04-17T06:12:42.957 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.949+0000 7f371938c740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f37101adb50 con 0x5577e6a949e0 2024-04-17T06:12:42.957 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.949+0000 7f371938c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f37101ab9c0 con 0x5577e6a9d7b0 2024-04-17T06:12:42.957 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.949+0000 7f370ffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5577e6a940c0 0x5577e6a9cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:12:42.957 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.949+0000 7f370ffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5577e6a940c0 0x5577e6a9cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58874/0 (socket says 172.21.15.47:58874) 2024-04-17T06:12:42.957 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.949+0000 7f370ffff640 1 -- 172.21.15.47:0/1789501070 learned_addr learned my addr 172.21.15.47:0/1789501070 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:12:42.957 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.949+0000 7f3714d45640 1 --2- 172.21.15.47:0/1789501070 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5577e6a9d7b0 0x5577e6ab5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:12:42.957 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.949+0000 7f3707fff640 1 --2- 172.21.15.47:0/1789501070 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5577e6a949e0 0x5577e6aa92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:12:42.957 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.949+0000 7f370ffff640 1 -- 172.21.15.47:0/1789501070 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5577e6a9d7b0 msgr2=0x5577e6ab5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:12:42.958 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.949+0000 7f370ffff640 1 --2- 172.21.15.47:0/1789501070 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5577e6a9d7b0 0x5577e6ab5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:42.958 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.949+0000 7f370ffff640 1 -- 172.21.15.47:0/1789501070 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5577e6a949e0 msgr2=0x5577e6aa92a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:12:42.958 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.949+0000 7f370ffff640 1 --2- 172.21.15.47:0/1789501070 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5577e6a949e0 0x5577e6aa92a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:42.958 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.949+0000 7f370ffff640 1 -- 172.21.15.47:0/1789501070 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577e6a98d80 con 0x5577e6a940c0 2024-04-17T06:12:42.958 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.949+0000 7f3707fff640 1 --2- 172.21.15.47:0/1789501070 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5577e6a949e0 0x5577e6aa92a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:12:42.958 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.949+0000 7f3714d45640 1 --2- 172.21.15.47:0/1789501070 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5577e6a9d7b0 0x5577e6ab5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:12:42.959 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.949+0000 7f370ffff640 1 --2- 172.21.15.47:0/1789501070 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5577e6a940c0 0x5577e6a9cf60 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f3700009dd0 tx=0x7f3700005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f045037ec4da2b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:12:42.959 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.949+0000 7f3705ffb640 1 -- 172.21.15.47:0/1789501070 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3700003da0 con 0x5577e6a940c0 2024-04-17T06:12:42.959 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.949+0000 7f3705ffb640 1 -- 172.21.15.47:0/1789501070 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f370001f530 con 0x5577e6a940c0 2024-04-17T06:12:42.959 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.949+0000 7f3705ffb640 1 -- 172.21.15.47:0/1789501070 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3700010b00 con 0x5577e6a940c0 2024-04-17T06:12:42.959 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.950+0000 7f371938c740 1 -- 172.21.15.47:0/1789501070 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5577e6a940c0 msgr2=0x5577e6a9cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:12:42.959 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.950+0000 7f371938c740 1 --2- 172.21.15.47:0/1789501070 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5577e6a940c0 0x5577e6a9cf60 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f3700009dd0 tx=0x7f3700005f90 comp rx=0 tx=0).stop 2024-04-17T06:12:42.959 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.950+0000 7f371938c740 1 -- 172.21.15.47:0/1789501070 shutdown_connections 2024-04-17T06:12:42.960 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.950+0000 7f371938c740 1 --2- 172.21.15.47:0/1789501070 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5577e6a9d7b0 0x5577e6ab5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:42.960 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.950+0000 7f371938c740 1 --2- 172.21.15.47:0/1789501070 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5577e6a949e0 0x5577e6aa92a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:42.960 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.950+0000 7f371938c740 1 --2- 172.21.15.47:0/1789501070 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5577e6a940c0 0x5577e6a9cf60 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:42.960 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.950+0000 7f371938c740 1 -- 172.21.15.47:0/1789501070 >> 172.21.15.47:0/1789501070 conn(0x5577e6a8e9e0 msgr2=0x5577e6a90650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:12:42.960 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.950+0000 7f371938c740 1 -- 172.21.15.47:0/1789501070 shutdown_connections 2024-04-17T06:12:42.961 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.950+0000 7f371938c740 1 -- 172.21.15.47:0/1789501070 wait complete. 2024-04-17T06:12:42.961 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.950+0000 7f371938c740 1 -- start start 2024-04-17T06:12:42.961 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.951+0000 7f371938c740 1 Processor -- start 2024-04-17T06:12:42.961 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.951+0000 7f371938c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5577e6a940c0 0x5577e6a97710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:42.961 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.951+0000 7f371938c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5577e6a949e0 0x5577e6a97c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:42.961 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.951+0000 7f371938c740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5577e6a9d7b0 0x5577e6a98190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:42.961 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.951+0000 7f371938c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f37101ac010 con 0x5577e6a949e0 2024-04-17T06:12:42.961 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.951+0000 7f371938c740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f37101ad480 con 0x5577e6a9d7b0 2024-04-17T06:12:42.961 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.951+0000 7f371938c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f37101ac6e0 con 0x5577e6a940c0 2024-04-17T06:12:42.962 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.951+0000 7f371938c740 10 client.0 _ll_register_callbacks cb 0x7f36f8098a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:12:42.962 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.951+0000 7f3707fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5577e6a949e0 0x5577e6a97c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:12:42.962 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.951+0000 7f3707fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5577e6a949e0 0x5577e6a97c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58884/0 (socket says 172.21.15.47:58884) 2024-04-17T06:12:42.962 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.951+0000 7f3707fff640 1 -- 172.21.15.47:0/3790864429 learned_addr learned my addr 172.21.15.47:0/3790864429 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:12:42.962 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.951+0000 7f370ffff640 1 --2- 172.21.15.47:0/3790864429 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5577e6a940c0 0x5577e6a97710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:12:42.962 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.951+0000 7f3714d45640 1 --2- 172.21.15.47:0/3790864429 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5577e6a9d7b0 0x5577e6a98190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:12:42.962 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.951+0000 7f3707fff640 1 -- 172.21.15.47:0/3790864429 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5577e6a940c0 msgr2=0x5577e6a97710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:12:42.962 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.951+0000 7f3707fff640 1 --2- 172.21.15.47:0/3790864429 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5577e6a940c0 0x5577e6a97710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:42.962 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.951+0000 7f3707fff640 1 -- 172.21.15.47:0/3790864429 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5577e6a9d7b0 msgr2=0x5577e6a98190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:12:42.962 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.951+0000 7f3707fff640 1 --2- 172.21.15.47:0/3790864429 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5577e6a9d7b0 0x5577e6a98190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:42.962 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.951+0000 7f3707fff640 1 -- 172.21.15.47:0/3790864429 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577e6b12ec0 con 0x5577e6a949e0 2024-04-17T06:12:42.963 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.951+0000 7f3707fff640 1 --2- 172.21.15.47:0/3790864429 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5577e6a949e0 0x5577e6a97c50 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f36f80ce510 tx=0x7f36f809c0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee1ec24592517efa server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:12:42.963 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.951+0000 7f36ef7fe640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-17T06:12:42.963 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.952+0000 7f36ef7fe640 1 -- 172.21.15.47:0/3790864429 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f80a0b10 con 0x5577e6a949e0 2024-04-17T06:12:42.963 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.952+0000 7f36ef7fe640 1 -- 172.21.15.47:0/3790864429 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8469+0+0 (secure 0 0 0) 0x7f36f800fa30 con 0x5577e6a949e0 2024-04-17T06:12:42.963 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.952+0000 7f371938c740 1 -- 172.21.15.47:0/3790864429 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f370001f530 con 0x5577e6a949e0 2024-04-17T06:12:42.963 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.952+0000 7f36ef7fe640 1 -- 172.21.15.47:0/3790864429 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f36f80a53d0 con 0x5577e6a949e0 2024-04-17T06:12:42.963 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.952+0000 7f371938c740 10 client.16353 fetch_fsmap learned FSMap version 36 2024-04-17T06:12:42.963 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.952+0000 7f371938c740 1 -- 172.21.15.47:0/3790864429 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5577e6b14160 con 0x5577e6a949e0 2024-04-17T06:12:42.963 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.952+0000 7f36ef7fe640 1 -- 172.21.15.47:0/3790864429 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f370001f530 con 0x5577e6a949e0 2024-04-17T06:12:42.963 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.952+0000 7f36ef7fe640 1 -- 172.21.15.47:0/3790864429 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 36) ==== 8339+0+0 (secure 0 0 0) 0x7f36f8020030 con 0x5577e6a949e0 2024-04-17T06:12:42.963 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.952+0000 7f371938c740 10 client.16353 fetch_fsmap finished waiting for FSMap version 36 2024-04-17T06:12:42.963 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.952+0000 7f371938c740 10 client.16353 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-17T06:12:42.963 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.952+0000 7f371938c740 20 client.16353 populate_metadata read hostname 'smithi047' 2024-04-17T06:12:42.963 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.952+0000 7f371938c740 1 --2- 172.21.15.47:0/3790864429 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x5577e6b14860 0x5577e6b16c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:42.963 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.952+0000 7f371938c740 4 client.16353 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-17T06:12:42.964 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.952+0000 7f371938c740 1 -- 172.21.15.47:0/3790864429 --> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5577e6a01060 con 0x5577e6b14860 2024-04-17T06:12:42.964 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.952+0000 7f370ffff640 1 --2- 172.21.15.47:0/3790864429 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x5577e6b14860 0x5577e6b16c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:12:42.964 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.953+0000 7f370ffff640 1 --2- 172.21.15.47:0/3790864429 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x5577e6b14860 0x5577e6b16c80 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a5a4f6f94c2ca7a server_cookie=c2ffe32acf39c24b in_seq=0 out_seq=0 2024-04-17T06:12:42.964 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.953+0000 7f36ef7fe640 10 client.16353 ms_handle_connect on v2:172.21.15.98:6838/2841287917 2024-04-17T06:12:42.965 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.961+0000 7f36ef7fe640 1 -- 172.21.15.47:0/3790864429 <== mds.0 v2:172.21.15.98:6838/2841287917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5577e6a01060 con 0x5577e6b14860 2024-04-17T06:12:42.965 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.961+0000 7f36ef7fe640 10 client.16353 handle_command_reply: tid=0 2024-04-17T06:12:42.965 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.961+0000 7f371938c740 1 client.16353 shutdown 2024-04-17T06:12:42.965 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.961+0000 7f371938c740 1 -- 172.21.15.47:0/3790864429 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5577e6a949e0 msgr2=0x5577e6a97c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:12:42.965 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.961+0000 7f371938c740 1 --2- 172.21.15.47:0/3790864429 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5577e6a949e0 0x5577e6a97c50 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f36f80ce510 tx=0x7f36f809c0c0 comp rx=0 tx=0).stop 2024-04-17T06:12:42.965 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.961+0000 7f371938c740 1 -- 172.21.15.47:0/3790864429 shutdown_connections 2024-04-17T06:12:42.965 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.961+0000 7f371938c740 1 --2- 172.21.15.47:0/3790864429 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x5577e6b14860 0x5577e6b16c80 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:42.966 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.961+0000 7f371938c740 1 --2- 172.21.15.47:0/3790864429 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5577e6a9d7b0 0x5577e6a98190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:42.966 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.961+0000 7f36ef7fe640 0 client.16353 ms_handle_reset on v2:172.21.15.98:6838/2841287917 2024-04-17T06:12:42.966 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.961+0000 7f371938c740 1 --2- 172.21.15.47:0/3790864429 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5577e6a949e0 0x5577e6a97c50 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:42.966 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.961+0000 7f371938c740 1 --2- 172.21.15.47:0/3790864429 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5577e6a940c0 0x5577e6a97710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:42.966 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.961+0000 7f371938c740 1 -- 172.21.15.47:0/3790864429 >> 172.21.15.47:0/3790864429 conn(0x5577e6a8e9e0 msgr2=0x5577e6a8d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:12:42.966 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.961+0000 7f371938c740 1 -- 172.21.15.47:0/3790864429 shutdown_connections 2024-04-17T06:12:42.966 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.961+0000 7f371938c740 1 -- 172.21.15.47:0/3790864429 wait complete. 2024-04-17T06:12:42.966 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.961+0000 7f371938c740 20 client.16353 trim_cache size 0 max 16384 2024-04-17T06:12:42.966 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-04-17T06:12:42.966 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-04-17T06:12:42.966 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-04-17T06:12:42.967 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-04-17T06:12:42.967 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.962+0000 7f37167a8640 1 -- 172.21.15.47:0/3650008317 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f36fc0a3dc0 msgr2=0x7f36fc0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:12:42.967 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.962+0000 7f37167a8640 1 --2- 172.21.15.47:0/3650008317 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f36fc0a3dc0 0x7f36fc0a6280 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f370804c000 tx=0x7f3708045000 comp rx=0 tx=0).stop 2024-04-17T06:12:42.968 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.962+0000 7f37167a8640 1 -- 172.21.15.47:0/3650008317 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3710107340 msgr2=0x7f3710198ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:12:42.968 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.962+0000 7f37167a8640 1 --2- 172.21.15.47:0/3650008317 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3710107340 0x7f3710198ce0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f36f800c970 tx=0x7f36f800ce30 comp rx=0 tx=0).stop 2024-04-17T06:12:42.968 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.962+0000 7f37167a8640 1 -- 172.21.15.47:0/3650008317 shutdown_connections 2024-04-17T06:12:42.968 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.962+0000 7f37167a8640 1 --2- 172.21.15.47:0/3650008317 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f371019d010 0x7f371019d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:42.968 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.962+0000 7f37167a8640 1 --2- 172.21.15.47:0/3650008317 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f36fc0a3dc0 0x7f36fc0a6280 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:42.968 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.962+0000 7f37167a8640 1 --2- 172.21.15.47:0/3650008317 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3710107340 0x7f3710198ce0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:42.968 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.962+0000 7f37167a8640 1 --2- 172.21.15.47:0/3650008317 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3710100730 0x7f37101987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:42.968 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.962+0000 7f37167a8640 1 -- 172.21.15.47:0/3650008317 >> 172.21.15.47:0/3650008317 conn(0x7f37100773a0 msgr2=0x7f3710105360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:12:42.968 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.962+0000 7f37167a8640 1 -- 172.21.15.47:0/3650008317 shutdown_connections 2024-04-17T06:12:42.968 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:42.962+0000 7f37167a8640 1 -- 172.21.15.47:0/3650008317 wait complete. 2024-04-17T06:12:44.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:12:43 smithi103 ceph-mon[36243]: pgmap v360: 97 pgs: 97 active+clean; 5.7 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:12:44.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:12:43 smithi098 ceph-mon[36516]: pgmap v360: 97 pgs: 97 active+clean; 5.7 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:12:44.388 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:12:43 smithi047 ceph-mon[32223]: pgmap v360: 97 pgs: 97 active+clean; 5.7 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:12:46.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:12:45 smithi103 ceph-mon[36243]: pgmap v361: 97 pgs: 97 active+clean; 5.7 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 21 KiB/s wr, 18 op/s 2024-04-17T06:12:46.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:12:45 smithi098 ceph-mon[36516]: pgmap v361: 97 pgs: 97 active+clean; 5.7 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 21 KiB/s wr, 18 op/s 2024-04-17T06:12:46.388 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:12:45 smithi047 ceph-mon[32223]: pgmap v361: 97 pgs: 97 active+clean; 5.7 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 21 KiB/s wr, 18 op/s 2024-04-17T06:12:48.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:12:47 smithi103 ceph-mon[36243]: pgmap v362: 97 pgs: 97 active+clean; 5.7 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:12:48.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:12:47 smithi098 ceph-mon[36516]: pgmap v362: 97 pgs: 97 active+clean; 5.7 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:12:48.388 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:12:47 smithi047 ceph-mon[32223]: pgmap v362: 97 pgs: 97 active+clean; 5.7 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:12:50.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:12:49 smithi103 ceph-mon[36243]: pgmap v363: 97 pgs: 97 active+clean; 5.7 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:12:50.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:12:49 smithi098 ceph-mon[36516]: pgmap v363: 97 pgs: 97 active+clean; 5.7 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:12:50.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:12:49 smithi047 ceph-mon[32223]: pgmap v363: 97 pgs: 97 active+clean; 5.7 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:12:52.191 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:12:51 smithi047 ceph-mon[32223]: pgmap v364: 97 pgs: 97 active+clean; 5.7 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-04-17T06:12:52.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:12:51 smithi103 ceph-mon[36243]: pgmap v364: 97 pgs: 97 active+clean; 5.7 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-04-17T06:12:52.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:12:51 smithi098 ceph-mon[36516]: pgmap v364: 97 pgs: 97 active+clean; 5.7 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-04-17T06:12:54.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:12:53 smithi103 ceph-mon[36243]: pgmap v365: 97 pgs: 97 active+clean; 5.7 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:12:54.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:12:53 smithi098 ceph-mon[36516]: pgmap v365: 97 pgs: 97 active+clean; 5.7 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:12:54.388 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:12:53 smithi047 ceph-mon[32223]: pgmap v365: 97 pgs: 97 active+clean; 5.7 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-17T06:12:56.062 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.062+0000 7f6ed202a640 1 -- 172.21.15.47:0/394883212 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ecc1072a0 msgr2=0x7f6ecc10ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:12:56.062 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.062+0000 7f6ed202a640 1 --2- 172.21.15.47:0/394883212 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ecc1072a0 0x7f6ecc10ee70 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f6ec4009900 tx=0x7f6ec402bc30 comp rx=0 tx=0).stop 2024-04-17T06:12:56.062 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.062+0000 7f6ed202a640 1 -- 172.21.15.47:0/394883212 shutdown_connections 2024-04-17T06:12:56.062 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.062+0000 7f6ed202a640 1 --2- 172.21.15.47:0/394883212 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ecc1072a0 0x7f6ecc10ee70 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:56.062 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.062+0000 7f6ed202a640 1 --2- 172.21.15.47:0/394883212 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6ecc102b80 0x7f6ecc106bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:56.062 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.062+0000 7f6ed202a640 1 --2- 172.21.15.47:0/394883212 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ecc1021d0 0x7f6ecc1025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.062+0000 7f6ed202a640 1 -- 172.21.15.47:0/394883212 >> 172.21.15.47:0/394883212 conn(0x7f6ecc0fda30 msgr2=0x7f6ecc0ffe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:12:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.062+0000 7f6ed202a640 1 -- 172.21.15.47:0/394883212 shutdown_connections 2024-04-17T06:12:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.062+0000 7f6ed202a640 1 -- 172.21.15.47:0/394883212 wait complete. 2024-04-17T06:12:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.062+0000 7f6ed202a640 1 Processor -- start 2024-04-17T06:12:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.063+0000 7f6ed202a640 1 -- start start 2024-04-17T06:12:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.063+0000 7f6ed202a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ecc1021d0 0x7f6ecc1987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.063+0000 7f6ed202a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6ecc1072a0 0x7f6ecc198ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.063+0000 7f6ed202a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ecc19d010 0x7f6ecc19d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.063+0000 7f6ed202a640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f6ecc112360 con 0x7f6ecc19d010 2024-04-17T06:12:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.063+0000 7f6ed202a640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6ecc1121e0 con 0x7f6ecc1072a0 2024-04-17T06:12:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.063+0000 7f6ed202a640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6ecc1124e0 con 0x7f6ecc1021d0 2024-04-17T06:12:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.063+0000 7f6ecbfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ecc19d010 0x7f6ecc19d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:12:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.063+0000 7f6ecbfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ecc19d010 0x7f6ecc19d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34930/0 (socket says 172.21.15.47:34930) 2024-04-17T06:12:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.063+0000 7f6ecbfff640 1 -- 172.21.15.47:0/771767484 learned_addr learned my addr 172.21.15.47:0/771767484 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:12:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.063+0000 7f6ecb7fe640 1 --2- 172.21.15.47:0/771767484 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ecc1021d0 0x7f6ecc1987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:12:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.063+0000 7f6ecaffd640 1 --2- 172.21.15.47:0/771767484 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6ecc1072a0 0x7f6ecc198ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:12:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.063+0000 7f6ecbfff640 1 -- 172.21.15.47:0/771767484 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ecc1021d0 msgr2=0x7f6ecc1987a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:12:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.063+0000 7f6ecbfff640 1 --2- 172.21.15.47:0/771767484 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ecc1021d0 0x7f6ecc1987a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.063+0000 7f6ecbfff640 1 -- 172.21.15.47:0/771767484 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6ecc1072a0 msgr2=0x7f6ecc198ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:12:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.063+0000 7f6ecbfff640 1 --2- 172.21.15.47:0/771767484 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6ecc1072a0 0x7f6ecc198ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.063+0000 7f6ecbfff640 1 -- 172.21.15.47:0/771767484 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ecc19db00 con 0x7f6ecc19d010 2024-04-17T06:12:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.063+0000 7f6ecb7fe640 1 --2- 172.21.15.47:0/771767484 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ecc1021d0 0x7f6ecc1987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:12:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.064+0000 7f6ecbfff640 1 --2- 172.21.15.47:0/771767484 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ecc19d010 0x7f6ecc19d4c0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f6ec4009230 tx=0x7f6ec402cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:12:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.064+0000 7f6ec8ff9640 1 -- 172.21.15.47:0/771767484 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec4041070 con 0x7f6ecc19d010 2024-04-17T06:12:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.064+0000 7f6ed202a640 1 -- 172.21.15.47:0/771767484 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ecc19dd90 con 0x7f6ecc19d010 2024-04-17T06:12:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.064+0000 7f6ec8ff9640 1 -- 172.21.15.47:0/771767484 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ec4004450 con 0x7f6ecc19d010 2024-04-17T06:12:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.064+0000 7f6ec8ff9640 1 -- 172.21.15.47:0/771767484 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec4003d90 con 0x7f6ecc19d010 2024-04-17T06:12:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.064+0000 7f6ed202a640 1 -- 172.21.15.47:0/771767484 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ecc1a5550 con 0x7f6ecc19d010 2024-04-17T06:12:56.071 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.066+0000 7f6ec8ff9640 1 -- 172.21.15.47:0/771767484 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6ec4044070 con 0x7f6ecc19d010 2024-04-17T06:12:56.071 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.067+0000 7f6ed4c13740 1 Processor -- start 2024-04-17T06:12:56.071 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.067+0000 7f6ed4c13740 1 -- start start 2024-04-17T06:12:56.071 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.067+0000 7f6ec8ff9640 1 --2- 172.21.15.47:0/771767484 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f6eb40a3dc0 0x7f6eb40a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:56.071 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.067+0000 7f6ed4c13740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5630d6dd0b60 0x5630d6dd0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:56.071 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.067+0000 7f6ed4c13740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5630d6dd1480 0x5630d6dde680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:56.071 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.067+0000 7f6ed4c13740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5630d6ddebc0 0x5630d6de0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:56.071 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.067+0000 7f6ed4c13740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5630d6d367d0 con 0x5630d6ddebc0 2024-04-17T06:12:56.072 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.067+0000 7f6ed4c13740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5630d6cd3f70 con 0x5630d6dd0b60 2024-04-17T06:12:56.072 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.067+0000 7f6ed4c13740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5630d6d32790 con 0x5630d6dd1480 2024-04-17T06:12:56.072 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.067+0000 7f6ecaffd640 1 --2- 172.21.15.47:0/771767484 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f6eb40a3dc0 0x7f6eb40a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:12:56.072 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.067+0000 7f6ecbfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5630d6ddebc0 0x5630d6de0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:12:56.072 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.067+0000 7f6ecbfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5630d6ddebc0 0x5630d6de0fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34940/0 (socket says 172.21.15.47:34940) 2024-04-17T06:12:56.072 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.067+0000 7f6ecbfff640 1 -- 172.21.15.47:0/4223522776 learned_addr learned my addr 172.21.15.47:0/4223522776 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:12:56.072 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.067+0000 7f6ec8ff9640 1 -- 172.21.15.47:0/771767484 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8469+0+0 (secure 0 0 0) 0x7f6ec40f2300 con 0x7f6ecc19d010 2024-04-17T06:12:56.072 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.067+0000 7f6ecb7fe640 1 --2- 172.21.15.47:0/4223522776 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5630d6dd0b60 0x5630d6dd0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:12:56.072 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.067+0000 7f6ecaffd640 1 --2- 172.21.15.47:0/4223522776 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5630d6dd1480 0x5630d6dde680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:12:56.072 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.068+0000 7f6ecbfff640 1 -- 172.21.15.47:0/4223522776 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5630d6dd1480 msgr2=0x5630d6dde680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:12:56.072 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.068+0000 7f6ecbfff640 1 --2- 172.21.15.47:0/4223522776 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5630d6dd1480 0x5630d6dde680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:56.072 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.068+0000 7f6ecbfff640 1 -- 172.21.15.47:0/4223522776 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5630d6dd0b60 msgr2=0x5630d6dd0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:12:56.073 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.068+0000 7f6ecbfff640 1 --2- 172.21.15.47:0/4223522776 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5630d6dd0b60 0x5630d6dd0f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:56.073 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.068+0000 7f6ecbfff640 1 -- 172.21.15.47:0/4223522776 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630d6a49210 con 0x5630d6ddebc0 2024-04-17T06:12:56.073 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.068+0000 7f6ecaffd640 1 --2- 172.21.15.47:0/771767484 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f6eb40a3dc0 0x7f6eb40a6280 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc108770 tx=0x7f6ec00127c0 comp rx=0 tx=0).ready entity=mgr.24923 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:12:56.073 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.068+0000 7f6ecbfff640 1 --2- 172.21.15.47:0/4223522776 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5630d6ddebc0 0x5630d6de0fb0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f6ec402cfd0 tx=0x7f6ec4009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=894b892a57087c83 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:12:56.073 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.068+0000 7f6eb1ffb640 1 -- 172.21.15.47:0/4223522776 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec40f1070 con 0x5630d6ddebc0 2024-04-17T06:12:56.073 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.068+0000 7f6eb1ffb640 1 -- 172.21.15.47:0/4223522776 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ec4004c80 con 0x5630d6ddebc0 2024-04-17T06:12:56.073 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.068+0000 7f6eb1ffb640 1 -- 172.21.15.47:0/4223522776 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec404b040 con 0x5630d6ddebc0 2024-04-17T06:12:56.073 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.069+0000 7f6ed4c13740 1 -- 172.21.15.47:0/4223522776 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5630d6ddebc0 msgr2=0x5630d6de0fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:12:56.073 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.069+0000 7f6ed4c13740 1 --2- 172.21.15.47:0/4223522776 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5630d6ddebc0 0x5630d6de0fb0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f6ec402cfd0 tx=0x7f6ec4009680 comp rx=0 tx=0).stop 2024-04-17T06:12:56.073 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.069+0000 7f6ed4c13740 1 -- 172.21.15.47:0/4223522776 shutdown_connections 2024-04-17T06:12:56.073 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.069+0000 7f6ed4c13740 1 --2- 172.21.15.47:0/4223522776 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5630d6ddebc0 0x5630d6de0fb0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:56.073 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.069+0000 7f6ed4c13740 1 --2- 172.21.15.47:0/4223522776 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5630d6dd1480 0x5630d6dde680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:56.073 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.069+0000 7f6ed4c13740 1 --2- 172.21.15.47:0/4223522776 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5630d6dd0b60 0x5630d6dd0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:56.073 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.069+0000 7f6ed4c13740 1 -- 172.21.15.47:0/4223522776 >> 172.21.15.47:0/4223522776 conn(0x5630d6dcb9e0 msgr2=0x5630d6dcbdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:12:56.074 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.069+0000 7f6ed4c13740 1 -- 172.21.15.47:0/4223522776 shutdown_connections 2024-04-17T06:12:56.074 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.069+0000 7f6ed4c13740 1 -- 172.21.15.47:0/4223522776 wait complete. 2024-04-17T06:12:56.074 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.069+0000 7f6ed4c13740 1 -- start start 2024-04-17T06:12:56.074 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.069+0000 7f6ed4c13740 1 Processor -- start 2024-04-17T06:12:56.074 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.070+0000 7f6ed4c13740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5630d6dd0b60 0x5630d6ddbb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:56.074 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.070+0000 7f6ed4c13740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5630d6dd1480 0x5630d6ddc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:56.074 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.070+0000 7f6ed4c13740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5630d6ddebc0 0x5630d6ddc600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:56.074 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.070+0000 7f6ed4c13740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5630d6de15b0 con 0x5630d6ddebc0 2024-04-17T06:12:56.074 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.070+0000 7f6ed4c13740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5630d6cbee90 con 0x5630d6dd1480 2024-04-17T06:12:56.074 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.070+0000 7f6ed4c13740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5630d6de1730 con 0x5630d6dd0b60 2024-04-17T06:12:56.074 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.070+0000 7f6ed4c13740 10 client.0 _ll_register_callbacks cb 0x5630d6cd4d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:12:56.074 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.070+0000 7f6ecbfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5630d6ddebc0 0x5630d6ddc600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:12:56.074 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.070+0000 7f6ecbfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5630d6ddebc0 0x5630d6ddc600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34950/0 (socket says 172.21.15.47:34950) 2024-04-17T06:12:56.074 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.070+0000 7f6ecbfff640 1 -- 172.21.15.47:0/1387022335 learned_addr learned my addr 172.21.15.47:0/1387022335 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:12:56.075 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.070+0000 7f6ecaffd640 1 --2- 172.21.15.47:0/1387022335 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5630d6dd1480 0x5630d6ddc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:12:56.075 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.070+0000 7f6ecb7fe640 1 --2- 172.21.15.47:0/1387022335 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5630d6dd0b60 0x5630d6ddbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:12:56.075 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.070+0000 7f6ecbfff640 1 -- 172.21.15.47:0/1387022335 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5630d6dd0b60 msgr2=0x5630d6ddbb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:12:56.075 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.070+0000 7f6ecbfff640 1 --2- 172.21.15.47:0/1387022335 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5630d6dd0b60 0x5630d6ddbb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:56.075 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.070+0000 7f6ecbfff640 1 -- 172.21.15.47:0/1387022335 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5630d6dd1480 msgr2=0x5630d6ddc0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:12:56.075 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.070+0000 7f6ecbfff640 1 --2- 172.21.15.47:0/1387022335 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5630d6dd1480 0x5630d6ddc0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:56.075 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.070+0000 7f6ecbfff640 1 -- 172.21.15.47:0/1387022335 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630d6b15620 con 0x5630d6ddebc0 2024-04-17T06:12:56.075 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.070+0000 7f6ecbfff640 1 --2- 172.21.15.47:0/1387022335 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5630d6ddebc0 0x5630d6ddc600 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f6ec40ba7e0 tx=0x7f6ec4004ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3caca0a9e01e30ad server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:12:56.075 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.070+0000 7f6eaaffd640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-17T06:12:56.075 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.070+0000 7f6eaaffd640 1 -- 172.21.15.47:0/1387022335 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec40f1030 con 0x5630d6ddebc0 2024-04-17T06:12:56.075 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.070+0000 7f6eaaffd640 1 -- 172.21.15.47:0/1387022335 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8469+0+0 (secure 0 0 0) 0x7f6ec40c6d60 con 0x5630d6ddebc0 2024-04-17T06:12:56.075 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.070+0000 7f6ed4c13740 1 -- 172.21.15.47:0/1387022335 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ec4004c80 con 0x5630d6ddebc0 2024-04-17T06:12:56.075 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.071+0000 7f6eaaffd640 1 -- 172.21.15.47:0/1387022335 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f6ec404b070 con 0x5630d6ddebc0 2024-04-17T06:12:56.076 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.071+0000 7f6ed4c13740 10 client.16365 fetch_fsmap learned FSMap version 36 2024-04-17T06:12:56.076 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.071+0000 7f6ed4c13740 1 -- 172.21.15.47:0/1387022335 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5630d6de15b0 con 0x5630d6ddebc0 2024-04-17T06:12:56.076 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.071+0000 7f6eaaffd640 1 -- 172.21.15.47:0/1387022335 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ec4004c80 con 0x5630d6ddebc0 2024-04-17T06:12:56.076 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.071+0000 7f6eaaffd640 1 -- 172.21.15.47:0/1387022335 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 36) ==== 8339+0+0 (secure 0 0 0) 0x7f6ec40efc10 con 0x5630d6ddebc0 2024-04-17T06:12:56.076 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.071+0000 7f6ed4c13740 10 client.16365 fetch_fsmap finished waiting for FSMap version 36 2024-04-17T06:12:56.076 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.071+0000 7f6ed4c13740 10 client.16365 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-17T06:12:56.076 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.071+0000 7f6ed4c13740 20 client.16365 populate_metadata read hostname 'smithi047' 2024-04-17T06:12:56.076 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.071+0000 7f6ed4c13740 1 --2- 172.21.15.47:0/1387022335 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x5630d6dee980 0x5630d6df0d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:56.076 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.071+0000 7f6ed4c13740 4 client.16365 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T06:12:56.076 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.071+0000 7f6ed4c13740 1 -- 172.21.15.47:0/1387022335 --> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5630d6d441f0 con 0x5630d6dee980 2024-04-17T06:12:56.076 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.071+0000 7f6ecb7fe640 1 --2- 172.21.15.47:0/1387022335 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x5630d6dee980 0x5630d6df0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:12:56.076 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.072+0000 7f6ecb7fe640 1 --2- 172.21.15.47:0/1387022335 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x5630d6dee980 0x5630d6df0d70 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7ecd3685aa65f8f server_cookie=1d7b8d16beea2f4 in_seq=0 out_seq=0 2024-04-17T06:12:56.076 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.072+0000 7f6eaaffd640 10 client.16365 ms_handle_connect on v2:172.21.15.98:6838/2841287917 2024-04-17T06:12:56.076 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.072+0000 7f6eaaffd640 1 -- 172.21.15.47:0/1387022335 <== mds.0 v2:172.21.15.98:6838/2841287917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5630d6d441f0 con 0x5630d6dee980 2024-04-17T06:12:56.077 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.073+0000 7f6eaaffd640 10 client.16365 handle_command_reply: tid=0 2024-04-17T06:12:56.077 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.073+0000 7f6ed4c13740 1 client.16365 shutdown 2024-04-17T06:12:56.077 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.073+0000 7f6ed4c13740 1 -- 172.21.15.47:0/1387022335 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5630d6ddebc0 msgr2=0x5630d6ddc600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:12:56.077 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.073+0000 7f6ed4c13740 1 --2- 172.21.15.47:0/1387022335 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5630d6ddebc0 0x5630d6ddc600 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f6ec40ba7e0 tx=0x7f6ec4004ee0 comp rx=0 tx=0).stop 2024-04-17T06:12:56.077 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.073+0000 7f6ed4c13740 1 -- 172.21.15.47:0/1387022335 shutdown_connections 2024-04-17T06:12:56.077 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.073+0000 7f6ed4c13740 1 --2- 172.21.15.47:0/1387022335 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x5630d6dee980 0x5630d6df0d70 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:56.077 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.073+0000 7f6ed4c13740 1 --2- 172.21.15.47:0/1387022335 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5630d6ddebc0 0x5630d6ddc600 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:56.077 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.073+0000 7f6ed4c13740 1 --2- 172.21.15.47:0/1387022335 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5630d6dd1480 0x5630d6ddc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:56.077 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.073+0000 7f6eaaffd640 0 client.16365 ms_handle_reset on v2:172.21.15.98:6838/2841287917 2024-04-17T06:12:56.077 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.073+0000 7f6ed4c13740 1 --2- 172.21.15.47:0/1387022335 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5630d6dd0b60 0x5630d6ddbb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:56.077 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.073+0000 7f6ed4c13740 1 -- 172.21.15.47:0/1387022335 >> 172.21.15.47:0/1387022335 conn(0x5630d6dcb9e0 msgr2=0x5630d6dcd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:12:56.077 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.073+0000 7f6ed4c13740 1 -- 172.21.15.47:0/1387022335 shutdown_connections 2024-04-17T06:12:56.077 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.073+0000 7f6ed4c13740 1 -- 172.21.15.47:0/1387022335 wait complete. 2024-04-17T06:12:56.078 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.073+0000 7f6ed4c13740 20 client.16365 trim_cache size 0 max 16384 2024-04-17T06:12:56.088 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.088+0000 7f6ed4c13740 1 Processor -- start 2024-04-17T06:12:56.088 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.088+0000 7f6ed4c13740 1 -- start start 2024-04-17T06:12:56.088 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.088+0000 7f6ed4c13740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5630d6dd0b60 0x5630d6dde2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:56.088 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.088+0000 7f6ed4c13740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5630d6dd1480 0x5630d6dea470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:56.088 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.088+0000 7f6ed4c13740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5630d6ddebc0 0x5630d6dcd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:56.088 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.088+0000 7f6ed4c13740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f6ecc1ae130 con 0x5630d6ddebc0 2024-04-17T06:12:56.088 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.088+0000 7f6ed4c13740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6ecc1ab9f0 con 0x5630d6dd1480 2024-04-17T06:12:56.088 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.088+0000 7f6ed4c13740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6ecc1a9840 con 0x5630d6dd0b60 2024-04-17T06:12:56.089 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.088+0000 7f6ecbfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5630d6ddebc0 0x5630d6dcd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:12:56.089 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.088+0000 7f6ecbfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5630d6ddebc0 0x5630d6dcd540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34966/0 (socket says 172.21.15.47:34966) 2024-04-17T06:12:56.089 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.088+0000 7f6ecbfff640 1 -- 172.21.15.47:0/104420542 learned_addr learned my addr 172.21.15.47:0/104420542 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:12:56.089 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.088+0000 7f6ecb7fe640 1 --2- 172.21.15.47:0/104420542 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5630d6dd0b60 0x5630d6dde2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:12:56.089 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.088+0000 7f6ecbfff640 1 -- 172.21.15.47:0/104420542 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5630d6dd0b60 msgr2=0x5630d6dde2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:12:56.089 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.088+0000 7f6ecbfff640 1 --2- 172.21.15.47:0/104420542 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5630d6dd0b60 0x5630d6dde2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:56.089 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.089+0000 7f6ecbfff640 1 -- 172.21.15.47:0/104420542 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5630d6dd1480 msgr2=0x5630d6dea470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:12:56.089 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.089+0000 7f6ecbfff640 1 --2- 172.21.15.47:0/104420542 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5630d6dd1480 0x5630d6dea470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:56.089 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.089+0000 7f6ecbfff640 1 -- 172.21.15.47:0/104420542 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630d6df13f0 con 0x5630d6ddebc0 2024-04-17T06:12:56.089 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.089+0000 7f6ecbfff640 1 --2- 172.21.15.47:0/104420542 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5630d6ddebc0 0x5630d6dcd540 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f6ec40bad50 tx=0x7f6ec4005be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cfa26888207ae64 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:12:56.089 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.089+0000 7f6eb17fa640 1 -- 172.21.15.47:0/104420542 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec40f1070 con 0x5630d6ddebc0 2024-04-17T06:12:56.089 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.089+0000 7f6eb17fa640 1 -- 172.21.15.47:0/104420542 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ec40a4070 con 0x5630d6ddebc0 2024-04-17T06:12:56.089 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.089+0000 7f6eb17fa640 1 -- 172.21.15.47:0/104420542 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec40c6ff0 con 0x5630d6ddebc0 2024-04-17T06:12:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.089+0000 7f6ed4c13740 1 -- 172.21.15.47:0/104420542 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5630d6ddebc0 msgr2=0x5630d6dcd540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:12:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.089+0000 7f6ed4c13740 1 --2- 172.21.15.47:0/104420542 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5630d6ddebc0 0x5630d6dcd540 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f6ec40bad50 tx=0x7f6ec4005be0 comp rx=0 tx=0).stop 2024-04-17T06:12:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.089+0000 7f6ed4c13740 1 -- 172.21.15.47:0/104420542 shutdown_connections 2024-04-17T06:12:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.089+0000 7f6ed4c13740 1 --2- 172.21.15.47:0/104420542 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5630d6ddebc0 0x5630d6dcd540 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.089+0000 7f6ed4c13740 1 --2- 172.21.15.47:0/104420542 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5630d6dd1480 0x5630d6dea470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.089+0000 7f6ed4c13740 1 --2- 172.21.15.47:0/104420542 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5630d6dd0b60 0x5630d6dde2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.089+0000 7f6ed4c13740 1 -- 172.21.15.47:0/104420542 >> 172.21.15.47:0/104420542 conn(0x5630d6dcb9e0 msgr2=0x5630d6dcce30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:12:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.090+0000 7f6ed4c13740 1 -- 172.21.15.47:0/104420542 shutdown_connections 2024-04-17T06:12:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.090+0000 7f6ed4c13740 1 -- 172.21.15.47:0/104420542 wait complete. 2024-04-17T06:12:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.090+0000 7f6ed4c13740 1 -- start start 2024-04-17T06:12:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.090+0000 7f6ed4c13740 1 Processor -- start 2024-04-17T06:12:56.093 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.090+0000 7f6ed4c13740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5630d6dd0b60 0x5630d6dd8980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:56.093 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.090+0000 7f6ed4c13740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5630d6dd1480 0x5630d6dd8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:56.093 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.090+0000 7f6ed4c13740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5630d6ddebc0 0x5630d6dd9400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:56.094 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.091+0000 7f6ed4c13740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f6ecc1a9eb0 con 0x5630d6dd1480 2024-04-17T06:12:56.094 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.091+0000 7f6ed4c13740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6ecc1ab320 con 0x5630d6ddebc0 2024-04-17T06:12:56.094 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.091+0000 7f6ed4c13740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6ecc1aa580 con 0x5630d6dd0b60 2024-04-17T06:12:56.094 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.091+0000 7f6ed4c13740 10 client.0 _ll_register_callbacks cb 0x7f6ebc0099e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:12:56.094 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.091+0000 7f6ecaffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5630d6dd1480 0x5630d6dd8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:12:56.094 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.091+0000 7f6ecaffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5630d6dd1480 0x5630d6dd8ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34974/0 (socket says 172.21.15.47:34974) 2024-04-17T06:12:56.094 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.091+0000 7f6ecaffd640 1 -- 172.21.15.47:0/3657627412 learned_addr learned my addr 172.21.15.47:0/3657627412 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:12:56.094 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.091+0000 7f6ecb7fe640 1 --2- 172.21.15.47:0/3657627412 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5630d6dd0b60 0x5630d6dd8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:12:56.094 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.091+0000 7f6ecbfff640 1 --2- 172.21.15.47:0/3657627412 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5630d6ddebc0 0x5630d6dd9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:12:56.094 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.091+0000 7f6ecaffd640 1 -- 172.21.15.47:0/3657627412 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5630d6dd0b60 msgr2=0x5630d6dd8980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:12:56.094 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.091+0000 7f6ecaffd640 1 --2- 172.21.15.47:0/3657627412 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5630d6dd0b60 0x5630d6dd8980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:56.094 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.091+0000 7f6ecaffd640 1 -- 172.21.15.47:0/3657627412 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5630d6ddebc0 msgr2=0x5630d6dd9400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:12:56.095 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.091+0000 7f6ecaffd640 1 --2- 172.21.15.47:0/3657627412 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5630d6ddebc0 0x5630d6dd9400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:56.095 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.091+0000 7f6ecaffd640 1 -- 172.21.15.47:0/3657627412 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630d6dd5330 con 0x5630d6dd1480 2024-04-17T06:12:56.095 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.091+0000 7f6ecaffd640 1 --2- 172.21.15.47:0/3657627412 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5630d6dd1480 0x5630d6dd8ec0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f6ec0005fd0 tx=0x7f6ec00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1701b20484961c0d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:12:56.095 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.091+0000 7f6eaaffd640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-17T06:12:56.095 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.091+0000 7f6eaaffd640 1 -- 172.21.15.47:0/3657627412 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec001f8c0 con 0x5630d6dd1480 2024-04-17T06:12:56.095 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.091+0000 7f6eaaffd640 1 -- 172.21.15.47:0/3657627412 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8469+0+0 (secure 0 0 0) 0x7f6ec001fa60 con 0x5630d6dd1480 2024-04-17T06:12:56.095 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.091+0000 7f6ed4c13740 1 -- 172.21.15.47:0/3657627412 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ec40a4070 con 0x5630d6dd1480 2024-04-17T06:12:56.095 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.091+0000 7f6eaaffd640 1 -- 172.21.15.47:0/3657627412 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f6ec001fc60 con 0x5630d6dd1480 2024-04-17T06:12:56.095 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.091+0000 7f6ed4c13740 10 client.16371 fetch_fsmap learned FSMap version 36 2024-04-17T06:12:56.096 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.091+0000 7f6ed4c13740 1 -- 172.21.15.47:0/3657627412 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5630d6e59710 con 0x5630d6dd1480 2024-04-17T06:12:56.096 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.092+0000 7f6eaaffd640 1 -- 172.21.15.47:0/3657627412 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ec40a4070 con 0x5630d6dd1480 2024-04-17T06:12:56.096 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.092+0000 7f6eaaffd640 1 -- 172.21.15.47:0/3657627412 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 36) ==== 8339+0+0 (secure 0 0 0) 0x7f6ec0027020 con 0x5630d6dd1480 2024-04-17T06:12:56.096 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.092+0000 7f6ed4c13740 10 client.16371 fetch_fsmap finished waiting for FSMap version 36 2024-04-17T06:12:56.096 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.092+0000 7f6ed4c13740 10 client.16371 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-17T06:12:56.096 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.092+0000 7f6ed4c13740 20 client.16371 populate_metadata read hostname 'smithi047' 2024-04-17T06:12:56.096 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.092+0000 7f6ed4c13740 1 --2- 172.21.15.47:0/3657627412 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x5630d6e59ef0 0x5630d6e5c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:56.096 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.092+0000 7f6ed4c13740 4 client.16371 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-17T06:12:56.096 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.092+0000 7f6ed4c13740 1 -- 172.21.15.47:0/3657627412 --> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5630d6d3e060 con 0x5630d6e59ef0 2024-04-17T06:12:56.096 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.092+0000 7f6ecb7fe640 1 --2- 172.21.15.47:0/3657627412 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x5630d6e59ef0 0x5630d6e5c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:12:56.096 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.092+0000 7f6eaaffd640 10 client.16371 ms_handle_connect on v2:172.21.15.98:6838/2841287917 2024-04-17T06:12:56.096 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.092+0000 7f6ecb7fe640 1 --2- 172.21.15.47:0/3657627412 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x5630d6e59ef0 0x5630d6e5c310 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97e03977a43b6c05 server_cookie=6bbe6e48177c6f93 in_seq=0 out_seq=0 2024-04-17T06:12:56.099 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.100+0000 7f6eaaffd640 1 -- 172.21.15.47:0/3657627412 <== mds.0 v2:172.21.15.98:6838/2841287917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5630d6d3e060 con 0x5630d6e59ef0 2024-04-17T06:12:56.099 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.100+0000 7f6eaaffd640 10 client.16371 handle_command_reply: tid=0 2024-04-17T06:12:56.099 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.100+0000 7f6ed4c13740 1 client.16371 shutdown 2024-04-17T06:12:56.099 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.101+0000 7f6ed4c13740 1 -- 172.21.15.47:0/3657627412 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5630d6dd1480 msgr2=0x5630d6dd8ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:12:56.099 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.101+0000 7f6ed4c13740 1 --2- 172.21.15.47:0/3657627412 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5630d6dd1480 0x5630d6dd8ec0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f6ec0005fd0 tx=0x7f6ec00046c0 comp rx=0 tx=0).stop 2024-04-17T06:12:56.099 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.101+0000 7f6ed4c13740 1 -- 172.21.15.47:0/3657627412 shutdown_connections 2024-04-17T06:12:56.099 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.101+0000 7f6ed4c13740 1 --2- 172.21.15.47:0/3657627412 >> [v2:172.21.15.98:6838/2841287917,v1:172.21.15.98:6839/2841287917] conn(0x5630d6e59ef0 0x5630d6e5c310 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:56.100 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.101+0000 7f6ed4c13740 1 --2- 172.21.15.47:0/3657627412 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5630d6ddebc0 0x5630d6dd9400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:56.100 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.101+0000 7f6ed4c13740 1 --2- 172.21.15.47:0/3657627412 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5630d6dd1480 0x5630d6dd8ec0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:56.100 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.101+0000 7f6ed4c13740 1 --2- 172.21.15.47:0/3657627412 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5630d6dd0b60 0x5630d6dd8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:56.100 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.101+0000 7f6eaaffd640 0 client.16371 ms_handle_reset on v2:172.21.15.98:6838/2841287917 2024-04-17T06:12:56.100 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.101+0000 7f6ed4c13740 1 -- 172.21.15.47:0/3657627412 >> 172.21.15.47:0/3657627412 conn(0x5630d6dcb9e0 msgr2=0x5630d6dcd6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:12:56.100 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.101+0000 7f6ed4c13740 1 -- 172.21.15.47:0/3657627412 shutdown_connections 2024-04-17T06:12:56.100 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.101+0000 7f6ed4c13740 1 -- 172.21.15.47:0/3657627412 wait complete. 2024-04-17T06:12:56.100 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.101+0000 7f6ed4c13740 20 client.16371 trim_cache size 0 max 16384 2024-04-17T06:12:56.100 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-04-17T06:12:56.100 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-04-17T06:12:56.100 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-04-17T06:12:56.101 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-04-17T06:12:56.101 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.101+0000 7f6ed202a640 1 -- 172.21.15.47:0/771767484 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f6eb40a3dc0 msgr2=0x7f6eb40a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:12:56.101 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.101+0000 7f6ed202a640 1 --2- 172.21.15.47:0/771767484 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f6eb40a3dc0 0x7f6eb40a6280 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc108770 tx=0x7f6ec00127c0 comp rx=0 tx=0).stop 2024-04-17T06:12:56.102 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.101+0000 7f6ed202a640 1 -- 172.21.15.47:0/771767484 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ecc19d010 msgr2=0x7f6ecc19d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:12:56.102 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.101+0000 7f6ed202a640 1 --2- 172.21.15.47:0/771767484 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ecc19d010 0x7f6ecc19d4c0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f6ec4009230 tx=0x7f6ec402cba0 comp rx=0 tx=0).stop 2024-04-17T06:12:56.102 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.101+0000 7f6ed202a640 1 -- 172.21.15.47:0/771767484 shutdown_connections 2024-04-17T06:12:56.102 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.101+0000 7f6ed202a640 1 --2- 172.21.15.47:0/771767484 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f6eb40a3dc0 0x7f6eb40a6280 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:56.102 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.101+0000 7f6ed202a640 1 --2- 172.21.15.47:0/771767484 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ecc19d010 0x7f6ecc19d4c0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:56.102 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.101+0000 7f6ed202a640 1 --2- 172.21.15.47:0/771767484 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6ecc1072a0 0x7f6ecc198ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:56.102 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.101+0000 7f6ed202a640 1 --2- 172.21.15.47:0/771767484 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ecc1021d0 0x7f6ecc1987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:56.102 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.101+0000 7f6ed202a640 1 -- 172.21.15.47:0/771767484 >> 172.21.15.47:0/771767484 conn(0x7f6ecc0fda30 msgr2=0x7f6ecc0fe470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:12:56.102 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.102+0000 7f6ed202a640 1 -- 172.21.15.47:0/771767484 shutdown_connections 2024-04-17T06:12:56.102 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-04-17T06:12:56.102+0000 7f6ed202a640 1 -- 172.21.15.47:0/771767484 wait complete. 2024-04-17T06:12:56.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:12:55 smithi103 ceph-mon[36243]: pgmap v366: 97 pgs: 97 active+clean; 5.8 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-04-17T06:12:56.284 INFO:tasks.workunit.client.0.smithi047.stdout:561/600: open fd = 3 2024-04-17T06:12:56.284 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:56.285 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:56.285 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:12:56.285 INFO:tasks.workunit.client.0.smithi047.stdout:562/600: open fd = 3 2024-04-17T06:12:56.285 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:56.285 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:56.285 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:12:56.285 INFO:tasks.workunit.client.0.smithi047.stdout:563/600: open fd = 3 2024-04-17T06:12:56.285 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:56.285 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:56.285 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:12:56.285 INFO:tasks.workunit.client.0.smithi047.stdout:564/600: open fd = 3 2024-04-17T06:12:56.285 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:56.285 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:56.285 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:12:56.285 INFO:tasks.workunit.client.0.smithi047.stdout:565/600: open fd = 3 2024-04-17T06:12:56.285 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:56.286 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:56.286 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:12:56.286 INFO:tasks.workunit.client.0.smithi047.stdout:566/600: open fd = 3 2024-04-17T06:12:56.286 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:56.286 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:56.286 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:12:56.286 INFO:tasks.workunit.client.0.smithi047.stdout:567/600: open fd = 3 2024-04-17T06:12:56.286 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:56.286 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:56.286 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:12:56.286 INFO:tasks.workunit.client.0.smithi047.stdout:568/600: open fd = 3 2024-04-17T06:12:56.286 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:56.286 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:56.286 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:12:56.286 INFO:tasks.workunit.client.0.smithi047.stdout:569/600: open fd = 3 2024-04-17T06:12:56.287 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:56.287 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:56.287 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:12:56.287 INFO:tasks.workunit.client.0.smithi047.stdout:570/600: open fd = 3 2024-04-17T06:12:56.287 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:56.287 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:56.287 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:12:56.287 INFO:tasks.workunit.client.0.smithi047.stdout:571/600: open fd = 3 2024-04-17T06:12:56.287 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:56.287 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:56.287 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:12:56.287 INFO:tasks.workunit.client.0.smithi047.stdout:572/600: open fd = 3 2024-04-17T06:12:56.287 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:56.287 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:56.288 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:12:56.288 INFO:tasks.workunit.client.0.smithi047.stdout:573/600: open fd = 3 2024-04-17T06:12:56.288 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:56.288 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:56.288 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:12:56.288 INFO:tasks.workunit.client.0.smithi047.stdout:574/600: open fd = 3 2024-04-17T06:12:56.288 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:56.288 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:56.288 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:12:56.288 INFO:tasks.workunit.client.0.smithi047.stdout:575/600: open fd = 3 2024-04-17T06:12:56.288 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:56.288 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:56.288 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:12:56.288 INFO:tasks.workunit.client.0.smithi047.stdout:576/600: open fd = 3 2024-04-17T06:12:56.288 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:56.289 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:56.289 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:12:56.289 INFO:tasks.workunit.client.0.smithi047.stdout:577/600: open fd = 3 2024-04-17T06:12:56.289 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:56.289 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:56.289 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:12:56.289 INFO:tasks.workunit.client.0.smithi047.stdout:578/600: open fd = 3 2024-04-17T06:12:56.289 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:56.289 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:56.289 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:12:56.289 INFO:tasks.workunit.client.0.smithi047.stdout:579/600: open fd = 3 2024-04-17T06:12:56.289 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:56.289 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:56.289 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:12:56.289 INFO:tasks.workunit.client.0.smithi047.stdout:580/600: open fd = 3 2024-04-17T06:12:56.290 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:56.290 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:56.290 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:12:56.290 INFO:tasks.workunit.client.0.smithi047.stdout:581/600: open fd = 3 2024-04-17T06:12:56.290 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:56.290 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:56.290 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:12:56.290 INFO:tasks.workunit.client.0.smithi047.stdout:582/600: open fd = 3 2024-04-17T06:12:56.290 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:56.290 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:56.290 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:12:56.290 INFO:tasks.workunit.client.0.smithi047.stdout:583/600: open fd = 3 2024-04-17T06:12:56.290 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:56.291 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:56.291 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:12:56.291 INFO:tasks.workunit.client.0.smithi047.stdout:584/600: open fd = 3 2024-04-17T06:12:56.291 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:56.291 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:56.291 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:12:56.291 INFO:tasks.workunit.client.0.smithi047.stdout:585/600: open fd = 3 2024-04-17T06:12:56.291 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:56.291 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:56.291 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:12:56.291 INFO:tasks.workunit.client.0.smithi047.stdout:586/600: open fd = 3 2024-04-17T06:12:56.291 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:56.291 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:56.291 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:12:56.291 INFO:tasks.workunit.client.0.smithi047.stdout:587/600: open fd = 3 2024-04-17T06:12:56.292 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:56.292 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:56.292 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:12:56.292 INFO:tasks.workunit.client.0.smithi047.stdout:588/600: open fd = 3 2024-04-17T06:12:56.292 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:56.292 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:56.292 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:12:56.292 INFO:tasks.workunit.client.0.smithi047.stdout:589/600: open fd = 3 2024-04-17T06:12:56.292 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:56.292 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:56.292 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:12:56.292 INFO:tasks.workunit.client.0.smithi047.stdout:590/600: open fd = 3 2024-04-17T06:12:56.292 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:56.292 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:56.292 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:12:56.293 INFO:tasks.workunit.client.0.smithi047.stdout:591/600: open fd = 3 2024-04-17T06:12:56.293 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:56.293 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:56.293 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:12:56.293 INFO:tasks.workunit.client.0.smithi047.stdout:592/600: open fd = 3 2024-04-17T06:12:56.293 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:56.293 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:56.293 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:12:56.293 INFO:tasks.workunit.client.0.smithi047.stdout:593/600: open fd = 3 2024-04-17T06:12:56.293 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:56.293 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:56.293 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:12:56.293 INFO:tasks.workunit.client.0.smithi047.stdout:594/600: open fd = 3 2024-04-17T06:12:56.293 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:56.293 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:56.293 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:12:56.294 INFO:tasks.workunit.client.0.smithi047.stdout:595/600: open fd = 3 2024-04-17T06:12:56.294 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:56.294 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:56.294 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:12:56.294 INFO:tasks.workunit.client.0.smithi047.stdout:596/600: open fd = 3 2024-04-17T06:12:56.294 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:56.294 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:56.294 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:12:56.294 INFO:tasks.workunit.client.0.smithi047.stdout:597/600: open fd = 3 2024-04-17T06:12:56.294 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:56.294 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:56.294 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:12:56.294 INFO:tasks.workunit.client.0.smithi047.stdout:598/600: open fd = 3 2024-04-17T06:12:56.294 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:56.294 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:56.295 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:12:56.295 INFO:tasks.workunit.client.0.smithi047.stdout:599/600: open fd = 3 2024-04-17T06:12:56.295 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:56.295 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:56.295 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:12:56.295 INFO:tasks.workunit.client.0.smithi047.stdout:600/600: open fd = 3 2024-04-17T06:12:56.295 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:56.295 INFO:tasks.workunit.client.0.smithi047.stdout:write ret = 32 2024-04-17T06:12:56.295 INFO:tasks.workunit.client.0.smithi047.stdout:pread ret = 64 2024-04-17T06:12:56.295 INFO:tasks.workunit.client.0.smithi047.stderr:+ echo OK 2024-04-17T06:12:56.296 INFO:tasks.workunit.client.0.smithi047.stdout:OK 2024-04-17T06:12:56.297 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-17T06:12:56.297 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-17T06:12:56.366 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:12:55 smithi047 ceph-mon[32223]: pgmap v366: 97 pgs: 97 active+clean; 5.8 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-04-17T06:12:56.382 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:12:55 smithi098 ceph-mon[36516]: pgmap v366: 97 pgs: 97 active+clean; 5.8 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-04-17T06:12:56.394 INFO:tasks.workunit:Stopping ['fs/test_o_trunc.sh'] on client.0... 2024-04-17T06:12:56.394 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-17T06:12:56.718 DEBUG:teuthology.parallel:result is None 2024-04-17T06:12:56.718 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-04-17T06:12:56.752 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-04-17T06:12:56.752 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-04-17T06:12:56.767 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi047.front.sepia.ceph.com 2024-04-17T06:12:56.801 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-04-17T06:12:56.810 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-04-17T06:12:56.810 INFO:teuthology.task.exec:Executing custom commands... 2024-04-17T06:12:56.810 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi047.front.sepia.ceph.com 2024-04-17T06:12:56.810 DEBUG:teuthology.orchestra.run.smithi047:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-04-17T06:12:56.878 DEBUG:teuthology.orchestra.run.smithi047:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-04-17T06:12:56.949 DEBUG:teuthology.orchestra.run.smithi047:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-17T06:12:57.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.098+0000 7ff5b9e47640 1 -- 172.21.15.47:0/2541117293 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff5b41016b0 msgr2=0x7ff5b410e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:12:57.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.098+0000 7ff5b9e47640 1 --2- 172.21.15.47:0/2541117293 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff5b41016b0 0x7ff5b410e400 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7ff5a8009900 tx=0x7ff5a802bc30 comp rx=0 tx=0).stop 2024-04-17T06:12:57.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.098+0000 7ff5b9e47640 1 -- 172.21.15.47:0/2541117293 shutdown_connections 2024-04-17T06:12:57.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.098+0000 7ff5b9e47640 1 --2- 172.21.15.47:0/2541117293 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5b410e940 0x7ff5b4110d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:57.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.098+0000 7ff5b9e47640 1 --2- 172.21.15.47:0/2541117293 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff5b41016b0 0x7ff5b410e400 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:57.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.098+0000 7ff5b9e47640 1 --2- 172.21.15.47:0/2541117293 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff5b4100d90 0x7ff5b4101170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:57.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.098+0000 7ff5b9e47640 1 -- 172.21.15.47:0/2541117293 >> 172.21.15.47:0/2541117293 conn(0x7ff5b4077fe0 msgr2=0x7ff5b40783f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:12:57.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.098+0000 7ff5b9e47640 1 -- 172.21.15.47:0/2541117293 shutdown_connections 2024-04-17T06:12:57.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.098+0000 7ff5b9e47640 1 -- 172.21.15.47:0/2541117293 wait complete. 2024-04-17T06:12:57.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.098+0000 7ff5b9e47640 1 Processor -- start 2024-04-17T06:12:57.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.099+0000 7ff5b9e47640 1 -- start start 2024-04-17T06:12:57.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.099+0000 7ff5b9e47640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff5b4100d90 0x7ff5b41a0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:57.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.099+0000 7ff5b9e47640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5b41016b0 0x7ff5b41a0c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:57.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.099+0000 7ff5b9e47640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff5b410e940 0x7ff5b419a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:57.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.099+0000 7ff5b9e47640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b4114590 con 0x7ff5b4100d90 2024-04-17T06:12:57.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.099+0000 7ff5b9e47640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b4114410 con 0x7ff5b410e940 2024-04-17T06:12:57.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.099+0000 7ff5b9e47640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b4114710 con 0x7ff5b41016b0 2024-04-17T06:12:57.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.099+0000 7ff5b37fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff5b4100d90 0x7ff5b41a0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:12:57.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.099+0000 7ff5b37fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff5b4100d90 0x7ff5b41a0740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34996/0 (socket says 172.21.15.47:34996) 2024-04-17T06:12:57.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.099+0000 7ff5b37fe640 1 -- 172.21.15.47:0/2448971497 learned_addr learned my addr 172.21.15.47:0/2448971497 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:12:57.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.099+0000 7ff5b2ffd640 1 --2- 172.21.15.47:0/2448971497 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5b41016b0 0x7ff5b41a0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:12:57.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.099+0000 7ff5b3fff640 1 --2- 172.21.15.47:0/2448971497 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff5b410e940 0x7ff5b419a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:12:57.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.099+0000 7ff5b37fe640 1 -- 172.21.15.47:0/2448971497 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5b41016b0 msgr2=0x7ff5b41a0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:12:57.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.099+0000 7ff5b37fe640 1 --2- 172.21.15.47:0/2448971497 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5b41016b0 0x7ff5b41a0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:57.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.099+0000 7ff5b37fe640 1 -- 172.21.15.47:0/2448971497 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff5b410e940 msgr2=0x7ff5b419a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:12:57.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.099+0000 7ff5b37fe640 1 --2- 172.21.15.47:0/2448971497 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff5b410e940 0x7ff5b419a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:57.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.099+0000 7ff5b37fe640 1 -- 172.21.15.47:0/2448971497 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5b419b070 con 0x7ff5b4100d90 2024-04-17T06:12:57.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.100+0000 7ff5b37fe640 1 --2- 172.21.15.47:0/2448971497 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff5b4100d90 0x7ff5b41a0740 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7ff5a400c890 tx=0x7ff5a400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:12:57.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.100+0000 7ff5b0ff9640 1 -- 172.21.15.47:0/2448971497 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a4004220 con 0x7ff5b4100d90 2024-04-17T06:12:57.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.100+0000 7ff5b9e47640 1 -- 172.21.15.47:0/2448971497 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5b419b360 con 0x7ff5b4100d90 2024-04-17T06:12:57.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.100+0000 7ff5b0ff9640 1 -- 172.21.15.47:0/2448971497 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5a40043c0 con 0x7ff5b4100d90 2024-04-17T06:12:57.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.100+0000 7ff5b0ff9640 1 -- 172.21.15.47:0/2448971497 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a4004a00 con 0x7ff5b4100d90 2024-04-17T06:12:57.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.100+0000 7ff5b9e47640 1 -- 172.21.15.47:0/2448971497 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5b41a7600 con 0x7ff5b4100d90 2024-04-17T06:12:57.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.101+0000 7ff5b9e47640 1 -- 172.21.15.47:0/2448971497 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5b404fcf0 con 0x7ff5b4100d90 2024-04-17T06:12:57.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.102+0000 7ff5b0ff9640 1 -- 172.21.15.47:0/2448971497 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff5a40079b0 con 0x7ff5b4100d90 2024-04-17T06:12:57.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.102+0000 7ff5b0ff9640 1 --2- 172.21.15.47:0/2448971497 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7ff59c0a3dc0 0x7ff59c0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:57.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.103+0000 7ff5b2ffd640 1 --2- 172.21.15.47:0/2448971497 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7ff59c0a3dc0 0x7ff59c0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:12:57.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.103+0000 7ff5b0ff9640 1 -- 172.21.15.47:0/2448971497 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8469+0+0 (secure 0 0 0) 0x7ff5a40d08d0 con 0x7ff5b4100d90 2024-04-17T06:12:57.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.103+0000 7ff5b2ffd640 1 --2- 172.21.15.47:0/2448971497 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7ff59c0a3dc0 0x7ff59c0a6280 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff5a8009230 tx=0x7ff5a8004f10 comp rx=0 tx=0).ready entity=mgr.24923 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:12:57.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.106+0000 7ff5b0ff9640 1 -- 172.21.15.47:0/2448971497 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7ff5a4014030 con 0x7ff5b4100d90 2024-04-17T06:12:57.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.316+0000 7ff5b9e47640 1 -- 172.21.15.47:0/2448971497 --> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7ff5b40623a0 con 0x7ff59c0a3dc0 2024-04-17T06:12:57.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.324+0000 7ff5b0ff9640 1 -- 172.21.15.47:0/2448971497 <== mgr.24923 v2:172.21.15.47:6800/255579447 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+350 (secure 0 0 0) 0x7ff5b40623a0 con 0x7ff59c0a3dc0 2024-04-17T06:12:57.322 INFO:teuthology.orchestra.run.smithi047.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-17T00:00:00", "created": "2024-04-17T06:01:19", "first": "2024-04-17T06:02:00", "last": "2024-04-17T06:12:00", "last_pruned": "2024-04-17T06:12:00", "created_count": 11, "pruned_count": 5, "active": true} 2024-04-17T06:12:57.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.326+0000 7ff5b9e47640 1 -- 172.21.15.47:0/2448971497 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7ff59c0a3dc0 msgr2=0x7ff59c0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:12:57.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.326+0000 7ff5b9e47640 1 --2- 172.21.15.47:0/2448971497 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7ff59c0a3dc0 0x7ff59c0a6280 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff5a8009230 tx=0x7ff5a8004f10 comp rx=0 tx=0).stop 2024-04-17T06:12:57.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.326+0000 7ff5b9e47640 1 -- 172.21.15.47:0/2448971497 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff5b4100d90 msgr2=0x7ff5b41a0740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:12:57.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.326+0000 7ff5b9e47640 1 --2- 172.21.15.47:0/2448971497 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff5b4100d90 0x7ff5b41a0740 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7ff5a400c890 tx=0x7ff5a400cd50 comp rx=0 tx=0).stop 2024-04-17T06:12:57.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.326+0000 7ff5b9e47640 1 -- 172.21.15.47:0/2448971497 shutdown_connections 2024-04-17T06:12:57.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.326+0000 7ff5b9e47640 1 --2- 172.21.15.47:0/2448971497 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff5b410e940 0x7ff5b419a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:57.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.326+0000 7ff5b9e47640 1 --2- 172.21.15.47:0/2448971497 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5b41016b0 0x7ff5b41a0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:57.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.326+0000 7ff5b9e47640 1 --2- 172.21.15.47:0/2448971497 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7ff59c0a3dc0 0x7ff59c0a6280 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:57.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.326+0000 7ff5b9e47640 1 --2- 172.21.15.47:0/2448971497 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff5b4100d90 0x7ff5b41a0740 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:57.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.326+0000 7ff5b9e47640 1 -- 172.21.15.47:0/2448971497 >> 172.21.15.47:0/2448971497 conn(0x7ff5b4077fe0 msgr2=0x7ff5b410fe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:12:57.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.326+0000 7ff5b9e47640 1 -- 172.21.15.47:0/2448971497 shutdown_connections 2024-04-17T06:12:57.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.326+0000 7ff5b9e47640 1 -- 172.21.15.47:0/2448971497 wait complete. 2024-04-17T06:12:57.336 DEBUG:teuthology.orchestra.run.smithi047:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-17T06:12:57.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.445+0000 7f32807f0640 1 -- 172.21.15.47:0/2316324642 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f32781006f0 msgr2=0x7f3278100ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:12:57.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.445+0000 7f32807f0640 1 --2- 172.21.15.47:0/2316324642 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f32781006f0 0x7f3278100ad0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f3274009900 tx=0x7f327402bc30 comp rx=0 tx=0).stop 2024-04-17T06:12:57.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.445+0000 7f32807f0640 1 -- 172.21.15.47:0/2316324642 shutdown_connections 2024-04-17T06:12:57.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.445+0000 7f32807f0640 1 --2- 172.21.15.47:0/2316324642 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3278107300 0x7f327810ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:57.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.445+0000 7f32807f0640 1 --2- 172.21.15.47:0/2316324642 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f32781010a0 0x7f3278106bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:57.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.445+0000 7f32807f0640 1 --2- 172.21.15.47:0/2316324642 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f32781006f0 0x7f3278100ad0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:57.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.445+0000 7f32807f0640 1 -- 172.21.15.47:0/2316324642 >> 172.21.15.47:0/2316324642 conn(0x7f32780773e0 msgr2=0x7f32780fe770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:12:57.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.445+0000 7f32807f0640 1 -- 172.21.15.47:0/2316324642 shutdown_connections 2024-04-17T06:12:57.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.445+0000 7f32807f0640 1 -- 172.21.15.47:0/2316324642 wait complete. 2024-04-17T06:12:57.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.446+0000 7f32807f0640 1 Processor -- start 2024-04-17T06:12:57.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.446+0000 7f32807f0640 1 -- start start 2024-04-17T06:12:57.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.446+0000 7f32807f0640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f32781006f0 0x7f327819e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:57.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.446+0000 7f32807f0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3278107300 0x7f327819e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:57.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.446+0000 7f32807f0640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f32781985a0 0x7f3278198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:57.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.446+0000 7f32807f0640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f3278112340 con 0x7f32781985a0 2024-04-17T06:12:57.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.446+0000 7f32807f0640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f32781121c0 con 0x7f32781006f0 2024-04-17T06:12:57.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.446+0000 7f32807f0640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f32781124c0 con 0x7f3278107300 2024-04-17T06:12:57.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.446+0000 7f327ed8d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f32781985a0 0x7f3278198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:12:57.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.446+0000 7f327ed8d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f32781985a0 0x7f3278198a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35018/0 (socket says 172.21.15.47:35018) 2024-04-17T06:12:57.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.446+0000 7f327ed8d640 1 -- 172.21.15.47:0/247060383 learned_addr learned my addr 172.21.15.47:0/247060383 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:12:57.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.447+0000 7f327ed8d640 1 -- 172.21.15.47:0/247060383 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3278107300 msgr2=0x7f327819e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:12:57.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.447+0000 7f327ed8d640 1 --2- 172.21.15.47:0/247060383 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3278107300 0x7f327819e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:57.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.447+0000 7f327ed8d640 1 -- 172.21.15.47:0/247060383 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f32781006f0 msgr2=0x7f327819e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:12:57.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.447+0000 7f327e58c640 1 --2- 172.21.15.47:0/247060383 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f32781006f0 0x7f327819e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:12:57.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.447+0000 7f327ed8d640 1 --2- 172.21.15.47:0/247060383 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f32781006f0 0x7f327819e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:57.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.447+0000 7f327ed8d640 1 -- 172.21.15.47:0/247060383 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32781990c0 con 0x7f32781985a0 2024-04-17T06:12:57.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.447+0000 7f327e58c640 1 --2- 172.21.15.47:0/247060383 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f32781006f0 0x7f327819e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:12:57.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.447+0000 7f327ed8d640 1 --2- 172.21.15.47:0/247060383 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f32781985a0 0x7f3278198a50 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f3270009fd0 tx=0x7f327000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:12:57.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.447+0000 7f326f7fe640 1 -- 172.21.15.47:0/247060383 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3270017070 con 0x7f32781985a0 2024-04-17T06:12:57.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.447+0000 7f326f7fe640 1 -- 172.21.15.47:0/247060383 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3270003e50 con 0x7f32781985a0 2024-04-17T06:12:57.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.447+0000 7f32807f0640 1 -- 172.21.15.47:0/247060383 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32781993b0 con 0x7f32781985a0 2024-04-17T06:12:57.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.447+0000 7f326f7fe640 1 -- 172.21.15.47:0/247060383 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32700106c0 con 0x7f32781985a0 2024-04-17T06:12:57.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.447+0000 7f32807f0640 1 -- 172.21.15.47:0/247060383 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32781a55b0 con 0x7f32781985a0 2024-04-17T06:12:57.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.448+0000 7f32807f0640 1 -- 172.21.15.47:0/247060383 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f327804fcf0 con 0x7f32781985a0 2024-04-17T06:12:57.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.449+0000 7f326f7fe640 1 -- 172.21.15.47:0/247060383 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3270003970 con 0x7f32781985a0 2024-04-17T06:12:57.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.450+0000 7f326f7fe640 1 --2- 172.21.15.47:0/247060383 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f32640a3dc0 0x7f32640a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:57.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.450+0000 7f327e58c640 1 --2- 172.21.15.47:0/247060383 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f32640a3dc0 0x7f32640a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:12:57.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.450+0000 7f326f7fe640 1 -- 172.21.15.47:0/247060383 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8469+0+0 (secure 0 0 0) 0x7f32700d0570 con 0x7f32781985a0 2024-04-17T06:12:57.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.450+0000 7f327e58c640 1 --2- 172.21.15.47:0/247060383 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f32640a3dc0 0x7f32640a6280 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3274036040 tx=0x7f3274000bc0 comp rx=0 tx=0).ready entity=mgr.24923 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:12:57.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.453+0000 7f326f7fe640 1 -- 172.21.15.47:0/247060383 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f327009cc40 con 0x7f32781985a0 2024-04-17T06:12:57.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.666+0000 7f32807f0640 1 -- 172.21.15.47:0/247060383 --> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f32781010a0 con 0x7f32640a3dc0 2024-04-17T06:12:57.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.673+0000 7f326f7fe640 1 -- 172.21.15.47:0/247060383 <== mgr.24923 v2:172.21.15.47:6800/255579447 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f32781010a0 con 0x7f32640a3dc0 2024-04-17T06:12:57.671 INFO:teuthology.orchestra.run.smithi047.stdout:/ 1m 6m3h 2024-04-17T06:12:57.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.675+0000 7f32807f0640 1 -- 172.21.15.47:0/247060383 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f32640a3dc0 msgr2=0x7f32640a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:12:57.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.675+0000 7f32807f0640 1 --2- 172.21.15.47:0/247060383 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f32640a3dc0 0x7f32640a6280 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3274036040 tx=0x7f3274000bc0 comp rx=0 tx=0).stop 2024-04-17T06:12:57.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.675+0000 7f32807f0640 1 -- 172.21.15.47:0/247060383 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f32781985a0 msgr2=0x7f3278198a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:12:57.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.675+0000 7f32807f0640 1 --2- 172.21.15.47:0/247060383 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f32781985a0 0x7f3278198a50 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f3270009fd0 tx=0x7f327000b260 comp rx=0 tx=0).stop 2024-04-17T06:12:57.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.675+0000 7f32807f0640 1 -- 172.21.15.47:0/247060383 shutdown_connections 2024-04-17T06:12:57.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.675+0000 7f32807f0640 1 --2- 172.21.15.47:0/247060383 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f32640a3dc0 0x7f32640a6280 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:57.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.675+0000 7f32807f0640 1 --2- 172.21.15.47:0/247060383 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f32781985a0 0x7f3278198a50 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:57.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.675+0000 7f32807f0640 1 --2- 172.21.15.47:0/247060383 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3278107300 0x7f327819e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:57.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.675+0000 7f32807f0640 1 --2- 172.21.15.47:0/247060383 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f32781006f0 0x7f327819e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:57.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.675+0000 7f32807f0640 1 -- 172.21.15.47:0/247060383 >> 172.21.15.47:0/247060383 conn(0x7f32780773e0 msgr2=0x7f32781053a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:12:57.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.675+0000 7f32807f0640 1 -- 172.21.15.47:0/247060383 shutdown_connections 2024-04-17T06:12:57.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.675+0000 7f32807f0640 1 -- 172.21.15.47:0/247060383 wait complete. 2024-04-17T06:12:57.685 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-04-17T06:12:57.698 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T06:12:57.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.806+0000 7efe5b035640 1 -- 172.21.15.47:0/2694802437 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efe54101a50 msgr2=0x7efe54105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:12:57.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.806+0000 7efe5b035640 1 --2- 172.21.15.47:0/2694802437 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efe54101a50 0x7efe54105a90 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7efe3c009900 tx=0x7efe3c02bc30 comp rx=0 tx=0).stop 2024-04-17T06:12:57.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.806+0000 7efe5b035640 1 -- 172.21.15.47:0/2694802437 shutdown_connections 2024-04-17T06:12:57.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.806+0000 7efe5b035640 1 --2- 172.21.15.47:0/2694802437 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efe54106170 0x7efe5410dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:57.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.806+0000 7efe5b035640 1 --2- 172.21.15.47:0/2694802437 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efe54101a50 0x7efe54105a90 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:57.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.806+0000 7efe5b035640 1 --2- 172.21.15.47:0/2694802437 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efe541010a0 0x7efe54101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:57.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.806+0000 7efe5b035640 1 -- 172.21.15.47:0/2694802437 >> 172.21.15.47:0/2694802437 conn(0x7efe540771e0 msgr2=0x7efe540775f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:12:57.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.806+0000 7efe5b035640 1 -- 172.21.15.47:0/2694802437 shutdown_connections 2024-04-17T06:12:57.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.806+0000 7efe5b035640 1 -- 172.21.15.47:0/2694802437 wait complete. 2024-04-17T06:12:57.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.806+0000 7efe5b035640 1 Processor -- start 2024-04-17T06:12:57.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.806+0000 7efe5b035640 1 -- start start 2024-04-17T06:12:57.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.807+0000 7efe5b035640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efe541010a0 0x7efe5419e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:57.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.807+0000 7efe5b035640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efe54106170 0x7efe5419eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:57.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.807+0000 7efe5b035640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efe541986b0 0x7efe54198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:57.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.807+0000 7efe5b035640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7efe54111260 con 0x7efe54106170 2024-04-17T06:12:57.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.807+0000 7efe5b035640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7efe541110e0 con 0x7efe541010a0 2024-04-17T06:12:57.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.807+0000 7efe5b035640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7efe541113e0 con 0x7efe541986b0 2024-04-17T06:12:57.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.807+0000 7efe53fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efe54106170 0x7efe5419eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:12:57.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.807+0000 7efe53fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efe54106170 0x7efe5419eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35042/0 (socket says 172.21.15.47:35042) 2024-04-17T06:12:57.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.807+0000 7efe53fff640 1 -- 172.21.15.47:0/847246225 learned_addr learned my addr 172.21.15.47:0/847246225 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:12:57.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.807+0000 7efe58dd1640 1 --2- 172.21.15.47:0/847246225 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efe541010a0 0x7efe5419e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:12:57.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.807+0000 7efe595d2640 1 --2- 172.21.15.47:0/847246225 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efe541986b0 0x7efe54198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:12:57.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.807+0000 7efe53fff640 1 -- 172.21.15.47:0/847246225 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efe541986b0 msgr2=0x7efe54198b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:12:57.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.807+0000 7efe53fff640 1 --2- 172.21.15.47:0/847246225 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efe541986b0 0x7efe54198b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:57.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.807+0000 7efe53fff640 1 -- 172.21.15.47:0/847246225 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efe541010a0 msgr2=0x7efe5419e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:12:57.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.807+0000 7efe53fff640 1 --2- 172.21.15.47:0/847246225 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efe541010a0 0x7efe5419e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:57.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.807+0000 7efe53fff640 1 -- 172.21.15.47:0/847246225 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe541992b0 con 0x7efe54106170 2024-04-17T06:12:57.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.807+0000 7efe58dd1640 1 --2- 172.21.15.47:0/847246225 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efe541010a0 0x7efe5419e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T06:12:57.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.807+0000 7efe53fff640 1 --2- 172.21.15.47:0/847246225 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efe54106170 0x7efe5419eb30 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7efe3c036040 tx=0x7efe3c02ce20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:12:57.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.808+0000 7efe51ffb640 1 -- 172.21.15.47:0/847246225 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe3c041070 con 0x7efe54106170 2024-04-17T06:12:57.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.808+0000 7efe51ffb640 1 -- 172.21.15.47:0/847246225 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe3c004450 con 0x7efe54106170 2024-04-17T06:12:57.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.808+0000 7efe5b035640 1 -- 172.21.15.47:0/847246225 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe54199540 con 0x7efe54106170 2024-04-17T06:12:57.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.808+0000 7efe51ffb640 1 -- 172.21.15.47:0/847246225 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe3c003e20 con 0x7efe54106170 2024-04-17T06:12:57.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.808+0000 7efe5b035640 1 -- 172.21.15.47:0/847246225 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe541a5680 con 0x7efe54106170 2024-04-17T06:12:57.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.809+0000 7efe5b035640 1 -- 172.21.15.47:0/847246225 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe54102570 con 0x7efe54106170 2024-04-17T06:12:57.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.810+0000 7efe51ffb640 1 -- 172.21.15.47:0/847246225 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7efe3c044070 con 0x7efe54106170 2024-04-17T06:12:57.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.811+0000 7efe51ffb640 1 --2- 172.21.15.47:0/847246225 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7efe400a3d20 0x7efe400a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:57.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.811+0000 7efe58dd1640 1 --2- 172.21.15.47:0/847246225 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7efe400a3d20 0x7efe400a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:12:57.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.811+0000 7efe51ffb640 1 -- 172.21.15.47:0/847246225 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8469+0+0 (secure 0 0 0) 0x7efe3c0f22c0 con 0x7efe54106170 2024-04-17T06:12:57.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.811+0000 7efe58dd1640 1 --2- 172.21.15.47:0/847246225 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7efe400a3d20 0x7efe400a61e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7efe44009d60 tx=0x7efe44009110 comp rx=0 tx=0).ready entity=mgr.24923 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:12:57.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:57.815+0000 7efe51ffb640 1 -- 172.21.15.47:0/847246225 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7efe3c0bead0 con 0x7efe54106170 2024-04-17T06:12:58.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.003+0000 7efe5b035640 1 -- 172.21.15.47:0/847246225 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efe54101a50 con 0x7efe54106170 2024-04-17T06:12:58.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.004+0000 7efe51ffb640 1 -- 172.21.15.47:0/847246225 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v104) ==== 74+0+23006 (secure 0 0 0) 0x7efe3c0be3d0 con 0x7efe54106170 2024-04-17T06:12:58.003 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-17T06:12:58.003 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":104,"fsid":"565db2d8-fc7e-11ee-bc91-c7b262605968","created":"2024-04-17T05:51:19.888865+0000","modified":"2024-04-17T06:12:15.699533+0000","last_up_change":"2024-04-17T05:58:24.380131+0000","last_in_change":"2024-04-17T05:58:04.418236+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T05:54:37.734306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T05:59:12.458010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T05:59:12.901339+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":102,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8453f300-533d-4736-a343-55619c08e966","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":1290127544},{"type":"v1","addr":"172.21.15.47:6803","nonce":1290127544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":1290127544},{"type":"v1","addr":"172.21.15.47:6805","nonce":1290127544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":1290127544},{"type":"v1","addr":"172.21.15.47:6809","nonce":1290127544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":1290127544},{"type":"v1","addr":"172.21.15.47:6807","nonce":1290127544}]},"public_addr":"172.21.15.47:6803/1290127544","cluster_addr":"172.21.15.47:6805/1290127544","heartbeat_back_addr":"172.21.15.47:6809/1290127544","heartbeat_front_addr":"172.21.15.47:6807/1290127544","state":["exists","up"]},{"osd":1,"uuid":"80b0f03e-c1bc-4658-8028-1e6b8eedcc47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":1180430954},{"type":"v1","addr":"172.21.15.98:6801","nonce":1180430954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":1180430954},{"type":"v1","addr":"172.21.15.98:6803","nonce":1180430954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":1180430954},{"type":"v1","addr":"172.21.15.98:6807","nonce":1180430954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":1180430954},{"type":"v1","addr":"172.21.15.98:6805","nonce":1180430954}]},"public_addr":"172.21.15.98:6801/1180430954","cluster_addr":"172.21.15.98:6803/1180430954","heartbeat_back_addr":"172.21.15.98:6807/1180430954","heartbeat_front_addr":"172.21.15.98:6805/1180430954","state":["exists","up"]},{"osd":2,"uuid":"0fc9d383-05b4-498d-b530-fccf7aeddd7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":494676216},{"type":"v1","addr":"172.21.15.103:6801","nonce":494676216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":494676216},{"type":"v1","addr":"172.21.15.103:6803","nonce":494676216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":494676216},{"type":"v1","addr":"172.21.15.103:6807","nonce":494676216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":494676216},{"type":"v1","addr":"172.21.15.103:6805","nonce":494676216}]},"public_addr":"172.21.15.103:6801/494676216","cluster_addr":"172.21.15.103:6803/494676216","heartbeat_back_addr":"172.21.15.103:6807/494676216","heartbeat_front_addr":"172.21.15.103:6805/494676216","state":["exists","up"]},{"osd":3,"uuid":"f6e0b0d5-0fb3-42d0-9e84-f7cfef21a998","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":1998369447},{"type":"v1","addr":"172.21.15.47:6811","nonce":1998369447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":1998369447},{"type":"v1","addr":"172.21.15.47:6813","nonce":1998369447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":1998369447},{"type":"v1","addr":"172.21.15.47:6817","nonce":1998369447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":1998369447},{"type":"v1","addr":"172.21.15.47:6815","nonce":1998369447}]},"public_addr":"172.21.15.47:6811/1998369447","cluster_addr":"172.21.15.47:6813/1998369447","heartbeat_back_addr":"172.21.15.47:6817/1998369447","heartbeat_front_addr":"172.21.15.47:6815/1998369447","state":["exists","up"]},{"osd":4,"uuid":"2c26217e-02df-466d-bc01-0149d4aec3c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":3060482485},{"type":"v1","addr":"172.21.15.98:6809","nonce":3060482485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":3060482485},{"type":"v1","addr":"172.21.15.98:6811","nonce":3060482485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":3060482485},{"type":"v1","addr":"172.21.15.98:6815","nonce":3060482485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":3060482485},{"type":"v1","addr":"172.21.15.98:6813","nonce":3060482485}]},"public_addr":"172.21.15.98:6809/3060482485","cluster_addr":"172.21.15.98:6811/3060482485","heartbeat_back_addr":"172.21.15.98:6815/3060482485","heartbeat_front_addr":"172.21.15.98:6813/3060482485","state":["exists","up"]},{"osd":5,"uuid":"740bf3e0-bea8-496b-8d6f-e42d0b017258","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":4181363567},{"type":"v1","addr":"172.21.15.103:6809","nonce":4181363567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":4181363567},{"type":"v1","addr":"172.21.15.103:6811","nonce":4181363567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":4181363567},{"type":"v1","addr":"172.21.15.103:6815","nonce":4181363567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":4181363567},{"type":"v1","addr":"172.21.15.103:6813","nonce":4181363567}]},"public_addr":"172.21.15.103:6809/4181363567","cluster_addr":"172.21.15.103:6811/4181363567","heartbeat_back_addr":"172.21.15.103:6815/4181363567","heartbeat_front_addr":"172.21.15.103:6813/4181363567","state":["exists","up"]},{"osd":6,"uuid":"356a6255-2c07-4f0d-a233-26be04c92b38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":2517529668},{"type":"v1","addr":"172.21.15.47:6819","nonce":2517529668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2517529668},{"type":"v1","addr":"172.21.15.47:6821","nonce":2517529668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2517529668},{"type":"v1","addr":"172.21.15.47:6825","nonce":2517529668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2517529668},{"type":"v1","addr":"172.21.15.47:6823","nonce":2517529668}]},"public_addr":"172.21.15.47:6819/2517529668","cluster_addr":"172.21.15.47:6821/2517529668","heartbeat_back_addr":"172.21.15.47:6825/2517529668","heartbeat_front_addr":"172.21.15.47:6823/2517529668","state":["exists","up"]},{"osd":7,"uuid":"349d0120-8290-417d-a800-5a8985f554f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":4225498496},{"type":"v1","addr":"172.21.15.98:6817","nonce":4225498496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":4225498496},{"type":"v1","addr":"172.21.15.98:6819","nonce":4225498496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":4225498496},{"type":"v1","addr":"172.21.15.98:6823","nonce":4225498496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":4225498496},{"type":"v1","addr":"172.21.15.98:6821","nonce":4225498496}]},"public_addr":"172.21.15.98:6817/4225498496","cluster_addr":"172.21.15.98:6819/4225498496","heartbeat_back_addr":"172.21.15.98:6823/4225498496","heartbeat_front_addr":"172.21.15.98:6821/4225498496","state":["exists","up"]},{"osd":8,"uuid":"cee4712f-321b-42c0-92ba-e15144c00f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":3433905520},{"type":"v1","addr":"172.21.15.103:6817","nonce":3433905520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":3433905520},{"type":"v1","addr":"172.21.15.103:6819","nonce":3433905520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":3433905520},{"type":"v1","addr":"172.21.15.103:6823","nonce":3433905520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":3433905520},{"type":"v1","addr":"172.21.15.103:6821","nonce":3433905520}]},"public_addr":"172.21.15.103:6817/3433905520","cluster_addr":"172.21.15.103:6819/3433905520","heartbeat_back_addr":"172.21.15.103:6823/3433905520","heartbeat_front_addr":"172.21.15.103:6821/3433905520","state":["exists","up"]},{"osd":9,"uuid":"63fc73c1-5a22-487d-80b8-9894e2ddd207","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":728359554},{"type":"v1","addr":"172.21.15.47:6827","nonce":728359554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":728359554},{"type":"v1","addr":"172.21.15.47:6829","nonce":728359554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":728359554},{"type":"v1","addr":"172.21.15.47:6833","nonce":728359554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":728359554},{"type":"v1","addr":"172.21.15.47:6831","nonce":728359554}]},"public_addr":"172.21.15.47:6827/728359554","cluster_addr":"172.21.15.47:6829/728359554","heartbeat_back_addr":"172.21.15.47:6833/728359554","heartbeat_front_addr":"172.21.15.47:6831/728359554","state":["exists","up"]},{"osd":10,"uuid":"9565bcc3-62ee-48eb-a07a-03a81e0cc8e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":468205689},{"type":"v1","addr":"172.21.15.98:6825","nonce":468205689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":468205689},{"type":"v1","addr":"172.21.15.98:6827","nonce":468205689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":468205689},{"type":"v1","addr":"172.21.15.98:6831","nonce":468205689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":468205689},{"type":"v1","addr":"172.21.15.98:6829","nonce":468205689}]},"public_addr":"172.21.15.98:6825/468205689","cluster_addr":"172.21.15.98:6827/468205689","heartbeat_back_addr":"172.21.15.98:6831/468205689","heartbeat_front_addr":"172.21.15.98:6829/468205689","state":["exists","up"]},{"osd":11,"uuid":"2e85e8d9-bf6c-4a12-b124-a4690dd61108","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":2639831013},{"type":"v1","addr":"172.21.15.103:6825","nonce":2639831013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":2639831013},{"type":"v1","addr":"172.21.15.103:6827","nonce":2639831013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":2639831013},{"type":"v1","addr":"172.21.15.103:6831","nonce":2639831013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":2639831013},{"type":"v1","addr":"172.21.15.103:6829","nonce":2639831013}]},"public_addr":"172.21.15.103:6825/2639831013","cluster_addr":"172.21.15.103:6827/2639831013","heartbeat_back_addr":"172.21.15.103:6831/2639831013","heartbeat_front_addr":"172.21.15.103:6829/2639831013","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:53:47.366438+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:54:10.953559+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:54:34.320332+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:54:59.751360+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:55:23.982052+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:55:48.425017+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:56:14.793368+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:56:40.440830+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:57:04.623680+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:57:31.215262+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:57:57.040346+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:58:22.671572+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:6800/3157902806":"2024-04-18T06:01:18.246671+0000","172.21.15.47:6801/3157902806":"2024-04-18T06:01:18.246671+0000","172.21.15.47:0/3681048987":"2024-04-18T06:01:18.246671+0000","172.21.15.47:0/3009926768":"2024-04-18T06:01:18.246671+0000","172.21.15.47:0/2970926607":"2024-04-18T06:01:18.246671+0000","172.21.15.47:6801/3632157175":"2024-04-18T05:52:13.645775+0000","172.21.15.47:0/4288219269":"2024-04-18T05:52:13.645775+0000","172.21.15.47:0/2584487544":"2024-04-18T05:52:13.645775+0000","172.21.15.47:6801/953064786":"2024-04-18T05:51:45.979351+0000","172.21.15.47:0/3484500927":"2024-04-18T05:51:45.979351+0000","172.21.15.47:6800/3632157175":"2024-04-18T05:52:13.645775+0000","172.21.15.47:0/112268670":"2024-04-18T05:51:45.979351+0000","172.21.15.47:0/1638423256":"2024-04-18T05:51:45.979351+0000","172.21.15.47:0/3275066738":"2024-04-18T06:01:18.246671+0000","172.21.15.47:6800/953064786":"2024-04-18T05:51:45.979351+0000","172.21.15.47:0/849479010":"2024-04-18T05:52:13.645775+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T06:12:58.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.006+0000 7efe5b035640 1 -- 172.21.15.47:0/847246225 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7efe400a3d20 msgr2=0x7efe400a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:12:58.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.006+0000 7efe5b035640 1 --2- 172.21.15.47:0/847246225 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7efe400a3d20 0x7efe400a61e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7efe44009d60 tx=0x7efe44009110 comp rx=0 tx=0).stop 2024-04-17T06:12:58.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.007+0000 7efe5b035640 1 -- 172.21.15.47:0/847246225 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efe54106170 msgr2=0x7efe5419eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:12:58.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.007+0000 7efe5b035640 1 --2- 172.21.15.47:0/847246225 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efe54106170 0x7efe5419eb30 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7efe3c036040 tx=0x7efe3c02ce20 comp rx=0 tx=0).stop 2024-04-17T06:12:58.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.007+0000 7efe5b035640 1 -- 172.21.15.47:0/847246225 shutdown_connections 2024-04-17T06:12:58.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.007+0000 7efe5b035640 1 --2- 172.21.15.47:0/847246225 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efe541986b0 0x7efe54198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:58.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.007+0000 7efe5b035640 1 --2- 172.21.15.47:0/847246225 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7efe400a3d20 0x7efe400a61e0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:58.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.007+0000 7efe5b035640 1 --2- 172.21.15.47:0/847246225 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efe54106170 0x7efe5419eb30 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:58.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.007+0000 7efe5b035640 1 --2- 172.21.15.47:0/847246225 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efe541010a0 0x7efe5419e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:58.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.007+0000 7efe5b035640 1 -- 172.21.15.47:0/847246225 >> 172.21.15.47:0/847246225 conn(0x7efe540771e0 msgr2=0x7efe5406a920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:12:58.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.007+0000 7efe5b035640 1 -- 172.21.15.47:0/847246225 shutdown_connections 2024-04-17T06:12:58.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.007+0000 7efe5b035640 1 -- 172.21.15.47:0/847246225 wait complete. 2024-04-17T06:12:58.015 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:12:57 smithi047 ceph-mon[32223]: pgmap v367: 97 pgs: 97 active+clean; 5.8 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.3 KiB/s wr, 9 op/s 2024-04-17T06:12:58.016 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T05:54:37.734306+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T05:59:12.458010+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T05:59:12.901339+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '102', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 17, 'snap_epoch': 102, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T06:12:58.017 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T06:12:58.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.128+0000 7f61be44a640 1 -- 172.21.15.47:0/1318469199 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f61b81021f0 msgr2=0x7f61b81025d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:12:58.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.128+0000 7f61be44a640 1 --2- 172.21.15.47:0/1318469199 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f61b81021f0 0x7f61b81025d0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f61a8009900 tx=0x7f61a802bc30 comp rx=0 tx=0).stop 2024-04-17T06:12:58.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.128+0000 7f61be44a640 1 -- 172.21.15.47:0/1318469199 shutdown_connections 2024-04-17T06:12:58.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.128+0000 7f61be44a640 1 --2- 172.21.15.47:0/1318469199 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f61b81072c0 0x7f61b810ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:58.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.128+0000 7f61be44a640 1 --2- 172.21.15.47:0/1318469199 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f61b8102ba0 0x7f61b8106be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:58.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.128+0000 7f61be44a640 1 --2- 172.21.15.47:0/1318469199 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f61b81021f0 0x7f61b81025d0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:58.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.128+0000 7f61be44a640 1 -- 172.21.15.47:0/1318469199 >> 172.21.15.47:0/1318469199 conn(0x7f61b80fda70 msgr2=0x7f61b80ffe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:12:58.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.128+0000 7f61be44a640 1 -- 172.21.15.47:0/1318469199 shutdown_connections 2024-04-17T06:12:58.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.128+0000 7f61be44a640 1 -- 172.21.15.47:0/1318469199 wait complete. 2024-04-17T06:12:58.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.129+0000 7f61be44a640 1 Processor -- start 2024-04-17T06:12:58.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.129+0000 7f61be44a640 1 -- start start 2024-04-17T06:12:58.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.129+0000 7f61be44a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f61b81021f0 0x7f61b819e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:58.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.129+0000 7f61be44a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f61b81072c0 0x7f61b819eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:58.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.129+0000 7f61be44a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f61b8198680 0x7f61b8198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:58.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.129+0000 7f61be44a640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f61b8112380 con 0x7f61b8198680 2024-04-17T06:12:58.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.129+0000 7f61be44a640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f61b8112200 con 0x7f61b81021f0 2024-04-17T06:12:58.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.129+0000 7f61be44a640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f61b8112500 con 0x7f61b81072c0 2024-04-17T06:12:58.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.129+0000 7f61bc9e7640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f61b8198680 0x7f61b8198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:12:58.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.129+0000 7f61bc9e7640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f61b8198680 0x7f61b8198b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35054/0 (socket says 172.21.15.47:35054) 2024-04-17T06:12:58.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.129+0000 7f61bc9e7640 1 -- 172.21.15.47:0/1246705630 learned_addr learned my addr 172.21.15.47:0/1246705630 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:12:58.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.130+0000 7f61b77fe640 1 --2- 172.21.15.47:0/1246705630 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f61b81072c0 0x7f61b819eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:12:58.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.130+0000 7f61b7fff640 1 --2- 172.21.15.47:0/1246705630 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f61b81021f0 0x7f61b819e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:12:58.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.130+0000 7f61bc9e7640 1 -- 172.21.15.47:0/1246705630 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f61b81072c0 msgr2=0x7f61b819eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:12:58.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.130+0000 7f61bc9e7640 1 --2- 172.21.15.47:0/1246705630 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f61b81072c0 0x7f61b819eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:58.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.130+0000 7f61bc9e7640 1 -- 172.21.15.47:0/1246705630 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f61b81021f0 msgr2=0x7f61b819e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:12:58.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.130+0000 7f61bc9e7640 1 --2- 172.21.15.47:0/1246705630 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f61b81021f0 0x7f61b819e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:58.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.130+0000 7f61bc9e7640 1 -- 172.21.15.47:0/1246705630 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61b8199280 con 0x7f61b8198680 2024-04-17T06:12:58.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.130+0000 7f61bc9e7640 1 --2- 172.21.15.47:0/1246705630 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f61b8198680 0x7f61b8198b00 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f61b000eaa0 tx=0x7f61b000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:12:58.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.130+0000 7f61b57fa640 1 -- 172.21.15.47:0/1246705630 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b0004d20 con 0x7f61b8198680 2024-04-17T06:12:58.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.130+0000 7f61be44a640 1 -- 172.21.15.47:0/1246705630 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61b81a5360 con 0x7f61b8198680 2024-04-17T06:12:58.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.130+0000 7f61b57fa640 1 -- 172.21.15.47:0/1246705630 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61b0005020 con 0x7f61b8198680 2024-04-17T06:12:58.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.130+0000 7f61b57fa640 1 -- 172.21.15.47:0/1246705630 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b00116d0 con 0x7f61b8198680 2024-04-17T06:12:58.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.130+0000 7f61be44a640 1 -- 172.21.15.47:0/1246705630 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61b81a5770 con 0x7f61b8198680 2024-04-17T06:12:58.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.131+0000 7f61be44a640 1 -- 172.21.15.47:0/1246705630 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61b81036c0 con 0x7f61b8198680 2024-04-17T06:12:58.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.132+0000 7f61b57fa640 1 -- 172.21.15.47:0/1246705630 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f61b00051d0 con 0x7f61b8198680 2024-04-17T06:12:58.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.133+0000 7f61b57fa640 1 --2- 172.21.15.47:0/1246705630 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f61a00a3d20 0x7f61a00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:58.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.133+0000 7f61b7fff640 1 --2- 172.21.15.47:0/1246705630 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f61a00a3d20 0x7f61a00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:12:58.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.133+0000 7f61b57fa640 1 -- 172.21.15.47:0/1246705630 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8469+0+0 (secure 0 0 0) 0x7f61b00d0990 con 0x7f61b8198680 2024-04-17T06:12:58.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.133+0000 7f61b7fff640 1 --2- 172.21.15.47:0/1246705630 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f61a00a3d20 0x7f61a00a61e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f61a8036040 tx=0x7f61a8005fd0 comp rx=0 tx=0).ready entity=mgr.24923 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:12:58.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.138+0000 7f61b57fa640 1 -- 172.21.15.47:0/1246705630 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f61b0016030 con 0x7f61b8198680 2024-04-17T06:12:58.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:12:57 smithi103 ceph-mon[36243]: pgmap v367: 97 pgs: 97 active+clean; 5.8 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.3 KiB/s wr, 9 op/s 2024-04-17T06:12:58.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.324+0000 7f61be44a640 1 -- 172.21.15.47:0/1246705630 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f61b8102ba0 con 0x7f61b8198680 2024-04-17T06:12:58.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.325+0000 7f61b57fa640 1 -- 172.21.15.47:0/1246705630 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v104) ==== 93+0+10 (secure 0 0 0) 0x7f61b009d000 con 0x7f61b8198680 2024-04-17T06:12:58.323 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-04-17T06:12:58.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.327+0000 7f61be44a640 1 -- 172.21.15.47:0/1246705630 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f61a00a3d20 msgr2=0x7f61a00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:12:58.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.327+0000 7f61be44a640 1 --2- 172.21.15.47:0/1246705630 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f61a00a3d20 0x7f61a00a61e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f61a8036040 tx=0x7f61a8005fd0 comp rx=0 tx=0).stop 2024-04-17T06:12:58.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.327+0000 7f61be44a640 1 -- 172.21.15.47:0/1246705630 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f61b8198680 msgr2=0x7f61b8198b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:12:58.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.327+0000 7f61be44a640 1 --2- 172.21.15.47:0/1246705630 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f61b8198680 0x7f61b8198b00 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f61b000eaa0 tx=0x7f61b000eea0 comp rx=0 tx=0).stop 2024-04-17T06:12:58.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.327+0000 7f61be44a640 1 -- 172.21.15.47:0/1246705630 shutdown_connections 2024-04-17T06:12:58.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.327+0000 7f61be44a640 1 --2- 172.21.15.47:0/1246705630 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f61a00a3d20 0x7f61a00a61e0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:58.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.327+0000 7f61be44a640 1 --2- 172.21.15.47:0/1246705630 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f61b8198680 0x7f61b8198b00 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:58.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.327+0000 7f61be44a640 1 --2- 172.21.15.47:0/1246705630 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f61b81072c0 0x7f61b819eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:58.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.327+0000 7f61be44a640 1 --2- 172.21.15.47:0/1246705630 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f61b81021f0 0x7f61b819e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:58.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.327+0000 7f61be44a640 1 -- 172.21.15.47:0/1246705630 >> 172.21.15.47:0/1246705630 conn(0x7f61b80fda70 msgr2=0x7f61b80ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:12:58.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.327+0000 7f61be44a640 1 -- 172.21.15.47:0/1246705630 shutdown_connections 2024-04-17T06:12:58.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.327+0000 7f61be44a640 1 -- 172.21.15.47:0/1246705630 wait complete. 2024-04-17T06:12:58.336 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T06:12:58.382 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:12:57 smithi098 ceph-mon[36516]: pgmap v367: 97 pgs: 97 active+clean; 5.8 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.3 KiB/s wr, 9 op/s 2024-04-17T06:12:58.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.448+0000 7f4247ac8640 1 -- 172.21.15.47:0/3028016064 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4240106170 msgr2=0x7f424010dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:12:58.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.448+0000 7f4247ac8640 1 --2- 172.21.15.47:0/3028016064 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4240106170 0x7f424010dd40 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f4238009ca0 tx=0x7f423802c0f0 comp rx=0 tx=0).stop 2024-04-17T06:12:58.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.448+0000 7f4247ac8640 1 -- 172.21.15.47:0/3028016064 shutdown_connections 2024-04-17T06:12:58.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.448+0000 7f4247ac8640 1 --2- 172.21.15.47:0/3028016064 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4240106170 0x7f424010dd40 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:58.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.448+0000 7f4247ac8640 1 --2- 172.21.15.47:0/3028016064 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4240101a50 0x7f4240105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:58.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.448+0000 7f4247ac8640 1 --2- 172.21.15.47:0/3028016064 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f42401010a0 0x7f4240101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:58.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.448+0000 7f4247ac8640 1 -- 172.21.15.47:0/3028016064 >> 172.21.15.47:0/3028016064 conn(0x7f4240077f60 msgr2=0x7f4240078370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:12:58.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.448+0000 7f4247ac8640 1 -- 172.21.15.47:0/3028016064 shutdown_connections 2024-04-17T06:12:58.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.448+0000 7f4247ac8640 1 -- 172.21.15.47:0/3028016064 wait complete. 2024-04-17T06:12:58.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.448+0000 7f4247ac8640 1 Processor -- start 2024-04-17T06:12:58.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.448+0000 7f4247ac8640 1 -- start start 2024-04-17T06:12:58.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.449+0000 7f4247ac8640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f42401010a0 0x7f424019e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:58.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.449+0000 7f4247ac8640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4240106170 0x7f424019e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:58.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.449+0000 7f4247ac8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4240198600 0x7f4240198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:58.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.449+0000 7f4247ac8640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f4240112340 con 0x7f42401010a0 2024-04-17T06:12:58.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.449+0000 7f4247ac8640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f42401121c0 con 0x7f4240106170 2024-04-17T06:12:58.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.449+0000 7f4247ac8640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f42401124c0 con 0x7f4240198600 2024-04-17T06:12:58.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.449+0000 7f4245864640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f42401010a0 0x7f424019e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:12:58.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.449+0000 7f4245864640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f42401010a0 0x7f424019e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35070/0 (socket says 172.21.15.47:35070) 2024-04-17T06:12:58.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.449+0000 7f4245864640 1 -- 172.21.15.47:0/2826897061 learned_addr learned my addr 172.21.15.47:0/2826897061 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:12:58.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.449+0000 7f4245063640 1 --2- 172.21.15.47:0/2826897061 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4240106170 0x7f424019e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:12:58.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.449+0000 7f4246065640 1 --2- 172.21.15.47:0/2826897061 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4240198600 0x7f4240198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:12:58.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.449+0000 7f4245864640 1 -- 172.21.15.47:0/2826897061 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4240198600 msgr2=0x7f4240198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:12:58.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.449+0000 7f4245864640 1 --2- 172.21.15.47:0/2826897061 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4240198600 0x7f4240198ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:58.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.449+0000 7f4245864640 1 -- 172.21.15.47:0/2826897061 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4240106170 msgr2=0x7f424019e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:12:58.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.449+0000 7f4245864640 1 --2- 172.21.15.47:0/2826897061 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4240106170 0x7f424019e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:58.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.449+0000 7f4245864640 1 -- 172.21.15.47:0/2826897061 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4240199120 con 0x7f42401010a0 2024-04-17T06:12:58.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.449+0000 7f4245864640 1 --2- 172.21.15.47:0/2826897061 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f42401010a0 0x7f424019e450 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f423c00d770 tx=0x7f423c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:12:58.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.450+0000 7f4236ffd640 1 -- 172.21.15.47:0/2826897061 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f423c017070 con 0x7f42401010a0 2024-04-17T06:12:58.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.450+0000 7f4236ffd640 1 -- 172.21.15.47:0/2826897061 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f423c005500 con 0x7f42401010a0 2024-04-17T06:12:58.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.450+0000 7f4247ac8640 1 -- 172.21.15.47:0/2826897061 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4240199410 con 0x7f42401010a0 2024-04-17T06:12:58.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.450+0000 7f4236ffd640 1 -- 172.21.15.47:0/2826897061 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f423c004e00 con 0x7f42401010a0 2024-04-17T06:12:58.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.450+0000 7f4247ac8640 1 -- 172.21.15.47:0/2826897061 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42401a5610 con 0x7f42401010a0 2024-04-17T06:12:58.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.451+0000 7f4247ac8640 1 -- 172.21.15.47:0/2826897061 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4240068d80 con 0x7f42401010a0 2024-04-17T06:12:58.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.452+0000 7f4236ffd640 1 -- 172.21.15.47:0/2826897061 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f423c004fa0 con 0x7f42401010a0 2024-04-17T06:12:58.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.452+0000 7f4236ffd640 1 --2- 172.21.15.47:0/2826897061 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f422c0a3d20 0x7f422c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:58.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.453+0000 7f4245063640 1 --2- 172.21.15.47:0/2826897061 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f422c0a3d20 0x7f422c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:12:58.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.453+0000 7f4236ffd640 1 -- 172.21.15.47:0/2826897061 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8469+0+0 (secure 0 0 0) 0x7f423c0d01a0 con 0x7f42401010a0 2024-04-17T06:12:58.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.453+0000 7f4245063640 1 --2- 172.21.15.47:0/2826897061 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f422c0a3d20 0x7f422c0a61e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f42401008c0 tx=0x7f4230005f90 comp rx=0 tx=0).ready entity=mgr.24923 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:12:58.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.457+0000 7f4236ffd640 1 -- 172.21.15.47:0/2826897061 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f423c09c220 con 0x7f42401010a0 2024-04-17T06:12:58.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.647+0000 7f4247ac8640 1 -- 172.21.15.47:0/2826897061 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f4240101a50 con 0x7f42401010a0 2024-04-17T06:12:58.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.647+0000 7f4236ffd640 1 -- 172.21.15.47:0/2826897061 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v104) ==== 104+0+11 (secure 0 0 0) 0x7f423c0a0520 con 0x7f42401010a0 2024-04-17T06:12:58.645 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-04-17T06:12:58.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.649+0000 7f4247ac8640 1 -- 172.21.15.47:0/2826897061 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f422c0a3d20 msgr2=0x7f422c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:12:58.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.649+0000 7f4247ac8640 1 --2- 172.21.15.47:0/2826897061 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f422c0a3d20 0x7f422c0a61e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f42401008c0 tx=0x7f4230005f90 comp rx=0 tx=0).stop 2024-04-17T06:12:58.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.649+0000 7f4247ac8640 1 -- 172.21.15.47:0/2826897061 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f42401010a0 msgr2=0x7f424019e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:12:58.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.649+0000 7f4247ac8640 1 --2- 172.21.15.47:0/2826897061 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f42401010a0 0x7f424019e450 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f423c00d770 tx=0x7f423c00dc30 comp rx=0 tx=0).stop 2024-04-17T06:12:58.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.649+0000 7f4247ac8640 1 -- 172.21.15.47:0/2826897061 shutdown_connections 2024-04-17T06:12:58.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.649+0000 7f4247ac8640 1 --2- 172.21.15.47:0/2826897061 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4240198600 0x7f4240198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:58.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.649+0000 7f4247ac8640 1 --2- 172.21.15.47:0/2826897061 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4240106170 0x7f424019e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:58.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.649+0000 7f4247ac8640 1 --2- 172.21.15.47:0/2826897061 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f422c0a3d20 0x7f422c0a61e0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:58.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.649+0000 7f4247ac8640 1 --2- 172.21.15.47:0/2826897061 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f42401010a0 0x7f424019e450 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:58.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.649+0000 7f4247ac8640 1 -- 172.21.15.47:0/2826897061 >> 172.21.15.47:0/2826897061 conn(0x7f4240077f60 msgr2=0x7f42400fee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:12:58.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.649+0000 7f4247ac8640 1 -- 172.21.15.47:0/2826897061 shutdown_connections 2024-04-17T06:12:58.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.649+0000 7f4247ac8640 1 -- 172.21.15.47:0/2826897061 wait complete. 2024-04-17T06:12:58.659 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T06:12:58.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.770+0000 7f52b1349640 1 -- 172.21.15.47:0/526657092 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52ac1014b0 msgr2=0x7f52ac105250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:12:58.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.770+0000 7f52b1349640 1 --2- 172.21.15.47:0/526657092 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52ac1014b0 0x7f52ac105250 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f5294009900 tx=0x7f529402bc30 comp rx=0 tx=0).stop 2024-04-17T06:12:58.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.770+0000 7f52b1349640 1 -- 172.21.15.47:0/526657092 shutdown_connections 2024-04-17T06:12:58.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.770+0000 7f52b1349640 1 --2- 172.21.15.47:0/526657092 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f52ac105980 0x7f52ac10d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:58.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.770+0000 7f52b1349640 1 --2- 172.21.15.47:0/526657092 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52ac1014b0 0x7f52ac105250 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:58.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.770+0000 7f52b1349640 1 --2- 172.21.15.47:0/526657092 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f52ac100b00 0x7f52ac100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:58.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.770+0000 7f52b1349640 1 -- 172.21.15.47:0/526657092 >> 172.21.15.47:0/526657092 conn(0x7f52ac0760f0 msgr2=0x7f52ac076500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:12:58.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.770+0000 7f52b1349640 1 -- 172.21.15.47:0/526657092 shutdown_connections 2024-04-17T06:12:58.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.770+0000 7f52b1349640 1 -- 172.21.15.47:0/526657092 wait complete. 2024-04-17T06:12:58.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.771+0000 7f52b1349640 1 Processor -- start 2024-04-17T06:12:58.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.771+0000 7f52b1349640 1 -- start start 2024-04-17T06:12:58.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.771+0000 7f52b1349640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f52ac100b00 0x7f52ac19e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:58.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.771+0000 7f52b1349640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52ac105980 0x7f52ac19ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:58.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.771+0000 7f52b1349640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f52ac1986f0 0x7f52ac198b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:58.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.771+0000 7f52b1349640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f52ac111220 con 0x7f52ac105980 2024-04-17T06:12:58.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.771+0000 7f52b1349640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f52ac1110a0 con 0x7f52ac100b00 2024-04-17T06:12:58.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.771+0000 7f52b1349640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f52ac1113a0 con 0x7f52ac1986f0 2024-04-17T06:12:58.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.772+0000 7f52a27fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52ac105980 0x7f52ac19ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:12:58.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.772+0000 7f52a27fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52ac105980 0x7f52ac19ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35098/0 (socket says 172.21.15.47:35098) 2024-04-17T06:12:58.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.772+0000 7f52a27fc640 1 -- 172.21.15.47:0/2428602217 learned_addr learned my addr 172.21.15.47:0/2428602217 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:12:58.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.772+0000 7f52ab7fe640 1 --2- 172.21.15.47:0/2428602217 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f52ac1986f0 0x7f52ac198b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:12:58.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.772+0000 7f52aaffd640 1 --2- 172.21.15.47:0/2428602217 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f52ac100b00 0x7f52ac19e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:12:58.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.772+0000 7f52a27fc640 1 -- 172.21.15.47:0/2428602217 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f52ac1986f0 msgr2=0x7f52ac198b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:12:58.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.772+0000 7f52a27fc640 1 --2- 172.21.15.47:0/2428602217 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f52ac1986f0 0x7f52ac198b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:58.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.772+0000 7f52a27fc640 1 -- 172.21.15.47:0/2428602217 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f52ac100b00 msgr2=0x7f52ac19e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:12:58.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.772+0000 7f52a27fc640 1 --2- 172.21.15.47:0/2428602217 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f52ac100b00 0x7f52ac19e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:58.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.772+0000 7f52a27fc640 1 -- 172.21.15.47:0/2428602217 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52ac1992f0 con 0x7f52ac105980 2024-04-17T06:12:58.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.772+0000 7f52aaffd640 1 --2- 172.21.15.47:0/2428602217 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f52ac100b00 0x7f52ac19e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:12:58.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.772+0000 7f52a27fc640 1 --2- 172.21.15.47:0/2428602217 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52ac105980 0x7f52ac19ead0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f5294036040 tx=0x7f529402ce20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:12:58.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.772+0000 7f52a8ff9640 1 -- 172.21.15.47:0/2428602217 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5294041070 con 0x7f52ac105980 2024-04-17T06:12:58.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.772+0000 7f52a8ff9640 1 -- 172.21.15.47:0/2428602217 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52940054e0 con 0x7f52ac105980 2024-04-17T06:12:58.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.772+0000 7f52b1349640 1 -- 172.21.15.47:0/2428602217 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52ac199580 con 0x7f52ac105980 2024-04-17T06:12:58.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.772+0000 7f52a8ff9640 1 -- 172.21.15.47:0/2428602217 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5294034440 con 0x7f52ac105980 2024-04-17T06:12:58.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.772+0000 7f52b1349640 1 -- 172.21.15.47:0/2428602217 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52ac1a5620 con 0x7f52ac105980 2024-04-17T06:12:58.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.773+0000 7f52b1349640 1 -- 172.21.15.47:0/2428602217 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52ac101d00 con 0x7f52ac105980 2024-04-17T06:12:58.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.775+0000 7f52a8ff9640 1 -- 172.21.15.47:0/2428602217 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5294044070 con 0x7f52ac105980 2024-04-17T06:12:58.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.775+0000 7f52a8ff9640 1 --2- 172.21.15.47:0/2428602217 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f52980a3d20 0x7f52980a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:58.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.776+0000 7f52aaffd640 1 --2- 172.21.15.47:0/2428602217 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f52980a3d20 0x7f52980a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:12:58.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.776+0000 7f52a8ff9640 1 -- 172.21.15.47:0/2428602217 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8469+0+0 (secure 0 0 0) 0x7f52940f22d0 con 0x7f52ac105980 2024-04-17T06:12:58.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.776+0000 7f52aaffd640 1 --2- 172.21.15.47:0/2428602217 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f52980a3d20 0x7f52980a61e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f52ac077830 tx=0x7f529c005d30 comp rx=0 tx=0).ready entity=mgr.24923 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:12:58.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.780+0000 7f52a8ff9640 1 -- 172.21.15.47:0/2428602217 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f52940beb40 con 0x7f52ac105980 2024-04-17T06:12:58.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.968+0000 7f52b1349640 1 -- 172.21.15.47:0/2428602217 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f52ac1014b0 con 0x7f52ac105980 2024-04-17T06:12:58.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.968+0000 7f52a8ff9640 1 -- 172.21.15.47:0/2428602217 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v104) ==== 100+0+11 (secure 0 0 0) 0x7f52940be440 con 0x7f52ac105980 2024-04-17T06:12:58.967 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-04-17T06:12:58.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.970+0000 7f52b1349640 1 -- 172.21.15.47:0/2428602217 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f52980a3d20 msgr2=0x7f52980a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:12:58.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.970+0000 7f52b1349640 1 --2- 172.21.15.47:0/2428602217 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f52980a3d20 0x7f52980a61e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f52ac077830 tx=0x7f529c005d30 comp rx=0 tx=0).stop 2024-04-17T06:12:58.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.970+0000 7f52b1349640 1 -- 172.21.15.47:0/2428602217 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52ac105980 msgr2=0x7f52ac19ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:12:58.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.970+0000 7f52b1349640 1 --2- 172.21.15.47:0/2428602217 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52ac105980 0x7f52ac19ead0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f5294036040 tx=0x7f529402ce20 comp rx=0 tx=0).stop 2024-04-17T06:12:58.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.970+0000 7f52b1349640 1 -- 172.21.15.47:0/2428602217 shutdown_connections 2024-04-17T06:12:58.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.970+0000 7f52b1349640 1 --2- 172.21.15.47:0/2428602217 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f52ac1986f0 0x7f52ac198b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:58.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.970+0000 7f52b1349640 1 --2- 172.21.15.47:0/2428602217 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f52980a3d20 0x7f52980a61e0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:58.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.970+0000 7f52b1349640 1 --2- 172.21.15.47:0/2428602217 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52ac105980 0x7f52ac19ead0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:58.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.970+0000 7f52b1349640 1 --2- 172.21.15.47:0/2428602217 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f52ac100b00 0x7f52ac19e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:58.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.970+0000 7f52b1349640 1 -- 172.21.15.47:0/2428602217 >> 172.21.15.47:0/2428602217 conn(0x7f52ac0760f0 msgr2=0x7f52ac104500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:12:58.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.971+0000 7f52b1349640 1 -- 172.21.15.47:0/2428602217 shutdown_connections 2024-04-17T06:12:58.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:58.971+0000 7f52b1349640 1 -- 172.21.15.47:0/2428602217 wait complete. 2024-04-17T06:12:58.980 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-04-17T06:12:59.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:59.088+0000 7f051d332640 1 -- 172.21.15.47:0/3885923845 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f05181011b0 msgr2=0x7f0518101630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:12:59.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:59.088+0000 7f051d332640 1 --2- 172.21.15.47:0/3885923845 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f05181011b0 0x7f0518101630 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f050c009900 tx=0x7f050c02bc80 comp rx=0 tx=0).stop 2024-04-17T06:12:59.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:59.088+0000 7f051d332640 1 -- 172.21.15.47:0/3885923845 shutdown_connections 2024-04-17T06:12:59.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:59.088+0000 7f051d332640 1 --2- 172.21.15.47:0/3885923845 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0518101b70 0x7f051810dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:59.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:59.088+0000 7f051d332640 1 --2- 172.21.15.47:0/3885923845 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f05181011b0 0x7f0518101630 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:59.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:59.088+0000 7f051d332640 1 --2- 172.21.15.47:0/3885923845 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0518105ef0 0x7f05181062d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:59.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:59.088+0000 7f051d332640 1 -- 172.21.15.47:0/3885923845 >> 172.21.15.47:0/3885923845 conn(0x7f0518077c30 msgr2=0x7f0518078040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:12:59.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:59.088+0000 7f051d332640 1 -- 172.21.15.47:0/3885923845 shutdown_connections 2024-04-17T06:12:59.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:59.088+0000 7f051d332640 1 -- 172.21.15.47:0/3885923845 wait complete. 2024-04-17T06:12:59.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:59.088+0000 7f051d332640 1 Processor -- start 2024-04-17T06:12:59.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:59.089+0000 7f051d332640 1 -- start start 2024-04-17T06:12:59.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:59.089+0000 7f051d332640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f05181011b0 0x7f051819e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:59.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:59.089+0000 7f051d332640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0518101b70 0x7f051819eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:59.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:59.089+0000 7f051d332640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0518105ef0 0x7f0518198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:59.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:59.089+0000 7f051d332640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f05181112a0 con 0x7f0518101b70 2024-04-17T06:12:59.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:59.089+0000 7f051d332640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0518111120 con 0x7f0518105ef0 2024-04-17T06:12:59.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:59.089+0000 7f051d332640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0518111420 con 0x7f05181011b0 2024-04-17T06:12:59.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:59.089+0000 7f05167fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0518101b70 0x7f051819eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:12:59.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:59.089+0000 7f05167fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0518101b70 0x7f051819eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35126/0 (socket says 172.21.15.47:35126) 2024-04-17T06:12:59.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:59.089+0000 7f05167fc640 1 -- 172.21.15.47:0/1368698928 learned_addr learned my addr 172.21.15.47:0/1368698928 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:12:59.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:59.089+0000 7f05177fe640 1 --2- 172.21.15.47:0/1368698928 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0518105ef0 0x7f0518198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:12:59.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:59.089+0000 7f0516ffd640 1 --2- 172.21.15.47:0/1368698928 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f05181011b0 0x7f051819e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:12:59.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:59.089+0000 7f05167fc640 1 -- 172.21.15.47:0/1368698928 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f05181011b0 msgr2=0x7f051819e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:12:59.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:59.089+0000 7f05167fc640 1 --2- 172.21.15.47:0/1368698928 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f05181011b0 0x7f051819e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:59.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:59.089+0000 7f05167fc640 1 -- 172.21.15.47:0/1368698928 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0518105ef0 msgr2=0x7f0518198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:12:59.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:59.089+0000 7f05167fc640 1 --2- 172.21.15.47:0/1368698928 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0518105ef0 0x7f0518198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:12:59.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:59.089+0000 7f05167fc640 1 -- 172.21.15.47:0/1368698928 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0518198ff0 con 0x7f0518101b70 2024-04-17T06:12:59.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:59.089+0000 7f05177fe640 1 --2- 172.21.15.47:0/1368698928 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0518105ef0 0x7f0518198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:12:59.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:59.089+0000 7f05167fc640 1 --2- 172.21.15.47:0/1368698928 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0518101b70 0x7f051819eb60 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f050c02c290 tx=0x7f050c0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:12:59.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:59.090+0000 7f04fffff640 1 -- 172.21.15.47:0/1368698928 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f050c043020 con 0x7f0518101b70 2024-04-17T06:12:59.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:59.090+0000 7f04fffff640 1 -- 172.21.15.47:0/1368698928 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f050c0054e0 con 0x7f0518101b70 2024-04-17T06:12:59.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:59.090+0000 7f051d332640 1 -- 172.21.15.47:0/1368698928 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0518199280 con 0x7f0518101b70 2024-04-17T06:12:59.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:59.090+0000 7f04fffff640 1 -- 172.21.15.47:0/1368698928 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f050c003da0 con 0x7f0518101b70 2024-04-17T06:12:59.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:59.090+0000 7f051d332640 1 -- 172.21.15.47:0/1368698928 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05181a55a0 con 0x7f0518101b70 2024-04-17T06:12:59.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:59.091+0000 7f051d332640 1 -- 172.21.15.47:0/1368698928 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0518102580 con 0x7f0518101b70 2024-04-17T06:12:59.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:59.092+0000 7f04fffff640 1 -- 172.21.15.47:0/1368698928 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f050c046020 con 0x7f0518101b70 2024-04-17T06:12:59.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:59.092+0000 7f04fffff640 1 --2- 172.21.15.47:0/1368698928 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f05000a3d20 0x7f05000a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:12:59.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:59.092+0000 7f0516ffd640 1 --2- 172.21.15.47:0/1368698928 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f05000a3d20 0x7f05000a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:12:59.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:59.092+0000 7f04fffff640 1 -- 172.21.15.47:0/1368698928 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8469+0+0 (secure 0 0 0) 0x7f050c0f2960 con 0x7f0518101b70 2024-04-17T06:12:59.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:59.093+0000 7f0516ffd640 1 --2- 172.21.15.47:0/1368698928 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f05000a3d20 0x7f05000a61e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f05080058c0 tx=0x7f0508005be0 comp rx=0 tx=0).ready entity=mgr.24923 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:12:59.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:59.097+0000 7f04fffff640 1 -- 172.21.15.47:0/1368698928 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f050c04d020 con 0x7f0518101b70 2024-04-17T06:12:59.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:12:58 smithi103 ceph-mon[36243]: from='client.16377 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T06:12:59.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:12:58 smithi103 ceph-mon[36243]: from='client.16383 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T06:12:59.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:12:58 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/847246225' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T06:12:59.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:12:58 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/1246705630' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-17T06:12:59.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:12:58 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/2826897061' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-17T06:12:59.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:59.334+0000 7f051d332640 1 -- 172.21.15.47:0/1368698928 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f0518199c10 con 0x7f0518101b70 2024-04-17T06:12:59.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:59.342+0000 7f04fffff640 1 -- 172.21.15.47:0/1368698928 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v26) ==== 81+0+746059 (secure 0 0 0) 0x7f050c0bf140 con 0x7f0518101b70 2024-04-17T06:12:59.351 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-17T06:12:59.352 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-17T06:12:59.352 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 26, 2024-04-17T06:12:59.352 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.352 INFO:teuthology.orchestra.run.smithi047.stdout: "active_gid": 24923, 2024-04-17T06:12:59.352 INFO:teuthology.orchestra.run.smithi047.stdout: "active_name": "x", 2024-04-17T06:12:59.352 INFO:teuthology.orchestra.run.smithi047.stdout: "active_addrs": { 2024-04-17T06:12:59.352 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-17T06:12:59.352 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:12:59.352 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-17T06:12:59.352 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6800", 2024-04-17T06:12:59.352 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 255579447 2024-04-17T06:12:59.352 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.353 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:12:59.353 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-17T06:12:59.353 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6801", 2024-04-17T06:12:59.353 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 255579447 2024-04-17T06:12:59.353 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:12:59.353 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-17T06:12:59.353 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.353 INFO:teuthology.orchestra.run.smithi047.stdout: "active_addr": "172.21.15.47:6801/255579447", 2024-04-17T06:12:59.353 INFO:teuthology.orchestra.run.smithi047.stdout: "active_change": "2024-04-17T06:01:18.246873+0000", 2024-04-17T06:12:59.353 INFO:teuthology.orchestra.run.smithi047.stdout: "active_mgr_features": 4540701547738038271, 2024-04-17T06:12:59.353 INFO:teuthology.orchestra.run.smithi047.stdout: "available": true, 2024-04-17T06:12:59.353 INFO:teuthology.orchestra.run.smithi047.stdout: "standbys": [ 2024-04-17T06:12:59.353 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:12:59.354 INFO:teuthology.orchestra.run.smithi047.stdout: "gid": 24880, 2024-04-17T06:12:59.354 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "z", 2024-04-17T06:12:59.354 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr_features": 4540701547738038271, 2024-04-17T06:12:59.354 INFO:teuthology.orchestra.run.smithi047.stdout: "available_modules": [ 2024-04-17T06:12:59.354 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:12:59.354 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "alerts", 2024-04-17T06:12:59.354 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-17T06:12:59.354 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-17T06:12:59.354 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-17T06:12:59.354 INFO:teuthology.orchestra.run.smithi047.stdout: "interval": { 2024-04-17T06:12:59.354 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "interval", 2024-04-17T06:12:59.354 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-17T06:12:59.354 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.355 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.355 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "60", 2024-04-17T06:12:59.355 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.355 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.355 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.355 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "How frequently to reexamine health status", 2024-04-17T06:12:59.355 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.355 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.355 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.355 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.355 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-17T06:12:59.355 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-17T06:12:59.355 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.356 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.356 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.356 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:12:59.356 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.356 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.356 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:12:59.356 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:12:59.356 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:12:59.356 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:12:59.356 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:12:59.356 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:12:59.356 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:12:59.357 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:12:59.357 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.357 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.357 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.357 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.357 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.357 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-17T06:12:59.357 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-17T06:12:59.357 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.357 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.357 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.357 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.358 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.358 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.358 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.358 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.358 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.358 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.358 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.358 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.358 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-17T06:12:59.358 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-17T06:12:59.358 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.358 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.359 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.359 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-17T06:12:59.359 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.359 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.359 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:12:59.359 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:12:59.359 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:12:59.359 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:12:59.359 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:12:59.359 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:12:59.359 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:12:59.359 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:12:59.360 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.360 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.360 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.360 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.360 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.360 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-17T06:12:59.360 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-17T06:12:59.360 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.360 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.360 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.360 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.360 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.360 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.361 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.361 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.361 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.361 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.361 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.361 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.361 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_destination": { 2024-04-17T06:12:59.361 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_destination", 2024-04-17T06:12:59.361 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.361 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.361 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.362 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:12:59.362 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.362 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.362 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.362 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Email address to send alerts to", 2024-04-17T06:12:59.362 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.362 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.362 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.362 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.362 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_from_name": { 2024-04-17T06:12:59.362 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_from_name", 2024-04-17T06:12:59.362 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.363 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.363 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.363 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "Ceph", 2024-04-17T06:12:59.363 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.363 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.363 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.363 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Email From: name", 2024-04-17T06:12:59.363 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.363 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.363 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.363 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.363 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_host": { 2024-04-17T06:12:59.363 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_host", 2024-04-17T06:12:59.364 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.364 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.364 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.364 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:12:59.364 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.364 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.364 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.364 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "SMTP server", 2024-04-17T06:12:59.364 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.364 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.364 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.364 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.364 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_password": { 2024-04-17T06:12:59.365 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_password", 2024-04-17T06:12:59.365 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.365 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.365 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.365 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:12:59.365 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.365 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.365 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.365 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Password to authenticate with", 2024-04-17T06:12:59.365 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.365 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.365 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.365 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.366 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_port": { 2024-04-17T06:12:59.366 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_port", 2024-04-17T06:12:59.366 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:12:59.366 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.366 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.366 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "465", 2024-04-17T06:12:59.366 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.366 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.366 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.366 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "SMTP port", 2024-04-17T06:12:59.366 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.366 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.366 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.367 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.367 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_sender": { 2024-04-17T06:12:59.367 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_sender", 2024-04-17T06:12:59.367 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.367 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.367 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.367 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:12:59.367 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.367 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.367 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.367 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "SMTP envelope sender", 2024-04-17T06:12:59.367 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.367 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.368 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.368 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.368 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_ssl": { 2024-04-17T06:12:59.368 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_ssl", 2024-04-17T06:12:59.368 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.368 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.368 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.368 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:12:59.368 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.368 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.368 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.368 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-17T06:12:59.368 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.369 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.369 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.369 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.369 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_user": { 2024-04-17T06:12:59.369 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_user", 2024-04-17T06:12:59.369 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.369 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.369 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.369 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:12:59.369 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.369 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.369 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.370 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "User to authenticate as", 2024-04-17T06:12:59.370 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.370 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.370 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.370 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:12:59.370 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:12:59.370 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.370 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:12:59.370 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "balancer", 2024-04-17T06:12:59.370 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-17T06:12:59.370 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-17T06:12:59.371 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-17T06:12:59.371 INFO:teuthology.orchestra.run.smithi047.stdout: "active": { 2024-04-17T06:12:59.371 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "active", 2024-04-17T06:12:59.371 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.371 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.371 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.371 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:12:59.371 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.371 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.371 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.371 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "automatically balance PGs across cluster", 2024-04-17T06:12:59.371 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.371 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.372 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.372 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.372 INFO:teuthology.orchestra.run.smithi047.stdout: "begin_time": { 2024-04-17T06:12:59.372 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "begin_time", 2024-04-17T06:12:59.372 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.372 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.372 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.372 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0000", 2024-04-17T06:12:59.372 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.372 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.372 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.373 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "beginning time of day to automatically balance", 2024-04-17T06:12:59.373 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-17T06:12:59.373 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.373 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.373 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.373 INFO:teuthology.orchestra.run.smithi047.stdout: "begin_weekday": { 2024-04-17T06:12:59.373 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "begin_weekday", 2024-04-17T06:12:59.373 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "uint", 2024-04-17T06:12:59.373 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.373 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.373 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-04-17T06:12:59.373 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "0", 2024-04-17T06:12:59.374 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "6", 2024-04-17T06:12:59.374 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.374 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-17T06:12:59.374 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-17T06:12:59.374 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.374 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.374 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.374 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_compat_max_iterations": { 2024-04-17T06:12:59.374 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "crush_compat_max_iterations", 2024-04-17T06:12:59.374 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "uint", 2024-04-17T06:12:59.374 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.374 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.375 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "25", 2024-04-17T06:12:59.375 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "1", 2024-04-17T06:12:59.375 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "250", 2024-04-17T06:12:59.375 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.375 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-17T06:12:59.375 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.375 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.375 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.375 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.375 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_compat_metrics": { 2024-04-17T06:12:59.375 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "crush_compat_metrics", 2024-04-17T06:12:59.375 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.375 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.376 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.376 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "pgs,objects,bytes", 2024-04-17T06:12:59.376 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.376 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.376 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.376 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-17T06:12:59.376 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-17T06:12:59.376 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.376 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.376 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.376 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_compat_step": { 2024-04-17T06:12:59.376 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "crush_compat_step", 2024-04-17T06:12:59.376 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "float", 2024-04-17T06:12:59.377 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.377 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.377 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0.5", 2024-04-17T06:12:59.377 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "0.001", 2024-04-17T06:12:59.377 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "0.999", 2024-04-17T06:12:59.377 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.377 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "aggressiveness of optimization", 2024-04-17T06:12:59.377 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-17T06:12:59.377 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.377 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.377 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.377 INFO:teuthology.orchestra.run.smithi047.stdout: "end_time": { 2024-04-17T06:12:59.377 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "end_time", 2024-04-17T06:12:59.378 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.378 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.378 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.378 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "2359", 2024-04-17T06:12:59.378 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.378 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.378 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.378 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "ending time of day to automatically balance", 2024-04-17T06:12:59.378 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-17T06:12:59.378 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.378 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.378 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.379 INFO:teuthology.orchestra.run.smithi047.stdout: "end_weekday": { 2024-04-17T06:12:59.379 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "end_weekday", 2024-04-17T06:12:59.379 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "uint", 2024-04-17T06:12:59.379 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.379 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.379 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-04-17T06:12:59.379 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "0", 2024-04-17T06:12:59.379 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "6", 2024-04-17T06:12:59.379 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.379 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-17T06:12:59.379 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-17T06:12:59.379 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.379 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.380 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.380 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-17T06:12:59.380 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-17T06:12:59.380 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.380 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.380 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.380 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:12:59.380 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.380 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.380 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:12:59.380 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:12:59.380 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:12:59.380 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:12:59.380 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:12:59.381 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:12:59.381 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:12:59.381 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:12:59.381 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.381 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.381 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.381 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.381 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.381 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-17T06:12:59.381 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-17T06:12:59.381 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.381 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.381 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.381 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.382 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.382 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.382 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.382 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.382 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.382 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.382 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.382 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.382 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-17T06:12:59.382 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-17T06:12:59.382 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.383 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.383 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.383 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-17T06:12:59.383 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.383 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.383 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:12:59.383 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:12:59.383 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:12:59.383 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:12:59.383 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:12:59.383 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:12:59.383 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:12:59.383 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:12:59.383 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.384 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.384 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.384 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.384 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.384 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-17T06:12:59.384 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-17T06:12:59.384 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.384 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.384 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.384 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.384 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.384 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.385 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.385 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.385 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.385 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.385 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.385 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.385 INFO:teuthology.orchestra.run.smithi047.stdout: "min_score": { 2024-04-17T06:12:59.385 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "min_score", 2024-04-17T06:12:59.385 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "float", 2024-04-17T06:12:59.385 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.385 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.385 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-04-17T06:12:59.385 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.385 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.386 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.386 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-17T06:12:59.386 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.386 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.386 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.386 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.386 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": { 2024-04-17T06:12:59.386 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "mode", 2024-04-17T06:12:59.386 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.386 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.386 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.386 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "upmap", 2024-04-17T06:12:59.386 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.387 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.387 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:12:59.387 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-compat", 2024-04-17T06:12:59.387 INFO:teuthology.orchestra.run.smithi047.stdout: "none", 2024-04-17T06:12:59.387 INFO:teuthology.orchestra.run.smithi047.stdout: "read", 2024-04-17T06:12:59.387 INFO:teuthology.orchestra.run.smithi047.stdout: "upmap", 2024-04-17T06:12:59.387 INFO:teuthology.orchestra.run.smithi047.stdout: "upmap-read" 2024-04-17T06:12:59.387 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:12:59.387 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Balancer mode", 2024-04-17T06:12:59.387 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.387 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.387 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.388 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.388 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_ids": { 2024-04-17T06:12:59.388 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "pool_ids", 2024-04-17T06:12:59.388 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.388 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.388 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.388 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:12:59.388 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.388 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.388 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.388 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-17T06:12:59.388 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.389 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.389 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.389 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.389 INFO:teuthology.orchestra.run.smithi047.stdout: "sleep_interval": { 2024-04-17T06:12:59.389 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sleep_interval", 2024-04-17T06:12:59.389 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-17T06:12:59.389 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.389 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.389 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "60", 2024-04-17T06:12:59.389 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.389 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.389 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.389 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-17T06:12:59.389 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.390 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.390 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.390 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.390 INFO:teuthology.orchestra.run.smithi047.stdout: "upmap_max_deviation": { 2024-04-17T06:12:59.390 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "upmap_max_deviation", 2024-04-17T06:12:59.390 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:12:59.390 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.390 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.390 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "5", 2024-04-17T06:12:59.390 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "1", 2024-04-17T06:12:59.390 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.390 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.391 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-17T06:12:59.391 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-17T06:12:59.391 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.391 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.391 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.391 INFO:teuthology.orchestra.run.smithi047.stdout: "upmap_max_optimizations": { 2024-04-17T06:12:59.391 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "upmap_max_optimizations", 2024-04-17T06:12:59.391 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "uint", 2024-04-17T06:12:59.391 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.391 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.391 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-04-17T06:12:59.391 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.391 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.392 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.392 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-17T06:12:59.392 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.392 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.392 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.392 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:12:59.392 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:12:59.392 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.392 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:12:59.392 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "cephadm", 2024-04-17T06:12:59.392 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-17T06:12:59.392 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-17T06:12:59.392 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-17T06:12:59.393 INFO:teuthology.orchestra.run.smithi047.stdout: "agent_down_multiplier": { 2024-04-17T06:12:59.393 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "agent_down_multiplier", 2024-04-17T06:12:59.393 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "float", 2024-04-17T06:12:59.393 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.393 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.393 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "3.0", 2024-04-17T06:12:59.393 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.393 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.393 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.393 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-17T06:12:59.393 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.393 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.393 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.393 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.394 INFO:teuthology.orchestra.run.smithi047.stdout: "agent_refresh_rate": { 2024-04-17T06:12:59.394 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "agent_refresh_rate", 2024-04-17T06:12:59.394 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-17T06:12:59.394 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.394 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.394 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "20", 2024-04-17T06:12:59.394 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.394 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.394 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.394 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-17T06:12:59.394 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.394 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.395 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.395 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.395 INFO:teuthology.orchestra.run.smithi047.stdout: "agent_starting_port": { 2024-04-17T06:12:59.395 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "agent_starting_port", 2024-04-17T06:12:59.395 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:12:59.395 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.395 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.395 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "4721", 2024-04-17T06:12:59.395 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.395 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.395 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.395 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-17T06:12:59.395 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.396 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.396 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.396 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.396 INFO:teuthology.orchestra.run.smithi047.stdout: "allow_ptrace": { 2024-04-17T06:12:59.396 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "allow_ptrace", 2024-04-17T06:12:59.396 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.396 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.396 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.396 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.396 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.396 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.396 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.396 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-17T06:12:59.396 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-17T06:12:59.397 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.397 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.397 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.397 INFO:teuthology.orchestra.run.smithi047.stdout: "autotune_interval": { 2024-04-17T06:12:59.397 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "autotune_interval", 2024-04-17T06:12:59.397 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-17T06:12:59.397 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.397 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.397 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "600", 2024-04-17T06:12:59.397 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.397 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.397 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.397 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-17T06:12:59.397 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.398 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.398 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.398 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.398 INFO:teuthology.orchestra.run.smithi047.stdout: "autotune_memory_target_ratio": { 2024-04-17T06:12:59.398 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "autotune_memory_target_ratio", 2024-04-17T06:12:59.398 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "float", 2024-04-17T06:12:59.398 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.398 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.398 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0.7", 2024-04-17T06:12:59.398 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.398 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.398 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.398 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-17T06:12:59.399 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.399 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.399 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.399 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.399 INFO:teuthology.orchestra.run.smithi047.stdout: "cephadm_log_destination": { 2024-04-17T06:12:59.399 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "cephadm_log_destination", 2024-04-17T06:12:59.399 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.399 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.399 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.399 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:12:59.399 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.399 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.399 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:12:59.399 INFO:teuthology.orchestra.run.smithi047.stdout: "file", 2024-04-17T06:12:59.400 INFO:teuthology.orchestra.run.smithi047.stdout: "file,syslog", 2024-04-17T06:12:59.400 INFO:teuthology.orchestra.run.smithi047.stdout: "syslog" 2024-04-17T06:12:59.400 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:12:59.400 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-17T06:12:59.400 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.400 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.400 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.400 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.400 INFO:teuthology.orchestra.run.smithi047.stdout: "cgroups_split": { 2024-04-17T06:12:59.400 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "cgroups_split", 2024-04-17T06:12:59.400 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.400 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.400 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.401 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:12:59.401 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.401 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.401 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.401 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-17T06:12:59.401 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.401 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.401 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.401 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.401 INFO:teuthology.orchestra.run.smithi047.stdout: "config_checks_enabled": { 2024-04-17T06:12:59.401 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "config_checks_enabled", 2024-04-17T06:12:59.401 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.401 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.402 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.402 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.402 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.402 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.402 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.402 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-17T06:12:59.402 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.402 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.402 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.402 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.402 INFO:teuthology.orchestra.run.smithi047.stdout: "config_dashboard": { 2024-04-17T06:12:59.403 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "config_dashboard", 2024-04-17T06:12:59.403 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.403 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.403 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.403 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:12:59.403 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.403 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.403 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.403 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-17T06:12:59.403 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.403 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.403 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.403 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.404 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_alertmanager": { 2024-04-17T06:12:59.404 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_alertmanager", 2024-04-17T06:12:59.404 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.404 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.404 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.404 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-17T06:12:59.404 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.404 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.404 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.404 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Prometheus container image", 2024-04-17T06:12:59.404 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.404 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.405 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.405 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.405 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_base": { 2024-04-17T06:12:59.405 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_base", 2024-04-17T06:12:59.405 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.405 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.405 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.405 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-17T06:12:59.405 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.405 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.405 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.405 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Container image name, without the tag", 2024-04-17T06:12:59.406 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.406 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.406 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.406 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.406 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_elasticsearch": { 2024-04-17T06:12:59.406 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_elasticsearch", 2024-04-17T06:12:59.406 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.406 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.406 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.406 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-17T06:12:59.406 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.406 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.407 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.407 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "elasticsearch container image", 2024-04-17T06:12:59.407 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.407 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.407 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.407 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.407 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_grafana": { 2024-04-17T06:12:59.407 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_grafana", 2024-04-17T06:12:59.407 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.407 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.407 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.407 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-17T06:12:59.407 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.408 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.408 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.408 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Prometheus container image", 2024-04-17T06:12:59.408 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.408 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.408 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.408 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.408 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_haproxy": { 2024-04-17T06:12:59.408 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_haproxy", 2024-04-17T06:12:59.408 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.408 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.408 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.409 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-17T06:12:59.409 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.409 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.409 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.409 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "HAproxy container image", 2024-04-17T06:12:59.409 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.409 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.409 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.409 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.409 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_jaeger_agent": { 2024-04-17T06:12:59.409 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_jaeger_agent", 2024-04-17T06:12:59.409 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.410 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.410 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.410 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-17T06:12:59.410 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.410 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.410 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.410 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Jaeger agent container image", 2024-04-17T06:12:59.410 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.410 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.410 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.410 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.410 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_jaeger_collector": { 2024-04-17T06:12:59.411 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_jaeger_collector", 2024-04-17T06:12:59.411 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.411 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.411 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.411 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-17T06:12:59.411 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.411 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.411 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.411 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Jaeger collector container image", 2024-04-17T06:12:59.411 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.411 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.411 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.411 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.412 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_jaeger_query": { 2024-04-17T06:12:59.412 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_jaeger_query", 2024-04-17T06:12:59.412 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.412 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.412 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.412 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-17T06:12:59.412 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.412 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.412 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.412 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Jaeger query container image", 2024-04-17T06:12:59.412 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.413 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.413 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.413 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.413 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_keepalived": { 2024-04-17T06:12:59.413 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_keepalived", 2024-04-17T06:12:59.413 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.413 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.413 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.413 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-17T06:12:59.413 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.413 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.413 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.413 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Keepalived container image", 2024-04-17T06:12:59.414 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.414 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.414 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.414 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.414 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_loki": { 2024-04-17T06:12:59.414 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_loki", 2024-04-17T06:12:59.414 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.414 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.414 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.414 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-17T06:12:59.414 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.414 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.415 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.415 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Loki container image", 2024-04-17T06:12:59.415 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.415 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.415 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.415 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.415 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_node_exporter": { 2024-04-17T06:12:59.415 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_node_exporter", 2024-04-17T06:12:59.415 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.415 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.415 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.415 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-17T06:12:59.415 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.415 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.416 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.416 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Prometheus container image", 2024-04-17T06:12:59.416 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.416 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.416 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.416 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.416 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_nvmeof": { 2024-04-17T06:12:59.416 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_nvmeof", 2024-04-17T06:12:59.416 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.416 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.416 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.416 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-17T06:12:59.417 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.417 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.417 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.417 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Nvme-of container image", 2024-04-17T06:12:59.417 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.417 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.417 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.417 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.417 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_prometheus": { 2024-04-17T06:12:59.417 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_prometheus", 2024-04-17T06:12:59.417 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.417 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.417 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.418 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-17T06:12:59.418 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.418 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.418 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.418 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Prometheus container image", 2024-04-17T06:12:59.418 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.418 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.418 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.418 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.418 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_promtail": { 2024-04-17T06:12:59.418 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_promtail", 2024-04-17T06:12:59.418 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.418 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.418 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.419 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-17T06:12:59.419 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.419 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.419 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.419 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Promtail container image", 2024-04-17T06:12:59.419 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.419 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.419 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.419 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.419 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_samba": { 2024-04-17T06:12:59.419 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_samba", 2024-04-17T06:12:59.419 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.419 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.419 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.420 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-17T06:12:59.420 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.420 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.420 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.420 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Samba/SMB container image", 2024-04-17T06:12:59.420 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.420 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.420 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.420 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.420 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_snmp_gateway": { 2024-04-17T06:12:59.420 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_snmp_gateway", 2024-04-17T06:12:59.420 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.420 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.421 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.421 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-17T06:12:59.421 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.421 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.421 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.421 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "SNMP Gateway container image", 2024-04-17T06:12:59.421 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.421 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.421 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.421 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.421 INFO:teuthology.orchestra.run.smithi047.stdout: "container_init": { 2024-04-17T06:12:59.421 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_init", 2024-04-17T06:12:59.421 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.421 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.422 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.422 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:12:59.422 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.422 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.422 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.422 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Run podman/docker with `--init`", 2024-04-17T06:12:59.422 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.422 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.422 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.422 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.422 INFO:teuthology.orchestra.run.smithi047.stdout: "daemon_cache_timeout": { 2024-04-17T06:12:59.422 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "daemon_cache_timeout", 2024-04-17T06:12:59.423 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-17T06:12:59.423 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.423 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.423 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "600", 2024-04-17T06:12:59.423 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.423 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.423 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.423 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-17T06:12:59.423 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.423 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.423 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.423 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.423 INFO:teuthology.orchestra.run.smithi047.stdout: "default_cephadm_command_timeout": { 2024-04-17T06:12:59.424 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "default_cephadm_command_timeout", 2024-04-17T06:12:59.424 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:12:59.424 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.424 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.424 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "900", 2024-04-17T06:12:59.424 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.424 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.424 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.424 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-17T06:12:59.424 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.424 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.424 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.425 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.425 INFO:teuthology.orchestra.run.smithi047.stdout: "default_registry": { 2024-04-17T06:12:59.425 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "default_registry", 2024-04-17T06:12:59.425 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.425 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.425 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.425 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "docker.io", 2024-04-17T06:12:59.425 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.425 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.425 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.425 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-17T06:12:59.425 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.426 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.426 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.426 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.426 INFO:teuthology.orchestra.run.smithi047.stdout: "device_cache_timeout": { 2024-04-17T06:12:59.426 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "device_cache_timeout", 2024-04-17T06:12:59.426 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-17T06:12:59.426 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.426 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.426 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "1800", 2024-04-17T06:12:59.426 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.426 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.426 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.427 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "seconds to cache device inventory", 2024-04-17T06:12:59.427 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.427 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.427 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.427 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.427 INFO:teuthology.orchestra.run.smithi047.stdout: "device_enhanced_scan": { 2024-04-17T06:12:59.427 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "device_enhanced_scan", 2024-04-17T06:12:59.427 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.427 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.427 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.427 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.427 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.427 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.427 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.428 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-17T06:12:59.428 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.428 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.428 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.428 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.428 INFO:teuthology.orchestra.run.smithi047.stdout: "facts_cache_timeout": { 2024-04-17T06:12:59.428 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "facts_cache_timeout", 2024-04-17T06:12:59.428 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-17T06:12:59.428 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.428 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.428 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "60", 2024-04-17T06:12:59.428 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.428 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.429 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.429 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "seconds to cache host facts data", 2024-04-17T06:12:59.429 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.429 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.429 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.429 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.429 INFO:teuthology.orchestra.run.smithi047.stdout: "grafana_dashboards_path": { 2024-04-17T06:12:59.429 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "grafana_dashboards_path", 2024-04-17T06:12:59.429 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.429 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.429 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.429 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-17T06:12:59.430 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.430 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.430 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.430 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-17T06:12:59.430 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.430 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.430 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.430 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.430 INFO:teuthology.orchestra.run.smithi047.stdout: "host_check_interval": { 2024-04-17T06:12:59.430 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "host_check_interval", 2024-04-17T06:12:59.430 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-17T06:12:59.430 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.430 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.431 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "600", 2024-04-17T06:12:59.431 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.431 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.431 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.431 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how frequently to perform a host check", 2024-04-17T06:12:59.431 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.431 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.431 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.431 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.431 INFO:teuthology.orchestra.run.smithi047.stdout: "hw_monitoring": { 2024-04-17T06:12:59.431 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "hw_monitoring", 2024-04-17T06:12:59.431 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.431 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.432 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.432 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.432 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.432 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.432 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.432 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-17T06:12:59.432 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.432 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.432 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.432 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.432 INFO:teuthology.orchestra.run.smithi047.stdout: "inventory_list_all": { 2024-04-17T06:12:59.432 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "inventory_list_all", 2024-04-17T06:12:59.432 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.433 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.433 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.433 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.433 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.433 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.433 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.433 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-17T06:12:59.433 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.433 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.433 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.433 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.433 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-17T06:12:59.433 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-17T06:12:59.434 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.434 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.434 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.434 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:12:59.434 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.434 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.434 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:12:59.434 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:12:59.434 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:12:59.434 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:12:59.434 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:12:59.434 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:12:59.435 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:12:59.435 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:12:59.435 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.435 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.435 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.435 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.435 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.435 INFO:teuthology.orchestra.run.smithi047.stdout: "log_refresh_metadata": { 2024-04-17T06:12:59.435 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_refresh_metadata", 2024-04-17T06:12:59.435 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.435 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.435 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.436 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.436 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.436 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.436 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.436 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-17T06:12:59.436 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.436 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.436 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.436 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.436 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-17T06:12:59.436 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-17T06:12:59.437 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.437 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.437 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.437 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:12:59.437 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.437 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.437 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.437 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-17T06:12:59.437 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.437 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.437 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.437 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.438 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-17T06:12:59.438 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-17T06:12:59.438 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.438 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.438 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.438 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-17T06:12:59.438 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.438 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.438 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:12:59.438 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:12:59.438 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:12:59.438 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:12:59.438 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:12:59.439 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:12:59.439 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:12:59.439 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:12:59.439 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.439 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.439 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.439 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.439 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.439 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-17T06:12:59.439 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-17T06:12:59.439 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.439 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.439 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.440 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.440 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.440 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.440 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.440 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.440 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.440 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.440 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.440 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.440 INFO:teuthology.orchestra.run.smithi047.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-17T06:12:59.440 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-17T06:12:59.440 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.440 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.441 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.441 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.441 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.441 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.441 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.441 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-17T06:12:59.441 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.441 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.441 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.441 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.441 INFO:teuthology.orchestra.run.smithi047.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-17T06:12:59.441 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-17T06:12:59.441 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.441 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.442 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.442 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "*", 2024-04-17T06:12:59.442 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.442 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.442 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.442 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-17T06:12:59.442 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.442 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.442 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.442 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.442 INFO:teuthology.orchestra.run.smithi047.stdout: "max_count_per_host": { 2024-04-17T06:12:59.442 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "max_count_per_host", 2024-04-17T06:12:59.442 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:12:59.443 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.443 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.443 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-04-17T06:12:59.443 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.443 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.443 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.443 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "max number of daemons per service per host", 2024-04-17T06:12:59.443 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.443 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.443 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.443 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.444 INFO:teuthology.orchestra.run.smithi047.stdout: "max_osd_draining_count": { 2024-04-17T06:12:59.444 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "max_osd_draining_count", 2024-04-17T06:12:59.444 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:12:59.444 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.444 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.444 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-04-17T06:12:59.444 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.444 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.444 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.444 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-17T06:12:59.444 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.444 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.444 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.445 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.445 INFO:teuthology.orchestra.run.smithi047.stdout: "migration_current": { 2024-04-17T06:12:59.445 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "migration_current", 2024-04-17T06:12:59.445 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:12:59.445 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.445 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.445 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:12:59.445 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.445 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.445 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.445 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "internal - do not modify", 2024-04-17T06:12:59.446 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.446 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.446 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.446 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.446 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": { 2024-04-17T06:12:59.446 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "mode", 2024-04-17T06:12:59.446 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.446 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.446 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.446 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "root", 2024-04-17T06:12:59.446 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.446 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.447 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:12:59.447 INFO:teuthology.orchestra.run.smithi047.stdout: "cephadm-package", 2024-04-17T06:12:59.447 INFO:teuthology.orchestra.run.smithi047.stdout: "root" 2024-04-17T06:12:59.447 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:12:59.447 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "mode for remote execution of cephadm", 2024-04-17T06:12:59.447 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.447 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.447 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.447 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.447 INFO:teuthology.orchestra.run.smithi047.stdout: "oob_default_addr": { 2024-04-17T06:12:59.447 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "oob_default_addr", 2024-04-17T06:12:59.447 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.447 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.448 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.448 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "169.254.1.1", 2024-04-17T06:12:59.448 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.448 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.448 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.448 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-17T06:12:59.448 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.448 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.448 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.448 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.448 INFO:teuthology.orchestra.run.smithi047.stdout: "prometheus_alerts_path": { 2024-04-17T06:12:59.448 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "prometheus_alerts_path", 2024-04-17T06:12:59.448 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.449 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.449 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.449 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-17T06:12:59.449 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.449 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.449 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.449 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-17T06:12:59.449 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.449 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.449 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.449 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.449 INFO:teuthology.orchestra.run.smithi047.stdout: "registry_insecure": { 2024-04-17T06:12:59.449 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "registry_insecure", 2024-04-17T06:12:59.449 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.450 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.450 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.450 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.450 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.450 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.450 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.450 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-17T06:12:59.450 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.450 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.450 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.450 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.450 INFO:teuthology.orchestra.run.smithi047.stdout: "registry_password": { 2024-04-17T06:12:59.450 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "registry_password", 2024-04-17T06:12:59.451 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.451 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.451 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.451 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:12:59.451 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.451 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.451 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.451 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-17T06:12:59.451 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.451 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.451 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.451 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.451 INFO:teuthology.orchestra.run.smithi047.stdout: "registry_url": { 2024-04-17T06:12:59.451 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "registry_url", 2024-04-17T06:12:59.452 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.452 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.452 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.452 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:12:59.452 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.452 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.452 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.452 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-17T06:12:59.452 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.452 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.452 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.452 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.453 INFO:teuthology.orchestra.run.smithi047.stdout: "registry_username": { 2024-04-17T06:12:59.453 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "registry_username", 2024-04-17T06:12:59.453 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.453 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.453 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.453 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:12:59.453 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.453 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.453 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.453 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-17T06:12:59.453 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.454 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.454 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.454 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.454 INFO:teuthology.orchestra.run.smithi047.stdout: "secure_monitoring_stack": { 2024-04-17T06:12:59.454 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "secure_monitoring_stack", 2024-04-17T06:12:59.454 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.454 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.454 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.454 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.454 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.454 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.454 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.454 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-17T06:12:59.455 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.455 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.455 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.455 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.455 INFO:teuthology.orchestra.run.smithi047.stdout: "service_discovery_port": { 2024-04-17T06:12:59.455 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "service_discovery_port", 2024-04-17T06:12:59.455 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:12:59.455 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.455 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.455 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "8765", 2024-04-17T06:12:59.455 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.455 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.455 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.455 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "cephadm service discovery port", 2024-04-17T06:12:59.456 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.456 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.456 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.456 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.456 INFO:teuthology.orchestra.run.smithi047.stdout: "ssh_config_file": { 2024-04-17T06:12:59.456 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ssh_config_file", 2024-04-17T06:12:59.456 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.456 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.456 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.456 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:12:59.456 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.456 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.456 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.456 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-17T06:12:59.457 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.457 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.457 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.457 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.457 INFO:teuthology.orchestra.run.smithi047.stdout: "use_agent": { 2024-04-17T06:12:59.457 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "use_agent", 2024-04-17T06:12:59.457 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.457 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.457 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.457 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.457 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.457 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.457 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.458 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-17T06:12:59.458 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.458 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.458 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.458 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.458 INFO:teuthology.orchestra.run.smithi047.stdout: "use_repo_digest": { 2024-04-17T06:12:59.458 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "use_repo_digest", 2024-04-17T06:12:59.458 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.458 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.458 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.458 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:12:59.458 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.458 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.459 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.459 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-17T06:12:59.459 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.459 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.459 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.459 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.459 INFO:teuthology.orchestra.run.smithi047.stdout: "warn_on_failed_host_check": { 2024-04-17T06:12:59.459 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "warn_on_failed_host_check", 2024-04-17T06:12:59.459 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.459 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.459 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.459 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:12:59.459 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.459 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.460 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.460 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "raise a health warning if the host check fails", 2024-04-17T06:12:59.460 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.460 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.460 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.460 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.460 INFO:teuthology.orchestra.run.smithi047.stdout: "warn_on_stray_daemons": { 2024-04-17T06:12:59.460 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "warn_on_stray_daemons", 2024-04-17T06:12:59.460 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.460 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.460 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.460 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:12:59.460 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.460 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.460 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.461 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-17T06:12:59.461 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.461 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.461 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.461 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.461 INFO:teuthology.orchestra.run.smithi047.stdout: "warn_on_stray_hosts": { 2024-04-17T06:12:59.461 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "warn_on_stray_hosts", 2024-04-17T06:12:59.461 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.461 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.461 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.461 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:12:59.461 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.462 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.462 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.462 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-17T06:12:59.462 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.462 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.462 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.462 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:12:59.462 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:12:59.462 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.462 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:12:59.462 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "crash", 2024-04-17T06:12:59.462 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-17T06:12:59.462 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-17T06:12:59.463 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-17T06:12:59.463 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-17T06:12:59.463 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-17T06:12:59.463 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.463 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.463 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.463 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:12:59.463 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.463 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.463 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:12:59.463 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:12:59.463 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:12:59.463 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:12:59.464 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:12:59.464 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:12:59.464 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:12:59.464 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:12:59.464 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.464 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.464 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.464 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.464 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.464 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-17T06:12:59.464 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-17T06:12:59.464 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.464 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.464 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.465 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.465 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.465 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.465 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.465 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.465 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.465 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.465 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.465 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.465 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-17T06:12:59.465 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-17T06:12:59.465 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.465 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.466 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.466 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-17T06:12:59.466 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.466 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.466 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:12:59.466 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:12:59.466 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:12:59.466 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:12:59.466 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:12:59.466 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:12:59.466 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:12:59.466 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:12:59.466 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.467 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.467 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.467 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.467 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.467 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-17T06:12:59.467 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-17T06:12:59.467 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.467 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.467 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.467 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.467 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.467 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.468 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.468 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.468 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.468 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.468 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.468 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.468 INFO:teuthology.orchestra.run.smithi047.stdout: "retain_interval": { 2024-04-17T06:12:59.468 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "retain_interval", 2024-04-17T06:12:59.468 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-17T06:12:59.468 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.468 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.468 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "31536000", 2024-04-17T06:12:59.468 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.468 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.469 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.469 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-17T06:12:59.469 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.469 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.469 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.469 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.469 INFO:teuthology.orchestra.run.smithi047.stdout: "warn_recent_interval": { 2024-04-17T06:12:59.469 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "warn_recent_interval", 2024-04-17T06:12:59.469 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-17T06:12:59.469 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.469 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.469 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "1209600", 2024-04-17T06:12:59.469 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.469 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.470 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.470 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-17T06:12:59.470 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.470 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.470 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.470 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:12:59.470 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:12:59.470 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.470 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:12:59.470 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "dashboard", 2024-04-17T06:12:59.470 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-17T06:12:59.470 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-17T06:12:59.470 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-17T06:12:59.470 INFO:teuthology.orchestra.run.smithi047.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-17T06:12:59.471 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-17T06:12:59.471 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:12:59.471 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.471 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.471 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-04-17T06:12:59.471 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.471 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.471 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.471 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.471 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.471 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.471 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.471 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.471 INFO:teuthology.orchestra.run.smithi047.stdout: "ALERTMANAGER_API_HOST": { 2024-04-17T06:12:59.472 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-17T06:12:59.472 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.472 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.472 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.472 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:12:59.472 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.472 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.472 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.472 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.472 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.472 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.472 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.472 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.473 INFO:teuthology.orchestra.run.smithi047.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-17T06:12:59.473 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-17T06:12:59.473 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.473 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.473 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.473 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:12:59.473 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.473 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.473 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.473 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.473 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.473 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.473 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.474 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.474 INFO:teuthology.orchestra.run.smithi047.stdout: "AUDIT_API_ENABLED": { 2024-04-17T06:12:59.474 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "AUDIT_API_ENABLED", 2024-04-17T06:12:59.474 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.474 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.474 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.474 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.474 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.474 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.474 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.474 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.474 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.474 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.474 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.475 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.475 INFO:teuthology.orchestra.run.smithi047.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-17T06:12:59.475 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-17T06:12:59.475 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.475 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.475 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.475 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:12:59.475 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.475 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.475 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.475 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.475 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.475 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.476 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.476 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.476 INFO:teuthology.orchestra.run.smithi047.stdout: "ENABLE_BROWSABLE_API": { 2024-04-17T06:12:59.476 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-17T06:12:59.476 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.476 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.476 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.476 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:12:59.476 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.476 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.476 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.476 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.476 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.477 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.477 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.477 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.477 INFO:teuthology.orchestra.run.smithi047.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-17T06:12:59.477 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-17T06:12:59.477 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.477 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.477 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.477 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:12:59.477 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.477 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.477 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.477 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.478 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.478 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.478 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.478 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.478 INFO:teuthology.orchestra.run.smithi047.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-17T06:12:59.478 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-17T06:12:59.478 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.478 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.478 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.478 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:12:59.478 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.478 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.478 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.479 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.479 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.479 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.479 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.479 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.479 INFO:teuthology.orchestra.run.smithi047.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-17T06:12:59.479 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-17T06:12:59.479 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.479 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.479 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.479 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:12:59.479 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.480 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.480 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.480 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.480 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.480 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.480 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.480 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.480 INFO:teuthology.orchestra.run.smithi047.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-17T06:12:59.480 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-17T06:12:59.480 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.480 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.480 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.480 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:12:59.480 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.481 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.481 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.481 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.481 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.481 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.481 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.481 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.481 INFO:teuthology.orchestra.run.smithi047.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-17T06:12:59.481 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-17T06:12:59.481 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.481 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.481 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.481 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:12:59.481 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.482 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.482 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.482 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.482 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.482 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.482 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.482 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.482 INFO:teuthology.orchestra.run.smithi047.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-17T06:12:59.482 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-17T06:12:59.482 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.482 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.482 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.483 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:12:59.483 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.483 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.483 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.483 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.483 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.483 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.483 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.483 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.483 INFO:teuthology.orchestra.run.smithi047.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-17T06:12:59.483 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-17T06:12:59.483 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.483 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.484 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.484 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:12:59.484 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.484 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.484 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.484 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.484 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.484 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.484 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.484 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.484 INFO:teuthology.orchestra.run.smithi047.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-17T06:12:59.484 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-17T06:12:59.484 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.484 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.485 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.485 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:12:59.485 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.485 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.485 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.485 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.485 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.485 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.485 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.485 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.485 INFO:teuthology.orchestra.run.smithi047.stdout: "GRAFANA_API_PASSWORD": { 2024-04-17T06:12:59.485 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-17T06:12:59.485 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.485 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.486 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.486 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "admin", 2024-04-17T06:12:59.486 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.486 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.486 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.486 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.486 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.486 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.486 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.486 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.486 INFO:teuthology.orchestra.run.smithi047.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-17T06:12:59.486 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-17T06:12:59.487 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.487 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.487 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.487 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:12:59.487 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.487 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.487 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.487 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.487 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.487 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.487 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.487 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.487 INFO:teuthology.orchestra.run.smithi047.stdout: "GRAFANA_API_URL": { 2024-04-17T06:12:59.488 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "GRAFANA_API_URL", 2024-04-17T06:12:59.488 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.488 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.488 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.488 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:12:59.488 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.488 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.488 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.488 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.488 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.488 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.488 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.488 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.488 INFO:teuthology.orchestra.run.smithi047.stdout: "GRAFANA_API_USERNAME": { 2024-04-17T06:12:59.489 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-17T06:12:59.489 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.489 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.489 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.489 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "admin", 2024-04-17T06:12:59.489 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.489 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.489 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.489 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.489 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.489 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.489 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.489 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.489 INFO:teuthology.orchestra.run.smithi047.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-17T06:12:59.490 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-17T06:12:59.490 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.490 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.490 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.490 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:12:59.490 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.490 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.490 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.490 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.490 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.490 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.490 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.490 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.490 INFO:teuthology.orchestra.run.smithi047.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-17T06:12:59.491 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-17T06:12:59.491 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.491 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.491 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.491 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.491 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.491 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.491 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.491 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.491 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.491 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.491 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.491 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.492 INFO:teuthology.orchestra.run.smithi047.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-17T06:12:59.492 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-17T06:12:59.492 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.492 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.492 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.492 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:12:59.492 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.492 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.492 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.492 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.492 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.492 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.492 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.493 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.493 INFO:teuthology.orchestra.run.smithi047.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-17T06:12:59.493 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-17T06:12:59.493 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.493 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.493 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.493 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:12:59.493 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.493 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.493 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.493 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.493 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.493 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.494 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.494 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.494 INFO:teuthology.orchestra.run.smithi047.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-17T06:12:59.494 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-17T06:12:59.494 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.494 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.494 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.494 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "[]", 2024-04-17T06:12:59.494 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.494 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.494 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.494 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.494 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.494 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.495 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.495 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.495 INFO:teuthology.orchestra.run.smithi047.stdout: "MULTICLUSTER_CONFIG": { 2024-04-17T06:12:59.495 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-17T06:12:59.495 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.495 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.495 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.495 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "{}", 2024-04-17T06:12:59.495 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.495 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.495 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.495 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.495 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.496 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.496 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.496 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.496 INFO:teuthology.orchestra.run.smithi047.stdout: "PROMETHEUS_API_HOST": { 2024-04-17T06:12:59.496 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-17T06:12:59.496 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.496 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.496 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.496 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:12:59.496 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.496 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.496 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.496 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.496 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.497 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.497 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.497 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.497 INFO:teuthology.orchestra.run.smithi047.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-17T06:12:59.497 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-17T06:12:59.497 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.497 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.497 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.497 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:12:59.497 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.497 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.497 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.497 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.498 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.498 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.498 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.498 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.498 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-17T06:12:59.498 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-17T06:12:59.498 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.498 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.498 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.498 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.498 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.499 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.499 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.499 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.499 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.499 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.499 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.499 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.499 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-17T06:12:59.499 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-17T06:12:59.499 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.499 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.499 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.499 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.499 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.500 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.500 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.500 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.500 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.500 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.500 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.500 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.500 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-17T06:12:59.500 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-17T06:12:59.500 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.500 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.500 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.500 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:12:59.500 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.501 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.501 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.501 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.501 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.501 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.501 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.501 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.501 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-17T06:12:59.501 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-17T06:12:59.501 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.501 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.501 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.501 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:12:59.501 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.502 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.502 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.502 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.502 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.502 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.502 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.502 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.502 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-17T06:12:59.502 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-17T06:12:59.502 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.502 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.502 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.502 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.503 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.503 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.503 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.503 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.503 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.503 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.503 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.503 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.503 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-17T06:12:59.503 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-17T06:12:59.503 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.503 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.503 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.504 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.504 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.504 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.504 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.504 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.504 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.504 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.504 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.504 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.504 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-17T06:12:59.504 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-17T06:12:59.504 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.504 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.505 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.505 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.505 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.505 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.505 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.505 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.505 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.505 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.505 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.505 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.505 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_ENABLED": { 2024-04-17T06:12:59.505 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-17T06:12:59.505 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.506 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.506 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.506 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:12:59.506 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.506 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.506 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.506 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.506 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.506 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.506 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.506 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.506 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-17T06:12:59.506 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-17T06:12:59.507 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.507 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.507 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.507 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-17T06:12:59.507 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.507 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.507 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.507 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.507 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.507 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.507 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.507 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.507 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-17T06:12:59.508 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-17T06:12:59.508 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:12:59.508 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.508 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.508 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-04-17T06:12:59.508 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.508 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.508 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.508 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.508 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.508 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.508 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.509 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.509 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-17T06:12:59.509 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-17T06:12:59.509 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:12:59.509 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.509 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.509 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "8", 2024-04-17T06:12:59.509 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.509 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.509 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.509 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.509 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.509 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.509 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.510 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.510 INFO:teuthology.orchestra.run.smithi047.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-17T06:12:59.510 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-17T06:12:59.510 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:12:59.510 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.510 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.510 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "45", 2024-04-17T06:12:59.510 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.510 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.510 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.510 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.510 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.511 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.511 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.511 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.511 INFO:teuthology.orchestra.run.smithi047.stdout: "RGW_API_ACCESS_KEY": { 2024-04-17T06:12:59.511 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-17T06:12:59.511 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.511 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.511 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.511 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:12:59.511 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.511 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.511 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.511 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.512 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.512 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.512 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.512 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.512 INFO:teuthology.orchestra.run.smithi047.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-17T06:12:59.512 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-17T06:12:59.512 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.512 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.512 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.512 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "admin", 2024-04-17T06:12:59.512 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.512 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.512 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.512 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.513 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.513 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.513 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.513 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.513 INFO:teuthology.orchestra.run.smithi047.stdout: "RGW_API_SECRET_KEY": { 2024-04-17T06:12:59.513 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-17T06:12:59.513 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.513 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.513 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.513 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:12:59.513 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.513 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.513 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.513 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.514 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.514 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.514 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.514 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.514 INFO:teuthology.orchestra.run.smithi047.stdout: "RGW_API_SSL_VERIFY": { 2024-04-17T06:12:59.514 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-17T06:12:59.514 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.514 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.514 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.514 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:12:59.514 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.514 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.515 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.515 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.515 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.515 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.515 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.515 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.515 INFO:teuthology.orchestra.run.smithi047.stdout: "UNSAFE_TLS_v1_2": { 2024-04-17T06:12:59.515 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-17T06:12:59.515 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.515 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.515 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.515 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.515 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.515 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.515 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.516 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.516 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.516 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.516 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.516 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.516 INFO:teuthology.orchestra.run.smithi047.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-17T06:12:59.516 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-17T06:12:59.516 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:12:59.516 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.516 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.516 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-04-17T06:12:59.516 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.516 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.516 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.517 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.517 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.517 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.517 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.517 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.517 INFO:teuthology.orchestra.run.smithi047.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-17T06:12:59.517 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-17T06:12:59.517 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:12:59.517 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.517 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.517 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-04-17T06:12:59.517 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.518 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.518 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.518 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.518 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.518 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.518 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.518 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.518 INFO:teuthology.orchestra.run.smithi047.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-17T06:12:59.518 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-17T06:12:59.518 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:12:59.518 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.518 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.518 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "5", 2024-04-17T06:12:59.519 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.519 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.519 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.519 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.519 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.519 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.519 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.519 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.519 INFO:teuthology.orchestra.run.smithi047.stdout: "cross_origin_url": { 2024-04-17T06:12:59.519 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "cross_origin_url", 2024-04-17T06:12:59.519 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.519 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.519 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.519 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:12:59.520 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.520 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.520 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.520 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.520 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.520 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.520 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.520 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.520 INFO:teuthology.orchestra.run.smithi047.stdout: "crt_file": { 2024-04-17T06:12:59.520 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "crt_file", 2024-04-17T06:12:59.520 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.520 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.520 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.521 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:12:59.521 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.521 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.521 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.521 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.521 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.521 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.521 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.521 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.521 INFO:teuthology.orchestra.run.smithi047.stdout: "debug": { 2024-04-17T06:12:59.521 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "debug", 2024-04-17T06:12:59.521 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.521 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.521 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.521 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.522 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.522 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.522 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.522 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Enable/disable debug options", 2024-04-17T06:12:59.522 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.522 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.522 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.522 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.522 INFO:teuthology.orchestra.run.smithi047.stdout: "jwt_token_ttl": { 2024-04-17T06:12:59.522 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "jwt_token_ttl", 2024-04-17T06:12:59.522 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:12:59.522 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.522 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.523 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "28800", 2024-04-17T06:12:59.523 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.523 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.523 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.523 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.523 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.523 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.523 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.523 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.523 INFO:teuthology.orchestra.run.smithi047.stdout: "key_file": { 2024-04-17T06:12:59.523 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "key_file", 2024-04-17T06:12:59.523 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.523 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.524 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.524 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:12:59.524 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.524 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.524 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.524 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.524 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.524 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.524 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.524 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.524 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-17T06:12:59.524 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-17T06:12:59.524 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.524 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.524 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.525 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:12:59.525 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.525 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.525 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:12:59.525 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:12:59.525 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:12:59.525 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:12:59.525 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:12:59.525 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:12:59.525 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:12:59.525 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:12:59.526 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.526 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.526 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.526 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.526 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.526 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-17T06:12:59.526 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-17T06:12:59.526 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.526 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.526 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.526 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.526 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.526 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.527 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.527 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.527 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.527 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.527 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.527 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.527 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-17T06:12:59.527 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-17T06:12:59.527 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.527 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.527 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.527 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-17T06:12:59.527 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.527 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.528 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:12:59.528 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:12:59.528 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:12:59.528 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:12:59.528 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:12:59.528 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:12:59.528 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:12:59.528 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:12:59.528 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.528 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.528 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.528 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.528 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.529 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-17T06:12:59.529 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-17T06:12:59.529 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.529 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.529 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.529 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.529 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.529 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.529 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.529 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.529 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.529 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.529 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.529 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.530 INFO:teuthology.orchestra.run.smithi047.stdout: "motd": { 2024-04-17T06:12:59.530 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "motd", 2024-04-17T06:12:59.530 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.530 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.530 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.530 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:12:59.530 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.530 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.530 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.530 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "The message of the day", 2024-04-17T06:12:59.530 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.530 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.531 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.531 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.531 INFO:teuthology.orchestra.run.smithi047.stdout: "redirect_resolve_ip_addr": { 2024-04-17T06:12:59.531 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "redirect_resolve_ip_addr", 2024-04-17T06:12:59.531 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.533 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.533 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.533 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.533 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.533 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.534 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.534 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.534 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.534 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.534 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.534 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.534 INFO:teuthology.orchestra.run.smithi047.stdout: "server_addr": { 2024-04-17T06:12:59.534 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "server_addr", 2024-04-17T06:12:59.534 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.534 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.534 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.535 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "::", 2024-04-17T06:12:59.535 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.535 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.535 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.535 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.535 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.535 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.535 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.535 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.535 INFO:teuthology.orchestra.run.smithi047.stdout: "server_port": { 2024-04-17T06:12:59.535 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "server_port", 2024-04-17T06:12:59.535 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:12:59.536 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.536 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.536 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "8080", 2024-04-17T06:12:59.536 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.536 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.536 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.536 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.536 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.536 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.536 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.536 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.537 INFO:teuthology.orchestra.run.smithi047.stdout: "ssl": { 2024-04-17T06:12:59.537 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ssl", 2024-04-17T06:12:59.537 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.537 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.537 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.537 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:12:59.537 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.537 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.537 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.537 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.537 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.538 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.538 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.538 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.538 INFO:teuthology.orchestra.run.smithi047.stdout: "ssl_server_port": { 2024-04-17T06:12:59.538 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ssl_server_port", 2024-04-17T06:12:59.538 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:12:59.538 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.538 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.538 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "8443", 2024-04-17T06:12:59.538 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.538 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.538 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.539 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.539 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.539 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.539 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.539 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.539 INFO:teuthology.orchestra.run.smithi047.stdout: "standby_behaviour": { 2024-04-17T06:12:59.539 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "standby_behaviour", 2024-04-17T06:12:59.539 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.539 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.539 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.539 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "redirect", 2024-04-17T06:12:59.539 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.539 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.540 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:12:59.540 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:12:59.540 INFO:teuthology.orchestra.run.smithi047.stdout: "redirect" 2024-04-17T06:12:59.540 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:12:59.540 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.540 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.540 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.540 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.540 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.540 INFO:teuthology.orchestra.run.smithi047.stdout: "standby_error_status_code": { 2024-04-17T06:12:59.540 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "standby_error_status_code", 2024-04-17T06:12:59.540 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:12:59.540 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.540 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.541 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "500", 2024-04-17T06:12:59.541 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "400", 2024-04-17T06:12:59.541 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "599", 2024-04-17T06:12:59.541 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.541 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.541 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.541 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.541 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.541 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.541 INFO:teuthology.orchestra.run.smithi047.stdout: "url_prefix": { 2024-04-17T06:12:59.541 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "url_prefix", 2024-04-17T06:12:59.541 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.541 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.542 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.542 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:12:59.542 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.542 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.542 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.542 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.542 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.542 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.542 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.542 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:12:59.542 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:12:59.542 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.543 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:12:59.543 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "devicehealth", 2024-04-17T06:12:59.543 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-17T06:12:59.543 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-17T06:12:59.543 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-17T06:12:59.543 INFO:teuthology.orchestra.run.smithi047.stdout: "enable_monitoring": { 2024-04-17T06:12:59.543 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "enable_monitoring", 2024-04-17T06:12:59.543 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.543 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.543 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.543 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:12:59.543 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.543 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.544 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.544 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "monitor device health metrics", 2024-04-17T06:12:59.544 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.544 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.544 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.544 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.544 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-17T06:12:59.544 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-17T06:12:59.544 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.544 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.544 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.544 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:12:59.544 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.545 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.545 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:12:59.545 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:12:59.545 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:12:59.545 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:12:59.545 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:12:59.545 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:12:59.545 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:12:59.545 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:12:59.545 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.545 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.545 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.545 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.545 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.546 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-17T06:12:59.546 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-17T06:12:59.546 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.546 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.546 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.546 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.546 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.546 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.546 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.546 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.546 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.546 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.546 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.547 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.547 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-17T06:12:59.547 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-17T06:12:59.547 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.547 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.547 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.547 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-17T06:12:59.547 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.547 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.547 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:12:59.547 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:12:59.547 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:12:59.548 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:12:59.548 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:12:59.548 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:12:59.548 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:12:59.548 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:12:59.548 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.548 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.548 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.548 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.548 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.548 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-17T06:12:59.548 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-17T06:12:59.549 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.549 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.549 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.549 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.549 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.549 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.549 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.549 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.549 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.549 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.549 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.549 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.550 INFO:teuthology.orchestra.run.smithi047.stdout: "mark_out_threshold": { 2024-04-17T06:12:59.550 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "mark_out_threshold", 2024-04-17T06:12:59.550 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-17T06:12:59.550 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.550 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.550 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "2419200", 2024-04-17T06:12:59.550 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.550 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.550 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.550 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-17T06:12:59.550 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.551 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.551 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.551 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.551 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_name": { 2024-04-17T06:12:59.551 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "pool_name", 2024-04-17T06:12:59.551 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.551 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.551 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.551 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "device_health_metrics", 2024-04-17T06:12:59.551 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.551 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.551 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.552 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-17T06:12:59.552 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.552 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.552 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.552 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.552 INFO:teuthology.orchestra.run.smithi047.stdout: "retention_period": { 2024-04-17T06:12:59.552 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "retention_period", 2024-04-17T06:12:59.552 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-17T06:12:59.552 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.552 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.552 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "15552000", 2024-04-17T06:12:59.552 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.552 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.553 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.553 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how long to retain device health metrics", 2024-04-17T06:12:59.553 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.553 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.553 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.553 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.553 INFO:teuthology.orchestra.run.smithi047.stdout: "scrape_frequency": { 2024-04-17T06:12:59.553 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "scrape_frequency", 2024-04-17T06:12:59.553 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-17T06:12:59.553 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.553 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.553 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "86400", 2024-04-17T06:12:59.554 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.554 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.554 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.554 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-17T06:12:59.554 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.554 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.554 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.554 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.554 INFO:teuthology.orchestra.run.smithi047.stdout: "self_heal": { 2024-04-17T06:12:59.554 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "self_heal", 2024-04-17T06:12:59.554 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.554 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.555 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.555 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:12:59.555 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.555 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.555 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.555 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-17T06:12:59.555 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.555 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.555 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.555 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.555 INFO:teuthology.orchestra.run.smithi047.stdout: "sleep_interval": { 2024-04-17T06:12:59.555 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sleep_interval", 2024-04-17T06:12:59.556 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-17T06:12:59.556 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.556 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.556 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "600", 2024-04-17T06:12:59.556 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.556 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.556 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.556 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how frequently to wake up and check device health", 2024-04-17T06:12:59.556 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.556 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.556 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.556 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.557 INFO:teuthology.orchestra.run.smithi047.stdout: "warn_threshold": { 2024-04-17T06:12:59.557 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "warn_threshold", 2024-04-17T06:12:59.557 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-17T06:12:59.557 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.557 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.557 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "7257600", 2024-04-17T06:12:59.557 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.557 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.557 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.557 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-17T06:12:59.557 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.557 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.558 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.558 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:12:59.558 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:12:59.558 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.558 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:12:59.558 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "diskprediction_local", 2024-04-17T06:12:59.558 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-17T06:12:59.558 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-17T06:12:59.558 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-17T06:12:59.558 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-17T06:12:59.558 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-17T06:12:59.558 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.559 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.559 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.559 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:12:59.559 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.559 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.559 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:12:59.559 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:12:59.559 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:12:59.559 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:12:59.559 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:12:59.559 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:12:59.560 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:12:59.560 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:12:59.560 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.560 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.560 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.560 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.560 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.560 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-17T06:12:59.560 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-17T06:12:59.560 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.560 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.560 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.561 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.561 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.561 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.561 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.561 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.561 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.561 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.561 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.561 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.561 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-17T06:12:59.561 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-17T06:12:59.561 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.562 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.562 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.562 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-17T06:12:59.562 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.562 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.562 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:12:59.562 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:12:59.562 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:12:59.562 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:12:59.562 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:12:59.562 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:12:59.562 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:12:59.562 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:12:59.563 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.563 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.563 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.563 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.563 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.563 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-17T06:12:59.563 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-17T06:12:59.563 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.563 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.563 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.563 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.563 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.564 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.564 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.564 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.564 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.564 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.564 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.564 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.564 INFO:teuthology.orchestra.run.smithi047.stdout: "predict_interval": { 2024-04-17T06:12:59.564 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "predict_interval", 2024-04-17T06:12:59.564 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.564 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.564 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.564 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "86400", 2024-04-17T06:12:59.565 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.565 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.565 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.565 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.565 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.565 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.565 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.565 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.565 INFO:teuthology.orchestra.run.smithi047.stdout: "predictor_model": { 2024-04-17T06:12:59.565 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "predictor_model", 2024-04-17T06:12:59.565 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.565 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.565 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.566 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "prophetstor", 2024-04-17T06:12:59.566 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.566 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.566 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.566 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.566 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.566 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.566 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.566 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.567 INFO:teuthology.orchestra.run.smithi047.stdout: "sleep_interval": { 2024-04-17T06:12:59.567 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sleep_interval", 2024-04-17T06:12:59.567 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.567 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.567 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.567 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "600", 2024-04-17T06:12:59.567 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.567 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.567 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.567 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.567 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.567 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.567 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.568 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:12:59.568 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:12:59.568 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.568 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:12:59.568 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "influx", 2024-04-17T06:12:59.568 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": false, 2024-04-17T06:12:59.568 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "influxdb python module not found", 2024-04-17T06:12:59.568 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-17T06:12:59.568 INFO:teuthology.orchestra.run.smithi047.stdout: "batch_size": { 2024-04-17T06:12:59.568 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "batch_size", 2024-04-17T06:12:59.568 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:12:59.568 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.568 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.569 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "5000", 2024-04-17T06:12:59.569 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.569 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.569 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.569 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-17T06:12:59.569 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.569 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.569 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.569 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.569 INFO:teuthology.orchestra.run.smithi047.stdout: "database": { 2024-04-17T06:12:59.569 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "database", 2024-04-17T06:12:59.569 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.570 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.570 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.570 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "ceph", 2024-04-17T06:12:59.570 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.570 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.570 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.570 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-17T06:12:59.570 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.570 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.570 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.570 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.570 INFO:teuthology.orchestra.run.smithi047.stdout: "hostname": { 2024-04-17T06:12:59.571 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "hostname", 2024-04-17T06:12:59.571 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.571 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.571 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.571 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:12:59.571 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.571 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.571 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.571 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "InfluxDB server hostname", 2024-04-17T06:12:59.571 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.571 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.571 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.571 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.572 INFO:teuthology.orchestra.run.smithi047.stdout: "interval": { 2024-04-17T06:12:59.572 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "interval", 2024-04-17T06:12:59.572 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-17T06:12:59.572 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.572 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.572 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "30", 2024-04-17T06:12:59.572 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "5", 2024-04-17T06:12:59.572 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.572 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.572 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-17T06:12:59.572 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.572 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.573 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.573 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.573 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-17T06:12:59.573 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-17T06:12:59.573 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.573 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.573 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.573 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:12:59.573 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.573 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.573 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:12:59.573 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:12:59.574 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:12:59.574 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:12:59.574 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:12:59.574 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:12:59.574 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:12:59.574 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:12:59.574 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.574 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.574 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.574 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.574 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.575 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-17T06:12:59.575 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-17T06:12:59.575 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.575 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.575 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.575 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.575 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.575 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.575 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.575 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.575 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.575 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.576 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.576 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.576 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-17T06:12:59.576 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-17T06:12:59.576 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.576 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.576 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.576 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-17T06:12:59.576 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.576 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.576 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:12:59.576 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:12:59.576 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:12:59.577 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:12:59.577 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:12:59.577 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:12:59.577 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:12:59.577 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:12:59.577 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.577 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.577 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.577 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.577 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.577 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-17T06:12:59.577 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-17T06:12:59.577 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.578 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.578 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.578 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.578 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.578 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.578 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.578 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.578 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.578 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.578 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.578 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.578 INFO:teuthology.orchestra.run.smithi047.stdout: "password": { 2024-04-17T06:12:59.579 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "password", 2024-04-17T06:12:59.579 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.579 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.579 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.579 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:12:59.579 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.579 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.579 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.579 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "password of InfluxDB server user", 2024-04-17T06:12:59.579 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.579 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.579 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.579 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.580 INFO:teuthology.orchestra.run.smithi047.stdout: "port": { 2024-04-17T06:12:59.580 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "port", 2024-04-17T06:12:59.580 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:12:59.580 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.580 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.580 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "8086", 2024-04-17T06:12:59.580 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.580 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.580 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.580 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "InfluxDB server port", 2024-04-17T06:12:59.580 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.580 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.580 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.580 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.581 INFO:teuthology.orchestra.run.smithi047.stdout: "ssl": { 2024-04-17T06:12:59.581 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ssl", 2024-04-17T06:12:59.581 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.581 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.581 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.581 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "false", 2024-04-17T06:12:59.581 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.581 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.581 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.581 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-17T06:12:59.581 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.581 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.581 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.582 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.582 INFO:teuthology.orchestra.run.smithi047.stdout: "threads": { 2024-04-17T06:12:59.582 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "threads", 2024-04-17T06:12:59.582 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:12:59.582 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.582 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.582 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "5", 2024-04-17T06:12:59.582 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "1", 2024-04-17T06:12:59.582 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "32", 2024-04-17T06:12:59.582 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.582 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-17T06:12:59.582 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.582 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.583 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.583 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.583 INFO:teuthology.orchestra.run.smithi047.stdout: "username": { 2024-04-17T06:12:59.583 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "username", 2024-04-17T06:12:59.583 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.583 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.583 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.583 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:12:59.583 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.583 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.583 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.583 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "username of InfluxDB server user", 2024-04-17T06:12:59.584 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.584 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.584 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.584 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.584 INFO:teuthology.orchestra.run.smithi047.stdout: "verify_ssl": { 2024-04-17T06:12:59.584 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "verify_ssl", 2024-04-17T06:12:59.584 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.584 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.584 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.584 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "true", 2024-04-17T06:12:59.584 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.584 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.584 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.585 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-17T06:12:59.585 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.585 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.585 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.585 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:12:59.585 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:12:59.585 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.585 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:12:59.585 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "insights", 2024-04-17T06:12:59.585 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-17T06:12:59.585 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-17T06:12:59.585 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-17T06:12:59.585 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-17T06:12:59.585 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-17T06:12:59.586 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.586 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.586 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.586 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:12:59.586 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.586 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.586 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:12:59.586 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:12:59.586 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:12:59.586 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:12:59.586 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:12:59.586 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:12:59.587 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:12:59.587 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:12:59.587 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.587 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.587 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.587 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.587 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.587 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-17T06:12:59.587 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-17T06:12:59.587 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.587 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.588 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.588 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.588 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.588 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.588 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.588 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.588 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.588 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.588 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.588 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.588 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-17T06:12:59.588 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-17T06:12:59.588 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.589 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.589 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.589 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-17T06:12:59.589 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.589 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.589 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:12:59.589 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:12:59.589 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:12:59.589 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:12:59.589 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:12:59.589 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:12:59.590 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:12:59.590 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:12:59.590 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.590 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.590 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.590 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.590 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.590 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-17T06:12:59.590 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-17T06:12:59.590 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.590 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.590 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.591 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.591 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.591 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.591 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.591 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.591 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.591 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.591 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.591 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:12:59.591 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:12:59.591 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.592 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:12:59.592 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "iostat", 2024-04-17T06:12:59.592 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-17T06:12:59.592 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-17T06:12:59.592 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-17T06:12:59.592 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-17T06:12:59.592 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-17T06:12:59.592 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.592 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.592 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.592 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:12:59.592 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.592 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.593 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:12:59.593 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:12:59.593 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:12:59.593 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:12:59.593 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:12:59.593 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:12:59.593 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:12:59.593 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:12:59.593 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.593 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.593 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.593 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.594 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.594 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-17T06:12:59.594 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-17T06:12:59.594 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.594 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.594 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.594 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.594 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.594 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.594 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.594 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.595 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.595 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.595 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.595 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.595 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-17T06:12:59.595 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-17T06:12:59.595 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.595 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.595 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.595 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-17T06:12:59.595 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.595 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.595 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:12:59.596 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:12:59.596 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:12:59.596 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:12:59.596 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:12:59.596 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:12:59.596 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:12:59.596 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:12:59.596 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.596 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.596 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.596 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.596 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.597 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-17T06:12:59.597 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-17T06:12:59.597 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.597 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.597 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.597 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.597 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.597 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.597 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.597 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.597 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.597 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.598 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.598 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:12:59.598 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:12:59.598 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.598 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:12:59.598 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "k8sevents", 2024-04-17T06:12:59.598 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-17T06:12:59.598 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-17T06:12:59.598 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-17T06:12:59.598 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_event_retention_days": { 2024-04-17T06:12:59.598 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ceph_event_retention_days", 2024-04-17T06:12:59.598 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:12:59.599 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.599 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.599 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "7", 2024-04-17T06:12:59.599 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.599 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.599 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.599 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-17T06:12:59.599 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.599 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.599 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.599 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.599 INFO:teuthology.orchestra.run.smithi047.stdout: "config_check_secs": { 2024-04-17T06:12:59.599 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "config_check_secs", 2024-04-17T06:12:59.600 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:12:59.600 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.600 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.600 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-04-17T06:12:59.600 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "10", 2024-04-17T06:12:59.600 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.600 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.600 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-17T06:12:59.600 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.600 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.600 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.600 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.600 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-17T06:12:59.601 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-17T06:12:59.601 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.601 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.601 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.601 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:12:59.601 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.601 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.601 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:12:59.601 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:12:59.601 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:12:59.601 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:12:59.601 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:12:59.601 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:12:59.601 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:12:59.602 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:12:59.602 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.602 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.602 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.602 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.602 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.602 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-17T06:12:59.602 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-17T06:12:59.602 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.602 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.602 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.602 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.603 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.603 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.603 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.603 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.603 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.603 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.603 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.603 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.603 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-17T06:12:59.603 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-17T06:12:59.603 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.603 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.604 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.604 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-17T06:12:59.604 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.604 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.604 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:12:59.604 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:12:59.604 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:12:59.604 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:12:59.604 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:12:59.604 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:12:59.604 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:12:59.604 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:12:59.604 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.605 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.605 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.605 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.605 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.605 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-17T06:12:59.605 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-17T06:12:59.605 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.605 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.605 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.605 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.605 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.605 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.606 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.606 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.606 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.606 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.606 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.606 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:12:59.606 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:12:59.606 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.606 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:12:59.606 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "localpool", 2024-04-17T06:12:59.606 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-17T06:12:59.606 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-17T06:12:59.607 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-17T06:12:59.607 INFO:teuthology.orchestra.run.smithi047.stdout: "failure_domain": { 2024-04-17T06:12:59.607 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "failure_domain", 2024-04-17T06:12:59.607 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.607 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.607 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.607 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "host", 2024-04-17T06:12:59.607 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.607 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.607 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.607 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "failure domain for any created local pool", 2024-04-17T06:12:59.608 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-17T06:12:59.608 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.608 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.608 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.608 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-17T06:12:59.608 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-17T06:12:59.608 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.608 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.608 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.608 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:12:59.608 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.608 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.609 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:12:59.609 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:12:59.609 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:12:59.609 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:12:59.609 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:12:59.609 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:12:59.609 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:12:59.609 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:12:59.609 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.609 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.609 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.609 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.609 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.610 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-17T06:12:59.610 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-17T06:12:59.610 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.610 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.610 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.610 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.610 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.610 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.610 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.610 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.610 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.610 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.610 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.611 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.611 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-17T06:12:59.611 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-17T06:12:59.611 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.611 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.611 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.611 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-17T06:12:59.611 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.611 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.611 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:12:59.611 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:12:59.611 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:12:59.611 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:12:59.611 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:12:59.612 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:12:59.612 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:12:59.612 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:12:59.612 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.612 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.612 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.612 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.612 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.612 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-17T06:12:59.612 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-17T06:12:59.612 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.612 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.613 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.613 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.613 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.613 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.613 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.613 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.613 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.613 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.613 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.613 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.613 INFO:teuthology.orchestra.run.smithi047.stdout: "min_size": { 2024-04-17T06:12:59.613 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "min_size", 2024-04-17T06:12:59.614 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:12:59.614 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.614 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.614 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:12:59.614 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.614 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.614 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.614 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "default min_size for any created local pool", 2024-04-17T06:12:59.614 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-17T06:12:59.614 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.614 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.615 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.615 INFO:teuthology.orchestra.run.smithi047.stdout: "num_rep": { 2024-04-17T06:12:59.615 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "num_rep", 2024-04-17T06:12:59.615 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:12:59.615 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.615 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.615 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "3", 2024-04-17T06:12:59.615 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.615 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.615 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.615 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "default replica count for any created local pool", 2024-04-17T06:12:59.615 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.615 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.616 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.616 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.616 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num": { 2024-04-17T06:12:59.616 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "pg_num", 2024-04-17T06:12:59.616 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:12:59.616 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.616 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.616 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "128", 2024-04-17T06:12:59.616 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.616 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.616 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.616 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "default pg_num for any created local pool", 2024-04-17T06:12:59.616 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.617 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.617 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.617 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.617 INFO:teuthology.orchestra.run.smithi047.stdout: "prefix": { 2024-04-17T06:12:59.617 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "prefix", 2024-04-17T06:12:59.617 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.617 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.617 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.617 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:12:59.617 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.617 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.617 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.618 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "name prefix for any created local pool", 2024-04-17T06:12:59.618 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.618 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.618 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.618 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.618 INFO:teuthology.orchestra.run.smithi047.stdout: "subtree": { 2024-04-17T06:12:59.618 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "subtree", 2024-04-17T06:12:59.618 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.618 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.618 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.618 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "rack", 2024-04-17T06:12:59.618 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.619 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.619 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.619 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-17T06:12:59.619 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-17T06:12:59.619 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.619 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.619 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:12:59.619 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:12:59.619 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.619 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:12:59.619 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "mds_autoscaler", 2024-04-17T06:12:59.619 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-17T06:12:59.620 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-17T06:12:59.620 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-17T06:12:59.620 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-17T06:12:59.620 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-17T06:12:59.620 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.620 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.620 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.620 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:12:59.620 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.620 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.620 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:12:59.620 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:12:59.621 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:12:59.621 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:12:59.621 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:12:59.621 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:12:59.621 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:12:59.621 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:12:59.621 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.621 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.621 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.621 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.621 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.621 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-17T06:12:59.622 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-17T06:12:59.622 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.622 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.622 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.622 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.622 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.622 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.622 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.622 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.622 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.622 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.622 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.623 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.623 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-17T06:12:59.623 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-17T06:12:59.623 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.623 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.623 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.623 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-17T06:12:59.623 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.623 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.623 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:12:59.623 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:12:59.623 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:12:59.623 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:12:59.623 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:12:59.624 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:12:59.624 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:12:59.624 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:12:59.624 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.624 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.624 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.624 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.624 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.624 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-17T06:12:59.624 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-17T06:12:59.624 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.624 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.625 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.625 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.625 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.625 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.625 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.625 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.625 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.625 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.625 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.625 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:12:59.625 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:12:59.625 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.625 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:12:59.625 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "mirroring", 2024-04-17T06:12:59.626 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-17T06:12:59.626 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-17T06:12:59.626 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-17T06:12:59.626 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-17T06:12:59.626 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-17T06:12:59.626 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.626 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.626 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.626 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:12:59.626 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.626 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.626 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:12:59.627 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:12:59.627 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:12:59.627 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:12:59.627 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:12:59.627 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:12:59.627 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:12:59.627 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:12:59.627 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.627 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.627 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.627 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.627 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.628 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-17T06:12:59.628 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-17T06:12:59.628 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.628 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.628 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.628 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.628 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.628 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.628 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.628 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.628 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.628 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.628 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.629 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.629 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-17T06:12:59.629 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-17T06:12:59.629 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.629 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.629 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.629 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-17T06:12:59.629 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.629 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.629 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:12:59.629 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:12:59.629 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:12:59.630 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:12:59.630 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:12:59.630 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:12:59.630 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:12:59.630 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:12:59.630 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.630 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.630 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.630 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.630 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.630 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-17T06:12:59.630 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-17T06:12:59.631 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.631 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.631 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.631 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.631 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.631 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.631 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.631 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.631 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.631 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.631 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.631 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:12:59.631 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:12:59.632 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.632 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:12:59.632 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "nfs", 2024-04-17T06:12:59.632 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-17T06:12:59.632 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-17T06:12:59.632 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-17T06:12:59.632 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-17T06:12:59.632 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-17T06:12:59.632 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.632 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.632 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.632 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:12:59.632 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.633 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.633 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:12:59.633 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:12:59.633 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:12:59.633 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:12:59.633 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:12:59.633 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:12:59.633 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:12:59.633 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:12:59.633 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.633 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.633 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.633 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.634 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.634 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-17T06:12:59.634 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-17T06:12:59.634 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.634 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.634 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.634 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.634 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.634 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.634 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.634 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.634 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.634 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.635 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.635 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.635 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-17T06:12:59.635 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-17T06:12:59.635 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.635 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.635 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.635 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-17T06:12:59.635 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.635 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.635 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:12:59.635 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:12:59.635 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:12:59.636 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:12:59.636 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:12:59.636 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:12:59.636 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:12:59.636 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:12:59.636 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.636 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.636 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.636 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.636 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.637 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-17T06:12:59.637 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-17T06:12:59.637 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.637 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.637 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.637 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.637 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.637 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.637 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.637 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.637 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.637 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.638 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.638 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:12:59.638 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:12:59.638 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.638 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:12:59.638 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "orchestrator", 2024-04-17T06:12:59.638 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-17T06:12:59.638 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-17T06:12:59.638 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-17T06:12:59.638 INFO:teuthology.orchestra.run.smithi047.stdout: "fail_fs": { 2024-04-17T06:12:59.638 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "fail_fs", 2024-04-17T06:12:59.638 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.639 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.639 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.639 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.639 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.639 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.639 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.639 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-17T06:12:59.639 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.639 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.639 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.639 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.639 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-17T06:12:59.639 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-17T06:12:59.640 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.640 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.640 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.640 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:12:59.640 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.640 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.640 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:12:59.640 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:12:59.640 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:12:59.640 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:12:59.640 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:12:59.640 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:12:59.640 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:12:59.640 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:12:59.641 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.641 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.641 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.641 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.641 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.641 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-17T06:12:59.641 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-17T06:12:59.641 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.641 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.641 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.641 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.641 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.641 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.642 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.642 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.642 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.642 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.642 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.642 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.642 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-17T06:12:59.642 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-17T06:12:59.642 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.642 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.642 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.642 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-17T06:12:59.643 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.643 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.643 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:12:59.643 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:12:59.643 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:12:59.643 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:12:59.643 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:12:59.643 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:12:59.643 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:12:59.643 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:12:59.643 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.643 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.644 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.644 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.644 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.644 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-17T06:12:59.644 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-17T06:12:59.644 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.644 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.644 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.644 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.644 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.644 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.644 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.645 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.645 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.645 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.645 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.645 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.645 INFO:teuthology.orchestra.run.smithi047.stdout: "orchestrator": { 2024-04-17T06:12:59.645 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "orchestrator", 2024-04-17T06:12:59.645 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.645 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.645 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.645 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:12:59.645 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.645 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.645 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:12:59.646 INFO:teuthology.orchestra.run.smithi047.stdout: "cephadm", 2024-04-17T06:12:59.646 INFO:teuthology.orchestra.run.smithi047.stdout: "rook", 2024-04-17T06:12:59.646 INFO:teuthology.orchestra.run.smithi047.stdout: "test_orchestrator" 2024-04-17T06:12:59.646 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:12:59.646 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Orchestrator backend", 2024-04-17T06:12:59.646 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.646 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.646 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.646 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:12:59.646 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:12:59.646 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.646 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:12:59.646 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "osd_perf_query", 2024-04-17T06:12:59.646 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-17T06:12:59.647 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-17T06:12:59.647 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-17T06:12:59.647 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-17T06:12:59.647 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-17T06:12:59.647 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.647 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.647 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.647 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:12:59.647 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.647 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.647 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:12:59.647 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:12:59.647 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:12:59.648 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:12:59.648 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:12:59.648 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:12:59.648 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:12:59.648 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:12:59.648 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.648 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.648 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.648 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.648 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.648 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-17T06:12:59.648 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-17T06:12:59.649 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.649 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.649 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.649 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.649 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.649 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.649 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.649 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.649 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.649 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.649 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.649 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.649 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-17T06:12:59.650 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-17T06:12:59.650 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.650 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.650 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.650 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-17T06:12:59.650 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.650 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.650 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:12:59.650 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:12:59.650 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:12:59.650 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:12:59.650 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:12:59.650 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:12:59.651 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:12:59.651 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:12:59.651 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.651 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.651 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.651 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.651 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.651 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-17T06:12:59.651 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-17T06:12:59.651 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.651 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.652 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.652 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.652 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.652 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.652 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.652 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.652 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.652 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.652 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.652 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:12:59.652 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:12:59.652 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.652 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:12:59.653 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "osd_support", 2024-04-17T06:12:59.653 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-17T06:12:59.653 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-17T06:12:59.653 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-17T06:12:59.653 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-17T06:12:59.653 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-17T06:12:59.653 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.653 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.653 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.653 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:12:59.653 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.653 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.653 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:12:59.654 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:12:59.654 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:12:59.654 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:12:59.654 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:12:59.654 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:12:59.654 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:12:59.654 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:12:59.654 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.654 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.654 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.654 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.654 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.654 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-17T06:12:59.655 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-17T06:12:59.655 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.655 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.655 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.655 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.655 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.655 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.655 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.655 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.655 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.655 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.655 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.655 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.656 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-17T06:12:59.656 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-17T06:12:59.656 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.656 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.656 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.656 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-17T06:12:59.656 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.656 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.656 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:12:59.656 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:12:59.656 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:12:59.656 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:12:59.656 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:12:59.657 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:12:59.657 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:12:59.657 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:12:59.657 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.657 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.657 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.657 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.657 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.657 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-17T06:12:59.657 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-17T06:12:59.657 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.657 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.658 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.658 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.658 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.658 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.658 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.658 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.658 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.658 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.658 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.658 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:12:59.658 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:12:59.659 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.659 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:12:59.659 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "pg_autoscaler", 2024-04-17T06:12:59.659 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-17T06:12:59.659 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-17T06:12:59.659 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-17T06:12:59.659 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-17T06:12:59.659 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-17T06:12:59.659 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.659 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.659 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.659 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:12:59.659 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.660 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.660 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:12:59.660 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:12:59.660 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:12:59.660 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:12:59.660 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:12:59.660 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:12:59.660 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:12:59.660 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:12:59.660 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.660 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.660 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.660 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.660 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.661 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-17T06:12:59.661 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-17T06:12:59.661 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.661 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.661 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.661 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.661 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.661 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.661 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.661 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.661 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.661 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.661 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.662 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.662 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-17T06:12:59.662 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-17T06:12:59.662 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.662 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.662 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.662 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-17T06:12:59.662 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.662 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.662 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:12:59.662 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:12:59.662 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:12:59.663 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:12:59.663 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:12:59.663 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:12:59.663 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:12:59.663 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:12:59.663 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.663 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.663 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.663 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.663 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.663 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-17T06:12:59.663 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-17T06:12:59.663 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.664 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.664 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.664 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.664 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.664 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.664 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.664 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.664 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.664 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.664 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.664 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.664 INFO:teuthology.orchestra.run.smithi047.stdout: "sleep_interval": { 2024-04-17T06:12:59.664 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sleep_interval", 2024-04-17T06:12:59.664 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-17T06:12:59.665 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.665 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.665 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "60", 2024-04-17T06:12:59.665 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.665 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.665 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.665 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.665 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.665 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.665 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.665 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.665 INFO:teuthology.orchestra.run.smithi047.stdout: "threshold": { 2024-04-17T06:12:59.665 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "threshold", 2024-04-17T06:12:59.665 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "float", 2024-04-17T06:12:59.665 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.666 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.666 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "3.0", 2024-04-17T06:12:59.666 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "1.0", 2024-04-17T06:12:59.666 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.666 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.666 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "scaling threshold", 2024-04-17T06:12:59.666 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-17T06:12:59.666 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.666 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.666 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:12:59.666 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:12:59.666 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.666 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:12:59.667 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "progress", 2024-04-17T06:12:59.667 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-17T06:12:59.667 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-17T06:12:59.667 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-17T06:12:59.667 INFO:teuthology.orchestra.run.smithi047.stdout: "allow_pg_recovery_event": { 2024-04-17T06:12:59.667 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "allow_pg_recovery_event", 2024-04-17T06:12:59.667 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.667 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.667 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.667 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.667 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.667 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.667 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.667 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-17T06:12:59.668 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.668 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.668 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.668 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.668 INFO:teuthology.orchestra.run.smithi047.stdout: "enabled": { 2024-04-17T06:12:59.668 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "enabled", 2024-04-17T06:12:59.668 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.668 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.668 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.668 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:12:59.668 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.668 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.669 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.669 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.669 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.669 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.669 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.669 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.669 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-17T06:12:59.669 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-17T06:12:59.669 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.669 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.669 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.669 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:12:59.669 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.670 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.670 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:12:59.670 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:12:59.670 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:12:59.670 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:12:59.670 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:12:59.670 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:12:59.670 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:12:59.670 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:12:59.670 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.670 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.670 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.670 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.671 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.671 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-17T06:12:59.671 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-17T06:12:59.671 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.671 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.671 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.671 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.671 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.671 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.671 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.671 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.671 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.672 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.672 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.672 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.672 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-17T06:12:59.672 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-17T06:12:59.672 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.672 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.672 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.672 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-17T06:12:59.672 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.672 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.672 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:12:59.673 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:12:59.673 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:12:59.673 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:12:59.673 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:12:59.673 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:12:59.673 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:12:59.673 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:12:59.673 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.673 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.673 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.673 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.674 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.674 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-17T06:12:59.674 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-17T06:12:59.674 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.674 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.674 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.674 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.674 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.674 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.674 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.674 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.674 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.675 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.675 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.675 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.675 INFO:teuthology.orchestra.run.smithi047.stdout: "max_completed_events": { 2024-04-17T06:12:59.675 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "max_completed_events", 2024-04-17T06:12:59.675 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:12:59.675 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.675 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.675 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "50", 2024-04-17T06:12:59.675 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.675 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.675 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.676 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "number of past completed events to remember", 2024-04-17T06:12:59.676 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.676 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.676 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.676 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.676 INFO:teuthology.orchestra.run.smithi047.stdout: "sleep_interval": { 2024-04-17T06:12:59.676 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sleep_interval", 2024-04-17T06:12:59.676 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-17T06:12:59.676 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.676 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.676 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "5", 2024-04-17T06:12:59.676 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.677 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.677 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.677 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how long the module is going to sleep", 2024-04-17T06:12:59.677 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.677 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.677 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.677 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:12:59.677 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:12:59.677 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.677 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:12:59.677 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "prometheus", 2024-04-17T06:12:59.677 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-17T06:12:59.678 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-17T06:12:59.678 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-17T06:12:59.678 INFO:teuthology.orchestra.run.smithi047.stdout: "cache": { 2024-04-17T06:12:59.678 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "cache", 2024-04-17T06:12:59.678 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.678 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.678 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.678 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:12:59.678 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.678 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.678 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.679 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.679 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.679 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.679 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.679 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.679 INFO:teuthology.orchestra.run.smithi047.stdout: "exclude_perf_counters": { 2024-04-17T06:12:59.679 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "exclude_perf_counters", 2024-04-17T06:12:59.679 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.679 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.679 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.679 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:12:59.679 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.680 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.680 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.680 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-17T06:12:59.680 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-17T06:12:59.680 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.680 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.680 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.680 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-17T06:12:59.680 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-17T06:12:59.680 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.680 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.680 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.681 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:12:59.681 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.681 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.681 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:12:59.681 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:12:59.681 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:12:59.681 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:12:59.681 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:12:59.681 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:12:59.681 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:12:59.681 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:12:59.681 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.681 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.681 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.682 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.682 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.682 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-17T06:12:59.682 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-17T06:12:59.682 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.682 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.682 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.682 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.682 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.682 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.682 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.682 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.683 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.683 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.683 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.683 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.683 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-17T06:12:59.683 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-17T06:12:59.683 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.683 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.683 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.683 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-17T06:12:59.683 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.683 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.683 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:12:59.683 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:12:59.684 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:12:59.684 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:12:59.684 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:12:59.684 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:12:59.684 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:12:59.684 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:12:59.684 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.684 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.684 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.684 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.684 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.684 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-17T06:12:59.684 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-17T06:12:59.684 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.685 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.685 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.685 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.685 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.685 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.685 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.685 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.685 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.685 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.685 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.685 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.685 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_stats_pools": { 2024-04-17T06:12:59.685 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rbd_stats_pools", 2024-04-17T06:12:59.685 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.686 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.686 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.686 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:12:59.686 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.686 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.686 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.686 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.686 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.686 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.686 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.686 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.686 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-17T06:12:59.686 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-17T06:12:59.687 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:12:59.687 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.687 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.687 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "300", 2024-04-17T06:12:59.687 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.687 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.687 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.687 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.687 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.687 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.687 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.687 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.688 INFO:teuthology.orchestra.run.smithi047.stdout: "scrape_interval": { 2024-04-17T06:12:59.688 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "scrape_interval", 2024-04-17T06:12:59.688 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "float", 2024-04-17T06:12:59.688 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.688 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.688 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "15.0", 2024-04-17T06:12:59.688 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.688 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.688 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.688 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.688 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.688 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.689 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.689 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.689 INFO:teuthology.orchestra.run.smithi047.stdout: "server_addr": { 2024-04-17T06:12:59.689 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "server_addr", 2024-04-17T06:12:59.689 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.689 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.689 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.689 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "::", 2024-04-17T06:12:59.689 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.689 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.689 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.689 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-17T06:12:59.689 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.690 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.690 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.690 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.690 INFO:teuthology.orchestra.run.smithi047.stdout: "server_port": { 2024-04-17T06:12:59.690 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "server_port", 2024-04-17T06:12:59.690 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:12:59.690 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.690 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.690 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "9283", 2024-04-17T06:12:59.690 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.690 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.690 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.690 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-17T06:12:59.691 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.691 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.691 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.691 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.691 INFO:teuthology.orchestra.run.smithi047.stdout: "stale_cache_strategy": { 2024-04-17T06:12:59.691 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "stale_cache_strategy", 2024-04-17T06:12:59.691 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.691 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.691 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.691 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "log", 2024-04-17T06:12:59.691 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.691 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.691 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.691 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.692 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.692 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.692 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.692 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.692 INFO:teuthology.orchestra.run.smithi047.stdout: "standby_behaviour": { 2024-04-17T06:12:59.692 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "standby_behaviour", 2024-04-17T06:12:59.692 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.692 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.692 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.692 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "default", 2024-04-17T06:12:59.692 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.692 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.692 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:12:59.693 INFO:teuthology.orchestra.run.smithi047.stdout: "default", 2024-04-17T06:12:59.693 INFO:teuthology.orchestra.run.smithi047.stdout: "error" 2024-04-17T06:12:59.693 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:12:59.693 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.693 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.693 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.693 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.693 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.693 INFO:teuthology.orchestra.run.smithi047.stdout: "standby_error_status_code": { 2024-04-17T06:12:59.693 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "standby_error_status_code", 2024-04-17T06:12:59.693 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:12:59.693 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.694 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.694 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "500", 2024-04-17T06:12:59.694 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "400", 2024-04-17T06:12:59.694 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "599", 2024-04-17T06:12:59.694 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.694 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.694 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.694 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.694 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.694 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:12:59.694 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:12:59.695 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.695 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:12:59.695 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rbd_support", 2024-04-17T06:12:59.695 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-17T06:12:59.695 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-17T06:12:59.695 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-17T06:12:59.695 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-17T06:12:59.695 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-17T06:12:59.695 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.695 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.695 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.695 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:12:59.695 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.696 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.696 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:12:59.696 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:12:59.696 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:12:59.696 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:12:59.696 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:12:59.696 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:12:59.696 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:12:59.696 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:12:59.696 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.696 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.696 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.696 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.696 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.696 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-17T06:12:59.697 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-17T06:12:59.697 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.697 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.697 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.697 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.697 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.697 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.697 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.697 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.697 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.697 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.697 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.697 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.698 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-17T06:12:59.698 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-17T06:12:59.698 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.698 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.698 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.698 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-17T06:12:59.698 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.698 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.698 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:12:59.698 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:12:59.698 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:12:59.698 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:12:59.698 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:12:59.699 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:12:59.699 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:12:59.699 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:12:59.699 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.699 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.699 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.699 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.699 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.699 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-17T06:12:59.699 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-17T06:12:59.699 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.699 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.700 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.700 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.700 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.700 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.700 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.700 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.700 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.700 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.700 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.700 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.700 INFO:teuthology.orchestra.run.smithi047.stdout: "max_concurrent_snap_create": { 2024-04-17T06:12:59.700 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "max_concurrent_snap_create", 2024-04-17T06:12:59.700 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:12:59.700 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.701 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.701 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-04-17T06:12:59.701 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.701 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.701 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.701 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.701 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.701 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.701 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.701 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.701 INFO:teuthology.orchestra.run.smithi047.stdout: "mirror_snapshot_schedule": { 2024-04-17T06:12:59.701 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "mirror_snapshot_schedule", 2024-04-17T06:12:59.701 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.701 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.702 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.702 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:12:59.702 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.702 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.702 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.702 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.702 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.702 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.702 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.702 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.702 INFO:teuthology.orchestra.run.smithi047.stdout: "trash_purge_schedule": { 2024-04-17T06:12:59.703 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "trash_purge_schedule", 2024-04-17T06:12:59.703 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.703 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.703 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.703 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:12:59.703 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.703 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.703 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.703 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.703 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.704 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.704 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.704 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:12:59.704 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:12:59.704 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.704 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:12:59.704 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "restful", 2024-04-17T06:12:59.704 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-17T06:12:59.704 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-17T06:12:59.704 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-17T06:12:59.704 INFO:teuthology.orchestra.run.smithi047.stdout: "enable_auth": { 2024-04-17T06:12:59.704 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "enable_auth", 2024-04-17T06:12:59.705 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.705 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.705 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.705 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:12:59.705 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.705 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.705 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.705 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.705 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.705 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.705 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.705 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.705 INFO:teuthology.orchestra.run.smithi047.stdout: "key_file": { 2024-04-17T06:12:59.705 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "key_file", 2024-04-17T06:12:59.706 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.706 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.706 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.706 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:12:59.706 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.706 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.706 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.706 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.706 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.706 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.706 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.706 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.706 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-17T06:12:59.706 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-17T06:12:59.707 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.707 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.707 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.707 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:12:59.707 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.707 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.707 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:12:59.707 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:12:59.707 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:12:59.707 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:12:59.707 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:12:59.707 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:12:59.707 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:12:59.708 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:12:59.708 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.708 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.708 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.708 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.708 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.708 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-17T06:12:59.708 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-17T06:12:59.708 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.708 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.708 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.708 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.709 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.709 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.709 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.709 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.709 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.709 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.709 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.709 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.709 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-17T06:12:59.709 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-17T06:12:59.709 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.709 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.709 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.710 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-17T06:12:59.710 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.710 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.710 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:12:59.710 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:12:59.710 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:12:59.710 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:12:59.710 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:12:59.710 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:12:59.710 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:12:59.710 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:12:59.710 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.710 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.710 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.711 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.711 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.711 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-17T06:12:59.711 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-17T06:12:59.711 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.711 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.711 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.711 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.711 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.711 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.711 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.712 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.712 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.712 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.712 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.712 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.712 INFO:teuthology.orchestra.run.smithi047.stdout: "server_addr": { 2024-04-17T06:12:59.712 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "server_addr", 2024-04-17T06:12:59.712 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.712 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.712 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.712 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:12:59.712 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.712 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.713 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.713 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.713 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.713 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.713 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.713 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.713 INFO:teuthology.orchestra.run.smithi047.stdout: "server_port": { 2024-04-17T06:12:59.713 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "server_port", 2024-04-17T06:12:59.713 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.713 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.713 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.713 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:12:59.713 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.714 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.714 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.714 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.714 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.714 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.714 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.714 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:12:59.714 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:12:59.714 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.714 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:12:59.714 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rgw", 2024-04-17T06:12:59.714 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-17T06:12:59.715 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-17T06:12:59.715 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-17T06:12:59.715 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-17T06:12:59.715 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-17T06:12:59.715 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.715 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.715 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.715 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:12:59.715 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.715 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.715 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:12:59.715 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:12:59.715 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:12:59.715 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:12:59.716 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:12:59.716 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:12:59.716 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:12:59.716 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:12:59.716 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.716 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.716 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.716 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.716 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.716 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-17T06:12:59.716 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-17T06:12:59.716 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.716 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.717 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.717 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.717 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.717 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.717 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.717 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.717 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.717 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.717 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.717 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.717 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-17T06:12:59.717 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-17T06:12:59.717 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.718 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.718 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.718 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-17T06:12:59.718 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.718 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.718 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:12:59.718 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:12:59.718 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:12:59.718 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:12:59.718 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:12:59.718 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:12:59.719 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:12:59.719 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:12:59.719 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.719 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.719 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.719 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.719 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.719 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-17T06:12:59.719 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-17T06:12:59.719 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.719 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.720 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.720 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.720 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.720 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.720 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.720 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.720 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.720 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.720 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.720 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:12:59.720 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:12:59.720 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.720 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:12:59.721 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rook", 2024-04-17T06:12:59.721 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-17T06:12:59.721 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-17T06:12:59.721 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-17T06:12:59.721 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-17T06:12:59.721 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-17T06:12:59.721 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.721 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.721 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.721 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:12:59.721 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.721 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.721 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:12:59.721 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:12:59.722 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:12:59.722 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:12:59.722 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:12:59.722 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:12:59.722 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:12:59.722 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:12:59.722 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.722 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.722 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.722 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.722 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.722 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-17T06:12:59.723 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-17T06:12:59.723 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.723 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.723 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.723 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.723 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.723 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.723 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.723 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.723 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.723 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.723 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.723 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.724 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-17T06:12:59.724 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-17T06:12:59.724 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.724 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.724 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.724 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-17T06:12:59.724 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.724 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.724 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:12:59.724 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:12:59.724 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:12:59.724 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:12:59.724 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:12:59.725 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:12:59.725 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:12:59.725 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:12:59.725 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.725 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.725 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.725 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.725 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.725 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-17T06:12:59.725 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-17T06:12:59.725 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.725 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.726 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.726 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.726 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.726 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.726 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.726 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.726 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.726 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.726 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.726 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.726 INFO:teuthology.orchestra.run.smithi047.stdout: "storage_class": { 2024-04-17T06:12:59.726 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "storage_class", 2024-04-17T06:12:59.726 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.727 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.727 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.727 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "local", 2024-04-17T06:12:59.727 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.727 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.727 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.727 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-17T06:12:59.727 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.727 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.727 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.727 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:12:59.727 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:12:59.727 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.728 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:12:59.728 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "selftest", 2024-04-17T06:12:59.728 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-17T06:12:59.728 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-17T06:12:59.728 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-17T06:12:59.728 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-17T06:12:59.728 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-17T06:12:59.728 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.728 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.728 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.728 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:12:59.728 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.728 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.729 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:12:59.729 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:12:59.729 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:12:59.729 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:12:59.729 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:12:59.729 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:12:59.729 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:12:59.729 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:12:59.729 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.729 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.729 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.729 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.730 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.730 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-17T06:12:59.730 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-17T06:12:59.730 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.730 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.730 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.730 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.730 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.730 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.730 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.730 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.730 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.731 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.731 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.731 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.731 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-17T06:12:59.731 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-17T06:12:59.731 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.731 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.731 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.731 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-17T06:12:59.731 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.731 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.731 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:12:59.731 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:12:59.732 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:12:59.732 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:12:59.732 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:12:59.732 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:12:59.732 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:12:59.732 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:12:59.732 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.732 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.732 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.732 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.732 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.732 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-17T06:12:59.732 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-17T06:12:59.733 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.733 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.733 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.733 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.733 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.733 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.733 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.733 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.733 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.733 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.733 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.733 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.733 INFO:teuthology.orchestra.run.smithi047.stdout: "roption1": { 2024-04-17T06:12:59.733 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "roption1", 2024-04-17T06:12:59.734 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.734 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.734 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.734 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:12:59.734 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.734 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.734 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.734 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.734 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.734 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.734 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.734 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.735 INFO:teuthology.orchestra.run.smithi047.stdout: "roption2": { 2024-04-17T06:12:59.735 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "roption2", 2024-04-17T06:12:59.735 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.735 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.735 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.735 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "xyz", 2024-04-17T06:12:59.735 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.735 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.735 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.735 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.735 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.735 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.735 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.736 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.736 INFO:teuthology.orchestra.run.smithi047.stdout: "rwoption1": { 2024-04-17T06:12:59.736 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rwoption1", 2024-04-17T06:12:59.736 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.736 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.736 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.736 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:12:59.736 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.736 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.736 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.736 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.736 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.737 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.737 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.737 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.737 INFO:teuthology.orchestra.run.smithi047.stdout: "rwoption2": { 2024-04-17T06:12:59.737 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rwoption2", 2024-04-17T06:12:59.737 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:12:59.737 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.737 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.737 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:12:59.737 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.737 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.737 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.737 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.738 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.738 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.738 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.738 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.738 INFO:teuthology.orchestra.run.smithi047.stdout: "rwoption3": { 2024-04-17T06:12:59.738 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rwoption3", 2024-04-17T06:12:59.738 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "float", 2024-04-17T06:12:59.738 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.738 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.738 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:12:59.738 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.738 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.739 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.739 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.739 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.739 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.739 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.739 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.739 INFO:teuthology.orchestra.run.smithi047.stdout: "rwoption4": { 2024-04-17T06:12:59.739 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rwoption4", 2024-04-17T06:12:59.739 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.739 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.739 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.739 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:12:59.739 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.740 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.740 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.740 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.740 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.740 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.740 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.740 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.740 INFO:teuthology.orchestra.run.smithi047.stdout: "rwoption5": { 2024-04-17T06:12:59.740 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rwoption5", 2024-04-17T06:12:59.740 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.740 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.741 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.741 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:12:59.741 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.741 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.741 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.741 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.741 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.741 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.741 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.741 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.741 INFO:teuthology.orchestra.run.smithi047.stdout: "rwoption6": { 2024-04-17T06:12:59.741 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rwoption6", 2024-04-17T06:12:59.741 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.742 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.742 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.742 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:12:59.742 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.742 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.742 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.742 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.742 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.742 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.742 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.742 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.743 INFO:teuthology.orchestra.run.smithi047.stdout: "rwoption7": { 2024-04-17T06:12:59.743 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rwoption7", 2024-04-17T06:12:59.743 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:12:59.743 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.743 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.743 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:12:59.743 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "1", 2024-04-17T06:12:59.743 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "42", 2024-04-17T06:12:59.743 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.743 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.743 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.743 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.744 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.744 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.744 INFO:teuthology.orchestra.run.smithi047.stdout: "testkey": { 2024-04-17T06:12:59.744 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "testkey", 2024-04-17T06:12:59.744 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.744 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.744 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.744 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:12:59.744 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.744 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.744 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.744 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.744 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.745 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.745 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.745 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.745 INFO:teuthology.orchestra.run.smithi047.stdout: "testlkey": { 2024-04-17T06:12:59.745 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "testlkey", 2024-04-17T06:12:59.745 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.745 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.745 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.745 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:12:59.745 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.745 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.745 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.745 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.746 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.746 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.746 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.746 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.746 INFO:teuthology.orchestra.run.smithi047.stdout: "testnewline": { 2024-04-17T06:12:59.746 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "testnewline", 2024-04-17T06:12:59.746 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.746 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.746 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.746 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:12:59.746 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.746 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.746 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.747 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.747 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.747 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.747 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.747 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:12:59.747 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:12:59.747 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.747 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:12:59.747 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "snap_schedule", 2024-04-17T06:12:59.747 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-17T06:12:59.747 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-17T06:12:59.747 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-17T06:12:59.748 INFO:teuthology.orchestra.run.smithi047.stdout: "allow_m_granularity": { 2024-04-17T06:12:59.748 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "allow_m_granularity", 2024-04-17T06:12:59.748 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.748 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.748 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.748 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.748 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.748 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.748 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.748 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "allow minute scheduled snapshots", 2024-04-17T06:12:59.748 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.748 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.748 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.749 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.749 INFO:teuthology.orchestra.run.smithi047.stdout: "dump_on_update": { 2024-04-17T06:12:59.749 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "dump_on_update", 2024-04-17T06:12:59.749 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.749 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.749 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.749 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.749 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.749 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.749 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.749 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "dump database to debug log on update", 2024-04-17T06:12:59.749 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.749 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.750 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.750 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.750 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-17T06:12:59.750 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-17T06:12:59.750 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.750 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.750 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.750 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:12:59.750 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.750 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.750 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:12:59.750 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:12:59.750 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:12:59.751 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:12:59.751 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:12:59.751 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:12:59.751 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:12:59.751 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:12:59.751 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.751 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.751 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.751 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.751 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.751 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-17T06:12:59.751 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-17T06:12:59.751 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.752 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.752 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.752 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.752 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.752 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.752 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.752 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.752 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.752 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.752 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.752 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.752 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-17T06:12:59.752 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-17T06:12:59.752 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.753 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.753 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.753 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-17T06:12:59.753 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.753 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.753 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:12:59.753 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:12:59.753 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:12:59.753 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:12:59.753 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:12:59.753 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:12:59.753 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:12:59.754 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:12:59.754 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.754 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.754 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.754 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.754 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.754 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-17T06:12:59.754 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-17T06:12:59.754 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.754 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.754 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.754 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.755 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.755 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.755 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.755 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.755 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.755 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.755 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.755 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:12:59.755 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:12:59.755 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.755 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:12:59.755 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "stats", 2024-04-17T06:12:59.755 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-17T06:12:59.756 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-17T06:12:59.756 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-17T06:12:59.756 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-17T06:12:59.756 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-17T06:12:59.756 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.756 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.756 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.756 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:12:59.756 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.756 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.756 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:12:59.756 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:12:59.756 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:12:59.756 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:12:59.757 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:12:59.757 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:12:59.757 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:12:59.757 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:12:59.757 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.757 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.757 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.757 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.757 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.757 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-17T06:12:59.757 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-17T06:12:59.757 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.758 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.758 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.758 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.758 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.758 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.758 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.758 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.758 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.758 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.758 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.758 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.758 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-17T06:12:59.759 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-17T06:12:59.759 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.759 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.759 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.759 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-17T06:12:59.759 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.759 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.759 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:12:59.759 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:12:59.759 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:12:59.759 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:12:59.759 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:12:59.759 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:12:59.760 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:12:59.760 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:12:59.760 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.760 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.760 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.760 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.760 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.760 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-17T06:12:59.760 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-17T06:12:59.760 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.760 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.760 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.761 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.761 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.761 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.761 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.761 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.761 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.761 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.761 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.761 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:12:59.761 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:12:59.761 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.761 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:12:59.761 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "status", 2024-04-17T06:12:59.762 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-17T06:12:59.762 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-17T06:12:59.762 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-17T06:12:59.762 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-17T06:12:59.762 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-17T06:12:59.762 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.762 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.762 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.762 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:12:59.762 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.762 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.762 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:12:59.763 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:12:59.763 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:12:59.763 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:12:59.763 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:12:59.763 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:12:59.763 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:12:59.763 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:12:59.763 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.763 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.763 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.763 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.763 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.764 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-17T06:12:59.764 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-17T06:12:59.764 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.764 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.764 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.764 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.764 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.764 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.764 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.764 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.764 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.764 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.764 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.765 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.765 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-17T06:12:59.765 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-17T06:12:59.765 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.765 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.765 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.765 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-17T06:12:59.765 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.765 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.765 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:12:59.765 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:12:59.765 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:12:59.766 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:12:59.766 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:12:59.766 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:12:59.766 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:12:59.766 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:12:59.766 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.766 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.766 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.766 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.766 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.766 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-17T06:12:59.767 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-17T06:12:59.767 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.767 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.767 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.767 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.767 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.767 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.767 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.767 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.767 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.767 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.767 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.767 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:12:59.768 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:12:59.768 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.768 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:12:59.768 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "telegraf", 2024-04-17T06:12:59.768 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-17T06:12:59.768 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-17T06:12:59.768 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-17T06:12:59.768 INFO:teuthology.orchestra.run.smithi047.stdout: "address": { 2024-04-17T06:12:59.768 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "address", 2024-04-17T06:12:59.768 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.768 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.768 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.768 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-17T06:12:59.769 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.769 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.769 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.769 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.769 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.769 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.769 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.769 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.769 INFO:teuthology.orchestra.run.smithi047.stdout: "interval": { 2024-04-17T06:12:59.769 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "interval", 2024-04-17T06:12:59.769 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-17T06:12:59.769 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.769 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.769 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "15", 2024-04-17T06:12:59.770 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.770 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.770 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.770 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.770 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.770 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.770 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.770 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.770 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-17T06:12:59.770 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-17T06:12:59.770 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.770 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.771 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.771 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:12:59.771 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.771 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.771 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:12:59.771 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:12:59.771 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:12:59.771 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:12:59.771 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:12:59.771 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:12:59.771 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:12:59.771 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:12:59.771 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.772 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.772 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.772 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.772 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.772 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-17T06:12:59.772 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-17T06:12:59.772 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.772 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.772 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.772 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.772 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.772 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.773 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.773 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.773 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.773 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.773 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.773 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.773 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-17T06:12:59.773 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-17T06:12:59.773 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.773 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.773 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.773 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-17T06:12:59.773 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.774 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.774 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:12:59.774 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:12:59.774 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:12:59.774 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:12:59.774 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:12:59.774 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:12:59.774 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:12:59.774 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:12:59.774 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.774 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.774 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.775 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.775 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.775 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-17T06:12:59.775 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-17T06:12:59.775 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.775 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.775 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.775 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.775 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.775 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.775 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.775 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.775 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.776 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.776 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.776 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:12:59.776 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:12:59.776 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.776 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:12:59.776 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "telemetry", 2024-04-17T06:12:59.776 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-17T06:12:59.776 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-17T06:12:59.776 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-17T06:12:59.776 INFO:teuthology.orchestra.run.smithi047.stdout: "channel_basic": { 2024-04-17T06:12:59.776 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "channel_basic", 2024-04-17T06:12:59.777 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.777 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.777 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.777 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:12:59.777 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.777 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.777 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.777 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-17T06:12:59.777 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.777 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.777 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.777 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.778 INFO:teuthology.orchestra.run.smithi047.stdout: "channel_crash": { 2024-04-17T06:12:59.778 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "channel_crash", 2024-04-17T06:12:59.778 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.778 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.778 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.778 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:12:59.778 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.778 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.778 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.778 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-17T06:12:59.778 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.778 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.778 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.779 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.779 INFO:teuthology.orchestra.run.smithi047.stdout: "channel_device": { 2024-04-17T06:12:59.779 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "channel_device", 2024-04-17T06:12:59.779 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.779 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.779 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.779 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:12:59.779 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.779 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.779 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.779 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-17T06:12:59.779 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.779 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.780 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.780 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.780 INFO:teuthology.orchestra.run.smithi047.stdout: "channel_ident": { 2024-04-17T06:12:59.780 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "channel_ident", 2024-04-17T06:12:59.780 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.780 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.780 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.780 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.780 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.780 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.780 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.781 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-17T06:12:59.781 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.781 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.781 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.781 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.781 INFO:teuthology.orchestra.run.smithi047.stdout: "channel_perf": { 2024-04-17T06:12:59.781 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "channel_perf", 2024-04-17T06:12:59.781 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.781 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.781 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.781 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.781 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.781 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.782 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.782 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-17T06:12:59.782 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.782 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.782 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.782 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.782 INFO:teuthology.orchestra.run.smithi047.stdout: "contact": { 2024-04-17T06:12:59.782 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "contact", 2024-04-17T06:12:59.782 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.782 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.782 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.782 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:12:59.783 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.783 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.783 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.783 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.783 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.783 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.783 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.783 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.783 INFO:teuthology.orchestra.run.smithi047.stdout: "description": { 2024-04-17T06:12:59.783 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "description", 2024-04-17T06:12:59.783 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.783 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.783 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.784 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:12:59.784 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.784 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.784 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.784 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.784 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.784 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.784 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.784 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.784 INFO:teuthology.orchestra.run.smithi047.stdout: "device_url": { 2024-04-17T06:12:59.784 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "device_url", 2024-04-17T06:12:59.784 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.784 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.785 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.785 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-17T06:12:59.785 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.785 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.785 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.785 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.785 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.785 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.785 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.785 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.785 INFO:teuthology.orchestra.run.smithi047.stdout: "enabled": { 2024-04-17T06:12:59.785 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "enabled", 2024-04-17T06:12:59.786 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.786 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.786 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.786 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.786 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.786 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.786 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.786 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.786 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.786 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.786 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.786 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.787 INFO:teuthology.orchestra.run.smithi047.stdout: "interval": { 2024-04-17T06:12:59.787 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "interval", 2024-04-17T06:12:59.787 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:12:59.787 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.787 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.787 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "24", 2024-04-17T06:12:59.787 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "8", 2024-04-17T06:12:59.787 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.787 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.787 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.787 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.787 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.787 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.788 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.788 INFO:teuthology.orchestra.run.smithi047.stdout: "last_opt_revision": { 2024-04-17T06:12:59.788 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "last_opt_revision", 2024-04-17T06:12:59.788 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:12:59.788 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.788 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.788 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "1", 2024-04-17T06:12:59.788 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.788 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.788 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.788 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.788 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.788 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.789 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.789 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.789 INFO:teuthology.orchestra.run.smithi047.stdout: "leaderboard": { 2024-04-17T06:12:59.789 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "leaderboard", 2024-04-17T06:12:59.789 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.789 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.789 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.789 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.789 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.789 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.789 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.789 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.789 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.789 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.790 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.790 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.790 INFO:teuthology.orchestra.run.smithi047.stdout: "leaderboard_description": { 2024-04-17T06:12:59.790 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "leaderboard_description", 2024-04-17T06:12:59.790 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.790 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.790 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.790 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:12:59.790 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.790 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.790 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.790 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.791 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.791 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.791 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.791 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.791 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-17T06:12:59.791 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-17T06:12:59.791 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.791 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.791 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.791 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:12:59.791 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.791 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.791 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:12:59.791 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:12:59.792 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:12:59.792 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:12:59.792 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:12:59.792 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:12:59.792 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:12:59.792 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:12:59.792 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.792 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.792 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.792 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.792 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.792 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-17T06:12:59.792 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-17T06:12:59.793 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.793 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.793 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.793 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.793 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.793 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.793 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.793 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.793 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.793 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.793 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.793 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.793 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-17T06:12:59.794 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-17T06:12:59.794 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.794 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.794 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.794 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-17T06:12:59.794 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.794 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.794 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:12:59.794 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:12:59.794 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:12:59.794 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:12:59.794 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:12:59.795 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:12:59.795 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:12:59.795 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:12:59.795 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.795 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.795 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.795 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.795 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.795 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-17T06:12:59.795 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-17T06:12:59.795 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.795 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.796 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.796 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.796 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.796 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.796 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.796 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.796 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.796 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.796 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.796 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.796 INFO:teuthology.orchestra.run.smithi047.stdout: "organization": { 2024-04-17T06:12:59.796 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "organization", 2024-04-17T06:12:59.796 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.797 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.797 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.797 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:12:59.797 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.797 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.797 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.797 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.797 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.797 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.797 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.797 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.797 INFO:teuthology.orchestra.run.smithi047.stdout: "proxy": { 2024-04-17T06:12:59.798 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "proxy", 2024-04-17T06:12:59.798 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.798 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.798 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.798 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:12:59.798 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.798 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.798 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.798 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.798 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.798 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.798 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.799 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.799 INFO:teuthology.orchestra.run.smithi047.stdout: "url": { 2024-04-17T06:12:59.799 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "url", 2024-04-17T06:12:59.799 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.799 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.799 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.799 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-17T06:12:59.799 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.799 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.799 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.799 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.799 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.799 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.800 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.800 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:12:59.800 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:12:59.800 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.800 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:12:59.800 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "test_orchestrator", 2024-04-17T06:12:59.800 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-17T06:12:59.800 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-17T06:12:59.800 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-17T06:12:59.800 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-17T06:12:59.800 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-17T06:12:59.800 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.801 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.801 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.801 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:12:59.801 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.801 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.801 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:12:59.801 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:12:59.801 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:12:59.801 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:12:59.801 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:12:59.801 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:12:59.801 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:12:59.801 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:12:59.802 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.802 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.802 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.802 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.802 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.802 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-17T06:12:59.802 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-17T06:12:59.802 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.802 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.802 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.802 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.802 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.802 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.803 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.803 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.803 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.803 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.803 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.803 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.803 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-17T06:12:59.803 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-17T06:12:59.803 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.803 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.803 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.803 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-17T06:12:59.804 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.804 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.804 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:12:59.804 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:12:59.804 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:12:59.804 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:12:59.804 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:12:59.804 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:12:59.804 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:12:59.804 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:12:59.804 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.804 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.804 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.805 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.805 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.805 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-17T06:12:59.805 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-17T06:12:59.805 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.805 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.805 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.805 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.805 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.805 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.805 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.805 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.805 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.806 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.806 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.806 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:12:59.806 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:12:59.806 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.806 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:12:59.806 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "volumes", 2024-04-17T06:12:59.806 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-17T06:12:59.806 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-17T06:12:59.806 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-17T06:12:59.806 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-17T06:12:59.806 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-17T06:12:59.807 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.807 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.807 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.807 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:12:59.807 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.807 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.807 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:12:59.807 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:12:59.807 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:12:59.807 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:12:59.807 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:12:59.808 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:12:59.808 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:12:59.808 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:12:59.808 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.808 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.808 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.808 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.808 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.808 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-17T06:12:59.808 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-17T06:12:59.808 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.808 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.808 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.809 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.809 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.809 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.809 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.809 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.809 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.809 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.809 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.809 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.809 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-17T06:12:59.809 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-17T06:12:59.809 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.809 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.810 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.810 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-17T06:12:59.810 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.810 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.810 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:12:59.810 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:12:59.810 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:12:59.810 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:12:59.810 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:12:59.810 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:12:59.810 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:12:59.810 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:12:59.810 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.811 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.811 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.811 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.811 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.811 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-17T06:12:59.811 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-17T06:12:59.811 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.811 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.811 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.811 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.811 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.811 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.812 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.812 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.812 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.812 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.812 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.812 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.812 INFO:teuthology.orchestra.run.smithi047.stdout: "max_concurrent_clones": { 2024-04-17T06:12:59.812 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "max_concurrent_clones", 2024-04-17T06:12:59.812 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:12:59.812 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.812 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.813 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "4", 2024-04-17T06:12:59.813 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.813 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.813 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.813 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-17T06:12:59.813 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.813 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.813 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.813 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.813 INFO:teuthology.orchestra.run.smithi047.stdout: "periodic_async_work": { 2024-04-17T06:12:59.813 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "periodic_async_work", 2024-04-17T06:12:59.813 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.813 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.813 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.814 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.814 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.814 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.814 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.814 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Periodically check for async work", 2024-04-17T06:12:59.814 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.814 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.814 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.814 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.814 INFO:teuthology.orchestra.run.smithi047.stdout: "snapshot_clone_delay": { 2024-04-17T06:12:59.814 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "snapshot_clone_delay", 2024-04-17T06:12:59.814 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:12:59.815 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.815 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.815 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-04-17T06:12:59.815 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.815 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.815 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.815 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-17T06:12:59.815 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.815 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.815 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.815 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.815 INFO:teuthology.orchestra.run.smithi047.stdout: "snapshot_clone_no_wait": { 2024-04-17T06:12:59.815 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "snapshot_clone_no_wait", 2024-04-17T06:12:59.816 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.816 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.816 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.816 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:12:59.816 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.816 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.816 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.816 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-17T06:12:59.816 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.816 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.816 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.816 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:12:59.816 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:12:59.817 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.817 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:12:59.817 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "zabbix", 2024-04-17T06:12:59.817 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-17T06:12:59.817 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-17T06:12:59.817 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-17T06:12:59.817 INFO:teuthology.orchestra.run.smithi047.stdout: "discovery_interval": { 2024-04-17T06:12:59.817 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "discovery_interval", 2024-04-17T06:12:59.817 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "uint", 2024-04-17T06:12:59.817 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.817 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.818 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "100", 2024-04-17T06:12:59.818 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.818 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.818 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.818 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.818 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.818 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.818 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.818 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.818 INFO:teuthology.orchestra.run.smithi047.stdout: "identifier": { 2024-04-17T06:12:59.818 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "identifier", 2024-04-17T06:12:59.819 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.819 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.819 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.819 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:12:59.819 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.819 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.819 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.819 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.819 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.819 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.819 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.819 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.819 INFO:teuthology.orchestra.run.smithi047.stdout: "interval": { 2024-04-17T06:12:59.820 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "interval", 2024-04-17T06:12:59.820 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-17T06:12:59.820 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.820 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.820 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "60", 2024-04-17T06:12:59.820 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.820 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.820 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.820 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.820 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.820 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.820 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.821 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.821 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-17T06:12:59.821 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-17T06:12:59.821 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.821 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.821 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.821 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:12:59.821 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.821 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.821 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:12:59.821 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:12:59.821 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:12:59.821 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:12:59.822 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:12:59.822 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:12:59.822 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:12:59.822 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:12:59.822 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.822 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.822 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.822 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.822 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.822 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-17T06:12:59.822 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-17T06:12:59.822 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.823 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.823 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.823 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.823 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.823 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.823 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.823 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.823 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.823 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.823 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.823 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.824 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-17T06:12:59.824 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-17T06:12:59.824 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.824 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.824 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.824 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-17T06:12:59.824 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.824 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.824 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:12:59.824 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:12:59.824 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:12:59.825 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:12:59.825 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:12:59.825 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:12:59.825 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:12:59.825 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:12:59.825 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.825 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.825 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.825 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.825 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.825 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-17T06:12:59.825 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-17T06:12:59.825 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.825 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.826 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.826 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.826 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.826 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.826 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.826 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.826 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.826 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.826 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.826 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.826 INFO:teuthology.orchestra.run.smithi047.stdout: "zabbix_host": { 2024-04-17T06:12:59.827 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "zabbix_host", 2024-04-17T06:12:59.827 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.827 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.827 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.827 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:12:59.827 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.827 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.827 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.827 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.827 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.827 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.827 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.828 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.828 INFO:teuthology.orchestra.run.smithi047.stdout: "zabbix_port": { 2024-04-17T06:12:59.828 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "zabbix_port", 2024-04-17T06:12:59.828 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:12:59.828 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.828 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.828 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10051", 2024-04-17T06:12:59.828 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.828 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.828 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.828 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.828 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.829 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.829 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.829 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.829 INFO:teuthology.orchestra.run.smithi047.stdout: "zabbix_sender": { 2024-04-17T06:12:59.829 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "zabbix_sender", 2024-04-17T06:12:59.829 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.829 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.829 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.829 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-17T06:12:59.829 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.829 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.829 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.829 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.830 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.830 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.830 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.830 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:12:59.830 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:12:59.830 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:12:59.830 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-17T06:12:59.830 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.830 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:12:59.830 INFO:teuthology.orchestra.run.smithi047.stdout: "gid": 24917, 2024-04-17T06:12:59.830 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "y", 2024-04-17T06:12:59.830 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr_features": 4540701547738038271, 2024-04-17T06:12:59.830 INFO:teuthology.orchestra.run.smithi047.stdout: "available_modules": [ 2024-04-17T06:12:59.831 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:12:59.831 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "alerts", 2024-04-17T06:12:59.831 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-17T06:12:59.831 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-17T06:12:59.831 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-17T06:12:59.831 INFO:teuthology.orchestra.run.smithi047.stdout: "interval": { 2024-04-17T06:12:59.831 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "interval", 2024-04-17T06:12:59.831 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-17T06:12:59.831 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.831 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.831 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "60", 2024-04-17T06:12:59.831 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.831 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.831 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.832 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "How frequently to reexamine health status", 2024-04-17T06:12:59.832 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.832 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.832 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.832 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.832 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-17T06:12:59.832 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-17T06:12:59.832 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.832 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.832 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.832 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:12:59.832 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.833 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.833 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:12:59.833 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:12:59.833 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:12:59.833 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:12:59.833 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:12:59.833 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:12:59.833 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:12:59.833 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:12:59.833 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.833 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.833 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.833 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.833 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.834 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-17T06:12:59.834 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-17T06:12:59.834 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.834 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.834 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.834 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.834 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.834 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.834 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.834 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.834 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.834 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.834 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.834 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.835 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-17T06:12:59.835 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-17T06:12:59.835 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.835 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.835 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.835 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-17T06:12:59.835 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.835 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.835 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:12:59.835 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:12:59.835 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:12:59.835 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:12:59.835 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:12:59.835 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:12:59.836 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:12:59.836 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:12:59.836 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.836 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.836 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.836 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.836 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.836 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-17T06:12:59.836 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-17T06:12:59.836 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.836 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.836 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.836 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.837 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.837 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.837 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.837 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.837 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.837 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.837 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.837 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.837 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_destination": { 2024-04-17T06:12:59.837 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_destination", 2024-04-17T06:12:59.837 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.837 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.838 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.838 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:12:59.838 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.838 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.838 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.838 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Email address to send alerts to", 2024-04-17T06:12:59.838 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.838 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.838 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.838 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.838 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_from_name": { 2024-04-17T06:12:59.838 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_from_name", 2024-04-17T06:12:59.839 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.839 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.839 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.839 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "Ceph", 2024-04-17T06:12:59.839 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.839 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.839 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.839 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Email From: name", 2024-04-17T06:12:59.839 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.839 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.839 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.839 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.839 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_host": { 2024-04-17T06:12:59.839 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_host", 2024-04-17T06:12:59.840 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.840 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.840 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.840 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:12:59.840 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.840 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.840 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.840 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "SMTP server", 2024-04-17T06:12:59.840 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.840 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.840 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.840 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.840 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_password": { 2024-04-17T06:12:59.840 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_password", 2024-04-17T06:12:59.841 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.841 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.841 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.841 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:12:59.841 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.841 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.841 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.841 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Password to authenticate with", 2024-04-17T06:12:59.841 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.841 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.841 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.841 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.841 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_port": { 2024-04-17T06:12:59.841 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_port", 2024-04-17T06:12:59.842 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:12:59.842 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.842 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.842 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "465", 2024-04-17T06:12:59.842 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.842 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.842 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.842 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "SMTP port", 2024-04-17T06:12:59.842 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.842 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.842 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.843 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.843 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_sender": { 2024-04-17T06:12:59.843 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_sender", 2024-04-17T06:12:59.843 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.843 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.843 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.843 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:12:59.843 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.843 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.843 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.843 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "SMTP envelope sender", 2024-04-17T06:12:59.843 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.844 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.844 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.844 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.844 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_ssl": { 2024-04-17T06:12:59.844 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_ssl", 2024-04-17T06:12:59.844 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.844 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.844 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.844 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:12:59.844 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.844 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.844 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.844 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-17T06:12:59.845 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.845 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.845 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.845 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.845 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_user": { 2024-04-17T06:12:59.845 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_user", 2024-04-17T06:12:59.845 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.845 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.845 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.845 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:12:59.845 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.845 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.845 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.846 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "User to authenticate as", 2024-04-17T06:12:59.846 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.846 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.846 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.846 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:12:59.846 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:12:59.846 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.846 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:12:59.846 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "balancer", 2024-04-17T06:12:59.846 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-17T06:12:59.846 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-17T06:12:59.846 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-17T06:12:59.846 INFO:teuthology.orchestra.run.smithi047.stdout: "active": { 2024-04-17T06:12:59.847 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "active", 2024-04-17T06:12:59.847 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.847 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.847 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.847 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:12:59.847 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.847 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.847 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.847 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "automatically balance PGs across cluster", 2024-04-17T06:12:59.847 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.847 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.847 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.847 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.848 INFO:teuthology.orchestra.run.smithi047.stdout: "begin_time": { 2024-04-17T06:12:59.848 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "begin_time", 2024-04-17T06:12:59.848 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.848 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.848 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.848 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0000", 2024-04-17T06:12:59.848 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.848 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.848 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.848 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "beginning time of day to automatically balance", 2024-04-17T06:12:59.848 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-17T06:12:59.848 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.848 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.849 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.849 INFO:teuthology.orchestra.run.smithi047.stdout: "begin_weekday": { 2024-04-17T06:12:59.849 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "begin_weekday", 2024-04-17T06:12:59.849 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "uint", 2024-04-17T06:12:59.849 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.849 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.849 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-04-17T06:12:59.849 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "0", 2024-04-17T06:12:59.849 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "6", 2024-04-17T06:12:59.849 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.849 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-17T06:12:59.849 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-17T06:12:59.849 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.850 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.850 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.850 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_compat_max_iterations": { 2024-04-17T06:12:59.850 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "crush_compat_max_iterations", 2024-04-17T06:12:59.850 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "uint", 2024-04-17T06:12:59.850 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.850 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.850 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "25", 2024-04-17T06:12:59.850 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "1", 2024-04-17T06:12:59.850 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "250", 2024-04-17T06:12:59.850 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.850 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-17T06:12:59.851 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.851 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.851 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.851 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.851 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_compat_metrics": { 2024-04-17T06:12:59.851 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "crush_compat_metrics", 2024-04-17T06:12:59.851 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.851 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.851 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.851 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "pgs,objects,bytes", 2024-04-17T06:12:59.851 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.851 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.851 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.851 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-17T06:12:59.852 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-17T06:12:59.852 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.852 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.852 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.852 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_compat_step": { 2024-04-17T06:12:59.852 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "crush_compat_step", 2024-04-17T06:12:59.852 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "float", 2024-04-17T06:12:59.852 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.852 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.852 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0.5", 2024-04-17T06:12:59.852 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "0.001", 2024-04-17T06:12:59.852 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "0.999", 2024-04-17T06:12:59.852 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.852 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "aggressiveness of optimization", 2024-04-17T06:12:59.853 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-17T06:12:59.853 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.853 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.853 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.853 INFO:teuthology.orchestra.run.smithi047.stdout: "end_time": { 2024-04-17T06:12:59.853 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "end_time", 2024-04-17T06:12:59.853 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.853 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.853 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.853 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "2359", 2024-04-17T06:12:59.853 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.853 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.854 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.854 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "ending time of day to automatically balance", 2024-04-17T06:12:59.854 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-17T06:12:59.854 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.854 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.854 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.854 INFO:teuthology.orchestra.run.smithi047.stdout: "end_weekday": { 2024-04-17T06:12:59.854 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "end_weekday", 2024-04-17T06:12:59.854 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "uint", 2024-04-17T06:12:59.854 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.854 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.854 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-04-17T06:12:59.855 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "0", 2024-04-17T06:12:59.855 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "6", 2024-04-17T06:12:59.855 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.855 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-17T06:12:59.855 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-17T06:12:59.855 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.855 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.855 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.855 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-17T06:12:59.855 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-17T06:12:59.855 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.855 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.855 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.856 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:12:59.856 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.856 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.856 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:12:59.856 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:12:59.856 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:12:59.856 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:12:59.856 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:12:59.856 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:12:59.856 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:12:59.856 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:12:59.856 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.857 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.857 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.857 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.857 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.857 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-17T06:12:59.857 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-17T06:12:59.857 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.857 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.857 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.857 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.857 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.857 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.858 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.858 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.858 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.858 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.858 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.858 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.858 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-17T06:12:59.858 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-17T06:12:59.858 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.858 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.858 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.858 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-17T06:12:59.859 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.859 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.859 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:12:59.859 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:12:59.859 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:12:59.859 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:12:59.859 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:12:59.859 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:12:59.859 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:12:59.859 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:12:59.859 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.859 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.860 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.860 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.860 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.860 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-17T06:12:59.860 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-17T06:12:59.860 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.860 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.860 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.860 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.860 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.860 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.860 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.860 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.861 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.861 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.861 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.861 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.861 INFO:teuthology.orchestra.run.smithi047.stdout: "min_score": { 2024-04-17T06:12:59.861 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "min_score", 2024-04-17T06:12:59.861 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "float", 2024-04-17T06:12:59.861 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.861 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.861 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-04-17T06:12:59.861 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.861 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.861 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.861 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-17T06:12:59.862 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.862 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.862 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.862 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.862 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": { 2024-04-17T06:12:59.862 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "mode", 2024-04-17T06:12:59.862 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.862 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.862 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.862 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "upmap", 2024-04-17T06:12:59.862 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.862 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.862 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:12:59.862 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-compat", 2024-04-17T06:12:59.863 INFO:teuthology.orchestra.run.smithi047.stdout: "none", 2024-04-17T06:12:59.863 INFO:teuthology.orchestra.run.smithi047.stdout: "read", 2024-04-17T06:12:59.863 INFO:teuthology.orchestra.run.smithi047.stdout: "upmap", 2024-04-17T06:12:59.863 INFO:teuthology.orchestra.run.smithi047.stdout: "upmap-read" 2024-04-17T06:12:59.863 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:12:59.863 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Balancer mode", 2024-04-17T06:12:59.863 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.863 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.863 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.863 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.863 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_ids": { 2024-04-17T06:12:59.863 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "pool_ids", 2024-04-17T06:12:59.863 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.863 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.864 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.864 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:12:59.864 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.864 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.864 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.864 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-17T06:12:59.864 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.864 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.864 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.864 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.864 INFO:teuthology.orchestra.run.smithi047.stdout: "sleep_interval": { 2024-04-17T06:12:59.864 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sleep_interval", 2024-04-17T06:12:59.864 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-17T06:12:59.864 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.865 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.865 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "60", 2024-04-17T06:12:59.865 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.865 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.865 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.865 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-17T06:12:59.865 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.865 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.865 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.865 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.865 INFO:teuthology.orchestra.run.smithi047.stdout: "upmap_max_deviation": { 2024-04-17T06:12:59.865 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "upmap_max_deviation", 2024-04-17T06:12:59.865 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:12:59.865 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.866 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.866 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "5", 2024-04-17T06:12:59.866 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "1", 2024-04-17T06:12:59.866 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.866 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.866 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-17T06:12:59.866 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-17T06:12:59.866 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.866 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.866 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.866 INFO:teuthology.orchestra.run.smithi047.stdout: "upmap_max_optimizations": { 2024-04-17T06:12:59.866 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "upmap_max_optimizations", 2024-04-17T06:12:59.866 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "uint", 2024-04-17T06:12:59.867 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.867 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.867 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-04-17T06:12:59.867 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.867 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.867 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.867 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-17T06:12:59.867 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.867 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.867 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.867 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:12:59.867 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:12:59.867 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.868 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:12:59.868 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "cephadm", 2024-04-17T06:12:59.868 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-17T06:12:59.868 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-17T06:12:59.868 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-17T06:12:59.868 INFO:teuthology.orchestra.run.smithi047.stdout: "agent_down_multiplier": { 2024-04-17T06:12:59.868 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "agent_down_multiplier", 2024-04-17T06:12:59.868 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "float", 2024-04-17T06:12:59.868 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.868 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.868 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "3.0", 2024-04-17T06:12:59.868 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.868 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.868 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.869 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-17T06:12:59.869 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.869 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.869 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.869 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.869 INFO:teuthology.orchestra.run.smithi047.stdout: "agent_refresh_rate": { 2024-04-17T06:12:59.869 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "agent_refresh_rate", 2024-04-17T06:12:59.869 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-17T06:12:59.869 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.869 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.869 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "20", 2024-04-17T06:12:59.870 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.870 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.870 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.870 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-17T06:12:59.870 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.870 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.870 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.870 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.870 INFO:teuthology.orchestra.run.smithi047.stdout: "agent_starting_port": { 2024-04-17T06:12:59.870 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "agent_starting_port", 2024-04-17T06:12:59.870 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:12:59.870 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.870 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.871 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "4721", 2024-04-17T06:12:59.871 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.871 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.871 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.871 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-17T06:12:59.871 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.871 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.871 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.871 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.871 INFO:teuthology.orchestra.run.smithi047.stdout: "allow_ptrace": { 2024-04-17T06:12:59.871 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "allow_ptrace", 2024-04-17T06:12:59.871 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.872 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.872 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.872 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.872 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.872 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.872 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.872 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-17T06:12:59.872 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-17T06:12:59.872 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.872 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.872 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.872 INFO:teuthology.orchestra.run.smithi047.stdout: "autotune_interval": { 2024-04-17T06:12:59.872 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "autotune_interval", 2024-04-17T06:12:59.872 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-17T06:12:59.873 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.873 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.873 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "600", 2024-04-17T06:12:59.873 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.873 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.873 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.873 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-17T06:12:59.873 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.873 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.873 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.873 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.873 INFO:teuthology.orchestra.run.smithi047.stdout: "autotune_memory_target_ratio": { 2024-04-17T06:12:59.873 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "autotune_memory_target_ratio", 2024-04-17T06:12:59.873 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "float", 2024-04-17T06:12:59.874 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.874 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.874 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0.7", 2024-04-17T06:12:59.874 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.874 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.874 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.874 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-17T06:12:59.874 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.874 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.874 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.874 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.874 INFO:teuthology.orchestra.run.smithi047.stdout: "cephadm_log_destination": { 2024-04-17T06:12:59.874 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "cephadm_log_destination", 2024-04-17T06:12:59.874 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.875 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.875 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.875 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:12:59.875 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.875 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.875 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:12:59.875 INFO:teuthology.orchestra.run.smithi047.stdout: "file", 2024-04-17T06:12:59.875 INFO:teuthology.orchestra.run.smithi047.stdout: "file,syslog", 2024-04-17T06:12:59.875 INFO:teuthology.orchestra.run.smithi047.stdout: "syslog" 2024-04-17T06:12:59.875 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:12:59.875 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-17T06:12:59.875 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.876 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.876 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.876 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.876 INFO:teuthology.orchestra.run.smithi047.stdout: "cgroups_split": { 2024-04-17T06:12:59.876 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "cgroups_split", 2024-04-17T06:12:59.876 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.876 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.876 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.876 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:12:59.876 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.876 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.876 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.876 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-17T06:12:59.876 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.877 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.877 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.877 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.877 INFO:teuthology.orchestra.run.smithi047.stdout: "config_checks_enabled": { 2024-04-17T06:12:59.877 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "config_checks_enabled", 2024-04-17T06:12:59.877 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.877 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.877 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.877 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.877 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.877 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.877 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.877 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-17T06:12:59.877 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.878 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.878 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.878 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.878 INFO:teuthology.orchestra.run.smithi047.stdout: "config_dashboard": { 2024-04-17T06:12:59.878 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "config_dashboard", 2024-04-17T06:12:59.878 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.878 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.878 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.878 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:12:59.878 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.878 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.878 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.878 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-17T06:12:59.879 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.879 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.879 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.879 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.879 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_alertmanager": { 2024-04-17T06:12:59.879 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_alertmanager", 2024-04-17T06:12:59.879 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.879 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.879 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.879 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-17T06:12:59.879 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.879 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.879 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.880 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Prometheus container image", 2024-04-17T06:12:59.880 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.880 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.880 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.880 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.880 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_base": { 2024-04-17T06:12:59.880 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_base", 2024-04-17T06:12:59.880 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.880 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.880 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.880 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-17T06:12:59.880 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.880 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.880 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.881 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Container image name, without the tag", 2024-04-17T06:12:59.881 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.881 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.881 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.881 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.881 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_elasticsearch": { 2024-04-17T06:12:59.881 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_elasticsearch", 2024-04-17T06:12:59.881 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.881 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.881 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.881 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-17T06:12:59.881 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.881 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.882 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.882 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "elasticsearch container image", 2024-04-17T06:12:59.882 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.882 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.882 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.882 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.882 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_grafana": { 2024-04-17T06:12:59.882 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_grafana", 2024-04-17T06:12:59.882 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.882 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.882 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.882 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-17T06:12:59.882 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.883 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.883 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.883 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Prometheus container image", 2024-04-17T06:12:59.883 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.883 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.883 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.883 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.883 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_haproxy": { 2024-04-17T06:12:59.883 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_haproxy", 2024-04-17T06:12:59.883 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.883 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.883 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.883 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-17T06:12:59.883 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.884 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.884 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.884 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "HAproxy container image", 2024-04-17T06:12:59.884 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.884 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.884 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.884 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.884 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_jaeger_agent": { 2024-04-17T06:12:59.884 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_jaeger_agent", 2024-04-17T06:12:59.884 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.884 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.884 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.884 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-17T06:12:59.884 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.885 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.885 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.885 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Jaeger agent container image", 2024-04-17T06:12:59.885 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.885 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.885 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.885 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.885 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_jaeger_collector": { 2024-04-17T06:12:59.885 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_jaeger_collector", 2024-04-17T06:12:59.885 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.885 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.885 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.885 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-17T06:12:59.885 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.886 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.886 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.886 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Jaeger collector container image", 2024-04-17T06:12:59.886 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.886 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.886 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.886 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.886 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_jaeger_query": { 2024-04-17T06:12:59.886 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_jaeger_query", 2024-04-17T06:12:59.886 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.886 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.886 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.886 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-17T06:12:59.887 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.887 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.887 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.887 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Jaeger query container image", 2024-04-17T06:12:59.887 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.887 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.887 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.887 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.887 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_keepalived": { 2024-04-17T06:12:59.887 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_keepalived", 2024-04-17T06:12:59.887 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.887 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.888 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.888 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-17T06:12:59.888 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.888 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.888 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.888 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Keepalived container image", 2024-04-17T06:12:59.888 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.888 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.888 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.888 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.888 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_loki": { 2024-04-17T06:12:59.889 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_loki", 2024-04-17T06:12:59.889 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.889 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.889 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.889 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-17T06:12:59.889 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.889 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.889 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.889 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Loki container image", 2024-04-17T06:12:59.889 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.889 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.889 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.889 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.890 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_node_exporter": { 2024-04-17T06:12:59.890 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_node_exporter", 2024-04-17T06:12:59.890 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.890 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.890 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.890 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-17T06:12:59.890 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.890 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.890 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.890 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Prometheus container image", 2024-04-17T06:12:59.890 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.890 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.890 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.891 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.891 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_nvmeof": { 2024-04-17T06:12:59.891 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_nvmeof", 2024-04-17T06:12:59.891 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.891 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.891 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.891 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-17T06:12:59.891 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.891 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.891 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.891 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Nvme-of container image", 2024-04-17T06:12:59.891 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.891 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.891 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.892 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.892 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_prometheus": { 2024-04-17T06:12:59.892 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_prometheus", 2024-04-17T06:12:59.892 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.892 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.892 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.892 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-17T06:12:59.892 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.892 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.892 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.892 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Prometheus container image", 2024-04-17T06:12:59.892 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.892 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.892 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.892 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.893 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_promtail": { 2024-04-17T06:12:59.893 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_promtail", 2024-04-17T06:12:59.893 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.893 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.893 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.893 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-17T06:12:59.893 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.893 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.893 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.893 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Promtail container image", 2024-04-17T06:12:59.893 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.893 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.893 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.894 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.894 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_samba": { 2024-04-17T06:12:59.894 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_samba", 2024-04-17T06:12:59.894 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.894 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.894 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.894 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-17T06:12:59.894 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.894 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.894 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.894 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Samba/SMB container image", 2024-04-17T06:12:59.894 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.895 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.895 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.895 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.895 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_snmp_gateway": { 2024-04-17T06:12:59.895 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_snmp_gateway", 2024-04-17T06:12:59.895 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.895 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.895 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.895 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-17T06:12:59.895 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.895 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.895 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.895 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "SNMP Gateway container image", 2024-04-17T06:12:59.896 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.896 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.896 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.896 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.896 INFO:teuthology.orchestra.run.smithi047.stdout: "container_init": { 2024-04-17T06:12:59.896 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_init", 2024-04-17T06:12:59.896 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.896 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.896 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.896 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:12:59.896 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.896 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.896 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.897 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Run podman/docker with `--init`", 2024-04-17T06:12:59.897 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.897 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.897 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.897 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.897 INFO:teuthology.orchestra.run.smithi047.stdout: "daemon_cache_timeout": { 2024-04-17T06:12:59.897 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "daemon_cache_timeout", 2024-04-17T06:12:59.897 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-17T06:12:59.897 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.897 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.897 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "600", 2024-04-17T06:12:59.897 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.898 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.898 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.898 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-17T06:12:59.898 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.898 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.898 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.898 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.898 INFO:teuthology.orchestra.run.smithi047.stdout: "default_cephadm_command_timeout": { 2024-04-17T06:12:59.898 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "default_cephadm_command_timeout", 2024-04-17T06:12:59.898 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:12:59.898 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.898 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.899 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "900", 2024-04-17T06:12:59.899 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.899 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.899 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.899 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-17T06:12:59.899 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.899 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.899 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.899 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.899 INFO:teuthology.orchestra.run.smithi047.stdout: "default_registry": { 2024-04-17T06:12:59.899 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "default_registry", 2024-04-17T06:12:59.900 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.900 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.900 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.900 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "docker.io", 2024-04-17T06:12:59.900 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.900 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.900 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.900 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-17T06:12:59.900 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.900 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.900 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.900 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.901 INFO:teuthology.orchestra.run.smithi047.stdout: "device_cache_timeout": { 2024-04-17T06:12:59.901 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "device_cache_timeout", 2024-04-17T06:12:59.901 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-17T06:12:59.901 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.901 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.901 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "1800", 2024-04-17T06:12:59.901 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.901 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.901 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.901 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "seconds to cache device inventory", 2024-04-17T06:12:59.901 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.901 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.902 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.902 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.902 INFO:teuthology.orchestra.run.smithi047.stdout: "device_enhanced_scan": { 2024-04-17T06:12:59.902 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "device_enhanced_scan", 2024-04-17T06:12:59.902 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.902 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.902 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.902 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.902 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.902 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.902 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.902 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-17T06:12:59.903 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.903 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.903 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.903 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.903 INFO:teuthology.orchestra.run.smithi047.stdout: "facts_cache_timeout": { 2024-04-17T06:12:59.903 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "facts_cache_timeout", 2024-04-17T06:12:59.903 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-17T06:12:59.903 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.903 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.903 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "60", 2024-04-17T06:12:59.903 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.903 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.904 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.904 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "seconds to cache host facts data", 2024-04-17T06:12:59.904 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.904 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.904 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.904 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.904 INFO:teuthology.orchestra.run.smithi047.stdout: "grafana_dashboards_path": { 2024-04-17T06:12:59.904 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "grafana_dashboards_path", 2024-04-17T06:12:59.904 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.904 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.904 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.904 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-17T06:12:59.905 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.905 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.905 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.905 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-17T06:12:59.905 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.905 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.905 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.905 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.905 INFO:teuthology.orchestra.run.smithi047.stdout: "host_check_interval": { 2024-04-17T06:12:59.905 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "host_check_interval", 2024-04-17T06:12:59.905 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-17T06:12:59.905 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.906 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.906 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "600", 2024-04-17T06:12:59.906 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.906 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.906 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.906 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how frequently to perform a host check", 2024-04-17T06:12:59.906 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.906 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.906 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.906 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.906 INFO:teuthology.orchestra.run.smithi047.stdout: "hw_monitoring": { 2024-04-17T06:12:59.907 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "hw_monitoring", 2024-04-17T06:12:59.907 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.907 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.907 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.907 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.907 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.907 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.907 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.907 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-17T06:12:59.907 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.907 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.907 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.908 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.908 INFO:teuthology.orchestra.run.smithi047.stdout: "inventory_list_all": { 2024-04-17T06:12:59.908 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "inventory_list_all", 2024-04-17T06:12:59.908 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.908 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.908 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.908 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.908 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.908 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.908 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.908 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-17T06:12:59.909 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.909 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.909 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.909 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.909 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-17T06:12:59.909 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-17T06:12:59.909 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.909 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.909 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.909 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:12:59.909 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.909 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.910 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:12:59.910 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:12:59.910 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:12:59.910 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:12:59.910 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:12:59.910 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:12:59.910 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:12:59.910 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:12:59.910 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.910 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.910 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.910 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.911 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.911 INFO:teuthology.orchestra.run.smithi047.stdout: "log_refresh_metadata": { 2024-04-17T06:12:59.911 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_refresh_metadata", 2024-04-17T06:12:59.911 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.911 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.911 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.911 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.911 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.911 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.911 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.911 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-17T06:12:59.911 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.912 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.912 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.912 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.912 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-17T06:12:59.912 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-17T06:12:59.912 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.912 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.912 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.912 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:12:59.912 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.912 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.912 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.913 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-17T06:12:59.913 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.913 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.913 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.913 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.913 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-17T06:12:59.913 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-17T06:12:59.913 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.913 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.913 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.913 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-17T06:12:59.913 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.914 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.914 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:12:59.914 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:12:59.914 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:12:59.914 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:12:59.914 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:12:59.914 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:12:59.914 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:12:59.914 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:12:59.914 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.914 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.914 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.915 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.915 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.915 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-17T06:12:59.915 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-17T06:12:59.915 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.915 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.915 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.915 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.915 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.915 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.915 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.916 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.916 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.916 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.916 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.916 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.916 INFO:teuthology.orchestra.run.smithi047.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-17T06:12:59.916 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-17T06:12:59.916 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.916 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.916 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.916 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.916 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.917 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.917 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.917 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-17T06:12:59.917 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.917 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.917 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.917 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.917 INFO:teuthology.orchestra.run.smithi047.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-17T06:12:59.917 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-17T06:12:59.917 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.917 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.917 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.918 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "*", 2024-04-17T06:12:59.918 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.918 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.918 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.918 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-17T06:12:59.918 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.918 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.918 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.918 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.918 INFO:teuthology.orchestra.run.smithi047.stdout: "max_count_per_host": { 2024-04-17T06:12:59.918 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "max_count_per_host", 2024-04-17T06:12:59.919 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:12:59.919 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.919 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.919 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-04-17T06:12:59.919 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.919 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.919 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.919 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "max number of daemons per service per host", 2024-04-17T06:12:59.919 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.919 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.919 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.919 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.920 INFO:teuthology.orchestra.run.smithi047.stdout: "max_osd_draining_count": { 2024-04-17T06:12:59.920 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "max_osd_draining_count", 2024-04-17T06:12:59.920 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:12:59.920 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.920 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.920 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-04-17T06:12:59.920 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.920 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.920 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.920 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-17T06:12:59.920 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.920 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.920 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.921 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.921 INFO:teuthology.orchestra.run.smithi047.stdout: "migration_current": { 2024-04-17T06:12:59.921 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "migration_current", 2024-04-17T06:12:59.921 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:12:59.921 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.921 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.921 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:12:59.921 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.921 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.921 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.921 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "internal - do not modify", 2024-04-17T06:12:59.922 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.922 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.922 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.922 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.922 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": { 2024-04-17T06:12:59.922 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "mode", 2024-04-17T06:12:59.922 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.922 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.922 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.922 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "root", 2024-04-17T06:12:59.922 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.922 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.923 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:12:59.923 INFO:teuthology.orchestra.run.smithi047.stdout: "cephadm-package", 2024-04-17T06:12:59.923 INFO:teuthology.orchestra.run.smithi047.stdout: "root" 2024-04-17T06:12:59.923 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:12:59.923 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "mode for remote execution of cephadm", 2024-04-17T06:12:59.923 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.923 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.923 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.923 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.923 INFO:teuthology.orchestra.run.smithi047.stdout: "oob_default_addr": { 2024-04-17T06:12:59.923 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "oob_default_addr", 2024-04-17T06:12:59.923 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.924 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.924 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.924 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "169.254.1.1", 2024-04-17T06:12:59.924 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.924 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.924 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.924 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-17T06:12:59.924 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.924 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.924 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.924 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.924 INFO:teuthology.orchestra.run.smithi047.stdout: "prometheus_alerts_path": { 2024-04-17T06:12:59.924 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "prometheus_alerts_path", 2024-04-17T06:12:59.925 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.925 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.925 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.925 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-17T06:12:59.925 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.925 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.925 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.925 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-17T06:12:59.925 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.925 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.925 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.925 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.926 INFO:teuthology.orchestra.run.smithi047.stdout: "registry_insecure": { 2024-04-17T06:12:59.926 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "registry_insecure", 2024-04-17T06:12:59.926 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.926 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.926 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.926 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.926 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.926 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.926 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.926 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-17T06:12:59.926 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.927 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.927 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.927 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.927 INFO:teuthology.orchestra.run.smithi047.stdout: "registry_password": { 2024-04-17T06:12:59.927 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "registry_password", 2024-04-17T06:12:59.927 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.927 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.927 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.927 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:12:59.927 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.927 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.927 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.928 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-17T06:12:59.928 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.928 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.928 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.928 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.928 INFO:teuthology.orchestra.run.smithi047.stdout: "registry_url": { 2024-04-17T06:12:59.928 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "registry_url", 2024-04-17T06:12:59.928 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.928 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.928 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.928 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:12:59.929 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.929 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.929 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.929 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-17T06:12:59.929 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.929 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.929 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.929 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.929 INFO:teuthology.orchestra.run.smithi047.stdout: "registry_username": { 2024-04-17T06:12:59.929 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "registry_username", 2024-04-17T06:12:59.929 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.929 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.929 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.930 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:12:59.930 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.930 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.930 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.930 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-17T06:12:59.930 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.930 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.930 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.930 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.930 INFO:teuthology.orchestra.run.smithi047.stdout: "secure_monitoring_stack": { 2024-04-17T06:12:59.930 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "secure_monitoring_stack", 2024-04-17T06:12:59.931 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.931 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.931 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.931 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.931 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.931 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.931 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.931 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-17T06:12:59.931 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.931 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.931 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.931 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.931 INFO:teuthology.orchestra.run.smithi047.stdout: "service_discovery_port": { 2024-04-17T06:12:59.932 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "service_discovery_port", 2024-04-17T06:12:59.932 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:12:59.932 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.932 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.932 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "8765", 2024-04-17T06:12:59.932 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.932 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.932 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.932 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "cephadm service discovery port", 2024-04-17T06:12:59.932 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.932 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.932 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.933 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.933 INFO:teuthology.orchestra.run.smithi047.stdout: "ssh_config_file": { 2024-04-17T06:12:59.933 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ssh_config_file", 2024-04-17T06:12:59.933 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.933 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.933 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.933 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:12:59.933 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.933 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.933 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.933 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-17T06:12:59.933 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.934 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.934 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.934 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.934 INFO:teuthology.orchestra.run.smithi047.stdout: "use_agent": { 2024-04-17T06:12:59.934 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "use_agent", 2024-04-17T06:12:59.934 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.934 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.934 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.934 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.934 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.934 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.934 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.935 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-17T06:12:59.935 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.935 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.935 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.935 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.935 INFO:teuthology.orchestra.run.smithi047.stdout: "use_repo_digest": { 2024-04-17T06:12:59.935 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "use_repo_digest", 2024-04-17T06:12:59.935 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.935 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.935 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.935 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:12:59.935 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.935 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.936 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.936 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-17T06:12:59.936 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.936 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.936 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.936 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.936 INFO:teuthology.orchestra.run.smithi047.stdout: "warn_on_failed_host_check": { 2024-04-17T06:12:59.936 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "warn_on_failed_host_check", 2024-04-17T06:12:59.936 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.936 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.936 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.937 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:12:59.937 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.937 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.937 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.937 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "raise a health warning if the host check fails", 2024-04-17T06:12:59.937 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.937 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.937 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.937 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.937 INFO:teuthology.orchestra.run.smithi047.stdout: "warn_on_stray_daemons": { 2024-04-17T06:12:59.937 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "warn_on_stray_daemons", 2024-04-17T06:12:59.937 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.937 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.938 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.938 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:12:59.938 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.938 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.938 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.938 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-17T06:12:59.938 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.938 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.938 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.938 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.939 INFO:teuthology.orchestra.run.smithi047.stdout: "warn_on_stray_hosts": { 2024-04-17T06:12:59.939 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "warn_on_stray_hosts", 2024-04-17T06:12:59.939 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.939 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.939 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.939 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:12:59.939 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.939 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.939 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.939 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-17T06:12:59.939 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.939 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.939 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.940 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:12:59.940 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:12:59.940 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.940 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:12:59.940 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "crash", 2024-04-17T06:12:59.940 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-17T06:12:59.940 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-17T06:12:59.940 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-17T06:12:59.940 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-17T06:12:59.940 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-17T06:12:59.940 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.940 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.940 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.940 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:12:59.941 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.941 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.941 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:12:59.941 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:12:59.941 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:12:59.941 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:12:59.941 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:12:59.941 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:12:59.941 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:12:59.941 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:12:59.941 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.941 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.941 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.941 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.942 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.942 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-17T06:12:59.942 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-17T06:12:59.942 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.942 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.942 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.942 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.942 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.942 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.942 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.942 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.942 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.943 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.943 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.943 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.943 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-17T06:12:59.943 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-17T06:12:59.943 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.943 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.943 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.943 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-17T06:12:59.943 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.943 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.943 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:12:59.943 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:12:59.943 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:12:59.944 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:12:59.944 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:12:59.944 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:12:59.944 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:12:59.944 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:12:59.944 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.944 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.944 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.944 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.944 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.944 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-17T06:12:59.944 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-17T06:12:59.944 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.944 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.945 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.945 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.945 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.945 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.945 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.945 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.945 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.945 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.945 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.945 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.945 INFO:teuthology.orchestra.run.smithi047.stdout: "retain_interval": { 2024-04-17T06:12:59.945 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "retain_interval", 2024-04-17T06:12:59.945 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-17T06:12:59.945 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.946 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.946 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "31536000", 2024-04-17T06:12:59.946 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.946 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.946 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.946 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-17T06:12:59.946 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.946 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.946 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.946 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.946 INFO:teuthology.orchestra.run.smithi047.stdout: "warn_recent_interval": { 2024-04-17T06:12:59.946 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "warn_recent_interval", 2024-04-17T06:12:59.946 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-17T06:12:59.947 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.947 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:12:59.947 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "1209600", 2024-04-17T06:12:59.947 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.947 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.947 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.947 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-17T06:12:59.947 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.947 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.947 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.947 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:12:59.947 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:12:59.948 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.948 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:12:59.948 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "dashboard", 2024-04-17T06:12:59.948 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-17T06:12:59.948 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-17T06:12:59.948 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-17T06:12:59.948 INFO:teuthology.orchestra.run.smithi047.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-17T06:12:59.948 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-17T06:12:59.948 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:12:59.948 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.948 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.948 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-04-17T06:12:59.948 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.948 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.949 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.949 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.949 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.949 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.949 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.949 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.949 INFO:teuthology.orchestra.run.smithi047.stdout: "ALERTMANAGER_API_HOST": { 2024-04-17T06:12:59.949 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-17T06:12:59.949 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.949 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.949 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.949 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:12:59.949 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.950 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.950 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.950 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.950 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.950 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.950 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.950 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.950 INFO:teuthology.orchestra.run.smithi047.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-17T06:12:59.950 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-17T06:12:59.950 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.950 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.950 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.950 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:12:59.951 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.951 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.951 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.951 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.951 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.951 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.951 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.951 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.951 INFO:teuthology.orchestra.run.smithi047.stdout: "AUDIT_API_ENABLED": { 2024-04-17T06:12:59.951 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "AUDIT_API_ENABLED", 2024-04-17T06:12:59.951 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.951 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.952 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.952 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.952 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.952 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.952 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.952 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.952 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.952 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.952 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.952 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.952 INFO:teuthology.orchestra.run.smithi047.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-17T06:12:59.952 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-17T06:12:59.952 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.953 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.953 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.953 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:12:59.953 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.953 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.953 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.953 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.953 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.953 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.953 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.953 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.953 INFO:teuthology.orchestra.run.smithi047.stdout: "ENABLE_BROWSABLE_API": { 2024-04-17T06:12:59.953 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-17T06:12:59.954 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.954 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.954 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.954 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:12:59.954 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.954 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.954 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.954 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.954 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.954 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.954 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.954 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.954 INFO:teuthology.orchestra.run.smithi047.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-17T06:12:59.955 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-17T06:12:59.955 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.955 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.955 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.955 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:12:59.955 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.955 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.955 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.955 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.955 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.955 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.955 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.955 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.955 INFO:teuthology.orchestra.run.smithi047.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-17T06:12:59.956 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-17T06:12:59.956 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.956 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.956 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.956 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:12:59.956 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.956 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.956 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.956 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.956 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.956 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.956 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.956 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.956 INFO:teuthology.orchestra.run.smithi047.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-17T06:12:59.957 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-17T06:12:59.957 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.957 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.957 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.957 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:12:59.957 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.957 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.957 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.957 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.957 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.957 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.957 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.958 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.958 INFO:teuthology.orchestra.run.smithi047.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-17T06:12:59.958 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-17T06:12:59.958 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.958 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.958 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.958 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:12:59.958 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.958 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.958 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.958 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.959 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.959 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.959 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.959 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.959 INFO:teuthology.orchestra.run.smithi047.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-17T06:12:59.959 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-17T06:12:59.959 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.959 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.959 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.959 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:12:59.959 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.959 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.959 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.959 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.960 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.960 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.960 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.960 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.960 INFO:teuthology.orchestra.run.smithi047.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-17T06:12:59.960 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-17T06:12:59.960 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.960 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.960 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.960 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:12:59.960 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.960 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.960 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.960 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.961 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.961 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.961 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.961 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.961 INFO:teuthology.orchestra.run.smithi047.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-17T06:12:59.961 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-17T06:12:59.961 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.961 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.961 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.961 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:12:59.961 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.961 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.961 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.961 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.962 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.962 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.962 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.962 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.962 INFO:teuthology.orchestra.run.smithi047.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-17T06:12:59.962 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-17T06:12:59.962 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.962 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.962 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.962 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:12:59.962 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.962 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.962 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.963 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.963 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.963 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.963 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.963 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.963 INFO:teuthology.orchestra.run.smithi047.stdout: "GRAFANA_API_PASSWORD": { 2024-04-17T06:12:59.963 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-17T06:12:59.963 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.963 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.963 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.963 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "admin", 2024-04-17T06:12:59.963 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.963 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.963 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.964 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.964 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.964 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.964 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.964 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.964 INFO:teuthology.orchestra.run.smithi047.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-17T06:12:59.964 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-17T06:12:59.964 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.964 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.964 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.964 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:12:59.964 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.964 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.964 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.965 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.965 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.965 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.965 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.965 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.965 INFO:teuthology.orchestra.run.smithi047.stdout: "GRAFANA_API_URL": { 2024-04-17T06:12:59.965 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "GRAFANA_API_URL", 2024-04-17T06:12:59.965 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.965 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.965 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.965 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:12:59.965 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.965 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.965 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.966 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.966 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.966 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.966 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.966 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.966 INFO:teuthology.orchestra.run.smithi047.stdout: "GRAFANA_API_USERNAME": { 2024-04-17T06:12:59.966 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-17T06:12:59.966 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.966 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.966 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.966 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "admin", 2024-04-17T06:12:59.966 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.966 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.967 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.967 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.967 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.967 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.967 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.967 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.967 INFO:teuthology.orchestra.run.smithi047.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-17T06:12:59.967 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-17T06:12:59.967 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.967 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.967 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.967 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:12:59.968 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.968 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.968 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.968 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.968 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.968 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.968 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.968 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.968 INFO:teuthology.orchestra.run.smithi047.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-17T06:12:59.968 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-17T06:12:59.968 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.968 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.969 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.969 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.969 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.969 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.969 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.969 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.969 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.969 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.969 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.969 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.969 INFO:teuthology.orchestra.run.smithi047.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-17T06:12:59.970 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-17T06:12:59.970 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.970 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.970 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.970 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:12:59.970 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.970 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.970 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.970 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.970 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.970 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.970 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.970 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.971 INFO:teuthology.orchestra.run.smithi047.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-17T06:12:59.971 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-17T06:12:59.971 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.971 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.971 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.971 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:12:59.971 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.971 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.971 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.971 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.971 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.972 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.972 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.972 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.972 INFO:teuthology.orchestra.run.smithi047.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-17T06:12:59.972 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-17T06:12:59.972 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.972 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.972 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.972 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "[]", 2024-04-17T06:12:59.972 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.972 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.972 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.973 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.973 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.973 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.973 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.973 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.973 INFO:teuthology.orchestra.run.smithi047.stdout: "MULTICLUSTER_CONFIG": { 2024-04-17T06:12:59.973 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-17T06:12:59.973 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.973 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.973 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.973 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "{}", 2024-04-17T06:12:59.973 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.974 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.974 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.974 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.974 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.974 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.974 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.974 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.974 INFO:teuthology.orchestra.run.smithi047.stdout: "PROMETHEUS_API_HOST": { 2024-04-17T06:12:59.974 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-17T06:12:59.974 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.974 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.974 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.975 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:12:59.975 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.975 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.975 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.975 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.975 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.975 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.975 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.975 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.975 INFO:teuthology.orchestra.run.smithi047.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-17T06:12:59.975 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-17T06:12:59.975 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.976 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.976 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.976 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:12:59.976 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.976 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.976 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.976 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.976 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.976 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.976 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.976 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.976 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-17T06:12:59.977 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-17T06:12:59.977 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.977 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.977 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.977 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.977 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.977 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.977 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.977 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.977 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.977 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.977 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.978 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.978 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-17T06:12:59.978 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-17T06:12:59.978 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.978 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.978 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.978 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.978 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.978 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.978 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.978 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.978 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.979 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.979 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.979 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.979 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-17T06:12:59.979 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-17T06:12:59.979 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.979 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.979 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.979 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:12:59.979 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.979 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.980 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.980 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.980 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.980 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.980 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.980 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.980 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-17T06:12:59.980 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-17T06:12:59.980 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.980 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.980 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.980 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:12:59.981 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.981 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.981 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.981 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.981 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.981 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.981 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.981 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.981 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-17T06:12:59.981 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-17T06:12:59.981 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.981 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.982 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.982 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.982 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.982 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.982 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.982 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.982 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.982 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.982 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.982 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.982 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-17T06:12:59.983 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-17T06:12:59.983 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.983 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.983 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.983 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.983 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.983 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.983 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.983 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.983 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.983 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.983 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.984 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.984 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-17T06:12:59.984 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-17T06:12:59.984 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.984 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.984 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.984 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.984 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.984 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.984 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.984 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.984 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.984 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.985 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.985 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.985 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_ENABLED": { 2024-04-17T06:12:59.985 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-17T06:12:59.985 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.985 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.985 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.985 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:12:59.985 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.985 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.985 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.985 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.985 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.986 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.986 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.986 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.986 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-17T06:12:59.986 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-17T06:12:59.986 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.986 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.986 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.986 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-17T06:12:59.986 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.986 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.986 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.987 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.987 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.987 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.987 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.987 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.987 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-17T06:12:59.987 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-17T06:12:59.987 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:12:59.987 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.987 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.987 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-04-17T06:12:59.987 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.987 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.988 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.988 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.988 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.988 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.988 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.988 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.988 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-17T06:12:59.988 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-17T06:12:59.988 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:12:59.988 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.988 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.988 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "8", 2024-04-17T06:12:59.988 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.989 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.989 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.989 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.989 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.989 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.989 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.989 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.989 INFO:teuthology.orchestra.run.smithi047.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-17T06:12:59.989 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-17T06:12:59.989 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:12:59.989 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.989 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.990 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "45", 2024-04-17T06:12:59.990 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.990 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.990 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.990 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.990 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.990 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.990 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.990 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.990 INFO:teuthology.orchestra.run.smithi047.stdout: "RGW_API_ACCESS_KEY": { 2024-04-17T06:12:59.990 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-17T06:12:59.990 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.990 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.991 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.991 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:12:59.991 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.991 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.991 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.991 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.991 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.991 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.991 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.991 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.991 INFO:teuthology.orchestra.run.smithi047.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-17T06:12:59.991 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-17T06:12:59.991 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.992 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.992 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.992 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "admin", 2024-04-17T06:12:59.992 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.992 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.992 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.992 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.992 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.992 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.992 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.992 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.992 INFO:teuthology.orchestra.run.smithi047.stdout: "RGW_API_SECRET_KEY": { 2024-04-17T06:12:59.993 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-17T06:12:59.993 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.993 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.993 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.993 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:12:59.993 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.993 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.993 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.993 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.993 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.993 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.993 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.993 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.994 INFO:teuthology.orchestra.run.smithi047.stdout: "RGW_API_SSL_VERIFY": { 2024-04-17T06:12:59.994 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-17T06:12:59.994 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.994 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.994 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.994 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:12:59.994 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.994 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.994 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.994 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.994 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.994 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.995 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.995 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.995 INFO:teuthology.orchestra.run.smithi047.stdout: "UNSAFE_TLS_v1_2": { 2024-04-17T06:12:59.995 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-17T06:12:59.995 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:12:59.995 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.995 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.995 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:12:59.995 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.995 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.995 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.995 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.995 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.996 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.996 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.996 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.996 INFO:teuthology.orchestra.run.smithi047.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-17T06:12:59.996 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-17T06:12:59.996 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:12:59.996 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.996 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.996 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-04-17T06:12:59.996 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.996 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.996 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.997 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.997 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.997 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.997 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.997 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.997 INFO:teuthology.orchestra.run.smithi047.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-17T06:12:59.997 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-17T06:12:59.997 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:12:59.997 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.997 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.997 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-04-17T06:12:59.997 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.997 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.998 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.998 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.998 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.998 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.998 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.998 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.998 INFO:teuthology.orchestra.run.smithi047.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-17T06:12:59.998 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-17T06:12:59.998 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:12:59.998 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:12:59.998 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:12:59.999 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "5", 2024-04-17T06:12:59.999 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:12:59.999 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:12:59.999 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:12:59.999 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:12:59.999 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:12:59.999 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:12:59.999 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:12:59.999 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:12:59.999 INFO:teuthology.orchestra.run.smithi047.stdout: "cross_origin_url": { 2024-04-17T06:12:59.999 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "cross_origin_url", 2024-04-17T06:12:59.999 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:12:59.999 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.000 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.000 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.000 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.000 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.000 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.000 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.000 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.000 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.000 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.000 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.000 INFO:teuthology.orchestra.run.smithi047.stdout: "crt_file": { 2024-04-17T06:13:00.000 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "crt_file", 2024-04-17T06:13:00.000 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.001 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.001 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.001 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.001 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.001 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.001 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.001 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.001 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.001 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.001 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.001 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.001 INFO:teuthology.orchestra.run.smithi047.stdout: "debug": { 2024-04-17T06:13:00.001 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "debug", 2024-04-17T06:13:00.002 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.002 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.002 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.002 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.002 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.002 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.002 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.002 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Enable/disable debug options", 2024-04-17T06:13:00.002 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.002 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.002 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.002 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.003 INFO:teuthology.orchestra.run.smithi047.stdout: "jwt_token_ttl": { 2024-04-17T06:13:00.003 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "jwt_token_ttl", 2024-04-17T06:13:00.003 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:13:00.003 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.003 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.003 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "28800", 2024-04-17T06:13:00.003 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.003 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.003 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.003 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.003 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.003 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.003 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.004 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.004 INFO:teuthology.orchestra.run.smithi047.stdout: "key_file": { 2024-04-17T06:13:00.004 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "key_file", 2024-04-17T06:13:00.004 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.004 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.004 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.004 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.004 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.004 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.004 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.004 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.004 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.004 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.005 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.005 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.005 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-17T06:13:00.005 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-17T06:13:00.005 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.005 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.005 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.005 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.005 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.005 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.005 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.005 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.006 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.006 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.006 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.006 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.006 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.006 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.006 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.006 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.006 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.006 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.006 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.006 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-17T06:13:00.007 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-17T06:13:00.007 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.007 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.007 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.007 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.007 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.007 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.007 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.007 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.007 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.007 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.007 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.008 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.008 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-17T06:13:00.008 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-17T06:13:00.008 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.008 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.008 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.008 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-17T06:13:00.008 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.008 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.008 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.008 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.008 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.008 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.009 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.009 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.009 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.009 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.009 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.009 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.009 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.009 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.009 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.009 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-17T06:13:00.009 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-17T06:13:00.009 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.009 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.010 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.010 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.010 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.010 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.010 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.010 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.010 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.010 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.010 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.010 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.010 INFO:teuthology.orchestra.run.smithi047.stdout: "motd": { 2024-04-17T06:13:00.010 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "motd", 2024-04-17T06:13:00.011 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.011 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.011 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.011 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.011 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.011 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.011 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.011 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "The message of the day", 2024-04-17T06:13:00.011 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.011 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.011 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.012 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.012 INFO:teuthology.orchestra.run.smithi047.stdout: "redirect_resolve_ip_addr": { 2024-04-17T06:13:00.012 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "redirect_resolve_ip_addr", 2024-04-17T06:13:00.012 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.012 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.012 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.012 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.012 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.012 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.012 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.012 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.012 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.013 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.013 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.013 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.013 INFO:teuthology.orchestra.run.smithi047.stdout: "server_addr": { 2024-04-17T06:13:00.013 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "server_addr", 2024-04-17T06:13:00.013 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.013 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.013 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.013 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "::", 2024-04-17T06:13:00.013 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.013 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.013 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.014 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.014 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.014 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.014 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.014 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.014 INFO:teuthology.orchestra.run.smithi047.stdout: "server_port": { 2024-04-17T06:13:00.014 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "server_port", 2024-04-17T06:13:00.014 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:13:00.014 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.014 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.014 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "8080", 2024-04-17T06:13:00.015 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.015 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.015 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.015 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.015 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.015 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.015 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.015 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.015 INFO:teuthology.orchestra.run.smithi047.stdout: "ssl": { 2024-04-17T06:13:00.015 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ssl", 2024-04-17T06:13:00.015 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.015 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.016 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.016 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:13:00.016 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.016 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.016 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.016 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.016 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.016 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.016 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.016 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.016 INFO:teuthology.orchestra.run.smithi047.stdout: "ssl_server_port": { 2024-04-17T06:13:00.016 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ssl_server_port", 2024-04-17T06:13:00.016 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:13:00.017 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.017 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.017 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "8443", 2024-04-17T06:13:00.017 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.017 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.017 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.017 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.017 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.017 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.017 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.017 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.017 INFO:teuthology.orchestra.run.smithi047.stdout: "standby_behaviour": { 2024-04-17T06:13:00.018 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "standby_behaviour", 2024-04-17T06:13:00.018 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.018 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.018 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.018 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "redirect", 2024-04-17T06:13:00.018 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.018 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.018 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.018 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.018 INFO:teuthology.orchestra.run.smithi047.stdout: "redirect" 2024-04-17T06:13:00.018 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.018 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.019 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.019 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.019 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.019 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.019 INFO:teuthology.orchestra.run.smithi047.stdout: "standby_error_status_code": { 2024-04-17T06:13:00.019 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "standby_error_status_code", 2024-04-17T06:13:00.019 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:13:00.019 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.019 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.019 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "500", 2024-04-17T06:13:00.019 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "400", 2024-04-17T06:13:00.019 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "599", 2024-04-17T06:13:00.019 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.020 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.020 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.020 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.020 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.020 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.020 INFO:teuthology.orchestra.run.smithi047.stdout: "url_prefix": { 2024-04-17T06:13:00.020 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "url_prefix", 2024-04-17T06:13:00.020 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.020 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.020 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.020 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.020 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.021 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.021 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.021 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.021 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.021 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.021 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.021 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.021 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.021 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.021 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:13:00.021 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "devicehealth", 2024-04-17T06:13:00.021 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-17T06:13:00.021 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-17T06:13:00.021 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-17T06:13:00.022 INFO:teuthology.orchestra.run.smithi047.stdout: "enable_monitoring": { 2024-04-17T06:13:00.022 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "enable_monitoring", 2024-04-17T06:13:00.022 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.022 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.022 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.022 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:13:00.022 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.022 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.022 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.022 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "monitor device health metrics", 2024-04-17T06:13:00.022 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.023 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.023 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.023 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.023 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-17T06:13:00.023 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-17T06:13:00.023 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.023 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.023 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.023 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.023 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.023 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.023 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.023 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.024 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.024 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.024 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.024 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.024 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.024 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.024 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.024 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.024 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.024 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.024 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.024 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-17T06:13:00.024 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-17T06:13:00.025 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.025 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.025 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.025 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.025 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.025 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.025 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.025 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.025 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.025 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.025 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.026 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.026 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-17T06:13:00.026 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-17T06:13:00.026 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.026 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.026 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.026 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-17T06:13:00.026 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.026 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.026 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.026 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.026 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.026 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.027 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.027 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.027 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.027 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.027 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.027 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.027 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.027 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.027 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.027 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-17T06:13:00.027 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-17T06:13:00.028 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.028 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.028 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.028 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.028 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.028 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.028 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.028 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.028 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.028 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.028 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.028 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.029 INFO:teuthology.orchestra.run.smithi047.stdout: "mark_out_threshold": { 2024-04-17T06:13:00.029 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "mark_out_threshold", 2024-04-17T06:13:00.029 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-17T06:13:00.029 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.029 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.029 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "2419200", 2024-04-17T06:13:00.029 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.029 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.029 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.029 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-17T06:13:00.029 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.029 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.030 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.030 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.030 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_name": { 2024-04-17T06:13:00.030 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "pool_name", 2024-04-17T06:13:00.030 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.030 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.030 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.030 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "device_health_metrics", 2024-04-17T06:13:00.030 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.030 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.030 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.031 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-17T06:13:00.031 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.031 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.031 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.031 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.031 INFO:teuthology.orchestra.run.smithi047.stdout: "retention_period": { 2024-04-17T06:13:00.031 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "retention_period", 2024-04-17T06:13:00.031 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-17T06:13:00.031 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.031 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.031 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "15552000", 2024-04-17T06:13:00.031 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.032 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.032 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.032 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how long to retain device health metrics", 2024-04-17T06:13:00.032 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.032 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.032 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.032 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.032 INFO:teuthology.orchestra.run.smithi047.stdout: "scrape_frequency": { 2024-04-17T06:13:00.032 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "scrape_frequency", 2024-04-17T06:13:00.032 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-17T06:13:00.032 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.032 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.032 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "86400", 2024-04-17T06:13:00.033 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.033 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.033 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.033 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-17T06:13:00.033 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.033 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.033 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.033 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.033 INFO:teuthology.orchestra.run.smithi047.stdout: "self_heal": { 2024-04-17T06:13:00.033 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "self_heal", 2024-04-17T06:13:00.033 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.033 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.034 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.034 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:13:00.034 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.034 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.034 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.034 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-17T06:13:00.034 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.034 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.034 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.034 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.034 INFO:teuthology.orchestra.run.smithi047.stdout: "sleep_interval": { 2024-04-17T06:13:00.034 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sleep_interval", 2024-04-17T06:13:00.035 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-17T06:13:00.035 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.035 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.035 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "600", 2024-04-17T06:13:00.035 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.035 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.035 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.035 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how frequently to wake up and check device health", 2024-04-17T06:13:00.035 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.035 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.035 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.035 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.036 INFO:teuthology.orchestra.run.smithi047.stdout: "warn_threshold": { 2024-04-17T06:13:00.036 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "warn_threshold", 2024-04-17T06:13:00.036 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-17T06:13:00.036 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.036 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.036 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "7257600", 2024-04-17T06:13:00.036 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.036 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.036 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.036 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-17T06:13:00.036 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.037 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.037 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.037 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.037 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.037 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.037 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:13:00.037 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "diskprediction_local", 2024-04-17T06:13:00.037 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-17T06:13:00.037 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-17T06:13:00.037 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-17T06:13:00.037 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-17T06:13:00.038 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-17T06:13:00.038 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.038 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.038 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.038 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.038 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.038 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.038 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.038 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.038 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.038 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.039 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.039 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.039 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.039 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.039 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.039 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.039 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.039 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.039 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.039 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-17T06:13:00.039 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-17T06:13:00.039 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.040 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.040 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.040 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.040 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.040 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.040 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.040 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.040 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.040 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.040 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.040 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.040 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-17T06:13:00.041 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-17T06:13:00.041 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.041 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.041 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.041 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-17T06:13:00.041 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.041 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.041 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.041 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.041 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.041 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.041 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.041 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.042 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.042 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.042 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.042 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.042 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.042 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.042 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.042 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-17T06:13:00.042 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-17T06:13:00.042 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.042 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.043 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.043 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.043 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.043 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.043 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.043 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.043 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.043 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.043 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.043 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.043 INFO:teuthology.orchestra.run.smithi047.stdout: "predict_interval": { 2024-04-17T06:13:00.043 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "predict_interval", 2024-04-17T06:13:00.043 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.044 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.044 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.044 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "86400", 2024-04-17T06:13:00.044 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.044 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.044 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.044 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.044 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.044 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.044 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.044 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.044 INFO:teuthology.orchestra.run.smithi047.stdout: "predictor_model": { 2024-04-17T06:13:00.045 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "predictor_model", 2024-04-17T06:13:00.045 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.045 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.045 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.045 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "prophetstor", 2024-04-17T06:13:00.045 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.045 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.045 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.045 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.045 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.045 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.045 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.046 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.046 INFO:teuthology.orchestra.run.smithi047.stdout: "sleep_interval": { 2024-04-17T06:13:00.046 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sleep_interval", 2024-04-17T06:13:00.046 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.046 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.046 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.046 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "600", 2024-04-17T06:13:00.046 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.046 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.046 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.046 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.046 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.046 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.047 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.047 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.047 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.047 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.047 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:13:00.047 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "influx", 2024-04-17T06:13:00.047 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": false, 2024-04-17T06:13:00.047 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "influxdb python module not found", 2024-04-17T06:13:00.047 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-17T06:13:00.047 INFO:teuthology.orchestra.run.smithi047.stdout: "batch_size": { 2024-04-17T06:13:00.047 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "batch_size", 2024-04-17T06:13:00.047 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:13:00.048 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.048 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.048 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "5000", 2024-04-17T06:13:00.048 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.048 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.048 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.048 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-17T06:13:00.048 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.048 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.048 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.048 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.048 INFO:teuthology.orchestra.run.smithi047.stdout: "database": { 2024-04-17T06:13:00.048 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "database", 2024-04-17T06:13:00.049 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.049 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.049 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.049 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "ceph", 2024-04-17T06:13:00.049 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.049 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.049 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.049 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-17T06:13:00.049 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.049 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.049 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.049 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.050 INFO:teuthology.orchestra.run.smithi047.stdout: "hostname": { 2024-04-17T06:13:00.050 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "hostname", 2024-04-17T06:13:00.050 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.050 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.050 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.050 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.050 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.050 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.050 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.050 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "InfluxDB server hostname", 2024-04-17T06:13:00.050 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.051 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.051 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.051 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.051 INFO:teuthology.orchestra.run.smithi047.stdout: "interval": { 2024-04-17T06:13:00.051 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "interval", 2024-04-17T06:13:00.051 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-17T06:13:00.051 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.051 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.051 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "30", 2024-04-17T06:13:00.051 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "5", 2024-04-17T06:13:00.051 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.052 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.052 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-17T06:13:00.052 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.052 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.052 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.052 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.052 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-17T06:13:00.052 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-17T06:13:00.052 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.052 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.052 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.052 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.052 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.053 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.053 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.053 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.053 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.053 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.053 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.053 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.053 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.053 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.053 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.053 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.053 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.054 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.054 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.054 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-17T06:13:00.054 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-17T06:13:00.054 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.054 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.054 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.054 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.054 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.054 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.054 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.054 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.054 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.055 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.055 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.055 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.055 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-17T06:13:00.055 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-17T06:13:00.055 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.055 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.055 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.055 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-17T06:13:00.055 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.055 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.055 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.055 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.056 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.056 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.056 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.056 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.056 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.056 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.056 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.056 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.056 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.056 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.056 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.056 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-17T06:13:00.056 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-17T06:13:00.056 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.057 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.057 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.057 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.057 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.057 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.057 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.057 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.057 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.057 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.057 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.057 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.057 INFO:teuthology.orchestra.run.smithi047.stdout: "password": { 2024-04-17T06:13:00.057 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "password", 2024-04-17T06:13:00.058 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.058 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.058 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.058 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.058 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.058 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.058 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.058 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "password of InfluxDB server user", 2024-04-17T06:13:00.058 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.058 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.058 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.058 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.059 INFO:teuthology.orchestra.run.smithi047.stdout: "port": { 2024-04-17T06:13:00.059 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "port", 2024-04-17T06:13:00.059 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:13:00.059 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.059 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.059 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "8086", 2024-04-17T06:13:00.059 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.059 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.059 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.059 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "InfluxDB server port", 2024-04-17T06:13:00.059 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.059 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.060 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.060 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.060 INFO:teuthology.orchestra.run.smithi047.stdout: "ssl": { 2024-04-17T06:13:00.060 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ssl", 2024-04-17T06:13:00.060 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.060 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.060 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.060 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "false", 2024-04-17T06:13:00.060 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.060 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.060 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.060 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-17T06:13:00.060 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.060 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.061 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.061 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.061 INFO:teuthology.orchestra.run.smithi047.stdout: "threads": { 2024-04-17T06:13:00.061 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "threads", 2024-04-17T06:13:00.061 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:13:00.061 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.061 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.061 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "5", 2024-04-17T06:13:00.061 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "1", 2024-04-17T06:13:00.061 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "32", 2024-04-17T06:13:00.061 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.061 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-17T06:13:00.061 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.062 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.062 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.062 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.062 INFO:teuthology.orchestra.run.smithi047.stdout: "username": { 2024-04-17T06:13:00.062 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "username", 2024-04-17T06:13:00.062 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.062 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.062 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.062 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.062 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.062 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.062 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.063 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "username of InfluxDB server user", 2024-04-17T06:13:00.063 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.063 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.063 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.063 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.063 INFO:teuthology.orchestra.run.smithi047.stdout: "verify_ssl": { 2024-04-17T06:13:00.063 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "verify_ssl", 2024-04-17T06:13:00.063 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.063 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.063 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.063 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "true", 2024-04-17T06:13:00.063 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.064 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.064 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.064 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-17T06:13:00.064 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.064 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.064 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.064 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.064 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.064 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.064 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:13:00.064 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "insights", 2024-04-17T06:13:00.064 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-17T06:13:00.064 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-17T06:13:00.064 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-17T06:13:00.065 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-17T06:13:00.065 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-17T06:13:00.065 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.065 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.065 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.065 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.065 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.065 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.065 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.065 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.065 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.066 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.066 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.066 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.066 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.066 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.066 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.066 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.066 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.066 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.066 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.066 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-17T06:13:00.066 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-17T06:13:00.067 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.067 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.067 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.067 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.067 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.067 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.067 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.067 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.067 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.067 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.067 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.067 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.067 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-17T06:13:00.067 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-17T06:13:00.068 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.068 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.068 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.068 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-17T06:13:00.068 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.068 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.068 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.068 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.068 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.068 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.068 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.068 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.069 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.069 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.069 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.069 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.069 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.069 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.069 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.069 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-17T06:13:00.069 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-17T06:13:00.069 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.069 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.069 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.069 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.069 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.070 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.070 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.070 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.070 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.070 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.070 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.070 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.070 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.070 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.070 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:13:00.070 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "iostat", 2024-04-17T06:13:00.070 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-17T06:13:00.070 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-17T06:13:00.071 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-17T06:13:00.071 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-17T06:13:00.071 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-17T06:13:00.071 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.071 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.071 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.071 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.071 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.071 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.071 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.071 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.071 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.071 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.072 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.072 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.072 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.072 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.072 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.072 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.072 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.072 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.072 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.072 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-17T06:13:00.072 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-17T06:13:00.072 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.072 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.072 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.073 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.073 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.073 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.073 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.073 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.073 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.073 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.073 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.073 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.073 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-17T06:13:00.073 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-17T06:13:00.073 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.073 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.074 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.074 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-17T06:13:00.074 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.074 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.074 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.074 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.074 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.074 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.074 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.074 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.074 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.074 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.074 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.075 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.075 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.075 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.075 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.075 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-17T06:13:00.075 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-17T06:13:00.075 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.075 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.075 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.075 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.075 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.075 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.075 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.076 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.076 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.076 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.076 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.076 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.076 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.076 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.076 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:13:00.076 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "k8sevents", 2024-04-17T06:13:00.076 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-17T06:13:00.076 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-17T06:13:00.076 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-17T06:13:00.076 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_event_retention_days": { 2024-04-17T06:13:00.077 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ceph_event_retention_days", 2024-04-17T06:13:00.077 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:13:00.077 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.077 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.077 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "7", 2024-04-17T06:13:00.077 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.077 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.077 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.077 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-17T06:13:00.077 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.077 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.077 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.078 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.078 INFO:teuthology.orchestra.run.smithi047.stdout: "config_check_secs": { 2024-04-17T06:13:00.078 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "config_check_secs", 2024-04-17T06:13:00.078 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:13:00.078 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.078 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.078 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-04-17T06:13:00.078 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "10", 2024-04-17T06:13:00.078 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.078 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.078 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-17T06:13:00.078 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.079 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.079 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.079 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.079 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-17T06:13:00.079 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-17T06:13:00.079 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.079 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.079 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.079 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.079 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.079 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.079 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.079 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.079 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.080 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.080 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.080 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.080 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.080 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.080 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.080 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.080 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.080 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.080 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.080 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-17T06:13:00.080 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-17T06:13:00.080 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.081 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.081 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.081 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.081 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.081 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.081 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.081 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.081 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.081 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.081 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.081 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.081 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-17T06:13:00.081 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-17T06:13:00.082 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.082 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.082 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.082 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-17T06:13:00.082 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.082 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.082 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.082 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.082 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.082 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.082 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.082 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.082 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.083 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.083 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.083 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.083 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.083 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.083 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.083 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-17T06:13:00.083 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-17T06:13:00.083 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.083 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.083 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.083 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.083 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.083 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.084 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.084 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.084 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.084 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.084 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.084 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.084 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.084 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.084 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:13:00.084 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "localpool", 2024-04-17T06:13:00.084 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-17T06:13:00.084 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-17T06:13:00.084 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-17T06:13:00.084 INFO:teuthology.orchestra.run.smithi047.stdout: "failure_domain": { 2024-04-17T06:13:00.085 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "failure_domain", 2024-04-17T06:13:00.085 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.085 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.085 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.085 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "host", 2024-04-17T06:13:00.085 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.085 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.085 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.085 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "failure domain for any created local pool", 2024-04-17T06:13:00.085 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-17T06:13:00.085 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.085 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.085 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.085 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-17T06:13:00.086 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-17T06:13:00.086 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.086 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.086 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.086 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.086 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.086 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.086 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.086 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.086 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.086 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.086 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.086 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.087 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.087 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.087 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.087 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.087 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.087 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.087 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.087 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-17T06:13:00.087 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-17T06:13:00.088 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.088 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.088 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.088 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.088 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.088 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.088 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.088 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.088 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.088 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.088 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.088 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.089 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-17T06:13:00.089 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-17T06:13:00.089 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.089 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.089 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.089 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-17T06:13:00.089 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.089 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.089 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.089 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.089 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.089 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.089 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.089 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.090 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.090 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.090 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.090 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.090 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.090 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.090 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.090 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-17T06:13:00.090 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-17T06:13:00.090 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.090 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.090 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.091 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.091 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.091 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.091 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.091 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.091 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.091 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.091 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.091 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.091 INFO:teuthology.orchestra.run.smithi047.stdout: "min_size": { 2024-04-17T06:13:00.091 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "min_size", 2024-04-17T06:13:00.091 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:13:00.091 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.092 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.092 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.092 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.092 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.092 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.092 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "default min_size for any created local pool", 2024-04-17T06:13:00.092 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-17T06:13:00.092 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.092 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.092 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.092 INFO:teuthology.orchestra.run.smithi047.stdout: "num_rep": { 2024-04-17T06:13:00.092 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "num_rep", 2024-04-17T06:13:00.092 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:13:00.092 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.093 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.093 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "3", 2024-04-17T06:13:00.093 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.093 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.093 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.093 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "default replica count for any created local pool", 2024-04-17T06:13:00.093 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.093 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.093 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.093 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.093 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num": { 2024-04-17T06:13:00.093 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "pg_num", 2024-04-17T06:13:00.093 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:13:00.093 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.094 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.094 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "128", 2024-04-17T06:13:00.094 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.094 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.094 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.094 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "default pg_num for any created local pool", 2024-04-17T06:13:00.094 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.094 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.094 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.094 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.094 INFO:teuthology.orchestra.run.smithi047.stdout: "prefix": { 2024-04-17T06:13:00.095 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "prefix", 2024-04-17T06:13:00.095 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.095 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.095 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.095 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.095 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.095 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.095 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.095 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "name prefix for any created local pool", 2024-04-17T06:13:00.095 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.095 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.095 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.096 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.096 INFO:teuthology.orchestra.run.smithi047.stdout: "subtree": { 2024-04-17T06:13:00.096 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "subtree", 2024-04-17T06:13:00.096 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.096 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.096 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.096 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "rack", 2024-04-17T06:13:00.096 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.096 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.096 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.096 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-17T06:13:00.096 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-17T06:13:00.096 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.097 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.097 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.097 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.097 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.097 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:13:00.097 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "mds_autoscaler", 2024-04-17T06:13:00.097 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-17T06:13:00.097 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-17T06:13:00.097 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-17T06:13:00.097 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-17T06:13:00.097 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-17T06:13:00.097 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.097 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.097 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.098 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.098 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.098 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.098 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.098 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.098 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.098 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.098 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.098 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.098 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.098 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.098 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.098 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.099 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.099 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.099 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.099 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-17T06:13:00.099 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-17T06:13:00.099 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.099 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.099 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.099 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.099 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.099 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.099 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.100 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.100 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.100 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.100 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.100 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.100 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-17T06:13:00.100 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-17T06:13:00.100 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.100 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.100 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.100 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-17T06:13:00.100 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.101 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.101 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.101 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.101 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.101 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.101 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.101 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.101 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.101 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.101 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.101 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.101 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.101 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.101 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.102 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-17T06:13:00.102 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-17T06:13:00.102 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.102 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.102 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.102 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.102 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.102 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.102 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.102 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.102 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.102 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.103 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.103 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.103 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.103 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.103 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:13:00.103 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "mirroring", 2024-04-17T06:13:00.103 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-17T06:13:00.103 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-17T06:13:00.103 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-17T06:13:00.103 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-17T06:13:00.103 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-17T06:13:00.103 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.103 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.103 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.104 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.104 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.104 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.104 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.104 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.104 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.104 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.104 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.104 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.104 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.104 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.104 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.104 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.105 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.105 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.105 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.105 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-17T06:13:00.105 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-17T06:13:00.105 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.105 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.105 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.105 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.105 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.105 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.105 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.105 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.105 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.106 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.106 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.106 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.106 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-17T06:13:00.106 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-17T06:13:00.106 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.106 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.106 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.106 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-17T06:13:00.106 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.106 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.106 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.106 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.107 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.107 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.107 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.107 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.107 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.107 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.107 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.107 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.107 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.107 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.107 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.107 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-17T06:13:00.107 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-17T06:13:00.108 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.108 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.108 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.108 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.108 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.108 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.108 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.108 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.108 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.108 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.109 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.109 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.109 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.109 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.109 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:13:00.109 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "nfs", 2024-04-17T06:13:00.109 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-17T06:13:00.109 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-17T06:13:00.109 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-17T06:13:00.109 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-17T06:13:00.109 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-17T06:13:00.109 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.109 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.109 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.110 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.110 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.110 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.110 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.110 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.110 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.110 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.110 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.110 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.110 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.110 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.110 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.110 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.110 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.111 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.111 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.111 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-17T06:13:00.111 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-17T06:13:00.111 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.111 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.111 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.111 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.111 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.111 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.111 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.111 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.111 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.111 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.111 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.111 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.112 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-17T06:13:00.112 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-17T06:13:00.112 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.112 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.112 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.112 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-17T06:13:00.112 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.112 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.112 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.112 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.112 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.112 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.112 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.112 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.112 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.112 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.112 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.113 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.113 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.113 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.113 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.113 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-17T06:13:00.113 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-17T06:13:00.113 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.113 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.113 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.113 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.113 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.113 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.113 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.113 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.113 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.113 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.114 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.114 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.114 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.114 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.114 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:13:00.114 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "orchestrator", 2024-04-17T06:13:00.114 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-17T06:13:00.114 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-17T06:13:00.114 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-17T06:13:00.114 INFO:teuthology.orchestra.run.smithi047.stdout: "fail_fs": { 2024-04-17T06:13:00.114 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "fail_fs", 2024-04-17T06:13:00.114 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.114 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.114 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.114 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.115 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.115 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.115 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.115 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-17T06:13:00.115 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.115 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.115 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.115 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.115 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-17T06:13:00.115 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-17T06:13:00.115 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.115 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.115 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.115 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.115 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.115 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.116 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.116 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.116 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.116 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.116 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.116 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.116 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.116 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.116 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.116 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.116 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.116 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.116 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.116 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-17T06:13:00.116 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-17T06:13:00.116 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.117 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.117 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.117 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.117 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.117 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.117 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.117 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.117 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.117 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.117 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.117 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.117 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-17T06:13:00.117 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-17T06:13:00.118 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.118 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.118 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.118 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-17T06:13:00.118 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.118 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.118 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.118 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.118 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.118 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.118 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.118 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.118 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.118 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.119 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.119 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.119 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.119 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.119 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.119 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-17T06:13:00.119 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-17T06:13:00.119 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.119 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.119 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.119 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.119 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.119 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.119 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.119 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.119 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.120 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.120 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.120 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.120 INFO:teuthology.orchestra.run.smithi047.stdout: "orchestrator": { 2024-04-17T06:13:00.120 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "orchestrator", 2024-04-17T06:13:00.120 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.120 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.120 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.120 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.120 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.120 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.120 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.120 INFO:teuthology.orchestra.run.smithi047.stdout: "cephadm", 2024-04-17T06:13:00.120 INFO:teuthology.orchestra.run.smithi047.stdout: "rook", 2024-04-17T06:13:00.120 INFO:teuthology.orchestra.run.smithi047.stdout: "test_orchestrator" 2024-04-17T06:13:00.120 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.121 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Orchestrator backend", 2024-04-17T06:13:00.121 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.121 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.121 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.121 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.121 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.121 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.121 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:13:00.121 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "osd_perf_query", 2024-04-17T06:13:00.121 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-17T06:13:00.121 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-17T06:13:00.121 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-17T06:13:00.121 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-17T06:13:00.121 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-17T06:13:00.121 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.121 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.122 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.122 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.122 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.122 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.122 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.122 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.122 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.122 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.122 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.122 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.122 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.122 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.122 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.123 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.123 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.123 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.123 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.123 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-17T06:13:00.123 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-17T06:13:00.123 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.123 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.123 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.123 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.123 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.123 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.123 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.123 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.123 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.123 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.124 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.124 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.124 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-17T06:13:00.124 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-17T06:13:00.124 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.124 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.124 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.124 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-17T06:13:00.124 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.124 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.124 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.124 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.124 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.124 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.124 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.124 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.125 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.125 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.125 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.125 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.125 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.125 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.125 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.125 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-17T06:13:00.125 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-17T06:13:00.125 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.125 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.125 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.125 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.125 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.125 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.126 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.126 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.126 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.126 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.126 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.126 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.126 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.126 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.126 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:13:00.126 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "osd_support", 2024-04-17T06:13:00.126 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-17T06:13:00.126 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-17T06:13:00.126 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-17T06:13:00.126 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-17T06:13:00.126 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-17T06:13:00.127 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.127 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.127 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.127 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.127 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.127 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.127 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.127 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.127 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.127 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.127 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.127 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.127 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.127 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.127 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.128 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.128 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.128 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.128 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.128 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-17T06:13:00.128 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-17T06:13:00.128 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.128 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.128 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.128 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.128 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.128 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.128 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.128 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.129 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.129 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.129 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.129 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.129 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-17T06:13:00.129 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-17T06:13:00.129 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.129 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.129 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.129 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-17T06:13:00.129 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.129 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.129 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.129 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.129 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.130 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.130 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.130 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.130 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.130 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.130 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.130 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.130 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.130 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.130 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.130 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-17T06:13:00.130 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-17T06:13:00.130 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.130 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.130 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.131 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.131 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.131 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.131 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.131 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.131 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.131 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.131 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.131 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.131 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.131 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.131 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:13:00.131 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "pg_autoscaler", 2024-04-17T06:13:00.131 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-17T06:13:00.131 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-17T06:13:00.131 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-17T06:13:00.131 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-17T06:13:00.132 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-17T06:13:00.132 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.132 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.132 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.132 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.132 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.132 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.132 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.132 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.132 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.132 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.132 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.132 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.132 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.132 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.132 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.133 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.133 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.133 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.133 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.133 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-17T06:13:00.133 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-17T06:13:00.133 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.133 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.133 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.133 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.133 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.133 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.133 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.133 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.133 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.133 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.133 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.134 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.134 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-17T06:13:00.134 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-17T06:13:00.134 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.134 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.134 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.134 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-17T06:13:00.134 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.134 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.134 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.134 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.134 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.134 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.134 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.134 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.134 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.135 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.135 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.135 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.135 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.135 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.135 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.135 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-17T06:13:00.135 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-17T06:13:00.135 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.135 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.135 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.135 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.135 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.135 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.135 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.135 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.136 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.136 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.136 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.136 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.136 INFO:teuthology.orchestra.run.smithi047.stdout: "sleep_interval": { 2024-04-17T06:13:00.136 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sleep_interval", 2024-04-17T06:13:00.136 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-17T06:13:00.136 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.136 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.136 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "60", 2024-04-17T06:13:00.136 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.136 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.136 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.136 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.136 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.136 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.137 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.137 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.137 INFO:teuthology.orchestra.run.smithi047.stdout: "threshold": { 2024-04-17T06:13:00.137 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "threshold", 2024-04-17T06:13:00.137 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "float", 2024-04-17T06:13:00.137 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.137 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.137 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "3.0", 2024-04-17T06:13:00.137 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "1.0", 2024-04-17T06:13:00.137 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.137 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.137 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "scaling threshold", 2024-04-17T06:13:00.137 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-17T06:13:00.137 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.138 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.138 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.138 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.138 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.138 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:13:00.138 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "progress", 2024-04-17T06:13:00.138 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-17T06:13:00.138 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-17T06:13:00.138 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-17T06:13:00.138 INFO:teuthology.orchestra.run.smithi047.stdout: "allow_pg_recovery_event": { 2024-04-17T06:13:00.138 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "allow_pg_recovery_event", 2024-04-17T06:13:00.138 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.139 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.139 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.139 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.139 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.139 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.139 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.139 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-17T06:13:00.139 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.139 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.139 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.139 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.139 INFO:teuthology.orchestra.run.smithi047.stdout: "enabled": { 2024-04-17T06:13:00.139 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "enabled", 2024-04-17T06:13:00.139 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.139 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.139 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.140 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:13:00.140 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.140 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.140 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.140 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.140 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.140 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.140 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.140 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.140 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-17T06:13:00.140 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-17T06:13:00.140 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.140 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.140 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.140 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.140 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.141 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.141 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.141 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.141 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.141 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.141 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.141 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.141 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.141 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.141 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.141 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.141 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.141 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.141 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.141 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-17T06:13:00.141 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-17T06:13:00.142 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.142 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.142 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.142 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.142 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.142 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.142 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.142 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.142 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.142 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.142 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.142 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.142 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-17T06:13:00.142 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-17T06:13:00.142 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.142 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.143 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.143 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-17T06:13:00.143 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.143 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.143 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.143 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.143 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.143 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.143 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.143 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.143 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.143 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.143 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.143 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.143 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.143 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.144 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.144 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-17T06:13:00.144 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-17T06:13:00.144 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.144 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.144 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.144 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.144 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.144 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.144 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.144 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.144 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.144 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.144 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.144 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.144 INFO:teuthology.orchestra.run.smithi047.stdout: "max_completed_events": { 2024-04-17T06:13:00.145 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "max_completed_events", 2024-04-17T06:13:00.145 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:13:00.145 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.145 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.145 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "50", 2024-04-17T06:13:00.145 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.145 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.145 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.145 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "number of past completed events to remember", 2024-04-17T06:13:00.145 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.145 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.145 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.145 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.145 INFO:teuthology.orchestra.run.smithi047.stdout: "sleep_interval": { 2024-04-17T06:13:00.146 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sleep_interval", 2024-04-17T06:13:00.146 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-17T06:13:00.146 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.146 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.146 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "5", 2024-04-17T06:13:00.146 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.146 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.146 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.146 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how long the module is going to sleep", 2024-04-17T06:13:00.146 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.146 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.146 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.146 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.146 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.146 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.147 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:13:00.147 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "prometheus", 2024-04-17T06:13:00.147 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-17T06:13:00.147 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-17T06:13:00.147 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-17T06:13:00.147 INFO:teuthology.orchestra.run.smithi047.stdout: "cache": { 2024-04-17T06:13:00.147 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "cache", 2024-04-17T06:13:00.147 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.147 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.147 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.147 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:13:00.147 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.147 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.147 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.147 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.147 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.148 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.148 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.148 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.148 INFO:teuthology.orchestra.run.smithi047.stdout: "exclude_perf_counters": { 2024-04-17T06:13:00.148 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "exclude_perf_counters", 2024-04-17T06:13:00.148 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.148 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.148 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.148 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:13:00.148 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.148 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.148 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.148 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-17T06:13:00.148 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-17T06:13:00.148 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.149 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.149 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.149 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-17T06:13:00.149 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-17T06:13:00.149 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.149 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.149 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.149 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.149 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.149 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.149 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.149 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.149 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.149 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.149 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.149 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.149 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.150 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.150 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.150 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.150 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.150 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.150 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.150 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-17T06:13:00.150 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-17T06:13:00.150 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.150 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.150 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.150 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.150 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.150 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.150 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.150 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.151 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.151 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.151 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.151 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.151 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-17T06:13:00.151 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-17T06:13:00.151 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.151 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.151 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.151 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-17T06:13:00.151 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.151 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.151 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.151 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.151 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.152 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.152 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.152 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.152 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.152 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.152 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.152 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.152 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.152 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.152 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.152 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-17T06:13:00.152 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-17T06:13:00.152 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.152 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.152 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.152 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.153 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.153 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.153 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.153 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.153 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.153 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.153 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.153 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.153 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_stats_pools": { 2024-04-17T06:13:00.153 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rbd_stats_pools", 2024-04-17T06:13:00.153 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.153 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.153 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.153 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.153 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.153 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.154 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.154 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.154 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.154 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.154 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.154 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.154 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-17T06:13:00.154 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-17T06:13:00.154 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:13:00.154 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.154 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.154 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "300", 2024-04-17T06:13:00.154 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.154 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.154 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.155 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.155 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.155 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.155 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.155 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.155 INFO:teuthology.orchestra.run.smithi047.stdout: "scrape_interval": { 2024-04-17T06:13:00.155 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "scrape_interval", 2024-04-17T06:13:00.155 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "float", 2024-04-17T06:13:00.155 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.155 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.155 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "15.0", 2024-04-17T06:13:00.155 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.155 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.155 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.155 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.155 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.156 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.156 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.156 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.156 INFO:teuthology.orchestra.run.smithi047.stdout: "server_addr": { 2024-04-17T06:13:00.156 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "server_addr", 2024-04-17T06:13:00.156 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.156 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.156 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.156 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "::", 2024-04-17T06:13:00.156 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.156 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.156 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.156 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-17T06:13:00.156 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.156 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.156 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.157 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.157 INFO:teuthology.orchestra.run.smithi047.stdout: "server_port": { 2024-04-17T06:13:00.157 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "server_port", 2024-04-17T06:13:00.157 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:13:00.157 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.157 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.157 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "9283", 2024-04-17T06:13:00.157 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.157 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.157 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.157 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-17T06:13:00.157 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.157 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.157 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.157 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.158 INFO:teuthology.orchestra.run.smithi047.stdout: "stale_cache_strategy": { 2024-04-17T06:13:00.158 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "stale_cache_strategy", 2024-04-17T06:13:00.158 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.158 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.158 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.158 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "log", 2024-04-17T06:13:00.158 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.158 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.158 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.158 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.158 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.158 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.158 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.158 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.158 INFO:teuthology.orchestra.run.smithi047.stdout: "standby_behaviour": { 2024-04-17T06:13:00.159 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "standby_behaviour", 2024-04-17T06:13:00.159 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.159 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.159 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.159 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "default", 2024-04-17T06:13:00.159 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.159 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.159 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.159 INFO:teuthology.orchestra.run.smithi047.stdout: "default", 2024-04-17T06:13:00.159 INFO:teuthology.orchestra.run.smithi047.stdout: "error" 2024-04-17T06:13:00.159 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.159 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.159 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.159 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.159 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.159 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.160 INFO:teuthology.orchestra.run.smithi047.stdout: "standby_error_status_code": { 2024-04-17T06:13:00.160 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "standby_error_status_code", 2024-04-17T06:13:00.160 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:13:00.160 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.160 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.160 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "500", 2024-04-17T06:13:00.160 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "400", 2024-04-17T06:13:00.160 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "599", 2024-04-17T06:13:00.160 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.160 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.160 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.160 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.160 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.160 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.160 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.160 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.161 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:13:00.161 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rbd_support", 2024-04-17T06:13:00.161 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-17T06:13:00.161 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-17T06:13:00.161 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-17T06:13:00.161 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-17T06:13:00.161 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-17T06:13:00.161 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.161 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.161 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.161 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.161 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.161 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.161 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.161 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.161 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.162 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.162 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.162 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.162 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.162 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.162 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.162 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.162 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.162 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.162 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.162 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-17T06:13:00.162 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-17T06:13:00.162 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.162 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.162 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.163 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.163 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.163 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.163 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.163 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.163 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.163 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.163 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.163 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.163 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-17T06:13:00.163 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-17T06:13:00.163 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.163 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.163 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.163 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-17T06:13:00.163 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.164 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.164 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.164 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.164 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.164 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.164 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.164 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.164 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.164 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.164 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.164 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.164 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.164 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.164 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.164 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-17T06:13:00.165 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-17T06:13:00.165 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.165 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.165 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.165 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.165 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.165 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.165 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.165 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.165 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.165 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.165 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.165 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.165 INFO:teuthology.orchestra.run.smithi047.stdout: "max_concurrent_snap_create": { 2024-04-17T06:13:00.165 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "max_concurrent_snap_create", 2024-04-17T06:13:00.166 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:13:00.166 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.166 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.166 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-04-17T06:13:00.166 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.166 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.166 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.166 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.166 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.166 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.166 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.166 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.166 INFO:teuthology.orchestra.run.smithi047.stdout: "mirror_snapshot_schedule": { 2024-04-17T06:13:00.166 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "mirror_snapshot_schedule", 2024-04-17T06:13:00.166 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.167 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.167 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.167 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.167 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.167 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.167 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.167 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.167 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.167 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.167 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.167 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.167 INFO:teuthology.orchestra.run.smithi047.stdout: "trash_purge_schedule": { 2024-04-17T06:13:00.167 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "trash_purge_schedule", 2024-04-17T06:13:00.167 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.167 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.167 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.168 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.168 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.168 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.168 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.168 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.168 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.168 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.168 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.168 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.168 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.168 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.168 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:13:00.168 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "restful", 2024-04-17T06:13:00.168 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-17T06:13:00.168 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-17T06:13:00.168 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-17T06:13:00.169 INFO:teuthology.orchestra.run.smithi047.stdout: "enable_auth": { 2024-04-17T06:13:00.169 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "enable_auth", 2024-04-17T06:13:00.169 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.169 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.169 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.169 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:13:00.169 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.169 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.169 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.169 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.169 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.169 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.169 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.169 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.169 INFO:teuthology.orchestra.run.smithi047.stdout: "key_file": { 2024-04-17T06:13:00.170 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "key_file", 2024-04-17T06:13:00.170 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.170 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.170 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.170 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.170 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.170 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.170 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.170 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.170 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.170 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.170 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.170 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.170 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-17T06:13:00.170 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-17T06:13:00.171 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.171 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.171 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.171 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.171 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.171 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.171 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.171 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.171 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.171 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.171 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.171 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.171 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.171 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.171 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.172 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.172 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.172 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.172 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.172 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-17T06:13:00.172 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-17T06:13:00.172 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.172 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.172 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.172 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.172 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.172 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.172 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.172 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.172 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.172 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.173 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.173 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.173 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-17T06:13:00.173 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-17T06:13:00.173 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.173 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.173 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.173 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-17T06:13:00.173 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.173 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.173 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.173 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.173 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.173 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.173 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.173 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.174 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.174 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.174 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.174 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.174 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.174 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.174 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.174 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-17T06:13:00.174 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-17T06:13:00.174 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.174 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.174 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.174 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.174 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.174 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.175 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.175 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.175 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.175 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.175 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.175 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.175 INFO:teuthology.orchestra.run.smithi047.stdout: "server_addr": { 2024-04-17T06:13:00.175 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "server_addr", 2024-04-17T06:13:00.175 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.175 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.175 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.175 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.175 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.175 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.175 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.175 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.175 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.176 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.176 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.176 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.176 INFO:teuthology.orchestra.run.smithi047.stdout: "server_port": { 2024-04-17T06:13:00.176 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "server_port", 2024-04-17T06:13:00.176 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.176 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.176 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.176 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.176 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.176 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.176 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.176 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.176 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.176 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.176 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.176 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.177 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.177 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.177 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:13:00.177 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rgw", 2024-04-17T06:13:00.177 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-17T06:13:00.177 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-17T06:13:00.177 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-17T06:13:00.177 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-17T06:13:00.177 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-17T06:13:00.177 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.177 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.177 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.177 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.178 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.178 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.178 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.178 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.178 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.178 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.178 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.178 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.178 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.178 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.178 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.178 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.178 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.178 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.178 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.179 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-17T06:13:00.179 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-17T06:13:00.179 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.179 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.179 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.179 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.179 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.179 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.179 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.179 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.179 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.179 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.179 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.179 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.179 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-17T06:13:00.179 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-17T06:13:00.180 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.180 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.180 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.180 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-17T06:13:00.180 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.180 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.180 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.180 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.180 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.180 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.180 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.180 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.180 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.180 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.180 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.180 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.181 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.181 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.181 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.181 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-17T06:13:00.181 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-17T06:13:00.181 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.181 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.181 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.181 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.181 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.181 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.181 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.181 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.181 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.181 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.181 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.181 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.182 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.182 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.182 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:13:00.182 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rook", 2024-04-17T06:13:00.182 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-17T06:13:00.182 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-17T06:13:00.182 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-17T06:13:00.182 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-17T06:13:00.182 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-17T06:13:00.182 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.182 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.182 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.182 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.182 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.182 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.182 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.183 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.183 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.183 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.183 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.183 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.183 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.183 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.183 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.183 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.183 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.183 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.183 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.183 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-17T06:13:00.183 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-17T06:13:00.183 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.183 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.184 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.184 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.184 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.184 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.184 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.184 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.184 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.184 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.184 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.184 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.184 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-17T06:13:00.184 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-17T06:13:00.184 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.184 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.184 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.184 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-17T06:13:00.185 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.185 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.185 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.185 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.185 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.185 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.185 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.185 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.185 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.185 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.185 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.185 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.185 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.185 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.185 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.185 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-17T06:13:00.185 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-17T06:13:00.186 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.186 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.186 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.186 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.186 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.186 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.186 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.186 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.186 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.186 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.186 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.186 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.186 INFO:teuthology.orchestra.run.smithi047.stdout: "storage_class": { 2024-04-17T06:13:00.186 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "storage_class", 2024-04-17T06:13:00.186 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.186 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.187 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.187 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "local", 2024-04-17T06:13:00.187 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.187 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.187 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.187 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-17T06:13:00.187 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.187 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.187 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.187 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.187 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.187 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.187 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:13:00.187 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "selftest", 2024-04-17T06:13:00.187 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-17T06:13:00.188 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-17T06:13:00.188 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-17T06:13:00.188 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-17T06:13:00.188 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-17T06:13:00.188 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.188 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.188 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.188 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.188 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.188 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.188 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.188 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.188 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.188 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.188 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.188 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.188 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.189 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.189 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.189 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.189 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.189 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.189 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.189 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-17T06:13:00.189 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-17T06:13:00.189 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.189 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.189 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.189 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.189 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.189 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.189 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.189 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.189 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.190 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.190 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.190 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.190 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-17T06:13:00.190 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-17T06:13:00.190 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.190 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.190 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.190 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-17T06:13:00.190 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.190 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.190 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.190 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.191 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.191 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.191 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.191 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.191 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.191 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.191 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.191 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.191 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.191 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.191 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.191 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-17T06:13:00.191 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-17T06:13:00.191 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.192 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.192 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.192 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.192 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.192 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.192 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.192 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.192 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.192 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.192 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.192 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.192 INFO:teuthology.orchestra.run.smithi047.stdout: "roption1": { 2024-04-17T06:13:00.192 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "roption1", 2024-04-17T06:13:00.193 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.193 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.193 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.193 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.193 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.193 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.193 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.193 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.193 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.193 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.193 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.193 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.193 INFO:teuthology.orchestra.run.smithi047.stdout: "roption2": { 2024-04-17T06:13:00.193 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "roption2", 2024-04-17T06:13:00.193 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.193 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.194 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.194 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "xyz", 2024-04-17T06:13:00.194 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.194 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.194 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.194 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.194 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.194 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.194 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.194 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.194 INFO:teuthology.orchestra.run.smithi047.stdout: "rwoption1": { 2024-04-17T06:13:00.194 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rwoption1", 2024-04-17T06:13:00.194 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.194 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.194 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.195 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.195 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.195 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.195 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.195 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.195 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.195 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.195 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.195 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.195 INFO:teuthology.orchestra.run.smithi047.stdout: "rwoption2": { 2024-04-17T06:13:00.195 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rwoption2", 2024-04-17T06:13:00.195 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:13:00.195 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.195 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.195 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.195 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.196 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.196 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.196 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.196 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.196 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.196 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.196 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.196 INFO:teuthology.orchestra.run.smithi047.stdout: "rwoption3": { 2024-04-17T06:13:00.196 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rwoption3", 2024-04-17T06:13:00.196 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "float", 2024-04-17T06:13:00.196 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.196 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.196 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.196 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.197 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.197 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.197 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.197 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.197 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.197 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.197 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.197 INFO:teuthology.orchestra.run.smithi047.stdout: "rwoption4": { 2024-04-17T06:13:00.197 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rwoption4", 2024-04-17T06:13:00.197 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.197 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.197 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.197 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.197 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.197 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.198 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.198 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.198 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.198 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.198 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.198 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.198 INFO:teuthology.orchestra.run.smithi047.stdout: "rwoption5": { 2024-04-17T06:13:00.198 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rwoption5", 2024-04-17T06:13:00.198 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.198 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.198 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.198 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.198 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.198 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.198 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.198 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.199 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.199 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.199 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.199 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.199 INFO:teuthology.orchestra.run.smithi047.stdout: "rwoption6": { 2024-04-17T06:13:00.199 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rwoption6", 2024-04-17T06:13:00.199 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.199 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.199 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.199 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:13:00.199 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.199 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.199 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.199 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.199 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.199 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.200 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.200 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.200 INFO:teuthology.orchestra.run.smithi047.stdout: "rwoption7": { 2024-04-17T06:13:00.200 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rwoption7", 2024-04-17T06:13:00.200 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:13:00.200 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.200 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.200 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.200 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "1", 2024-04-17T06:13:00.200 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "42", 2024-04-17T06:13:00.200 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.200 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.200 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.200 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.200 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.200 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.201 INFO:teuthology.orchestra.run.smithi047.stdout: "testkey": { 2024-04-17T06:13:00.201 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "testkey", 2024-04-17T06:13:00.201 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.201 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.201 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.201 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.201 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.201 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.201 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.201 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.201 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.201 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.201 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.201 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.201 INFO:teuthology.orchestra.run.smithi047.stdout: "testlkey": { 2024-04-17T06:13:00.201 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "testlkey", 2024-04-17T06:13:00.201 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.202 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.202 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.202 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.202 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.202 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.202 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.202 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.202 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.202 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.202 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.202 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.202 INFO:teuthology.orchestra.run.smithi047.stdout: "testnewline": { 2024-04-17T06:13:00.202 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "testnewline", 2024-04-17T06:13:00.202 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.202 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.203 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.203 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.203 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.203 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.203 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.203 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.203 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.203 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.203 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.203 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.203 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.203 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.203 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:13:00.203 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "snap_schedule", 2024-04-17T06:13:00.203 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-17T06:13:00.203 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-17T06:13:00.204 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-17T06:13:00.204 INFO:teuthology.orchestra.run.smithi047.stdout: "allow_m_granularity": { 2024-04-17T06:13:00.204 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "allow_m_granularity", 2024-04-17T06:13:00.204 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.204 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.204 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.204 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.204 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.204 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.204 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.204 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "allow minute scheduled snapshots", 2024-04-17T06:13:00.204 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.204 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.204 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.204 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.204 INFO:teuthology.orchestra.run.smithi047.stdout: "dump_on_update": { 2024-04-17T06:13:00.205 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "dump_on_update", 2024-04-17T06:13:00.205 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.205 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.205 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.205 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.205 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.205 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.205 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.205 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "dump database to debug log on update", 2024-04-17T06:13:00.205 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.205 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.205 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.205 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.205 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-17T06:13:00.205 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-17T06:13:00.205 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.206 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.206 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.206 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.206 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.206 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.206 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.206 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.206 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.206 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.206 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.206 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.206 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.206 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.206 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.206 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.206 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.207 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.207 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.207 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-17T06:13:00.207 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-17T06:13:00.207 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.207 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.207 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.207 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.207 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.207 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.207 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.207 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.207 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.207 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.207 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.207 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.208 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-17T06:13:00.208 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-17T06:13:00.208 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.208 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.208 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.208 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-17T06:13:00.208 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.208 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.208 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.208 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.208 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.208 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.208 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.208 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.208 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.208 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.209 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.209 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.209 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.209 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.209 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.209 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-17T06:13:00.209 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-17T06:13:00.209 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.209 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.209 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.209 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.209 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.209 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.209 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.209 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.209 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.210 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.210 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.210 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.210 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.210 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.210 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:13:00.210 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "stats", 2024-04-17T06:13:00.210 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-17T06:13:00.210 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-17T06:13:00.210 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-17T06:13:00.210 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-17T06:13:00.210 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-17T06:13:00.210 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.210 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.210 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.211 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.211 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.211 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.211 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.211 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.211 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.211 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.211 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.211 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.211 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.211 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.211 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.211 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.211 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.211 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.211 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.212 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-17T06:13:00.212 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-17T06:13:00.212 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.212 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.212 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.212 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.212 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.212 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.212 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.212 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.212 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.212 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.212 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.212 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.212 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-17T06:13:00.212 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-17T06:13:00.212 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.213 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.213 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.213 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-17T06:13:00.213 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.213 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.213 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.213 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.213 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.213 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.213 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.213 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.213 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.213 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.213 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.213 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.213 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.213 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.214 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.214 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-17T06:13:00.214 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-17T06:13:00.214 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.214 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.214 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.214 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.214 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.214 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.214 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.214 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.214 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.214 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.214 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.214 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.214 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.215 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.215 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:13:00.215 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "status", 2024-04-17T06:13:00.215 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-17T06:13:00.215 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-17T06:13:00.215 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-17T06:13:00.215 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-17T06:13:00.215 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-17T06:13:00.215 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.215 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.215 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.215 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.215 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.215 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.215 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.216 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.216 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.216 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.216 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.216 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.216 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.216 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.216 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.216 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.216 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.216 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.216 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.216 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-17T06:13:00.216 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-17T06:13:00.216 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.216 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.217 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.217 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.217 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.217 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.217 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.217 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.217 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.217 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.217 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.217 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.217 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-17T06:13:00.217 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-17T06:13:00.217 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.217 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.217 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.217 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-17T06:13:00.218 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.218 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.218 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.218 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.218 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.218 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.218 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.218 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.218 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.218 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.218 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.218 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.218 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.218 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.218 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.219 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-17T06:13:00.219 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-17T06:13:00.219 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.219 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.219 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.219 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.219 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.219 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.219 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.219 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.219 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.219 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.219 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.219 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.219 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.219 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.220 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:13:00.220 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "telegraf", 2024-04-17T06:13:00.220 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-17T06:13:00.220 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-17T06:13:00.220 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-17T06:13:00.220 INFO:teuthology.orchestra.run.smithi047.stdout: "address": { 2024-04-17T06:13:00.220 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "address", 2024-04-17T06:13:00.220 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.220 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.220 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.220 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-17T06:13:00.220 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.220 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.220 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.220 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.220 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.221 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.221 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.221 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.221 INFO:teuthology.orchestra.run.smithi047.stdout: "interval": { 2024-04-17T06:13:00.221 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "interval", 2024-04-17T06:13:00.221 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-17T06:13:00.221 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.221 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.221 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "15", 2024-04-17T06:13:00.221 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.221 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.221 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.221 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.221 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.221 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.221 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.221 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.222 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-17T06:13:00.222 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-17T06:13:00.222 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.222 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.222 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.222 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.222 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.222 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.222 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.222 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.222 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.222 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.222 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.222 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.223 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.223 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.223 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.223 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.223 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.223 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.223 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.223 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-17T06:13:00.223 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-17T06:13:00.223 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.223 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.223 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.223 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.224 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.224 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.224 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.224 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.224 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.224 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.224 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.224 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.224 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-17T06:13:00.224 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-17T06:13:00.224 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.224 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.224 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.224 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-17T06:13:00.224 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.225 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.225 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.225 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.225 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.225 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.225 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.225 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.225 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.225 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.225 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.225 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.225 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.225 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.225 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.225 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-17T06:13:00.225 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-17T06:13:00.225 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.226 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.226 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.226 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.226 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.226 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.226 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.226 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.226 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.226 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.226 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.226 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.226 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.226 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.227 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:13:00.227 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "telemetry", 2024-04-17T06:13:00.227 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-17T06:13:00.227 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-17T06:13:00.227 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-17T06:13:00.227 INFO:teuthology.orchestra.run.smithi047.stdout: "channel_basic": { 2024-04-17T06:13:00.227 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "channel_basic", 2024-04-17T06:13:00.227 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.227 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.227 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.227 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:13:00.227 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.227 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.227 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.227 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-17T06:13:00.228 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.228 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.228 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.228 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.228 INFO:teuthology.orchestra.run.smithi047.stdout: "channel_crash": { 2024-04-17T06:13:00.228 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "channel_crash", 2024-04-17T06:13:00.228 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.228 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.228 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.228 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:13:00.228 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.228 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.228 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.229 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-17T06:13:00.229 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.229 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.229 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.229 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.229 INFO:teuthology.orchestra.run.smithi047.stdout: "channel_device": { 2024-04-17T06:13:00.229 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "channel_device", 2024-04-17T06:13:00.229 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.229 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.229 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.229 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:13:00.229 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.229 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.229 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.229 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-17T06:13:00.230 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.230 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.230 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.230 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.230 INFO:teuthology.orchestra.run.smithi047.stdout: "channel_ident": { 2024-04-17T06:13:00.230 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "channel_ident", 2024-04-17T06:13:00.230 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.230 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.230 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.230 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.230 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.230 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.230 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.230 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-17T06:13:00.230 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.231 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.231 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.231 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.231 INFO:teuthology.orchestra.run.smithi047.stdout: "channel_perf": { 2024-04-17T06:13:00.231 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "channel_perf", 2024-04-17T06:13:00.231 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.231 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.231 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.231 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.231 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.231 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.231 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.231 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-17T06:13:00.231 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.231 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.231 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.232 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.232 INFO:teuthology.orchestra.run.smithi047.stdout: "contact": { 2024-04-17T06:13:00.232 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "contact", 2024-04-17T06:13:00.232 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.232 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.232 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.232 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.232 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.232 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.232 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.232 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.232 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.232 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.232 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.232 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.232 INFO:teuthology.orchestra.run.smithi047.stdout: "description": { 2024-04-17T06:13:00.233 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "description", 2024-04-17T06:13:00.233 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.233 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.233 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.233 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.233 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.233 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.233 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.233 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.233 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.233 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.233 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.233 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.233 INFO:teuthology.orchestra.run.smithi047.stdout: "device_url": { 2024-04-17T06:13:00.233 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "device_url", 2024-04-17T06:13:00.233 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.233 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.234 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.234 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-17T06:13:00.234 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.234 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.234 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.234 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.234 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.234 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.234 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.234 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.234 INFO:teuthology.orchestra.run.smithi047.stdout: "enabled": { 2024-04-17T06:13:00.234 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "enabled", 2024-04-17T06:13:00.234 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.234 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.234 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.235 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.235 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.235 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.235 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.235 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.235 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.235 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.235 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.235 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.235 INFO:teuthology.orchestra.run.smithi047.stdout: "interval": { 2024-04-17T06:13:00.235 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "interval", 2024-04-17T06:13:00.235 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:13:00.235 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.235 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.235 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "24", 2024-04-17T06:13:00.235 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "8", 2024-04-17T06:13:00.235 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.236 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.236 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.236 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.236 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.236 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.236 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.236 INFO:teuthology.orchestra.run.smithi047.stdout: "last_opt_revision": { 2024-04-17T06:13:00.236 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "last_opt_revision", 2024-04-17T06:13:00.236 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:13:00.236 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.236 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.236 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "1", 2024-04-17T06:13:00.236 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.236 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.236 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.236 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.237 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.237 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.237 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.237 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.237 INFO:teuthology.orchestra.run.smithi047.stdout: "leaderboard": { 2024-04-17T06:13:00.237 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "leaderboard", 2024-04-17T06:13:00.237 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.237 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.237 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.237 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.237 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.237 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.237 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.237 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.237 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.237 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.238 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.238 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.238 INFO:teuthology.orchestra.run.smithi047.stdout: "leaderboard_description": { 2024-04-17T06:13:00.238 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "leaderboard_description", 2024-04-17T06:13:00.238 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.238 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.238 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.238 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.238 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.238 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.239 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.239 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.239 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.239 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.239 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.239 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.239 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-17T06:13:00.239 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-17T06:13:00.239 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.239 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.239 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.239 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.239 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.239 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.239 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.239 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.240 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.240 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.240 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.240 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.240 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.240 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.240 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.240 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.240 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.240 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.240 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.240 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-17T06:13:00.240 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-17T06:13:00.240 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.240 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.240 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.241 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.241 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.241 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.241 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.241 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.241 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.241 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.241 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.241 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.241 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-17T06:13:00.241 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-17T06:13:00.241 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.241 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.241 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.241 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-17T06:13:00.241 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.242 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.242 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.242 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.242 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.242 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.242 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.242 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.242 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.242 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.242 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.242 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.242 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.242 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.242 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.242 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-17T06:13:00.243 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-17T06:13:00.243 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.243 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.243 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.243 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.243 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.243 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.243 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.243 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.243 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.243 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.243 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.243 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.243 INFO:teuthology.orchestra.run.smithi047.stdout: "organization": { 2024-04-17T06:13:00.243 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "organization", 2024-04-17T06:13:00.243 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.243 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.244 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.244 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.244 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.244 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.244 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.244 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.244 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.244 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.244 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.244 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.244 INFO:teuthology.orchestra.run.smithi047.stdout: "proxy": { 2024-04-17T06:13:00.244 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "proxy", 2024-04-17T06:13:00.244 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.244 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.244 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.245 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.245 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.245 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.245 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.245 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.245 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.245 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.245 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.245 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.245 INFO:teuthology.orchestra.run.smithi047.stdout: "url": { 2024-04-17T06:13:00.245 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "url", 2024-04-17T06:13:00.245 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.245 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.245 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.246 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-17T06:13:00.246 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.246 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.246 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.246 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.246 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.246 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.246 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.246 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.246 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.246 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.246 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:13:00.246 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "test_orchestrator", 2024-04-17T06:13:00.246 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-17T06:13:00.247 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-17T06:13:00.247 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-17T06:13:00.247 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-17T06:13:00.247 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-17T06:13:00.247 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.247 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.247 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.247 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.247 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.247 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.247 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.247 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.247 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.247 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.247 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.247 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.248 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.248 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.248 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.248 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.248 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.248 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.248 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.248 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-17T06:13:00.248 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-17T06:13:00.248 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.248 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.248 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.248 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.248 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.248 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.248 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.249 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.249 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.249 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.249 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.249 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.249 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-17T06:13:00.249 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-17T06:13:00.249 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.249 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.249 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.249 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-17T06:13:00.249 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.249 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.249 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.249 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.249 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.250 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.250 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.250 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.250 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.250 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.250 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.250 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.250 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.250 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.250 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.250 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-17T06:13:00.250 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-17T06:13:00.250 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.250 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.250 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.250 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.251 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.251 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.251 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.251 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.251 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.251 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.251 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.251 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.251 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.251 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.251 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:13:00.251 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "volumes", 2024-04-17T06:13:00.251 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-17T06:13:00.251 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-17T06:13:00.251 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-17T06:13:00.251 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-17T06:13:00.252 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-17T06:13:00.252 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.252 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.252 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.252 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.252 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.252 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.252 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.252 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.252 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.252 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.252 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.252 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.252 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.252 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.252 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.253 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.253 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.253 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.253 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.253 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-17T06:13:00.253 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-17T06:13:00.253 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.253 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.253 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.253 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.253 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.253 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.253 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.253 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.253 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.254 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.254 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.254 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.254 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-17T06:13:00.254 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-17T06:13:00.254 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.254 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.254 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.254 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-17T06:13:00.254 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.254 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.254 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.254 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.254 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.254 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.254 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.255 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.255 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.255 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.255 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.255 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.255 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.255 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.255 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.255 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-17T06:13:00.255 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-17T06:13:00.255 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.255 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.255 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.255 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.255 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.255 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.256 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.256 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.256 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.256 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.256 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.256 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.256 INFO:teuthology.orchestra.run.smithi047.stdout: "max_concurrent_clones": { 2024-04-17T06:13:00.256 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "max_concurrent_clones", 2024-04-17T06:13:00.256 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:13:00.256 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.256 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.256 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "4", 2024-04-17T06:13:00.256 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.256 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.256 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.256 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-17T06:13:00.256 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.257 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.257 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.257 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.257 INFO:teuthology.orchestra.run.smithi047.stdout: "periodic_async_work": { 2024-04-17T06:13:00.257 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "periodic_async_work", 2024-04-17T06:13:00.257 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.257 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.257 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.257 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.257 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.257 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.257 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.257 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Periodically check for async work", 2024-04-17T06:13:00.257 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.257 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.258 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.258 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.258 INFO:teuthology.orchestra.run.smithi047.stdout: "snapshot_clone_delay": { 2024-04-17T06:13:00.258 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "snapshot_clone_delay", 2024-04-17T06:13:00.258 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:13:00.258 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.258 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.258 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-04-17T06:13:00.258 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.258 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.258 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.258 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-17T06:13:00.258 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.258 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.258 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.258 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.259 INFO:teuthology.orchestra.run.smithi047.stdout: "snapshot_clone_no_wait": { 2024-04-17T06:13:00.259 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "snapshot_clone_no_wait", 2024-04-17T06:13:00.259 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.259 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.259 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.259 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:13:00.259 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.259 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.259 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.259 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-17T06:13:00.259 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.259 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.259 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.259 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.259 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.259 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.259 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:13:00.260 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "zabbix", 2024-04-17T06:13:00.260 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-17T06:13:00.260 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-17T06:13:00.260 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-17T06:13:00.260 INFO:teuthology.orchestra.run.smithi047.stdout: "discovery_interval": { 2024-04-17T06:13:00.260 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "discovery_interval", 2024-04-17T06:13:00.260 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "uint", 2024-04-17T06:13:00.260 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.260 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.260 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "100", 2024-04-17T06:13:00.260 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.260 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.260 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.260 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.260 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.260 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.261 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.261 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.261 INFO:teuthology.orchestra.run.smithi047.stdout: "identifier": { 2024-04-17T06:13:00.261 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "identifier", 2024-04-17T06:13:00.261 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.261 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.261 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.261 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.261 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.261 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.261 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.261 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.261 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.261 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.261 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.261 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.262 INFO:teuthology.orchestra.run.smithi047.stdout: "interval": { 2024-04-17T06:13:00.262 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "interval", 2024-04-17T06:13:00.262 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-17T06:13:00.262 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.262 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.262 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "60", 2024-04-17T06:13:00.262 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.262 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.262 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.262 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.262 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.262 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.262 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.262 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.262 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-17T06:13:00.262 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-17T06:13:00.263 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.263 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.263 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.263 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.263 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.263 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.263 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.263 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.263 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.263 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.263 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.263 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.263 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.263 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.263 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.263 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.263 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.264 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.264 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.264 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-17T06:13:00.264 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-17T06:13:00.264 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.264 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.264 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.264 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.264 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.264 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.264 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.264 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.264 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.264 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.264 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.264 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.264 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-17T06:13:00.265 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-17T06:13:00.265 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.265 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.265 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.265 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-17T06:13:00.265 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.265 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.265 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.265 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.265 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.265 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.265 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.265 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.265 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.265 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.266 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.266 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.266 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.266 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.266 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.266 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-17T06:13:00.266 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-17T06:13:00.266 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.266 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.266 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.266 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.266 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.266 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.266 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.266 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.266 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.267 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.267 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.267 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.267 INFO:teuthology.orchestra.run.smithi047.stdout: "zabbix_host": { 2024-04-17T06:13:00.267 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "zabbix_host", 2024-04-17T06:13:00.267 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.267 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.267 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.267 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.267 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.267 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.267 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.267 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.267 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.267 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.267 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.267 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.268 INFO:teuthology.orchestra.run.smithi047.stdout: "zabbix_port": { 2024-04-17T06:13:00.268 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "zabbix_port", 2024-04-17T06:13:00.268 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:13:00.268 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.268 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.268 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10051", 2024-04-17T06:13:00.268 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.268 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.268 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.268 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.268 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.268 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.268 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.268 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.268 INFO:teuthology.orchestra.run.smithi047.stdout: "zabbix_sender": { 2024-04-17T06:13:00.268 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "zabbix_sender", 2024-04-17T06:13:00.269 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.269 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.269 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.269 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-17T06:13:00.269 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.269 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.269 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.269 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.269 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.269 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.269 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.269 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.269 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.269 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.269 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-17T06:13:00.269 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.270 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.270 INFO:teuthology.orchestra.run.smithi047.stdout: "modules": [ 2024-04-17T06:13:00.270 INFO:teuthology.orchestra.run.smithi047.stdout: "cephadm", 2024-04-17T06:13:00.270 INFO:teuthology.orchestra.run.smithi047.stdout: "dashboard", 2024-04-17T06:13:00.270 INFO:teuthology.orchestra.run.smithi047.stdout: "iostat", 2024-04-17T06:13:00.270 INFO:teuthology.orchestra.run.smithi047.stdout: "nfs", 2024-04-17T06:13:00.270 INFO:teuthology.orchestra.run.smithi047.stdout: "restful", 2024-04-17T06:13:00.270 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_schedule" 2024-04-17T06:13:00.270 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.270 INFO:teuthology.orchestra.run.smithi047.stdout: "available_modules": [ 2024-04-17T06:13:00.270 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:13:00.270 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "alerts", 2024-04-17T06:13:00.270 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-17T06:13:00.270 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-17T06:13:00.271 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-17T06:13:00.271 INFO:teuthology.orchestra.run.smithi047.stdout: "interval": { 2024-04-17T06:13:00.271 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "interval", 2024-04-17T06:13:00.271 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-17T06:13:00.271 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.271 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.271 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "60", 2024-04-17T06:13:00.271 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.271 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.271 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.271 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "How frequently to reexamine health status", 2024-04-17T06:13:00.271 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.271 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.271 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.271 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.271 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-17T06:13:00.272 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-17T06:13:00.272 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.272 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.272 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.272 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.272 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.272 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.272 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.272 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.272 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.272 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.272 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.272 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.272 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.272 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.272 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.273 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.273 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.273 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.273 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.273 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-17T06:13:00.273 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-17T06:13:00.273 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.273 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.273 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.273 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.273 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.273 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.273 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.273 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.273 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.273 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.273 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.274 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.274 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-17T06:13:00.274 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-17T06:13:00.274 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.274 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.274 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.274 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-17T06:13:00.274 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.274 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.274 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.274 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.274 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.274 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.274 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.275 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.275 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.275 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.275 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.275 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.275 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.275 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.275 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.275 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-17T06:13:00.275 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-17T06:13:00.275 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.275 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.275 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.275 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.275 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.275 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.276 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.276 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.276 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.276 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.276 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.276 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.276 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_destination": { 2024-04-17T06:13:00.276 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_destination", 2024-04-17T06:13:00.276 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.276 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.276 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.276 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.276 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.276 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.276 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.276 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Email address to send alerts to", 2024-04-17T06:13:00.276 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.277 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.277 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.277 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.277 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_from_name": { 2024-04-17T06:13:00.277 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_from_name", 2024-04-17T06:13:00.277 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.277 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.277 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.277 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "Ceph", 2024-04-17T06:13:00.277 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.277 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.277 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.277 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Email From: name", 2024-04-17T06:13:00.277 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.278 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.278 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.278 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.278 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_host": { 2024-04-17T06:13:00.278 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_host", 2024-04-17T06:13:00.278 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.278 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.278 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.278 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.278 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.278 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.278 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.278 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "SMTP server", 2024-04-17T06:13:00.278 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.278 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.278 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.279 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.279 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_password": { 2024-04-17T06:13:00.279 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_password", 2024-04-17T06:13:00.279 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.279 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.279 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.279 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.279 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.279 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.279 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.279 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Password to authenticate with", 2024-04-17T06:13:00.279 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.279 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.279 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.279 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.279 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_port": { 2024-04-17T06:13:00.280 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_port", 2024-04-17T06:13:00.280 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:13:00.280 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.280 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.280 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "465", 2024-04-17T06:13:00.280 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.280 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.280 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.280 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "SMTP port", 2024-04-17T06:13:00.280 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.280 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.280 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.280 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.280 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_sender": { 2024-04-17T06:13:00.280 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_sender", 2024-04-17T06:13:00.280 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.281 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.281 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.281 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.281 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.281 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.281 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.281 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "SMTP envelope sender", 2024-04-17T06:13:00.281 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.281 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.281 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.281 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.281 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_ssl": { 2024-04-17T06:13:00.281 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_ssl", 2024-04-17T06:13:00.281 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.281 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.281 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.282 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:13:00.282 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.282 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.282 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.282 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-17T06:13:00.282 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.282 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.282 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.282 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.282 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_user": { 2024-04-17T06:13:00.282 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_user", 2024-04-17T06:13:00.282 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.282 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.282 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.282 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.282 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.283 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.283 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.283 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "User to authenticate as", 2024-04-17T06:13:00.283 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.283 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.283 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.283 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.283 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.283 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.283 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:13:00.283 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "balancer", 2024-04-17T06:13:00.283 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-17T06:13:00.283 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-17T06:13:00.283 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-17T06:13:00.283 INFO:teuthology.orchestra.run.smithi047.stdout: "active": { 2024-04-17T06:13:00.283 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "active", 2024-04-17T06:13:00.284 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.284 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.284 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.284 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:13:00.284 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.284 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.284 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.284 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "automatically balance PGs across cluster", 2024-04-17T06:13:00.284 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.284 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.284 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.284 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.284 INFO:teuthology.orchestra.run.smithi047.stdout: "begin_time": { 2024-04-17T06:13:00.284 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "begin_time", 2024-04-17T06:13:00.284 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.284 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.284 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.285 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0000", 2024-04-17T06:13:00.285 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.285 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.285 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.285 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "beginning time of day to automatically balance", 2024-04-17T06:13:00.285 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-17T06:13:00.285 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.285 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.285 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.285 INFO:teuthology.orchestra.run.smithi047.stdout: "begin_weekday": { 2024-04-17T06:13:00.285 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "begin_weekday", 2024-04-17T06:13:00.285 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "uint", 2024-04-17T06:13:00.285 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.285 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.285 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-04-17T06:13:00.285 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "0", 2024-04-17T06:13:00.286 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "6", 2024-04-17T06:13:00.286 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.286 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-17T06:13:00.286 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-17T06:13:00.286 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.286 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.286 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.286 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_compat_max_iterations": { 2024-04-17T06:13:00.286 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "crush_compat_max_iterations", 2024-04-17T06:13:00.286 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "uint", 2024-04-17T06:13:00.286 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.286 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.286 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "25", 2024-04-17T06:13:00.286 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "1", 2024-04-17T06:13:00.286 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "250", 2024-04-17T06:13:00.286 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.287 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-17T06:13:00.287 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.287 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.287 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.287 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.287 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_compat_metrics": { 2024-04-17T06:13:00.287 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "crush_compat_metrics", 2024-04-17T06:13:00.287 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.287 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.287 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.287 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "pgs,objects,bytes", 2024-04-17T06:13:00.287 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.287 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.287 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.287 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-17T06:13:00.287 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-17T06:13:00.288 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.288 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.288 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.288 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_compat_step": { 2024-04-17T06:13:00.288 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "crush_compat_step", 2024-04-17T06:13:00.288 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "float", 2024-04-17T06:13:00.288 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.288 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.288 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0.5", 2024-04-17T06:13:00.288 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "0.001", 2024-04-17T06:13:00.288 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "0.999", 2024-04-17T06:13:00.288 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.288 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "aggressiveness of optimization", 2024-04-17T06:13:00.288 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-17T06:13:00.288 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.289 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.289 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.289 INFO:teuthology.orchestra.run.smithi047.stdout: "end_time": { 2024-04-17T06:13:00.289 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "end_time", 2024-04-17T06:13:00.289 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.289 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.289 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.289 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "2359", 2024-04-17T06:13:00.289 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.289 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.289 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.289 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "ending time of day to automatically balance", 2024-04-17T06:13:00.289 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-17T06:13:00.289 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.289 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.289 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.289 INFO:teuthology.orchestra.run.smithi047.stdout: "end_weekday": { 2024-04-17T06:13:00.290 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "end_weekday", 2024-04-17T06:13:00.290 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "uint", 2024-04-17T06:13:00.290 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.290 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.290 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-04-17T06:13:00.290 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "0", 2024-04-17T06:13:00.290 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "6", 2024-04-17T06:13:00.290 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.290 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-17T06:13:00.290 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-17T06:13:00.290 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.290 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.290 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.290 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-17T06:13:00.290 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-17T06:13:00.291 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.291 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.291 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.291 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.291 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.291 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.291 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.291 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.291 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.291 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.291 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.291 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.291 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.292 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.292 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.292 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.292 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.292 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.292 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.292 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-17T06:13:00.292 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-17T06:13:00.292 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.292 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.292 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.292 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.292 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.292 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.292 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.293 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.293 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.293 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.293 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.293 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.293 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-17T06:13:00.293 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-17T06:13:00.293 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.293 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.293 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.293 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-17T06:13:00.293 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.293 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.293 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.294 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.294 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.294 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.294 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.294 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.294 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.294 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.294 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.294 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.294 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.294 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.294 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.294 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-17T06:13:00.294 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-17T06:13:00.295 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.295 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.295 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.295 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.295 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.295 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.295 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.295 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.295 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.295 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.295 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.295 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.295 INFO:teuthology.orchestra.run.smithi047.stdout: "min_score": { 2024-04-17T06:13:00.295 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "min_score", 2024-04-17T06:13:00.295 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "float", 2024-04-17T06:13:00.296 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.296 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.296 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-04-17T06:13:00.296 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.296 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.296 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.296 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-17T06:13:00.296 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.296 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.296 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.296 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.296 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": { 2024-04-17T06:13:00.296 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "mode", 2024-04-17T06:13:00.296 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.296 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.296 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.297 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "upmap", 2024-04-17T06:13:00.297 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.297 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.297 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.297 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-compat", 2024-04-17T06:13:00.297 INFO:teuthology.orchestra.run.smithi047.stdout: "none", 2024-04-17T06:13:00.297 INFO:teuthology.orchestra.run.smithi047.stdout: "read", 2024-04-17T06:13:00.297 INFO:teuthology.orchestra.run.smithi047.stdout: "upmap", 2024-04-17T06:13:00.297 INFO:teuthology.orchestra.run.smithi047.stdout: "upmap-read" 2024-04-17T06:13:00.297 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.297 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Balancer mode", 2024-04-17T06:13:00.298 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.298 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.298 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.298 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.298 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_ids": { 2024-04-17T06:13:00.298 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "pool_ids", 2024-04-17T06:13:00.298 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.298 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.298 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.298 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.298 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.298 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.298 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.298 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-17T06:13:00.298 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.298 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.299 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.299 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.299 INFO:teuthology.orchestra.run.smithi047.stdout: "sleep_interval": { 2024-04-17T06:13:00.299 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sleep_interval", 2024-04-17T06:13:00.299 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-17T06:13:00.299 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.299 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.299 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "60", 2024-04-17T06:13:00.299 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.299 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.299 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.299 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-17T06:13:00.299 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.299 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.299 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.300 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.300 INFO:teuthology.orchestra.run.smithi047.stdout: "upmap_max_deviation": { 2024-04-17T06:13:00.300 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "upmap_max_deviation", 2024-04-17T06:13:00.300 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:13:00.300 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.300 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.300 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "5", 2024-04-17T06:13:00.300 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "1", 2024-04-17T06:13:00.300 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.300 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.300 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-17T06:13:00.300 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-17T06:13:00.300 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.300 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.300 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.300 INFO:teuthology.orchestra.run.smithi047.stdout: "upmap_max_optimizations": { 2024-04-17T06:13:00.301 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "upmap_max_optimizations", 2024-04-17T06:13:00.301 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "uint", 2024-04-17T06:13:00.301 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.301 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.301 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-04-17T06:13:00.301 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.301 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.301 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.301 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-17T06:13:00.301 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.301 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.301 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.301 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.301 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.301 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.302 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:13:00.302 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "cephadm", 2024-04-17T06:13:00.302 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-17T06:13:00.302 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-17T06:13:00.302 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-17T06:13:00.302 INFO:teuthology.orchestra.run.smithi047.stdout: "agent_down_multiplier": { 2024-04-17T06:13:00.302 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "agent_down_multiplier", 2024-04-17T06:13:00.302 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "float", 2024-04-17T06:13:00.302 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.302 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.302 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "3.0", 2024-04-17T06:13:00.302 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.302 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.302 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.303 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-17T06:13:00.303 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.303 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.303 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.303 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.303 INFO:teuthology.orchestra.run.smithi047.stdout: "agent_refresh_rate": { 2024-04-17T06:13:00.303 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "agent_refresh_rate", 2024-04-17T06:13:00.303 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-17T06:13:00.303 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.303 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.303 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "20", 2024-04-17T06:13:00.303 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.303 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.303 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.303 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-17T06:13:00.303 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.304 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.304 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.304 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.304 INFO:teuthology.orchestra.run.smithi047.stdout: "agent_starting_port": { 2024-04-17T06:13:00.304 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "agent_starting_port", 2024-04-17T06:13:00.304 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:13:00.304 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.304 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.304 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "4721", 2024-04-17T06:13:00.304 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.304 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.304 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.304 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-17T06:13:00.304 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.304 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.305 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.305 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.305 INFO:teuthology.orchestra.run.smithi047.stdout: "allow_ptrace": { 2024-04-17T06:13:00.305 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "allow_ptrace", 2024-04-17T06:13:00.305 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.305 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.305 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.305 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.305 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.305 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.305 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.305 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-17T06:13:00.305 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-17T06:13:00.305 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.305 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.306 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.306 INFO:teuthology.orchestra.run.smithi047.stdout: "autotune_interval": { 2024-04-17T06:13:00.306 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "autotune_interval", 2024-04-17T06:13:00.306 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-17T06:13:00.306 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.306 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.306 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "600", 2024-04-17T06:13:00.306 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.306 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.306 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.306 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-17T06:13:00.306 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.306 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.306 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.306 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.307 INFO:teuthology.orchestra.run.smithi047.stdout: "autotune_memory_target_ratio": { 2024-04-17T06:13:00.307 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "autotune_memory_target_ratio", 2024-04-17T06:13:00.307 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "float", 2024-04-17T06:13:00.307 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.307 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.307 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0.7", 2024-04-17T06:13:00.307 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.307 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.307 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.307 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-17T06:13:00.307 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.307 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.307 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.307 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.307 INFO:teuthology.orchestra.run.smithi047.stdout: "cephadm_log_destination": { 2024-04-17T06:13:00.307 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "cephadm_log_destination", 2024-04-17T06:13:00.308 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.308 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.308 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.308 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.308 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.308 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.308 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.308 INFO:teuthology.orchestra.run.smithi047.stdout: "file", 2024-04-17T06:13:00.308 INFO:teuthology.orchestra.run.smithi047.stdout: "file,syslog", 2024-04-17T06:13:00.308 INFO:teuthology.orchestra.run.smithi047.stdout: "syslog" 2024-04-17T06:13:00.308 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.308 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-17T06:13:00.308 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.308 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.308 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.309 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.309 INFO:teuthology.orchestra.run.smithi047.stdout: "cgroups_split": { 2024-04-17T06:13:00.309 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "cgroups_split", 2024-04-17T06:13:00.309 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.309 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.309 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.309 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:13:00.309 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.309 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.309 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.309 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-17T06:13:00.309 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.309 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.309 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.309 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.310 INFO:teuthology.orchestra.run.smithi047.stdout: "config_checks_enabled": { 2024-04-17T06:13:00.310 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "config_checks_enabled", 2024-04-17T06:13:00.310 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.310 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.310 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.310 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.310 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.310 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.310 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.310 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-17T06:13:00.310 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.310 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.310 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.310 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.311 INFO:teuthology.orchestra.run.smithi047.stdout: "config_dashboard": { 2024-04-17T06:13:00.311 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "config_dashboard", 2024-04-17T06:13:00.311 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.311 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.311 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.311 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:13:00.311 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.311 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.311 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.311 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-17T06:13:00.311 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.311 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.311 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.311 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.311 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_alertmanager": { 2024-04-17T06:13:00.311 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_alertmanager", 2024-04-17T06:13:00.312 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.312 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.312 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.312 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-17T06:13:00.312 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.312 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.312 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.312 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Prometheus container image", 2024-04-17T06:13:00.312 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.312 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.312 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.312 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.312 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_base": { 2024-04-17T06:13:00.312 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_base", 2024-04-17T06:13:00.313 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.313 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.313 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.313 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-17T06:13:00.313 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.313 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.313 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.313 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Container image name, without the tag", 2024-04-17T06:13:00.313 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.313 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.313 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.313 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.313 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_elasticsearch": { 2024-04-17T06:13:00.313 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_elasticsearch", 2024-04-17T06:13:00.313 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.314 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.314 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.314 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-17T06:13:00.314 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.314 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.314 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.314 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "elasticsearch container image", 2024-04-17T06:13:00.314 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.314 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.314 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.314 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.314 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_grafana": { 2024-04-17T06:13:00.314 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_grafana", 2024-04-17T06:13:00.314 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.315 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.315 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.315 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-17T06:13:00.315 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.315 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.315 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.315 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Prometheus container image", 2024-04-17T06:13:00.315 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.315 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.315 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.315 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.315 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_haproxy": { 2024-04-17T06:13:00.315 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_haproxy", 2024-04-17T06:13:00.315 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.315 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.316 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.316 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-17T06:13:00.316 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.316 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.316 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.316 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "HAproxy container image", 2024-04-17T06:13:00.316 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.316 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.316 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.316 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.316 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_jaeger_agent": { 2024-04-17T06:13:00.316 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_jaeger_agent", 2024-04-17T06:13:00.316 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.316 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.316 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.316 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-17T06:13:00.317 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.317 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.317 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.317 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Jaeger agent container image", 2024-04-17T06:13:00.317 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.317 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.317 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.317 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.317 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_jaeger_collector": { 2024-04-17T06:13:00.317 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_jaeger_collector", 2024-04-17T06:13:00.317 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.317 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.317 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.317 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-17T06:13:00.318 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.318 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.318 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.318 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Jaeger collector container image", 2024-04-17T06:13:00.318 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.318 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.318 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.318 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.318 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_jaeger_query": { 2024-04-17T06:13:00.318 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_jaeger_query", 2024-04-17T06:13:00.318 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.318 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.318 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.318 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-17T06:13:00.319 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.319 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.319 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.319 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Jaeger query container image", 2024-04-17T06:13:00.319 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.319 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.319 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.319 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.319 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_keepalived": { 2024-04-17T06:13:00.319 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_keepalived", 2024-04-17T06:13:00.319 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.319 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.319 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.319 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-17T06:13:00.319 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.320 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.320 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.320 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Keepalived container image", 2024-04-17T06:13:00.320 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.320 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.320 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.320 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.320 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_loki": { 2024-04-17T06:13:00.320 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_loki", 2024-04-17T06:13:00.320 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.320 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.320 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.320 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-17T06:13:00.320 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.321 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.321 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.321 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Loki container image", 2024-04-17T06:13:00.321 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.321 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.321 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.321 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.321 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_node_exporter": { 2024-04-17T06:13:00.321 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_node_exporter", 2024-04-17T06:13:00.321 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.321 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.321 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.321 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-17T06:13:00.321 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.321 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.322 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.322 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Prometheus container image", 2024-04-17T06:13:00.322 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.322 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.322 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.322 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.322 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_nvmeof": { 2024-04-17T06:13:00.322 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_nvmeof", 2024-04-17T06:13:00.322 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.322 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.322 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.322 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-17T06:13:00.322 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.322 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.323 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.323 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Nvme-of container image", 2024-04-17T06:13:00.323 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.323 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.323 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.323 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.323 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_prometheus": { 2024-04-17T06:13:00.323 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_prometheus", 2024-04-17T06:13:00.323 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.323 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.323 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.323 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-17T06:13:00.324 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.324 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.324 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.324 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Prometheus container image", 2024-04-17T06:13:00.324 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.324 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.324 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.324 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.324 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_promtail": { 2024-04-17T06:13:00.324 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_promtail", 2024-04-17T06:13:00.324 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.324 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.325 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.325 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-17T06:13:00.325 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.325 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.325 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.325 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Promtail container image", 2024-04-17T06:13:00.325 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.325 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.325 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.325 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.325 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_samba": { 2024-04-17T06:13:00.325 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_samba", 2024-04-17T06:13:00.325 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.325 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.325 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.326 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-17T06:13:00.326 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.326 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.326 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.326 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Samba/SMB container image", 2024-04-17T06:13:00.326 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.326 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.326 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.326 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.326 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_snmp_gateway": { 2024-04-17T06:13:00.326 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_snmp_gateway", 2024-04-17T06:13:00.326 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.326 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.327 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.327 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-17T06:13:00.327 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.327 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.327 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.327 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "SNMP Gateway container image", 2024-04-17T06:13:00.327 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.327 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.327 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.327 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.327 INFO:teuthology.orchestra.run.smithi047.stdout: "container_init": { 2024-04-17T06:13:00.327 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_init", 2024-04-17T06:13:00.327 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.327 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.327 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.328 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:13:00.328 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.328 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.328 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.328 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Run podman/docker with `--init`", 2024-04-17T06:13:00.328 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.328 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.328 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.328 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.328 INFO:teuthology.orchestra.run.smithi047.stdout: "daemon_cache_timeout": { 2024-04-17T06:13:00.328 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "daemon_cache_timeout", 2024-04-17T06:13:00.328 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-17T06:13:00.328 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.328 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.329 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "600", 2024-04-17T06:13:00.329 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.329 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.329 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.329 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-17T06:13:00.329 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.329 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.329 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.329 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.329 INFO:teuthology.orchestra.run.smithi047.stdout: "default_cephadm_command_timeout": { 2024-04-17T06:13:00.329 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "default_cephadm_command_timeout", 2024-04-17T06:13:00.329 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:13:00.329 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.329 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.329 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "900", 2024-04-17T06:13:00.329 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.330 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.330 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.330 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-17T06:13:00.330 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.330 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.330 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.330 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.330 INFO:teuthology.orchestra.run.smithi047.stdout: "default_registry": { 2024-04-17T06:13:00.330 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "default_registry", 2024-04-17T06:13:00.330 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.330 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.330 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.330 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "docker.io", 2024-04-17T06:13:00.330 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.330 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.331 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.331 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-17T06:13:00.331 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.331 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.331 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.331 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.331 INFO:teuthology.orchestra.run.smithi047.stdout: "device_cache_timeout": { 2024-04-17T06:13:00.331 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "device_cache_timeout", 2024-04-17T06:13:00.331 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-17T06:13:00.331 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.331 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.331 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "1800", 2024-04-17T06:13:00.331 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.331 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.331 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.332 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "seconds to cache device inventory", 2024-04-17T06:13:00.332 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.332 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.332 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.332 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.332 INFO:teuthology.orchestra.run.smithi047.stdout: "device_enhanced_scan": { 2024-04-17T06:13:00.332 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "device_enhanced_scan", 2024-04-17T06:13:00.332 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.332 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.332 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.332 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.332 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.332 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.332 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.332 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-17T06:13:00.333 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.333 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.333 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.333 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.333 INFO:teuthology.orchestra.run.smithi047.stdout: "facts_cache_timeout": { 2024-04-17T06:13:00.333 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "facts_cache_timeout", 2024-04-17T06:13:00.333 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-17T06:13:00.333 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.333 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.333 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "60", 2024-04-17T06:13:00.333 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.333 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.333 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.333 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "seconds to cache host facts data", 2024-04-17T06:13:00.334 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.334 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.334 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.334 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.334 INFO:teuthology.orchestra.run.smithi047.stdout: "grafana_dashboards_path": { 2024-04-17T06:13:00.334 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "grafana_dashboards_path", 2024-04-17T06:13:00.334 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.334 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.334 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.334 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-17T06:13:00.334 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.334 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.334 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.334 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-17T06:13:00.334 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.335 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.335 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.335 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.335 INFO:teuthology.orchestra.run.smithi047.stdout: "host_check_interval": { 2024-04-17T06:13:00.335 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "host_check_interval", 2024-04-17T06:13:00.335 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-17T06:13:00.335 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.335 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.335 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "600", 2024-04-17T06:13:00.335 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.335 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.335 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.335 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how frequently to perform a host check", 2024-04-17T06:13:00.335 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.335 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.336 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.336 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.336 INFO:teuthology.orchestra.run.smithi047.stdout: "hw_monitoring": { 2024-04-17T06:13:00.336 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "hw_monitoring", 2024-04-17T06:13:00.336 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.336 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.336 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.336 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.336 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.336 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.336 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.336 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-17T06:13:00.336 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.336 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.337 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.337 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.337 INFO:teuthology.orchestra.run.smithi047.stdout: "inventory_list_all": { 2024-04-17T06:13:00.337 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "inventory_list_all", 2024-04-17T06:13:00.337 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.337 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.337 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.337 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.337 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.337 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.337 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.337 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-17T06:13:00.337 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.337 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.338 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.338 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.338 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-17T06:13:00.338 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-17T06:13:00.338 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.338 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.338 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.338 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.338 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.338 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.338 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.338 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.338 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.339 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.339 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.339 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.339 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.339 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.339 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.339 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.339 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.339 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.339 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.339 INFO:teuthology.orchestra.run.smithi047.stdout: "log_refresh_metadata": { 2024-04-17T06:13:00.339 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_refresh_metadata", 2024-04-17T06:13:00.340 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.340 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.340 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.340 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.340 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.340 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.340 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.340 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-17T06:13:00.340 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.340 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.340 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.340 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.340 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-17T06:13:00.340 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-17T06:13:00.340 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.340 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.341 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.341 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:13:00.341 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.341 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.341 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.341 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-17T06:13:00.341 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.341 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.341 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.341 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.341 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-17T06:13:00.341 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-17T06:13:00.341 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.341 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.341 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.342 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-17T06:13:00.342 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.342 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.342 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.342 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.342 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.342 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.342 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.342 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.342 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.342 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.342 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.342 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.342 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.342 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.343 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.343 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-17T06:13:00.343 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-17T06:13:00.343 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.343 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.343 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.343 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.343 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.343 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.343 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.343 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.343 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.343 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.343 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.343 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.343 INFO:teuthology.orchestra.run.smithi047.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-17T06:13:00.344 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-17T06:13:00.344 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.344 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.344 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.344 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.344 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.344 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.344 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.344 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-17T06:13:00.344 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.344 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.344 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.344 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.344 INFO:teuthology.orchestra.run.smithi047.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-17T06:13:00.344 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-17T06:13:00.344 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.345 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.345 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.345 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "*", 2024-04-17T06:13:00.345 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.345 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.345 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.345 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-17T06:13:00.345 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.345 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.345 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.345 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.345 INFO:teuthology.orchestra.run.smithi047.stdout: "max_count_per_host": { 2024-04-17T06:13:00.345 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "max_count_per_host", 2024-04-17T06:13:00.345 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:13:00.345 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.345 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.346 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-04-17T06:13:00.346 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.346 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.346 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.346 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "max number of daemons per service per host", 2024-04-17T06:13:00.346 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.346 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.346 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.346 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.346 INFO:teuthology.orchestra.run.smithi047.stdout: "max_osd_draining_count": { 2024-04-17T06:13:00.346 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "max_osd_draining_count", 2024-04-17T06:13:00.347 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:13:00.347 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.347 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.347 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-04-17T06:13:00.347 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.347 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.347 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.347 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-17T06:13:00.347 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.347 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.347 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.347 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.347 INFO:teuthology.orchestra.run.smithi047.stdout: "migration_current": { 2024-04-17T06:13:00.347 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "migration_current", 2024-04-17T06:13:00.347 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:13:00.348 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.348 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.348 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.348 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.348 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.348 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.348 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "internal - do not modify", 2024-04-17T06:13:00.348 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.348 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.348 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.348 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.348 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": { 2024-04-17T06:13:00.348 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "mode", 2024-04-17T06:13:00.348 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.348 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.348 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.348 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "root", 2024-04-17T06:13:00.349 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.349 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.349 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.349 INFO:teuthology.orchestra.run.smithi047.stdout: "cephadm-package", 2024-04-17T06:13:00.349 INFO:teuthology.orchestra.run.smithi047.stdout: "root" 2024-04-17T06:13:00.349 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.349 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "mode for remote execution of cephadm", 2024-04-17T06:13:00.349 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.349 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.349 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.349 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.349 INFO:teuthology.orchestra.run.smithi047.stdout: "oob_default_addr": { 2024-04-17T06:13:00.349 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "oob_default_addr", 2024-04-17T06:13:00.349 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.350 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.350 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.350 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "169.254.1.1", 2024-04-17T06:13:00.350 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.350 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.350 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.350 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-17T06:13:00.350 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.350 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.350 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.350 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.350 INFO:teuthology.orchestra.run.smithi047.stdout: "prometheus_alerts_path": { 2024-04-17T06:13:00.350 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "prometheus_alerts_path", 2024-04-17T06:13:00.350 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.350 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.351 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.351 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-17T06:13:00.351 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.351 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.351 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.351 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-17T06:13:00.351 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.351 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.351 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.351 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.351 INFO:teuthology.orchestra.run.smithi047.stdout: "registry_insecure": { 2024-04-17T06:13:00.351 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "registry_insecure", 2024-04-17T06:13:00.351 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.351 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.351 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.351 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.352 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.352 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.352 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.352 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-17T06:13:00.352 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.352 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.352 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.352 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.352 INFO:teuthology.orchestra.run.smithi047.stdout: "registry_password": { 2024-04-17T06:13:00.352 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "registry_password", 2024-04-17T06:13:00.352 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.352 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.352 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.352 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.352 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.352 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.352 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.353 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-17T06:13:00.353 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.353 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.353 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.353 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.353 INFO:teuthology.orchestra.run.smithi047.stdout: "registry_url": { 2024-04-17T06:13:00.353 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "registry_url", 2024-04-17T06:13:00.353 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.353 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.353 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.353 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.353 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.353 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.353 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.353 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-17T06:13:00.353 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.354 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.354 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.354 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.354 INFO:teuthology.orchestra.run.smithi047.stdout: "registry_username": { 2024-04-17T06:13:00.354 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "registry_username", 2024-04-17T06:13:00.354 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.354 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.354 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.354 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.354 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.354 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.354 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.354 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-17T06:13:00.354 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.354 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.355 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.355 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.355 INFO:teuthology.orchestra.run.smithi047.stdout: "secure_monitoring_stack": { 2024-04-17T06:13:00.355 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "secure_monitoring_stack", 2024-04-17T06:13:00.355 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.355 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.355 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.355 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.355 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.355 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.355 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.355 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-17T06:13:00.355 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.355 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.355 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.355 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.355 INFO:teuthology.orchestra.run.smithi047.stdout: "service_discovery_port": { 2024-04-17T06:13:00.356 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "service_discovery_port", 2024-04-17T06:13:00.356 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:13:00.356 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.356 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.356 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "8765", 2024-04-17T06:13:00.356 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.356 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.356 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.356 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "cephadm service discovery port", 2024-04-17T06:13:00.356 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.356 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.356 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.356 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.356 INFO:teuthology.orchestra.run.smithi047.stdout: "ssh_config_file": { 2024-04-17T06:13:00.356 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ssh_config_file", 2024-04-17T06:13:00.356 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.357 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.357 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.357 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.357 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.357 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.357 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.357 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-17T06:13:00.357 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.357 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.357 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.357 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.357 INFO:teuthology.orchestra.run.smithi047.stdout: "use_agent": { 2024-04-17T06:13:00.357 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "use_agent", 2024-04-17T06:13:00.357 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.357 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.357 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.358 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.358 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.358 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.358 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.358 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-17T06:13:00.358 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.358 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.358 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.358 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.358 INFO:teuthology.orchestra.run.smithi047.stdout: "use_repo_digest": { 2024-04-17T06:13:00.358 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "use_repo_digest", 2024-04-17T06:13:00.358 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.358 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.358 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.358 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:13:00.359 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.359 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.359 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.359 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-17T06:13:00.359 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.359 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.359 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.359 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.359 INFO:teuthology.orchestra.run.smithi047.stdout: "warn_on_failed_host_check": { 2024-04-17T06:13:00.359 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "warn_on_failed_host_check", 2024-04-17T06:13:00.359 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.359 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.359 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.359 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:13:00.359 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.359 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.360 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.360 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "raise a health warning if the host check fails", 2024-04-17T06:13:00.360 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.360 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.360 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.360 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.360 INFO:teuthology.orchestra.run.smithi047.stdout: "warn_on_stray_daemons": { 2024-04-17T06:13:00.360 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "warn_on_stray_daemons", 2024-04-17T06:13:00.360 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.360 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.360 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.360 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:13:00.360 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.360 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.360 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.360 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-17T06:13:00.361 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.361 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.361 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.361 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.361 INFO:teuthology.orchestra.run.smithi047.stdout: "warn_on_stray_hosts": { 2024-04-17T06:13:00.361 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "warn_on_stray_hosts", 2024-04-17T06:13:00.361 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.361 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.361 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.361 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:13:00.361 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.361 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.361 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.361 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-17T06:13:00.361 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.361 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.362 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.362 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.362 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.362 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.362 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:13:00.362 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "crash", 2024-04-17T06:13:00.362 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-17T06:13:00.362 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-17T06:13:00.362 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-17T06:13:00.362 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-17T06:13:00.362 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-17T06:13:00.362 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.362 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.362 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.362 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.362 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.363 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.363 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.363 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.363 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.363 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.363 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.363 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.363 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.363 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.363 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.363 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.363 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.363 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.363 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.363 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-17T06:13:00.364 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-17T06:13:00.364 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.364 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.364 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.364 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.364 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.364 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.364 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.364 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.364 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.364 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.364 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.364 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.364 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-17T06:13:00.364 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-17T06:13:00.364 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.364 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.365 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.365 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-17T06:13:00.365 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.365 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.365 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.365 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.365 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.365 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.365 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.365 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.365 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.365 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.365 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.365 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.365 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.365 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.366 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.366 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-17T06:13:00.366 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-17T06:13:00.366 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.366 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.366 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.366 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.366 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.366 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.366 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.366 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.366 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.366 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.366 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.366 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.367 INFO:teuthology.orchestra.run.smithi047.stdout: "retain_interval": { 2024-04-17T06:13:00.367 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "retain_interval", 2024-04-17T06:13:00.367 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-17T06:13:00.367 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.367 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.367 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "31536000", 2024-04-17T06:13:00.367 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.367 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.367 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.367 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-17T06:13:00.367 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.367 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.367 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.367 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.367 INFO:teuthology.orchestra.run.smithi047.stdout: "warn_recent_interval": { 2024-04-17T06:13:00.368 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "warn_recent_interval", 2024-04-17T06:13:00.368 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-17T06:13:00.368 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.368 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.368 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "1209600", 2024-04-17T06:13:00.368 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.368 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.368 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.368 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-17T06:13:00.368 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.368 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.368 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.368 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.368 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.368 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.369 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:13:00.369 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "dashboard", 2024-04-17T06:13:00.369 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-17T06:13:00.369 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-17T06:13:00.369 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-17T06:13:00.369 INFO:teuthology.orchestra.run.smithi047.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-17T06:13:00.369 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-17T06:13:00.369 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:13:00.369 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.369 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.369 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-04-17T06:13:00.369 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.369 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.369 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.369 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.370 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.370 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.370 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.370 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.370 INFO:teuthology.orchestra.run.smithi047.stdout: "ALERTMANAGER_API_HOST": { 2024-04-17T06:13:00.370 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-17T06:13:00.370 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.370 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.370 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.370 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.370 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.370 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.370 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.370 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.371 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.371 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.371 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.371 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.371 INFO:teuthology.orchestra.run.smithi047.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-17T06:13:00.371 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-17T06:13:00.371 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.371 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.371 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.371 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:13:00.371 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.371 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.371 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.371 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.372 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.372 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.372 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.372 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.372 INFO:teuthology.orchestra.run.smithi047.stdout: "AUDIT_API_ENABLED": { 2024-04-17T06:13:00.372 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "AUDIT_API_ENABLED", 2024-04-17T06:13:00.373 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.373 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.373 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.373 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.373 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.373 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.373 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.373 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.373 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.373 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.373 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.374 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.374 INFO:teuthology.orchestra.run.smithi047.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-17T06:13:00.374 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-17T06:13:00.374 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.374 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.374 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.374 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:13:00.374 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.374 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.374 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.374 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.374 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.374 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.374 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.375 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.375 INFO:teuthology.orchestra.run.smithi047.stdout: "ENABLE_BROWSABLE_API": { 2024-04-17T06:13:00.375 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-17T06:13:00.375 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.375 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.375 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.375 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:13:00.375 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.375 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.375 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.375 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.375 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.375 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.375 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.375 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.376 INFO:teuthology.orchestra.run.smithi047.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-17T06:13:00.376 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-17T06:13:00.376 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.376 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.376 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.376 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:13:00.376 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.376 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.376 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.376 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.376 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.376 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.376 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.376 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.376 INFO:teuthology.orchestra.run.smithi047.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-17T06:13:00.376 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-17T06:13:00.376 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.377 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.377 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.377 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:13:00.377 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.377 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.377 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.377 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.377 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.377 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.377 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.377 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.377 INFO:teuthology.orchestra.run.smithi047.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-17T06:13:00.377 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-17T06:13:00.377 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.378 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.378 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.378 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:13:00.378 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.378 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.378 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.378 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.378 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.378 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.378 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.378 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.378 INFO:teuthology.orchestra.run.smithi047.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-17T06:13:00.378 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-17T06:13:00.378 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.379 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.379 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.379 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:13:00.379 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.379 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.379 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.379 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.379 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.379 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.379 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.379 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.379 INFO:teuthology.orchestra.run.smithi047.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-17T06:13:00.379 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-17T06:13:00.379 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.380 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.380 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.380 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:13:00.380 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.380 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.380 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.380 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.380 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.380 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.380 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.380 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.380 INFO:teuthology.orchestra.run.smithi047.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-17T06:13:00.380 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-17T06:13:00.380 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.380 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.381 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.381 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:13:00.381 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.381 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.381 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.381 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.381 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.381 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.381 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.381 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.381 INFO:teuthology.orchestra.run.smithi047.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-17T06:13:00.381 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-17T06:13:00.381 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.381 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.381 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.382 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:13:00.382 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.382 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.382 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.382 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.382 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.382 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.382 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.382 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.382 INFO:teuthology.orchestra.run.smithi047.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-17T06:13:00.382 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-17T06:13:00.382 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.382 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.383 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.383 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.383 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.383 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.383 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.383 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.383 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.383 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.383 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.383 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.383 INFO:teuthology.orchestra.run.smithi047.stdout: "GRAFANA_API_PASSWORD": { 2024-04-17T06:13:00.383 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-17T06:13:00.383 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.383 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.384 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.384 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "admin", 2024-04-17T06:13:00.384 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.384 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.384 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.384 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.384 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.384 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.384 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.384 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.384 INFO:teuthology.orchestra.run.smithi047.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-17T06:13:00.384 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-17T06:13:00.384 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.384 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.384 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.385 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:13:00.385 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.385 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.385 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.385 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.385 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.385 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.385 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.385 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.385 INFO:teuthology.orchestra.run.smithi047.stdout: "GRAFANA_API_URL": { 2024-04-17T06:13:00.385 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "GRAFANA_API_URL", 2024-04-17T06:13:00.385 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.385 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.385 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.385 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.386 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.386 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.386 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.386 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.386 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.386 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.386 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.386 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.386 INFO:teuthology.orchestra.run.smithi047.stdout: "GRAFANA_API_USERNAME": { 2024-04-17T06:13:00.386 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-17T06:13:00.386 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.386 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.386 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.386 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "admin", 2024-04-17T06:13:00.387 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.387 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.387 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.387 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.387 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.387 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.387 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.387 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.387 INFO:teuthology.orchestra.run.smithi047.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-17T06:13:00.387 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-17T06:13:00.387 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.387 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.387 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.387 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.388 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.388 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.388 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.388 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.388 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.388 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.388 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.388 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.389 INFO:teuthology.orchestra.run.smithi047.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-17T06:13:00.389 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-17T06:13:00.389 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.389 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.389 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.389 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.389 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.389 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.389 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.389 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.389 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.389 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.389 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.389 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.390 INFO:teuthology.orchestra.run.smithi047.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-17T06:13:00.390 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-17T06:13:00.390 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.390 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.390 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.390 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:13:00.390 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.390 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.390 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.390 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.390 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.390 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.390 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.390 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.390 INFO:teuthology.orchestra.run.smithi047.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-17T06:13:00.391 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-17T06:13:00.391 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.391 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.391 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.391 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.391 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.391 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.391 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.391 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.391 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.391 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.391 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.391 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.391 INFO:teuthology.orchestra.run.smithi047.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-17T06:13:00.392 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-17T06:13:00.392 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.392 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.392 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.392 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "[]", 2024-04-17T06:13:00.392 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.392 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.392 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.392 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.392 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.392 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.392 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.392 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.393 INFO:teuthology.orchestra.run.smithi047.stdout: "MULTICLUSTER_CONFIG": { 2024-04-17T06:13:00.393 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-17T06:13:00.393 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.393 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.393 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.393 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "{}", 2024-04-17T06:13:00.393 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.393 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.393 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.393 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.393 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.393 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.393 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.393 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.393 INFO:teuthology.orchestra.run.smithi047.stdout: "PROMETHEUS_API_HOST": { 2024-04-17T06:13:00.394 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-17T06:13:00.394 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.394 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.394 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.394 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.394 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.394 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.394 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.394 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.394 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.394 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.394 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.394 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.394 INFO:teuthology.orchestra.run.smithi047.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-17T06:13:00.394 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-17T06:13:00.395 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.395 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.395 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.395 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:13:00.395 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.395 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.395 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.395 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.395 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.395 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.395 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.395 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.395 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-17T06:13:00.395 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-17T06:13:00.395 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.396 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.396 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.396 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.396 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.396 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.396 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.396 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.396 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.396 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.396 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.396 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.396 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-17T06:13:00.396 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-17T06:13:00.396 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.396 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.396 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.397 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.397 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.397 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.397 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.397 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.397 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.397 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.397 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.397 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.397 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-17T06:13:00.397 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-17T06:13:00.397 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.397 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.398 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.398 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:13:00.398 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.398 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.398 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.398 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.398 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.398 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.398 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.398 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.398 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-17T06:13:00.398 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-17T06:13:00.398 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.398 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.399 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.399 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:13:00.399 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.399 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.399 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.399 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.399 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.399 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.399 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.399 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.399 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-17T06:13:00.399 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-17T06:13:00.399 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.399 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.399 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.399 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.400 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.400 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.400 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.400 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.400 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.400 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.400 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.400 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.400 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-17T06:13:00.400 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-17T06:13:00.400 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.400 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.400 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.400 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.400 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.401 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.401 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.401 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.401 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.401 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.401 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.401 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.401 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-17T06:13:00.401 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-17T06:13:00.401 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.401 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.401 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.401 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.401 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.401 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.402 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.402 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.402 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.402 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.402 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.402 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.402 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_ENABLED": { 2024-04-17T06:13:00.402 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-17T06:13:00.402 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.402 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.402 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.402 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:13:00.402 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.402 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.403 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.403 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.403 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.403 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.403 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.403 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.403 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-17T06:13:00.403 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-17T06:13:00.403 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.403 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.403 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.403 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-17T06:13:00.403 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.403 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.403 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.403 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.404 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.404 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.404 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.404 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.404 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-17T06:13:00.404 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-17T06:13:00.404 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:13:00.404 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.404 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.404 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-04-17T06:13:00.404 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.404 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.404 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.404 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.404 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.405 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.405 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.405 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.405 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-17T06:13:00.405 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-17T06:13:00.405 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:13:00.405 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.405 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.405 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "8", 2024-04-17T06:13:00.405 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.405 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.405 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.405 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.405 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.405 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.405 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.406 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.406 INFO:teuthology.orchestra.run.smithi047.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-17T06:13:00.406 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-17T06:13:00.406 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:13:00.406 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.406 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.406 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "45", 2024-04-17T06:13:00.406 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.406 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.406 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.406 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.406 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.406 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.406 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.406 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.407 INFO:teuthology.orchestra.run.smithi047.stdout: "RGW_API_ACCESS_KEY": { 2024-04-17T06:13:00.407 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-17T06:13:00.407 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.407 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.407 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.407 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.407 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.407 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.407 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.407 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.407 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.407 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.407 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.407 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.408 INFO:teuthology.orchestra.run.smithi047.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-17T06:13:00.408 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-17T06:13:00.408 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.408 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.408 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.408 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "admin", 2024-04-17T06:13:00.408 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.408 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.408 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.408 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.408 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.408 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.408 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.408 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.408 INFO:teuthology.orchestra.run.smithi047.stdout: "RGW_API_SECRET_KEY": { 2024-04-17T06:13:00.409 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-17T06:13:00.409 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.409 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.409 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.409 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.409 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.409 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.409 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.409 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.409 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.409 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.409 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.409 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.409 INFO:teuthology.orchestra.run.smithi047.stdout: "RGW_API_SSL_VERIFY": { 2024-04-17T06:13:00.409 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-17T06:13:00.410 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.410 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.410 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.410 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:13:00.410 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.410 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.410 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.410 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.410 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.410 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.410 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.410 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.410 INFO:teuthology.orchestra.run.smithi047.stdout: "UNSAFE_TLS_v1_2": { 2024-04-17T06:13:00.410 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-17T06:13:00.411 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.411 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.411 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.411 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.411 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.411 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.411 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.411 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.411 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.411 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.411 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.411 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.411 INFO:teuthology.orchestra.run.smithi047.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-17T06:13:00.411 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-17T06:13:00.411 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:13:00.411 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.412 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.412 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-04-17T06:13:00.412 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.412 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.412 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.412 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.412 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.412 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.412 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.412 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.412 INFO:teuthology.orchestra.run.smithi047.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-17T06:13:00.412 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-17T06:13:00.412 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:13:00.412 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.412 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.413 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-04-17T06:13:00.413 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.413 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.413 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.413 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.413 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.413 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.413 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.413 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.413 INFO:teuthology.orchestra.run.smithi047.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-17T06:13:00.413 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-17T06:13:00.413 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:13:00.413 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.413 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.413 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "5", 2024-04-17T06:13:00.413 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.414 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.414 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.414 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.414 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.414 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.414 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.414 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.414 INFO:teuthology.orchestra.run.smithi047.stdout: "cross_origin_url": { 2024-04-17T06:13:00.414 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "cross_origin_url", 2024-04-17T06:13:00.414 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.414 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.414 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.414 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.414 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.415 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.415 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.415 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.415 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.415 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.415 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.415 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.415 INFO:teuthology.orchestra.run.smithi047.stdout: "crt_file": { 2024-04-17T06:13:00.415 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "crt_file", 2024-04-17T06:13:00.415 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.415 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.415 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.415 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.415 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.415 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.415 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.416 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.416 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.416 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.416 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.416 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.416 INFO:teuthology.orchestra.run.smithi047.stdout: "debug": { 2024-04-17T06:13:00.416 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "debug", 2024-04-17T06:13:00.416 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.416 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.416 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.416 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.416 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.416 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.416 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.416 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Enable/disable debug options", 2024-04-17T06:13:00.417 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.417 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.417 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.417 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.417 INFO:teuthology.orchestra.run.smithi047.stdout: "jwt_token_ttl": { 2024-04-17T06:13:00.417 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "jwt_token_ttl", 2024-04-17T06:13:00.417 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:13:00.417 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.417 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.417 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "28800", 2024-04-17T06:13:00.417 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.417 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.417 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.417 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.417 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.418 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.418 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.418 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.418 INFO:teuthology.orchestra.run.smithi047.stdout: "key_file": { 2024-04-17T06:13:00.418 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "key_file", 2024-04-17T06:13:00.418 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.418 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.418 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.418 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.418 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.418 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.418 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.418 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.418 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.418 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.419 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.419 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.419 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-17T06:13:00.419 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-17T06:13:00.419 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.419 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.419 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.419 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.419 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.419 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.419 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.419 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.419 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.419 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.419 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.419 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.420 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.420 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.420 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.420 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.420 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.420 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.420 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.420 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-17T06:13:00.420 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-17T06:13:00.420 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.420 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.420 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.420 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.420 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.420 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.421 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.421 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.421 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.421 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.421 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.421 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.421 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-17T06:13:00.421 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-17T06:13:00.421 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.421 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.421 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.421 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-17T06:13:00.421 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.421 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.421 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.421 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.422 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.422 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.422 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.422 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.422 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.422 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.422 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.422 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.422 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.422 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.422 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.422 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-17T06:13:00.422 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-17T06:13:00.422 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.422 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.422 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.423 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.423 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.423 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.423 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.423 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.423 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.423 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.423 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.423 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.423 INFO:teuthology.orchestra.run.smithi047.stdout: "motd": { 2024-04-17T06:13:00.423 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "motd", 2024-04-17T06:13:00.423 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.423 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.423 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.423 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.424 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.424 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.424 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.424 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "The message of the day", 2024-04-17T06:13:00.424 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.424 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.424 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.424 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.424 INFO:teuthology.orchestra.run.smithi047.stdout: "redirect_resolve_ip_addr": { 2024-04-17T06:13:00.424 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "redirect_resolve_ip_addr", 2024-04-17T06:13:00.424 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.424 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.425 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.425 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.425 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.425 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.425 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.425 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.425 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.425 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.425 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.425 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.425 INFO:teuthology.orchestra.run.smithi047.stdout: "server_addr": { 2024-04-17T06:13:00.425 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "server_addr", 2024-04-17T06:13:00.425 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.425 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.425 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.426 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "::", 2024-04-17T06:13:00.426 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.426 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.426 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.426 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.426 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.426 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.426 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.426 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.426 INFO:teuthology.orchestra.run.smithi047.stdout: "server_port": { 2024-04-17T06:13:00.426 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "server_port", 2024-04-17T06:13:00.426 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:13:00.426 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.427 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.427 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "8080", 2024-04-17T06:13:00.427 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.427 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.427 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.427 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.427 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.427 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.427 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.427 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.427 INFO:teuthology.orchestra.run.smithi047.stdout: "ssl": { 2024-04-17T06:13:00.427 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ssl", 2024-04-17T06:13:00.427 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.427 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.427 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.428 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:13:00.428 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.428 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.428 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.428 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.428 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.428 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.428 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.428 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.428 INFO:teuthology.orchestra.run.smithi047.stdout: "ssl_server_port": { 2024-04-17T06:13:00.428 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ssl_server_port", 2024-04-17T06:13:00.429 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:13:00.429 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.429 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.429 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "8443", 2024-04-17T06:13:00.429 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.429 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.429 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.429 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.429 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.429 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.429 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.429 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.429 INFO:teuthology.orchestra.run.smithi047.stdout: "standby_behaviour": { 2024-04-17T06:13:00.430 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "standby_behaviour", 2024-04-17T06:13:00.430 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.430 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.430 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.430 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "redirect", 2024-04-17T06:13:00.430 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.430 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.430 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.430 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.430 INFO:teuthology.orchestra.run.smithi047.stdout: "redirect" 2024-04-17T06:13:00.430 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.430 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.430 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.430 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.430 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.431 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.431 INFO:teuthology.orchestra.run.smithi047.stdout: "standby_error_status_code": { 2024-04-17T06:13:00.431 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "standby_error_status_code", 2024-04-17T06:13:00.431 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:13:00.431 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.431 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.431 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "500", 2024-04-17T06:13:00.431 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "400", 2024-04-17T06:13:00.431 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "599", 2024-04-17T06:13:00.431 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.431 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.431 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.431 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.431 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.431 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.432 INFO:teuthology.orchestra.run.smithi047.stdout: "url_prefix": { 2024-04-17T06:13:00.432 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "url_prefix", 2024-04-17T06:13:00.432 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.432 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.432 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.432 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.432 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.432 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.432 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.432 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.432 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.432 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.432 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.432 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.432 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.432 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.432 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:13:00.433 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "devicehealth", 2024-04-17T06:13:00.433 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-17T06:13:00.433 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-17T06:13:00.433 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-17T06:13:00.433 INFO:teuthology.orchestra.run.smithi047.stdout: "enable_monitoring": { 2024-04-17T06:13:00.433 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "enable_monitoring", 2024-04-17T06:13:00.433 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.433 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.433 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.433 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:13:00.433 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.433 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.433 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.433 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "monitor device health metrics", 2024-04-17T06:13:00.433 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.433 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.434 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.434 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.434 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-17T06:13:00.434 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-17T06:13:00.434 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.434 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.434 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.434 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.434 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.434 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.434 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.434 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.434 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.434 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.434 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.434 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.435 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.435 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.435 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.435 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.435 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.435 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.435 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.435 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-17T06:13:00.435 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-17T06:13:00.435 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.435 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.435 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.435 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.435 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.435 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.435 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.435 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.436 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.436 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.436 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.436 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.436 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-17T06:13:00.436 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-17T06:13:00.436 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.436 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.436 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.436 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-17T06:13:00.436 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.436 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.436 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.436 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.436 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.436 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.437 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.437 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.437 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.437 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.437 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.437 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.437 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.437 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.437 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.437 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-17T06:13:00.437 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-17T06:13:00.437 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.437 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.437 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.437 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.437 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.437 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.438 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.438 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.438 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.438 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.438 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.438 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.438 INFO:teuthology.orchestra.run.smithi047.stdout: "mark_out_threshold": { 2024-04-17T06:13:00.438 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "mark_out_threshold", 2024-04-17T06:13:00.438 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-17T06:13:00.438 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.438 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.438 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "2419200", 2024-04-17T06:13:00.438 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.438 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.438 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.438 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-17T06:13:00.439 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.439 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.439 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.439 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.439 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_name": { 2024-04-17T06:13:00.439 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "pool_name", 2024-04-17T06:13:00.439 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.439 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.439 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.439 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "device_health_metrics", 2024-04-17T06:13:00.439 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.440 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.440 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.440 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-17T06:13:00.440 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.440 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.440 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.440 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.440 INFO:teuthology.orchestra.run.smithi047.stdout: "retention_period": { 2024-04-17T06:13:00.440 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "retention_period", 2024-04-17T06:13:00.440 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-17T06:13:00.440 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.440 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.440 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "15552000", 2024-04-17T06:13:00.440 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.440 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.441 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.441 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how long to retain device health metrics", 2024-04-17T06:13:00.441 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.441 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.441 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.441 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.441 INFO:teuthology.orchestra.run.smithi047.stdout: "scrape_frequency": { 2024-04-17T06:13:00.441 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "scrape_frequency", 2024-04-17T06:13:00.441 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-17T06:13:00.441 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.441 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.441 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "86400", 2024-04-17T06:13:00.441 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.441 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.442 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.442 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-17T06:13:00.442 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.442 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.442 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.442 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.442 INFO:teuthology.orchestra.run.smithi047.stdout: "self_heal": { 2024-04-17T06:13:00.442 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "self_heal", 2024-04-17T06:13:00.442 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.442 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.442 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.442 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:13:00.442 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.442 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.442 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.442 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-17T06:13:00.443 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.443 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.443 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.443 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.443 INFO:teuthology.orchestra.run.smithi047.stdout: "sleep_interval": { 2024-04-17T06:13:00.443 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sleep_interval", 2024-04-17T06:13:00.443 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-17T06:13:00.443 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.443 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.443 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "600", 2024-04-17T06:13:00.443 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.443 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.443 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.443 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how frequently to wake up and check device health", 2024-04-17T06:13:00.443 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.443 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.444 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.444 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.444 INFO:teuthology.orchestra.run.smithi047.stdout: "warn_threshold": { 2024-04-17T06:13:00.444 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "warn_threshold", 2024-04-17T06:13:00.444 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-17T06:13:00.444 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.444 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.444 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "7257600", 2024-04-17T06:13:00.444 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.444 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.444 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.444 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-17T06:13:00.444 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.444 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.444 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.444 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.445 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.445 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.445 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:13:00.445 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "diskprediction_local", 2024-04-17T06:13:00.445 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-17T06:13:00.445 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-17T06:13:00.445 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-17T06:13:00.445 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-17T06:13:00.445 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-17T06:13:00.445 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.445 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.445 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.445 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.445 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.445 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.445 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.446 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.446 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.446 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.446 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.446 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.446 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.446 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.446 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.446 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.446 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.446 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.446 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.446 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-17T06:13:00.447 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-17T06:13:00.447 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.447 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.447 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.447 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.447 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.447 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.447 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.447 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.447 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.447 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.447 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.447 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.447 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-17T06:13:00.447 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-17T06:13:00.448 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.448 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.448 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.448 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-17T06:13:00.448 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.448 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.448 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.448 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.448 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.448 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.448 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.448 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.448 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.448 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.448 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.449 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.449 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.449 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.449 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.449 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-17T06:13:00.449 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-17T06:13:00.449 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.449 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.449 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.449 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.449 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.449 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.449 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.449 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.449 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.449 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.450 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.450 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.450 INFO:teuthology.orchestra.run.smithi047.stdout: "predict_interval": { 2024-04-17T06:13:00.450 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "predict_interval", 2024-04-17T06:13:00.450 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.450 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.450 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.450 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "86400", 2024-04-17T06:13:00.450 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.450 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.450 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.450 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.450 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.450 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.450 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.450 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.451 INFO:teuthology.orchestra.run.smithi047.stdout: "predictor_model": { 2024-04-17T06:13:00.451 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "predictor_model", 2024-04-17T06:13:00.451 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.451 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.451 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.451 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "prophetstor", 2024-04-17T06:13:00.451 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.451 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.451 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.451 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.451 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.451 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.451 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.451 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.451 INFO:teuthology.orchestra.run.smithi047.stdout: "sleep_interval": { 2024-04-17T06:13:00.451 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sleep_interval", 2024-04-17T06:13:00.452 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.452 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.452 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.452 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "600", 2024-04-17T06:13:00.452 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.452 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.452 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.452 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.452 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.452 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.452 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.452 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.452 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.452 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.452 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:13:00.452 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "influx", 2024-04-17T06:13:00.453 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": false, 2024-04-17T06:13:00.453 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "influxdb python module not found", 2024-04-17T06:13:00.453 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-17T06:13:00.453 INFO:teuthology.orchestra.run.smithi047.stdout: "batch_size": { 2024-04-17T06:13:00.453 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "batch_size", 2024-04-17T06:13:00.453 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:13:00.453 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.453 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.453 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "5000", 2024-04-17T06:13:00.453 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.453 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.453 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.453 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-17T06:13:00.453 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.453 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.454 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.454 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.454 INFO:teuthology.orchestra.run.smithi047.stdout: "database": { 2024-04-17T06:13:00.454 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "database", 2024-04-17T06:13:00.454 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.454 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.454 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.454 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "ceph", 2024-04-17T06:13:00.454 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.454 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.454 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.454 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-17T06:13:00.454 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.454 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.454 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.454 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.454 INFO:teuthology.orchestra.run.smithi047.stdout: "hostname": { 2024-04-17T06:13:00.455 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "hostname", 2024-04-17T06:13:00.455 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.455 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.455 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.455 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.455 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.455 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.455 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.455 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "InfluxDB server hostname", 2024-04-17T06:13:00.455 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.455 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.455 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.455 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.455 INFO:teuthology.orchestra.run.smithi047.stdout: "interval": { 2024-04-17T06:13:00.455 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "interval", 2024-04-17T06:13:00.455 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-17T06:13:00.456 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.456 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.456 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "30", 2024-04-17T06:13:00.456 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "5", 2024-04-17T06:13:00.456 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.456 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.456 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-17T06:13:00.456 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.456 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.456 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.456 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.456 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-17T06:13:00.456 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-17T06:13:00.456 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.456 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.457 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.457 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.457 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.457 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.457 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.457 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.457 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.457 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.457 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.457 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.457 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.457 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.457 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.457 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.457 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.457 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.458 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.458 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-17T06:13:00.458 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-17T06:13:00.458 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.458 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.458 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.458 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.458 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.458 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.458 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.458 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.458 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.458 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.458 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.458 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.458 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-17T06:13:00.459 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-17T06:13:00.459 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.459 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.459 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.459 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-17T06:13:00.459 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.459 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.459 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.459 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.459 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.459 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.459 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.459 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.459 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.459 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.460 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.460 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.460 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.460 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.460 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.460 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-17T06:13:00.460 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-17T06:13:00.460 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.460 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.460 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.460 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.460 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.460 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.460 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.460 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.460 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.461 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.461 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.461 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.461 INFO:teuthology.orchestra.run.smithi047.stdout: "password": { 2024-04-17T06:13:00.461 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "password", 2024-04-17T06:13:00.461 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.461 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.461 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.461 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.461 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.461 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.461 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.461 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "password of InfluxDB server user", 2024-04-17T06:13:00.461 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.461 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.461 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.461 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.462 INFO:teuthology.orchestra.run.smithi047.stdout: "port": { 2024-04-17T06:13:00.462 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "port", 2024-04-17T06:13:00.462 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:13:00.462 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.462 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.462 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "8086", 2024-04-17T06:13:00.462 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.462 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.462 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.462 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "InfluxDB server port", 2024-04-17T06:13:00.462 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.462 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.462 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.462 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.462 INFO:teuthology.orchestra.run.smithi047.stdout: "ssl": { 2024-04-17T06:13:00.462 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ssl", 2024-04-17T06:13:00.463 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.463 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.463 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.463 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "false", 2024-04-17T06:13:00.463 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.463 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.463 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.463 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-17T06:13:00.463 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.463 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.463 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.463 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.463 INFO:teuthology.orchestra.run.smithi047.stdout: "threads": { 2024-04-17T06:13:00.463 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "threads", 2024-04-17T06:13:00.463 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:13:00.463 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.464 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.464 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "5", 2024-04-17T06:13:00.464 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "1", 2024-04-17T06:13:00.464 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "32", 2024-04-17T06:13:00.464 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.464 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-17T06:13:00.464 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.464 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.464 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.464 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.464 INFO:teuthology.orchestra.run.smithi047.stdout: "username": { 2024-04-17T06:13:00.464 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "username", 2024-04-17T06:13:00.464 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.464 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.464 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.464 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.465 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.465 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.465 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.465 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "username of InfluxDB server user", 2024-04-17T06:13:00.465 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.465 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.465 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.465 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.465 INFO:teuthology.orchestra.run.smithi047.stdout: "verify_ssl": { 2024-04-17T06:13:00.465 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "verify_ssl", 2024-04-17T06:13:00.465 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.465 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.465 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.465 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "true", 2024-04-17T06:13:00.465 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.465 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.466 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.466 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-17T06:13:00.466 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.466 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.466 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.466 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.466 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.466 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.466 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:13:00.466 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "insights", 2024-04-17T06:13:00.466 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-17T06:13:00.466 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-17T06:13:00.466 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-17T06:13:00.466 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-17T06:13:00.466 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-17T06:13:00.466 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.467 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.467 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.467 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.467 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.467 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.467 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.467 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.467 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.467 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.467 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.467 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.467 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.467 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.467 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.467 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.467 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.468 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.468 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.468 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-17T06:13:00.468 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-17T06:13:00.468 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.468 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.468 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.468 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.468 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.468 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.468 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.468 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.468 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.468 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.468 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.468 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.468 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-17T06:13:00.469 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-17T06:13:00.469 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.469 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.469 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.469 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-17T06:13:00.469 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.469 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.469 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.469 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.469 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.469 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.469 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.469 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.469 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.469 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.469 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.469 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.470 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.470 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.470 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.470 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-17T06:13:00.470 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-17T06:13:00.470 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.470 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.470 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.470 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.470 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.470 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.470 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.470 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.470 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.471 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.471 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.471 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.471 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.471 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.471 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:13:00.471 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "iostat", 2024-04-17T06:13:00.471 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-17T06:13:00.471 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-17T06:13:00.471 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-17T06:13:00.471 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-17T06:13:00.471 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-17T06:13:00.471 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.471 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.471 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.471 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.472 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.472 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.472 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.472 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.472 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.472 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.472 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.472 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.472 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.472 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.472 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.472 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.472 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.472 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.472 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.472 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-17T06:13:00.473 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-17T06:13:00.473 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.473 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.473 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.473 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.473 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.473 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.473 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.473 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.473 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.473 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.473 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.473 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.473 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-17T06:13:00.474 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-17T06:13:00.474 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.474 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.474 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.474 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-17T06:13:00.474 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.474 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.474 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.474 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.474 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.474 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.474 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.474 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.474 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.474 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.475 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.475 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.475 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.475 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.475 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.475 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-17T06:13:00.475 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-17T06:13:00.475 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.475 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.475 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.475 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.475 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.475 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.475 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.475 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.475 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.476 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.476 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.476 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.476 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.476 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.476 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:13:00.476 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "k8sevents", 2024-04-17T06:13:00.476 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-17T06:13:00.476 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-17T06:13:00.476 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-17T06:13:00.476 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_event_retention_days": { 2024-04-17T06:13:00.476 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ceph_event_retention_days", 2024-04-17T06:13:00.476 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:13:00.476 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.476 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.476 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "7", 2024-04-17T06:13:00.477 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.477 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.477 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.477 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-17T06:13:00.477 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.477 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.477 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.477 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.477 INFO:teuthology.orchestra.run.smithi047.stdout: "config_check_secs": { 2024-04-17T06:13:00.477 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "config_check_secs", 2024-04-17T06:13:00.477 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:13:00.477 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.477 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.477 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-04-17T06:13:00.477 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "10", 2024-04-17T06:13:00.477 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.477 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.478 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-17T06:13:00.478 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.478 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.478 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.478 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.478 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-17T06:13:00.478 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-17T06:13:00.478 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.478 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.478 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.478 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.478 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.478 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.478 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.478 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.479 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.479 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.479 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.479 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.479 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.479 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.479 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.479 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.479 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.479 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.479 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.479 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-17T06:13:00.479 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-17T06:13:00.479 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.479 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.479 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.479 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.480 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.480 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.480 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.480 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.480 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.480 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.480 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.480 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.480 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-17T06:13:00.480 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-17T06:13:00.480 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.480 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.480 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.480 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-17T06:13:00.480 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.480 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.481 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.481 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.481 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.481 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.481 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.481 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.481 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.481 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.481 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.481 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.481 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.481 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.481 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.481 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-17T06:13:00.481 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-17T06:13:00.481 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.481 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.482 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.482 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.482 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.482 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.482 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.482 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.482 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.482 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.482 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.482 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.482 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.482 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.482 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:13:00.482 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "localpool", 2024-04-17T06:13:00.482 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-17T06:13:00.482 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-17T06:13:00.483 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-17T06:13:00.483 INFO:teuthology.orchestra.run.smithi047.stdout: "failure_domain": { 2024-04-17T06:13:00.483 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "failure_domain", 2024-04-17T06:13:00.483 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.483 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.483 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.483 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "host", 2024-04-17T06:13:00.483 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.483 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.483 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.483 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "failure domain for any created local pool", 2024-04-17T06:13:00.483 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-17T06:13:00.483 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.483 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.483 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.483 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-17T06:13:00.483 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-17T06:13:00.484 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.484 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.484 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.484 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.484 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.484 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.484 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.484 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.484 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.484 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.484 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.484 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.484 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.484 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.484 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.484 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.485 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.485 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.485 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.485 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-17T06:13:00.485 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-17T06:13:00.485 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.485 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.485 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.485 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.485 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.485 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.485 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.485 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.485 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.485 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.485 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.486 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.486 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-17T06:13:00.486 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-17T06:13:00.486 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.486 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.486 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.486 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-17T06:13:00.486 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.486 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.486 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.486 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.486 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.486 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.486 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.486 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.486 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.487 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.487 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.487 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.487 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.487 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.487 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.487 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-17T06:13:00.487 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-17T06:13:00.487 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.487 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.487 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.487 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.487 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.487 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.487 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.487 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.487 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.488 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.488 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.488 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.488 INFO:teuthology.orchestra.run.smithi047.stdout: "min_size": { 2024-04-17T06:13:00.488 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "min_size", 2024-04-17T06:13:00.488 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:13:00.488 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.488 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.488 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.488 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.488 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.488 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.488 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "default min_size for any created local pool", 2024-04-17T06:13:00.488 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-17T06:13:00.488 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.488 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.489 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.489 INFO:teuthology.orchestra.run.smithi047.stdout: "num_rep": { 2024-04-17T06:13:00.489 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "num_rep", 2024-04-17T06:13:00.489 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:13:00.489 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.489 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.489 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "3", 2024-04-17T06:13:00.489 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.489 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.489 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.489 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "default replica count for any created local pool", 2024-04-17T06:13:00.489 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.489 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.490 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.490 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.490 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num": { 2024-04-17T06:13:00.490 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "pg_num", 2024-04-17T06:13:00.490 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:13:00.490 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.490 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.490 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "128", 2024-04-17T06:13:00.490 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.490 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.490 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.490 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "default pg_num for any created local pool", 2024-04-17T06:13:00.490 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.490 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.490 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.491 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.491 INFO:teuthology.orchestra.run.smithi047.stdout: "prefix": { 2024-04-17T06:13:00.491 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "prefix", 2024-04-17T06:13:00.491 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.491 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.491 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.491 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.491 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.491 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.491 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.491 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "name prefix for any created local pool", 2024-04-17T06:13:00.491 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.491 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.491 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.492 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.492 INFO:teuthology.orchestra.run.smithi047.stdout: "subtree": { 2024-04-17T06:13:00.492 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "subtree", 2024-04-17T06:13:00.492 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.492 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.492 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.492 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "rack", 2024-04-17T06:13:00.492 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.492 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.492 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.492 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-17T06:13:00.492 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-17T06:13:00.492 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.492 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.493 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.493 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.493 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.493 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:13:00.493 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "mds_autoscaler", 2024-04-17T06:13:00.493 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-17T06:13:00.493 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-17T06:13:00.493 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-17T06:13:00.493 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-17T06:13:00.493 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-17T06:13:00.493 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.493 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.493 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.493 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.493 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.493 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.493 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.494 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.494 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.494 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.494 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.494 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.494 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.494 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.494 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.494 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.494 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.494 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.494 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.494 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-17T06:13:00.494 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-17T06:13:00.494 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.494 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.495 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.495 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.495 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.495 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.495 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.495 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.495 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.495 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.495 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.495 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.495 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-17T06:13:00.495 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-17T06:13:00.495 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.495 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.495 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.495 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-17T06:13:00.496 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.496 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.496 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.496 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.496 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.496 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.496 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.496 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.496 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.496 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.496 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.496 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.496 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.496 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.496 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.496 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-17T06:13:00.497 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-17T06:13:00.497 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.497 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.497 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.497 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.497 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.497 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.497 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.497 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.497 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.497 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.497 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.497 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.497 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.498 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.498 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:13:00.498 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "mirroring", 2024-04-17T06:13:00.498 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-17T06:13:00.498 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-17T06:13:00.498 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-17T06:13:00.498 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-17T06:13:00.498 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-17T06:13:00.498 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.498 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.498 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.498 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.498 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.498 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.498 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.499 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.499 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.499 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.499 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.499 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.499 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.499 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.499 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.499 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.499 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.499 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.499 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.499 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-17T06:13:00.499 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-17T06:13:00.499 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.499 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.500 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.500 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.500 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.500 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.500 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.500 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.500 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.500 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.500 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.500 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.500 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-17T06:13:00.500 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-17T06:13:00.500 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.500 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.500 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.500 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-17T06:13:00.501 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.501 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.501 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.501 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.501 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.501 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.501 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.501 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.501 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.501 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.501 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.501 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.501 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.501 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.501 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.501 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-17T06:13:00.502 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-17T06:13:00.502 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.502 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.502 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.502 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.502 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.502 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.502 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.502 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.502 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.502 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.502 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.502 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.502 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.502 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.502 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:13:00.503 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "nfs", 2024-04-17T06:13:00.503 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-17T06:13:00.503 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-17T06:13:00.503 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-17T06:13:00.503 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-17T06:13:00.503 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-17T06:13:00.503 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.503 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.503 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.503 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.503 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.503 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.503 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.503 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.503 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.504 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.504 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.504 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.504 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.504 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.504 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.504 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.504 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.504 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.504 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.504 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-17T06:13:00.504 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-17T06:13:00.504 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.504 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.504 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.504 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.505 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.505 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.505 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.505 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.505 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.505 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.505 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.505 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.505 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-17T06:13:00.505 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-17T06:13:00.505 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.505 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.505 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.505 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-17T06:13:00.505 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.505 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.506 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.506 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.506 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.506 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.506 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.506 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.506 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.506 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.506 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.506 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.506 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.506 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.506 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.506 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-17T06:13:00.506 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-17T06:13:00.507 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.507 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.507 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.507 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.507 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.507 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.507 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.507 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.507 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.507 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.507 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.507 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.507 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.507 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.507 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:13:00.507 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "orchestrator", 2024-04-17T06:13:00.508 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-17T06:13:00.508 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-17T06:13:00.508 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-17T06:13:00.508 INFO:teuthology.orchestra.run.smithi047.stdout: "fail_fs": { 2024-04-17T06:13:00.508 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "fail_fs", 2024-04-17T06:13:00.508 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.508 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.508 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.508 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.508 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.508 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.508 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.508 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-17T06:13:00.508 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.508 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.508 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.508 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.509 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-17T06:13:00.509 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-17T06:13:00.509 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.509 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.509 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.509 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.509 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.509 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.509 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.509 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.509 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.509 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.509 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.509 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.509 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.510 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.510 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.510 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.510 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.510 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.510 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.510 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-17T06:13:00.510 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-17T06:13:00.510 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.510 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.510 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.510 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.510 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.510 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.510 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.511 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.511 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.511 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.511 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.511 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.511 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-17T06:13:00.511 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-17T06:13:00.511 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.511 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.511 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.511 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-17T06:13:00.511 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.511 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.511 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.511 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.511 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.511 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.512 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.512 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.512 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.512 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.512 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.512 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.512 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.512 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.512 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.512 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-17T06:13:00.512 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-17T06:13:00.512 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.512 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.512 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.512 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.512 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.513 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.513 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.513 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.513 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.513 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.513 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.513 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.513 INFO:teuthology.orchestra.run.smithi047.stdout: "orchestrator": { 2024-04-17T06:13:00.513 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "orchestrator", 2024-04-17T06:13:00.513 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.513 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.513 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.513 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.513 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.513 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.513 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.513 INFO:teuthology.orchestra.run.smithi047.stdout: "cephadm", 2024-04-17T06:13:00.514 INFO:teuthology.orchestra.run.smithi047.stdout: "rook", 2024-04-17T06:13:00.514 INFO:teuthology.orchestra.run.smithi047.stdout: "test_orchestrator" 2024-04-17T06:13:00.514 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.514 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Orchestrator backend", 2024-04-17T06:13:00.514 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.514 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.514 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.514 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.514 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.514 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.514 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:13:00.514 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "osd_perf_query", 2024-04-17T06:13:00.514 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-17T06:13:00.514 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-17T06:13:00.514 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-17T06:13:00.514 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-17T06:13:00.515 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-17T06:13:00.515 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.515 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.515 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.515 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.515 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.515 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.515 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.515 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.515 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.515 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.515 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.515 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.515 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.515 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.515 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.516 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.516 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.516 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.516 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.516 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-17T06:13:00.516 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-17T06:13:00.516 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.516 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.516 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.516 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.516 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.516 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.516 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.516 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.516 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.516 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.516 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.517 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.517 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-17T06:13:00.517 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-17T06:13:00.517 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.517 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.517 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.517 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-17T06:13:00.517 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.517 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.517 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.517 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.517 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.517 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.517 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.517 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.517 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.518 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.518 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.518 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.518 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.518 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.518 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.518 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-17T06:13:00.518 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-17T06:13:00.518 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.518 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.518 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.518 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.518 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.518 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.518 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.519 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.519 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.519 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.519 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.519 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.519 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.519 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.519 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:13:00.519 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "osd_support", 2024-04-17T06:13:00.519 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-17T06:13:00.519 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-17T06:13:00.519 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-17T06:13:00.519 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-17T06:13:00.519 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-17T06:13:00.519 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.519 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.519 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.520 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.520 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.520 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.520 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.520 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.520 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.520 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.520 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.520 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.520 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.520 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.520 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.520 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.520 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.520 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.520 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.521 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-17T06:13:00.521 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-17T06:13:00.521 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.521 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.521 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.521 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.521 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.521 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.521 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.521 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.521 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.521 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.521 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.521 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.521 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-17T06:13:00.521 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-17T06:13:00.522 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.522 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.522 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.522 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-17T06:13:00.522 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.522 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.522 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.522 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.522 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.522 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.522 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.522 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.522 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.522 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.522 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.522 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.523 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.523 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.523 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.523 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-17T06:13:00.523 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-17T06:13:00.523 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.523 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.523 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.523 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.523 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.523 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.523 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.523 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.523 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.523 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.523 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.523 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.524 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.524 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.524 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:13:00.524 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "pg_autoscaler", 2024-04-17T06:13:00.524 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-17T06:13:00.524 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-17T06:13:00.524 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-17T06:13:00.524 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-17T06:13:00.524 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-17T06:13:00.524 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.524 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.524 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.524 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.525 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.525 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.525 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.525 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.525 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.525 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.525 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.525 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.525 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.525 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.525 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.525 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.525 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.525 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.525 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.525 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-17T06:13:00.526 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-17T06:13:00.526 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.526 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.526 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.526 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.526 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.526 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.526 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.526 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.526 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.526 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.526 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.526 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.526 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-17T06:13:00.527 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-17T06:13:00.527 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.527 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.527 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.527 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-17T06:13:00.527 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.527 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.527 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.527 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.527 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.527 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.527 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.527 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.527 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.527 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.527 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.528 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.528 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.528 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.528 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.528 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-17T06:13:00.528 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-17T06:13:00.528 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.528 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.528 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.528 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.528 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.528 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.528 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.529 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.529 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.529 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.529 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.529 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.529 INFO:teuthology.orchestra.run.smithi047.stdout: "sleep_interval": { 2024-04-17T06:13:00.529 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sleep_interval", 2024-04-17T06:13:00.529 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-17T06:13:00.529 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.529 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.529 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "60", 2024-04-17T06:13:00.529 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.529 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.529 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.530 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.530 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.530 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.530 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.530 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.530 INFO:teuthology.orchestra.run.smithi047.stdout: "threshold": { 2024-04-17T06:13:00.530 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "threshold", 2024-04-17T06:13:00.530 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "float", 2024-04-17T06:13:00.530 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.530 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.530 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "3.0", 2024-04-17T06:13:00.530 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "1.0", 2024-04-17T06:13:00.530 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.530 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.530 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "scaling threshold", 2024-04-17T06:13:00.531 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-17T06:13:00.531 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.531 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.531 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.531 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.531 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.531 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:13:00.531 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "progress", 2024-04-17T06:13:00.531 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-17T06:13:00.531 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-17T06:13:00.531 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-17T06:13:00.531 INFO:teuthology.orchestra.run.smithi047.stdout: "allow_pg_recovery_event": { 2024-04-17T06:13:00.531 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "allow_pg_recovery_event", 2024-04-17T06:13:00.531 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.531 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.532 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.532 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.532 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.532 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.532 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.532 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-17T06:13:00.532 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.532 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.532 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.533 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.533 INFO:teuthology.orchestra.run.smithi047.stdout: "enabled": { 2024-04-17T06:13:00.533 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "enabled", 2024-04-17T06:13:00.533 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.533 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.533 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.533 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:13:00.533 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.533 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.534 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.534 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.534 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.534 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.534 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.534 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.534 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-17T06:13:00.534 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-17T06:13:00.535 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.535 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.535 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.535 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.535 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.535 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.535 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.535 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.535 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.536 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.536 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.536 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.536 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.536 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.536 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.536 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.536 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.536 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.537 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.537 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-17T06:13:00.537 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-17T06:13:00.537 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.537 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.537 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.537 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.537 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.538 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.538 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.538 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.538 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.538 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.538 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.538 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.538 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-17T06:13:00.538 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-17T06:13:00.538 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.539 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.539 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.539 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-17T06:13:00.539 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.539 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.539 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.539 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.539 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.539 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.539 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.539 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.539 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.539 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.539 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.540 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.540 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.540 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.540 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.540 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-17T06:13:00.540 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-17T06:13:00.540 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.540 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.540 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.540 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.540 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.540 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.540 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.540 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.540 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.540 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.541 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.541 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.541 INFO:teuthology.orchestra.run.smithi047.stdout: "max_completed_events": { 2024-04-17T06:13:00.541 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "max_completed_events", 2024-04-17T06:13:00.541 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:13:00.541 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.541 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.541 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "50", 2024-04-17T06:13:00.541 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.541 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.541 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.541 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "number of past completed events to remember", 2024-04-17T06:13:00.541 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.541 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.541 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.541 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.542 INFO:teuthology.orchestra.run.smithi047.stdout: "sleep_interval": { 2024-04-17T06:13:00.542 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sleep_interval", 2024-04-17T06:13:00.542 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-17T06:13:00.542 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.542 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.542 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "5", 2024-04-17T06:13:00.542 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.542 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.542 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.542 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how long the module is going to sleep", 2024-04-17T06:13:00.542 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.542 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.542 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.542 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.542 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.542 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.543 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:13:00.543 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "prometheus", 2024-04-17T06:13:00.543 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-17T06:13:00.543 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-17T06:13:00.543 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-17T06:13:00.543 INFO:teuthology.orchestra.run.smithi047.stdout: "cache": { 2024-04-17T06:13:00.543 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "cache", 2024-04-17T06:13:00.543 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.543 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.543 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.543 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:13:00.543 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.543 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.543 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.543 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.543 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.544 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.544 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.544 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.544 INFO:teuthology.orchestra.run.smithi047.stdout: "exclude_perf_counters": { 2024-04-17T06:13:00.544 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "exclude_perf_counters", 2024-04-17T06:13:00.544 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.544 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.544 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.544 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:13:00.544 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.544 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.544 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.544 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-17T06:13:00.544 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-17T06:13:00.544 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.544 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.544 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.545 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-17T06:13:00.545 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-17T06:13:00.545 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.545 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.545 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.545 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.545 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.545 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.545 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.545 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.545 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.545 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.545 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.545 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.545 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.545 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.546 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.546 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.546 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.546 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.546 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.546 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-17T06:13:00.546 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-17T06:13:00.546 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.546 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.546 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.546 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.546 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.546 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.546 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.546 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.546 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.547 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.547 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.547 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.547 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-17T06:13:00.547 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-17T06:13:00.547 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.547 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.547 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.547 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-17T06:13:00.547 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.547 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.547 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.547 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.547 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.548 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.548 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.548 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.548 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.548 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.548 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.548 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.548 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.548 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.548 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.548 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-17T06:13:00.548 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-17T06:13:00.548 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.548 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.548 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.548 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.549 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.549 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.549 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.549 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.549 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.549 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.549 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.549 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.549 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_stats_pools": { 2024-04-17T06:13:00.549 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rbd_stats_pools", 2024-04-17T06:13:00.549 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.549 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.549 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.549 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.549 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.549 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.549 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.550 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.550 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.550 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.550 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.550 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.550 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-17T06:13:00.550 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-17T06:13:00.550 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:13:00.550 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.550 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.550 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "300", 2024-04-17T06:13:00.550 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.550 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.550 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.550 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.551 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.551 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.551 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.551 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.551 INFO:teuthology.orchestra.run.smithi047.stdout: "scrape_interval": { 2024-04-17T06:13:00.551 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "scrape_interval", 2024-04-17T06:13:00.551 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "float", 2024-04-17T06:13:00.551 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.551 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.551 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "15.0", 2024-04-17T06:13:00.551 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.551 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.551 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.551 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.551 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.551 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.551 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.552 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.552 INFO:teuthology.orchestra.run.smithi047.stdout: "server_addr": { 2024-04-17T06:13:00.552 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "server_addr", 2024-04-17T06:13:00.552 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.552 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.552 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.552 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "::", 2024-04-17T06:13:00.552 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.552 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.552 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.552 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-17T06:13:00.552 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.552 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.552 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.552 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.553 INFO:teuthology.orchestra.run.smithi047.stdout: "server_port": { 2024-04-17T06:13:00.553 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "server_port", 2024-04-17T06:13:00.553 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:13:00.553 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.553 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.553 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "9283", 2024-04-17T06:13:00.553 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.553 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.553 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.553 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-17T06:13:00.553 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.553 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.553 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.553 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.553 INFO:teuthology.orchestra.run.smithi047.stdout: "stale_cache_strategy": { 2024-04-17T06:13:00.553 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "stale_cache_strategy", 2024-04-17T06:13:00.553 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.554 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.554 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.554 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "log", 2024-04-17T06:13:00.554 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.554 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.554 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.554 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.554 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.554 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.554 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.554 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.554 INFO:teuthology.orchestra.run.smithi047.stdout: "standby_behaviour": { 2024-04-17T06:13:00.554 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "standby_behaviour", 2024-04-17T06:13:00.554 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.554 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.555 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.555 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "default", 2024-04-17T06:13:00.555 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.555 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.555 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.555 INFO:teuthology.orchestra.run.smithi047.stdout: "default", 2024-04-17T06:13:00.555 INFO:teuthology.orchestra.run.smithi047.stdout: "error" 2024-04-17T06:13:00.555 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.555 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.555 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.555 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.555 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.555 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.555 INFO:teuthology.orchestra.run.smithi047.stdout: "standby_error_status_code": { 2024-04-17T06:13:00.555 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "standby_error_status_code", 2024-04-17T06:13:00.555 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:13:00.556 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.556 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.556 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "500", 2024-04-17T06:13:00.556 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "400", 2024-04-17T06:13:00.556 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "599", 2024-04-17T06:13:00.556 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.556 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.556 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.556 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.556 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.556 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.556 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.556 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.556 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:13:00.556 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rbd_support", 2024-04-17T06:13:00.556 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-17T06:13:00.557 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-17T06:13:00.557 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-17T06:13:00.557 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-17T06:13:00.557 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-17T06:13:00.557 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.557 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.557 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.557 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.557 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.557 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.557 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.557 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.557 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.557 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.558 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.558 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.558 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.558 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.558 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.558 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.559 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.559 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.559 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.559 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-17T06:13:00.559 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-17T06:13:00.559 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.559 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.559 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.560 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.560 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.560 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.560 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.560 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.560 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.560 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.561 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.561 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.561 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-17T06:13:00.561 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-17T06:13:00.561 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.561 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.561 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.561 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-17T06:13:00.562 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.562 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.562 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.562 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.562 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.562 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.562 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.562 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.562 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.562 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.562 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.562 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.562 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.563 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.563 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.563 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-17T06:13:00.563 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-17T06:13:00.563 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.563 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.563 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.563 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.563 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.563 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.563 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.563 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.563 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.563 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.563 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.563 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.564 INFO:teuthology.orchestra.run.smithi047.stdout: "max_concurrent_snap_create": { 2024-04-17T06:13:00.564 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "max_concurrent_snap_create", 2024-04-17T06:13:00.564 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:13:00.564 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.564 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.564 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-04-17T06:13:00.564 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.564 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.564 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.564 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.564 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.564 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.564 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.564 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.564 INFO:teuthology.orchestra.run.smithi047.stdout: "mirror_snapshot_schedule": { 2024-04-17T06:13:00.564 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "mirror_snapshot_schedule", 2024-04-17T06:13:00.564 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.565 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.565 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.565 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.565 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.565 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.565 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.565 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.565 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.565 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.565 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.565 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.565 INFO:teuthology.orchestra.run.smithi047.stdout: "trash_purge_schedule": { 2024-04-17T06:13:00.565 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "trash_purge_schedule", 2024-04-17T06:13:00.565 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.565 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.566 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.566 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.566 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.566 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.566 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.566 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.566 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.566 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.566 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.566 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.566 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.566 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.566 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:13:00.567 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "restful", 2024-04-17T06:13:00.567 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-17T06:13:00.567 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-17T06:13:00.567 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-17T06:13:00.567 INFO:teuthology.orchestra.run.smithi047.stdout: "enable_auth": { 2024-04-17T06:13:00.567 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "enable_auth", 2024-04-17T06:13:00.567 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.567 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.567 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.567 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:13:00.567 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.567 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.567 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.567 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.567 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.567 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.568 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.568 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.568 INFO:teuthology.orchestra.run.smithi047.stdout: "key_file": { 2024-04-17T06:13:00.568 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "key_file", 2024-04-17T06:13:00.568 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.568 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.568 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.568 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.568 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.568 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.568 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.568 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.568 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.568 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.568 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.568 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.569 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-17T06:13:00.569 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-17T06:13:00.569 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.569 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.569 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.569 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.569 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.569 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.569 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.569 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.569 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.569 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.569 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.569 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.569 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.569 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.570 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.570 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.570 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.570 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.570 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.570 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-17T06:13:00.570 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-17T06:13:00.570 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.570 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.570 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.570 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.570 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.571 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.571 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.571 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.571 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.571 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.571 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.571 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.571 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-17T06:13:00.571 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-17T06:13:00.571 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.571 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.571 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.571 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-17T06:13:00.571 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.571 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.572 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.572 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.572 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.572 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.572 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.572 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.572 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.572 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.572 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.572 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.572 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.572 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.572 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.572 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-17T06:13:00.572 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-17T06:13:00.573 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.573 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.573 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.573 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.573 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.573 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.573 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.573 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.573 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.573 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.573 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.573 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.573 INFO:teuthology.orchestra.run.smithi047.stdout: "server_addr": { 2024-04-17T06:13:00.573 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "server_addr", 2024-04-17T06:13:00.573 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.573 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.573 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.574 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.574 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.574 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.574 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.574 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.574 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.574 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.574 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.574 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.574 INFO:teuthology.orchestra.run.smithi047.stdout: "server_port": { 2024-04-17T06:13:00.574 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "server_port", 2024-04-17T06:13:00.574 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.574 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.575 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.575 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.575 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.575 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.575 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.575 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.575 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.575 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.575 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.575 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.575 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.575 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.575 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:13:00.575 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rgw", 2024-04-17T06:13:00.575 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-17T06:13:00.575 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-17T06:13:00.576 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-17T06:13:00.576 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-17T06:13:00.576 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-17T06:13:00.576 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.576 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.576 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.576 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.576 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.576 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.576 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.576 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.576 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.576 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.576 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.576 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.576 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.576 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.577 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.577 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.577 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.577 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.577 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.577 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-17T06:13:00.577 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-17T06:13:00.577 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.577 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.577 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.577 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.577 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.577 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.577 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.577 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.577 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.578 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.578 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.578 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.578 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-17T06:13:00.578 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-17T06:13:00.578 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.578 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.578 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.578 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-17T06:13:00.578 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.578 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.578 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.578 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.578 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.578 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.579 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.579 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.579 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.579 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.579 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.579 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.579 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.579 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.579 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.579 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-17T06:13:00.579 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-17T06:13:00.579 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.579 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.579 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.579 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.579 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.580 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.580 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.580 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.580 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.580 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.580 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.580 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.580 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.580 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.580 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:13:00.580 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rook", 2024-04-17T06:13:00.580 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-17T06:13:00.580 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-17T06:13:00.580 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-17T06:13:00.580 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-17T06:13:00.580 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-17T06:13:00.580 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.581 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.581 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.581 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.581 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.581 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.581 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.581 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.581 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.581 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.581 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.581 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.581 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.581 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.581 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.581 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.581 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.581 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.582 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.582 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-17T06:13:00.582 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-17T06:13:00.582 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.582 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.582 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.582 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.582 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.582 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.582 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.582 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.582 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.582 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.582 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.583 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.583 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-17T06:13:00.583 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-17T06:13:00.583 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.583 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.583 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.583 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-17T06:13:00.583 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.583 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.583 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.583 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.583 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.583 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.583 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.583 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.583 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.583 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.584 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.584 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.584 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.584 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.584 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.584 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-17T06:13:00.584 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-17T06:13:00.584 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.584 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.584 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.584 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.584 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.584 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.584 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.584 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.584 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.585 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.585 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.585 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.585 INFO:teuthology.orchestra.run.smithi047.stdout: "storage_class": { 2024-04-17T06:13:00.585 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "storage_class", 2024-04-17T06:13:00.585 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.585 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.585 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.585 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "local", 2024-04-17T06:13:00.585 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.585 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.585 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.585 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-17T06:13:00.585 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.585 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.585 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.585 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.586 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.586 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.586 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:13:00.586 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "selftest", 2024-04-17T06:13:00.586 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-17T06:13:00.586 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-17T06:13:00.586 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-17T06:13:00.586 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-17T06:13:00.586 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-17T06:13:00.586 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.586 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.586 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.586 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.586 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.587 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.587 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.587 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.587 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.587 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.587 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.587 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.587 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.587 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.587 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.587 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.587 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.587 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.587 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.587 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-17T06:13:00.587 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-17T06:13:00.588 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.588 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.588 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.588 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.588 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.588 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.588 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.588 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.588 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.588 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.588 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.588 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.588 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-17T06:13:00.588 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-17T06:13:00.588 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.588 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.588 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.589 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-17T06:13:00.589 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.589 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.589 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.589 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.589 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.589 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.589 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.589 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.589 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.589 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.589 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.589 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.589 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.590 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.590 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.590 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-17T06:13:00.590 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-17T06:13:00.590 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.590 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.590 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.590 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.590 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.590 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.590 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.590 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.590 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.590 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.591 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.591 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.591 INFO:teuthology.orchestra.run.smithi047.stdout: "roption1": { 2024-04-17T06:13:00.591 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "roption1", 2024-04-17T06:13:00.591 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.591 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.591 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.591 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.591 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.591 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.591 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.591 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.591 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.592 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.592 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.592 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.592 INFO:teuthology.orchestra.run.smithi047.stdout: "roption2": { 2024-04-17T06:13:00.592 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "roption2", 2024-04-17T06:13:00.592 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.592 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.592 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.592 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "xyz", 2024-04-17T06:13:00.592 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.592 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.592 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.592 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.592 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.592 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.592 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.592 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.593 INFO:teuthology.orchestra.run.smithi047.stdout: "rwoption1": { 2024-04-17T06:13:00.593 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rwoption1", 2024-04-17T06:13:00.593 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.593 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.593 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.593 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.593 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.593 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.593 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.593 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.593 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.593 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.593 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.593 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.594 INFO:teuthology.orchestra.run.smithi047.stdout: "rwoption2": { 2024-04-17T06:13:00.594 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rwoption2", 2024-04-17T06:13:00.594 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:13:00.594 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.594 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.594 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.594 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.594 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.594 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.594 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.594 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.594 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.594 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.594 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.595 INFO:teuthology.orchestra.run.smithi047.stdout: "rwoption3": { 2024-04-17T06:13:00.595 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rwoption3", 2024-04-17T06:13:00.595 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "float", 2024-04-17T06:13:00.595 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.595 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.595 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.595 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.595 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.595 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.595 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.595 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.595 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.595 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.595 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.595 INFO:teuthology.orchestra.run.smithi047.stdout: "rwoption4": { 2024-04-17T06:13:00.595 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rwoption4", 2024-04-17T06:13:00.595 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.596 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.596 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.596 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.596 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.596 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.596 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.596 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.596 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.596 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.596 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.596 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.596 INFO:teuthology.orchestra.run.smithi047.stdout: "rwoption5": { 2024-04-17T06:13:00.596 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rwoption5", 2024-04-17T06:13:00.596 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.596 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.596 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.596 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.597 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.597 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.597 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.597 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.597 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.597 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.597 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.597 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.597 INFO:teuthology.orchestra.run.smithi047.stdout: "rwoption6": { 2024-04-17T06:13:00.597 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rwoption6", 2024-04-17T06:13:00.597 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.597 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.597 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.597 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:13:00.597 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.598 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.598 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.598 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.598 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.598 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.598 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.598 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.598 INFO:teuthology.orchestra.run.smithi047.stdout: "rwoption7": { 2024-04-17T06:13:00.598 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rwoption7", 2024-04-17T06:13:00.598 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:13:00.598 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.598 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.599 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.599 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "1", 2024-04-17T06:13:00.599 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "42", 2024-04-17T06:13:00.599 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.599 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.599 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.599 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.599 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.599 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.599 INFO:teuthology.orchestra.run.smithi047.stdout: "testkey": { 2024-04-17T06:13:00.599 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "testkey", 2024-04-17T06:13:00.599 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.599 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.599 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.599 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.599 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.600 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.600 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.600 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.600 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.600 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.600 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.600 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.600 INFO:teuthology.orchestra.run.smithi047.stdout: "testlkey": { 2024-04-17T06:13:00.600 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "testlkey", 2024-04-17T06:13:00.600 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.600 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.600 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.600 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.600 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.600 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.600 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.601 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.601 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.601 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.601 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.601 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.601 INFO:teuthology.orchestra.run.smithi047.stdout: "testnewline": { 2024-04-17T06:13:00.601 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "testnewline", 2024-04-17T06:13:00.601 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.601 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.601 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.601 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.601 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.601 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.601 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.601 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.601 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.601 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.602 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.602 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.602 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.602 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.602 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:13:00.602 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "snap_schedule", 2024-04-17T06:13:00.602 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-17T06:13:00.602 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-17T06:13:00.602 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-17T06:13:00.602 INFO:teuthology.orchestra.run.smithi047.stdout: "allow_m_granularity": { 2024-04-17T06:13:00.602 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "allow_m_granularity", 2024-04-17T06:13:00.602 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.602 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.602 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.603 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.603 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.603 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.603 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.603 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "allow minute scheduled snapshots", 2024-04-17T06:13:00.603 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.603 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.603 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.603 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.603 INFO:teuthology.orchestra.run.smithi047.stdout: "dump_on_update": { 2024-04-17T06:13:00.603 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "dump_on_update", 2024-04-17T06:13:00.603 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.603 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.603 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.603 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.603 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.604 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.604 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.604 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "dump database to debug log on update", 2024-04-17T06:13:00.604 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.604 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.604 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.604 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.604 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-17T06:13:00.604 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-17T06:13:00.604 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.604 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.604 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.604 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.604 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.604 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.604 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.605 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.605 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.605 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.605 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.605 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.605 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.605 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.605 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.605 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.605 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.605 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.605 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.605 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-17T06:13:00.605 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-17T06:13:00.605 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.605 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.606 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.606 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.606 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.606 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.606 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.606 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.606 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.606 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.606 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.606 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.606 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-17T06:13:00.606 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-17T06:13:00.606 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.606 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.606 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.607 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-17T06:13:00.607 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.607 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.607 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.607 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.607 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.607 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.607 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.607 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.607 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.607 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.607 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.607 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.607 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.607 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.607 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.608 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-17T06:13:00.608 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-17T06:13:00.608 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.608 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.608 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.608 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.608 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.608 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.608 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.608 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.608 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.608 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.608 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.608 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.608 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.608 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.608 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:13:00.609 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "stats", 2024-04-17T06:13:00.609 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-17T06:13:00.609 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-17T06:13:00.609 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-17T06:13:00.609 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-17T06:13:00.609 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-17T06:13:00.609 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.609 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.609 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.609 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.609 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.609 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.609 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.609 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.609 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.609 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.610 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.610 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.610 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.610 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.610 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.610 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.610 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.610 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.610 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.610 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-17T06:13:00.610 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-17T06:13:00.610 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.610 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.610 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.611 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.611 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.611 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.611 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.611 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.611 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.611 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.611 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.611 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.611 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-17T06:13:00.611 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-17T06:13:00.611 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.611 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.611 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.611 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-17T06:13:00.611 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.611 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.612 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.612 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.612 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.612 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.612 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.612 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.612 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.612 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.612 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.612 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.612 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.612 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.612 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.612 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-17T06:13:00.612 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-17T06:13:00.613 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.613 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.613 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.613 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.613 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.613 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.613 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.613 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.613 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.613 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.613 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.613 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.613 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.613 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.613 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:13:00.613 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "status", 2024-04-17T06:13:00.613 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-17T06:13:00.614 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-17T06:13:00.614 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-17T06:13:00.614 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-17T06:13:00.614 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-17T06:13:00.614 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.614 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.614 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.614 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.614 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.614 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.614 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.614 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.614 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.614 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.615 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.615 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.615 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.615 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.615 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.615 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.615 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.615 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.615 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.615 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-17T06:13:00.615 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-17T06:13:00.615 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.615 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.615 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.615 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.615 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.616 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.616 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.616 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.616 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.616 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.616 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.616 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.616 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-17T06:13:00.616 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-17T06:13:00.616 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.616 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.616 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.616 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-17T06:13:00.616 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.616 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.616 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.617 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.617 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.617 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.617 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.617 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.617 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.617 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.617 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.617 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.617 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.617 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.617 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.617 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-17T06:13:00.617 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-17T06:13:00.617 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.618 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.618 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.618 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.618 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.618 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.618 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.618 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.618 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.618 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.618 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.618 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.618 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.618 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.618 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:13:00.619 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "telegraf", 2024-04-17T06:13:00.619 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-17T06:13:00.619 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-17T06:13:00.619 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-17T06:13:00.619 INFO:teuthology.orchestra.run.smithi047.stdout: "address": { 2024-04-17T06:13:00.619 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "address", 2024-04-17T06:13:00.619 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.619 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.619 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.619 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-17T06:13:00.619 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.619 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.619 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.619 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.619 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.619 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.620 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.620 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.620 INFO:teuthology.orchestra.run.smithi047.stdout: "interval": { 2024-04-17T06:13:00.620 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "interval", 2024-04-17T06:13:00.620 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-17T06:13:00.620 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.620 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.620 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "15", 2024-04-17T06:13:00.620 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.620 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.620 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.620 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.620 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.620 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.620 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.620 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.621 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-17T06:13:00.621 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-17T06:13:00.621 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.621 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.621 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.621 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.621 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.621 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.621 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.621 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.621 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.621 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.621 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.621 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.621 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.621 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.621 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.622 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.622 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.622 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.622 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.622 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-17T06:13:00.622 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-17T06:13:00.622 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.622 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.622 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.622 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.622 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.622 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.622 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.622 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.623 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.623 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.623 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.623 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.623 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-17T06:13:00.623 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-17T06:13:00.623 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.623 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.623 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.623 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-17T06:13:00.623 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.623 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.623 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.623 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.623 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.623 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.624 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.624 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.624 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.624 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.624 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.624 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.624 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.624 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.624 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.624 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-17T06:13:00.624 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-17T06:13:00.624 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.624 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.624 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.624 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.624 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.625 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.625 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.625 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.625 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.625 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.625 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.625 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.625 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.625 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.625 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:13:00.625 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "telemetry", 2024-04-17T06:13:00.625 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-17T06:13:00.625 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-17T06:13:00.626 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-17T06:13:00.626 INFO:teuthology.orchestra.run.smithi047.stdout: "channel_basic": { 2024-04-17T06:13:00.626 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "channel_basic", 2024-04-17T06:13:00.626 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.626 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.626 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.626 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:13:00.626 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.626 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.626 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.626 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-17T06:13:00.626 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.626 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.627 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.627 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.627 INFO:teuthology.orchestra.run.smithi047.stdout: "channel_crash": { 2024-04-17T06:13:00.627 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "channel_crash", 2024-04-17T06:13:00.627 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.627 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.627 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.627 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:13:00.627 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.627 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.627 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.627 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-17T06:13:00.627 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.627 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.627 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.628 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.628 INFO:teuthology.orchestra.run.smithi047.stdout: "channel_device": { 2024-04-17T06:13:00.628 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "channel_device", 2024-04-17T06:13:00.628 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.628 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.628 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.628 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:13:00.628 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.628 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.628 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.628 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-17T06:13:00.628 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.628 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.629 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.629 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.629 INFO:teuthology.orchestra.run.smithi047.stdout: "channel_ident": { 2024-04-17T06:13:00.629 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "channel_ident", 2024-04-17T06:13:00.629 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.629 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.629 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.629 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.629 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.629 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.629 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.629 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-17T06:13:00.629 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.629 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.630 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.630 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.630 INFO:teuthology.orchestra.run.smithi047.stdout: "channel_perf": { 2024-04-17T06:13:00.630 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "channel_perf", 2024-04-17T06:13:00.630 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.630 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.630 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.630 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.630 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.630 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.630 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.630 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-17T06:13:00.630 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.630 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.631 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.631 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.631 INFO:teuthology.orchestra.run.smithi047.stdout: "contact": { 2024-04-17T06:13:00.631 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "contact", 2024-04-17T06:13:00.631 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.631 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.631 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.631 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.631 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.631 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.631 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.631 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.631 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.631 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.631 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.631 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.632 INFO:teuthology.orchestra.run.smithi047.stdout: "description": { 2024-04-17T06:13:00.632 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "description", 2024-04-17T06:13:00.632 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.632 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.632 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.632 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.632 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.632 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.632 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.632 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.632 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.632 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.632 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.632 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.632 INFO:teuthology.orchestra.run.smithi047.stdout: "device_url": { 2024-04-17T06:13:00.632 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "device_url", 2024-04-17T06:13:00.633 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.633 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.633 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.633 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-17T06:13:00.633 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.633 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.633 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.633 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.633 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.633 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.633 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.633 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.633 INFO:teuthology.orchestra.run.smithi047.stdout: "enabled": { 2024-04-17T06:13:00.633 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "enabled", 2024-04-17T06:13:00.634 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.634 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.634 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.634 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.634 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.634 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.634 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.634 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.635 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.635 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.635 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.635 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.635 INFO:teuthology.orchestra.run.smithi047.stdout: "interval": { 2024-04-17T06:13:00.635 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "interval", 2024-04-17T06:13:00.635 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:13:00.635 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.635 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.635 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "24", 2024-04-17T06:13:00.635 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "8", 2024-04-17T06:13:00.635 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.635 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.635 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.635 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.635 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.635 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.636 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.636 INFO:teuthology.orchestra.run.smithi047.stdout: "last_opt_revision": { 2024-04-17T06:13:00.636 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "last_opt_revision", 2024-04-17T06:13:00.636 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:13:00.636 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.636 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.636 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "1", 2024-04-17T06:13:00.636 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.636 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.636 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.636 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.636 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.636 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.636 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.636 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.636 INFO:teuthology.orchestra.run.smithi047.stdout: "leaderboard": { 2024-04-17T06:13:00.637 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "leaderboard", 2024-04-17T06:13:00.637 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.637 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.637 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.637 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.637 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.637 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.637 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.637 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.637 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.637 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.637 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.637 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.637 INFO:teuthology.orchestra.run.smithi047.stdout: "leaderboard_description": { 2024-04-17T06:13:00.637 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "leaderboard_description", 2024-04-17T06:13:00.637 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.637 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.638 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.638 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.638 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.638 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.638 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.638 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.638 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.638 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.638 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.638 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.638 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-17T06:13:00.638 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-17T06:13:00.638 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.638 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.639 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.639 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.639 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.639 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.639 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.639 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.639 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.639 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.639 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.639 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.639 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.639 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.639 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.639 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.639 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.639 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.639 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.640 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-17T06:13:00.640 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-17T06:13:00.640 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.640 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.640 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.640 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.640 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.640 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.640 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.640 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.640 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.640 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.640 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.640 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.640 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-17T06:13:00.640 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-17T06:13:00.640 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.641 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.641 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.641 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-17T06:13:00.641 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.641 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.641 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.641 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.641 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.641 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.641 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.641 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.641 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.641 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.641 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.641 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.641 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.642 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.642 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.642 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-17T06:13:00.642 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-17T06:13:00.642 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.642 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.642 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.642 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.642 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.642 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.642 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.642 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.642 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.642 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.642 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.643 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.643 INFO:teuthology.orchestra.run.smithi047.stdout: "organization": { 2024-04-17T06:13:00.643 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "organization", 2024-04-17T06:13:00.643 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.643 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.643 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.643 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.643 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.643 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.643 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.643 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.643 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.643 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.643 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.643 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.643 INFO:teuthology.orchestra.run.smithi047.stdout: "proxy": { 2024-04-17T06:13:00.644 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "proxy", 2024-04-17T06:13:00.644 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.644 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.644 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.644 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.644 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.644 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.644 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.644 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.644 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.644 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.644 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.644 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.644 INFO:teuthology.orchestra.run.smithi047.stdout: "url": { 2024-04-17T06:13:00.644 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "url", 2024-04-17T06:13:00.644 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.644 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.645 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.645 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-17T06:13:00.645 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.645 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.645 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.645 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.645 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.645 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.645 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.645 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.645 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.645 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.645 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:13:00.645 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "test_orchestrator", 2024-04-17T06:13:00.645 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-17T06:13:00.645 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-17T06:13:00.646 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-17T06:13:00.646 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-17T06:13:00.646 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-17T06:13:00.646 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.646 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.646 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.646 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.646 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.646 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.646 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.646 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.646 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.646 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.646 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.647 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.647 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.647 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.647 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.647 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.647 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.647 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.647 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.647 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-17T06:13:00.647 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-17T06:13:00.647 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.647 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.647 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.647 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.648 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.648 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.648 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.648 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.648 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.648 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.648 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.648 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.648 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-17T06:13:00.648 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-17T06:13:00.648 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.648 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.648 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.648 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-17T06:13:00.649 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.649 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.649 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.649 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.649 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.649 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.649 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.649 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.649 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.649 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.649 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.649 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.649 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.649 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.649 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.649 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-17T06:13:00.650 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-17T06:13:00.650 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.650 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.650 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.650 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.650 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.650 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.650 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.650 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.650 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.650 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.650 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.650 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.650 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.650 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.651 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:13:00.651 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "volumes", 2024-04-17T06:13:00.651 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-17T06:13:00.651 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-17T06:13:00.651 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-17T06:13:00.651 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-17T06:13:00.651 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-17T06:13:00.651 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.651 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.651 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.651 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.651 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.651 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.651 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.651 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.652 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.652 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.652 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.652 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.652 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.652 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.652 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.652 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.652 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.652 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.652 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.652 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-17T06:13:00.652 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-17T06:13:00.652 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.652 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.652 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.653 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.653 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.653 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.653 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.653 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.653 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.653 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.653 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.653 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.653 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-17T06:13:00.653 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-17T06:13:00.653 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.653 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.653 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.653 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-17T06:13:00.654 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.654 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.654 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.654 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.654 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.654 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.654 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.654 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.654 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.654 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.654 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.654 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.654 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.654 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.655 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.655 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-17T06:13:00.655 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-17T06:13:00.655 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.655 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.655 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.655 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.655 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.655 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.655 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.655 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.655 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.655 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.655 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.655 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.656 INFO:teuthology.orchestra.run.smithi047.stdout: "max_concurrent_clones": { 2024-04-17T06:13:00.656 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "max_concurrent_clones", 2024-04-17T06:13:00.656 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:13:00.656 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.656 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.656 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "4", 2024-04-17T06:13:00.656 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.656 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.656 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.656 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-17T06:13:00.656 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.656 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.656 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.656 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.656 INFO:teuthology.orchestra.run.smithi047.stdout: "periodic_async_work": { 2024-04-17T06:13:00.656 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "periodic_async_work", 2024-04-17T06:13:00.657 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.657 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.657 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.657 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.657 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.657 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.657 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.657 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Periodically check for async work", 2024-04-17T06:13:00.657 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.657 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.657 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.657 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.657 INFO:teuthology.orchestra.run.smithi047.stdout: "snapshot_clone_delay": { 2024-04-17T06:13:00.657 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "snapshot_clone_delay", 2024-04-17T06:13:00.657 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:13:00.657 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.658 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.658 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-04-17T06:13:00.658 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.658 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.658 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.658 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-17T06:13:00.658 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.658 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.658 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.658 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.658 INFO:teuthology.orchestra.run.smithi047.stdout: "snapshot_clone_no_wait": { 2024-04-17T06:13:00.658 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "snapshot_clone_no_wait", 2024-04-17T06:13:00.658 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.658 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.658 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.658 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-04-17T06:13:00.659 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.659 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.659 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.659 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-17T06:13:00.659 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.659 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.659 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.659 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.659 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.659 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.659 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:13:00.659 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "zabbix", 2024-04-17T06:13:00.659 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-04-17T06:13:00.659 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-04-17T06:13:00.659 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-04-17T06:13:00.660 INFO:teuthology.orchestra.run.smithi047.stdout: "discovery_interval": { 2024-04-17T06:13:00.660 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "discovery_interval", 2024-04-17T06:13:00.660 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "uint", 2024-04-17T06:13:00.660 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.660 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.660 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "100", 2024-04-17T06:13:00.660 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.660 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.660 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.660 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.660 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.660 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.660 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.660 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.660 INFO:teuthology.orchestra.run.smithi047.stdout: "identifier": { 2024-04-17T06:13:00.660 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "identifier", 2024-04-17T06:13:00.661 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.661 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.661 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.661 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.661 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.661 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.661 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.661 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.661 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.661 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.661 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.661 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.661 INFO:teuthology.orchestra.run.smithi047.stdout: "interval": { 2024-04-17T06:13:00.661 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "interval", 2024-04-17T06:13:00.661 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-04-17T06:13:00.662 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.662 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.662 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "60", 2024-04-17T06:13:00.662 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.662 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.662 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.662 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.662 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.662 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.662 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.662 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.662 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-04-17T06:13:00.662 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-04-17T06:13:00.662 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.662 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.663 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.663 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.663 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.663 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.663 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.663 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.663 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.663 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.663 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.663 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.663 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.663 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.663 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.663 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.663 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.663 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.664 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.664 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-04-17T06:13:00.664 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-04-17T06:13:00.664 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.664 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.664 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.664 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.664 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.664 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.664 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.664 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.664 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.664 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.664 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.664 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.665 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-04-17T06:13:00.665 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-04-17T06:13:00.665 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.665 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.665 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.665 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-04-17T06:13:00.665 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.665 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.665 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-04-17T06:13:00.665 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-04-17T06:13:00.665 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-04-17T06:13:00.665 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-04-17T06:13:00.665 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-04-17T06:13:00.665 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-04-17T06:13:00.665 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-04-17T06:13:00.666 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.666 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.666 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.666 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.666 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.666 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.666 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-04-17T06:13:00.666 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-04-17T06:13:00.666 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-04-17T06:13:00.666 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.666 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-04-17T06:13:00.666 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-04-17T06:13:00.666 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.666 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.666 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.667 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.667 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.667 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.667 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.667 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.667 INFO:teuthology.orchestra.run.smithi047.stdout: "zabbix_host": { 2024-04-17T06:13:00.667 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "zabbix_host", 2024-04-17T06:13:00.667 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.667 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.667 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.667 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-04-17T06:13:00.667 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.667 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.667 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.668 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.668 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.668 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.668 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.668 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.668 INFO:teuthology.orchestra.run.smithi047.stdout: "zabbix_port": { 2024-04-17T06:13:00.668 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "zabbix_port", 2024-04-17T06:13:00.668 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-04-17T06:13:00.668 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.668 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.668 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10051", 2024-04-17T06:13:00.668 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.668 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.668 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.668 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.668 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.669 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.669 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.669 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.669 INFO:teuthology.orchestra.run.smithi047.stdout: "zabbix_sender": { 2024-04-17T06:13:00.669 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "zabbix_sender", 2024-04-17T06:13:00.669 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-04-17T06:13:00.669 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-04-17T06:13:00.669 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-04-17T06:13:00.669 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-17T06:13:00.669 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-04-17T06:13:00.669 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-04-17T06:13:00.669 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-04-17T06:13:00.669 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-04-17T06:13:00.669 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-04-17T06:13:00.669 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-04-17T06:13:00.670 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-04-17T06:13:00.670 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.670 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.670 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.670 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.670 INFO:teuthology.orchestra.run.smithi047.stdout: "services": { 2024-04-17T06:13:00.670 INFO:teuthology.orchestra.run.smithi047.stdout: "dashboard": "https://172.21.15.47:8443/" 2024-04-17T06:13:00.670 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.670 INFO:teuthology.orchestra.run.smithi047.stdout: "always_on_modules": { 2024-04-17T06:13:00.670 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus": [ 2024-04-17T06:13:00.670 INFO:teuthology.orchestra.run.smithi047.stdout: "balancer", 2024-04-17T06:13:00.670 INFO:teuthology.orchestra.run.smithi047.stdout: "crash", 2024-04-17T06:13:00.670 INFO:teuthology.orchestra.run.smithi047.stdout: "devicehealth", 2024-04-17T06:13:00.670 INFO:teuthology.orchestra.run.smithi047.stdout: "orchestrator", 2024-04-17T06:13:00.670 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscaler", 2024-04-17T06:13:00.670 INFO:teuthology.orchestra.run.smithi047.stdout: "progress", 2024-04-17T06:13:00.671 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_support", 2024-04-17T06:13:00.671 INFO:teuthology.orchestra.run.smithi047.stdout: "status", 2024-04-17T06:13:00.671 INFO:teuthology.orchestra.run.smithi047.stdout: "telemetry", 2024-04-17T06:13:00.671 INFO:teuthology.orchestra.run.smithi047.stdout: "volumes" 2024-04-17T06:13:00.671 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.671 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific": [ 2024-04-17T06:13:00.671 INFO:teuthology.orchestra.run.smithi047.stdout: "balancer", 2024-04-17T06:13:00.671 INFO:teuthology.orchestra.run.smithi047.stdout: "crash", 2024-04-17T06:13:00.671 INFO:teuthology.orchestra.run.smithi047.stdout: "devicehealth", 2024-04-17T06:13:00.671 INFO:teuthology.orchestra.run.smithi047.stdout: "orchestrator", 2024-04-17T06:13:00.671 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscaler", 2024-04-17T06:13:00.671 INFO:teuthology.orchestra.run.smithi047.stdout: "progress", 2024-04-17T06:13:00.671 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_support", 2024-04-17T06:13:00.671 INFO:teuthology.orchestra.run.smithi047.stdout: "status", 2024-04-17T06:13:00.671 INFO:teuthology.orchestra.run.smithi047.stdout: "telemetry", 2024-04-17T06:13:00.671 INFO:teuthology.orchestra.run.smithi047.stdout: "volumes" 2024-04-17T06:13:00.672 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.672 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy": [ 2024-04-17T06:13:00.672 INFO:teuthology.orchestra.run.smithi047.stdout: "balancer", 2024-04-17T06:13:00.672 INFO:teuthology.orchestra.run.smithi047.stdout: "crash", 2024-04-17T06:13:00.672 INFO:teuthology.orchestra.run.smithi047.stdout: "devicehealth", 2024-04-17T06:13:00.672 INFO:teuthology.orchestra.run.smithi047.stdout: "orchestrator", 2024-04-17T06:13:00.672 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscaler", 2024-04-17T06:13:00.672 INFO:teuthology.orchestra.run.smithi047.stdout: "progress", 2024-04-17T06:13:00.672 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_support", 2024-04-17T06:13:00.672 INFO:teuthology.orchestra.run.smithi047.stdout: "status", 2024-04-17T06:13:00.672 INFO:teuthology.orchestra.run.smithi047.stdout: "telemetry", 2024-04-17T06:13:00.672 INFO:teuthology.orchestra.run.smithi047.stdout: "volumes" 2024-04-17T06:13:00.672 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.672 INFO:teuthology.orchestra.run.smithi047.stdout: "reef": [ 2024-04-17T06:13:00.673 INFO:teuthology.orchestra.run.smithi047.stdout: "balancer", 2024-04-17T06:13:00.673 INFO:teuthology.orchestra.run.smithi047.stdout: "crash", 2024-04-17T06:13:00.673 INFO:teuthology.orchestra.run.smithi047.stdout: "devicehealth", 2024-04-17T06:13:00.673 INFO:teuthology.orchestra.run.smithi047.stdout: "orchestrator", 2024-04-17T06:13:00.673 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscaler", 2024-04-17T06:13:00.673 INFO:teuthology.orchestra.run.smithi047.stdout: "progress", 2024-04-17T06:13:00.673 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_support", 2024-04-17T06:13:00.673 INFO:teuthology.orchestra.run.smithi047.stdout: "status", 2024-04-17T06:13:00.674 INFO:teuthology.orchestra.run.smithi047.stdout: "telemetry", 2024-04-17T06:13:00.674 INFO:teuthology.orchestra.run.smithi047.stdout: "volumes" 2024-04-17T06:13:00.674 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-17T06:13:00.674 INFO:teuthology.orchestra.run.smithi047.stdout: "squid": [ 2024-04-17T06:13:00.674 INFO:teuthology.orchestra.run.smithi047.stdout: "balancer", 2024-04-17T06:13:00.674 INFO:teuthology.orchestra.run.smithi047.stdout: "crash", 2024-04-17T06:13:00.674 INFO:teuthology.orchestra.run.smithi047.stdout: "devicehealth", 2024-04-17T06:13:00.674 INFO:teuthology.orchestra.run.smithi047.stdout: "orchestrator", 2024-04-17T06:13:00.675 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscaler", 2024-04-17T06:13:00.675 INFO:teuthology.orchestra.run.smithi047.stdout: "progress", 2024-04-17T06:13:00.675 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_support", 2024-04-17T06:13:00.675 INFO:teuthology.orchestra.run.smithi047.stdout: "status", 2024-04-17T06:13:00.675 INFO:teuthology.orchestra.run.smithi047.stdout: "telemetry", 2024-04-17T06:13:00.675 INFO:teuthology.orchestra.run.smithi047.stdout: "volumes" 2024-04-17T06:13:00.675 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-17T06:13:00.676 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.676 INFO:teuthology.orchestra.run.smithi047.stdout: "last_failure_osd_epoch": 89, 2024-04-17T06:13:00.676 INFO:teuthology.orchestra.run.smithi047.stdout: "active_clients": [ 2024-04-17T06:13:00.676 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:13:00.676 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "devicehealth", 2024-04-17T06:13:00.676 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-17T06:13:00.676 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:13:00.676 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-17T06:13:00.677 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:0", 2024-04-17T06:13:00.677 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 3021192321 2024-04-17T06:13:00.677 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.677 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-17T06:13:00.677 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.677 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:13:00.677 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "libcephsqlite", 2024-04-17T06:13:00.677 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-17T06:13:00.678 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:13:00.678 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-17T06:13:00.678 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:0", 2024-04-17T06:13:00.678 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 2361749294 2024-04-17T06:13:00.678 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.678 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-17T06:13:00.678 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.678 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:13:00.678 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rbd_support", 2024-04-17T06:13:00.679 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-17T06:13:00.679 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:13:00.679 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-17T06:13:00.679 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:0", 2024-04-17T06:13:00.679 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 1293410992 2024-04-17T06:13:00.679 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.679 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-17T06:13:00.680 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.680 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:13:00.680 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "snap_schedule", 2024-04-17T06:13:00.680 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-17T06:13:00.680 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:13:00.680 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-17T06:13:00.680 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:0", 2024-04-17T06:13:00.680 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 1493533641 2024-04-17T06:13:00.681 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.681 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-17T06:13:00.681 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.681 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:13:00.681 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "snap_schedule", 2024-04-17T06:13:00.681 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-17T06:13:00.681 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:13:00.682 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-17T06:13:00.682 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:0", 2024-04-17T06:13:00.682 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 3818433982 2024-04-17T06:13:00.682 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.682 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-17T06:13:00.682 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:00.682 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:13:00.682 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "volumes", 2024-04-17T06:13:00.682 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-17T06:13:00.682 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-17T06:13:00.682 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-17T06:13:00.683 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:0", 2024-04-17T06:13:00.683 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 546053331 2024-04-17T06:13:00.683 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.683 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-17T06:13:00.683 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:00.683 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-17T06:13:00.683 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-17T06:13:00.684 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:12:59 smithi103 ceph-mon[36243]: pgmap v368: 97 pgs: 97 active+clean; 5.8 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.3 KiB/s wr, 9 op/s 2024-04-17T06:13:00.684 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:12:59 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/2428602217' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-17T06:13:00.684 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:12:59 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/1368698928' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-17T06:13:00.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:59.348+0000 7f051d332640 1 -- 172.21.15.47:0/1368698928 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f05000a3d20 msgr2=0x7f05000a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:13:00.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:59.348+0000 7f051d332640 1 --2- 172.21.15.47:0/1368698928 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f05000a3d20 0x7f05000a61e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f05080058c0 tx=0x7f0508005be0 comp rx=0 tx=0).stop 2024-04-17T06:13:00.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:59.348+0000 7f051d332640 1 -- 172.21.15.47:0/1368698928 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0518101b70 msgr2=0x7f051819eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:13:00.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:59.348+0000 7f051d332640 1 --2- 172.21.15.47:0/1368698928 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0518101b70 0x7f051819eb60 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f050c02c290 tx=0x7f050c0045e0 comp rx=0 tx=0).stop 2024-04-17T06:13:00.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:59.348+0000 7f051d332640 1 -- 172.21.15.47:0/1368698928 shutdown_connections 2024-04-17T06:13:00.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:59.348+0000 7f051d332640 1 --2- 172.21.15.47:0/1368698928 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0518105ef0 0x7f0518198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:13:00.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:59.348+0000 7f051d332640 1 --2- 172.21.15.47:0/1368698928 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f05000a3d20 0x7f05000a61e0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:13:00.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:59.348+0000 7f051d332640 1 --2- 172.21.15.47:0/1368698928 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0518101b70 0x7f051819eb60 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:13:00.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:59.348+0000 7f051d332640 1 --2- 172.21.15.47:0/1368698928 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f05181011b0 0x7f051819e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:13:00.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:59.348+0000 7f051d332640 1 -- 172.21.15.47:0/1368698928 >> 172.21.15.47:0/1368698928 conn(0x7f0518077c30 msgr2=0x7f051806aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:13:00.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:59.348+0000 7f051d332640 1 -- 172.21.15.47:0/1368698928 shutdown_connections 2024-04-17T06:13:00.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:12:59.348+0000 7f051d332640 1 -- 172.21.15.47:0/1368698928 wait complete. 2024-04-17T06:13:00.685 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:12:58 smithi098 ceph-mon[36516]: from='client.16377 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T06:13:00.686 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:12:58 smithi098 ceph-mon[36516]: from='client.16383 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T06:13:00.686 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:12:58 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/847246225' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T06:13:00.686 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:12:58 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/1246705630' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-17T06:13:00.686 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:12:58 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/2826897061' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-17T06:13:00.686 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:12:59 smithi098 ceph-mon[36516]: pgmap v368: 97 pgs: 97 active+clean; 5.8 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.3 KiB/s wr, 9 op/s 2024-04-17T06:13:00.686 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:12:59 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/2428602217' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-17T06:13:00.686 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:12:59 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/1368698928' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-17T06:13:00.686 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:12:58 smithi047 ceph-mon[32223]: from='client.16377 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T06:13:00.686 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:12:58 smithi047 ceph-mon[32223]: from='client.16383 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T06:13:00.686 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:12:58 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/847246225' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T06:13:00.686 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:12:58 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/1246705630' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-17T06:13:00.686 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:12:58 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/2826897061' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-17T06:13:00.692 DEBUG:tasks.check_counter:Getting stats from a 2024-04-17T06:13:00.692 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 565db2d8-fc7e-11ee-bc91-c7b262605968 -- ceph daemon mds.a perf dump 2024-04-17T06:13:00.693 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:12:59 smithi047 ceph-mon[32223]: pgmap v368: 97 pgs: 97 active+clean; 5.8 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.3 KiB/s wr, 9 op/s 2024-04-17T06:13:00.693 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:12:59 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/2428602217' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-17T06:13:00.693 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:12:59 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/1368698928' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-17T06:13:00.955 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T06:13:02.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:13:01 smithi103 ceph-mon[36243]: pgmap v369: 97 pgs: 97 active+clean; 5.9 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 18 op/s 2024-04-17T06:13:02.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:13:01 smithi103 ceph-mon[36243]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-17T06:13:02.321 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:13:01 smithi047 ceph-mon[32223]: pgmap v369: 97 pgs: 97 active+clean; 5.9 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 18 op/s 2024-04-17T06:13:02.321 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:13:01 smithi047 ceph-mon[32223]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-17T06:13:02.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:13:01 smithi098 ceph-mon[36516]: pgmap v369: 97 pgs: 97 active+clean; 5.9 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 18 op/s 2024-04-17T06:13:02.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:13:01 smithi098 ceph-mon[36516]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-17T06:13:02.696 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-17T06:13:02.697 INFO:teuthology.orchestra.run.smithi047.stdout: "AsyncMessenger::Worker-0": { 2024-04-17T06:13:02.697 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_messages": 4, 2024-04-17T06:13:02.697 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages": 4, 2024-04-17T06:13:02.697 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_bytes": 3024, 2024-04-17T06:13:02.697 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_bytes": 448, 2024-04-17T06:13:02.697 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_created_connections": 4, 2024-04-17T06:13:02.697 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_active_connections": 0, 2024-04-17T06:13:02.697 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_total_time": 0.001444246, 2024-04-17T06:13:02.697 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_send_time": 0.000030208, 2024-04-17T06:13:02.697 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_recv_time": 0.000511595, 2024-04-17T06:13:02.697 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-17T06:13:02.697 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T06:13:02.698 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 3, 2024-04-17T06:13:02.698 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000687422, 2024-04-17T06:13:02.698 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000229140 2024-04-17T06:13:02.698 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:02.698 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_handle_ack_lat": { 2024-04-17T06:13:02.698 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-04-17T06:13:02.698 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-04-17T06:13:02.698 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-04-17T06:13:02.698 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:02.698 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-04-17T06:13:02.698 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_encrypted_bytes": 448 2024-04-17T06:13:02.698 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:02.698 INFO:teuthology.orchestra.run.smithi047.stdout: "AsyncMessenger::Worker-1": { 2024-04-17T06:13:02.699 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_messages": 2, 2024-04-17T06:13:02.699 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages": 171, 2024-04-17T06:13:02.699 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_bytes": 320, 2024-04-17T06:13:02.699 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_bytes": 186336, 2024-04-17T06:13:02.699 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_created_connections": 5, 2024-04-17T06:13:02.699 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_active_connections": 3, 2024-04-17T06:13:02.699 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_total_time": 0.015562035, 2024-04-17T06:13:02.699 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_send_time": 0.011096670, 2024-04-17T06:13:02.699 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_recv_time": 0.002034616, 2024-04-17T06:13:02.699 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-17T06:13:02.700 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T06:13:02.700 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 168, 2024-04-17T06:13:02.700 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.017719613, 2024-04-17T06:13:02.700 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000105473 2024-04-17T06:13:02.700 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:02.700 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_handle_ack_lat": { 2024-04-17T06:13:02.700 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-04-17T06:13:02.700 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-04-17T06:13:02.700 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-04-17T06:13:02.700 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:02.700 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-17T06:13:02.700 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_encrypted_bytes": 186336 2024-04-17T06:13:02.700 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:02.700 INFO:teuthology.orchestra.run.smithi047.stdout: "AsyncMessenger::Worker-2": { 2024-04-17T06:13:02.701 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_messages": 236, 2024-04-17T06:13:02.701 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages": 225, 2024-04-17T06:13:02.701 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_bytes": 1593776, 2024-04-17T06:13:02.701 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_bytes": 138304, 2024-04-17T06:13:02.701 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_created_connections": 2, 2024-04-17T06:13:02.701 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_active_connections": 1, 2024-04-17T06:13:02.701 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_total_time": 0.046539572, 2024-04-17T06:13:02.701 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_send_time": 0.015574062, 2024-04-17T06:13:02.701 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_recv_time": 0.035086915, 2024-04-17T06:13:02.701 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_fast_dispatch_time": 0.004549624, 2024-04-17T06:13:02.701 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T06:13:02.701 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 224, 2024-04-17T06:13:02.702 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.010862632, 2024-04-17T06:13:02.702 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000048493 2024-04-17T06:13:02.702 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:02.702 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_handle_ack_lat": { 2024-04-17T06:13:02.702 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-04-17T06:13:02.702 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-04-17T06:13:02.702 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-04-17T06:13:02.702 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:02.702 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_encrypted_bytes": 1593776, 2024-04-17T06:13:02.702 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_encrypted_bytes": 138304 2024-04-17T06:13:02.702 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:02.703 INFO:teuthology.orchestra.run.smithi047.stdout: "cct": { 2024-04-17T06:13:02.703 INFO:teuthology.orchestra.run.smithi047.stdout: "total_workers": 0, 2024-04-17T06:13:02.703 INFO:teuthology.orchestra.run.smithi047.stdout: "unhealthy_workers": 0 2024-04-17T06:13:02.703 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:02.703 INFO:teuthology.orchestra.run.smithi047.stdout: "mempool": { 2024-04-17T06:13:02.703 INFO:teuthology.orchestra.run.smithi047.stdout: "bloom_filter_bytes": 0, 2024-04-17T06:13:02.704 INFO:teuthology.orchestra.run.smithi047.stdout: "bloom_filter_items": 0, 2024-04-17T06:13:02.704 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_alloc_bytes": 0, 2024-04-17T06:13:02.704 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_alloc_items": 0, 2024-04-17T06:13:02.704 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_data_bytes": 0, 2024-04-17T06:13:02.704 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_data_items": 0, 2024-04-17T06:13:02.704 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-17T06:13:02.704 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_onode_items": 0, 2024-04-17T06:13:02.704 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-17T06:13:02.704 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_meta_items": 0, 2024-04-17T06:13:02.705 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_other_bytes": 0, 2024-04-17T06:13:02.705 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_other_items": 0, 2024-04-17T06:13:02.705 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-17T06:13:02.705 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_buffer_items": 0, 2024-04-17T06:13:02.705 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_extent_bytes": 0, 2024-04-17T06:13:02.705 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_extent_items": 0, 2024-04-17T06:13:02.705 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_blob_bytes": 0, 2024-04-17T06:13:02.705 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_blob_items": 0, 2024-04-17T06:13:02.706 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-17T06:13:02.706 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_shared_blob_items": 0, 2024-04-17T06:13:02.706 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-17T06:13:02.706 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_inline_bl_items": 0, 2024-04-17T06:13:02.706 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_fsck_bytes": 0, 2024-04-17T06:13:02.706 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_fsck_items": 0, 2024-04-17T06:13:02.706 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_txc_bytes": 0, 2024-04-17T06:13:02.706 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_txc_items": 0, 2024-04-17T06:13:02.706 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-17T06:13:02.706 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_writing_deferred_items": 0, 2024-04-17T06:13:02.706 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_writing_bytes": 0, 2024-04-17T06:13:02.707 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_writing_items": 0, 2024-04-17T06:13:02.707 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_bytes": 0, 2024-04-17T06:13:02.707 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_items": 0, 2024-04-17T06:13:02.707 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_file_reader_bytes": 0, 2024-04-17T06:13:02.707 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_file_reader_items": 0, 2024-04-17T06:13:02.707 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_file_writer_bytes": 0, 2024-04-17T06:13:02.707 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_file_writer_items": 0, 2024-04-17T06:13:02.707 INFO:teuthology.orchestra.run.smithi047.stdout: "buffer_anon_bytes": 149034, 2024-04-17T06:13:02.707 INFO:teuthology.orchestra.run.smithi047.stdout: "buffer_anon_items": 32, 2024-04-17T06:13:02.707 INFO:teuthology.orchestra.run.smithi047.stdout: "buffer_meta_bytes": 0, 2024-04-17T06:13:02.707 INFO:teuthology.orchestra.run.smithi047.stdout: "buffer_meta_items": 0, 2024-04-17T06:13:02.707 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_bytes": 0, 2024-04-17T06:13:02.707 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_items": 0, 2024-04-17T06:13:02.707 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_mapbl_bytes": 0, 2024-04-17T06:13:02.708 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_mapbl_items": 0, 2024-04-17T06:13:02.708 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_pglog_bytes": 0, 2024-04-17T06:13:02.708 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_pglog_items": 0, 2024-04-17T06:13:02.708 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_bytes": 0, 2024-04-17T06:13:02.708 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_items": 0, 2024-04-17T06:13:02.708 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_mapping_bytes": 0, 2024-04-17T06:13:02.708 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_mapping_items": 0, 2024-04-17T06:13:02.708 INFO:teuthology.orchestra.run.smithi047.stdout: "pgmap_bytes": 0, 2024-04-17T06:13:02.708 INFO:teuthology.orchestra.run.smithi047.stdout: "pgmap_items": 0, 2024-04-17T06:13:02.708 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_co_bytes": 928, 2024-04-17T06:13:02.708 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_co_items": 2, 2024-04-17T06:13:02.708 INFO:teuthology.orchestra.run.smithi047.stdout: "unittest_1_bytes": 0, 2024-04-17T06:13:02.708 INFO:teuthology.orchestra.run.smithi047.stdout: "unittest_1_items": 0, 2024-04-17T06:13:02.708 INFO:teuthology.orchestra.run.smithi047.stdout: "unittest_2_bytes": 0, 2024-04-17T06:13:02.709 INFO:teuthology.orchestra.run.smithi047.stdout: "unittest_2_items": 0 2024-04-17T06:13:02.709 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:02.709 INFO:teuthology.orchestra.run.smithi047.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-17T06:13:02.709 INFO:teuthology.orchestra.run.smithi047.stdout: "val": 0, 2024-04-17T06:13:02.709 INFO:teuthology.orchestra.run.smithi047.stdout: "max": 104857600, 2024-04-17T06:13:02.709 INFO:teuthology.orchestra.run.smithi047.stdout: "get_started": 0, 2024-04-17T06:13:02.709 INFO:teuthology.orchestra.run.smithi047.stdout: "get": 238, 2024-04-17T06:13:02.709 INFO:teuthology.orchestra.run.smithi047.stdout: "get_sum": 1560485, 2024-04-17T06:13:02.709 INFO:teuthology.orchestra.run.smithi047.stdout: "get_or_fail_fail": 0, 2024-04-17T06:13:02.709 INFO:teuthology.orchestra.run.smithi047.stdout: "get_or_fail_success": 238, 2024-04-17T06:13:02.709 INFO:teuthology.orchestra.run.smithi047.stdout: "take": 0, 2024-04-17T06:13:02.709 INFO:teuthology.orchestra.run.smithi047.stdout: "take_sum": 0, 2024-04-17T06:13:02.709 INFO:teuthology.orchestra.run.smithi047.stdout: "put": 238, 2024-04-17T06:13:02.709 INFO:teuthology.orchestra.run.smithi047.stdout: "put_sum": 1560485, 2024-04-17T06:13:02.710 INFO:teuthology.orchestra.run.smithi047.stdout: "wait": { 2024-04-17T06:13:02.710 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-04-17T06:13:02.710 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-04-17T06:13:02.710 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-04-17T06:13:02.710 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:02.710 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:02.710 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-17T06:13:03.451 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-04-17T06:13:03.451 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-04-17T06:13:03.451 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.a 2024-04-17T06:13:03.452 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.a 2024-04-17T06:13:03.452 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.a 2024-04-17T06:13:03.452 DEBUG:tasks.check_counter:Getting stats from d 2024-04-17T06:13:03.452 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 565db2d8-fc7e-11ee-bc91-c7b262605968 -- ceph daemon mds.d perf dump 2024-04-17T06:13:03.710 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T06:13:04.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:13:03 smithi047 ceph-mon[32223]: pgmap v370: 97 pgs: 97 active+clean; 5.9 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 KiB/s wr, 14 op/s 2024-04-17T06:13:04.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:13:03 smithi103 ceph-mon[36243]: pgmap v370: 97 pgs: 97 active+clean; 5.9 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 KiB/s wr, 14 op/s 2024-04-17T06:13:04.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:13:03 smithi098 ceph-mon[36516]: pgmap v370: 97 pgs: 97 active+clean; 5.9 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 KiB/s wr, 14 op/s 2024-04-17T06:13:05.526 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-17T06:13:05.526 INFO:teuthology.orchestra.run.smithi047.stdout: "AsyncMessenger::Worker-0": { 2024-04-17T06:13:05.527 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_messages": 0, 2024-04-17T06:13:05.527 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages": 2, 2024-04-17T06:13:05.527 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_bytes": 0, 2024-04-17T06:13:05.527 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_bytes": 0, 2024-04-17T06:13:05.527 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_created_connections": 4, 2024-04-17T06:13:05.527 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_active_connections": 0, 2024-04-17T06:13:05.527 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_total_time": 0.001228470, 2024-04-17T06:13:05.527 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_send_time": 0.000000000, 2024-04-17T06:13:05.527 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_recv_time": 0.000327091, 2024-04-17T06:13:05.527 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-17T06:13:05.527 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T06:13:05.527 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-04-17T06:13:05.527 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-04-17T06:13:05.528 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-04-17T06:13:05.528 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:05.528 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_handle_ack_lat": { 2024-04-17T06:13:05.528 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-04-17T06:13:05.528 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-04-17T06:13:05.528 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-04-17T06:13:05.528 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:05.528 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-17T06:13:05.528 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_encrypted_bytes": 0 2024-04-17T06:13:05.528 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:05.528 INFO:teuthology.orchestra.run.smithi047.stdout: "AsyncMessenger::Worker-1": { 2024-04-17T06:13:05.528 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_messages": 245, 2024-04-17T06:13:05.528 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages": 232, 2024-04-17T06:13:05.528 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_bytes": 1598160, 2024-04-17T06:13:05.529 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_bytes": 141952, 2024-04-17T06:13:05.529 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_created_connections": 2, 2024-04-17T06:13:05.529 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_active_connections": 1, 2024-04-17T06:13:05.529 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_total_time": 0.048524456, 2024-04-17T06:13:05.529 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_send_time": 0.015859671, 2024-04-17T06:13:05.529 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_recv_time": 0.036756106, 2024-04-17T06:13:05.529 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_fast_dispatch_time": 0.004858504, 2024-04-17T06:13:05.529 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T06:13:05.529 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 232, 2024-04-17T06:13:05.529 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.012522301, 2024-04-17T06:13:05.529 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000053975 2024-04-17T06:13:05.529 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:05.529 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_handle_ack_lat": { 2024-04-17T06:13:05.529 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-04-17T06:13:05.530 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-04-17T06:13:05.530 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-04-17T06:13:05.530 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:05.530 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_encrypted_bytes": 1598160, 2024-04-17T06:13:05.530 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_encrypted_bytes": 141952 2024-04-17T06:13:05.530 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:05.530 INFO:teuthology.orchestra.run.smithi047.stdout: "AsyncMessenger::Worker-2": { 2024-04-17T06:13:05.530 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_messages": 2, 2024-04-17T06:13:05.530 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages": 176, 2024-04-17T06:13:05.530 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_bytes": 320, 2024-04-17T06:13:05.530 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_bytes": 187216, 2024-04-17T06:13:05.530 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_created_connections": 5, 2024-04-17T06:13:05.531 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_active_connections": 3, 2024-04-17T06:13:05.531 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_total_time": 0.015433903, 2024-04-17T06:13:05.531 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_send_time": 0.011016099, 2024-04-17T06:13:05.531 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_recv_time": 0.001732615, 2024-04-17T06:13:05.531 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-17T06:13:05.531 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T06:13:05.531 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 173, 2024-04-17T06:13:05.531 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.015016744, 2024-04-17T06:13:05.531 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000086801 2024-04-17T06:13:05.531 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:05.531 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_handle_ack_lat": { 2024-04-17T06:13:05.531 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-04-17T06:13:05.531 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-04-17T06:13:05.531 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-04-17T06:13:05.532 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:05.532 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-17T06:13:05.532 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_encrypted_bytes": 187216 2024-04-17T06:13:05.532 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:05.532 INFO:teuthology.orchestra.run.smithi047.stdout: "cct": { 2024-04-17T06:13:05.532 INFO:teuthology.orchestra.run.smithi047.stdout: "total_workers": 0, 2024-04-17T06:13:05.532 INFO:teuthology.orchestra.run.smithi047.stdout: "unhealthy_workers": 0 2024-04-17T06:13:05.532 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:05.532 INFO:teuthology.orchestra.run.smithi047.stdout: "mempool": { 2024-04-17T06:13:05.532 INFO:teuthology.orchestra.run.smithi047.stdout: "bloom_filter_bytes": 0, 2024-04-17T06:13:05.532 INFO:teuthology.orchestra.run.smithi047.stdout: "bloom_filter_items": 0, 2024-04-17T06:13:05.532 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_alloc_bytes": 0, 2024-04-17T06:13:05.532 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_alloc_items": 0, 2024-04-17T06:13:05.532 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_data_bytes": 0, 2024-04-17T06:13:05.532 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_data_items": 0, 2024-04-17T06:13:05.533 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-17T06:13:05.533 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_onode_items": 0, 2024-04-17T06:13:05.533 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-17T06:13:05.533 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_meta_items": 0, 2024-04-17T06:13:05.533 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_other_bytes": 0, 2024-04-17T06:13:05.533 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_other_items": 0, 2024-04-17T06:13:05.533 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-17T06:13:05.533 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_buffer_items": 0, 2024-04-17T06:13:05.533 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_extent_bytes": 0, 2024-04-17T06:13:05.533 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_extent_items": 0, 2024-04-17T06:13:05.533 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_blob_bytes": 0, 2024-04-17T06:13:05.533 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_blob_items": 0, 2024-04-17T06:13:05.533 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-17T06:13:05.533 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_shared_blob_items": 0, 2024-04-17T06:13:05.533 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-17T06:13:05.534 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_inline_bl_items": 0, 2024-04-17T06:13:05.534 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_fsck_bytes": 0, 2024-04-17T06:13:05.534 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_fsck_items": 0, 2024-04-17T06:13:05.534 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_txc_bytes": 0, 2024-04-17T06:13:05.534 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_txc_items": 0, 2024-04-17T06:13:05.534 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-17T06:13:05.534 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_writing_deferred_items": 0, 2024-04-17T06:13:05.534 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_writing_bytes": 0, 2024-04-17T06:13:05.534 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_writing_items": 0, 2024-04-17T06:13:05.534 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_bytes": 0, 2024-04-17T06:13:05.534 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_items": 0, 2024-04-17T06:13:05.534 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_file_reader_bytes": 0, 2024-04-17T06:13:05.534 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_file_reader_items": 0, 2024-04-17T06:13:05.535 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_file_writer_bytes": 0, 2024-04-17T06:13:05.535 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_file_writer_items": 0, 2024-04-17T06:13:05.535 INFO:teuthology.orchestra.run.smithi047.stdout: "buffer_anon_bytes": 149007, 2024-04-17T06:13:05.535 INFO:teuthology.orchestra.run.smithi047.stdout: "buffer_anon_items": 30, 2024-04-17T06:13:05.535 INFO:teuthology.orchestra.run.smithi047.stdout: "buffer_meta_bytes": 0, 2024-04-17T06:13:05.535 INFO:teuthology.orchestra.run.smithi047.stdout: "buffer_meta_items": 0, 2024-04-17T06:13:05.535 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_bytes": 0, 2024-04-17T06:13:05.535 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_items": 0, 2024-04-17T06:13:05.535 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_mapbl_bytes": 0, 2024-04-17T06:13:05.535 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_mapbl_items": 0, 2024-04-17T06:13:05.535 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_pglog_bytes": 0, 2024-04-17T06:13:05.535 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_pglog_items": 0, 2024-04-17T06:13:05.535 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_bytes": 0, 2024-04-17T06:13:05.535 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_items": 0, 2024-04-17T06:13:05.536 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_mapping_bytes": 0, 2024-04-17T06:13:05.536 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_mapping_items": 0, 2024-04-17T06:13:05.536 INFO:teuthology.orchestra.run.smithi047.stdout: "pgmap_bytes": 0, 2024-04-17T06:13:05.536 INFO:teuthology.orchestra.run.smithi047.stdout: "pgmap_items": 0, 2024-04-17T06:13:05.536 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_co_bytes": 928, 2024-04-17T06:13:05.536 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_co_items": 2, 2024-04-17T06:13:05.536 INFO:teuthology.orchestra.run.smithi047.stdout: "unittest_1_bytes": 0, 2024-04-17T06:13:05.536 INFO:teuthology.orchestra.run.smithi047.stdout: "unittest_1_items": 0, 2024-04-17T06:13:05.536 INFO:teuthology.orchestra.run.smithi047.stdout: "unittest_2_bytes": 0, 2024-04-17T06:13:05.536 INFO:teuthology.orchestra.run.smithi047.stdout: "unittest_2_items": 0 2024-04-17T06:13:05.536 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:05.536 INFO:teuthology.orchestra.run.smithi047.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-17T06:13:05.536 INFO:teuthology.orchestra.run.smithi047.stdout: "val": 0, 2024-04-17T06:13:05.537 INFO:teuthology.orchestra.run.smithi047.stdout: "max": 104857600, 2024-04-17T06:13:05.537 INFO:teuthology.orchestra.run.smithi047.stdout: "get_started": 0, 2024-04-17T06:13:05.537 INFO:teuthology.orchestra.run.smithi047.stdout: "get": 243, 2024-04-17T06:13:05.537 INFO:teuthology.orchestra.run.smithi047.stdout: "get_sum": 1561135, 2024-04-17T06:13:05.537 INFO:teuthology.orchestra.run.smithi047.stdout: "get_or_fail_fail": 0, 2024-04-17T06:13:05.537 INFO:teuthology.orchestra.run.smithi047.stdout: "get_or_fail_success": 243, 2024-04-17T06:13:05.537 INFO:teuthology.orchestra.run.smithi047.stdout: "take": 0, 2024-04-17T06:13:05.537 INFO:teuthology.orchestra.run.smithi047.stdout: "take_sum": 0, 2024-04-17T06:13:05.537 INFO:teuthology.orchestra.run.smithi047.stdout: "put": 243, 2024-04-17T06:13:05.537 INFO:teuthology.orchestra.run.smithi047.stdout: "put_sum": 1561135, 2024-04-17T06:13:05.537 INFO:teuthology.orchestra.run.smithi047.stdout: "wait": { 2024-04-17T06:13:05.537 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-04-17T06:13:05.537 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-04-17T06:13:05.538 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-04-17T06:13:05.538 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:05.538 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:05.538 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-17T06:13:06.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:13:05 smithi047 ceph-mon[32223]: pgmap v371: 97 pgs: 97 active+clean; 5.9 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 63 KiB/s wr, 34 op/s 2024-04-17T06:13:06.254 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-04-17T06:13:06.254 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-04-17T06:13:06.254 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-04-17T06:13:06.254 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-04-17T06:13:06.254 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-04-17T06:13:06.254 DEBUG:tasks.check_counter:Getting stats from g 2024-04-17T06:13:06.255 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 565db2d8-fc7e-11ee-bc91-c7b262605968 -- ceph daemon mds.g perf dump 2024-04-17T06:13:06.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:13:05 smithi103 ceph-mon[36243]: pgmap v371: 97 pgs: 97 active+clean; 5.9 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 63 KiB/s wr, 34 op/s 2024-04-17T06:13:06.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:13:05 smithi098 ceph-mon[36516]: pgmap v371: 97 pgs: 97 active+clean; 5.9 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 63 KiB/s wr, 34 op/s 2024-04-17T06:13:06.508 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T06:13:08.252 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:13:07 smithi047 ceph-mon[32223]: pgmap v372: 97 pgs: 97 active+clean; 5.9 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 58 KiB/s wr, 29 op/s 2024-04-17T06:13:08.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:13:07 smithi103 ceph-mon[36243]: pgmap v372: 97 pgs: 97 active+clean; 5.9 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 58 KiB/s wr, 29 op/s 2024-04-17T06:13:08.294 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-17T06:13:08.294 INFO:teuthology.orchestra.run.smithi047.stdout: "AsyncMessenger::Worker-0": { 2024-04-17T06:13:08.294 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_messages": 0, 2024-04-17T06:13:08.294 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages": 2, 2024-04-17T06:13:08.294 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_bytes": 0, 2024-04-17T06:13:08.294 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_bytes": 0, 2024-04-17T06:13:08.294 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_created_connections": 4, 2024-04-17T06:13:08.294 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_active_connections": 0, 2024-04-17T06:13:08.294 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_total_time": 0.001236768, 2024-04-17T06:13:08.295 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_send_time": 0.000000000, 2024-04-17T06:13:08.295 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_recv_time": 0.000263601, 2024-04-17T06:13:08.295 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-17T06:13:08.295 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T06:13:08.295 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-04-17T06:13:08.295 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-04-17T06:13:08.295 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-04-17T06:13:08.295 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:08.295 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_handle_ack_lat": { 2024-04-17T06:13:08.295 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-04-17T06:13:08.295 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-04-17T06:13:08.295 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-04-17T06:13:08.295 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:08.295 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-17T06:13:08.296 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_encrypted_bytes": 0 2024-04-17T06:13:08.296 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:08.296 INFO:teuthology.orchestra.run.smithi047.stdout: "AsyncMessenger::Worker-1": { 2024-04-17T06:13:08.296 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_messages": 242, 2024-04-17T06:13:08.296 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages": 229, 2024-04-17T06:13:08.296 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_bytes": 1597344, 2024-04-17T06:13:08.296 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_bytes": 140032, 2024-04-17T06:13:08.296 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_created_connections": 2, 2024-04-17T06:13:08.296 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_active_connections": 1, 2024-04-17T06:13:08.296 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_total_time": 0.049637216, 2024-04-17T06:13:08.296 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_send_time": 0.016149647, 2024-04-17T06:13:08.296 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_recv_time": 0.037609127, 2024-04-17T06:13:08.296 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_fast_dispatch_time": 0.004783318, 2024-04-17T06:13:08.296 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T06:13:08.297 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 229, 2024-04-17T06:13:08.297 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.012598200, 2024-04-17T06:13:08.297 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000055013 2024-04-17T06:13:08.297 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:08.297 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_handle_ack_lat": { 2024-04-17T06:13:08.297 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-04-17T06:13:08.297 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-04-17T06:13:08.297 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-04-17T06:13:08.297 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:08.297 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_encrypted_bytes": 1597344, 2024-04-17T06:13:08.297 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_encrypted_bytes": 140032 2024-04-17T06:13:08.297 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:08.297 INFO:teuthology.orchestra.run.smithi047.stdout: "AsyncMessenger::Worker-2": { 2024-04-17T06:13:08.297 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_messages": 2, 2024-04-17T06:13:08.297 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages": 173, 2024-04-17T06:13:08.298 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_bytes": 320, 2024-04-17T06:13:08.298 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_bytes": 186688, 2024-04-17T06:13:08.298 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_created_connections": 5, 2024-04-17T06:13:08.298 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_active_connections": 3, 2024-04-17T06:13:08.298 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_total_time": 0.015258692, 2024-04-17T06:13:08.298 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_send_time": 0.011061314, 2024-04-17T06:13:08.298 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_recv_time": 0.001469478, 2024-04-17T06:13:08.298 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-17T06:13:08.298 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T06:13:08.298 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 170, 2024-04-17T06:13:08.298 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.012569484, 2024-04-17T06:13:08.298 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000073938 2024-04-17T06:13:08.298 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:08.298 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_handle_ack_lat": { 2024-04-17T06:13:08.299 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-04-17T06:13:08.299 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-04-17T06:13:08.299 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-04-17T06:13:08.299 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:08.299 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-17T06:13:08.299 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_encrypted_bytes": 186688 2024-04-17T06:13:08.299 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:08.299 INFO:teuthology.orchestra.run.smithi047.stdout: "cct": { 2024-04-17T06:13:08.299 INFO:teuthology.orchestra.run.smithi047.stdout: "total_workers": 0, 2024-04-17T06:13:08.299 INFO:teuthology.orchestra.run.smithi047.stdout: "unhealthy_workers": 0 2024-04-17T06:13:08.299 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:08.299 INFO:teuthology.orchestra.run.smithi047.stdout: "mempool": { 2024-04-17T06:13:08.299 INFO:teuthology.orchestra.run.smithi047.stdout: "bloom_filter_bytes": 0, 2024-04-17T06:13:08.299 INFO:teuthology.orchestra.run.smithi047.stdout: "bloom_filter_items": 0, 2024-04-17T06:13:08.299 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_alloc_bytes": 0, 2024-04-17T06:13:08.300 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_alloc_items": 0, 2024-04-17T06:13:08.300 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_data_bytes": 0, 2024-04-17T06:13:08.300 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_data_items": 0, 2024-04-17T06:13:08.300 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-17T06:13:08.300 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_onode_items": 0, 2024-04-17T06:13:08.300 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-17T06:13:08.300 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_meta_items": 0, 2024-04-17T06:13:08.300 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_other_bytes": 0, 2024-04-17T06:13:08.300 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_other_items": 0, 2024-04-17T06:13:08.300 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-17T06:13:08.300 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_buffer_items": 0, 2024-04-17T06:13:08.300 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_extent_bytes": 0, 2024-04-17T06:13:08.300 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_extent_items": 0, 2024-04-17T06:13:08.300 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_blob_bytes": 0, 2024-04-17T06:13:08.301 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_blob_items": 0, 2024-04-17T06:13:08.301 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-17T06:13:08.301 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_shared_blob_items": 0, 2024-04-17T06:13:08.301 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-17T06:13:08.301 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_inline_bl_items": 0, 2024-04-17T06:13:08.301 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_fsck_bytes": 0, 2024-04-17T06:13:08.301 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_fsck_items": 0, 2024-04-17T06:13:08.301 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_txc_bytes": 0, 2024-04-17T06:13:08.301 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_txc_items": 0, 2024-04-17T06:13:08.301 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-17T06:13:08.301 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_writing_deferred_items": 0, 2024-04-17T06:13:08.301 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_writing_bytes": 0, 2024-04-17T06:13:08.301 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_writing_items": 0, 2024-04-17T06:13:08.301 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_bytes": 0, 2024-04-17T06:13:08.301 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_items": 0, 2024-04-17T06:13:08.302 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_file_reader_bytes": 0, 2024-04-17T06:13:08.302 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_file_reader_items": 0, 2024-04-17T06:13:08.302 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_file_writer_bytes": 0, 2024-04-17T06:13:08.302 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_file_writer_items": 0, 2024-04-17T06:13:08.302 INFO:teuthology.orchestra.run.smithi047.stdout: "buffer_anon_bytes": 149007, 2024-04-17T06:13:08.302 INFO:teuthology.orchestra.run.smithi047.stdout: "buffer_anon_items": 30, 2024-04-17T06:13:08.302 INFO:teuthology.orchestra.run.smithi047.stdout: "buffer_meta_bytes": 0, 2024-04-17T06:13:08.302 INFO:teuthology.orchestra.run.smithi047.stdout: "buffer_meta_items": 0, 2024-04-17T06:13:08.302 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_bytes": 0, 2024-04-17T06:13:08.302 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_items": 0, 2024-04-17T06:13:08.302 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_mapbl_bytes": 0, 2024-04-17T06:13:08.302 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_mapbl_items": 0, 2024-04-17T06:13:08.302 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_pglog_bytes": 0, 2024-04-17T06:13:08.303 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_pglog_items": 0, 2024-04-17T06:13:08.303 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_bytes": 0, 2024-04-17T06:13:08.303 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_items": 0, 2024-04-17T06:13:08.303 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_mapping_bytes": 0, 2024-04-17T06:13:08.303 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_mapping_items": 0, 2024-04-17T06:13:08.303 INFO:teuthology.orchestra.run.smithi047.stdout: "pgmap_bytes": 0, 2024-04-17T06:13:08.303 INFO:teuthology.orchestra.run.smithi047.stdout: "pgmap_items": 0, 2024-04-17T06:13:08.303 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_co_bytes": 928, 2024-04-17T06:13:08.303 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_co_items": 2, 2024-04-17T06:13:08.303 INFO:teuthology.orchestra.run.smithi047.stdout: "unittest_1_bytes": 0, 2024-04-17T06:13:08.303 INFO:teuthology.orchestra.run.smithi047.stdout: "unittest_1_items": 0, 2024-04-17T06:13:08.303 INFO:teuthology.orchestra.run.smithi047.stdout: "unittest_2_bytes": 0, 2024-04-17T06:13:08.303 INFO:teuthology.orchestra.run.smithi047.stdout: "unittest_2_items": 0 2024-04-17T06:13:08.303 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:08.304 INFO:teuthology.orchestra.run.smithi047.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-17T06:13:08.304 INFO:teuthology.orchestra.run.smithi047.stdout: "val": 0, 2024-04-17T06:13:08.304 INFO:teuthology.orchestra.run.smithi047.stdout: "max": 104857600, 2024-04-17T06:13:08.304 INFO:teuthology.orchestra.run.smithi047.stdout: "get_started": 0, 2024-04-17T06:13:08.304 INFO:teuthology.orchestra.run.smithi047.stdout: "get": 240, 2024-04-17T06:13:08.304 INFO:teuthology.orchestra.run.smithi047.stdout: "get_sum": 1560745, 2024-04-17T06:13:08.304 INFO:teuthology.orchestra.run.smithi047.stdout: "get_or_fail_fail": 0, 2024-04-17T06:13:08.304 INFO:teuthology.orchestra.run.smithi047.stdout: "get_or_fail_success": 240, 2024-04-17T06:13:08.304 INFO:teuthology.orchestra.run.smithi047.stdout: "take": 0, 2024-04-17T06:13:08.304 INFO:teuthology.orchestra.run.smithi047.stdout: "take_sum": 0, 2024-04-17T06:13:08.304 INFO:teuthology.orchestra.run.smithi047.stdout: "put": 240, 2024-04-17T06:13:08.304 INFO:teuthology.orchestra.run.smithi047.stdout: "put_sum": 1560745, 2024-04-17T06:13:08.304 INFO:teuthology.orchestra.run.smithi047.stdout: "wait": { 2024-04-17T06:13:08.304 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-04-17T06:13:08.304 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-04-17T06:13:08.305 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-04-17T06:13:08.305 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:08.305 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:08.305 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-17T06:13:08.382 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:13:07 smithi098 ceph-mon[36516]: pgmap v372: 97 pgs: 97 active+clean; 5.9 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 58 KiB/s wr, 29 op/s 2024-04-17T06:13:09.165 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-04-17T06:13:09.166 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-04-17T06:13:09.166 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.g 2024-04-17T06:13:09.166 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.g 2024-04-17T06:13:09.166 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.g 2024-04-17T06:13:09.166 DEBUG:tasks.check_counter:Getting stats from j 2024-04-17T06:13:09.166 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 565db2d8-fc7e-11ee-bc91-c7b262605968 -- ceph daemon mds.j perf dump 2024-04-17T06:13:09.418 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T06:13:10.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:13:09 smithi047 ceph-mon[32223]: pgmap v373: 97 pgs: 97 active+clean; 5.9 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 58 KiB/s wr, 29 op/s 2024-04-17T06:13:10.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:13:09 smithi047 ceph-mon[32223]: osdmap e105: 12 total, 12 up, 12 in 2024-04-17T06:13:10.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:13:09 smithi103 ceph-mon[36243]: pgmap v373: 97 pgs: 97 active+clean; 5.9 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 58 KiB/s wr, 29 op/s 2024-04-17T06:13:10.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:13:09 smithi103 ceph-mon[36243]: osdmap e105: 12 total, 12 up, 12 in 2024-04-17T06:13:10.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:13:09 smithi098 ceph-mon[36516]: pgmap v373: 97 pgs: 97 active+clean; 5.9 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 58 KiB/s wr, 29 op/s 2024-04-17T06:13:10.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:13:09 smithi098 ceph-mon[36516]: osdmap e105: 12 total, 12 up, 12 in 2024-04-17T06:13:11.288 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-17T06:13:11.288 INFO:teuthology.orchestra.run.smithi047.stdout: "AsyncMessenger::Worker-0": { 2024-04-17T06:13:11.288 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_messages": 244, 2024-04-17T06:13:11.288 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages": 233, 2024-04-17T06:13:11.288 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_bytes": 1595952, 2024-04-17T06:13:11.288 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_bytes": 143424, 2024-04-17T06:13:11.288 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_created_connections": 4, 2024-04-17T06:13:11.289 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_active_connections": 0, 2024-04-17T06:13:11.289 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_total_time": 0.050194713, 2024-04-17T06:13:11.289 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_send_time": 0.015923351, 2024-04-17T06:13:11.289 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_recv_time": 0.037387286, 2024-04-17T06:13:11.289 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_fast_dispatch_time": 0.004791394, 2024-04-17T06:13:11.289 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T06:13:11.289 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 232, 2024-04-17T06:13:11.289 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.011164956, 2024-04-17T06:13:11.289 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000048124 2024-04-17T06:13:11.289 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:11.289 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_handle_ack_lat": { 2024-04-17T06:13:11.289 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-04-17T06:13:11.289 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-04-17T06:13:11.289 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-04-17T06:13:11.289 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:11.290 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_encrypted_bytes": 1595952, 2024-04-17T06:13:11.290 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_encrypted_bytes": 143424 2024-04-17T06:13:11.290 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:11.290 INFO:teuthology.orchestra.run.smithi047.stdout: "AsyncMessenger::Worker-1": { 2024-04-17T06:13:11.290 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_messages": 6, 2024-04-17T06:13:11.290 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages": 180, 2024-04-17T06:13:11.290 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_bytes": 3344, 2024-04-17T06:13:11.290 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_bytes": 188016, 2024-04-17T06:13:11.290 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_created_connections": 5, 2024-04-17T06:13:11.290 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_active_connections": 3, 2024-04-17T06:13:11.290 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_total_time": 0.015399797, 2024-04-17T06:13:11.290 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_send_time": 0.010501516, 2024-04-17T06:13:11.290 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_recv_time": 0.002233905, 2024-04-17T06:13:11.291 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-17T06:13:11.291 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T06:13:11.291 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 178, 2024-04-17T06:13:11.291 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.018482322, 2024-04-17T06:13:11.291 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000103833 2024-04-17T06:13:11.291 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:11.291 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_handle_ack_lat": { 2024-04-17T06:13:11.291 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-04-17T06:13:11.291 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-04-17T06:13:11.291 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-04-17T06:13:11.291 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:11.291 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-04-17T06:13:11.291 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_encrypted_bytes": 188016 2024-04-17T06:13:11.291 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:11.292 INFO:teuthology.orchestra.run.smithi047.stdout: "AsyncMessenger::Worker-2": { 2024-04-17T06:13:11.292 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_messages": 0, 2024-04-17T06:13:11.292 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages": 2, 2024-04-17T06:13:11.292 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_bytes": 0, 2024-04-17T06:13:11.292 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_bytes": 0, 2024-04-17T06:13:11.292 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_created_connections": 2, 2024-04-17T06:13:11.292 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_active_connections": 1, 2024-04-17T06:13:11.292 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_total_time": 0.000406769, 2024-04-17T06:13:11.292 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_send_time": 0.000000000, 2024-04-17T06:13:11.292 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_recv_time": 0.000341179, 2024-04-17T06:13:11.292 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-17T06:13:11.292 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T06:13:11.292 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-04-17T06:13:11.292 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-04-17T06:13:11.292 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-04-17T06:13:11.293 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:11.293 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_handle_ack_lat": { 2024-04-17T06:13:11.293 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-04-17T06:13:11.293 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-04-17T06:13:11.293 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-04-17T06:13:11.293 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:11.293 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-17T06:13:11.293 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_encrypted_bytes": 0 2024-04-17T06:13:11.293 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:11.293 INFO:teuthology.orchestra.run.smithi047.stdout: "cct": { 2024-04-17T06:13:11.293 INFO:teuthology.orchestra.run.smithi047.stdout: "total_workers": 0, 2024-04-17T06:13:11.293 INFO:teuthology.orchestra.run.smithi047.stdout: "unhealthy_workers": 0 2024-04-17T06:13:11.293 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:11.293 INFO:teuthology.orchestra.run.smithi047.stdout: "mempool": { 2024-04-17T06:13:11.293 INFO:teuthology.orchestra.run.smithi047.stdout: "bloom_filter_bytes": 0, 2024-04-17T06:13:11.293 INFO:teuthology.orchestra.run.smithi047.stdout: "bloom_filter_items": 0, 2024-04-17T06:13:11.294 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_alloc_bytes": 0, 2024-04-17T06:13:11.294 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_alloc_items": 0, 2024-04-17T06:13:11.294 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_data_bytes": 0, 2024-04-17T06:13:11.294 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_data_items": 0, 2024-04-17T06:13:11.294 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-17T06:13:11.294 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_onode_items": 0, 2024-04-17T06:13:11.294 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-17T06:13:11.294 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_meta_items": 0, 2024-04-17T06:13:11.294 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_other_bytes": 0, 2024-04-17T06:13:11.294 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_other_items": 0, 2024-04-17T06:13:11.294 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-17T06:13:11.294 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_buffer_items": 0, 2024-04-17T06:13:11.294 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_extent_bytes": 0, 2024-04-17T06:13:11.294 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_extent_items": 0, 2024-04-17T06:13:11.295 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_blob_bytes": 0, 2024-04-17T06:13:11.295 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_blob_items": 0, 2024-04-17T06:13:11.295 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-17T06:13:11.295 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_shared_blob_items": 0, 2024-04-17T06:13:11.295 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-17T06:13:11.295 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_inline_bl_items": 0, 2024-04-17T06:13:11.295 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_fsck_bytes": 0, 2024-04-17T06:13:11.295 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_fsck_items": 0, 2024-04-17T06:13:11.295 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_txc_bytes": 0, 2024-04-17T06:13:11.295 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_txc_items": 0, 2024-04-17T06:13:11.295 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-17T06:13:11.295 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_writing_deferred_items": 0, 2024-04-17T06:13:11.295 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_writing_bytes": 0, 2024-04-17T06:13:11.295 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_writing_items": 0, 2024-04-17T06:13:11.296 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_bytes": 0, 2024-04-17T06:13:11.296 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_items": 0, 2024-04-17T06:13:11.296 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_file_reader_bytes": 0, 2024-04-17T06:13:11.296 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_file_reader_items": 0, 2024-04-17T06:13:11.296 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_file_writer_bytes": 0, 2024-04-17T06:13:11.296 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_file_writer_items": 0, 2024-04-17T06:13:11.296 INFO:teuthology.orchestra.run.smithi047.stdout: "buffer_anon_bytes": 148994, 2024-04-17T06:13:11.296 INFO:teuthology.orchestra.run.smithi047.stdout: "buffer_anon_items": 31, 2024-04-17T06:13:11.296 INFO:teuthology.orchestra.run.smithi047.stdout: "buffer_meta_bytes": 0, 2024-04-17T06:13:11.296 INFO:teuthology.orchestra.run.smithi047.stdout: "buffer_meta_items": 0, 2024-04-17T06:13:11.296 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_bytes": 0, 2024-04-17T06:13:11.296 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_items": 0, 2024-04-17T06:13:11.296 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_mapbl_bytes": 0, 2024-04-17T06:13:11.296 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_mapbl_items": 0, 2024-04-17T06:13:11.297 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_pglog_bytes": 0, 2024-04-17T06:13:11.297 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_pglog_items": 0, 2024-04-17T06:13:11.297 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_bytes": 0, 2024-04-17T06:13:11.297 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_items": 0, 2024-04-17T06:13:11.297 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_mapping_bytes": 0, 2024-04-17T06:13:11.297 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_mapping_items": 0, 2024-04-17T06:13:11.297 INFO:teuthology.orchestra.run.smithi047.stdout: "pgmap_bytes": 0, 2024-04-17T06:13:11.297 INFO:teuthology.orchestra.run.smithi047.stdout: "pgmap_items": 0, 2024-04-17T06:13:11.297 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_co_bytes": 928, 2024-04-17T06:13:11.297 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_co_items": 2, 2024-04-17T06:13:11.297 INFO:teuthology.orchestra.run.smithi047.stdout: "unittest_1_bytes": 0, 2024-04-17T06:13:11.297 INFO:teuthology.orchestra.run.smithi047.stdout: "unittest_1_items": 0, 2024-04-17T06:13:11.297 INFO:teuthology.orchestra.run.smithi047.stdout: "unittest_2_bytes": 0, 2024-04-17T06:13:11.297 INFO:teuthology.orchestra.run.smithi047.stdout: "unittest_2_items": 0 2024-04-17T06:13:11.297 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-17T06:13:11.298 INFO:teuthology.orchestra.run.smithi047.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-17T06:13:11.298 INFO:teuthology.orchestra.run.smithi047.stdout: "val": 0, 2024-04-17T06:13:11.298 INFO:teuthology.orchestra.run.smithi047.stdout: "max": 104857600, 2024-04-17T06:13:11.298 INFO:teuthology.orchestra.run.smithi047.stdout: "get_started": 0, 2024-04-17T06:13:11.298 INFO:teuthology.orchestra.run.smithi047.stdout: "get": 246, 2024-04-17T06:13:11.298 INFO:teuthology.orchestra.run.smithi047.stdout: "get_sum": 1561525, 2024-04-17T06:13:11.298 INFO:teuthology.orchestra.run.smithi047.stdout: "get_or_fail_fail": 0, 2024-04-17T06:13:11.298 INFO:teuthology.orchestra.run.smithi047.stdout: "get_or_fail_success": 246, 2024-04-17T06:13:11.298 INFO:teuthology.orchestra.run.smithi047.stdout: "take": 0, 2024-04-17T06:13:11.298 INFO:teuthology.orchestra.run.smithi047.stdout: "take_sum": 0, 2024-04-17T06:13:11.298 INFO:teuthology.orchestra.run.smithi047.stdout: "put": 246, 2024-04-17T06:13:11.298 INFO:teuthology.orchestra.run.smithi047.stdout: "put_sum": 1561525, 2024-04-17T06:13:11.298 INFO:teuthology.orchestra.run.smithi047.stdout: "wait": { 2024-04-17T06:13:11.298 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-04-17T06:13:11.299 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-04-17T06:13:11.299 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-04-17T06:13:11.299 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:11.299 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-17T06:13:11.299 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-17T06:13:12.022 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-04-17T06:13:12.023 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-04-17T06:13:12.023 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-04-17T06:13:12.023 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-04-17T06:13:12.023 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-04-17T06:13:12.023 DEBUG:tasks.check_counter:Getting stats from b 2024-04-17T06:13:12.023 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 565db2d8-fc7e-11ee-bc91-c7b262605968 -- ceph daemon mds.b perf dump 2024-04-17T06:13:12.274 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/mon.b/config 2024-04-17T06:13:12.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:13:11 smithi103 ceph-mon[36243]: pgmap v375: 97 pgs: 97 active+clean; 5.9 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 59 KiB/s wr, 28 op/s 2024-04-17T06:13:12.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:13:11 smithi098 ceph-mon[36516]: pgmap v375: 97 pgs: 97 active+clean; 5.9 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 59 KiB/s wr, 28 op/s 2024-04-17T06:13:12.388 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:13:11 smithi047 ceph-mon[32223]: pgmap v375: 97 pgs: 97 active+clean; 5.9 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 59 KiB/s wr, 28 op/s 2024-04-17T06:13:14.011 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:13:13 smithi098 ceph-mon[36516]: pgmap v376: 97 pgs: 97 active+clean; 5.9 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 59 KiB/s wr, 28 op/s 2024-04-17T06:13:14.161 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-04-17T06:13:14.161 INFO:teuthology.orchestra.run.smithi098.stdout: "AsyncMessenger::Worker-0": { 2024-04-17T06:13:14.161 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_messages": 1266, 2024-04-17T06:13:14.161 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages": 905, 2024-04-17T06:13:14.161 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_bytes": 2077578, 2024-04-17T06:13:14.161 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_bytes": 1586100, 2024-04-17T06:13:14.161 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_created_connections": 101, 2024-04-17T06:13:14.161 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_active_connections": 4, 2024-04-17T06:13:14.161 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_total_time": 0.183578494, 2024-04-17T06:13:14.161 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_send_time": 0.049752698, 2024-04-17T06:13:14.161 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_recv_time": 0.095138017, 2024-04-17T06:13:14.162 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_fast_dispatch_time": 0.034560745, 2024-04-17T06:13:14.162 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T06:13:14.162 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 908, 2024-04-17T06:13:14.162 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.381590401, 2024-04-17T06:13:14.162 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000420253 2024-04-17T06:13:14.162 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:14.162 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_handle_ack_lat": { 2024-04-17T06:13:14.162 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 1000, 2024-04-17T06:13:14.162 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000183810, 2024-04-17T06:13:14.163 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000183 2024-04-17T06:13:14.163 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:14.163 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_encrypted_bytes": 1742928, 2024-04-17T06:13:14.163 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_encrypted_bytes": 154432 2024-04-17T06:13:14.163 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:14.163 INFO:teuthology.orchestra.run.smithi098.stdout: "AsyncMessenger::Worker-1": { 2024-04-17T06:13:14.163 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_messages": 1765, 2024-04-17T06:13:14.163 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages": 1146, 2024-04-17T06:13:14.163 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_bytes": 906023, 2024-04-17T06:13:14.163 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_bytes": 1227198, 2024-04-17T06:13:14.163 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_created_connections": 15, 2024-04-17T06:13:14.163 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_active_connections": 6, 2024-04-17T06:13:14.163 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_total_time": 0.220351954, 2024-04-17T06:13:14.164 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_send_time": 0.058102037, 2024-04-17T06:13:14.164 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_recv_time": 0.082740678, 2024-04-17T06:13:14.164 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_fast_dispatch_time": 0.076030636, 2024-04-17T06:13:14.164 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T06:13:14.164 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 1146, 2024-04-17T06:13:14.164 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.434819636, 2024-04-17T06:13:14.164 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000379423 2024-04-17T06:13:14.164 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:14.164 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_handle_ack_lat": { 2024-04-17T06:13:14.164 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 2172, 2024-04-17T06:13:14.164 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000391544, 2024-04-17T06:13:14.164 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000180 2024-04-17T06:13:14.165 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:14.165 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_encrypted_bytes": 358064, 2024-04-17T06:13:14.165 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_encrypted_bytes": 709360 2024-04-17T06:13:14.165 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:14.165 INFO:teuthology.orchestra.run.smithi098.stdout: "AsyncMessenger::Worker-2": { 2024-04-17T06:13:14.165 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_messages": 1747, 2024-04-17T06:13:14.165 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages": 1015, 2024-04-17T06:13:14.165 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_bytes": 597239, 2024-04-17T06:13:14.165 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_bytes": 3849560, 2024-04-17T06:13:14.165 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_created_connections": 23, 2024-04-17T06:13:14.165 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_active_connections": 8, 2024-04-17T06:13:14.165 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_total_time": 0.198310045, 2024-04-17T06:13:14.165 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_send_time": 0.047213207, 2024-04-17T06:13:14.166 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_recv_time": 0.068773396, 2024-04-17T06:13:14.166 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_fast_dispatch_time": 0.075474921, 2024-04-17T06:13:14.166 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T06:13:14.166 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 1007, 2024-04-17T06:13:14.166 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.869286147, 2024-04-17T06:13:14.166 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000863243 2024-04-17T06:13:14.166 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:14.166 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_handle_ack_lat": { 2024-04-17T06:13:14.166 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 1029, 2024-04-17T06:13:14.166 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000181328, 2024-04-17T06:13:14.166 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000176 2024-04-17T06:13:14.166 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:14.166 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_encrypted_bytes": 58288, 2024-04-17T06:13:14.167 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_encrypted_bytes": 31152 2024-04-17T06:13:14.167 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:14.167 INFO:teuthology.orchestra.run.smithi098.stdout: "cct": { 2024-04-17T06:13:14.167 INFO:teuthology.orchestra.run.smithi098.stdout: "total_workers": 1, 2024-04-17T06:13:14.167 INFO:teuthology.orchestra.run.smithi098.stdout: "unhealthy_workers": 0 2024-04-17T06:13:14.167 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:14.167 INFO:teuthology.orchestra.run.smithi098.stdout: "finisher-MDSRank": { 2024-04-17T06:13:14.167 INFO:teuthology.orchestra.run.smithi098.stdout: "queue_len": 0, 2024-04-17T06:13:14.167 INFO:teuthology.orchestra.run.smithi098.stdout: "complete_latency": { 2024-04-17T06:13:14.167 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 1195, 2024-04-17T06:13:14.168 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.172895780, 2024-04-17T06:13:14.168 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000144682 2024-04-17T06:13:14.168 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-04-17T06:13:14.168 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:14.168 INFO:teuthology.orchestra.run.smithi098.stdout: "finisher-PurgeQueue": { 2024-04-17T06:13:14.168 INFO:teuthology.orchestra.run.smithi098.stdout: "queue_len": 0, 2024-04-17T06:13:14.168 INFO:teuthology.orchestra.run.smithi098.stdout: "complete_latency": { 2024-04-17T06:13:14.168 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 2, 2024-04-17T06:13:14.168 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000018122, 2024-04-17T06:13:14.168 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000009061 2024-04-17T06:13:14.168 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-04-17T06:13:14.168 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:14.168 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": { 2024-04-17T06:13:14.168 INFO:teuthology.orchestra.run.smithi098.stdout: "request": 186, 2024-04-17T06:13:14.169 INFO:teuthology.orchestra.run.smithi098.stdout: "reply": 185, 2024-04-17T06:13:14.169 INFO:teuthology.orchestra.run.smithi098.stdout: "reply_latency": { 2024-04-17T06:13:14.169 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 185, 2024-04-17T06:13:14.169 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.185795756, 2024-04-17T06:13:14.169 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.001004301 2024-04-17T06:13:14.169 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:14.169 INFO:teuthology.orchestra.run.smithi098.stdout: "slow_reply": 0, 2024-04-17T06:13:14.169 INFO:teuthology.orchestra.run.smithi098.stdout: "forward": 1, 2024-04-17T06:13:14.169 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_fetch_complete": 0, 2024-04-17T06:13:14.169 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_fetch_keys": 0, 2024-04-17T06:13:14.169 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_commit": 301, 2024-04-17T06:13:14.169 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_split": 0, 2024-04-17T06:13:14.169 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_merge": 0, 2024-04-17T06:13:14.169 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes": 42, 2024-04-17T06:13:14.170 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_top": 7, 2024-04-17T06:13:14.170 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_bottom": 35, 2024-04-17T06:13:14.170 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_pin_tail": 0, 2024-04-17T06:13:14.170 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_pinned": 32, 2024-04-17T06:13:14.170 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_expired": 1, 2024-04-17T06:13:14.170 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_with_caps": 3, 2024-04-17T06:13:14.170 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": 3, 2024-04-17T06:13:14.170 INFO:teuthology.orchestra.run.smithi098.stdout: "subtrees": 2, 2024-04-17T06:13:14.170 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse": 229, 2024-04-17T06:13:14.170 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_hit": 220, 2024-04-17T06:13:14.170 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_forward": 1, 2024-04-17T06:13:14.170 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_discover": 0, 2024-04-17T06:13:14.171 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_dir_fetch": 0, 2024-04-17T06:13:14.171 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_remote_ino": 0, 2024-04-17T06:13:14.171 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_lock": 0, 2024-04-17T06:13:14.171 INFO:teuthology.orchestra.run.smithi098.stdout: "load_cent": 0, 2024-04-17T06:13:14.171 INFO:teuthology.orchestra.run.smithi098.stdout: "q": 0, 2024-04-17T06:13:14.171 INFO:teuthology.orchestra.run.smithi098.stdout: "exported": 1, 2024-04-17T06:13:14.171 INFO:teuthology.orchestra.run.smithi098.stdout: "exported_inodes": 1, 2024-04-17T06:13:14.171 INFO:teuthology.orchestra.run.smithi098.stdout: "imported": 1, 2024-04-17T06:13:14.171 INFO:teuthology.orchestra.run.smithi098.stdout: "imported_inodes": 11, 2024-04-17T06:13:14.171 INFO:teuthology.orchestra.run.smithi098.stdout: "openino_dir_fetch": 0, 2024-04-17T06:13:14.171 INFO:teuthology.orchestra.run.smithi098.stdout: "openino_backtrace_fetch": 0, 2024-04-17T06:13:14.171 INFO:teuthology.orchestra.run.smithi098.stdout: "openino_peer_discover": 0, 2024-04-17T06:13:14.171 INFO:teuthology.orchestra.run.smithi098.stdout: "root_rfiles": 4, 2024-04-17T06:13:14.171 INFO:teuthology.orchestra.run.smithi098.stdout: "root_rbytes": 234, 2024-04-17T06:13:14.171 INFO:teuthology.orchestra.run.smithi098.stdout: "root_rsnaps": 6, 2024-04-17T06:13:14.172 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_backtrace_fetch": 0, 2024-04-17T06:13:14.172 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_set_tag": 0, 2024-04-17T06:13:14.172 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_backtrace_repaired": 0, 2024-04-17T06:13:14.172 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_inotable_repaired": 0, 2024-04-17T06:13:14.172 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_dir_inodes": 0, 2024-04-17T06:13:14.172 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_dir_base_inodes": 0, 2024-04-17T06:13:14.172 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_dirfrag_rstats": 0, 2024-04-17T06:13:14.172 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_file_inodes": 0, 2024-04-17T06:13:14.172 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_inode_file_caps": 2, 2024-04-17T06:13:14.172 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_revoke": 14, 2024-04-17T06:13:14.172 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_grant": 12, 2024-04-17T06:13:14.172 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_trunc": 1, 2024-04-17T06:13:14.172 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-17T06:13:14.173 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-17T06:13:14.173 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_caps": 21, 2024-04-17T06:13:14.173 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_caps_dirty": 4, 2024-04-17T06:13:14.173 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_cap_release": 2, 2024-04-17T06:13:14.173 INFO:teuthology.orchestra.run.smithi098.stdout: "process_request_cap_release": 22 2024-04-17T06:13:14.173 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:14.173 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_cache": { 2024-04-17T06:13:14.173 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_update": 0, 2024-04-17T06:13:14.173 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_update_receipt": 0, 2024-04-17T06:13:14.173 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_try_discover": 0, 2024-04-17T06:13:14.174 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_send_discover": 0, 2024-04-17T06:13:14.174 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_handle_discover": 7, 2024-04-17T06:13:14.174 INFO:teuthology.orchestra.run.smithi098.stdout: "num_strays": 2, 2024-04-17T06:13:14.174 INFO:teuthology.orchestra.run.smithi098.stdout: "num_strays_delayed": 0, 2024-04-17T06:13:14.174 INFO:teuthology.orchestra.run.smithi098.stdout: "num_strays_enqueuing": 0, 2024-04-17T06:13:14.174 INFO:teuthology.orchestra.run.smithi098.stdout: "strays_created": 2, 2024-04-17T06:13:14.174 INFO:teuthology.orchestra.run.smithi098.stdout: "strays_enqueued": 0, 2024-04-17T06:13:14.174 INFO:teuthology.orchestra.run.smithi098.stdout: "strays_reintegrated": 0, 2024-04-17T06:13:14.174 INFO:teuthology.orchestra.run.smithi098.stdout: "strays_migrated": 0, 2024-04-17T06:13:14.174 INFO:teuthology.orchestra.run.smithi098.stdout: "num_recovering_processing": 0, 2024-04-17T06:13:14.174 INFO:teuthology.orchestra.run.smithi098.stdout: "num_recovering_enqueued": 0, 2024-04-17T06:13:14.174 INFO:teuthology.orchestra.run.smithi098.stdout: "num_recovering_prioritized": 0, 2024-04-17T06:13:14.174 INFO:teuthology.orchestra.run.smithi098.stdout: "recovery_started": 0, 2024-04-17T06:13:14.174 INFO:teuthology.orchestra.run.smithi098.stdout: "recovery_completed": 0, 2024-04-17T06:13:14.175 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_quiesce_path": 0, 2024-04-17T06:13:14.175 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_quiesce_inode": 0, 2024-04-17T06:13:14.175 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_enqueue_scrub": 0, 2024-04-17T06:13:14.175 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_exportdir": 1, 2024-04-17T06:13:14.175 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_flush": 0, 2024-04-17T06:13:14.175 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_fragmentdir": 0, 2024-04-17T06:13:14.175 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_fragstats": 0, 2024-04-17T06:13:14.175 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_inodestats": 0 2024-04-17T06:13:14.175 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:14.175 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_log": { 2024-04-17T06:13:14.175 INFO:teuthology.orchestra.run.smithi098.stdout: "evlrg": 0, 2024-04-17T06:13:14.175 INFO:teuthology.orchestra.run.smithi098.stdout: "evadd": 249, 2024-04-17T06:13:14.175 INFO:teuthology.orchestra.run.smithi098.stdout: "evex": 228, 2024-04-17T06:13:14.175 INFO:teuthology.orchestra.run.smithi098.stdout: "evtrm": 228, 2024-04-17T06:13:14.175 INFO:teuthology.orchestra.run.smithi098.stdout: "ev": 21, 2024-04-17T06:13:14.176 INFO:teuthology.orchestra.run.smithi098.stdout: "evexg": 0, 2024-04-17T06:13:14.176 INFO:teuthology.orchestra.run.smithi098.stdout: "evexd": 2, 2024-04-17T06:13:14.176 INFO:teuthology.orchestra.run.smithi098.stdout: "segadd": 54, 2024-04-17T06:13:14.176 INFO:teuthology.orchestra.run.smithi098.stdout: "segex": 53, 2024-04-17T06:13:14.176 INFO:teuthology.orchestra.run.smithi098.stdout: "segtrm": 53, 2024-04-17T06:13:14.176 INFO:teuthology.orchestra.run.smithi098.stdout: "seg": 1, 2024-04-17T06:13:14.176 INFO:teuthology.orchestra.run.smithi098.stdout: "segmjr": 1, 2024-04-17T06:13:14.176 INFO:teuthology.orchestra.run.smithi098.stdout: "segexg": 0, 2024-04-17T06:13:14.176 INFO:teuthology.orchestra.run.smithi098.stdout: "segexd": 1, 2024-04-17T06:13:14.176 INFO:teuthology.orchestra.run.smithi098.stdout: "expos": 6683816, 2024-04-17T06:13:14.176 INFO:teuthology.orchestra.run.smithi098.stdout: "wrpos": 6814585, 2024-04-17T06:13:14.176 INFO:teuthology.orchestra.run.smithi098.stdout: "rdpos": 0, 2024-04-17T06:13:14.176 INFO:teuthology.orchestra.run.smithi098.stdout: "jlat": { 2024-04-17T06:13:14.176 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 213, 2024-04-17T06:13:14.177 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.407125964, 2024-04-17T06:13:14.177 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.001911389 2024-04-17T06:13:14.177 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:14.177 INFO:teuthology.orchestra.run.smithi098.stdout: "replayed": 0 2024-04-17T06:13:14.177 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:14.177 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_mem": { 2024-04-17T06:13:14.177 INFO:teuthology.orchestra.run.smithi098.stdout: "ino": 35, 2024-04-17T06:13:14.177 INFO:teuthology.orchestra.run.smithi098.stdout: "ino+": 36, 2024-04-17T06:13:14.177 INFO:teuthology.orchestra.run.smithi098.stdout: "ino-": 1, 2024-04-17T06:13:14.177 INFO:teuthology.orchestra.run.smithi098.stdout: "dir": 20, 2024-04-17T06:13:14.177 INFO:teuthology.orchestra.run.smithi098.stdout: "dir+": 20, 2024-04-17T06:13:14.177 INFO:teuthology.orchestra.run.smithi098.stdout: "dir-": 0, 2024-04-17T06:13:14.177 INFO:teuthology.orchestra.run.smithi098.stdout: "dn": 42, 2024-04-17T06:13:14.177 INFO:teuthology.orchestra.run.smithi098.stdout: "dn+": 43, 2024-04-17T06:13:14.177 INFO:teuthology.orchestra.run.smithi098.stdout: "dn-": 1, 2024-04-17T06:13:14.178 INFO:teuthology.orchestra.run.smithi098.stdout: "cap": 3, 2024-04-17T06:13:14.178 INFO:teuthology.orchestra.run.smithi098.stdout: "cap+": 19, 2024-04-17T06:13:14.178 INFO:teuthology.orchestra.run.smithi098.stdout: "cap-": 16, 2024-04-17T06:13:14.178 INFO:teuthology.orchestra.run.smithi098.stdout: "rss": 52704, 2024-04-17T06:13:14.178 INFO:teuthology.orchestra.run.smithi098.stdout: "heap": 231800 2024-04-17T06:13:14.178 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:14.178 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_server": { 2024-04-17T06:13:14.178 INFO:teuthology.orchestra.run.smithi098.stdout: "dispatch_client_request": 247, 2024-04-17T06:13:14.178 INFO:teuthology.orchestra.run.smithi098.stdout: "dispatch_server_request": 5, 2024-04-17T06:13:14.179 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_request": 186, 2024-04-17T06:13:14.179 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_session": 30, 2024-04-17T06:13:14.179 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_peer_request": 12, 2024-04-17T06:13:14.179 INFO:teuthology.orchestra.run.smithi098.stdout: "req_create_latency": { 2024-04-17T06:13:14.179 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 5, 2024-04-17T06:13:14.179 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.005692465, 2024-04-17T06:13:14.179 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.001138493 2024-04-17T06:13:14.179 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:14.179 INFO:teuthology.orchestra.run.smithi098.stdout: "req_getattr_latency": { 2024-04-17T06:13:14.179 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 124, 2024-04-17T06:13:14.179 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.024466134, 2024-04-17T06:13:14.179 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000197307 2024-04-17T06:13:14.179 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:14.179 INFO:teuthology.orchestra.run.smithi098.stdout: "req_getfilelock_latency": { 2024-04-17T06:13:14.179 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T06:13:14.180 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T06:13:14.180 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T06:13:14.180 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:14.180 INFO:teuthology.orchestra.run.smithi098.stdout: "req_link_latency": { 2024-04-17T06:13:14.180 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T06:13:14.180 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T06:13:14.180 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T06:13:14.180 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:14.180 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookup_latency": { 2024-04-17T06:13:14.180 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 1, 2024-04-17T06:13:14.180 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000261664, 2024-04-17T06:13:14.180 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000261664 2024-04-17T06:13:14.180 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:14.180 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookuphash_latency": { 2024-04-17T06:13:14.181 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T06:13:14.181 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T06:13:14.181 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T06:13:14.181 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:14.181 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookupino_latency": { 2024-04-17T06:13:14.181 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 1, 2024-04-17T06:13:14.181 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000113884, 2024-04-17T06:13:14.181 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000113884 2024-04-17T06:13:14.181 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:14.181 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookupname_latency": { 2024-04-17T06:13:14.181 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T06:13:14.181 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T06:13:14.181 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T06:13:14.181 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:14.181 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookupparent_latency": { 2024-04-17T06:13:14.182 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T06:13:14.182 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T06:13:14.182 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T06:13:14.182 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:14.182 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookupsnap_latency": { 2024-04-17T06:13:14.182 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T06:13:14.182 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T06:13:14.182 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T06:13:14.182 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:14.182 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lssnap_latency": { 2024-04-17T06:13:14.183 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 12, 2024-04-17T06:13:14.183 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.002272381, 2024-04-17T06:13:14.183 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000189365 2024-04-17T06:13:14.183 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:14.184 INFO:teuthology.orchestra.run.smithi098.stdout: "req_mkdir_latency": { 2024-04-17T06:13:14.184 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 8, 2024-04-17T06:13:14.184 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.013591083, 2024-04-17T06:13:14.184 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.001698885 2024-04-17T06:13:14.184 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:14.185 INFO:teuthology.orchestra.run.smithi098.stdout: "req_mknod_latency": { 2024-04-17T06:13:14.185 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T06:13:14.185 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T06:13:14.185 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T06:13:14.185 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:14.185 INFO:teuthology.orchestra.run.smithi098.stdout: "req_mksnap_latency": { 2024-04-17T06:13:14.185 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 12, 2024-04-17T06:13:14.185 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.057308240, 2024-04-17T06:13:14.185 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.004775686 2024-04-17T06:13:14.185 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:14.185 INFO:teuthology.orchestra.run.smithi098.stdout: "req_open_latency": { 2024-04-17T06:13:14.185 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 1, 2024-04-17T06:13:14.185 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000221854, 2024-04-17T06:13:14.185 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000221854 2024-04-17T06:13:14.186 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:14.186 INFO:teuthology.orchestra.run.smithi098.stdout: "req_readdir_latency": { 2024-04-17T06:13:14.186 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 1, 2024-04-17T06:13:14.186 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000411317, 2024-04-17T06:13:14.186 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000411317 2024-04-17T06:13:14.186 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:14.186 INFO:teuthology.orchestra.run.smithi098.stdout: "req_rename_latency": { 2024-04-17T06:13:14.186 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 2, 2024-04-17T06:13:14.186 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.003894746, 2024-04-17T06:13:14.186 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.001947373 2024-04-17T06:13:14.186 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:14.186 INFO:teuthology.orchestra.run.smithi098.stdout: "req_renamesnap_latency": { 2024-04-17T06:13:14.186 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T06:13:14.186 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T06:13:14.187 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T06:13:14.187 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:14.187 INFO:teuthology.orchestra.run.smithi098.stdout: "req_snapdiff_latency": { 2024-04-17T06:13:14.187 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T06:13:14.187 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T06:13:14.187 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T06:13:14.187 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:14.187 INFO:teuthology.orchestra.run.smithi098.stdout: "req_rmdir_latency": { 2024-04-17T06:13:14.187 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 2, 2024-04-17T06:13:14.187 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.020165693, 2024-04-17T06:13:14.187 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.010082846 2024-04-17T06:13:14.187 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:14.187 INFO:teuthology.orchestra.run.smithi098.stdout: "req_rmsnap_latency": { 2024-04-17T06:13:14.187 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 6, 2024-04-17T06:13:14.188 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.026937764, 2024-04-17T06:13:14.188 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.004489627 2024-04-17T06:13:14.188 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:14.188 INFO:teuthology.orchestra.run.smithi098.stdout: "req_rmxattr_latency": { 2024-04-17T06:13:14.188 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T06:13:14.188 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T06:13:14.188 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T06:13:14.188 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:14.188 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setattr_latency": { 2024-04-17T06:13:14.188 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 2, 2024-04-17T06:13:14.188 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.007114717, 2024-04-17T06:13:14.188 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.003557358 2024-04-17T06:13:14.189 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:14.189 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setdirlayout_latency": { 2024-04-17T06:13:14.189 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T06:13:14.189 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T06:13:14.189 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T06:13:14.189 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:14.189 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setfilelock_latency": { 2024-04-17T06:13:14.189 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T06:13:14.189 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T06:13:14.189 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T06:13:14.189 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:14.189 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setlayout_latency": { 2024-04-17T06:13:14.189 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T06:13:14.189 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T06:13:14.190 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T06:13:14.190 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:14.190 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setxattr_latency": { 2024-04-17T06:13:14.190 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 8, 2024-04-17T06:13:14.190 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.023343814, 2024-04-17T06:13:14.190 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.002917976 2024-04-17T06:13:14.190 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:14.190 INFO:teuthology.orchestra.run.smithi098.stdout: "req_symlink_latency": { 2024-04-17T06:13:14.190 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T06:13:14.190 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T06:13:14.190 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T06:13:14.190 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:14.191 INFO:teuthology.orchestra.run.smithi098.stdout: "req_unlink_latency": { 2024-04-17T06:13:14.191 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T06:13:14.191 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T06:13:14.191 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T06:13:14.191 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:14.191 INFO:teuthology.orchestra.run.smithi098.stdout: "cap_revoke_eviction": 0, 2024-04-17T06:13:14.191 INFO:teuthology.orchestra.run.smithi098.stdout: "cap_acquisition_throttle": 0, 2024-04-17T06:13:14.191 INFO:teuthology.orchestra.run.smithi098.stdout: "req_getvxattr_latency": { 2024-04-17T06:13:14.191 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T06:13:14.191 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T06:13:14.191 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T06:13:14.191 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-04-17T06:13:14.191 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:14.191 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_sessions": { 2024-04-17T06:13:14.192 INFO:teuthology.orchestra.run.smithi098.stdout: "session_count": 2, 2024-04-17T06:13:14.192 INFO:teuthology.orchestra.run.smithi098.stdout: "session_add": 4, 2024-04-17T06:13:14.192 INFO:teuthology.orchestra.run.smithi098.stdout: "session_remove": 2, 2024-04-17T06:13:14.192 INFO:teuthology.orchestra.run.smithi098.stdout: "sessions_open": 2, 2024-04-17T06:13:14.192 INFO:teuthology.orchestra.run.smithi098.stdout: "sessions_stale": 0, 2024-04-17T06:13:14.192 INFO:teuthology.orchestra.run.smithi098.stdout: "total_load": 31, 2024-04-17T06:13:14.192 INFO:teuthology.orchestra.run.smithi098.stdout: "average_load": 15, 2024-04-17T06:13:14.192 INFO:teuthology.orchestra.run.smithi098.stdout: "avg_session_uptime": 695, 2024-04-17T06:13:14.192 INFO:teuthology.orchestra.run.smithi098.stdout: "mdthresh_evicted": 0 2024-04-17T06:13:14.192 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:14.192 INFO:teuthology.orchestra.run.smithi098.stdout: "mempool": { 2024-04-17T06:13:14.192 INFO:teuthology.orchestra.run.smithi098.stdout: "bloom_filter_bytes": 0, 2024-04-17T06:13:14.192 INFO:teuthology.orchestra.run.smithi098.stdout: "bloom_filter_items": 0, 2024-04-17T06:13:14.192 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_alloc_bytes": 0, 2024-04-17T06:13:14.193 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_alloc_items": 0, 2024-04-17T06:13:14.193 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_data_bytes": 0, 2024-04-17T06:13:14.193 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_data_items": 0, 2024-04-17T06:13:14.193 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-17T06:13:14.193 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_onode_items": 0, 2024-04-17T06:13:14.193 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-17T06:13:14.193 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_meta_items": 0, 2024-04-17T06:13:14.193 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_other_bytes": 0, 2024-04-17T06:13:14.193 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_other_items": 0, 2024-04-17T06:13:14.193 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-17T06:13:14.193 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_buffer_items": 0, 2024-04-17T06:13:14.193 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_extent_bytes": 0, 2024-04-17T06:13:14.193 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_extent_items": 0, 2024-04-17T06:13:14.193 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_blob_bytes": 0, 2024-04-17T06:13:14.193 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_blob_items": 0, 2024-04-17T06:13:14.194 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-17T06:13:14.194 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_shared_blob_items": 0, 2024-04-17T06:13:14.194 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-17T06:13:14.194 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_inline_bl_items": 0, 2024-04-17T06:13:14.194 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_fsck_bytes": 0, 2024-04-17T06:13:14.194 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_fsck_items": 0, 2024-04-17T06:13:14.194 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_txc_bytes": 0, 2024-04-17T06:13:14.194 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_txc_items": 0, 2024-04-17T06:13:14.194 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-17T06:13:14.194 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_deferred_items": 0, 2024-04-17T06:13:14.194 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_bytes": 0, 2024-04-17T06:13:14.194 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_items": 0, 2024-04-17T06:13:14.195 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_bytes": 0, 2024-04-17T06:13:14.195 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_items": 0, 2024-04-17T06:13:14.195 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_reader_bytes": 0, 2024-04-17T06:13:14.195 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_reader_items": 0, 2024-04-17T06:13:14.195 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_writer_bytes": 0, 2024-04-17T06:13:14.195 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_writer_items": 0, 2024-04-17T06:13:14.195 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_anon_bytes": 374662, 2024-04-17T06:13:14.195 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_anon_items": 154, 2024-04-17T06:13:14.195 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_meta_bytes": 616, 2024-04-17T06:13:14.195 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_meta_items": 7, 2024-04-17T06:13:14.195 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_bytes": 0, 2024-04-17T06:13:14.195 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_items": 0, 2024-04-17T06:13:14.195 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_mapbl_bytes": 0, 2024-04-17T06:13:14.196 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_mapbl_items": 0, 2024-04-17T06:13:14.196 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_pglog_bytes": 0, 2024-04-17T06:13:14.196 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_pglog_items": 0, 2024-04-17T06:13:14.196 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_bytes": 8152, 2024-04-17T06:13:14.196 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_items": 170, 2024-04-17T06:13:14.196 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_mapping_bytes": 0, 2024-04-17T06:13:14.196 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_mapping_items": 0, 2024-04-17T06:13:14.196 INFO:teuthology.orchestra.run.smithi098.stdout: "pgmap_bytes": 0, 2024-04-17T06:13:14.196 INFO:teuthology.orchestra.run.smithi098.stdout: "pgmap_items": 0, 2024-04-17T06:13:14.196 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_co_bytes": 179516, 2024-04-17T06:13:14.196 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_co_items": 1472, 2024-04-17T06:13:14.196 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_1_bytes": 0, 2024-04-17T06:13:14.197 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_1_items": 0, 2024-04-17T06:13:14.197 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_2_bytes": 0, 2024-04-17T06:13:14.197 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_2_items": 0 2024-04-17T06:13:14.197 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:14.197 INFO:teuthology.orchestra.run.smithi098.stdout: "objecter": { 2024-04-17T06:13:14.197 INFO:teuthology.orchestra.run.smithi098.stdout: "op_active": 0, 2024-04-17T06:13:14.197 INFO:teuthology.orchestra.run.smithi098.stdout: "op_laggy": 0, 2024-04-17T06:13:14.197 INFO:teuthology.orchestra.run.smithi098.stdout: "op_send": 1312, 2024-04-17T06:13:14.197 INFO:teuthology.orchestra.run.smithi098.stdout: "op_send_bytes": 3684276, 2024-04-17T06:13:14.197 INFO:teuthology.orchestra.run.smithi098.stdout: "op_resend": 35, 2024-04-17T06:13:14.198 INFO:teuthology.orchestra.run.smithi098.stdout: "op_reply": 1312, 2024-04-17T06:13:14.198 INFO:teuthology.orchestra.run.smithi098.stdout: "op_latency": { 2024-04-17T06:13:14.198 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 1312, 2024-04-17T06:13:14.198 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 21.705703588, 2024-04-17T06:13:14.198 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.016543981 2024-04-17T06:13:14.198 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:14.198 INFO:teuthology.orchestra.run.smithi098.stdout: "op_inflight": 0, 2024-04-17T06:13:14.198 INFO:teuthology.orchestra.run.smithi098.stdout: "oplen_avg": { 2024-04-17T06:13:14.198 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 1312, 2024-04-17T06:13:14.198 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 1958 2024-04-17T06:13:14.198 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:14.198 INFO:teuthology.orchestra.run.smithi098.stdout: "op": 1312, 2024-04-17T06:13:14.198 INFO:teuthology.orchestra.run.smithi098.stdout: "op_r": 0, 2024-04-17T06:13:14.198 INFO:teuthology.orchestra.run.smithi098.stdout: "op_w": 1312, 2024-04-17T06:13:14.199 INFO:teuthology.orchestra.run.smithi098.stdout: "op_rmw": 0, 2024-04-17T06:13:14.199 INFO:teuthology.orchestra.run.smithi098.stdout: "op_pg": 0, 2024-04-17T06:13:14.199 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_stat": 282, 2024-04-17T06:13:14.199 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_create": 24, 2024-04-17T06:13:14.199 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_read": 0, 2024-04-17T06:13:14.199 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_write": 213, 2024-04-17T06:13:14.199 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_writefull": 182, 2024-04-17T06:13:14.199 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_writesame": 0, 2024-04-17T06:13:14.199 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_append": 0, 2024-04-17T06:13:14.199 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_zero": 0, 2024-04-17T06:13:14.200 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_truncate": 0, 2024-04-17T06:13:14.200 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_delete": 506, 2024-04-17T06:13:14.200 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_mapext": 0, 2024-04-17T06:13:14.200 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_sparse_read": 0, 2024-04-17T06:13:14.200 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_clonerange": 0, 2024-04-17T06:13:14.200 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_getxattr": 0, 2024-04-17T06:13:14.200 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_setxattr": 48, 2024-04-17T06:13:14.200 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_cmpxattr": 0, 2024-04-17T06:13:14.200 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_rmxattr": 0, 2024-04-17T06:13:14.200 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_resetxattrs": 0, 2024-04-17T06:13:14.200 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_call": 0, 2024-04-17T06:13:14.200 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_watch": 0, 2024-04-17T06:13:14.200 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_notify": 0, 2024-04-17T06:13:14.200 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_src_cmpxattr": 0, 2024-04-17T06:13:14.201 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_pgls": 0, 2024-04-17T06:13:14.201 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_pgls_filter": 0, 2024-04-17T06:13:14.201 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_other": 1, 2024-04-17T06:13:14.201 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_active": 0, 2024-04-17T06:13:14.201 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_send": 0, 2024-04-17T06:13:14.201 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_resend": 0, 2024-04-17T06:13:14.201 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_ping": 0, 2024-04-17T06:13:14.201 INFO:teuthology.orchestra.run.smithi098.stdout: "poolop_active": 0, 2024-04-17T06:13:14.201 INFO:teuthology.orchestra.run.smithi098.stdout: "poolop_send": 0, 2024-04-17T06:13:14.201 INFO:teuthology.orchestra.run.smithi098.stdout: "poolop_resend": 0, 2024-04-17T06:13:14.201 INFO:teuthology.orchestra.run.smithi098.stdout: "poolstat_active": 0, 2024-04-17T06:13:14.201 INFO:teuthology.orchestra.run.smithi098.stdout: "poolstat_send": 0, 2024-04-17T06:13:14.201 INFO:teuthology.orchestra.run.smithi098.stdout: "poolstat_resend": 0, 2024-04-17T06:13:14.201 INFO:teuthology.orchestra.run.smithi098.stdout: "statfs_active": 0, 2024-04-17T06:13:14.202 INFO:teuthology.orchestra.run.smithi098.stdout: "statfs_send": 0, 2024-04-17T06:13:14.202 INFO:teuthology.orchestra.run.smithi098.stdout: "statfs_resend": 0, 2024-04-17T06:13:14.202 INFO:teuthology.orchestra.run.smithi098.stdout: "command_active": 0, 2024-04-17T06:13:14.202 INFO:teuthology.orchestra.run.smithi098.stdout: "command_send": 0, 2024-04-17T06:13:14.202 INFO:teuthology.orchestra.run.smithi098.stdout: "command_resend": 0, 2024-04-17T06:13:14.202 INFO:teuthology.orchestra.run.smithi098.stdout: "map_epoch": 105, 2024-04-17T06:13:14.202 INFO:teuthology.orchestra.run.smithi098.stdout: "map_full": 0, 2024-04-17T06:13:14.202 INFO:teuthology.orchestra.run.smithi098.stdout: "map_inc": 20, 2024-04-17T06:13:14.202 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_sessions": 12, 2024-04-17T06:13:14.202 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_session_open": 12, 2024-04-17T06:13:14.202 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_session_close": 0, 2024-04-17T06:13:14.202 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_laggy": 0, 2024-04-17T06:13:14.202 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_wr": 690, 2024-04-17T06:13:14.202 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_rd": 0, 2024-04-17T06:13:14.203 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_del": 12 2024-04-17T06:13:14.203 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:14.203 INFO:teuthology.orchestra.run.smithi098.stdout: "oft": { 2024-04-17T06:13:14.203 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_total_objs": 1, 2024-04-17T06:13:14.203 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_total_kv_pairs": 0, 2024-04-17T06:13:14.203 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_total_updates": 0, 2024-04-17T06:13:14.203 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_total_removes": 0 2024-04-17T06:13:14.203 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:14.203 INFO:teuthology.orchestra.run.smithi098.stdout: "osd-slow-ops": { 2024-04-17T06:13:14.203 INFO:teuthology.orchestra.run.smithi098.stdout: "slow_ops_count": 0 2024-04-17T06:13:14.203 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:14.203 INFO:teuthology.orchestra.run.smithi098.stdout: "purge_queue": { 2024-04-17T06:13:14.203 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executing_ops": 0, 2024-04-17T06:13:14.203 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executing_ops_high_water": 0, 2024-04-17T06:13:14.203 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executing": 0, 2024-04-17T06:13:14.204 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executing_high_water": 0, 2024-04-17T06:13:14.204 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executed_ops": 0, 2024-04-17T06:13:14.204 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executed": 0, 2024-04-17T06:13:14.204 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_item_in_journal": 0 2024-04-17T06:13:14.204 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:14.204 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-17T06:13:14.204 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-04-17T06:13:14.204 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 104857600, 2024-04-17T06:13:14.204 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-04-17T06:13:14.204 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 4774, 2024-04-17T06:13:14.204 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 3083925, 2024-04-17T06:13:14.204 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-04-17T06:13:14.204 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 4774, 2024-04-17T06:13:14.204 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 0, 2024-04-17T06:13:14.204 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 0, 2024-04-17T06:13:14.205 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 4774, 2024-04-17T06:13:14.205 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 3083925, 2024-04-17T06:13:14.205 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-04-17T06:13:14.205 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T06:13:14.205 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T06:13:14.205 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T06:13:14.205 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-04-17T06:13:14.205 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:14.205 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-objecter_bytes": { 2024-04-17T06:13:14.205 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-04-17T06:13:14.205 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 104857600, 2024-04-17T06:13:14.205 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-04-17T06:13:14.205 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 0, 2024-04-17T06:13:14.205 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 0, 2024-04-17T06:13:14.206 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-04-17T06:13:14.206 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 0, 2024-04-17T06:13:14.206 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 1312, 2024-04-17T06:13:14.206 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 3684276, 2024-04-17T06:13:14.206 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 1312, 2024-04-17T06:13:14.206 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 3684276, 2024-04-17T06:13:14.206 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-04-17T06:13:14.206 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T06:13:14.206 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T06:13:14.206 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T06:13:14.206 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-04-17T06:13:14.206 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:14.206 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-objecter_ops": { 2024-04-17T06:13:14.206 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-04-17T06:13:14.207 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 1024, 2024-04-17T06:13:14.207 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-04-17T06:13:14.207 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 0, 2024-04-17T06:13:14.207 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 0, 2024-04-17T06:13:14.207 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-04-17T06:13:14.207 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 0, 2024-04-17T06:13:14.207 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 1312, 2024-04-17T06:13:14.207 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 1312, 2024-04-17T06:13:14.207 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 1312, 2024-04-17T06:13:14.207 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 1312, 2024-04-17T06:13:14.207 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-04-17T06:13:14.207 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T06:13:14.207 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T06:13:14.207 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T06:13:14.207 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-04-17T06:13:14.208 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:14.208 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-write_buf_throttle": { 2024-04-17T06:13:14.208 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-04-17T06:13:14.208 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 3758096384, 2024-04-17T06:13:14.208 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-04-17T06:13:14.208 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 0, 2024-04-17T06:13:14.208 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 0, 2024-04-17T06:13:14.208 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-04-17T06:13:14.208 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 0, 2024-04-17T06:13:14.208 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 0, 2024-04-17T06:13:14.208 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 0, 2024-04-17T06:13:14.208 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 0, 2024-04-17T06:13:14.208 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 0, 2024-04-17T06:13:14.208 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-04-17T06:13:14.208 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T06:13:14.209 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T06:13:14.209 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T06:13:14.209 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-04-17T06:13:14.209 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:14.209 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-write_buf_throttle-0x55d2ac1b8500": { 2024-04-17T06:13:14.209 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-04-17T06:13:14.209 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 3758096384, 2024-04-17T06:13:14.209 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-04-17T06:13:14.209 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 249, 2024-04-17T06:13:14.209 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 2620281, 2024-04-17T06:13:14.209 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-04-17T06:13:14.209 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 249, 2024-04-17T06:13:14.210 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 0, 2024-04-17T06:13:14.210 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 0, 2024-04-17T06:13:14.210 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 213, 2024-04-17T06:13:14.210 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 2620281, 2024-04-17T06:13:14.210 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-04-17T06:13:14.210 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T06:13:14.210 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T06:13:14.210 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T06:13:14.210 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-04-17T06:13:14.210 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-04-17T06:13:14.210 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-04-17T06:13:14.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:13:13 smithi103 ceph-mon[36243]: pgmap v376: 97 pgs: 97 active+clean; 5.9 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 59 KiB/s wr, 28 op/s 2024-04-17T06:13:14.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:13:13 smithi047 ceph-mon[32223]: pgmap v376: 97 pgs: 97 active+clean; 5.9 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 59 KiB/s wr, 28 op/s 2024-04-17T06:13:14.919 INFO:tasks.check_counter:Daemon mds.b mds.exported=1 2024-04-17T06:13:14.919 INFO:tasks.check_counter:Daemon mds.b mds.imported=1 2024-04-17T06:13:14.920 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-04-17T06:13:14.920 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=12 2024-04-17T06:13:14.920 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=6 2024-04-17T06:13:14.920 DEBUG:tasks.check_counter:Getting stats from e 2024-04-17T06:13:14.920 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 565db2d8-fc7e-11ee-bc91-c7b262605968 -- ceph daemon mds.e perf dump 2024-04-17T06:13:15.181 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/mon.b/config 2024-04-17T06:13:15.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:13:15 smithi098 ceph-mon[36516]: pgmap v377: 97 pgs: 97 active+clean; 5.9 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 2.7 KiB/s wr, 7 op/s 2024-04-17T06:13:15.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:13:15 smithi098 ceph-mon[36516]: osdmap e106: 12 total, 12 up, 12 in 2024-04-17T06:13:16.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:13:15 smithi047 ceph-mon[32223]: pgmap v377: 97 pgs: 97 active+clean; 5.9 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 2.7 KiB/s wr, 7 op/s 2024-04-17T06:13:16.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:13:15 smithi047 ceph-mon[32223]: osdmap e106: 12 total, 12 up, 12 in 2024-04-17T06:13:16.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:13:15 smithi103 ceph-mon[36243]: pgmap v377: 97 pgs: 97 active+clean; 5.9 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 2.7 KiB/s wr, 7 op/s 2024-04-17T06:13:16.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:13:15 smithi103 ceph-mon[36243]: osdmap e106: 12 total, 12 up, 12 in 2024-04-17T06:13:16.875 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-04-17T06:13:16.875 INFO:teuthology.orchestra.run.smithi098.stdout: "AsyncMessenger::Worker-0": { 2024-04-17T06:13:16.876 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_messages": 0, 2024-04-17T06:13:16.876 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages": 2, 2024-04-17T06:13:16.876 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_bytes": 0, 2024-04-17T06:13:16.876 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_bytes": 0, 2024-04-17T06:13:16.876 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_created_connections": 4, 2024-04-17T06:13:16.876 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_active_connections": 0, 2024-04-17T06:13:16.876 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_total_time": 0.001001695, 2024-04-17T06:13:16.876 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_send_time": 0.000000000, 2024-04-17T06:13:16.876 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_recv_time": 0.000233417, 2024-04-17T06:13:16.876 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-17T06:13:16.876 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T06:13:16.876 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T06:13:16.876 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T06:13:16.876 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T06:13:16.877 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:16.877 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_handle_ack_lat": { 2024-04-17T06:13:16.877 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T06:13:16.877 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T06:13:16.877 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T06:13:16.877 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:16.877 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-17T06:13:16.877 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_encrypted_bytes": 0 2024-04-17T06:13:16.877 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:16.877 INFO:teuthology.orchestra.run.smithi098.stdout: "AsyncMessenger::Worker-1": { 2024-04-17T06:13:16.877 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_messages": 1, 2024-04-17T06:13:16.877 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages": 88, 2024-04-17T06:13:16.877 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_bytes": 160, 2024-04-17T06:13:16.878 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_bytes": 93520, 2024-04-17T06:13:16.878 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_created_connections": 3, 2024-04-17T06:13:16.878 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_active_connections": 2, 2024-04-17T06:13:16.878 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_total_time": 0.006586864, 2024-04-17T06:13:16.878 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_send_time": 0.004454925, 2024-04-17T06:13:16.878 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_recv_time": 0.000927256, 2024-04-17T06:13:16.878 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-17T06:13:16.878 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T06:13:16.878 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 86, 2024-04-17T06:13:16.878 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.005202704, 2024-04-17T06:13:16.878 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000060496 2024-04-17T06:13:16.878 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:16.878 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_handle_ack_lat": { 2024-04-17T06:13:16.878 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T06:13:16.879 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T06:13:16.879 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T06:13:16.879 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:16.879 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-17T06:13:16.879 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_encrypted_bytes": 93520 2024-04-17T06:13:16.879 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:16.879 INFO:teuthology.orchestra.run.smithi098.stdout: "AsyncMessenger::Worker-2": { 2024-04-17T06:13:16.879 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_messages": 121, 2024-04-17T06:13:16.879 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages": 118, 2024-04-17T06:13:16.879 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_bytes": 177584, 2024-04-17T06:13:16.879 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_bytes": 70896, 2024-04-17T06:13:16.879 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_created_connections": 2, 2024-04-17T06:13:16.879 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_active_connections": 1, 2024-04-17T06:13:16.879 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_total_time": 0.019210204, 2024-04-17T06:13:16.879 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_send_time": 0.008053108, 2024-04-17T06:13:16.880 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_recv_time": 0.006686754, 2024-04-17T06:13:16.880 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_fast_dispatch_time": 0.002912160, 2024-04-17T06:13:16.880 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T06:13:16.880 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 118, 2024-04-17T06:13:16.880 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.007113576, 2024-04-17T06:13:16.880 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000060284 2024-04-17T06:13:16.880 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:16.880 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_handle_ack_lat": { 2024-04-17T06:13:16.880 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T06:13:16.880 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T06:13:16.880 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T06:13:16.880 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:16.880 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_encrypted_bytes": 177584, 2024-04-17T06:13:16.880 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_encrypted_bytes": 70896 2024-04-17T06:13:16.880 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:16.881 INFO:teuthology.orchestra.run.smithi098.stdout: "cct": { 2024-04-17T06:13:16.881 INFO:teuthology.orchestra.run.smithi098.stdout: "total_workers": 0, 2024-04-17T06:13:16.881 INFO:teuthology.orchestra.run.smithi098.stdout: "unhealthy_workers": 0 2024-04-17T06:13:16.881 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:16.881 INFO:teuthology.orchestra.run.smithi098.stdout: "mempool": { 2024-04-17T06:13:16.881 INFO:teuthology.orchestra.run.smithi098.stdout: "bloom_filter_bytes": 0, 2024-04-17T06:13:16.881 INFO:teuthology.orchestra.run.smithi098.stdout: "bloom_filter_items": 0, 2024-04-17T06:13:16.881 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_alloc_bytes": 0, 2024-04-17T06:13:16.881 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_alloc_items": 0, 2024-04-17T06:13:16.881 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_data_bytes": 0, 2024-04-17T06:13:16.881 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_data_items": 0, 2024-04-17T06:13:16.881 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-17T06:13:16.881 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_onode_items": 0, 2024-04-17T06:13:16.881 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-17T06:13:16.882 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_meta_items": 0, 2024-04-17T06:13:16.882 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_other_bytes": 0, 2024-04-17T06:13:16.882 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_other_items": 0, 2024-04-17T06:13:16.882 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-17T06:13:16.882 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_buffer_items": 0, 2024-04-17T06:13:16.882 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_extent_bytes": 0, 2024-04-17T06:13:16.882 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_extent_items": 0, 2024-04-17T06:13:16.882 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_blob_bytes": 0, 2024-04-17T06:13:16.882 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_blob_items": 0, 2024-04-17T06:13:16.882 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-17T06:13:16.882 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_shared_blob_items": 0, 2024-04-17T06:13:16.882 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-17T06:13:16.882 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_inline_bl_items": 0, 2024-04-17T06:13:16.882 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_fsck_bytes": 0, 2024-04-17T06:13:16.883 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_fsck_items": 0, 2024-04-17T06:13:16.883 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_txc_bytes": 0, 2024-04-17T06:13:16.883 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_txc_items": 0, 2024-04-17T06:13:16.883 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-17T06:13:16.883 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_deferred_items": 0, 2024-04-17T06:13:16.883 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_bytes": 0, 2024-04-17T06:13:16.883 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_items": 0, 2024-04-17T06:13:16.883 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_bytes": 0, 2024-04-17T06:13:16.883 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_items": 0, 2024-04-17T06:13:16.883 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_reader_bytes": 0, 2024-04-17T06:13:16.883 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_reader_items": 0, 2024-04-17T06:13:16.883 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_writer_bytes": 0, 2024-04-17T06:13:16.883 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_writer_items": 0, 2024-04-17T06:13:16.883 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_anon_bytes": 148863, 2024-04-17T06:13:16.883 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_anon_items": 28, 2024-04-17T06:13:16.884 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_meta_bytes": 0, 2024-04-17T06:13:16.884 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_meta_items": 0, 2024-04-17T06:13:16.884 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_bytes": 0, 2024-04-17T06:13:16.884 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_items": 0, 2024-04-17T06:13:16.884 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_mapbl_bytes": 0, 2024-04-17T06:13:16.884 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_mapbl_items": 0, 2024-04-17T06:13:16.884 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_pglog_bytes": 0, 2024-04-17T06:13:16.884 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_pglog_items": 0, 2024-04-17T06:13:16.884 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_bytes": 0, 2024-04-17T06:13:16.884 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_items": 0, 2024-04-17T06:13:16.884 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_mapping_bytes": 0, 2024-04-17T06:13:16.884 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_mapping_items": 0, 2024-04-17T06:13:16.884 INFO:teuthology.orchestra.run.smithi098.stdout: "pgmap_bytes": 0, 2024-04-17T06:13:16.884 INFO:teuthology.orchestra.run.smithi098.stdout: "pgmap_items": 0, 2024-04-17T06:13:16.884 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_co_bytes": 928, 2024-04-17T06:13:16.885 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_co_items": 2, 2024-04-17T06:13:16.885 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_1_bytes": 0, 2024-04-17T06:13:16.885 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_1_items": 0, 2024-04-17T06:13:16.885 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_2_bytes": 0, 2024-04-17T06:13:16.885 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_2_items": 0 2024-04-17T06:13:16.885 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:16.885 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-17T06:13:16.885 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-04-17T06:13:16.885 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 104857600, 2024-04-17T06:13:16.885 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-04-17T06:13:16.885 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 118, 2024-04-17T06:13:16.885 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 158059, 2024-04-17T06:13:16.885 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-04-17T06:13:16.885 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 118, 2024-04-17T06:13:16.886 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 0, 2024-04-17T06:13:16.886 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 0, 2024-04-17T06:13:16.886 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 118, 2024-04-17T06:13:16.886 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 158059, 2024-04-17T06:13:16.886 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-04-17T06:13:16.886 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T06:13:16.886 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T06:13:16.886 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T06:13:16.886 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-04-17T06:13:16.886 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-04-17T06:13:16.886 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-04-17T06:13:17.547 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.e 2024-04-17T06:13:17.547 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.e 2024-04-17T06:13:17.547 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.e 2024-04-17T06:13:17.547 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.e 2024-04-17T06:13:17.547 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.e 2024-04-17T06:13:17.547 DEBUG:tasks.check_counter:Getting stats from h 2024-04-17T06:13:17.548 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 565db2d8-fc7e-11ee-bc91-c7b262605968 -- ceph daemon mds.h perf dump 2024-04-17T06:13:17.788 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/mon.b/config 2024-04-17T06:13:18.062 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:13:17 smithi098 ceph-mon[36516]: pgmap v379: 97 pgs: 97 active+clean; 5.9 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.4 KiB/s wr, 8 op/s 2024-04-17T06:13:18.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:13:17 smithi047 ceph-mon[32223]: pgmap v379: 97 pgs: 97 active+clean; 5.9 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.4 KiB/s wr, 8 op/s 2024-04-17T06:13:18.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:13:17 smithi103 ceph-mon[36243]: pgmap v379: 97 pgs: 97 active+clean; 5.9 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.4 KiB/s wr, 8 op/s 2024-04-17T06:13:19.537 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-04-17T06:13:19.537 INFO:teuthology.orchestra.run.smithi098.stdout: "AsyncMessenger::Worker-0": { 2024-04-17T06:13:19.537 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_messages": 629, 2024-04-17T06:13:19.537 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages": 884, 2024-04-17T06:13:19.537 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_bytes": 1801335, 2024-04-17T06:13:19.538 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_bytes": 431888, 2024-04-17T06:13:19.538 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_created_connections": 8, 2024-04-17T06:13:19.538 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_active_connections": 4, 2024-04-17T06:13:19.538 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_total_time": 0.118429478, 2024-04-17T06:13:19.538 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_send_time": 0.048821823, 2024-04-17T06:13:19.538 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_recv_time": 0.062428852, 2024-04-17T06:13:19.538 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_fast_dispatch_time": 0.010580044, 2024-04-17T06:13:19.538 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T06:13:19.538 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 883, 2024-04-17T06:13:19.538 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.057429220, 2024-04-17T06:13:19.538 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000065038 2024-04-17T06:13:19.538 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:19.538 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_handle_ack_lat": { 2024-04-17T06:13:19.538 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 847, 2024-04-17T06:13:19.539 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000231750, 2024-04-17T06:13:19.539 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000273 2024-04-17T06:13:19.539 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:19.539 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_encrypted_bytes": 1693392, 2024-04-17T06:13:19.539 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_encrypted_bytes": 151136 2024-04-17T06:13:19.539 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:19.539 INFO:teuthology.orchestra.run.smithi098.stdout: "AsyncMessenger::Worker-1": { 2024-04-17T06:13:19.539 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_messages": 31, 2024-04-17T06:13:19.539 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages": 206, 2024-04-17T06:13:19.539 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_bytes": 9480, 2024-04-17T06:13:19.539 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_bytes": 532492, 2024-04-17T06:13:19.539 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_created_connections": 15, 2024-04-17T06:13:19.539 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_active_connections": 4, 2024-04-17T06:13:19.539 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_total_time": 0.020108845, 2024-04-17T06:13:19.540 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_send_time": 0.008610263, 2024-04-17T06:13:19.540 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_recv_time": 0.005952029, 2024-04-17T06:13:19.540 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_fast_dispatch_time": 0.000697194, 2024-04-17T06:13:19.540 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T06:13:19.540 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 204, 2024-04-17T06:13:19.540 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.035332550, 2024-04-17T06:13:19.540 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000173198 2024-04-17T06:13:19.540 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:19.540 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_handle_ack_lat": { 2024-04-17T06:13:19.540 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 25, 2024-04-17T06:13:19.540 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000003586, 2024-04-17T06:13:19.540 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000143 2024-04-17T06:13:19.540 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:19.540 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_encrypted_bytes": 6112, 2024-04-17T06:13:19.540 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_encrypted_bytes": 479936 2024-04-17T06:13:19.540 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:19.541 INFO:teuthology.orchestra.run.smithi098.stdout: "AsyncMessenger::Worker-2": { 2024-04-17T06:13:19.541 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_messages": 25, 2024-04-17T06:13:19.541 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages": 29, 2024-04-17T06:13:19.541 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_bytes": 6787, 2024-04-17T06:13:19.541 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_bytes": 16648, 2024-04-17T06:13:19.541 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_created_connections": 7, 2024-04-17T06:13:19.541 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_active_connections": 4, 2024-04-17T06:13:19.541 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_total_time": 0.004039010, 2024-04-17T06:13:19.541 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_send_time": 0.000568171, 2024-04-17T06:13:19.541 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_recv_time": 0.002733909, 2024-04-17T06:13:19.541 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_fast_dispatch_time": 0.000597492, 2024-04-17T06:13:19.541 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T06:13:19.541 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 27, 2024-04-17T06:13:19.541 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.011474999, 2024-04-17T06:13:19.541 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000424999 2024-04-17T06:13:19.542 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:19.542 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_handle_ack_lat": { 2024-04-17T06:13:19.542 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 15, 2024-04-17T06:13:19.542 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000001704, 2024-04-17T06:13:19.542 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000113 2024-04-17T06:13:19.542 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:19.542 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_encrypted_bytes": 1840, 2024-04-17T06:13:19.542 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_encrypted_bytes": 2912 2024-04-17T06:13:19.542 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:19.542 INFO:teuthology.orchestra.run.smithi098.stdout: "cct": { 2024-04-17T06:13:19.542 INFO:teuthology.orchestra.run.smithi098.stdout: "total_workers": 1, 2024-04-17T06:13:19.542 INFO:teuthology.orchestra.run.smithi098.stdout: "unhealthy_workers": 0 2024-04-17T06:13:19.542 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:19.543 INFO:teuthology.orchestra.run.smithi098.stdout: "finisher-MDSRank": { 2024-04-17T06:13:19.543 INFO:teuthology.orchestra.run.smithi098.stdout: "queue_len": 0, 2024-04-17T06:13:19.543 INFO:teuthology.orchestra.run.smithi098.stdout: "complete_latency": { 2024-04-17T06:13:19.543 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 60, 2024-04-17T06:13:19.543 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.002239447, 2024-04-17T06:13:19.543 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000037324 2024-04-17T06:13:19.543 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-04-17T06:13:19.543 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:19.543 INFO:teuthology.orchestra.run.smithi098.stdout: "finisher-PurgeQueue": { 2024-04-17T06:13:19.543 INFO:teuthology.orchestra.run.smithi098.stdout: "queue_len": 0, 2024-04-17T06:13:19.543 INFO:teuthology.orchestra.run.smithi098.stdout: "complete_latency": { 2024-04-17T06:13:19.543 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 2, 2024-04-17T06:13:19.543 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000008183, 2024-04-17T06:13:19.543 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000004091 2024-04-17T06:13:19.544 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-04-17T06:13:19.544 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:19.544 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": { 2024-04-17T06:13:19.544 INFO:teuthology.orchestra.run.smithi098.stdout: "request": 7, 2024-04-17T06:13:19.544 INFO:teuthology.orchestra.run.smithi098.stdout: "reply": 6, 2024-04-17T06:13:19.544 INFO:teuthology.orchestra.run.smithi098.stdout: "reply_latency": { 2024-04-17T06:13:19.544 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 6, 2024-04-17T06:13:19.544 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.002299811, 2024-04-17T06:13:19.544 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000383301 2024-04-17T06:13:19.544 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:19.544 INFO:teuthology.orchestra.run.smithi098.stdout: "slow_reply": 0, 2024-04-17T06:13:19.544 INFO:teuthology.orchestra.run.smithi098.stdout: "forward": 1, 2024-04-17T06:13:19.544 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_fetch_complete": 0, 2024-04-17T06:13:19.544 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_fetch_keys": 0, 2024-04-17T06:13:19.545 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_commit": 11, 2024-04-17T06:13:19.545 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_split": 0, 2024-04-17T06:13:19.545 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_merge": 0, 2024-04-17T06:13:19.545 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes": 11, 2024-04-17T06:13:19.545 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_top": 0, 2024-04-17T06:13:19.545 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_bottom": 11, 2024-04-17T06:13:19.545 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_pin_tail": 0, 2024-04-17T06:13:19.545 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_pinned": 10, 2024-04-17T06:13:19.545 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_expired": 1, 2024-04-17T06:13:19.545 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_with_caps": 1, 2024-04-17T06:13:19.545 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": 1, 2024-04-17T06:13:19.545 INFO:teuthology.orchestra.run.smithi098.stdout: "subtrees": 2, 2024-04-17T06:13:19.545 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse": 13, 2024-04-17T06:13:19.545 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_hit": 8, 2024-04-17T06:13:19.546 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_forward": 1, 2024-04-17T06:13:19.546 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_discover": 2, 2024-04-17T06:13:19.546 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_dir_fetch": 0, 2024-04-17T06:13:19.546 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_remote_ino": 0, 2024-04-17T06:13:19.546 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_lock": 0, 2024-04-17T06:13:19.546 INFO:teuthology.orchestra.run.smithi098.stdout: "load_cent": 0, 2024-04-17T06:13:19.546 INFO:teuthology.orchestra.run.smithi098.stdout: "q": 0, 2024-04-17T06:13:19.546 INFO:teuthology.orchestra.run.smithi098.stdout: "exported": 0, 2024-04-17T06:13:19.546 INFO:teuthology.orchestra.run.smithi098.stdout: "exported_inodes": 0, 2024-04-17T06:13:19.546 INFO:teuthology.orchestra.run.smithi098.stdout: "imported": 0, 2024-04-17T06:13:19.546 INFO:teuthology.orchestra.run.smithi098.stdout: "imported_inodes": 0, 2024-04-17T06:13:19.546 INFO:teuthology.orchestra.run.smithi098.stdout: "openino_dir_fetch": 0, 2024-04-17T06:13:19.546 INFO:teuthology.orchestra.run.smithi098.stdout: "openino_backtrace_fetch": 0, 2024-04-17T06:13:19.546 INFO:teuthology.orchestra.run.smithi098.stdout: "openino_peer_discover": 0, 2024-04-17T06:13:19.547 INFO:teuthology.orchestra.run.smithi098.stdout: "root_rfiles": 4, 2024-04-17T06:13:19.547 INFO:teuthology.orchestra.run.smithi098.stdout: "root_rbytes": 234, 2024-04-17T06:13:19.547 INFO:teuthology.orchestra.run.smithi098.stdout: "root_rsnaps": 0, 2024-04-17T06:13:19.547 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_backtrace_fetch": 0, 2024-04-17T06:13:19.547 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_set_tag": 0, 2024-04-17T06:13:19.547 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_backtrace_repaired": 0, 2024-04-17T06:13:19.547 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_inotable_repaired": 0, 2024-04-17T06:13:19.547 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_dir_inodes": 0, 2024-04-17T06:13:19.547 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_dir_base_inodes": 0, 2024-04-17T06:13:19.547 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_dirfrag_rstats": 0, 2024-04-17T06:13:19.547 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_file_inodes": 0, 2024-04-17T06:13:19.547 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_inode_file_caps": 0, 2024-04-17T06:13:19.547 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_revoke": 0, 2024-04-17T06:13:19.547 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_grant": 0, 2024-04-17T06:13:19.547 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_trunc": 0, 2024-04-17T06:13:19.548 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-17T06:13:19.548 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-17T06:13:19.548 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_caps": 0, 2024-04-17T06:13:19.548 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_caps_dirty": 0, 2024-04-17T06:13:19.548 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_cap_release": 0, 2024-04-17T06:13:19.548 INFO:teuthology.orchestra.run.smithi098.stdout: "process_request_cap_release": 0 2024-04-17T06:13:19.548 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:19.548 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_cache": { 2024-04-17T06:13:19.548 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_update": 0, 2024-04-17T06:13:19.548 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_update_receipt": 0, 2024-04-17T06:13:19.548 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_try_discover": 0, 2024-04-17T06:13:19.548 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_send_discover": 3, 2024-04-17T06:13:19.548 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_handle_discover": 0, 2024-04-17T06:13:19.548 INFO:teuthology.orchestra.run.smithi098.stdout: "num_strays": 0, 2024-04-17T06:13:19.548 INFO:teuthology.orchestra.run.smithi098.stdout: "num_strays_delayed": 0, 2024-04-17T06:13:19.549 INFO:teuthology.orchestra.run.smithi098.stdout: "num_strays_enqueuing": 0, 2024-04-17T06:13:19.549 INFO:teuthology.orchestra.run.smithi098.stdout: "strays_created": 0, 2024-04-17T06:13:19.549 INFO:teuthology.orchestra.run.smithi098.stdout: "strays_enqueued": 0, 2024-04-17T06:13:19.549 INFO:teuthology.orchestra.run.smithi098.stdout: "strays_reintegrated": 0, 2024-04-17T06:13:19.549 INFO:teuthology.orchestra.run.smithi098.stdout: "strays_migrated": 0, 2024-04-17T06:13:19.549 INFO:teuthology.orchestra.run.smithi098.stdout: "num_recovering_processing": 0, 2024-04-17T06:13:19.549 INFO:teuthology.orchestra.run.smithi098.stdout: "num_recovering_enqueued": 0, 2024-04-17T06:13:19.549 INFO:teuthology.orchestra.run.smithi098.stdout: "num_recovering_prioritized": 0, 2024-04-17T06:13:19.549 INFO:teuthology.orchestra.run.smithi098.stdout: "recovery_started": 0, 2024-04-17T06:13:19.549 INFO:teuthology.orchestra.run.smithi098.stdout: "recovery_completed": 0, 2024-04-17T06:13:19.549 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_quiesce_path": 0, 2024-04-17T06:13:19.549 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_quiesce_inode": 0, 2024-04-17T06:13:19.549 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_enqueue_scrub": 0, 2024-04-17T06:13:19.549 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_exportdir": 0, 2024-04-17T06:13:19.549 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_flush": 0, 2024-04-17T06:13:19.550 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_fragmentdir": 0, 2024-04-17T06:13:19.550 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_fragstats": 0, 2024-04-17T06:13:19.550 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_inodestats": 0 2024-04-17T06:13:19.550 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:19.550 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_log": { 2024-04-17T06:13:19.550 INFO:teuthology.orchestra.run.smithi098.stdout: "evlrg": 0, 2024-04-17T06:13:19.550 INFO:teuthology.orchestra.run.smithi098.stdout: "evadd": 9, 2024-04-17T06:13:19.550 INFO:teuthology.orchestra.run.smithi098.stdout: "evex": 0, 2024-04-17T06:13:19.550 INFO:teuthology.orchestra.run.smithi098.stdout: "evtrm": 0, 2024-04-17T06:13:19.550 INFO:teuthology.orchestra.run.smithi098.stdout: "ev": 9, 2024-04-17T06:13:19.550 INFO:teuthology.orchestra.run.smithi098.stdout: "evexg": 0, 2024-04-17T06:13:19.550 INFO:teuthology.orchestra.run.smithi098.stdout: "evexd": 0, 2024-04-17T06:13:19.550 INFO:teuthology.orchestra.run.smithi098.stdout: "segadd": 2, 2024-04-17T06:13:19.550 INFO:teuthology.orchestra.run.smithi098.stdout: "segex": 0, 2024-04-17T06:13:19.551 INFO:teuthology.orchestra.run.smithi098.stdout: "segtrm": 0, 2024-04-17T06:13:19.551 INFO:teuthology.orchestra.run.smithi098.stdout: "seg": 2, 2024-04-17T06:13:19.551 INFO:teuthology.orchestra.run.smithi098.stdout: "segmjr": 2, 2024-04-17T06:13:19.551 INFO:teuthology.orchestra.run.smithi098.stdout: "segexg": 0, 2024-04-17T06:13:19.551 INFO:teuthology.orchestra.run.smithi098.stdout: "segexd": 0, 2024-04-17T06:13:19.551 INFO:teuthology.orchestra.run.smithi098.stdout: "expos": 4194304, 2024-04-17T06:13:19.551 INFO:teuthology.orchestra.run.smithi098.stdout: "wrpos": 4196796, 2024-04-17T06:13:19.551 INFO:teuthology.orchestra.run.smithi098.stdout: "rdpos": 0, 2024-04-17T06:13:19.551 INFO:teuthology.orchestra.run.smithi098.stdout: "jlat": { 2024-04-17T06:13:19.551 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 8, 2024-04-17T06:13:19.551 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.014091884, 2024-04-17T06:13:19.551 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.001761485 2024-04-17T06:13:19.551 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:19.551 INFO:teuthology.orchestra.run.smithi098.stdout: "replayed": 0 2024-04-17T06:13:19.551 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:19.551 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_mem": { 2024-04-17T06:13:19.552 INFO:teuthology.orchestra.run.smithi098.stdout: "ino": 14, 2024-04-17T06:13:19.552 INFO:teuthology.orchestra.run.smithi098.stdout: "ino+": 14, 2024-04-17T06:13:19.552 INFO:teuthology.orchestra.run.smithi098.stdout: "ino-": 0, 2024-04-17T06:13:19.552 INFO:teuthology.orchestra.run.smithi098.stdout: "dir": 13, 2024-04-17T06:13:19.552 INFO:teuthology.orchestra.run.smithi098.stdout: "dir+": 14, 2024-04-17T06:13:19.552 INFO:teuthology.orchestra.run.smithi098.stdout: "dir-": 1, 2024-04-17T06:13:19.552 INFO:teuthology.orchestra.run.smithi098.stdout: "dn": 11, 2024-04-17T06:13:19.552 INFO:teuthology.orchestra.run.smithi098.stdout: "dn+": 12, 2024-04-17T06:13:19.552 INFO:teuthology.orchestra.run.smithi098.stdout: "dn-": 1, 2024-04-17T06:13:19.552 INFO:teuthology.orchestra.run.smithi098.stdout: "cap": 1, 2024-04-17T06:13:19.552 INFO:teuthology.orchestra.run.smithi098.stdout: "cap+": 5, 2024-04-17T06:13:19.552 INFO:teuthology.orchestra.run.smithi098.stdout: "cap-": 4, 2024-04-17T06:13:19.552 INFO:teuthology.orchestra.run.smithi098.stdout: "rss": 51920, 2024-04-17T06:13:19.552 INFO:teuthology.orchestra.run.smithi098.stdout: "heap": 223608 2024-04-17T06:13:19.552 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:19.553 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_server": { 2024-04-17T06:13:19.553 INFO:teuthology.orchestra.run.smithi098.stdout: "dispatch_client_request": 12, 2024-04-17T06:13:19.553 INFO:teuthology.orchestra.run.smithi098.stdout: "dispatch_server_request": 0, 2024-04-17T06:13:19.553 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_request": 7, 2024-04-17T06:13:19.553 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_session": 22, 2024-04-17T06:13:19.553 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_peer_request": 1, 2024-04-17T06:13:19.553 INFO:teuthology.orchestra.run.smithi098.stdout: "req_create_latency": { 2024-04-17T06:13:19.553 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T06:13:19.553 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T06:13:19.553 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T06:13:19.553 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:19.553 INFO:teuthology.orchestra.run.smithi098.stdout: "req_getattr_latency": { 2024-04-17T06:13:19.553 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 2, 2024-04-17T06:13:19.553 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.001314087, 2024-04-17T06:13:19.553 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000657043 2024-04-17T06:13:19.554 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:19.554 INFO:teuthology.orchestra.run.smithi098.stdout: "req_getfilelock_latency": { 2024-04-17T06:13:19.554 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T06:13:19.554 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T06:13:19.554 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T06:13:19.554 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:19.554 INFO:teuthology.orchestra.run.smithi098.stdout: "req_link_latency": { 2024-04-17T06:13:19.554 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T06:13:19.554 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T06:13:19.554 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T06:13:19.554 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:19.554 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookup_latency": { 2024-04-17T06:13:19.554 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 3, 2024-04-17T06:13:19.554 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000888510, 2024-04-17T06:13:19.554 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000296170 2024-04-17T06:13:19.555 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:19.555 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookuphash_latency": { 2024-04-17T06:13:19.555 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T06:13:19.555 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T06:13:19.555 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T06:13:19.555 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:19.555 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookupino_latency": { 2024-04-17T06:13:19.555 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 1, 2024-04-17T06:13:19.555 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000097214, 2024-04-17T06:13:19.555 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000097214 2024-04-17T06:13:19.555 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:19.555 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookupname_latency": { 2024-04-17T06:13:19.555 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T06:13:19.555 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T06:13:19.555 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T06:13:19.555 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:19.556 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookupparent_latency": { 2024-04-17T06:13:19.556 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T06:13:19.556 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T06:13:19.556 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T06:13:19.556 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:19.556 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookupsnap_latency": { 2024-04-17T06:13:19.556 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T06:13:19.556 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T06:13:19.556 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T06:13:19.556 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:19.556 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lssnap_latency": { 2024-04-17T06:13:19.556 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T06:13:19.556 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T06:13:19.556 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T06:13:19.556 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:19.557 INFO:teuthology.orchestra.run.smithi098.stdout: "req_mkdir_latency": { 2024-04-17T06:13:19.557 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T06:13:19.557 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T06:13:19.557 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T06:13:19.558 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:19.558 INFO:teuthology.orchestra.run.smithi098.stdout: "req_mknod_latency": { 2024-04-17T06:13:19.558 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T06:13:19.558 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T06:13:19.558 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T06:13:19.558 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:19.558 INFO:teuthology.orchestra.run.smithi098.stdout: "req_mksnap_latency": { 2024-04-17T06:13:19.558 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T06:13:19.558 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T06:13:19.558 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T06:13:19.558 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:19.558 INFO:teuthology.orchestra.run.smithi098.stdout: "req_open_latency": { 2024-04-17T06:13:19.558 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T06:13:19.559 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T06:13:19.559 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T06:13:19.559 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:19.559 INFO:teuthology.orchestra.run.smithi098.stdout: "req_readdir_latency": { 2024-04-17T06:13:19.559 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T06:13:19.559 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T06:13:19.559 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T06:13:19.559 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:19.559 INFO:teuthology.orchestra.run.smithi098.stdout: "req_rename_latency": { 2024-04-17T06:13:19.559 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T06:13:19.559 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T06:13:19.559 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T06:13:19.559 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:19.559 INFO:teuthology.orchestra.run.smithi098.stdout: "req_renamesnap_latency": { 2024-04-17T06:13:19.559 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T06:13:19.560 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T06:13:19.560 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T06:13:19.560 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:19.560 INFO:teuthology.orchestra.run.smithi098.stdout: "req_snapdiff_latency": { 2024-04-17T06:13:19.560 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T06:13:19.560 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T06:13:19.560 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T06:13:19.560 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:19.560 INFO:teuthology.orchestra.run.smithi098.stdout: "req_rmdir_latency": { 2024-04-17T06:13:19.560 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T06:13:19.560 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T06:13:19.560 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T06:13:19.560 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:19.560 INFO:teuthology.orchestra.run.smithi098.stdout: "req_rmsnap_latency": { 2024-04-17T06:13:19.560 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T06:13:19.561 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T06:13:19.561 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T06:13:19.561 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:19.561 INFO:teuthology.orchestra.run.smithi098.stdout: "req_rmxattr_latency": { 2024-04-17T06:13:19.561 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T06:13:19.561 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T06:13:19.561 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T06:13:19.561 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:19.561 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setattr_latency": { 2024-04-17T06:13:19.561 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T06:13:19.561 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T06:13:19.561 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T06:13:19.561 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:19.561 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setdirlayout_latency": { 2024-04-17T06:13:19.561 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T06:13:19.561 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T06:13:19.562 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T06:13:19.562 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:19.562 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setfilelock_latency": { 2024-04-17T06:13:19.562 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T06:13:19.562 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T06:13:19.562 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T06:13:19.562 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:19.562 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setlayout_latency": { 2024-04-17T06:13:19.562 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T06:13:19.562 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T06:13:19.562 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T06:13:19.562 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:19.562 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setxattr_latency": { 2024-04-17T06:13:19.562 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T06:13:19.562 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T06:13:19.563 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T06:13:19.563 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:19.563 INFO:teuthology.orchestra.run.smithi098.stdout: "req_symlink_latency": { 2024-04-17T06:13:19.563 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T06:13:19.563 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T06:13:19.563 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T06:13:19.563 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:19.563 INFO:teuthology.orchestra.run.smithi098.stdout: "req_unlink_latency": { 2024-04-17T06:13:19.563 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T06:13:19.563 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T06:13:19.563 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T06:13:19.563 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:19.563 INFO:teuthology.orchestra.run.smithi098.stdout: "cap_revoke_eviction": 0, 2024-04-17T06:13:19.563 INFO:teuthology.orchestra.run.smithi098.stdout: "cap_acquisition_throttle": 0, 2024-04-17T06:13:19.563 INFO:teuthology.orchestra.run.smithi098.stdout: "req_getvxattr_latency": { 2024-04-17T06:13:19.564 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T06:13:19.564 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T06:13:19.564 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T06:13:19.564 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-04-17T06:13:19.564 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:19.564 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_sessions": { 2024-04-17T06:13:19.564 INFO:teuthology.orchestra.run.smithi098.stdout: "session_count": 1, 2024-04-17T06:13:19.564 INFO:teuthology.orchestra.run.smithi098.stdout: "session_add": 4, 2024-04-17T06:13:19.564 INFO:teuthology.orchestra.run.smithi098.stdout: "session_remove": 3, 2024-04-17T06:13:19.564 INFO:teuthology.orchestra.run.smithi098.stdout: "sessions_open": 1, 2024-04-17T06:13:19.564 INFO:teuthology.orchestra.run.smithi098.stdout: "sessions_stale": 0, 2024-04-17T06:13:19.564 INFO:teuthology.orchestra.run.smithi098.stdout: "total_load": 5, 2024-04-17T06:13:19.564 INFO:teuthology.orchestra.run.smithi098.stdout: "average_load": 1, 2024-04-17T06:13:19.564 INFO:teuthology.orchestra.run.smithi098.stdout: "avg_session_uptime": 716, 2024-04-17T06:13:19.564 INFO:teuthology.orchestra.run.smithi098.stdout: "mdthresh_evicted": 0 2024-04-17T06:13:19.565 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:19.565 INFO:teuthology.orchestra.run.smithi098.stdout: "mempool": { 2024-04-17T06:13:19.565 INFO:teuthology.orchestra.run.smithi098.stdout: "bloom_filter_bytes": 0, 2024-04-17T06:13:19.565 INFO:teuthology.orchestra.run.smithi098.stdout: "bloom_filter_items": 0, 2024-04-17T06:13:19.565 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_alloc_bytes": 0, 2024-04-17T06:13:19.565 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_alloc_items": 0, 2024-04-17T06:13:19.565 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_data_bytes": 0, 2024-04-17T06:13:19.565 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_data_items": 0, 2024-04-17T06:13:19.565 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-17T06:13:19.565 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_onode_items": 0, 2024-04-17T06:13:19.565 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-17T06:13:19.565 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_meta_items": 0, 2024-04-17T06:13:19.565 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_other_bytes": 0, 2024-04-17T06:13:19.565 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_other_items": 0, 2024-04-17T06:13:19.565 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-17T06:13:19.566 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_buffer_items": 0, 2024-04-17T06:13:19.566 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_extent_bytes": 0, 2024-04-17T06:13:19.566 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_extent_items": 0, 2024-04-17T06:13:19.566 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_blob_bytes": 0, 2024-04-17T06:13:19.566 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_blob_items": 0, 2024-04-17T06:13:19.566 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-17T06:13:19.566 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_shared_blob_items": 0, 2024-04-17T06:13:19.566 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-17T06:13:19.566 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_inline_bl_items": 0, 2024-04-17T06:13:19.566 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_fsck_bytes": 0, 2024-04-17T06:13:19.566 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_fsck_items": 0, 2024-04-17T06:13:19.566 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_txc_bytes": 0, 2024-04-17T06:13:19.566 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_txc_items": 0, 2024-04-17T06:13:19.566 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-17T06:13:19.566 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_deferred_items": 0, 2024-04-17T06:13:19.567 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_bytes": 0, 2024-04-17T06:13:19.567 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_items": 0, 2024-04-17T06:13:19.567 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_bytes": 0, 2024-04-17T06:13:19.567 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_items": 0, 2024-04-17T06:13:19.567 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_reader_bytes": 0, 2024-04-17T06:13:19.567 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_reader_items": 0, 2024-04-17T06:13:19.567 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_writer_bytes": 0, 2024-04-17T06:13:19.567 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_writer_items": 0, 2024-04-17T06:13:19.567 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_anon_bytes": 255559, 2024-04-17T06:13:19.567 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_anon_items": 89, 2024-04-17T06:13:19.567 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_meta_bytes": 0, 2024-04-17T06:13:19.567 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_meta_items": 0, 2024-04-17T06:13:19.567 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_bytes": 0, 2024-04-17T06:13:19.567 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_items": 0, 2024-04-17T06:13:19.567 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_mapbl_bytes": 0, 2024-04-17T06:13:19.568 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_mapbl_items": 0, 2024-04-17T06:13:19.568 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_pglog_bytes": 0, 2024-04-17T06:13:19.568 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_pglog_items": 0, 2024-04-17T06:13:19.568 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_bytes": 8032, 2024-04-17T06:13:19.568 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_items": 167, 2024-04-17T06:13:19.568 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_mapping_bytes": 0, 2024-04-17T06:13:19.568 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_mapping_items": 0, 2024-04-17T06:13:19.568 INFO:teuthology.orchestra.run.smithi098.stdout: "pgmap_bytes": 0, 2024-04-17T06:13:19.568 INFO:teuthology.orchestra.run.smithi098.stdout: "pgmap_items": 0, 2024-04-17T06:13:19.568 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_co_bytes": 56928, 2024-04-17T06:13:19.568 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_co_items": 228, 2024-04-17T06:13:19.568 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_1_bytes": 0, 2024-04-17T06:13:19.568 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_1_items": 0, 2024-04-17T06:13:19.568 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_2_bytes": 0, 2024-04-17T06:13:19.568 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_2_items": 0 2024-04-17T06:13:19.569 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:19.569 INFO:teuthology.orchestra.run.smithi098.stdout: "objecter": { 2024-04-17T06:13:19.569 INFO:teuthology.orchestra.run.smithi098.stdout: "op_active": 0, 2024-04-17T06:13:19.569 INFO:teuthology.orchestra.run.smithi098.stdout: "op_laggy": 0, 2024-04-17T06:13:19.569 INFO:teuthology.orchestra.run.smithi098.stdout: "op_send": 47, 2024-04-17T06:13:19.569 INFO:teuthology.orchestra.run.smithi098.stdout: "op_send_bytes": 13029, 2024-04-17T06:13:19.569 INFO:teuthology.orchestra.run.smithi098.stdout: "op_resend": 28, 2024-04-17T06:13:19.569 INFO:teuthology.orchestra.run.smithi098.stdout: "op_reply": 47, 2024-04-17T06:13:19.569 INFO:teuthology.orchestra.run.smithi098.stdout: "op_latency": { 2024-04-17T06:13:19.569 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 47, 2024-04-17T06:13:19.569 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.246996627, 2024-04-17T06:13:19.569 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.005255247 2024-04-17T06:13:19.569 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:19.569 INFO:teuthology.orchestra.run.smithi098.stdout: "op_inflight": 0, 2024-04-17T06:13:19.569 INFO:teuthology.orchestra.run.smithi098.stdout: "oplen_avg": { 2024-04-17T06:13:19.570 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 47, 2024-04-17T06:13:19.570 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 71 2024-04-17T06:13:19.570 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:19.570 INFO:teuthology.orchestra.run.smithi098.stdout: "op": 47, 2024-04-17T06:13:19.570 INFO:teuthology.orchestra.run.smithi098.stdout: "op_r": 0, 2024-04-17T06:13:19.570 INFO:teuthology.orchestra.run.smithi098.stdout: "op_w": 47, 2024-04-17T06:13:19.570 INFO:teuthology.orchestra.run.smithi098.stdout: "op_rmw": 0, 2024-04-17T06:13:19.570 INFO:teuthology.orchestra.run.smithi098.stdout: "op_pg": 0, 2024-04-17T06:13:19.570 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_stat": 0, 2024-04-17T06:13:19.570 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_create": 10, 2024-04-17T06:13:19.570 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_read": 0, 2024-04-17T06:13:19.570 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_write": 8, 2024-04-17T06:13:19.570 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_writefull": 8, 2024-04-17T06:13:19.570 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_writesame": 0, 2024-04-17T06:13:19.570 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_append": 0, 2024-04-17T06:13:19.571 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_zero": 0, 2024-04-17T06:13:19.571 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_truncate": 0, 2024-04-17T06:13:19.571 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_delete": 6, 2024-04-17T06:13:19.571 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_mapext": 0, 2024-04-17T06:13:19.571 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_sparse_read": 0, 2024-04-17T06:13:19.571 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_clonerange": 0, 2024-04-17T06:13:19.571 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_getxattr": 0, 2024-04-17T06:13:19.571 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_setxattr": 20, 2024-04-17T06:13:19.571 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_cmpxattr": 0, 2024-04-17T06:13:19.571 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_rmxattr": 0, 2024-04-17T06:13:19.571 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_resetxattrs": 0, 2024-04-17T06:13:19.571 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_call": 0, 2024-04-17T06:13:19.571 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_watch": 0, 2024-04-17T06:13:19.571 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_notify": 0, 2024-04-17T06:13:19.572 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_src_cmpxattr": 0, 2024-04-17T06:13:19.572 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_pgls": 0, 2024-04-17T06:13:19.572 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_pgls_filter": 0, 2024-04-17T06:13:19.572 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_other": 0, 2024-04-17T06:13:19.572 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_active": 0, 2024-04-17T06:13:19.572 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_send": 0, 2024-04-17T06:13:19.572 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_resend": 0, 2024-04-17T06:13:19.572 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_ping": 0, 2024-04-17T06:13:19.572 INFO:teuthology.orchestra.run.smithi098.stdout: "poolop_active": 0, 2024-04-17T06:13:19.572 INFO:teuthology.orchestra.run.smithi098.stdout: "poolop_send": 0, 2024-04-17T06:13:19.572 INFO:teuthology.orchestra.run.smithi098.stdout: "poolop_resend": 0, 2024-04-17T06:13:19.572 INFO:teuthology.orchestra.run.smithi098.stdout: "poolstat_active": 0, 2024-04-17T06:13:19.572 INFO:teuthology.orchestra.run.smithi098.stdout: "poolstat_send": 0, 2024-04-17T06:13:19.572 INFO:teuthology.orchestra.run.smithi098.stdout: "poolstat_resend": 0, 2024-04-17T06:13:19.572 INFO:teuthology.orchestra.run.smithi098.stdout: "statfs_active": 0, 2024-04-17T06:13:19.573 INFO:teuthology.orchestra.run.smithi098.stdout: "statfs_send": 0, 2024-04-17T06:13:19.573 INFO:teuthology.orchestra.run.smithi098.stdout: "statfs_resend": 0, 2024-04-17T06:13:19.573 INFO:teuthology.orchestra.run.smithi098.stdout: "command_active": 0, 2024-04-17T06:13:19.573 INFO:teuthology.orchestra.run.smithi098.stdout: "command_send": 0, 2024-04-17T06:13:19.573 INFO:teuthology.orchestra.run.smithi098.stdout: "command_resend": 0, 2024-04-17T06:13:19.573 INFO:teuthology.orchestra.run.smithi098.stdout: "map_epoch": 106, 2024-04-17T06:13:19.573 INFO:teuthology.orchestra.run.smithi098.stdout: "map_full": 0, 2024-04-17T06:13:19.573 INFO:teuthology.orchestra.run.smithi098.stdout: "map_inc": 20, 2024-04-17T06:13:19.573 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_sessions": 9, 2024-04-17T06:13:19.573 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_session_open": 9, 2024-04-17T06:13:19.573 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_session_close": 0, 2024-04-17T06:13:19.573 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_laggy": 0, 2024-04-17T06:13:19.573 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_wr": 17, 2024-04-17T06:13:19.573 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_rd": 0, 2024-04-17T06:13:19.573 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_del": 2 2024-04-17T06:13:19.574 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:19.574 INFO:teuthology.orchestra.run.smithi098.stdout: "oft": { 2024-04-17T06:13:19.574 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_total_objs": 1, 2024-04-17T06:13:19.574 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_total_kv_pairs": 0, 2024-04-17T06:13:19.574 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_total_updates": 0, 2024-04-17T06:13:19.574 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_total_removes": 0 2024-04-17T06:13:19.574 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:19.574 INFO:teuthology.orchestra.run.smithi098.stdout: "osd-slow-ops": { 2024-04-17T06:13:19.574 INFO:teuthology.orchestra.run.smithi098.stdout: "slow_ops_count": 0 2024-04-17T06:13:19.574 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:19.574 INFO:teuthology.orchestra.run.smithi098.stdout: "purge_queue": { 2024-04-17T06:13:19.574 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executing_ops": 0, 2024-04-17T06:13:19.574 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executing_ops_high_water": 0, 2024-04-17T06:13:19.575 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executing": 0, 2024-04-17T06:13:19.575 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executing_high_water": 0, 2024-04-17T06:13:19.575 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executed_ops": 0, 2024-04-17T06:13:19.575 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executed": 0, 2024-04-17T06:13:19.575 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_item_in_journal": 0 2024-04-17T06:13:19.575 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:19.575 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-17T06:13:19.575 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-04-17T06:13:19.575 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 104857600, 2024-04-17T06:13:19.575 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-04-17T06:13:19.575 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 681, 2024-04-17T06:13:19.575 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 1736840, 2024-04-17T06:13:19.575 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-04-17T06:13:19.575 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 681, 2024-04-17T06:13:19.575 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 0, 2024-04-17T06:13:19.576 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 0, 2024-04-17T06:13:19.576 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 681, 2024-04-17T06:13:19.576 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 1736840, 2024-04-17T06:13:19.576 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-04-17T06:13:19.576 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T06:13:19.576 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T06:13:19.576 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T06:13:19.576 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-04-17T06:13:19.576 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:19.576 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-objecter_bytes": { 2024-04-17T06:13:19.576 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-04-17T06:13:19.576 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 104857600, 2024-04-17T06:13:19.576 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-04-17T06:13:19.576 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 0, 2024-04-17T06:13:19.576 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 0, 2024-04-17T06:13:19.576 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-04-17T06:13:19.577 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 0, 2024-04-17T06:13:19.577 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 47, 2024-04-17T06:13:19.577 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 13029, 2024-04-17T06:13:19.577 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 47, 2024-04-17T06:13:19.577 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 13029, 2024-04-17T06:13:19.577 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-04-17T06:13:19.577 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T06:13:19.577 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T06:13:19.577 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T06:13:19.577 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-04-17T06:13:19.577 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:19.577 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-objecter_ops": { 2024-04-17T06:13:19.577 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-04-17T06:13:19.577 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 1024, 2024-04-17T06:13:19.578 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-04-17T06:13:19.578 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 0, 2024-04-17T06:13:19.578 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 0, 2024-04-17T06:13:19.578 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-04-17T06:13:19.578 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 0, 2024-04-17T06:13:19.578 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 47, 2024-04-17T06:13:19.578 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 47, 2024-04-17T06:13:19.578 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 47, 2024-04-17T06:13:19.578 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 47, 2024-04-17T06:13:19.578 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-04-17T06:13:19.578 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T06:13:19.578 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T06:13:19.578 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T06:13:19.578 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-04-17T06:13:19.578 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:19.579 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-write_buf_throttle": { 2024-04-17T06:13:19.579 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-04-17T06:13:19.579 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 3758096384, 2024-04-17T06:13:19.579 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-04-17T06:13:19.579 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 0, 2024-04-17T06:13:19.579 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 0, 2024-04-17T06:13:19.579 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-04-17T06:13:19.579 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 0, 2024-04-17T06:13:19.579 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 0, 2024-04-17T06:13:19.579 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 0, 2024-04-17T06:13:19.579 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 0, 2024-04-17T06:13:19.579 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 0, 2024-04-17T06:13:19.579 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-04-17T06:13:19.579 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T06:13:19.579 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T06:13:19.580 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T06:13:19.580 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-04-17T06:13:19.580 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:19.580 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-write_buf_throttle-0x55b6ac5a0500": { 2024-04-17T06:13:19.580 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-04-17T06:13:19.580 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 3758096384, 2024-04-17T06:13:19.580 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-04-17T06:13:19.580 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 9, 2024-04-17T06:13:19.580 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 2492, 2024-04-17T06:13:19.580 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-04-17T06:13:19.580 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 9, 2024-04-17T06:13:19.580 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 0, 2024-04-17T06:13:19.580 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 0, 2024-04-17T06:13:19.580 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 8, 2024-04-17T06:13:19.580 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 2492, 2024-04-17T06:13:19.581 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-04-17T06:13:19.581 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T06:13:19.581 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T06:13:19.581 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T06:13:19.581 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-04-17T06:13:19.581 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-04-17T06:13:19.581 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-04-17T06:13:20.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:13:19 smithi098 ceph-mon[36516]: pgmap v380: 97 pgs: 97 active+clean; 5.9 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.8 KiB/s wr, 7 op/s 2024-04-17T06:13:20.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:13:19 smithi047 ceph-mon[32223]: pgmap v380: 97 pgs: 97 active+clean; 5.9 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.8 KiB/s wr, 7 op/s 2024-04-17T06:13:20.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:13:19 smithi103 ceph-mon[36243]: pgmap v380: 97 pgs: 97 active+clean; 5.9 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.8 KiB/s wr, 7 op/s 2024-04-17T06:13:20.338 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-04-17T06:13:20.338 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-04-17T06:13:20.338 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=0 2024-04-17T06:13:20.338 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-04-17T06:13:20.338 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-04-17T06:13:20.338 DEBUG:tasks.check_counter:Getting stats from k 2024-04-17T06:13:20.339 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 565db2d8-fc7e-11ee-bc91-c7b262605968 -- ceph daemon mds.k perf dump 2024-04-17T06:13:20.578 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/mon.b/config 2024-04-17T06:13:22.059 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:13:21 smithi098 ceph-mon[36516]: pgmap v381: 97 pgs: 97 active+clean; 5.9 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-04-17T06:13:22.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:13:21 smithi047 ceph-mon[32223]: pgmap v381: 97 pgs: 97 active+clean; 5.9 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-04-17T06:13:22.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:13:21 smithi103 ceph-mon[36243]: pgmap v381: 97 pgs: 97 active+clean; 5.9 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-04-17T06:13:22.421 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-04-17T06:13:22.421 INFO:teuthology.orchestra.run.smithi098.stdout: "AsyncMessenger::Worker-0": { 2024-04-17T06:13:22.421 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_messages": 153, 2024-04-17T06:13:22.421 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages": 155, 2024-04-17T06:13:22.421 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_bytes": 411409, 2024-04-17T06:13:22.421 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_bytes": 48664, 2024-04-17T06:13:22.421 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_created_connections": 6, 2024-04-17T06:13:22.421 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_active_connections": 2, 2024-04-17T06:13:22.421 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_total_time": 0.015099627, 2024-04-17T06:13:22.422 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_send_time": 0.005377832, 2024-04-17T06:13:22.422 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_recv_time": 0.004020469, 2024-04-17T06:13:22.422 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_fast_dispatch_time": 0.003735131, 2024-04-17T06:13:22.422 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T06:13:22.422 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 153, 2024-04-17T06:13:22.422 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.006232199, 2024-04-17T06:13:22.422 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000040733 2024-04-17T06:13:22.422 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:22.422 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_handle_ack_lat": { 2024-04-17T06:13:22.422 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T06:13:22.422 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T06:13:22.422 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T06:13:22.422 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:22.423 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-17T06:13:22.423 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_encrypted_bytes": 0 2024-04-17T06:13:22.423 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:22.423 INFO:teuthology.orchestra.run.smithi098.stdout: "AsyncMessenger::Worker-1": { 2024-04-17T06:13:22.423 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_messages": 9, 2024-04-17T06:13:22.423 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages": 23, 2024-04-17T06:13:22.423 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_bytes": 5551, 2024-04-17T06:13:22.423 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_bytes": 114714, 2024-04-17T06:13:22.423 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_created_connections": 5, 2024-04-17T06:13:22.423 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_active_connections": 4, 2024-04-17T06:13:22.423 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_total_time": 0.002435408, 2024-04-17T06:13:22.423 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_send_time": 0.000739388, 2024-04-17T06:13:22.423 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_recv_time": 0.001437656, 2024-04-17T06:13:22.423 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_fast_dispatch_time": 0.000080477, 2024-04-17T06:13:22.424 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T06:13:22.424 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 22, 2024-04-17T06:13:22.424 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.004358036, 2024-04-17T06:13:22.424 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000198092 2024-04-17T06:13:22.424 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:22.424 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_handle_ack_lat": { 2024-04-17T06:13:22.424 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T06:13:22.424 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T06:13:22.424 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T06:13:22.424 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:22.424 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_encrypted_bytes": 3184, 2024-04-17T06:13:22.424 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_encrypted_bytes": 113376 2024-04-17T06:13:22.424 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:22.425 INFO:teuthology.orchestra.run.smithi098.stdout: "AsyncMessenger::Worker-2": { 2024-04-17T06:13:22.425 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_messages": 101, 2024-04-17T06:13:22.425 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages": 99, 2024-04-17T06:13:22.425 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_bytes": 184489, 2024-04-17T06:13:22.425 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_bytes": 35465, 2024-04-17T06:13:22.425 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_created_connections": 3, 2024-04-17T06:13:22.425 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_active_connections": 2, 2024-04-17T06:13:22.425 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_total_time": 0.010591534, 2024-04-17T06:13:22.425 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_send_time": 0.002988110, 2024-04-17T06:13:22.425 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_recv_time": 0.006392778, 2024-04-17T06:13:22.425 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_fast_dispatch_time": 0.002689514, 2024-04-17T06:13:22.425 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T06:13:22.425 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 98, 2024-04-17T06:13:22.425 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.004144971, 2024-04-17T06:13:22.426 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000042295 2024-04-17T06:13:22.426 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:22.426 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_handle_ack_lat": { 2024-04-17T06:13:22.426 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T06:13:22.426 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T06:13:22.426 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T06:13:22.426 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:22.426 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_encrypted_bytes": 167168, 2024-04-17T06:13:22.426 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_encrypted_bytes": 13840 2024-04-17T06:13:22.426 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:22.426 INFO:teuthology.orchestra.run.smithi098.stdout: "cct": { 2024-04-17T06:13:22.426 INFO:teuthology.orchestra.run.smithi098.stdout: "total_workers": 1, 2024-04-17T06:13:22.426 INFO:teuthology.orchestra.run.smithi098.stdout: "unhealthy_workers": 0 2024-04-17T06:13:22.427 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:22.427 INFO:teuthology.orchestra.run.smithi098.stdout: "finisher-MDSRank": { 2024-04-17T06:13:22.427 INFO:teuthology.orchestra.run.smithi098.stdout: "queue_len": 0, 2024-04-17T06:13:22.427 INFO:teuthology.orchestra.run.smithi098.stdout: "complete_latency": { 2024-04-17T06:13:22.427 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 372, 2024-04-17T06:13:22.427 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.009501370, 2024-04-17T06:13:22.427 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000025541 2024-04-17T06:13:22.427 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-04-17T06:13:22.427 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:22.427 INFO:teuthology.orchestra.run.smithi098.stdout: "finisher-PurgeQueue": { 2024-04-17T06:13:22.427 INFO:teuthology.orchestra.run.smithi098.stdout: "queue_len": 0, 2024-04-17T06:13:22.427 INFO:teuthology.orchestra.run.smithi098.stdout: "complete_latency": { 2024-04-17T06:13:22.427 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T06:13:22.427 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T06:13:22.428 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T06:13:22.428 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-04-17T06:13:22.428 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:22.428 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": { 2024-04-17T06:13:22.428 INFO:teuthology.orchestra.run.smithi098.stdout: "request": 0, 2024-04-17T06:13:22.428 INFO:teuthology.orchestra.run.smithi098.stdout: "reply": 0, 2024-04-17T06:13:22.428 INFO:teuthology.orchestra.run.smithi098.stdout: "reply_latency": { 2024-04-17T06:13:22.428 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T06:13:22.428 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T06:13:22.428 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T06:13:22.428 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:22.428 INFO:teuthology.orchestra.run.smithi098.stdout: "slow_reply": 0, 2024-04-17T06:13:22.428 INFO:teuthology.orchestra.run.smithi098.stdout: "forward": 0, 2024-04-17T06:13:22.428 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_fetch_complete": 0, 2024-04-17T06:13:22.429 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_fetch_keys": 0, 2024-04-17T06:13:22.429 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_commit": 0, 2024-04-17T06:13:22.429 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_split": 0, 2024-04-17T06:13:22.429 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_merge": 0, 2024-04-17T06:13:22.429 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes": 22, 2024-04-17T06:13:22.429 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_top": 0, 2024-04-17T06:13:22.429 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_bottom": 22, 2024-04-17T06:13:22.429 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_pin_tail": 0, 2024-04-17T06:13:22.429 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_pinned": 22, 2024-04-17T06:13:22.429 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_expired": 0, 2024-04-17T06:13:22.429 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_with_caps": 0, 2024-04-17T06:13:22.429 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": 0, 2024-04-17T06:13:22.429 INFO:teuthology.orchestra.run.smithi098.stdout: "subtrees": 2, 2024-04-17T06:13:22.429 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse": 0, 2024-04-17T06:13:22.430 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_hit": 0, 2024-04-17T06:13:22.430 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_forward": 0, 2024-04-17T06:13:22.430 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_discover": 0, 2024-04-17T06:13:22.430 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_dir_fetch": 0, 2024-04-17T06:13:22.430 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_remote_ino": 0, 2024-04-17T06:13:22.430 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_lock": 0, 2024-04-17T06:13:22.430 INFO:teuthology.orchestra.run.smithi098.stdout: "load_cent": 0, 2024-04-17T06:13:22.430 INFO:teuthology.orchestra.run.smithi098.stdout: "q": 0, 2024-04-17T06:13:22.430 INFO:teuthology.orchestra.run.smithi098.stdout: "exported": 0, 2024-04-17T06:13:22.430 INFO:teuthology.orchestra.run.smithi098.stdout: "exported_inodes": 0, 2024-04-17T06:13:22.430 INFO:teuthology.orchestra.run.smithi098.stdout: "imported": 0, 2024-04-17T06:13:22.430 INFO:teuthology.orchestra.run.smithi098.stdout: "imported_inodes": 0, 2024-04-17T06:13:22.430 INFO:teuthology.orchestra.run.smithi098.stdout: "openino_dir_fetch": 0, 2024-04-17T06:13:22.430 INFO:teuthology.orchestra.run.smithi098.stdout: "openino_backtrace_fetch": 0, 2024-04-17T06:13:22.431 INFO:teuthology.orchestra.run.smithi098.stdout: "openino_peer_discover": 0, 2024-04-17T06:13:22.431 INFO:teuthology.orchestra.run.smithi098.stdout: "root_rfiles": 4, 2024-04-17T06:13:22.431 INFO:teuthology.orchestra.run.smithi098.stdout: "root_rbytes": 234, 2024-04-17T06:13:22.431 INFO:teuthology.orchestra.run.smithi098.stdout: "root_rsnaps": 6, 2024-04-17T06:13:22.431 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_backtrace_fetch": 0, 2024-04-17T06:13:22.431 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_set_tag": 0, 2024-04-17T06:13:22.431 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_backtrace_repaired": 0, 2024-04-17T06:13:22.431 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_inotable_repaired": 0, 2024-04-17T06:13:22.431 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_dir_inodes": 0, 2024-04-17T06:13:22.431 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_dir_base_inodes": 0, 2024-04-17T06:13:22.431 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_dirfrag_rstats": 0, 2024-04-17T06:13:22.431 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_file_inodes": 0, 2024-04-17T06:13:22.431 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_inode_file_caps": 0, 2024-04-17T06:13:22.431 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_revoke": 0, 2024-04-17T06:13:22.432 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_grant": 0, 2024-04-17T06:13:22.432 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_trunc": 0, 2024-04-17T06:13:22.432 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-17T06:13:22.432 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-17T06:13:22.432 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_caps": 0, 2024-04-17T06:13:22.432 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_caps_dirty": 0, 2024-04-17T06:13:22.432 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_cap_release": 0, 2024-04-17T06:13:22.432 INFO:teuthology.orchestra.run.smithi098.stdout: "process_request_cap_release": 0 2024-04-17T06:13:22.432 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:22.432 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_cache": { 2024-04-17T06:13:22.432 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_update": 0, 2024-04-17T06:13:22.432 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_update_receipt": 0, 2024-04-17T06:13:22.432 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_try_discover": 0, 2024-04-17T06:13:22.432 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_send_discover": 0, 2024-04-17T06:13:22.433 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_handle_discover": 0, 2024-04-17T06:13:22.433 INFO:teuthology.orchestra.run.smithi098.stdout: "num_strays": 0, 2024-04-17T06:13:22.433 INFO:teuthology.orchestra.run.smithi098.stdout: "num_strays_delayed": 0, 2024-04-17T06:13:22.433 INFO:teuthology.orchestra.run.smithi098.stdout: "num_strays_enqueuing": 0, 2024-04-17T06:13:22.433 INFO:teuthology.orchestra.run.smithi098.stdout: "strays_created": 0, 2024-04-17T06:13:22.433 INFO:teuthology.orchestra.run.smithi098.stdout: "strays_enqueued": 0, 2024-04-17T06:13:22.433 INFO:teuthology.orchestra.run.smithi098.stdout: "strays_reintegrated": 0, 2024-04-17T06:13:22.433 INFO:teuthology.orchestra.run.smithi098.stdout: "strays_migrated": 0, 2024-04-17T06:13:22.433 INFO:teuthology.orchestra.run.smithi098.stdout: "num_recovering_processing": 0, 2024-04-17T06:13:22.433 INFO:teuthology.orchestra.run.smithi098.stdout: "num_recovering_enqueued": 0, 2024-04-17T06:13:22.433 INFO:teuthology.orchestra.run.smithi098.stdout: "num_recovering_prioritized": 0, 2024-04-17T06:13:22.433 INFO:teuthology.orchestra.run.smithi098.stdout: "recovery_started": 0, 2024-04-17T06:13:22.433 INFO:teuthology.orchestra.run.smithi098.stdout: "recovery_completed": 0, 2024-04-17T06:13:22.433 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_quiesce_path": 0, 2024-04-17T06:13:22.434 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_quiesce_inode": 0, 2024-04-17T06:13:22.434 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_enqueue_scrub": 0, 2024-04-17T06:13:22.434 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_exportdir": 0, 2024-04-17T06:13:22.434 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_flush": 0, 2024-04-17T06:13:22.434 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_fragmentdir": 0, 2024-04-17T06:13:22.434 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_fragstats": 0, 2024-04-17T06:13:22.434 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_inodestats": 0 2024-04-17T06:13:22.434 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:22.434 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_log": { 2024-04-17T06:13:22.434 INFO:teuthology.orchestra.run.smithi098.stdout: "evlrg": 0, 2024-04-17T06:13:22.434 INFO:teuthology.orchestra.run.smithi098.stdout: "evadd": 0, 2024-04-17T06:13:22.434 INFO:teuthology.orchestra.run.smithi098.stdout: "evex": 0, 2024-04-17T06:13:22.434 INFO:teuthology.orchestra.run.smithi098.stdout: "evtrm": 0, 2024-04-17T06:13:22.434 INFO:teuthology.orchestra.run.smithi098.stdout: "ev": 32, 2024-04-17T06:13:22.435 INFO:teuthology.orchestra.run.smithi098.stdout: "evexg": 0, 2024-04-17T06:13:22.435 INFO:teuthology.orchestra.run.smithi098.stdout: "evexd": 0, 2024-04-17T06:13:22.435 INFO:teuthology.orchestra.run.smithi098.stdout: "segadd": 0, 2024-04-17T06:13:22.435 INFO:teuthology.orchestra.run.smithi098.stdout: "segex": 0, 2024-04-17T06:13:22.435 INFO:teuthology.orchestra.run.smithi098.stdout: "segtrm": 0, 2024-04-17T06:13:22.435 INFO:teuthology.orchestra.run.smithi098.stdout: "seg": 2, 2024-04-17T06:13:22.435 INFO:teuthology.orchestra.run.smithi098.stdout: "segmjr": 5, 2024-04-17T06:13:22.435 INFO:teuthology.orchestra.run.smithi098.stdout: "segexg": 0, 2024-04-17T06:13:22.435 INFO:teuthology.orchestra.run.smithi098.stdout: "segexd": 0, 2024-04-17T06:13:22.435 INFO:teuthology.orchestra.run.smithi098.stdout: "expos": 6683816, 2024-04-17T06:13:22.435 INFO:teuthology.orchestra.run.smithi098.stdout: "wrpos": 6814585, 2024-04-17T06:13:22.435 INFO:teuthology.orchestra.run.smithi098.stdout: "rdpos": 6805783, 2024-04-17T06:13:22.435 INFO:teuthology.orchestra.run.smithi098.stdout: "jlat": { 2024-04-17T06:13:22.435 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T06:13:22.436 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T06:13:22.436 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T06:13:22.436 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:22.436 INFO:teuthology.orchestra.run.smithi098.stdout: "replayed": 32 2024-04-17T06:13:22.436 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:22.436 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_mem": { 2024-04-17T06:13:22.436 INFO:teuthology.orchestra.run.smithi098.stdout: "ino": 20, 2024-04-17T06:13:22.436 INFO:teuthology.orchestra.run.smithi098.stdout: "ino+": 20, 2024-04-17T06:13:22.436 INFO:teuthology.orchestra.run.smithi098.stdout: "ino-": 0, 2024-04-17T06:13:22.436 INFO:teuthology.orchestra.run.smithi098.stdout: "dir": 9, 2024-04-17T06:13:22.436 INFO:teuthology.orchestra.run.smithi098.stdout: "dir+": 9, 2024-04-17T06:13:22.436 INFO:teuthology.orchestra.run.smithi098.stdout: "dir-": 0, 2024-04-17T06:13:22.436 INFO:teuthology.orchestra.run.smithi098.stdout: "dn": 22, 2024-04-17T06:13:22.436 INFO:teuthology.orchestra.run.smithi098.stdout: "dn+": 22, 2024-04-17T06:13:22.436 INFO:teuthology.orchestra.run.smithi098.stdout: "dn-": 0, 2024-04-17T06:13:22.437 INFO:teuthology.orchestra.run.smithi098.stdout: "cap": 0, 2024-04-17T06:13:22.437 INFO:teuthology.orchestra.run.smithi098.stdout: "cap+": 0, 2024-04-17T06:13:22.437 INFO:teuthology.orchestra.run.smithi098.stdout: "cap-": 0, 2024-04-17T06:13:22.437 INFO:teuthology.orchestra.run.smithi098.stdout: "rss": 40388, 2024-04-17T06:13:22.437 INFO:teuthology.orchestra.run.smithi098.stdout: "heap": 215416 2024-04-17T06:13:22.437 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:22.437 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_server": { 2024-04-17T06:13:22.437 INFO:teuthology.orchestra.run.smithi098.stdout: "dispatch_client_request": 0, 2024-04-17T06:13:22.437 INFO:teuthology.orchestra.run.smithi098.stdout: "dispatch_server_request": 0, 2024-04-17T06:13:22.437 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_request": 0, 2024-04-17T06:13:22.437 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_session": 0, 2024-04-17T06:13:22.437 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_peer_request": 0, 2024-04-17T06:13:22.437 INFO:teuthology.orchestra.run.smithi098.stdout: "req_create_latency": { 2024-04-17T06:13:22.437 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T06:13:22.438 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T06:13:22.438 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T06:13:22.438 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:22.438 INFO:teuthology.orchestra.run.smithi098.stdout: "req_getattr_latency": { 2024-04-17T06:13:22.438 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T06:13:22.438 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T06:13:22.438 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T06:13:22.438 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:22.438 INFO:teuthology.orchestra.run.smithi098.stdout: "req_getfilelock_latency": { 2024-04-17T06:13:22.438 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T06:13:22.438 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T06:13:22.438 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T06:13:22.438 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:22.438 INFO:teuthology.orchestra.run.smithi098.stdout: "req_link_latency": { 2024-04-17T06:13:22.439 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T06:13:22.439 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T06:13:22.439 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T06:13:22.439 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:22.439 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookup_latency": { 2024-04-17T06:13:22.439 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T06:13:22.439 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T06:13:22.439 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T06:13:22.439 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:22.439 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookuphash_latency": { 2024-04-17T06:13:22.439 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T06:13:22.439 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T06:13:22.439 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T06:13:22.439 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:22.440 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookupino_latency": { 2024-04-17T06:13:22.440 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T06:13:22.440 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T06:13:22.440 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T06:13:22.440 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:22.440 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookupname_latency": { 2024-04-17T06:13:22.440 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T06:13:22.440 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T06:13:22.440 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T06:13:22.440 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:22.440 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookupparent_latency": { 2024-04-17T06:13:22.440 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T06:13:22.440 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T06:13:22.440 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T06:13:22.441 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:22.441 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookupsnap_latency": { 2024-04-17T06:13:22.441 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T06:13:22.441 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T06:13:22.441 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T06:13:22.441 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:22.441 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lssnap_latency": { 2024-04-17T06:13:22.441 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T06:13:22.441 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T06:13:22.441 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T06:13:22.441 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:22.441 INFO:teuthology.orchestra.run.smithi098.stdout: "req_mkdir_latency": { 2024-04-17T06:13:22.441 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T06:13:22.442 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T06:13:22.442 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T06:13:22.442 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:22.442 INFO:teuthology.orchestra.run.smithi098.stdout: "req_mknod_latency": { 2024-04-17T06:13:22.443 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T06:13:22.443 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T06:13:22.443 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T06:13:22.443 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:22.443 INFO:teuthology.orchestra.run.smithi098.stdout: "req_mksnap_latency": { 2024-04-17T06:13:22.443 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T06:13:22.443 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T06:13:22.443 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T06:13:22.443 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:22.443 INFO:teuthology.orchestra.run.smithi098.stdout: "req_open_latency": { 2024-04-17T06:13:22.443 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T06:13:22.443 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T06:13:22.443 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T06:13:22.443 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:22.443 INFO:teuthology.orchestra.run.smithi098.stdout: "req_readdir_latency": { 2024-04-17T06:13:22.444 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T06:13:22.444 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T06:13:22.444 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T06:13:22.444 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:22.444 INFO:teuthology.orchestra.run.smithi098.stdout: "req_rename_latency": { 2024-04-17T06:13:22.444 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T06:13:22.444 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T06:13:22.444 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T06:13:22.444 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:22.444 INFO:teuthology.orchestra.run.smithi098.stdout: "req_renamesnap_latency": { 2024-04-17T06:13:22.444 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T06:13:22.444 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T06:13:22.444 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T06:13:22.444 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:22.444 INFO:teuthology.orchestra.run.smithi098.stdout: "req_snapdiff_latency": { 2024-04-17T06:13:22.445 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T06:13:22.445 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T06:13:22.445 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T06:13:22.445 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:22.445 INFO:teuthology.orchestra.run.smithi098.stdout: "req_rmdir_latency": { 2024-04-17T06:13:22.445 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T06:13:22.445 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T06:13:22.445 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T06:13:22.445 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:22.445 INFO:teuthology.orchestra.run.smithi098.stdout: "req_rmsnap_latency": { 2024-04-17T06:13:22.445 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T06:13:22.445 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T06:13:22.445 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T06:13:22.445 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:22.445 INFO:teuthology.orchestra.run.smithi098.stdout: "req_rmxattr_latency": { 2024-04-17T06:13:22.446 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T06:13:22.446 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T06:13:22.446 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T06:13:22.446 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:22.446 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setattr_latency": { 2024-04-17T06:13:22.446 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T06:13:22.446 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T06:13:22.446 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T06:13:22.446 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:22.446 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setdirlayout_latency": { 2024-04-17T06:13:22.446 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T06:13:22.446 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T06:13:22.446 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T06:13:22.446 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:22.446 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setfilelock_latency": { 2024-04-17T06:13:22.447 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T06:13:22.447 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T06:13:22.447 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T06:13:22.447 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:22.447 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setlayout_latency": { 2024-04-17T06:13:22.447 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T06:13:22.447 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T06:13:22.447 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T06:13:22.447 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:22.447 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setxattr_latency": { 2024-04-17T06:13:22.447 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T06:13:22.447 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T06:13:22.447 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T06:13:22.447 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:22.448 INFO:teuthology.orchestra.run.smithi098.stdout: "req_symlink_latency": { 2024-04-17T06:13:22.448 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T06:13:22.448 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T06:13:22.448 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T06:13:22.448 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:22.448 INFO:teuthology.orchestra.run.smithi098.stdout: "req_unlink_latency": { 2024-04-17T06:13:22.448 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T06:13:22.448 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T06:13:22.448 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T06:13:22.448 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:22.448 INFO:teuthology.orchestra.run.smithi098.stdout: "cap_revoke_eviction": 0, 2024-04-17T06:13:22.448 INFO:teuthology.orchestra.run.smithi098.stdout: "cap_acquisition_throttle": 0, 2024-04-17T06:13:22.448 INFO:teuthology.orchestra.run.smithi098.stdout: "req_getvxattr_latency": { 2024-04-17T06:13:22.448 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T06:13:22.448 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T06:13:22.449 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T06:13:22.449 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-04-17T06:13:22.449 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:22.449 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_sessions": { 2024-04-17T06:13:22.449 INFO:teuthology.orchestra.run.smithi098.stdout: "session_count": 2, 2024-04-17T06:13:22.449 INFO:teuthology.orchestra.run.smithi098.stdout: "session_add": 2, 2024-04-17T06:13:22.449 INFO:teuthology.orchestra.run.smithi098.stdout: "session_remove": 0, 2024-04-17T06:13:22.449 INFO:teuthology.orchestra.run.smithi098.stdout: "sessions_open": 0, 2024-04-17T06:13:22.449 INFO:teuthology.orchestra.run.smithi098.stdout: "sessions_stale": 0, 2024-04-17T06:13:22.449 INFO:teuthology.orchestra.run.smithi098.stdout: "total_load": 0, 2024-04-17T06:13:22.449 INFO:teuthology.orchestra.run.smithi098.stdout: "average_load": 0, 2024-04-17T06:13:22.449 INFO:teuthology.orchestra.run.smithi098.stdout: "avg_session_uptime": 2091, 2024-04-17T06:13:22.449 INFO:teuthology.orchestra.run.smithi098.stdout: "mdthresh_evicted": 0 2024-04-17T06:13:22.449 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:22.449 INFO:teuthology.orchestra.run.smithi098.stdout: "mempool": { 2024-04-17T06:13:22.450 INFO:teuthology.orchestra.run.smithi098.stdout: "bloom_filter_bytes": 0, 2024-04-17T06:13:22.450 INFO:teuthology.orchestra.run.smithi098.stdout: "bloom_filter_items": 0, 2024-04-17T06:13:22.450 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_alloc_bytes": 0, 2024-04-17T06:13:22.450 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_alloc_items": 0, 2024-04-17T06:13:22.450 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_data_bytes": 0, 2024-04-17T06:13:22.450 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_data_items": 0, 2024-04-17T06:13:22.450 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-17T06:13:22.450 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_onode_items": 0, 2024-04-17T06:13:22.450 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-17T06:13:22.450 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_meta_items": 0, 2024-04-17T06:13:22.450 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_other_bytes": 0, 2024-04-17T06:13:22.450 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_other_items": 0, 2024-04-17T06:13:22.450 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-17T06:13:22.451 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_buffer_items": 0, 2024-04-17T06:13:22.451 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_extent_bytes": 0, 2024-04-17T06:13:22.451 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_extent_items": 0, 2024-04-17T06:13:22.451 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_blob_bytes": 0, 2024-04-17T06:13:22.451 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_blob_items": 0, 2024-04-17T06:13:22.451 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-17T06:13:22.451 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_shared_blob_items": 0, 2024-04-17T06:13:22.451 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-17T06:13:22.451 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_inline_bl_items": 0, 2024-04-17T06:13:22.451 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_fsck_bytes": 0, 2024-04-17T06:13:22.451 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_fsck_items": 0, 2024-04-17T06:13:22.451 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_txc_bytes": 0, 2024-04-17T06:13:22.451 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_txc_items": 0, 2024-04-17T06:13:22.451 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-17T06:13:22.452 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_deferred_items": 0, 2024-04-17T06:13:22.452 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_bytes": 0, 2024-04-17T06:13:22.452 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_items": 0, 2024-04-17T06:13:22.452 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_bytes": 0, 2024-04-17T06:13:22.452 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_items": 0, 2024-04-17T06:13:22.452 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_reader_bytes": 0, 2024-04-17T06:13:22.452 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_reader_items": 0, 2024-04-17T06:13:22.452 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_writer_bytes": 0, 2024-04-17T06:13:22.452 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_writer_items": 0, 2024-04-17T06:13:22.452 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_anon_bytes": 189404, 2024-04-17T06:13:22.452 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_anon_items": 71, 2024-04-17T06:13:22.452 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_meta_bytes": 0, 2024-04-17T06:13:22.452 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_meta_items": 0, 2024-04-17T06:13:22.452 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_bytes": 0, 2024-04-17T06:13:22.452 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_items": 0, 2024-04-17T06:13:22.453 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_mapbl_bytes": 0, 2024-04-17T06:13:22.453 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_mapbl_items": 0, 2024-04-17T06:13:22.453 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_pglog_bytes": 0, 2024-04-17T06:13:22.453 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_pglog_items": 0, 2024-04-17T06:13:22.453 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_bytes": 8032, 2024-04-17T06:13:22.453 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_items": 167, 2024-04-17T06:13:22.453 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_mapping_bytes": 0, 2024-04-17T06:13:22.453 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_mapping_items": 0, 2024-04-17T06:13:22.453 INFO:teuthology.orchestra.run.smithi098.stdout: "pgmap_bytes": 0, 2024-04-17T06:13:22.453 INFO:teuthology.orchestra.run.smithi098.stdout: "pgmap_items": 0, 2024-04-17T06:13:22.453 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_co_bytes": 113851, 2024-04-17T06:13:22.453 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_co_items": 965, 2024-04-17T06:13:22.453 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_1_bytes": 0, 2024-04-17T06:13:22.453 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_1_items": 0, 2024-04-17T06:13:22.453 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_2_bytes": 0, 2024-04-17T06:13:22.454 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_2_items": 0 2024-04-17T06:13:22.454 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:22.454 INFO:teuthology.orchestra.run.smithi098.stdout: "objecter": { 2024-04-17T06:13:22.454 INFO:teuthology.orchestra.run.smithi098.stdout: "op_active": 0, 2024-04-17T06:13:22.454 INFO:teuthology.orchestra.run.smithi098.stdout: "op_laggy": 0, 2024-04-17T06:13:22.454 INFO:teuthology.orchestra.run.smithi098.stdout: "op_send": 225, 2024-04-17T06:13:22.454 INFO:teuthology.orchestra.run.smithi098.stdout: "op_send_bytes": 26, 2024-04-17T06:13:22.454 INFO:teuthology.orchestra.run.smithi098.stdout: "op_resend": 4, 2024-04-17T06:13:22.454 INFO:teuthology.orchestra.run.smithi098.stdout: "op_reply": 225, 2024-04-17T06:13:22.454 INFO:teuthology.orchestra.run.smithi098.stdout: "op_latency": { 2024-04-17T06:13:22.454 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 225, 2024-04-17T06:13:22.454 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.115998367, 2024-04-17T06:13:22.454 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000515548 2024-04-17T06:13:22.454 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:22.455 INFO:teuthology.orchestra.run.smithi098.stdout: "op_inflight": 0, 2024-04-17T06:13:22.455 INFO:teuthology.orchestra.run.smithi098.stdout: "oplen_avg": { 2024-04-17T06:13:22.455 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 225, 2024-04-17T06:13:22.455 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 226 2024-04-17T06:13:22.455 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:22.455 INFO:teuthology.orchestra.run.smithi098.stdout: "op": 225, 2024-04-17T06:13:22.455 INFO:teuthology.orchestra.run.smithi098.stdout: "op_r": 225, 2024-04-17T06:13:22.455 INFO:teuthology.orchestra.run.smithi098.stdout: "op_w": 0, 2024-04-17T06:13:22.455 INFO:teuthology.orchestra.run.smithi098.stdout: "op_rmw": 0, 2024-04-17T06:13:22.455 INFO:teuthology.orchestra.run.smithi098.stdout: "op_pg": 0, 2024-04-17T06:13:22.455 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_stat": 134, 2024-04-17T06:13:22.455 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_create": 0, 2024-04-17T06:13:22.455 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_read": 88, 2024-04-17T06:13:22.455 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_write": 0, 2024-04-17T06:13:22.455 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_writefull": 0, 2024-04-17T06:13:22.456 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_writesame": 0, 2024-04-17T06:13:22.456 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_append": 0, 2024-04-17T06:13:22.456 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_zero": 0, 2024-04-17T06:13:22.456 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_truncate": 0, 2024-04-17T06:13:22.456 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_delete": 0, 2024-04-17T06:13:22.456 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_mapext": 0, 2024-04-17T06:13:22.456 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_sparse_read": 0, 2024-04-17T06:13:22.456 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_clonerange": 0, 2024-04-17T06:13:22.456 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_getxattr": 2, 2024-04-17T06:13:22.456 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_setxattr": 0, 2024-04-17T06:13:22.456 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_cmpxattr": 0, 2024-04-17T06:13:22.456 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_rmxattr": 0, 2024-04-17T06:13:22.456 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_resetxattrs": 0, 2024-04-17T06:13:22.456 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_call": 0, 2024-04-17T06:13:22.456 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_watch": 0, 2024-04-17T06:13:22.457 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_notify": 0, 2024-04-17T06:13:22.457 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_src_cmpxattr": 0, 2024-04-17T06:13:22.457 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_pgls": 0, 2024-04-17T06:13:22.457 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_pgls_filter": 0, 2024-04-17T06:13:22.457 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_other": 0, 2024-04-17T06:13:22.457 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_active": 0, 2024-04-17T06:13:22.457 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_send": 0, 2024-04-17T06:13:22.457 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_resend": 0, 2024-04-17T06:13:22.457 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_ping": 0, 2024-04-17T06:13:22.457 INFO:teuthology.orchestra.run.smithi098.stdout: "poolop_active": 0, 2024-04-17T06:13:22.457 INFO:teuthology.orchestra.run.smithi098.stdout: "poolop_send": 0, 2024-04-17T06:13:22.457 INFO:teuthology.orchestra.run.smithi098.stdout: "poolop_resend": 0, 2024-04-17T06:13:22.457 INFO:teuthology.orchestra.run.smithi098.stdout: "poolstat_active": 0, 2024-04-17T06:13:22.457 INFO:teuthology.orchestra.run.smithi098.stdout: "poolstat_send": 0, 2024-04-17T06:13:22.458 INFO:teuthology.orchestra.run.smithi098.stdout: "poolstat_resend": 0, 2024-04-17T06:13:22.458 INFO:teuthology.orchestra.run.smithi098.stdout: "statfs_active": 0, 2024-04-17T06:13:22.458 INFO:teuthology.orchestra.run.smithi098.stdout: "statfs_send": 0, 2024-04-17T06:13:22.458 INFO:teuthology.orchestra.run.smithi098.stdout: "statfs_resend": 0, 2024-04-17T06:13:22.458 INFO:teuthology.orchestra.run.smithi098.stdout: "command_active": 0, 2024-04-17T06:13:22.458 INFO:teuthology.orchestra.run.smithi098.stdout: "command_send": 0, 2024-04-17T06:13:22.458 INFO:teuthology.orchestra.run.smithi098.stdout: "command_resend": 0, 2024-04-17T06:13:22.458 INFO:teuthology.orchestra.run.smithi098.stdout: "map_epoch": 106, 2024-04-17T06:13:22.458 INFO:teuthology.orchestra.run.smithi098.stdout: "map_full": 0, 2024-04-17T06:13:22.458 INFO:teuthology.orchestra.run.smithi098.stdout: "map_inc": 2, 2024-04-17T06:13:22.458 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_sessions": 5, 2024-04-17T06:13:22.458 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_session_open": 5, 2024-04-17T06:13:22.458 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_session_close": 0, 2024-04-17T06:13:22.458 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_laggy": 0, 2024-04-17T06:13:22.458 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_wr": 0, 2024-04-17T06:13:22.459 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_rd": 2, 2024-04-17T06:13:22.459 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_del": 0 2024-04-17T06:13:22.459 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:22.459 INFO:teuthology.orchestra.run.smithi098.stdout: "oft": { 2024-04-17T06:13:22.459 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_total_objs": 0, 2024-04-17T06:13:22.459 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_total_kv_pairs": 0, 2024-04-17T06:13:22.459 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_total_updates": 0, 2024-04-17T06:13:22.459 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_total_removes": 0 2024-04-17T06:13:22.459 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:22.459 INFO:teuthology.orchestra.run.smithi098.stdout: "osd-slow-ops": { 2024-04-17T06:13:22.459 INFO:teuthology.orchestra.run.smithi098.stdout: "slow_ops_count": 0 2024-04-17T06:13:22.459 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:22.459 INFO:teuthology.orchestra.run.smithi098.stdout: "purge_queue": { 2024-04-17T06:13:22.459 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executing_ops": 0, 2024-04-17T06:13:22.459 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executing_ops_high_water": 0, 2024-04-17T06:13:22.459 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executing": 0, 2024-04-17T06:13:22.460 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executing_high_water": 0, 2024-04-17T06:13:22.460 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executed_ops": 0, 2024-04-17T06:13:22.460 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executed": 0, 2024-04-17T06:13:22.460 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_item_in_journal": 0 2024-04-17T06:13:22.460 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:22.460 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-17T06:13:22.460 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-04-17T06:13:22.460 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 104857600, 2024-04-17T06:13:22.460 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-04-17T06:13:22.460 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 259, 2024-04-17T06:13:22.460 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 573458, 2024-04-17T06:13:22.460 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-04-17T06:13:22.460 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 259, 2024-04-17T06:13:22.460 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 0, 2024-04-17T06:13:22.460 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 0, 2024-04-17T06:13:22.461 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 259, 2024-04-17T06:13:22.461 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 573458, 2024-04-17T06:13:22.461 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-04-17T06:13:22.461 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T06:13:22.461 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T06:13:22.461 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T06:13:22.461 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-04-17T06:13:22.461 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:22.461 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-objecter_bytes": { 2024-04-17T06:13:22.461 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-04-17T06:13:22.461 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 104857600, 2024-04-17T06:13:22.461 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-04-17T06:13:22.461 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 0, 2024-04-17T06:13:22.461 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 0, 2024-04-17T06:13:22.461 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-04-17T06:13:22.461 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 0, 2024-04-17T06:13:22.462 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 225, 2024-04-17T06:13:22.462 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 363195, 2024-04-17T06:13:22.462 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 225, 2024-04-17T06:13:22.462 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 363195, 2024-04-17T06:13:22.462 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-04-17T06:13:22.462 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T06:13:22.462 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T06:13:22.462 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T06:13:22.462 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-04-17T06:13:22.462 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:22.462 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-objecter_ops": { 2024-04-17T06:13:22.462 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-04-17T06:13:22.462 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 1024, 2024-04-17T06:13:22.462 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-04-17T06:13:22.463 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 0, 2024-04-17T06:13:22.463 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 0, 2024-04-17T06:13:22.463 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-04-17T06:13:22.463 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 0, 2024-04-17T06:13:22.463 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 225, 2024-04-17T06:13:22.463 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 225, 2024-04-17T06:13:22.463 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 225, 2024-04-17T06:13:22.463 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 225, 2024-04-17T06:13:22.463 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-04-17T06:13:22.463 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T06:13:22.463 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T06:13:22.463 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T06:13:22.463 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-04-17T06:13:22.463 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:22.463 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-write_buf_throttle": { 2024-04-17T06:13:22.464 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-04-17T06:13:22.464 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 3758096384, 2024-04-17T06:13:22.464 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-04-17T06:13:22.464 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 0, 2024-04-17T06:13:22.464 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 0, 2024-04-17T06:13:22.464 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-04-17T06:13:22.464 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 0, 2024-04-17T06:13:22.464 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 0, 2024-04-17T06:13:22.464 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 0, 2024-04-17T06:13:22.464 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 0, 2024-04-17T06:13:22.464 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 0, 2024-04-17T06:13:22.464 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-04-17T06:13:22.464 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T06:13:22.464 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T06:13:22.464 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T06:13:22.465 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-04-17T06:13:22.465 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T06:13:22.465 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-write_buf_throttle-0x559e855b66e0": { 2024-04-17T06:13:22.465 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-04-17T06:13:22.465 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 3758096384, 2024-04-17T06:13:22.465 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-04-17T06:13:22.465 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 0, 2024-04-17T06:13:22.465 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 0, 2024-04-17T06:13:22.465 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-04-17T06:13:22.465 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 0, 2024-04-17T06:13:22.465 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 0, 2024-04-17T06:13:22.465 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 0, 2024-04-17T06:13:22.465 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 0, 2024-04-17T06:13:22.465 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 0, 2024-04-17T06:13:22.466 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-04-17T06:13:22.466 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T06:13:22.466 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T06:13:22.466 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T06:13:22.466 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-04-17T06:13:22.466 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-04-17T06:13:22.466 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-04-17T06:13:23.284 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-04-17T06:13:23.284 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-04-17T06:13:23.284 INFO:tasks.check_counter:Daemon mds.k mds.root_rsnaps=6 2024-04-17T06:13:23.284 INFO:tasks.check_counter:Daemon mds.k mds_server.req_mksnap_latency.avgcount=0 2024-04-17T06:13:23.284 INFO:tasks.check_counter:Daemon mds.k mds_server.req_rmsnap_latency.avgcount=0 2024-04-17T06:13:23.284 DEBUG:tasks.check_counter:Getting stats from c 2024-04-17T06:13:23.284 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 565db2d8-fc7e-11ee-bc91-c7b262605968 -- ceph daemon mds.c perf dump 2024-04-17T06:13:23.534 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/mon.c/config 2024-04-17T06:13:24.005 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:13:23 smithi103 ceph-mon[36243]: pgmap v382: 97 pgs: 97 active+clean; 5.9 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-04-17T06:13:24.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:13:23 smithi098 ceph-mon[36516]: pgmap v382: 97 pgs: 97 active+clean; 5.9 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-04-17T06:13:24.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:13:23 smithi047 ceph-mon[32223]: pgmap v382: 97 pgs: 97 active+clean; 5.9 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-04-17T06:13:25.358 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-04-17T06:13:25.359 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-0": { 2024-04-17T06:13:25.359 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 2460, 2024-04-17T06:13:25.359 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 2462, 2024-04-17T06:13:25.359 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 697845, 2024-04-17T06:13:25.359 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 782287, 2024-04-17T06:13:25.359 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 6, 2024-04-17T06:13:25.359 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 2, 2024-04-17T06:13:25.359 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.226506012, 2024-04-17T06:13:25.359 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.076934531, 2024-04-17T06:13:25.359 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.054659891, 2024-04-17T06:13:25.359 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.076600845, 2024-04-17T06:13:25.359 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T06:13:25.359 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 2460, 2024-04-17T06:13:25.360 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.099993297, 2024-04-17T06:13:25.360 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000040647 2024-04-17T06:13:25.360 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:25.360 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-04-17T06:13:25.360 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:25.360 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:25.360 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:25.360 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:25.360 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-17T06:13:25.360 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 0 2024-04-17T06:13:25.360 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:25.360 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-1": { 2024-04-17T06:13:25.360 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 290, 2024-04-17T06:13:25.360 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 262, 2024-04-17T06:13:25.360 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 1686684, 2024-04-17T06:13:25.361 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 148922, 2024-04-17T06:13:25.361 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 3, 2024-04-17T06:13:25.361 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 2, 2024-04-17T06:13:25.361 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.052710474, 2024-04-17T06:13:25.361 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.016766046, 2024-04-17T06:13:25.361 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.038048684, 2024-04-17T06:13:25.361 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.006273752, 2024-04-17T06:13:25.361 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T06:13:25.361 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 262, 2024-04-17T06:13:25.361 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.016043684, 2024-04-17T06:13:25.361 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000061235 2024-04-17T06:13:25.361 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:25.361 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-04-17T06:13:25.361 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:25.362 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:25.362 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:25.362 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:25.362 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 1686128, 2024-04-17T06:13:25.362 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 148224 2024-04-17T06:13:25.362 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:25.362 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-2": { 2024-04-17T06:13:25.362 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 3, 2024-04-17T06:13:25.362 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 180, 2024-04-17T06:13:25.362 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 588, 2024-04-17T06:13:25.362 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 478110, 2024-04-17T06:13:25.362 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 6, 2024-04-17T06:13:25.362 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 4, 2024-04-17T06:13:25.363 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.010810425, 2024-04-17T06:13:25.363 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.007021798, 2024-04-17T06:13:25.363 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.001794134, 2024-04-17T06:13:25.363 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.000026414, 2024-04-17T06:13:25.363 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T06:13:25.363 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 177, 2024-04-17T06:13:25.363 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.014334762, 2024-04-17T06:13:25.363 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000080987 2024-04-17T06:13:25.363 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:25.363 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-04-17T06:13:25.363 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:25.363 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:25.363 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:25.363 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:25.363 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-17T06:13:25.363 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 477792 2024-04-17T06:13:25.364 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:25.364 INFO:teuthology.orchestra.run.smithi103.stdout: "cct": { 2024-04-17T06:13:25.364 INFO:teuthology.orchestra.run.smithi103.stdout: "total_workers": 1, 2024-04-17T06:13:25.364 INFO:teuthology.orchestra.run.smithi103.stdout: "unhealthy_workers": 0 2024-04-17T06:13:25.364 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:25.364 INFO:teuthology.orchestra.run.smithi103.stdout: "finisher-MDSRank": { 2024-04-17T06:13:25.364 INFO:teuthology.orchestra.run.smithi103.stdout: "queue_len": 0, 2024-04-17T06:13:25.364 INFO:teuthology.orchestra.run.smithi103.stdout: "complete_latency": { 2024-04-17T06:13:25.364 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 4157, 2024-04-17T06:13:25.364 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.122004178, 2024-04-17T06:13:25.364 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000029349 2024-04-17T06:13:25.364 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-04-17T06:13:25.364 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:25.364 INFO:teuthology.orchestra.run.smithi103.stdout: "finisher-PurgeQueue": { 2024-04-17T06:13:25.364 INFO:teuthology.orchestra.run.smithi103.stdout: "queue_len": 0, 2024-04-17T06:13:25.364 INFO:teuthology.orchestra.run.smithi103.stdout: "complete_latency": { 2024-04-17T06:13:25.365 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:25.365 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:25.365 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:25.365 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-04-17T06:13:25.365 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:25.365 INFO:teuthology.orchestra.run.smithi103.stdout: "mds": { 2024-04-17T06:13:25.365 INFO:teuthology.orchestra.run.smithi103.stdout: "request": 0, 2024-04-17T06:13:25.365 INFO:teuthology.orchestra.run.smithi103.stdout: "reply": 0, 2024-04-17T06:13:25.365 INFO:teuthology.orchestra.run.smithi103.stdout: "reply_latency": { 2024-04-17T06:13:25.365 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:25.365 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:25.365 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:25.365 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:25.365 INFO:teuthology.orchestra.run.smithi103.stdout: "slow_reply": 0, 2024-04-17T06:13:25.365 INFO:teuthology.orchestra.run.smithi103.stdout: "forward": 0, 2024-04-17T06:13:25.366 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_fetch_complete": 0, 2024-04-17T06:13:25.366 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_fetch_keys": 0, 2024-04-17T06:13:25.366 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_commit": 0, 2024-04-17T06:13:25.366 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_split": 0, 2024-04-17T06:13:25.366 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_merge": 0, 2024-04-17T06:13:25.366 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes": 0, 2024-04-17T06:13:25.366 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_top": 0, 2024-04-17T06:13:25.366 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_bottom": 0, 2024-04-17T06:13:25.366 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_pin_tail": 0, 2024-04-17T06:13:25.366 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_pinned": 0, 2024-04-17T06:13:25.366 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_expired": 0, 2024-04-17T06:13:25.366 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_with_caps": 0, 2024-04-17T06:13:25.366 INFO:teuthology.orchestra.run.smithi103.stdout: "caps": 0, 2024-04-17T06:13:25.366 INFO:teuthology.orchestra.run.smithi103.stdout: "subtrees": 1, 2024-04-17T06:13:25.366 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse": 0, 2024-04-17T06:13:25.367 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_hit": 0, 2024-04-17T06:13:25.367 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_forward": 0, 2024-04-17T06:13:25.367 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_discover": 0, 2024-04-17T06:13:25.367 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_dir_fetch": 0, 2024-04-17T06:13:25.367 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_remote_ino": 0, 2024-04-17T06:13:25.367 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_lock": 0, 2024-04-17T06:13:25.367 INFO:teuthology.orchestra.run.smithi103.stdout: "load_cent": 0, 2024-04-17T06:13:25.367 INFO:teuthology.orchestra.run.smithi103.stdout: "q": 0, 2024-04-17T06:13:25.367 INFO:teuthology.orchestra.run.smithi103.stdout: "exported": 0, 2024-04-17T06:13:25.367 INFO:teuthology.orchestra.run.smithi103.stdout: "exported_inodes": 0, 2024-04-17T06:13:25.367 INFO:teuthology.orchestra.run.smithi103.stdout: "imported": 0, 2024-04-17T06:13:25.367 INFO:teuthology.orchestra.run.smithi103.stdout: "imported_inodes": 0, 2024-04-17T06:13:25.368 INFO:teuthology.orchestra.run.smithi103.stdout: "openino_dir_fetch": 0, 2024-04-17T06:13:25.368 INFO:teuthology.orchestra.run.smithi103.stdout: "openino_backtrace_fetch": 0, 2024-04-17T06:13:25.368 INFO:teuthology.orchestra.run.smithi103.stdout: "openino_peer_discover": 0, 2024-04-17T06:13:25.368 INFO:teuthology.orchestra.run.smithi103.stdout: "root_rfiles": 0, 2024-04-17T06:13:25.368 INFO:teuthology.orchestra.run.smithi103.stdout: "root_rbytes": 0, 2024-04-17T06:13:25.368 INFO:teuthology.orchestra.run.smithi103.stdout: "root_rsnaps": 0, 2024-04-17T06:13:25.368 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_backtrace_fetch": 0, 2024-04-17T06:13:25.368 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_set_tag": 0, 2024-04-17T06:13:25.368 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_backtrace_repaired": 0, 2024-04-17T06:13:25.368 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_inotable_repaired": 0, 2024-04-17T06:13:25.368 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_dir_inodes": 0, 2024-04-17T06:13:25.368 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_dir_base_inodes": 0, 2024-04-17T06:13:25.368 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_dirfrag_rstats": 0, 2024-04-17T06:13:25.368 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_file_inodes": 0, 2024-04-17T06:13:25.368 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_inode_file_caps": 0, 2024-04-17T06:13:25.368 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_revoke": 0, 2024-04-17T06:13:25.369 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_grant": 0, 2024-04-17T06:13:25.369 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_trunc": 0, 2024-04-17T06:13:25.369 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-17T06:13:25.369 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-17T06:13:25.369 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_caps": 0, 2024-04-17T06:13:25.369 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_caps_dirty": 0, 2024-04-17T06:13:25.369 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_cap_release": 0, 2024-04-17T06:13:25.369 INFO:teuthology.orchestra.run.smithi103.stdout: "process_request_cap_release": 0 2024-04-17T06:13:25.369 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:25.369 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_cache": { 2024-04-17T06:13:25.369 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_update": 0, 2024-04-17T06:13:25.369 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_update_receipt": 0, 2024-04-17T06:13:25.369 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_try_discover": 0, 2024-04-17T06:13:25.369 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_send_discover": 0, 2024-04-17T06:13:25.369 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_handle_discover": 0, 2024-04-17T06:13:25.370 INFO:teuthology.orchestra.run.smithi103.stdout: "num_strays": 0, 2024-04-17T06:13:25.370 INFO:teuthology.orchestra.run.smithi103.stdout: "num_strays_delayed": 0, 2024-04-17T06:13:25.370 INFO:teuthology.orchestra.run.smithi103.stdout: "num_strays_enqueuing": 0, 2024-04-17T06:13:25.370 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_created": 0, 2024-04-17T06:13:25.370 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_enqueued": 0, 2024-04-17T06:13:25.370 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_reintegrated": 0, 2024-04-17T06:13:25.370 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_migrated": 0, 2024-04-17T06:13:25.370 INFO:teuthology.orchestra.run.smithi103.stdout: "num_recovering_processing": 0, 2024-04-17T06:13:25.370 INFO:teuthology.orchestra.run.smithi103.stdout: "num_recovering_enqueued": 0, 2024-04-17T06:13:25.370 INFO:teuthology.orchestra.run.smithi103.stdout: "num_recovering_prioritized": 0, 2024-04-17T06:13:25.370 INFO:teuthology.orchestra.run.smithi103.stdout: "recovery_started": 0, 2024-04-17T06:13:25.370 INFO:teuthology.orchestra.run.smithi103.stdout: "recovery_completed": 0, 2024-04-17T06:13:25.370 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_quiesce_path": 0, 2024-04-17T06:13:25.370 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_quiesce_inode": 0, 2024-04-17T06:13:25.370 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_enqueue_scrub": 0, 2024-04-17T06:13:25.371 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_exportdir": 0, 2024-04-17T06:13:25.371 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_flush": 0, 2024-04-17T06:13:25.371 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_fragmentdir": 0, 2024-04-17T06:13:25.371 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_fragstats": 0, 2024-04-17T06:13:25.371 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_inodestats": 0 2024-04-17T06:13:25.371 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:25.371 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_log": { 2024-04-17T06:13:25.371 INFO:teuthology.orchestra.run.smithi103.stdout: "evlrg": 0, 2024-04-17T06:13:25.371 INFO:teuthology.orchestra.run.smithi103.stdout: "evadd": 0, 2024-04-17T06:13:25.371 INFO:teuthology.orchestra.run.smithi103.stdout: "evex": 0, 2024-04-17T06:13:25.371 INFO:teuthology.orchestra.run.smithi103.stdout: "evtrm": 0, 2024-04-17T06:13:25.371 INFO:teuthology.orchestra.run.smithi103.stdout: "ev": 9, 2024-04-17T06:13:25.371 INFO:teuthology.orchestra.run.smithi103.stdout: "evexg": 0, 2024-04-17T06:13:25.371 INFO:teuthology.orchestra.run.smithi103.stdout: "evexd": 0, 2024-04-17T06:13:25.371 INFO:teuthology.orchestra.run.smithi103.stdout: "segadd": 0, 2024-04-17T06:13:25.372 INFO:teuthology.orchestra.run.smithi103.stdout: "segex": 0, 2024-04-17T06:13:25.372 INFO:teuthology.orchestra.run.smithi103.stdout: "segtrm": 0, 2024-04-17T06:13:25.372 INFO:teuthology.orchestra.run.smithi103.stdout: "seg": 2, 2024-04-17T06:13:25.372 INFO:teuthology.orchestra.run.smithi103.stdout: "segmjr": 2, 2024-04-17T06:13:25.372 INFO:teuthology.orchestra.run.smithi103.stdout: "segexg": 0, 2024-04-17T06:13:25.372 INFO:teuthology.orchestra.run.smithi103.stdout: "segexd": 0, 2024-04-17T06:13:25.372 INFO:teuthology.orchestra.run.smithi103.stdout: "expos": 4194304, 2024-04-17T06:13:25.372 INFO:teuthology.orchestra.run.smithi103.stdout: "wrpos": 4196796, 2024-04-17T06:13:25.372 INFO:teuthology.orchestra.run.smithi103.stdout: "rdpos": 4196655, 2024-04-17T06:13:25.372 INFO:teuthology.orchestra.run.smithi103.stdout: "jlat": { 2024-04-17T06:13:25.372 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:25.372 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:25.372 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:25.372 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:25.373 INFO:teuthology.orchestra.run.smithi103.stdout: "replayed": 9 2024-04-17T06:13:25.373 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:25.373 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_mem": { 2024-04-17T06:13:25.373 INFO:teuthology.orchestra.run.smithi103.stdout: "ino": 3, 2024-04-17T06:13:25.373 INFO:teuthology.orchestra.run.smithi103.stdout: "ino+": 3, 2024-04-17T06:13:25.373 INFO:teuthology.orchestra.run.smithi103.stdout: "ino-": 0, 2024-04-17T06:13:25.373 INFO:teuthology.orchestra.run.smithi103.stdout: "dir": 1, 2024-04-17T06:13:25.373 INFO:teuthology.orchestra.run.smithi103.stdout: "dir+": 1, 2024-04-17T06:13:25.373 INFO:teuthology.orchestra.run.smithi103.stdout: "dir-": 0, 2024-04-17T06:13:25.373 INFO:teuthology.orchestra.run.smithi103.stdout: "dn": 0, 2024-04-17T06:13:25.373 INFO:teuthology.orchestra.run.smithi103.stdout: "dn+": 0, 2024-04-17T06:13:25.373 INFO:teuthology.orchestra.run.smithi103.stdout: "dn-": 0, 2024-04-17T06:13:25.373 INFO:teuthology.orchestra.run.smithi103.stdout: "cap": 0, 2024-04-17T06:13:25.373 INFO:teuthology.orchestra.run.smithi103.stdout: "cap+": 0, 2024-04-17T06:13:25.373 INFO:teuthology.orchestra.run.smithi103.stdout: "cap-": 0, 2024-04-17T06:13:25.373 INFO:teuthology.orchestra.run.smithi103.stdout: "rss": 48004, 2024-04-17T06:13:25.374 INFO:teuthology.orchestra.run.smithi103.stdout: "heap": 215416 2024-04-17T06:13:25.374 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:25.374 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_server": { 2024-04-17T06:13:25.374 INFO:teuthology.orchestra.run.smithi103.stdout: "dispatch_client_request": 0, 2024-04-17T06:13:25.374 INFO:teuthology.orchestra.run.smithi103.stdout: "dispatch_server_request": 0, 2024-04-17T06:13:25.374 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_request": 0, 2024-04-17T06:13:25.374 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_session": 0, 2024-04-17T06:13:25.374 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_peer_request": 0, 2024-04-17T06:13:25.374 INFO:teuthology.orchestra.run.smithi103.stdout: "req_create_latency": { 2024-04-17T06:13:25.374 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:25.374 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:25.374 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:25.374 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:25.374 INFO:teuthology.orchestra.run.smithi103.stdout: "req_getattr_latency": { 2024-04-17T06:13:25.375 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:25.375 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:25.375 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:25.375 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:25.375 INFO:teuthology.orchestra.run.smithi103.stdout: "req_getfilelock_latency": { 2024-04-17T06:13:25.375 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:25.375 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:25.375 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:25.375 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:25.375 INFO:teuthology.orchestra.run.smithi103.stdout: "req_link_latency": { 2024-04-17T06:13:25.375 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:25.375 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:25.376 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:25.376 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:25.376 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookup_latency": { 2024-04-17T06:13:25.376 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:25.376 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:25.376 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:25.376 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:25.376 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookuphash_latency": { 2024-04-17T06:13:25.376 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:25.376 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:25.376 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:25.376 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:25.376 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupino_latency": { 2024-04-17T06:13:25.376 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:25.376 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:25.376 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:25.377 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:25.377 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupname_latency": { 2024-04-17T06:13:25.377 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:25.377 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:25.377 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:25.377 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:25.377 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupparent_latency": { 2024-04-17T06:13:25.377 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:25.377 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:25.377 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:25.378 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:25.378 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupsnap_latency": { 2024-04-17T06:13:25.378 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:25.378 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:25.378 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:25.378 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:25.378 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lssnap_latency": { 2024-04-17T06:13:25.378 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:25.378 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:25.378 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:25.378 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:25.378 INFO:teuthology.orchestra.run.smithi103.stdout: "req_mkdir_latency": { 2024-04-17T06:13:25.378 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:25.379 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:25.379 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:25.379 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:25.379 INFO:teuthology.orchestra.run.smithi103.stdout: "req_mknod_latency": { 2024-04-17T06:13:25.379 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:25.379 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:25.379 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:25.379 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:25.380 INFO:teuthology.orchestra.run.smithi103.stdout: "req_mksnap_latency": { 2024-04-17T06:13:25.380 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:25.380 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:25.380 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:25.380 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:25.380 INFO:teuthology.orchestra.run.smithi103.stdout: "req_open_latency": { 2024-04-17T06:13:25.380 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:25.380 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:25.380 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:25.380 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:25.380 INFO:teuthology.orchestra.run.smithi103.stdout: "req_readdir_latency": { 2024-04-17T06:13:25.380 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:25.381 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:25.381 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:25.381 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:25.381 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rename_latency": { 2024-04-17T06:13:25.381 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:25.381 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:25.381 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:25.381 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:25.381 INFO:teuthology.orchestra.run.smithi103.stdout: "req_renamesnap_latency": { 2024-04-17T06:13:25.381 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:25.381 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:25.381 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:25.382 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:25.382 INFO:teuthology.orchestra.run.smithi103.stdout: "req_snapdiff_latency": { 2024-04-17T06:13:25.382 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:25.382 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:25.382 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:25.382 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:25.382 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rmdir_latency": { 2024-04-17T06:13:25.382 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:25.382 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:25.382 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:25.382 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:25.382 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rmsnap_latency": { 2024-04-17T06:13:25.383 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:25.383 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:25.383 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:25.383 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:25.383 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rmxattr_latency": { 2024-04-17T06:13:25.383 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:25.383 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:25.383 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:25.383 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:25.383 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setattr_latency": { 2024-04-17T06:13:25.383 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:25.383 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:25.383 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:25.383 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:25.383 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setdirlayout_latency": { 2024-04-17T06:13:25.384 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:25.384 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:25.384 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:25.384 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:25.384 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setfilelock_latency": { 2024-04-17T06:13:25.384 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:25.384 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:25.384 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:25.384 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:25.384 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setlayout_latency": { 2024-04-17T06:13:25.384 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:25.384 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:25.384 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:25.384 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:25.385 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setxattr_latency": { 2024-04-17T06:13:25.385 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:25.385 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:25.385 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:25.385 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:25.385 INFO:teuthology.orchestra.run.smithi103.stdout: "req_symlink_latency": { 2024-04-17T06:13:25.385 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:25.385 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:25.385 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:25.385 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:25.385 INFO:teuthology.orchestra.run.smithi103.stdout: "req_unlink_latency": { 2024-04-17T06:13:25.385 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:25.385 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:25.385 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:25.385 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:25.386 INFO:teuthology.orchestra.run.smithi103.stdout: "cap_revoke_eviction": 0, 2024-04-17T06:13:25.386 INFO:teuthology.orchestra.run.smithi103.stdout: "cap_acquisition_throttle": 0, 2024-04-17T06:13:25.386 INFO:teuthology.orchestra.run.smithi103.stdout: "req_getvxattr_latency": { 2024-04-17T06:13:25.386 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:25.386 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:25.386 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:25.386 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-04-17T06:13:25.386 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:25.386 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_sessions": { 2024-04-17T06:13:25.386 INFO:teuthology.orchestra.run.smithi103.stdout: "session_count": 1, 2024-04-17T06:13:25.386 INFO:teuthology.orchestra.run.smithi103.stdout: "session_add": 4, 2024-04-17T06:13:25.386 INFO:teuthology.orchestra.run.smithi103.stdout: "session_remove": 3, 2024-04-17T06:13:25.387 INFO:teuthology.orchestra.run.smithi103.stdout: "sessions_open": 3, 2024-04-17T06:13:25.387 INFO:teuthology.orchestra.run.smithi103.stdout: "sessions_stale": 0, 2024-04-17T06:13:25.387 INFO:teuthology.orchestra.run.smithi103.stdout: "total_load": 0, 2024-04-17T06:13:25.387 INFO:teuthology.orchestra.run.smithi103.stdout: "average_load": 0, 2024-04-17T06:13:25.387 INFO:teuthology.orchestra.run.smithi103.stdout: "avg_session_uptime": 4737, 2024-04-17T06:13:25.387 INFO:teuthology.orchestra.run.smithi103.stdout: "mdthresh_evicted": 0 2024-04-17T06:13:25.387 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:25.387 INFO:teuthology.orchestra.run.smithi103.stdout: "mempool": { 2024-04-17T06:13:25.387 INFO:teuthology.orchestra.run.smithi103.stdout: "bloom_filter_bytes": 0, 2024-04-17T06:13:25.387 INFO:teuthology.orchestra.run.smithi103.stdout: "bloom_filter_items": 0, 2024-04-17T06:13:25.387 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_alloc_bytes": 0, 2024-04-17T06:13:25.388 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_alloc_items": 0, 2024-04-17T06:13:25.388 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_data_bytes": 0, 2024-04-17T06:13:25.388 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_data_items": 0, 2024-04-17T06:13:25.388 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-17T06:13:25.388 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_onode_items": 0, 2024-04-17T06:13:25.388 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-17T06:13:25.388 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_meta_items": 0, 2024-04-17T06:13:25.388 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_other_bytes": 0, 2024-04-17T06:13:25.388 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_other_items": 0, 2024-04-17T06:13:25.388 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-17T06:13:25.388 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_buffer_items": 0, 2024-04-17T06:13:25.389 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_extent_bytes": 0, 2024-04-17T06:13:25.389 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_extent_items": 0, 2024-04-17T06:13:25.389 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_blob_bytes": 0, 2024-04-17T06:13:25.389 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_blob_items": 0, 2024-04-17T06:13:25.389 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-17T06:13:25.389 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_shared_blob_items": 0, 2024-04-17T06:13:25.389 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-17T06:13:25.389 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_inline_bl_items": 0, 2024-04-17T06:13:25.389 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_fsck_bytes": 0, 2024-04-17T06:13:25.389 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_fsck_items": 0, 2024-04-17T06:13:25.389 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_txc_bytes": 0, 2024-04-17T06:13:25.389 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_txc_items": 0, 2024-04-17T06:13:25.389 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-17T06:13:25.390 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_deferred_items": 0, 2024-04-17T06:13:25.390 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_bytes": 0, 2024-04-17T06:13:25.390 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_items": 0, 2024-04-17T06:13:25.390 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_bytes": 0, 2024-04-17T06:13:25.390 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_items": 0, 2024-04-17T06:13:25.390 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_reader_bytes": 0, 2024-04-17T06:13:25.390 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_reader_items": 0, 2024-04-17T06:13:25.390 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_writer_bytes": 0, 2024-04-17T06:13:25.390 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_writer_items": 0, 2024-04-17T06:13:25.390 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_anon_bytes": 181343, 2024-04-17T06:13:25.390 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_anon_items": 47, 2024-04-17T06:13:25.390 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_meta_bytes": 0, 2024-04-17T06:13:25.390 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_meta_items": 0, 2024-04-17T06:13:25.390 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_bytes": 0, 2024-04-17T06:13:25.391 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_items": 0, 2024-04-17T06:13:25.391 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mapbl_bytes": 0, 2024-04-17T06:13:25.391 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mapbl_items": 0, 2024-04-17T06:13:25.391 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pglog_bytes": 0, 2024-04-17T06:13:25.391 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pglog_items": 0, 2024-04-17T06:13:25.391 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_bytes": 8032, 2024-04-17T06:13:25.391 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_items": 167, 2024-04-17T06:13:25.391 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_mapping_bytes": 0, 2024-04-17T06:13:25.391 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_mapping_items": 0, 2024-04-17T06:13:25.391 INFO:teuthology.orchestra.run.smithi103.stdout: "pgmap_bytes": 0, 2024-04-17T06:13:25.391 INFO:teuthology.orchestra.run.smithi103.stdout: "pgmap_items": 0, 2024-04-17T06:13:25.391 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_co_bytes": 8512, 2024-04-17T06:13:25.391 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_co_items": 13, 2024-04-17T06:13:25.391 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_1_bytes": 0, 2024-04-17T06:13:25.391 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_1_items": 0, 2024-04-17T06:13:25.392 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_2_bytes": 0, 2024-04-17T06:13:25.392 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_2_items": 0 2024-04-17T06:13:25.392 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:25.392 INFO:teuthology.orchestra.run.smithi103.stdout: "objecter": { 2024-04-17T06:13:25.392 INFO:teuthology.orchestra.run.smithi103.stdout: "op_active": 0, 2024-04-17T06:13:25.392 INFO:teuthology.orchestra.run.smithi103.stdout: "op_laggy": 0, 2024-04-17T06:13:25.392 INFO:teuthology.orchestra.run.smithi103.stdout: "op_send": 2463, 2024-04-17T06:13:25.392 INFO:teuthology.orchestra.run.smithi103.stdout: "op_send_bytes": 21, 2024-04-17T06:13:25.392 INFO:teuthology.orchestra.run.smithi103.stdout: "op_resend": 3, 2024-04-17T06:13:25.392 INFO:teuthology.orchestra.run.smithi103.stdout: "op_reply": 2463, 2024-04-17T06:13:25.392 INFO:teuthology.orchestra.run.smithi103.stdout: "op_latency": { 2024-04-17T06:13:25.392 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 2463, 2024-04-17T06:13:25.392 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 1.277976195, 2024-04-17T06:13:25.393 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000518869 2024-04-17T06:13:25.393 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:25.393 INFO:teuthology.orchestra.run.smithi103.stdout: "op_inflight": 0, 2024-04-17T06:13:25.393 INFO:teuthology.orchestra.run.smithi103.stdout: "oplen_avg": { 2024-04-17T06:13:25.393 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 2463, 2024-04-17T06:13:25.393 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 2464 2024-04-17T06:13:25.393 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:25.393 INFO:teuthology.orchestra.run.smithi103.stdout: "op": 2463, 2024-04-17T06:13:25.393 INFO:teuthology.orchestra.run.smithi103.stdout: "op_r": 2463, 2024-04-17T06:13:25.393 INFO:teuthology.orchestra.run.smithi103.stdout: "op_w": 0, 2024-04-17T06:13:25.393 INFO:teuthology.orchestra.run.smithi103.stdout: "op_rmw": 0, 2024-04-17T06:13:25.393 INFO:teuthology.orchestra.run.smithi103.stdout: "op_pg": 0, 2024-04-17T06:13:25.393 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_stat": 1606, 2024-04-17T06:13:25.394 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_create": 0, 2024-04-17T06:13:25.394 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_read": 855, 2024-04-17T06:13:25.394 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_write": 0, 2024-04-17T06:13:25.394 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_writefull": 0, 2024-04-17T06:13:25.394 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_writesame": 0, 2024-04-17T06:13:25.394 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_append": 0, 2024-04-17T06:13:25.394 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_zero": 0, 2024-04-17T06:13:25.394 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_truncate": 0, 2024-04-17T06:13:25.394 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_delete": 0, 2024-04-17T06:13:25.394 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_mapext": 0, 2024-04-17T06:13:25.394 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_sparse_read": 0, 2024-04-17T06:13:25.394 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_clonerange": 0, 2024-04-17T06:13:25.394 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_getxattr": 1, 2024-04-17T06:13:25.395 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_setxattr": 0, 2024-04-17T06:13:25.395 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_cmpxattr": 0, 2024-04-17T06:13:25.395 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_rmxattr": 0, 2024-04-17T06:13:25.395 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_resetxattrs": 0, 2024-04-17T06:13:25.395 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_call": 0, 2024-04-17T06:13:25.395 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_watch": 0, 2024-04-17T06:13:25.395 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_notify": 0, 2024-04-17T06:13:25.395 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_src_cmpxattr": 0, 2024-04-17T06:13:25.395 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_pgls": 0, 2024-04-17T06:13:25.395 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_pgls_filter": 0, 2024-04-17T06:13:25.395 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_other": 0, 2024-04-17T06:13:25.395 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_active": 0, 2024-04-17T06:13:25.395 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_send": 0, 2024-04-17T06:13:25.395 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_resend": 0, 2024-04-17T06:13:25.396 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_ping": 0, 2024-04-17T06:13:25.396 INFO:teuthology.orchestra.run.smithi103.stdout: "poolop_active": 0, 2024-04-17T06:13:25.396 INFO:teuthology.orchestra.run.smithi103.stdout: "poolop_send": 0, 2024-04-17T06:13:25.396 INFO:teuthology.orchestra.run.smithi103.stdout: "poolop_resend": 0, 2024-04-17T06:13:25.396 INFO:teuthology.orchestra.run.smithi103.stdout: "poolstat_active": 0, 2024-04-17T06:13:25.396 INFO:teuthology.orchestra.run.smithi103.stdout: "poolstat_send": 0, 2024-04-17T06:13:25.396 INFO:teuthology.orchestra.run.smithi103.stdout: "poolstat_resend": 0, 2024-04-17T06:13:25.396 INFO:teuthology.orchestra.run.smithi103.stdout: "statfs_active": 0, 2024-04-17T06:13:25.396 INFO:teuthology.orchestra.run.smithi103.stdout: "statfs_send": 0, 2024-04-17T06:13:25.396 INFO:teuthology.orchestra.run.smithi103.stdout: "statfs_resend": 0, 2024-04-17T06:13:25.396 INFO:teuthology.orchestra.run.smithi103.stdout: "command_active": 0, 2024-04-17T06:13:25.396 INFO:teuthology.orchestra.run.smithi103.stdout: "command_send": 0, 2024-04-17T06:13:25.396 INFO:teuthology.orchestra.run.smithi103.stdout: "command_resend": 0, 2024-04-17T06:13:25.397 INFO:teuthology.orchestra.run.smithi103.stdout: "map_epoch": 106, 2024-04-17T06:13:25.397 INFO:teuthology.orchestra.run.smithi103.stdout: "map_full": 0, 2024-04-17T06:13:25.397 INFO:teuthology.orchestra.run.smithi103.stdout: "map_inc": 20, 2024-04-17T06:13:25.397 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_sessions": 4, 2024-04-17T06:13:25.397 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_session_open": 4, 2024-04-17T06:13:25.397 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_session_close": 0, 2024-04-17T06:13:25.397 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_laggy": 0, 2024-04-17T06:13:25.397 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_wr": 0, 2024-04-17T06:13:25.397 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_rd": 2, 2024-04-17T06:13:25.397 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_del": 0 2024-04-17T06:13:25.397 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:25.397 INFO:teuthology.orchestra.run.smithi103.stdout: "oft": { 2024-04-17T06:13:25.397 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_objs": 0, 2024-04-17T06:13:25.397 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_kv_pairs": 0, 2024-04-17T06:13:25.398 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_updates": 0, 2024-04-17T06:13:25.398 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_removes": 0 2024-04-17T06:13:25.398 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:25.398 INFO:teuthology.orchestra.run.smithi103.stdout: "osd-slow-ops": { 2024-04-17T06:13:25.398 INFO:teuthology.orchestra.run.smithi103.stdout: "slow_ops_count": 0 2024-04-17T06:13:25.398 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:25.398 INFO:teuthology.orchestra.run.smithi103.stdout: "purge_queue": { 2024-04-17T06:13:25.398 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing_ops": 0, 2024-04-17T06:13:25.398 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing_ops_high_water": 0, 2024-04-17T06:13:25.398 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing": 0, 2024-04-17T06:13:25.398 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing_high_water": 0, 2024-04-17T06:13:25.398 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executed_ops": 0, 2024-04-17T06:13:25.399 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executed": 0, 2024-04-17T06:13:25.399 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_item_in_journal": 0 2024-04-17T06:13:25.399 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:25.399 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-17T06:13:25.399 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-04-17T06:13:25.399 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 104857600, 2024-04-17T06:13:25.399 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-04-17T06:13:25.399 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 2749, 2024-04-17T06:13:25.399 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 2120234, 2024-04-17T06:13:25.399 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-04-17T06:13:25.399 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 2749, 2024-04-17T06:13:25.399 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 0, 2024-04-17T06:13:25.399 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 0, 2024-04-17T06:13:25.400 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 2749, 2024-04-17T06:13:25.400 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 2120234, 2024-04-17T06:13:25.400 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-04-17T06:13:25.400 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:25.400 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:25.400 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:25.400 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-04-17T06:13:25.400 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:25.400 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-objecter_bytes": { 2024-04-17T06:13:25.400 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-04-17T06:13:25.400 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 104857600, 2024-04-17T06:13:25.400 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-04-17T06:13:25.400 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 0, 2024-04-17T06:13:25.400 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 0, 2024-04-17T06:13:25.400 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-04-17T06:13:25.401 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 0, 2024-04-17T06:13:25.401 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 2463, 2024-04-17T06:13:25.401 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 2497, 2024-04-17T06:13:25.401 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 2463, 2024-04-17T06:13:25.401 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 2497, 2024-04-17T06:13:25.401 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-04-17T06:13:25.401 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:25.401 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:25.401 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:25.401 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-04-17T06:13:25.401 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:25.401 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-objecter_ops": { 2024-04-17T06:13:25.401 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-04-17T06:13:25.401 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 1024, 2024-04-17T06:13:25.402 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-04-17T06:13:25.402 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 0, 2024-04-17T06:13:25.402 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 0, 2024-04-17T06:13:25.402 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-04-17T06:13:25.402 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 0, 2024-04-17T06:13:25.402 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 2463, 2024-04-17T06:13:25.402 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 2463, 2024-04-17T06:13:25.402 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 2463, 2024-04-17T06:13:25.402 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 2463, 2024-04-17T06:13:25.402 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-04-17T06:13:25.402 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:25.402 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:25.403 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:25.403 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-04-17T06:13:25.403 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:25.403 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-write_buf_throttle": { 2024-04-17T06:13:25.403 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-04-17T06:13:25.403 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 3758096384, 2024-04-17T06:13:25.403 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-04-17T06:13:25.403 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 0, 2024-04-17T06:13:25.403 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 0, 2024-04-17T06:13:25.403 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-04-17T06:13:25.403 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 0, 2024-04-17T06:13:25.403 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 0, 2024-04-17T06:13:25.403 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 0, 2024-04-17T06:13:25.403 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 0, 2024-04-17T06:13:25.404 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 0, 2024-04-17T06:13:25.404 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-04-17T06:13:25.404 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:25.404 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:25.404 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:25.404 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-04-17T06:13:25.404 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:25.404 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-write_buf_throttle-0x55d54ce0a6e0": { 2024-04-17T06:13:25.404 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-04-17T06:13:25.404 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 3758096384, 2024-04-17T06:13:25.404 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-04-17T06:13:25.404 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 0, 2024-04-17T06:13:25.404 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 0, 2024-04-17T06:13:25.404 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-04-17T06:13:25.405 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 0, 2024-04-17T06:13:25.405 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 0, 2024-04-17T06:13:25.405 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 0, 2024-04-17T06:13:25.405 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 0, 2024-04-17T06:13:25.405 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 0, 2024-04-17T06:13:25.405 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-04-17T06:13:25.405 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:25.405 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:25.405 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:25.405 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-04-17T06:13:25.406 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-04-17T06:13:25.406 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-04-17T06:13:26.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:13:25 smithi098 ceph-mon[36516]: pgmap v383: 97 pgs: 97 active+clean; 5.9 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-17T06:13:26.138 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:13:25 smithi103 ceph-mon[36243]: pgmap v383: 97 pgs: 97 active+clean; 5.9 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-17T06:13:26.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:13:25 smithi047 ceph-mon[32223]: pgmap v383: 97 pgs: 97 active+clean; 5.9 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-17T06:13:26.163 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-04-17T06:13:26.163 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-04-17T06:13:26.163 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=0 2024-04-17T06:13:26.163 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-04-17T06:13:26.163 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-04-17T06:13:26.164 DEBUG:tasks.check_counter:Getting stats from f 2024-04-17T06:13:26.164 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 565db2d8-fc7e-11ee-bc91-c7b262605968 -- ceph daemon mds.f perf dump 2024-04-17T06:13:26.413 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/mon.c/config 2024-04-17T06:13:28.044 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-04-17T06:13:28.044 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-0": { 2024-04-17T06:13:28.044 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 1461, 2024-04-17T06:13:28.044 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 1463, 2024-04-17T06:13:28.044 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 2640541, 2024-04-17T06:13:28.044 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 464662, 2024-04-17T06:13:28.044 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 6, 2024-04-17T06:13:28.044 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 2, 2024-04-17T06:13:28.044 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.143956253, 2024-04-17T06:13:28.044 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.041070169, 2024-04-17T06:13:28.045 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.044552094, 2024-04-17T06:13:28.045 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.044852966, 2024-04-17T06:13:28.045 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T06:13:28.045 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 1461, 2024-04-17T06:13:28.045 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.054025908, 2024-04-17T06:13:28.045 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000036978 2024-04-17T06:13:28.045 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:28.045 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-04-17T06:13:28.045 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:28.045 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:28.045 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:28.045 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:28.045 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-17T06:13:28.045 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 0 2024-04-17T06:13:28.046 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:28.046 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-1": { 2024-04-17T06:13:28.046 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 1055, 2024-04-17T06:13:28.046 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 1027, 2024-04-17T06:13:28.046 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 1874884, 2024-04-17T06:13:28.046 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 392779, 2024-04-17T06:13:28.046 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 4, 2024-04-17T06:13:28.046 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 3, 2024-04-17T06:13:28.046 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.133543385, 2024-04-17T06:13:28.046 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.038320433, 2024-04-17T06:13:28.047 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.060183478, 2024-04-17T06:13:28.047 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.037722333, 2024-04-17T06:13:28.047 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T06:13:28.047 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 1027, 2024-04-17T06:13:28.047 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.044012218, 2024-04-17T06:13:28.047 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000042855 2024-04-17T06:13:28.047 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:28.047 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-04-17T06:13:28.047 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:28.047 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:28.047 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:28.047 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:28.047 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 1686672, 2024-04-17T06:13:28.048 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 149504 2024-04-17T06:13:28.048 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:28.048 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-2": { 2024-04-17T06:13:28.048 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 849, 2024-04-17T06:13:28.048 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 1026, 2024-04-17T06:13:28.048 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 284912, 2024-04-17T06:13:28.048 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 747138, 2024-04-17T06:13:28.048 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 6, 2024-04-17T06:13:28.048 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 4, 2024-04-17T06:13:28.048 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.116589295, 2024-04-17T06:13:28.048 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.038932442, 2024-04-17T06:13:28.048 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.029390118, 2024-04-17T06:13:28.048 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.035415034, 2024-04-17T06:13:28.048 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T06:13:28.049 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 1023, 2024-04-17T06:13:28.049 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.065153178, 2024-04-17T06:13:28.049 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000063688 2024-04-17T06:13:28.049 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:28.049 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-04-17T06:13:28.049 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:28.049 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:28.049 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:28.049 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:28.049 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-17T06:13:28.049 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 477792 2024-04-17T06:13:28.049 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:28.049 INFO:teuthology.orchestra.run.smithi103.stdout: "cct": { 2024-04-17T06:13:28.049 INFO:teuthology.orchestra.run.smithi103.stdout: "total_workers": 1, 2024-04-17T06:13:28.050 INFO:teuthology.orchestra.run.smithi103.stdout: "unhealthy_workers": 0 2024-04-17T06:13:28.050 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:28.050 INFO:teuthology.orchestra.run.smithi103.stdout: "finisher-MDSRank": { 2024-04-17T06:13:28.050 INFO:teuthology.orchestra.run.smithi103.stdout: "queue_len": 0, 2024-04-17T06:13:28.050 INFO:teuthology.orchestra.run.smithi103.stdout: "complete_latency": { 2024-04-17T06:13:28.050 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 5309, 2024-04-17T06:13:28.050 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.150471413, 2024-04-17T06:13:28.050 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000028342 2024-04-17T06:13:28.050 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-04-17T06:13:28.050 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:28.050 INFO:teuthology.orchestra.run.smithi103.stdout: "finisher-PurgeQueue": { 2024-04-17T06:13:28.050 INFO:teuthology.orchestra.run.smithi103.stdout: "queue_len": 0, 2024-04-17T06:13:28.050 INFO:teuthology.orchestra.run.smithi103.stdout: "complete_latency": { 2024-04-17T06:13:28.051 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:28.051 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:28.051 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:28.051 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-04-17T06:13:28.051 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:28.051 INFO:teuthology.orchestra.run.smithi103.stdout: "mds": { 2024-04-17T06:13:28.051 INFO:teuthology.orchestra.run.smithi103.stdout: "request": 0, 2024-04-17T06:13:28.051 INFO:teuthology.orchestra.run.smithi103.stdout: "reply": 0, 2024-04-17T06:13:28.051 INFO:teuthology.orchestra.run.smithi103.stdout: "reply_latency": { 2024-04-17T06:13:28.051 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:28.051 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:28.052 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:28.052 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:28.052 INFO:teuthology.orchestra.run.smithi103.stdout: "slow_reply": 0, 2024-04-17T06:13:28.052 INFO:teuthology.orchestra.run.smithi103.stdout: "forward": 0, 2024-04-17T06:13:28.052 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_fetch_complete": 0, 2024-04-17T06:13:28.052 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_fetch_keys": 0, 2024-04-17T06:13:28.052 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_commit": 0, 2024-04-17T06:13:28.052 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_split": 0, 2024-04-17T06:13:28.052 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_merge": 0, 2024-04-17T06:13:28.052 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes": 2, 2024-04-17T06:13:28.052 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_top": 0, 2024-04-17T06:13:28.052 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_bottom": 2, 2024-04-17T06:13:28.052 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_pin_tail": 0, 2024-04-17T06:13:28.052 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_pinned": 2, 2024-04-17T06:13:28.053 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_expired": 0, 2024-04-17T06:13:28.053 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_with_caps": 0, 2024-04-17T06:13:28.053 INFO:teuthology.orchestra.run.smithi103.stdout: "caps": 0, 2024-04-17T06:13:28.053 INFO:teuthology.orchestra.run.smithi103.stdout: "subtrees": 1, 2024-04-17T06:13:28.053 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse": 0, 2024-04-17T06:13:28.053 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_hit": 0, 2024-04-17T06:13:28.053 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_forward": 0, 2024-04-17T06:13:28.053 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_discover": 0, 2024-04-17T06:13:28.053 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_dir_fetch": 0, 2024-04-17T06:13:28.053 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_remote_ino": 0, 2024-04-17T06:13:28.054 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_lock": 0, 2024-04-17T06:13:28.054 INFO:teuthology.orchestra.run.smithi103.stdout: "load_cent": 0, 2024-04-17T06:13:28.054 INFO:teuthology.orchestra.run.smithi103.stdout: "q": 0, 2024-04-17T06:13:28.054 INFO:teuthology.orchestra.run.smithi103.stdout: "exported": 0, 2024-04-17T06:13:28.054 INFO:teuthology.orchestra.run.smithi103.stdout: "exported_inodes": 0, 2024-04-17T06:13:28.054 INFO:teuthology.orchestra.run.smithi103.stdout: "imported": 0, 2024-04-17T06:13:28.054 INFO:teuthology.orchestra.run.smithi103.stdout: "imported_inodes": 0, 2024-04-17T06:13:28.054 INFO:teuthology.orchestra.run.smithi103.stdout: "openino_dir_fetch": 0, 2024-04-17T06:13:28.054 INFO:teuthology.orchestra.run.smithi103.stdout: "openino_backtrace_fetch": 0, 2024-04-17T06:13:28.054 INFO:teuthology.orchestra.run.smithi103.stdout: "openino_peer_discover": 0, 2024-04-17T06:13:28.054 INFO:teuthology.orchestra.run.smithi103.stdout: "root_rfiles": 0, 2024-04-17T06:13:28.054 INFO:teuthology.orchestra.run.smithi103.stdout: "root_rbytes": 0, 2024-04-17T06:13:28.054 INFO:teuthology.orchestra.run.smithi103.stdout: "root_rsnaps": 0, 2024-04-17T06:13:28.054 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_backtrace_fetch": 0, 2024-04-17T06:13:28.055 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_set_tag": 0, 2024-04-17T06:13:28.055 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_backtrace_repaired": 0, 2024-04-17T06:13:28.055 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_inotable_repaired": 0, 2024-04-17T06:13:28.055 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_dir_inodes": 0, 2024-04-17T06:13:28.055 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_dir_base_inodes": 0, 2024-04-17T06:13:28.055 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_dirfrag_rstats": 0, 2024-04-17T06:13:28.055 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_file_inodes": 0, 2024-04-17T06:13:28.055 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_inode_file_caps": 0, 2024-04-17T06:13:28.055 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_revoke": 0, 2024-04-17T06:13:28.055 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_grant": 0, 2024-04-17T06:13:28.055 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_trunc": 0, 2024-04-17T06:13:28.055 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-17T06:13:28.055 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-17T06:13:28.055 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_caps": 0, 2024-04-17T06:13:28.056 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_caps_dirty": 0, 2024-04-17T06:13:28.056 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_cap_release": 0, 2024-04-17T06:13:28.056 INFO:teuthology.orchestra.run.smithi103.stdout: "process_request_cap_release": 0 2024-04-17T06:13:28.056 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:28.056 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_cache": { 2024-04-17T06:13:28.056 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_update": 0, 2024-04-17T06:13:28.056 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_update_receipt": 0, 2024-04-17T06:13:28.056 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_try_discover": 0, 2024-04-17T06:13:28.056 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_send_discover": 0, 2024-04-17T06:13:28.056 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_handle_discover": 0, 2024-04-17T06:13:28.056 INFO:teuthology.orchestra.run.smithi103.stdout: "num_strays": 0, 2024-04-17T06:13:28.056 INFO:teuthology.orchestra.run.smithi103.stdout: "num_strays_delayed": 0, 2024-04-17T06:13:28.056 INFO:teuthology.orchestra.run.smithi103.stdout: "num_strays_enqueuing": 0, 2024-04-17T06:13:28.056 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_created": 0, 2024-04-17T06:13:28.057 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_enqueued": 0, 2024-04-17T06:13:28.057 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_reintegrated": 0, 2024-04-17T06:13:28.057 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_migrated": 0, 2024-04-17T06:13:28.057 INFO:teuthology.orchestra.run.smithi103.stdout: "num_recovering_processing": 0, 2024-04-17T06:13:28.057 INFO:teuthology.orchestra.run.smithi103.stdout: "num_recovering_enqueued": 0, 2024-04-17T06:13:28.057 INFO:teuthology.orchestra.run.smithi103.stdout: "num_recovering_prioritized": 0, 2024-04-17T06:13:28.057 INFO:teuthology.orchestra.run.smithi103.stdout: "recovery_started": 0, 2024-04-17T06:13:28.057 INFO:teuthology.orchestra.run.smithi103.stdout: "recovery_completed": 0, 2024-04-17T06:13:28.057 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_quiesce_path": 0, 2024-04-17T06:13:28.057 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_quiesce_inode": 0, 2024-04-17T06:13:28.057 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_enqueue_scrub": 0, 2024-04-17T06:13:28.057 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_exportdir": 0, 2024-04-17T06:13:28.057 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_flush": 0, 2024-04-17T06:13:28.057 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_fragmentdir": 0, 2024-04-17T06:13:28.057 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_fragstats": 0, 2024-04-17T06:13:28.058 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_inodestats": 0 2024-04-17T06:13:28.058 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:28.058 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_log": { 2024-04-17T06:13:28.058 INFO:teuthology.orchestra.run.smithi103.stdout: "evlrg": 0, 2024-04-17T06:13:28.058 INFO:teuthology.orchestra.run.smithi103.stdout: "evadd": 0, 2024-04-17T06:13:28.058 INFO:teuthology.orchestra.run.smithi103.stdout: "evex": 0, 2024-04-17T06:13:28.058 INFO:teuthology.orchestra.run.smithi103.stdout: "evtrm": 0, 2024-04-17T06:13:28.058 INFO:teuthology.orchestra.run.smithi103.stdout: "ev": 1945, 2024-04-17T06:13:28.058 INFO:teuthology.orchestra.run.smithi103.stdout: "evexg": 0, 2024-04-17T06:13:28.058 INFO:teuthology.orchestra.run.smithi103.stdout: "evexd": 0, 2024-04-17T06:13:28.058 INFO:teuthology.orchestra.run.smithi103.stdout: "segadd": 0, 2024-04-17T06:13:28.058 INFO:teuthology.orchestra.run.smithi103.stdout: "segex": 0, 2024-04-17T06:13:28.058 INFO:teuthology.orchestra.run.smithi103.stdout: "segtrm": 0, 2024-04-17T06:13:28.058 INFO:teuthology.orchestra.run.smithi103.stdout: "seg": 3, 2024-04-17T06:13:28.059 INFO:teuthology.orchestra.run.smithi103.stdout: "segmjr": 2, 2024-04-17T06:13:28.059 INFO:teuthology.orchestra.run.smithi103.stdout: "segexg": 0, 2024-04-17T06:13:28.059 INFO:teuthology.orchestra.run.smithi103.stdout: "segexd": 0, 2024-04-17T06:13:28.059 INFO:teuthology.orchestra.run.smithi103.stdout: "expos": 4194304, 2024-04-17T06:13:28.059 INFO:teuthology.orchestra.run.smithi103.stdout: "wrpos": 6461200, 2024-04-17T06:13:28.059 INFO:teuthology.orchestra.run.smithi103.stdout: "rdpos": 6458313, 2024-04-17T06:13:28.059 INFO:teuthology.orchestra.run.smithi103.stdout: "jlat": { 2024-04-17T06:13:28.059 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:28.059 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:28.059 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:28.059 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:28.059 INFO:teuthology.orchestra.run.smithi103.stdout: "replayed": 1945 2024-04-17T06:13:28.059 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:28.059 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_mem": { 2024-04-17T06:13:28.060 INFO:teuthology.orchestra.run.smithi103.stdout: "ino": 5, 2024-04-17T06:13:28.060 INFO:teuthology.orchestra.run.smithi103.stdout: "ino+": 23, 2024-04-17T06:13:28.060 INFO:teuthology.orchestra.run.smithi103.stdout: "ino-": 18, 2024-04-17T06:13:28.060 INFO:teuthology.orchestra.run.smithi103.stdout: "dir": 2, 2024-04-17T06:13:28.060 INFO:teuthology.orchestra.run.smithi103.stdout: "dir+": 11, 2024-04-17T06:13:28.060 INFO:teuthology.orchestra.run.smithi103.stdout: "dir-": 9, 2024-04-17T06:13:28.060 INFO:teuthology.orchestra.run.smithi103.stdout: "dn": 2, 2024-04-17T06:13:28.060 INFO:teuthology.orchestra.run.smithi103.stdout: "dn+": 21, 2024-04-17T06:13:28.060 INFO:teuthology.orchestra.run.smithi103.stdout: "dn-": 19, 2024-04-17T06:13:28.060 INFO:teuthology.orchestra.run.smithi103.stdout: "cap": 0, 2024-04-17T06:13:28.060 INFO:teuthology.orchestra.run.smithi103.stdout: "cap+": 0, 2024-04-17T06:13:28.060 INFO:teuthology.orchestra.run.smithi103.stdout: "cap-": 0, 2024-04-17T06:13:28.060 INFO:teuthology.orchestra.run.smithi103.stdout: "rss": 48028, 2024-04-17T06:13:28.060 INFO:teuthology.orchestra.run.smithi103.stdout: "heap": 215416 2024-04-17T06:13:28.060 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:28.061 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_server": { 2024-04-17T06:13:28.061 INFO:teuthology.orchestra.run.smithi103.stdout: "dispatch_client_request": 0, 2024-04-17T06:13:28.061 INFO:teuthology.orchestra.run.smithi103.stdout: "dispatch_server_request": 0, 2024-04-17T06:13:28.061 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_request": 0, 2024-04-17T06:13:28.061 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_session": 0, 2024-04-17T06:13:28.061 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_peer_request": 0, 2024-04-17T06:13:28.061 INFO:teuthology.orchestra.run.smithi103.stdout: "req_create_latency": { 2024-04-17T06:13:28.061 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:28.061 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:28.061 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:28.061 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:28.061 INFO:teuthology.orchestra.run.smithi103.stdout: "req_getattr_latency": { 2024-04-17T06:13:28.061 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:28.061 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:28.061 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:28.062 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:28.062 INFO:teuthology.orchestra.run.smithi103.stdout: "req_getfilelock_latency": { 2024-04-17T06:13:28.062 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:28.062 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:28.062 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:28.062 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:28.062 INFO:teuthology.orchestra.run.smithi103.stdout: "req_link_latency": { 2024-04-17T06:13:28.062 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:28.062 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:28.062 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:28.062 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:28.062 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookup_latency": { 2024-04-17T06:13:28.062 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:28.062 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:28.062 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:28.063 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:28.063 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookuphash_latency": { 2024-04-17T06:13:28.063 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:28.063 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:28.063 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:28.063 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:28.063 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupino_latency": { 2024-04-17T06:13:28.063 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:28.063 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:28.063 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:28.063 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:28.063 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupname_latency": { 2024-04-17T06:13:28.063 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:28.064 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:28.064 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:28.064 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:28.064 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupparent_latency": { 2024-04-17T06:13:28.064 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:28.064 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:28.064 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:28.064 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:28.064 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupsnap_latency": { 2024-04-17T06:13:28.064 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:28.064 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:28.065 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:28.065 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:28.065 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lssnap_latency": { 2024-04-17T06:13:28.065 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:28.065 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:28.065 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:28.065 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:28.065 INFO:teuthology.orchestra.run.smithi103.stdout: "req_mkdir_latency": { 2024-04-17T06:13:28.065 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:28.066 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:28.066 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:28.066 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:28.066 INFO:teuthology.orchestra.run.smithi103.stdout: "req_mknod_latency": { 2024-04-17T06:13:28.066 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:28.066 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:28.066 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:28.066 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:28.067 INFO:teuthology.orchestra.run.smithi103.stdout: "req_mksnap_latency": { 2024-04-17T06:13:28.067 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:28.067 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:28.067 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:28.067 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:28.067 INFO:teuthology.orchestra.run.smithi103.stdout: "req_open_latency": { 2024-04-17T06:13:28.067 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:28.067 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:28.067 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:28.067 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:28.067 INFO:teuthology.orchestra.run.smithi103.stdout: "req_readdir_latency": { 2024-04-17T06:13:28.067 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:28.067 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:28.068 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:28.068 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:28.068 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rename_latency": { 2024-04-17T06:13:28.068 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:28.068 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:28.068 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:28.068 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:28.068 INFO:teuthology.orchestra.run.smithi103.stdout: "req_renamesnap_latency": { 2024-04-17T06:13:28.068 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:28.069 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:28.069 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:28.069 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:28.069 INFO:teuthology.orchestra.run.smithi103.stdout: "req_snapdiff_latency": { 2024-04-17T06:13:28.069 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:28.069 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:28.069 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:28.069 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:28.069 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rmdir_latency": { 2024-04-17T06:13:28.069 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:28.069 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:28.069 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:28.069 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:28.069 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rmsnap_latency": { 2024-04-17T06:13:28.069 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:28.070 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:28.070 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:28.070 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:28.070 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rmxattr_latency": { 2024-04-17T06:13:28.070 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:28.070 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:28.070 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:28.070 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:28.070 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setattr_latency": { 2024-04-17T06:13:28.070 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:28.070 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:28.070 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:28.070 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:28.070 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setdirlayout_latency": { 2024-04-17T06:13:28.071 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:28.071 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:28.071 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:28.071 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:28.071 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setfilelock_latency": { 2024-04-17T06:13:28.071 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:28.071 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:28.071 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:28.071 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:28.071 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setlayout_latency": { 2024-04-17T06:13:28.071 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:28.071 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:28.071 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:28.071 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:28.072 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setxattr_latency": { 2024-04-17T06:13:28.072 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:28.072 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:28.072 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:28.072 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:28.072 INFO:teuthology.orchestra.run.smithi103.stdout: "req_symlink_latency": { 2024-04-17T06:13:28.072 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:28.072 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:28.072 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:28.072 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:28.072 INFO:teuthology.orchestra.run.smithi103.stdout: "req_unlink_latency": { 2024-04-17T06:13:28.072 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:28.072 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:28.072 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:28.072 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:28.073 INFO:teuthology.orchestra.run.smithi103.stdout: "cap_revoke_eviction": 0, 2024-04-17T06:13:28.073 INFO:teuthology.orchestra.run.smithi103.stdout: "cap_acquisition_throttle": 0, 2024-04-17T06:13:28.073 INFO:teuthology.orchestra.run.smithi103.stdout: "req_getvxattr_latency": { 2024-04-17T06:13:28.073 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:28.073 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:28.073 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:28.073 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-04-17T06:13:28.073 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:28.073 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_sessions": { 2024-04-17T06:13:28.073 INFO:teuthology.orchestra.run.smithi103.stdout: "session_count": 2, 2024-04-17T06:13:28.073 INFO:teuthology.orchestra.run.smithi103.stdout: "session_add": 5, 2024-04-17T06:13:28.073 INFO:teuthology.orchestra.run.smithi103.stdout: "session_remove": 3, 2024-04-17T06:13:28.073 INFO:teuthology.orchestra.run.smithi103.stdout: "sessions_open": 4, 2024-04-17T06:13:28.073 INFO:teuthology.orchestra.run.smithi103.stdout: "sessions_stale": 0, 2024-04-17T06:13:28.074 INFO:teuthology.orchestra.run.smithi103.stdout: "total_load": 0, 2024-04-17T06:13:28.074 INFO:teuthology.orchestra.run.smithi103.stdout: "average_load": 0, 2024-04-17T06:13:28.074 INFO:teuthology.orchestra.run.smithi103.stdout: "avg_session_uptime": 3402, 2024-04-17T06:13:28.074 INFO:teuthology.orchestra.run.smithi103.stdout: "mdthresh_evicted": 0 2024-04-17T06:13:28.074 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:28.074 INFO:teuthology.orchestra.run.smithi103.stdout: "mempool": { 2024-04-17T06:13:28.074 INFO:teuthology.orchestra.run.smithi103.stdout: "bloom_filter_bytes": 0, 2024-04-17T06:13:28.074 INFO:teuthology.orchestra.run.smithi103.stdout: "bloom_filter_items": 0, 2024-04-17T06:13:28.074 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_alloc_bytes": 0, 2024-04-17T06:13:28.075 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_alloc_items": 0, 2024-04-17T06:13:28.075 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_data_bytes": 0, 2024-04-17T06:13:28.075 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_data_items": 0, 2024-04-17T06:13:28.075 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-17T06:13:28.075 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_onode_items": 0, 2024-04-17T06:13:28.075 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-17T06:13:28.075 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_meta_items": 0, 2024-04-17T06:13:28.075 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_other_bytes": 0, 2024-04-17T06:13:28.075 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_other_items": 0, 2024-04-17T06:13:28.075 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-17T06:13:28.075 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_buffer_items": 0, 2024-04-17T06:13:28.075 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_extent_bytes": 0, 2024-04-17T06:13:28.075 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_extent_items": 0, 2024-04-17T06:13:28.075 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_blob_bytes": 0, 2024-04-17T06:13:28.076 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_blob_items": 0, 2024-04-17T06:13:28.076 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-17T06:13:28.076 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_shared_blob_items": 0, 2024-04-17T06:13:28.076 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-17T06:13:28.076 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_inline_bl_items": 0, 2024-04-17T06:13:28.076 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_fsck_bytes": 0, 2024-04-17T06:13:28.076 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_fsck_items": 0, 2024-04-17T06:13:28.076 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_txc_bytes": 0, 2024-04-17T06:13:28.076 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_txc_items": 0, 2024-04-17T06:13:28.076 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-17T06:13:28.076 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_deferred_items": 0, 2024-04-17T06:13:28.076 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_bytes": 0, 2024-04-17T06:13:28.076 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_items": 0, 2024-04-17T06:13:28.076 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_bytes": 0, 2024-04-17T06:13:28.076 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_items": 0, 2024-04-17T06:13:28.077 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_reader_bytes": 0, 2024-04-17T06:13:28.077 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_reader_items": 0, 2024-04-17T06:13:28.077 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_writer_bytes": 0, 2024-04-17T06:13:28.077 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_writer_items": 0, 2024-04-17T06:13:28.077 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_anon_bytes": 189460, 2024-04-17T06:13:28.077 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_anon_items": 52, 2024-04-17T06:13:28.077 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_meta_bytes": 0, 2024-04-17T06:13:28.077 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_meta_items": 0, 2024-04-17T06:13:28.077 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_bytes": 0, 2024-04-17T06:13:28.077 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_items": 0, 2024-04-17T06:13:28.077 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mapbl_bytes": 0, 2024-04-17T06:13:28.077 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mapbl_items": 0, 2024-04-17T06:13:28.077 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pglog_bytes": 0, 2024-04-17T06:13:28.078 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pglog_items": 0, 2024-04-17T06:13:28.078 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_bytes": 8032, 2024-04-17T06:13:28.078 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_items": 167, 2024-04-17T06:13:28.078 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_mapping_bytes": 0, 2024-04-17T06:13:28.078 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_mapping_items": 0, 2024-04-17T06:13:28.078 INFO:teuthology.orchestra.run.smithi103.stdout: "pgmap_bytes": 0, 2024-04-17T06:13:28.078 INFO:teuthology.orchestra.run.smithi103.stdout: "pgmap_items": 0, 2024-04-17T06:13:28.078 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_co_bytes": 18294, 2024-04-17T06:13:28.078 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_co_items": 141, 2024-04-17T06:13:28.078 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_1_bytes": 0, 2024-04-17T06:13:28.078 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_1_items": 0, 2024-04-17T06:13:28.078 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_2_bytes": 0, 2024-04-17T06:13:28.078 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_2_items": 0 2024-04-17T06:13:28.078 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:28.079 INFO:teuthology.orchestra.run.smithi103.stdout: "objecter": { 2024-04-17T06:13:28.079 INFO:teuthology.orchestra.run.smithi103.stdout: "op_active": 0, 2024-04-17T06:13:28.079 INFO:teuthology.orchestra.run.smithi103.stdout: "op_laggy": 0, 2024-04-17T06:13:28.079 INFO:teuthology.orchestra.run.smithi103.stdout: "op_send": 3073, 2024-04-17T06:13:28.079 INFO:teuthology.orchestra.run.smithi103.stdout: "op_send_bytes": 21, 2024-04-17T06:13:28.079 INFO:teuthology.orchestra.run.smithi103.stdout: "op_resend": 3, 2024-04-17T06:13:28.079 INFO:teuthology.orchestra.run.smithi103.stdout: "op_reply": 3073, 2024-04-17T06:13:28.079 INFO:teuthology.orchestra.run.smithi103.stdout: "op_latency": { 2024-04-17T06:13:28.079 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 3073, 2024-04-17T06:13:28.079 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 1.742967505, 2024-04-17T06:13:28.079 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000567187 2024-04-17T06:13:28.079 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:28.079 INFO:teuthology.orchestra.run.smithi103.stdout: "op_inflight": 0, 2024-04-17T06:13:28.079 INFO:teuthology.orchestra.run.smithi103.stdout: "oplen_avg": { 2024-04-17T06:13:28.080 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 3073, 2024-04-17T06:13:28.080 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 3074 2024-04-17T06:13:28.080 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:28.080 INFO:teuthology.orchestra.run.smithi103.stdout: "op": 3073, 2024-04-17T06:13:28.080 INFO:teuthology.orchestra.run.smithi103.stdout: "op_r": 3073, 2024-04-17T06:13:28.080 INFO:teuthology.orchestra.run.smithi103.stdout: "op_w": 0, 2024-04-17T06:13:28.080 INFO:teuthology.orchestra.run.smithi103.stdout: "op_rmw": 0, 2024-04-17T06:13:28.080 INFO:teuthology.orchestra.run.smithi103.stdout: "op_pg": 0, 2024-04-17T06:13:28.080 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_stat": 1610, 2024-04-17T06:13:28.080 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_create": 0, 2024-04-17T06:13:28.080 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_read": 1461, 2024-04-17T06:13:28.080 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_write": 0, 2024-04-17T06:13:28.080 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_writefull": 0, 2024-04-17T06:13:28.080 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_writesame": 0, 2024-04-17T06:13:28.081 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_append": 0, 2024-04-17T06:13:28.081 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_zero": 0, 2024-04-17T06:13:28.081 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_truncate": 0, 2024-04-17T06:13:28.081 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_delete": 0, 2024-04-17T06:13:28.081 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_mapext": 0, 2024-04-17T06:13:28.081 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_sparse_read": 0, 2024-04-17T06:13:28.081 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_clonerange": 0, 2024-04-17T06:13:28.081 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_getxattr": 1, 2024-04-17T06:13:28.081 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_setxattr": 0, 2024-04-17T06:13:28.081 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_cmpxattr": 0, 2024-04-17T06:13:28.081 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_rmxattr": 0, 2024-04-17T06:13:28.081 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_resetxattrs": 0, 2024-04-17T06:13:28.081 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_call": 0, 2024-04-17T06:13:28.081 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_watch": 0, 2024-04-17T06:13:28.082 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_notify": 0, 2024-04-17T06:13:28.082 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_src_cmpxattr": 0, 2024-04-17T06:13:28.082 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_pgls": 0, 2024-04-17T06:13:28.082 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_pgls_filter": 0, 2024-04-17T06:13:28.082 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_other": 0, 2024-04-17T06:13:28.082 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_active": 0, 2024-04-17T06:13:28.082 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_send": 0, 2024-04-17T06:13:28.082 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_resend": 0, 2024-04-17T06:13:28.082 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_ping": 0, 2024-04-17T06:13:28.082 INFO:teuthology.orchestra.run.smithi103.stdout: "poolop_active": 0, 2024-04-17T06:13:28.082 INFO:teuthology.orchestra.run.smithi103.stdout: "poolop_send": 0, 2024-04-17T06:13:28.082 INFO:teuthology.orchestra.run.smithi103.stdout: "poolop_resend": 0, 2024-04-17T06:13:28.082 INFO:teuthology.orchestra.run.smithi103.stdout: "poolstat_active": 0, 2024-04-17T06:13:28.083 INFO:teuthology.orchestra.run.smithi103.stdout: "poolstat_send": 0, 2024-04-17T06:13:28.083 INFO:teuthology.orchestra.run.smithi103.stdout: "poolstat_resend": 0, 2024-04-17T06:13:28.083 INFO:teuthology.orchestra.run.smithi103.stdout: "statfs_active": 0, 2024-04-17T06:13:28.083 INFO:teuthology.orchestra.run.smithi103.stdout: "statfs_send": 0, 2024-04-17T06:13:28.083 INFO:teuthology.orchestra.run.smithi103.stdout: "statfs_resend": 0, 2024-04-17T06:13:28.083 INFO:teuthology.orchestra.run.smithi103.stdout: "command_active": 0, 2024-04-17T06:13:28.083 INFO:teuthology.orchestra.run.smithi103.stdout: "command_send": 0, 2024-04-17T06:13:28.083 INFO:teuthology.orchestra.run.smithi103.stdout: "command_resend": 0, 2024-04-17T06:13:28.083 INFO:teuthology.orchestra.run.smithi103.stdout: "map_epoch": 106, 2024-04-17T06:13:28.083 INFO:teuthology.orchestra.run.smithi103.stdout: "map_full": 0, 2024-04-17T06:13:28.083 INFO:teuthology.orchestra.run.smithi103.stdout: "map_inc": 20, 2024-04-17T06:13:28.083 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_sessions": 5, 2024-04-17T06:13:28.083 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_session_open": 5, 2024-04-17T06:13:28.083 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_session_close": 0, 2024-04-17T06:13:28.083 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_laggy": 0, 2024-04-17T06:13:28.084 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_wr": 0, 2024-04-17T06:13:28.084 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_rd": 2, 2024-04-17T06:13:28.084 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_del": 0 2024-04-17T06:13:28.084 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:28.084 INFO:teuthology.orchestra.run.smithi103.stdout: "oft": { 2024-04-17T06:13:28.084 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_objs": 0, 2024-04-17T06:13:28.084 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_kv_pairs": 0, 2024-04-17T06:13:28.084 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_updates": 0, 2024-04-17T06:13:28.084 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_removes": 0 2024-04-17T06:13:28.084 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:28.084 INFO:teuthology.orchestra.run.smithi103.stdout: "osd-slow-ops": { 2024-04-17T06:13:28.084 INFO:teuthology.orchestra.run.smithi103.stdout: "slow_ops_count": 0 2024-04-17T06:13:28.084 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:28.085 INFO:teuthology.orchestra.run.smithi103.stdout: "purge_queue": { 2024-04-17T06:13:28.085 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing_ops": 0, 2024-04-17T06:13:28.085 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing_ops_high_water": 0, 2024-04-17T06:13:28.085 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing": 0, 2024-04-17T06:13:28.085 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing_high_water": 0, 2024-04-17T06:13:28.085 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executed_ops": 0, 2024-04-17T06:13:28.085 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executed": 0, 2024-04-17T06:13:28.085 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_item_in_journal": 0 2024-04-17T06:13:28.085 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:28.085 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-17T06:13:28.085 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-04-17T06:13:28.086 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 104857600, 2024-04-17T06:13:28.086 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-04-17T06:13:28.086 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 3361, 2024-04-17T06:13:28.086 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 4480270, 2024-04-17T06:13:28.086 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-04-17T06:13:28.086 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 3361, 2024-04-17T06:13:28.086 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 0, 2024-04-17T06:13:28.086 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 0, 2024-04-17T06:13:28.086 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 3361, 2024-04-17T06:13:28.086 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 4480270, 2024-04-17T06:13:28.086 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-04-17T06:13:28.086 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:28.086 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:28.086 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:28.087 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-04-17T06:13:28.087 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:28.087 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-objecter_bytes": { 2024-04-17T06:13:28.087 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-04-17T06:13:28.087 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 104857600, 2024-04-17T06:13:28.087 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-04-17T06:13:28.087 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 0, 2024-04-17T06:13:28.087 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 0, 2024-04-17T06:13:28.087 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-04-17T06:13:28.087 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 0, 2024-04-17T06:13:28.087 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 3073, 2024-04-17T06:13:28.087 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 2266901, 2024-04-17T06:13:28.087 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 3073, 2024-04-17T06:13:28.087 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 2266901, 2024-04-17T06:13:28.088 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-04-17T06:13:28.088 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:28.088 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:28.088 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:28.088 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-04-17T06:13:28.088 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:28.088 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-objecter_ops": { 2024-04-17T06:13:28.088 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-04-17T06:13:28.088 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 1024, 2024-04-17T06:13:28.088 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-04-17T06:13:28.088 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 0, 2024-04-17T06:13:28.088 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 0, 2024-04-17T06:13:28.088 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-04-17T06:13:28.088 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 0, 2024-04-17T06:13:28.088 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 3073, 2024-04-17T06:13:28.089 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 3073, 2024-04-17T06:13:28.089 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 3073, 2024-04-17T06:13:28.089 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 3073, 2024-04-17T06:13:28.089 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-04-17T06:13:28.089 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:28.089 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:28.089 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:28.089 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-04-17T06:13:28.089 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:28.089 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-write_buf_throttle": { 2024-04-17T06:13:28.089 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-04-17T06:13:28.089 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 3758096384, 2024-04-17T06:13:28.089 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-04-17T06:13:28.089 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 0, 2024-04-17T06:13:28.089 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 0, 2024-04-17T06:13:28.090 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-04-17T06:13:28.090 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 0, 2024-04-17T06:13:28.090 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 0, 2024-04-17T06:13:28.090 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 0, 2024-04-17T06:13:28.090 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 0, 2024-04-17T06:13:28.090 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 0, 2024-04-17T06:13:28.090 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-04-17T06:13:28.090 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:28.090 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:28.090 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:28.090 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-04-17T06:13:28.090 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:28.090 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-write_buf_throttle-0x55dcaa64e6e0": { 2024-04-17T06:13:28.090 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-04-17T06:13:28.091 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 3758096384, 2024-04-17T06:13:28.091 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-04-17T06:13:28.091 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 0, 2024-04-17T06:13:28.091 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 0, 2024-04-17T06:13:28.091 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-04-17T06:13:28.091 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 0, 2024-04-17T06:13:28.091 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 0, 2024-04-17T06:13:28.091 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 0, 2024-04-17T06:13:28.091 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 0, 2024-04-17T06:13:28.092 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 0, 2024-04-17T06:13:28.092 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-04-17T06:13:28.092 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:28.092 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:28.092 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:28.092 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-04-17T06:13:28.092 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-04-17T06:13:28.092 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-04-17T06:13:28.092 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:13:27 smithi103 ceph-mon[36243]: pgmap v384: 97 pgs: 97 active+clean; 5.9 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-04-17T06:13:28.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:13:27 smithi098 ceph-mon[36516]: pgmap v384: 97 pgs: 97 active+clean; 5.9 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-04-17T06:13:28.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:13:27 smithi047 ceph-mon[32223]: pgmap v384: 97 pgs: 97 active+clean; 5.9 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-04-17T06:13:28.876 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-04-17T06:13:28.876 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-04-17T06:13:28.876 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=0 2024-04-17T06:13:28.876 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-04-17T06:13:28.876 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-04-17T06:13:28.876 DEBUG:tasks.check_counter:Getting stats from i 2024-04-17T06:13:28.876 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 565db2d8-fc7e-11ee-bc91-c7b262605968 -- ceph daemon mds.i perf dump 2024-04-17T06:13:29.119 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/mon.c/config 2024-04-17T06:13:30.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:13:29 smithi098 ceph-mon[36516]: pgmap v385: 97 pgs: 97 active+clean; 5.9 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-17T06:13:30.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:13:29 smithi047 ceph-mon[32223]: pgmap v385: 97 pgs: 97 active+clean; 5.9 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-17T06:13:30.249 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:13:29 smithi103 ceph-mon[36243]: pgmap v385: 97 pgs: 97 active+clean; 5.9 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-17T06:13:30.780 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-04-17T06:13:30.780 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-0": { 2024-04-17T06:13:30.780 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 2487, 2024-04-17T06:13:30.780 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 2662, 2024-04-17T06:13:30.780 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 711837, 2024-04-17T06:13:30.780 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 3384566, 2024-04-17T06:13:30.781 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 9, 2024-04-17T06:13:30.781 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 4, 2024-04-17T06:13:30.781 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.234820566, 2024-04-17T06:13:30.781 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.080434368, 2024-04-17T06:13:30.781 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.081019084, 2024-04-17T06:13:30.781 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.061389733, 2024-04-17T06:13:30.781 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T06:13:30.781 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 2660, 2024-04-17T06:13:30.781 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.094638625, 2024-04-17T06:13:30.781 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000035578 2024-04-17T06:13:30.781 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:30.781 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-04-17T06:13:30.781 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 1261, 2024-04-17T06:13:30.782 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000350399, 2024-04-17T06:13:30.782 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000277 2024-04-17T06:13:30.782 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:30.782 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 1728, 2024-04-17T06:13:30.782 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 6432 2024-04-17T06:13:30.782 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:30.782 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-1": { 2024-04-17T06:13:30.782 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 636, 2024-04-17T06:13:30.783 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 669, 2024-04-17T06:13:30.783 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 161908, 2024-04-17T06:13:30.783 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 370064, 2024-04-17T06:13:30.783 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 11, 2024-04-17T06:13:30.783 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 6, 2024-04-17T06:13:30.783 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.058153528, 2024-04-17T06:13:30.783 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.021638741, 2024-04-17T06:13:30.783 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.016649956, 2024-04-17T06:13:30.783 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.014371035, 2024-04-17T06:13:30.783 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T06:13:30.783 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 666, 2024-04-17T06:13:30.784 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.029577109, 2024-04-17T06:13:30.784 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000044410 2024-04-17T06:13:30.784 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:30.784 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-04-17T06:13:30.784 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 22, 2024-04-17T06:13:30.784 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000002925, 2024-04-17T06:13:30.784 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000132 2024-04-17T06:13:30.784 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:30.784 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-04-17T06:13:30.784 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 133664 2024-04-17T06:13:30.784 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:30.784 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-2": { 2024-04-17T06:13:30.784 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 1662, 2024-04-17T06:13:30.785 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 4208, 2024-04-17T06:13:30.785 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 2115110, 2024-04-17T06:13:30.785 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 1863251, 2024-04-17T06:13:30.785 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 12, 2024-04-17T06:13:30.785 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 4, 2024-04-17T06:13:30.785 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.245540547, 2024-04-17T06:13:30.785 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.114196468, 2024-04-17T06:13:30.785 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.142131394, 2024-04-17T06:13:30.785 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.006184667, 2024-04-17T06:13:30.785 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T06:13:30.785 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 4208, 2024-04-17T06:13:30.786 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.131916095, 2024-04-17T06:13:30.786 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000031348 2024-04-17T06:13:30.786 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:30.786 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-04-17T06:13:30.786 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 3887, 2024-04-17T06:13:30.786 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.001051060, 2024-04-17T06:13:30.786 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000270 2024-04-17T06:13:30.786 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:30.786 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 1694880, 2024-04-17T06:13:30.786 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 505840 2024-04-17T06:13:30.786 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:30.786 INFO:teuthology.orchestra.run.smithi103.stdout: "cct": { 2024-04-17T06:13:30.787 INFO:teuthology.orchestra.run.smithi103.stdout: "total_workers": 1, 2024-04-17T06:13:30.787 INFO:teuthology.orchestra.run.smithi103.stdout: "unhealthy_workers": 0 2024-04-17T06:13:30.787 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:30.787 INFO:teuthology.orchestra.run.smithi103.stdout: "finisher-MDSRank": { 2024-04-17T06:13:30.787 INFO:teuthology.orchestra.run.smithi103.stdout: "queue_len": 0, 2024-04-17T06:13:30.787 INFO:teuthology.orchestra.run.smithi103.stdout: "complete_latency": { 2024-04-17T06:13:30.787 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 4412, 2024-04-17T06:13:30.787 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.892007489, 2024-04-17T06:13:30.787 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000202177 2024-04-17T06:13:30.787 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-04-17T06:13:30.787 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:30.787 INFO:teuthology.orchestra.run.smithi103.stdout: "finisher-PurgeQueue": { 2024-04-17T06:13:30.787 INFO:teuthology.orchestra.run.smithi103.stdout: "queue_len": 0, 2024-04-17T06:13:30.787 INFO:teuthology.orchestra.run.smithi103.stdout: "complete_latency": { 2024-04-17T06:13:30.788 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 16, 2024-04-17T06:13:30.788 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000258968, 2024-04-17T06:13:30.788 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000016185 2024-04-17T06:13:30.788 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-04-17T06:13:30.788 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:30.788 INFO:teuthology.orchestra.run.smithi103.stdout: "mds": { 2024-04-17T06:13:30.788 INFO:teuthology.orchestra.run.smithi103.stdout: "request": 605, 2024-04-17T06:13:30.788 INFO:teuthology.orchestra.run.smithi103.stdout: "reply": 604, 2024-04-17T06:13:30.788 INFO:teuthology.orchestra.run.smithi103.stdout: "reply_latency": { 2024-04-17T06:13:30.788 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 604, 2024-04-17T06:13:30.788 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 2.473183743, 2024-04-17T06:13:30.788 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.004094675 2024-04-17T06:13:30.788 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:30.789 INFO:teuthology.orchestra.run.smithi103.stdout: "slow_reply": 0, 2024-04-17T06:13:30.789 INFO:teuthology.orchestra.run.smithi103.stdout: "forward": 1, 2024-04-17T06:13:30.789 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_fetch_complete": 0, 2024-04-17T06:13:30.789 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_fetch_keys": 0, 2024-04-17T06:13:30.789 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_commit": 11, 2024-04-17T06:13:30.789 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_split": 0, 2024-04-17T06:13:30.789 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_merge": 0, 2024-04-17T06:13:30.789 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes": 15, 2024-04-17T06:13:30.789 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_top": 0, 2024-04-17T06:13:30.789 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_bottom": 2, 2024-04-17T06:13:30.789 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_pin_tail": 13, 2024-04-17T06:13:30.790 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_pinned": 13, 2024-04-17T06:13:30.790 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_expired": 17, 2024-04-17T06:13:30.790 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_with_caps": 0, 2024-04-17T06:13:30.790 INFO:teuthology.orchestra.run.smithi103.stdout: "caps": 0, 2024-04-17T06:13:30.790 INFO:teuthology.orchestra.run.smithi103.stdout: "subtrees": 3, 2024-04-17T06:13:30.790 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse": 614, 2024-04-17T06:13:30.790 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_hit": 606, 2024-04-17T06:13:30.790 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_forward": 0, 2024-04-17T06:13:30.790 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_discover": 3, 2024-04-17T06:13:30.790 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_dir_fetch": 0, 2024-04-17T06:13:30.790 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_remote_ino": 0, 2024-04-17T06:13:30.790 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_lock": 0, 2024-04-17T06:13:30.791 INFO:teuthology.orchestra.run.smithi103.stdout: "load_cent": 0, 2024-04-17T06:13:30.791 INFO:teuthology.orchestra.run.smithi103.stdout: "q": 0, 2024-04-17T06:13:30.791 INFO:teuthology.orchestra.run.smithi103.stdout: "exported": 1, 2024-04-17T06:13:30.791 INFO:teuthology.orchestra.run.smithi103.stdout: "exported_inodes": 11, 2024-04-17T06:13:30.791 INFO:teuthology.orchestra.run.smithi103.stdout: "imported": 1, 2024-04-17T06:13:30.791 INFO:teuthology.orchestra.run.smithi103.stdout: "imported_inodes": 1, 2024-04-17T06:13:30.791 INFO:teuthology.orchestra.run.smithi103.stdout: "openino_dir_fetch": 0, 2024-04-17T06:13:30.791 INFO:teuthology.orchestra.run.smithi103.stdout: "openino_backtrace_fetch": 0, 2024-04-17T06:13:30.791 INFO:teuthology.orchestra.run.smithi103.stdout: "openino_peer_discover": 1, 2024-04-17T06:13:30.791 INFO:teuthology.orchestra.run.smithi103.stdout: "root_rfiles": 4, 2024-04-17T06:13:30.791 INFO:teuthology.orchestra.run.smithi103.stdout: "root_rbytes": 234, 2024-04-17T06:13:30.791 INFO:teuthology.orchestra.run.smithi103.stdout: "root_rsnaps": 0, 2024-04-17T06:13:30.791 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_backtrace_fetch": 0, 2024-04-17T06:13:30.791 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_set_tag": 0, 2024-04-17T06:13:30.792 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_backtrace_repaired": 0, 2024-04-17T06:13:30.792 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_inotable_repaired": 0, 2024-04-17T06:13:30.792 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_dir_inodes": 0, 2024-04-17T06:13:30.792 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_dir_base_inodes": 0, 2024-04-17T06:13:30.792 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_dirfrag_rstats": 0, 2024-04-17T06:13:30.792 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_file_inodes": 0, 2024-04-17T06:13:30.792 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_inode_file_caps": 0, 2024-04-17T06:13:30.792 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_revoke": 604, 2024-04-17T06:13:30.793 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_grant": 601, 2024-04-17T06:13:30.793 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_trunc": 598, 2024-04-17T06:13:30.793 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-17T06:13:30.793 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-17T06:13:30.793 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_caps": 671, 2024-04-17T06:13:30.793 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_caps_dirty": 664, 2024-04-17T06:13:30.793 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_cap_release": 2, 2024-04-17T06:13:30.794 INFO:teuthology.orchestra.run.smithi103.stdout: "process_request_cap_release": 601 2024-04-17T06:13:30.794 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:30.794 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_cache": { 2024-04-17T06:13:30.794 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_update": 0, 2024-04-17T06:13:30.794 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_update_receipt": 0, 2024-04-17T06:13:30.794 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_try_discover": 0, 2024-04-17T06:13:30.795 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_send_discover": 4, 2024-04-17T06:13:30.795 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_handle_discover": 0, 2024-04-17T06:13:30.795 INFO:teuthology.orchestra.run.smithi103.stdout: "num_strays": 0, 2024-04-17T06:13:30.795 INFO:teuthology.orchestra.run.smithi103.stdout: "num_strays_delayed": 0, 2024-04-17T06:13:30.795 INFO:teuthology.orchestra.run.smithi103.stdout: "num_strays_enqueuing": 0, 2024-04-17T06:13:30.795 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_created": 1, 2024-04-17T06:13:30.795 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_enqueued": 1, 2024-04-17T06:13:30.795 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_reintegrated": 0, 2024-04-17T06:13:30.795 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_migrated": 0, 2024-04-17T06:13:30.795 INFO:teuthology.orchestra.run.smithi103.stdout: "num_recovering_processing": 0, 2024-04-17T06:13:30.795 INFO:teuthology.orchestra.run.smithi103.stdout: "num_recovering_enqueued": 0, 2024-04-17T06:13:30.796 INFO:teuthology.orchestra.run.smithi103.stdout: "num_recovering_prioritized": 0, 2024-04-17T06:13:30.796 INFO:teuthology.orchestra.run.smithi103.stdout: "recovery_started": 0, 2024-04-17T06:13:30.796 INFO:teuthology.orchestra.run.smithi103.stdout: "recovery_completed": 0, 2024-04-17T06:13:30.796 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_quiesce_path": 0, 2024-04-17T06:13:30.796 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_quiesce_inode": 0, 2024-04-17T06:13:30.796 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_enqueue_scrub": 0, 2024-04-17T06:13:30.796 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_exportdir": 1, 2024-04-17T06:13:30.796 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_flush": 0, 2024-04-17T06:13:30.796 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_fragmentdir": 0, 2024-04-17T06:13:30.796 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_fragstats": 0, 2024-04-17T06:13:30.796 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_inodestats": 0 2024-04-17T06:13:30.797 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:30.797 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_log": { 2024-04-17T06:13:30.797 INFO:teuthology.orchestra.run.smithi103.stdout: "evlrg": 0, 2024-04-17T06:13:30.797 INFO:teuthology.orchestra.run.smithi103.stdout: "evadd": 1947, 2024-04-17T06:13:30.797 INFO:teuthology.orchestra.run.smithi103.stdout: "evex": 0, 2024-04-17T06:13:30.797 INFO:teuthology.orchestra.run.smithi103.stdout: "evtrm": 0, 2024-04-17T06:13:30.797 INFO:teuthology.orchestra.run.smithi103.stdout: "ev": 1947, 2024-04-17T06:13:30.797 INFO:teuthology.orchestra.run.smithi103.stdout: "evexg": 0, 2024-04-17T06:13:30.797 INFO:teuthology.orchestra.run.smithi103.stdout: "evexd": 0, 2024-04-17T06:13:30.797 INFO:teuthology.orchestra.run.smithi103.stdout: "segadd": 3, 2024-04-17T06:13:30.797 INFO:teuthology.orchestra.run.smithi103.stdout: "segex": 0, 2024-04-17T06:13:30.797 INFO:teuthology.orchestra.run.smithi103.stdout: "segtrm": 0, 2024-04-17T06:13:30.797 INFO:teuthology.orchestra.run.smithi103.stdout: "seg": 3, 2024-04-17T06:13:30.798 INFO:teuthology.orchestra.run.smithi103.stdout: "segmjr": 2, 2024-04-17T06:13:30.798 INFO:teuthology.orchestra.run.smithi103.stdout: "segexg": 0, 2024-04-17T06:13:30.798 INFO:teuthology.orchestra.run.smithi103.stdout: "segexd": 0, 2024-04-17T06:13:30.798 INFO:teuthology.orchestra.run.smithi103.stdout: "expos": 4194304, 2024-04-17T06:13:30.798 INFO:teuthology.orchestra.run.smithi103.stdout: "wrpos": 6466454, 2024-04-17T06:13:30.798 INFO:teuthology.orchestra.run.smithi103.stdout: "rdpos": 0, 2024-04-17T06:13:30.798 INFO:teuthology.orchestra.run.smithi103.stdout: "jlat": { 2024-04-17T06:13:30.798 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 1867, 2024-04-17T06:13:30.798 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 3.692436008, 2024-04-17T06:13:30.798 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.001977737 2024-04-17T06:13:30.798 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:30.798 INFO:teuthology.orchestra.run.smithi103.stdout: "replayed": 0 2024-04-17T06:13:30.798 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:30.799 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_mem": { 2024-04-17T06:13:30.799 INFO:teuthology.orchestra.run.smithi103.stdout: "ino": 19, 2024-04-17T06:13:30.799 INFO:teuthology.orchestra.run.smithi103.stdout: "ino+": 32, 2024-04-17T06:13:30.799 INFO:teuthology.orchestra.run.smithi103.stdout: "ino-": 13, 2024-04-17T06:13:30.799 INFO:teuthology.orchestra.run.smithi103.stdout: "dir": 17, 2024-04-17T06:13:30.799 INFO:teuthology.orchestra.run.smithi103.stdout: "dir+": 21, 2024-04-17T06:13:30.799 INFO:teuthology.orchestra.run.smithi103.stdout: "dir-": 4, 2024-04-17T06:13:30.799 INFO:teuthology.orchestra.run.smithi103.stdout: "dn": 15, 2024-04-17T06:13:30.799 INFO:teuthology.orchestra.run.smithi103.stdout: "dn+": 32, 2024-04-17T06:13:30.799 INFO:teuthology.orchestra.run.smithi103.stdout: "dn-": 17, 2024-04-17T06:13:30.799 INFO:teuthology.orchestra.run.smithi103.stdout: "cap": 0, 2024-04-17T06:13:30.799 INFO:teuthology.orchestra.run.smithi103.stdout: "cap+": 4, 2024-04-17T06:13:30.799 INFO:teuthology.orchestra.run.smithi103.stdout: "cap-": 4, 2024-04-17T06:13:30.800 INFO:teuthology.orchestra.run.smithi103.stdout: "rss": 51748, 2024-04-17T06:13:30.800 INFO:teuthology.orchestra.run.smithi103.stdout: "heap": 223608 2024-04-17T06:13:30.800 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:30.800 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_server": { 2024-04-17T06:13:30.800 INFO:teuthology.orchestra.run.smithi103.stdout: "dispatch_client_request": 1812, 2024-04-17T06:13:30.800 INFO:teuthology.orchestra.run.smithi103.stdout: "dispatch_server_request": 1, 2024-04-17T06:13:30.800 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_request": 605, 2024-04-17T06:13:30.800 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_session": 31, 2024-04-17T06:13:30.800 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_peer_request": 6, 2024-04-17T06:13:30.800 INFO:teuthology.orchestra.run.smithi103.stdout: "req_create_latency": { 2024-04-17T06:13:30.800 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 1, 2024-04-17T06:13:30.800 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000408680, 2024-04-17T06:13:30.800 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000408680 2024-04-17T06:13:30.800 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:30.800 INFO:teuthology.orchestra.run.smithi103.stdout: "req_getattr_latency": { 2024-04-17T06:13:30.800 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 1, 2024-04-17T06:13:30.801 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000887422, 2024-04-17T06:13:30.801 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000887422 2024-04-17T06:13:30.801 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:30.801 INFO:teuthology.orchestra.run.smithi103.stdout: "req_getfilelock_latency": { 2024-04-17T06:13:30.801 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:30.801 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:30.801 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:30.801 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:30.801 INFO:teuthology.orchestra.run.smithi103.stdout: "req_link_latency": { 2024-04-17T06:13:30.802 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:30.802 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:30.802 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:30.802 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:30.802 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookup_latency": { 2024-04-17T06:13:30.802 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 2, 2024-04-17T06:13:30.802 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000879966, 2024-04-17T06:13:30.802 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000439983 2024-04-17T06:13:30.802 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:30.803 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookuphash_latency": { 2024-04-17T06:13:30.803 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:30.803 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:30.803 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:30.803 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:30.803 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupino_latency": { 2024-04-17T06:13:30.803 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:30.803 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:30.803 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:30.804 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:30.804 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupname_latency": { 2024-04-17T06:13:30.804 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:30.804 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:30.804 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:30.804 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:30.804 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupparent_latency": { 2024-04-17T06:13:30.805 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:30.805 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:30.806 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:30.806 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:30.806 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupsnap_latency": { 2024-04-17T06:13:30.806 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:30.806 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:30.806 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:30.806 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:30.806 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lssnap_latency": { 2024-04-17T06:13:30.807 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:30.807 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:30.807 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:30.807 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:30.807 INFO:teuthology.orchestra.run.smithi103.stdout: "req_mkdir_latency": { 2024-04-17T06:13:30.808 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:30.808 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:30.808 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:30.808 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:30.808 INFO:teuthology.orchestra.run.smithi103.stdout: "req_mknod_latency": { 2024-04-17T06:13:30.808 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:30.808 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:30.808 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:30.808 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:30.808 INFO:teuthology.orchestra.run.smithi103.stdout: "req_mksnap_latency": { 2024-04-17T06:13:30.808 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:30.808 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:30.808 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:30.808 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:30.809 INFO:teuthology.orchestra.run.smithi103.stdout: "req_open_latency": { 2024-04-17T06:13:30.809 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:30.809 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:30.809 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:30.809 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:30.809 INFO:teuthology.orchestra.run.smithi103.stdout: "req_readdir_latency": { 2024-04-17T06:13:30.809 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 1, 2024-04-17T06:13:30.809 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.001353851, 2024-04-17T06:13:30.809 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.001353851 2024-04-17T06:13:30.809 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:30.809 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rename_latency": { 2024-04-17T06:13:30.809 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:30.809 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:30.809 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:30.809 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:30.810 INFO:teuthology.orchestra.run.smithi103.stdout: "req_renamesnap_latency": { 2024-04-17T06:13:30.810 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:30.810 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:30.810 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:30.810 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:30.810 INFO:teuthology.orchestra.run.smithi103.stdout: "req_snapdiff_latency": { 2024-04-17T06:13:30.810 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:30.810 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:30.810 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:30.810 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:30.811 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rmdir_latency": { 2024-04-17T06:13:30.811 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:30.811 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:30.811 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:30.811 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:30.811 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rmsnap_latency": { 2024-04-17T06:13:30.811 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:30.811 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:30.811 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:30.811 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:30.811 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rmxattr_latency": { 2024-04-17T06:13:30.811 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:30.812 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:30.812 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:30.812 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:30.812 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setattr_latency": { 2024-04-17T06:13:30.812 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 598, 2024-04-17T06:13:30.812 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 2.466941928, 2024-04-17T06:13:30.812 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.004125320 2024-04-17T06:13:30.812 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:30.812 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setdirlayout_latency": { 2024-04-17T06:13:30.813 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:30.813 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:30.813 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:30.813 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:30.813 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setfilelock_latency": { 2024-04-17T06:13:30.813 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:30.813 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:30.813 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:30.813 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:30.814 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setlayout_latency": { 2024-04-17T06:13:30.814 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:30.814 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:30.814 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:30.814 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:30.814 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setxattr_latency": { 2024-04-17T06:13:30.814 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:30.814 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:30.814 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:30.814 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:30.815 INFO:teuthology.orchestra.run.smithi103.stdout: "req_symlink_latency": { 2024-04-17T06:13:30.815 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:30.815 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:30.815 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:30.815 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:30.815 INFO:teuthology.orchestra.run.smithi103.stdout: "req_unlink_latency": { 2024-04-17T06:13:30.815 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 1, 2024-04-17T06:13:30.815 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.002711896, 2024-04-17T06:13:30.815 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.002711896 2024-04-17T06:13:30.815 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:30.815 INFO:teuthology.orchestra.run.smithi103.stdout: "cap_revoke_eviction": 0, 2024-04-17T06:13:30.815 INFO:teuthology.orchestra.run.smithi103.stdout: "cap_acquisition_throttle": 0, 2024-04-17T06:13:30.815 INFO:teuthology.orchestra.run.smithi103.stdout: "req_getvxattr_latency": { 2024-04-17T06:13:30.816 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:30.816 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:30.816 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:30.816 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-04-17T06:13:30.816 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:30.816 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_sessions": { 2024-04-17T06:13:30.816 INFO:teuthology.orchestra.run.smithi103.stdout: "session_count": 2, 2024-04-17T06:13:30.816 INFO:teuthology.orchestra.run.smithi103.stdout: "session_add": 5, 2024-04-17T06:13:30.816 INFO:teuthology.orchestra.run.smithi103.stdout: "session_remove": 3, 2024-04-17T06:13:30.816 INFO:teuthology.orchestra.run.smithi103.stdout: "sessions_open": 2, 2024-04-17T06:13:30.816 INFO:teuthology.orchestra.run.smithi103.stdout: "sessions_stale": 0, 2024-04-17T06:13:30.817 INFO:teuthology.orchestra.run.smithi103.stdout: "total_load": 87, 2024-04-17T06:13:30.817 INFO:teuthology.orchestra.run.smithi103.stdout: "average_load": 43, 2024-04-17T06:13:30.817 INFO:teuthology.orchestra.run.smithi103.stdout: "avg_session_uptime": 709, 2024-04-17T06:13:30.817 INFO:teuthology.orchestra.run.smithi103.stdout: "mdthresh_evicted": 0 2024-04-17T06:13:30.817 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:30.817 INFO:teuthology.orchestra.run.smithi103.stdout: "mempool": { 2024-04-17T06:13:30.817 INFO:teuthology.orchestra.run.smithi103.stdout: "bloom_filter_bytes": 0, 2024-04-17T06:13:30.817 INFO:teuthology.orchestra.run.smithi103.stdout: "bloom_filter_items": 0, 2024-04-17T06:13:30.818 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_alloc_bytes": 0, 2024-04-17T06:13:30.818 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_alloc_items": 0, 2024-04-17T06:13:30.818 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_data_bytes": 0, 2024-04-17T06:13:30.818 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_data_items": 0, 2024-04-17T06:13:30.818 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-17T06:13:30.818 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_onode_items": 0, 2024-04-17T06:13:30.818 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-17T06:13:30.818 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_meta_items": 0, 2024-04-17T06:13:30.818 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_other_bytes": 0, 2024-04-17T06:13:30.819 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_other_items": 0, 2024-04-17T06:13:30.819 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-17T06:13:30.819 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_buffer_items": 0, 2024-04-17T06:13:30.819 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_extent_bytes": 0, 2024-04-17T06:13:30.819 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_extent_items": 0, 2024-04-17T06:13:30.819 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_blob_bytes": 0, 2024-04-17T06:13:30.819 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_blob_items": 0, 2024-04-17T06:13:30.819 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-17T06:13:30.819 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_shared_blob_items": 0, 2024-04-17T06:13:30.819 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-17T06:13:30.819 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_inline_bl_items": 0, 2024-04-17T06:13:30.819 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_fsck_bytes": 0, 2024-04-17T06:13:30.819 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_fsck_items": 0, 2024-04-17T06:13:30.820 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_txc_bytes": 0, 2024-04-17T06:13:30.820 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_txc_items": 0, 2024-04-17T06:13:30.820 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-17T06:13:30.820 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_deferred_items": 0, 2024-04-17T06:13:30.820 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_bytes": 0, 2024-04-17T06:13:30.820 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_items": 0, 2024-04-17T06:13:30.820 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_bytes": 0, 2024-04-17T06:13:30.820 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_items": 0, 2024-04-17T06:13:30.820 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_reader_bytes": 0, 2024-04-17T06:13:30.820 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_reader_items": 0, 2024-04-17T06:13:30.820 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_writer_bytes": 0, 2024-04-17T06:13:30.820 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_writer_items": 0, 2024-04-17T06:13:30.820 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_anon_bytes": 315124, 2024-04-17T06:13:30.820 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_anon_items": 118, 2024-04-17T06:13:30.820 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_meta_bytes": 0, 2024-04-17T06:13:30.821 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_meta_items": 0, 2024-04-17T06:13:30.821 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_bytes": 0, 2024-04-17T06:13:30.821 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_items": 0, 2024-04-17T06:13:30.821 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mapbl_bytes": 0, 2024-04-17T06:13:30.821 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mapbl_items": 0, 2024-04-17T06:13:30.821 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pglog_bytes": 0, 2024-04-17T06:13:30.821 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pglog_items": 0, 2024-04-17T06:13:30.821 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_bytes": 8032, 2024-04-17T06:13:30.821 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_items": 167, 2024-04-17T06:13:30.821 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_mapping_bytes": 0, 2024-04-17T06:13:30.821 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_mapping_items": 0, 2024-04-17T06:13:30.821 INFO:teuthology.orchestra.run.smithi103.stdout: "pgmap_bytes": 0, 2024-04-17T06:13:30.821 INFO:teuthology.orchestra.run.smithi103.stdout: "pgmap_items": 0, 2024-04-17T06:13:30.821 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_co_bytes": 84515, 2024-04-17T06:13:30.822 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_co_items": 501, 2024-04-17T06:13:30.822 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_1_bytes": 0, 2024-04-17T06:13:30.822 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_1_items": 0, 2024-04-17T06:13:30.822 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_2_bytes": 0, 2024-04-17T06:13:30.822 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_2_items": 0 2024-04-17T06:13:30.822 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:30.822 INFO:teuthology.orchestra.run.smithi103.stdout: "objecter": { 2024-04-17T06:13:30.822 INFO:teuthology.orchestra.run.smithi103.stdout: "op_active": 0, 2024-04-17T06:13:30.822 INFO:teuthology.orchestra.run.smithi103.stdout: "op_laggy": 0, 2024-04-17T06:13:30.822 INFO:teuthology.orchestra.run.smithi103.stdout: "op_send": 2562, 2024-04-17T06:13:30.822 INFO:teuthology.orchestra.run.smithi103.stdout: "op_send_bytes": 2287390, 2024-04-17T06:13:30.822 INFO:teuthology.orchestra.run.smithi103.stdout: "op_resend": 33, 2024-04-17T06:13:30.822 INFO:teuthology.orchestra.run.smithi103.stdout: "op_reply": 2562, 2024-04-17T06:13:30.822 INFO:teuthology.orchestra.run.smithi103.stdout: "op_latency": { 2024-04-17T06:13:30.823 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 2562, 2024-04-17T06:13:30.823 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 5.040905953, 2024-04-17T06:13:30.823 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.001967566 2024-04-17T06:13:30.824 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:30.824 INFO:teuthology.orchestra.run.smithi103.stdout: "op_inflight": 0, 2024-04-17T06:13:30.824 INFO:teuthology.orchestra.run.smithi103.stdout: "oplen_avg": { 2024-04-17T06:13:30.824 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 2562, 2024-04-17T06:13:30.824 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 2591 2024-04-17T06:13:30.824 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:30.824 INFO:teuthology.orchestra.run.smithi103.stdout: "op": 2562, 2024-04-17T06:13:30.824 INFO:teuthology.orchestra.run.smithi103.stdout: "op_r": 1, 2024-04-17T06:13:30.824 INFO:teuthology.orchestra.run.smithi103.stdout: "op_w": 2561, 2024-04-17T06:13:30.824 INFO:teuthology.orchestra.run.smithi103.stdout: "op_rmw": 0, 2024-04-17T06:13:30.824 INFO:teuthology.orchestra.run.smithi103.stdout: "op_pg": 0, 2024-04-17T06:13:30.824 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_stat": 0, 2024-04-17T06:13:30.825 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_create": 10, 2024-04-17T06:13:30.825 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_read": 1, 2024-04-17T06:13:30.825 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_write": 1868, 2024-04-17T06:13:30.825 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_writefull": 52, 2024-04-17T06:13:30.825 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_writesame": 0, 2024-04-17T06:13:30.825 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_append": 0, 2024-04-17T06:13:30.825 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_zero": 0, 2024-04-17T06:13:30.825 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_truncate": 0, 2024-04-17T06:13:30.825 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_delete": 13, 2024-04-17T06:13:30.825 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_mapext": 0, 2024-04-17T06:13:30.825 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_sparse_read": 0, 2024-04-17T06:13:30.825 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_clonerange": 0, 2024-04-17T06:13:30.825 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_getxattr": 0, 2024-04-17T06:13:30.825 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_setxattr": 20, 2024-04-17T06:13:30.825 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_cmpxattr": 0, 2024-04-17T06:13:30.826 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_rmxattr": 0, 2024-04-17T06:13:30.826 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_resetxattrs": 0, 2024-04-17T06:13:30.826 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_call": 0, 2024-04-17T06:13:30.826 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_watch": 0, 2024-04-17T06:13:30.826 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_notify": 0, 2024-04-17T06:13:30.826 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_src_cmpxattr": 0, 2024-04-17T06:13:30.826 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_pgls": 0, 2024-04-17T06:13:30.826 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_pgls_filter": 0, 2024-04-17T06:13:30.826 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_other": 598, 2024-04-17T06:13:30.826 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_active": 0, 2024-04-17T06:13:30.826 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_send": 0, 2024-04-17T06:13:30.826 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_resend": 0, 2024-04-17T06:13:30.826 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_ping": 0, 2024-04-17T06:13:30.826 INFO:teuthology.orchestra.run.smithi103.stdout: "poolop_active": 0, 2024-04-17T06:13:30.827 INFO:teuthology.orchestra.run.smithi103.stdout: "poolop_send": 0, 2024-04-17T06:13:30.827 INFO:teuthology.orchestra.run.smithi103.stdout: "poolop_resend": 0, 2024-04-17T06:13:30.827 INFO:teuthology.orchestra.run.smithi103.stdout: "poolstat_active": 0, 2024-04-17T06:13:30.827 INFO:teuthology.orchestra.run.smithi103.stdout: "poolstat_send": 0, 2024-04-17T06:13:30.827 INFO:teuthology.orchestra.run.smithi103.stdout: "poolstat_resend": 0, 2024-04-17T06:13:30.827 INFO:teuthology.orchestra.run.smithi103.stdout: "statfs_active": 0, 2024-04-17T06:13:30.827 INFO:teuthology.orchestra.run.smithi103.stdout: "statfs_send": 0, 2024-04-17T06:13:30.827 INFO:teuthology.orchestra.run.smithi103.stdout: "statfs_resend": 0, 2024-04-17T06:13:30.827 INFO:teuthology.orchestra.run.smithi103.stdout: "command_active": 0, 2024-04-17T06:13:30.827 INFO:teuthology.orchestra.run.smithi103.stdout: "command_send": 0, 2024-04-17T06:13:30.827 INFO:teuthology.orchestra.run.smithi103.stdout: "command_resend": 0, 2024-04-17T06:13:30.827 INFO:teuthology.orchestra.run.smithi103.stdout: "map_epoch": 106, 2024-04-17T06:13:30.827 INFO:teuthology.orchestra.run.smithi103.stdout: "map_full": 0, 2024-04-17T06:13:30.827 INFO:teuthology.orchestra.run.smithi103.stdout: "map_inc": 20, 2024-04-17T06:13:30.827 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_sessions": 10, 2024-04-17T06:13:30.828 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_session_open": 10, 2024-04-17T06:13:30.828 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_session_close": 0, 2024-04-17T06:13:30.828 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_laggy": 0, 2024-04-17T06:13:30.828 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_wr": 24, 2024-04-17T06:13:30.828 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_rd": 0, 2024-04-17T06:13:30.828 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_del": 5 2024-04-17T06:13:30.828 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:30.828 INFO:teuthology.orchestra.run.smithi103.stdout: "oft": { 2024-04-17T06:13:30.828 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_objs": 1, 2024-04-17T06:13:30.828 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_kv_pairs": 0, 2024-04-17T06:13:30.828 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_updates": 0, 2024-04-17T06:13:30.828 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_removes": 0 2024-04-17T06:13:30.828 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:30.828 INFO:teuthology.orchestra.run.smithi103.stdout: "osd-slow-ops": { 2024-04-17T06:13:30.829 INFO:teuthology.orchestra.run.smithi103.stdout: "slow_ops_count": 0 2024-04-17T06:13:30.829 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:30.829 INFO:teuthology.orchestra.run.smithi103.stdout: "purge_queue": { 2024-04-17T06:13:30.829 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing_ops": 0, 2024-04-17T06:13:30.829 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing_ops_high_water": 1, 2024-04-17T06:13:30.829 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing": 0, 2024-04-17T06:13:30.829 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing_high_water": 1, 2024-04-17T06:13:30.829 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executed_ops": 1, 2024-04-17T06:13:30.829 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executed": 1, 2024-04-17T06:13:30.829 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_item_in_journal": 0 2024-04-17T06:13:30.829 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:30.829 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-17T06:13:30.829 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-04-17T06:13:30.829 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 104857600, 2024-04-17T06:13:30.830 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-04-17T06:13:30.830 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 4781, 2024-04-17T06:13:30.830 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 2538581, 2024-04-17T06:13:30.830 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-04-17T06:13:30.830 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 4781, 2024-04-17T06:13:30.830 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 0, 2024-04-17T06:13:30.830 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 0, 2024-04-17T06:13:30.830 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 4781, 2024-04-17T06:13:30.830 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 2538581, 2024-04-17T06:13:30.830 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-04-17T06:13:30.830 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:30.830 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:30.830 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:30.830 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-04-17T06:13:30.830 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:30.831 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-objecter_bytes": { 2024-04-17T06:13:30.831 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-04-17T06:13:30.831 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 104857600, 2024-04-17T06:13:30.831 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-04-17T06:13:30.831 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 0, 2024-04-17T06:13:30.831 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 0, 2024-04-17T06:13:30.831 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-04-17T06:13:30.831 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 0, 2024-04-17T06:13:30.831 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 2562, 2024-04-17T06:13:30.831 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 2287531, 2024-04-17T06:13:30.831 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 2562, 2024-04-17T06:13:30.831 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 2287531, 2024-04-17T06:13:30.831 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-04-17T06:13:30.831 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:30.832 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:30.832 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:30.832 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-04-17T06:13:30.832 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:30.832 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-objecter_ops": { 2024-04-17T06:13:30.832 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-04-17T06:13:30.832 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 1024, 2024-04-17T06:13:30.832 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-04-17T06:13:30.832 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 0, 2024-04-17T06:13:30.832 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 0, 2024-04-17T06:13:30.832 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-04-17T06:13:30.832 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 0, 2024-04-17T06:13:30.832 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 2562, 2024-04-17T06:13:30.832 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 2562, 2024-04-17T06:13:30.832 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 2562, 2024-04-17T06:13:30.833 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 2562, 2024-04-17T06:13:30.833 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-04-17T06:13:30.833 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:30.833 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:30.833 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:30.833 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-04-17T06:13:30.833 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:30.833 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-write_buf_throttle": { 2024-04-17T06:13:30.833 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-04-17T06:13:30.833 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 3758096384, 2024-04-17T06:13:30.833 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-04-17T06:13:30.833 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 1, 2024-04-17T06:13:30.833 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 141, 2024-04-17T06:13:30.833 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-04-17T06:13:30.833 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 1, 2024-04-17T06:13:30.834 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 0, 2024-04-17T06:13:30.834 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 0, 2024-04-17T06:13:30.834 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 1, 2024-04-17T06:13:30.834 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 141, 2024-04-17T06:13:30.834 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-04-17T06:13:30.834 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:30.834 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:30.834 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:30.834 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-04-17T06:13:30.835 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:30.835 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-write_buf_throttle-0x561908438500": { 2024-04-17T06:13:30.835 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 599, 2024-04-17T06:13:30.835 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 3758096384, 2024-04-17T06:13:30.835 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-04-17T06:13:30.835 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 1947, 2024-04-17T06:13:30.835 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 2272150, 2024-04-17T06:13:30.835 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-04-17T06:13:30.835 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 1947, 2024-04-17T06:13:30.835 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 0, 2024-04-17T06:13:30.835 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 0, 2024-04-17T06:13:30.835 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 1867, 2024-04-17T06:13:30.835 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 2271551, 2024-04-17T06:13:30.835 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-04-17T06:13:30.835 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:30.836 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:30.836 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:30.836 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-04-17T06:13:30.836 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-04-17T06:13:30.836 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-04-17T06:13:31.605 INFO:tasks.check_counter:Daemon mds.i mds.exported=1 2024-04-17T06:13:31.606 INFO:tasks.check_counter:Daemon mds.i mds.imported=1 2024-04-17T06:13:31.606 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=0 2024-04-17T06:13:31.606 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-04-17T06:13:31.606 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-04-17T06:13:31.606 DEBUG:tasks.check_counter:Getting stats from l 2024-04-17T06:13:31.606 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 565db2d8-fc7e-11ee-bc91-c7b262605968 -- ceph daemon mds.l perf dump 2024-04-17T06:13:31.845 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/mon.c/config 2024-04-17T06:13:31.966 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:13:31 smithi103 ceph-mon[36243]: pgmap v386: 97 pgs: 97 active+clean; 5.9 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-17T06:13:32.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:13:31 smithi098 ceph-mon[36516]: pgmap v386: 97 pgs: 97 active+clean; 5.9 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-17T06:13:32.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:13:31 smithi047 ceph-mon[32223]: pgmap v386: 97 pgs: 97 active+clean; 5.9 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-17T06:13:33.586 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-04-17T06:13:33.586 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-0": { 2024-04-17T06:13:33.586 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 0, 2024-04-17T06:13:33.586 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 2, 2024-04-17T06:13:33.586 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 0, 2024-04-17T06:13:33.586 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 0, 2024-04-17T06:13:33.586 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 4, 2024-04-17T06:13:33.586 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 0, 2024-04-17T06:13:33.586 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.001114923, 2024-04-17T06:13:33.587 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.000000000, 2024-04-17T06:13:33.587 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.000299598, 2024-04-17T06:13:33.587 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-17T06:13:33.587 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T06:13:33.587 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:33.587 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:33.587 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:33.587 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:33.587 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-04-17T06:13:33.587 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:33.587 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:33.587 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:33.587 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:33.588 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-17T06:13:33.588 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 0 2024-04-17T06:13:33.588 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:33.588 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-1": { 2024-04-17T06:13:33.588 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 2, 2024-04-17T06:13:33.588 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 183, 2024-04-17T06:13:33.588 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 320, 2024-04-17T06:13:33.588 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 188448, 2024-04-17T06:13:33.588 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 5, 2024-04-17T06:13:33.588 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 3, 2024-04-17T06:13:33.588 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.012534827, 2024-04-17T06:13:33.588 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.008576864, 2024-04-17T06:13:33.588 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.001098002, 2024-04-17T06:13:33.588 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-17T06:13:33.588 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T06:13:33.589 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 180, 2024-04-17T06:13:33.589 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.021305743, 2024-04-17T06:13:33.589 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000118365 2024-04-17T06:13:33.589 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:33.589 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-04-17T06:13:33.589 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:33.589 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:33.589 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:33.589 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:33.589 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-17T06:13:33.589 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 188448 2024-04-17T06:13:33.589 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:33.589 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-2": { 2024-04-17T06:13:33.589 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 255, 2024-04-17T06:13:33.589 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 242, 2024-04-17T06:13:33.590 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 1600880, 2024-04-17T06:13:33.590 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 148352, 2024-04-17T06:13:33.590 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 2, 2024-04-17T06:13:33.590 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 1, 2024-04-17T06:13:33.590 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.053156757, 2024-04-17T06:13:33.590 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.016335816, 2024-04-17T06:13:33.590 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.038962660, 2024-04-17T06:13:33.590 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.006572716, 2024-04-17T06:13:33.590 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T06:13:33.590 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 242, 2024-04-17T06:13:33.590 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.012734021, 2024-04-17T06:13:33.590 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000052619 2024-04-17T06:13:33.590 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:33.590 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-04-17T06:13:33.591 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:33.591 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:33.591 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:33.591 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:33.591 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 1600880, 2024-04-17T06:13:33.591 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 148352 2024-04-17T06:13:33.591 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:33.591 INFO:teuthology.orchestra.run.smithi103.stdout: "cct": { 2024-04-17T06:13:33.591 INFO:teuthology.orchestra.run.smithi103.stdout: "total_workers": 0, 2024-04-17T06:13:33.591 INFO:teuthology.orchestra.run.smithi103.stdout: "unhealthy_workers": 0 2024-04-17T06:13:33.591 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:33.591 INFO:teuthology.orchestra.run.smithi103.stdout: "mempool": { 2024-04-17T06:13:33.591 INFO:teuthology.orchestra.run.smithi103.stdout: "bloom_filter_bytes": 0, 2024-04-17T06:13:33.591 INFO:teuthology.orchestra.run.smithi103.stdout: "bloom_filter_items": 0, 2024-04-17T06:13:33.591 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_alloc_bytes": 0, 2024-04-17T06:13:33.592 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_alloc_items": 0, 2024-04-17T06:13:33.592 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_data_bytes": 0, 2024-04-17T06:13:33.592 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_data_items": 0, 2024-04-17T06:13:33.592 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-17T06:13:33.592 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_onode_items": 0, 2024-04-17T06:13:33.592 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-17T06:13:33.592 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_meta_items": 0, 2024-04-17T06:13:33.592 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_other_bytes": 0, 2024-04-17T06:13:33.592 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_other_items": 0, 2024-04-17T06:13:33.592 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-17T06:13:33.592 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_buffer_items": 0, 2024-04-17T06:13:33.592 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_extent_bytes": 0, 2024-04-17T06:13:33.592 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_extent_items": 0, 2024-04-17T06:13:33.592 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_blob_bytes": 0, 2024-04-17T06:13:33.592 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_blob_items": 0, 2024-04-17T06:13:33.592 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-17T06:13:33.593 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_shared_blob_items": 0, 2024-04-17T06:13:33.593 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-17T06:13:33.593 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_inline_bl_items": 0, 2024-04-17T06:13:33.593 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_fsck_bytes": 0, 2024-04-17T06:13:33.593 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_fsck_items": 0, 2024-04-17T06:13:33.593 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_txc_bytes": 0, 2024-04-17T06:13:33.593 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_txc_items": 0, 2024-04-17T06:13:33.593 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-17T06:13:33.593 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_deferred_items": 0, 2024-04-17T06:13:33.593 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_bytes": 0, 2024-04-17T06:13:33.593 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_items": 0, 2024-04-17T06:13:33.593 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_bytes": 0, 2024-04-17T06:13:33.593 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_items": 0, 2024-04-17T06:13:33.593 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_reader_bytes": 0, 2024-04-17T06:13:33.593 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_reader_items": 0, 2024-04-17T06:13:33.594 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_writer_bytes": 0, 2024-04-17T06:13:33.594 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_writer_items": 0, 2024-04-17T06:13:33.594 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_anon_bytes": 149024, 2024-04-17T06:13:33.594 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_anon_items": 31, 2024-04-17T06:13:33.594 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_meta_bytes": 0, 2024-04-17T06:13:33.594 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_meta_items": 0, 2024-04-17T06:13:33.594 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_bytes": 0, 2024-04-17T06:13:33.594 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_items": 0, 2024-04-17T06:13:33.594 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mapbl_bytes": 0, 2024-04-17T06:13:33.594 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mapbl_items": 0, 2024-04-17T06:13:33.594 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pglog_bytes": 0, 2024-04-17T06:13:33.594 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pglog_items": 0, 2024-04-17T06:13:33.594 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_bytes": 0, 2024-04-17T06:13:33.594 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_items": 0, 2024-04-17T06:13:33.594 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_mapping_bytes": 0, 2024-04-17T06:13:33.595 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_mapping_items": 0, 2024-04-17T06:13:33.595 INFO:teuthology.orchestra.run.smithi103.stdout: "pgmap_bytes": 0, 2024-04-17T06:13:33.595 INFO:teuthology.orchestra.run.smithi103.stdout: "pgmap_items": 0, 2024-04-17T06:13:33.595 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_co_bytes": 928, 2024-04-17T06:13:33.595 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_co_items": 2, 2024-04-17T06:13:33.595 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_1_bytes": 0, 2024-04-17T06:13:33.595 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_1_items": 0, 2024-04-17T06:13:33.595 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_2_bytes": 0, 2024-04-17T06:13:33.595 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_2_items": 0 2024-04-17T06:13:33.595 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T06:13:33.595 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-17T06:13:33.595 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-04-17T06:13:33.596 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 104857600, 2024-04-17T06:13:33.596 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-04-17T06:13:33.596 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 253, 2024-04-17T06:13:33.596 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 1562435, 2024-04-17T06:13:33.596 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-04-17T06:13:33.596 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 253, 2024-04-17T06:13:33.596 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 0, 2024-04-17T06:13:33.596 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 0, 2024-04-17T06:13:33.596 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 253, 2024-04-17T06:13:33.596 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 1562435, 2024-04-17T06:13:33.596 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-04-17T06:13:33.596 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T06:13:33.596 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T06:13:33.596 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T06:13:33.596 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-04-17T06:13:33.596 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-04-17T06:13:33.596 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-04-17T06:13:34.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:13:33 smithi098 ceph-mon[36516]: pgmap v387: 97 pgs: 97 active+clean; 5.9 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-17T06:13:34.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:13:33 smithi047 ceph-mon[32223]: pgmap v387: 97 pgs: 97 active+clean; 5.9 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-17T06:13:34.275 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:13:33 smithi103 ceph-mon[36243]: pgmap v387: 97 pgs: 97 active+clean; 5.9 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-17T06:13:34.307 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-04-17T06:13:34.308 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-04-17T06:13:34.308 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.l 2024-04-17T06:13:34.308 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.l 2024-04-17T06:13:34.308 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.l 2024-04-17T06:13:34.308 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-17T06:13:34.320 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-17T06:13:34.320 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-17T06:13:34.320 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:13:34.343 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-17T06:13:34.343 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:13:34.404 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T06:13:34.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:34.564+0000 7fce7797c640 1 -- 172.21.15.47:0/288081075 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce70100e50 msgr2=0x7fce7010eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:13:34.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:34.564+0000 7fce7797c640 1 --2- 172.21.15.47:0/288081075 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce70100e50 0x7fce7010eea0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fce68009cd0 tx=0x7fce6802c0f0 comp rx=0 tx=0).stop 2024-04-17T06:13:34.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:34.564+0000 7fce7797c640 1 -- 172.21.15.47:0/288081075 shutdown_connections 2024-04-17T06:13:34.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:34.564+0000 7fce7797c640 1 --2- 172.21.15.47:0/288081075 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce70100e50 0x7fce7010eea0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:13:34.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:34.564+0000 7fce7797c640 1 --2- 172.21.15.47:0/288081075 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fce701004d0 0x7fce70100910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:13:34.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:34.564+0000 7fce7797c640 1 --2- 172.21.15.47:0/288081075 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fce701080e0 0x7fce701084c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:13:34.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:34.564+0000 7fce7797c640 1 -- 172.21.15.47:0/288081075 >> 172.21.15.47:0/288081075 conn(0x7fce70077490 msgr2=0x7fce700fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:13:34.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:34.564+0000 7fce7797c640 1 -- 172.21.15.47:0/288081075 shutdown_connections 2024-04-17T06:13:34.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:34.564+0000 7fce7797c640 1 -- 172.21.15.47:0/288081075 wait complete. 2024-04-17T06:13:34.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:34.565+0000 7fce7797c640 1 Processor -- start 2024-04-17T06:13:34.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:34.565+0000 7fce7797c640 1 -- start start 2024-04-17T06:13:34.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:34.565+0000 7fce7797c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fce701004d0 0x7fce7019e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:13:34.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:34.565+0000 7fce7797c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fce70100e50 0x7fce7019eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:13:34.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:34.565+0000 7fce7797c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce701080e0 0x7fce701986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:13:34.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:34.565+0000 7fce7797c640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fce701123e0 con 0x7fce701080e0 2024-04-17T06:13:34.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:34.565+0000 7fce7797c640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fce70112260 con 0x7fce701004d0 2024-04-17T06:13:34.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:34.565+0000 7fce7797c640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fce70112560 con 0x7fce70100e50 2024-04-17T06:13:34.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:34.565+0000 7fce75f19640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce701080e0 0x7fce701986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:13:34.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:34.565+0000 7fce75f19640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce701080e0 0x7fce701986e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46634/0 (socket says 172.21.15.47:46634) 2024-04-17T06:13:34.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:34.565+0000 7fce75f19640 1 -- 172.21.15.47:0/4257291614 learned_addr learned my addr 172.21.15.47:0/4257291614 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:13:34.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:34.566+0000 7fce74f17640 1 --2- 172.21.15.47:0/4257291614 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fce70100e50 0x7fce7019eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:13:34.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:34.566+0000 7fce75718640 1 --2- 172.21.15.47:0/4257291614 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fce701004d0 0x7fce7019e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:13:34.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:34.566+0000 7fce75f19640 1 -- 172.21.15.47:0/4257291614 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fce70100e50 msgr2=0x7fce7019eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:13:34.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:34.566+0000 7fce75f19640 1 --2- 172.21.15.47:0/4257291614 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fce70100e50 0x7fce7019eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:13:34.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:34.566+0000 7fce75f19640 1 -- 172.21.15.47:0/4257291614 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fce701004d0 msgr2=0x7fce7019e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:13:34.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:34.566+0000 7fce75f19640 1 --2- 172.21.15.47:0/4257291614 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fce701004d0 0x7fce7019e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:13:34.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:34.566+0000 7fce75f19640 1 -- 172.21.15.47:0/4257291614 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce70198fa0 con 0x7fce701080e0 2024-04-17T06:13:34.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:34.566+0000 7fce74f17640 1 --2- 172.21.15.47:0/4257291614 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fce70100e50 0x7fce7019eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:13:34.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:34.566+0000 7fce75718640 1 --2- 172.21.15.47:0/4257291614 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fce701004d0 0x7fce7019e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:13:34.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:34.566+0000 7fce75f19640 1 --2- 172.21.15.47:0/4257291614 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce701080e0 0x7fce701986e0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fce68037040 tx=0x7fce68035ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:13:34.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:34.566+0000 7fce667fc640 1 -- 172.21.15.47:0/4257291614 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce68039040 con 0x7fce701080e0 2024-04-17T06:13:34.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:34.566+0000 7fce667fc640 1 -- 172.21.15.47:0/4257291614 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce68044070 con 0x7fce701080e0 2024-04-17T06:13:34.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:34.566+0000 7fce7797c640 1 -- 172.21.15.47:0/4257291614 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce70199230 con 0x7fce701080e0 2024-04-17T06:13:34.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:34.566+0000 7fce667fc640 1 -- 172.21.15.47:0/4257291614 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce68004330 con 0x7fce701080e0 2024-04-17T06:13:34.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:34.566+0000 7fce7797c640 1 -- 172.21.15.47:0/4257291614 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce701a5550 con 0x7fce701080e0 2024-04-17T06:13:34.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:34.567+0000 7fce7797c640 1 -- 172.21.15.47:0/4257291614 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce701037f0 con 0x7fce701080e0 2024-04-17T06:13:34.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:34.568+0000 7fce667fc640 1 -- 172.21.15.47:0/4257291614 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fce68003970 con 0x7fce701080e0 2024-04-17T06:13:34.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:34.569+0000 7fce667fc640 1 --2- 172.21.15.47:0/4257291614 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fce5c0a3da0 0x7fce5c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:13:34.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:34.569+0000 7fce75718640 1 --2- 172.21.15.47:0/4257291614 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fce5c0a3da0 0x7fce5c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:13:34.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:34.569+0000 7fce667fc640 1 -- 172.21.15.47:0/4257291614 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8513+0+0 (secure 0 0 0) 0x7fce680f32e0 con 0x7fce701080e0 2024-04-17T06:13:34.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:34.569+0000 7fce75718640 1 --2- 172.21.15.47:0/4257291614 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fce5c0a3da0 0x7fce5c0a6260 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fce6c0058c0 tx=0x7fce6c005cb0 comp rx=0 tx=0).ready entity=mgr.24923 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:13:34.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:34.573+0000 7fce667fc640 1 -- 172.21.15.47:0/4257291614 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fce680bfa90 con 0x7fce701080e0 2024-04-17T06:13:34.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:34.763+0000 7fce7797c640 1 -- 172.21.15.47:0/4257291614 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fce70199bc0 con 0x7fce701080e0 2024-04-17T06:13:34.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:34.764+0000 7fce667fc640 1 -- 172.21.15.47:0/4257291614 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) ==== 74+0+23073 (secure 0 0 0) 0x7fce680bf390 con 0x7fce701080e0 2024-04-17T06:13:34.763 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-17T06:13:34.763 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":106,"fsid":"565db2d8-fc7e-11ee-bc91-c7b262605968","created":"2024-04-17T05:51:19.888865+0000","modified":"2024-04-17T06:13:14.788483+0000","last_up_change":"2024-04-17T05:58:24.380131+0000","last_in_change":"2024-04-17T05:58:04.418236+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T05:54:37.734306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T05:59:12.458010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T05:59:12.901339+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":19,"snap_epoch":105,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8453f300-533d-4736-a343-55619c08e966","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":1290127544},{"type":"v1","addr":"172.21.15.47:6803","nonce":1290127544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":1290127544},{"type":"v1","addr":"172.21.15.47:6805","nonce":1290127544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":1290127544},{"type":"v1","addr":"172.21.15.47:6809","nonce":1290127544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":1290127544},{"type":"v1","addr":"172.21.15.47:6807","nonce":1290127544}]},"public_addr":"172.21.15.47:6803/1290127544","cluster_addr":"172.21.15.47:6805/1290127544","heartbeat_back_addr":"172.21.15.47:6809/1290127544","heartbeat_front_addr":"172.21.15.47:6807/1290127544","state":["exists","up"]},{"osd":1,"uuid":"80b0f03e-c1bc-4658-8028-1e6b8eedcc47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":1180430954},{"type":"v1","addr":"172.21.15.98:6801","nonce":1180430954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":1180430954},{"type":"v1","addr":"172.21.15.98:6803","nonce":1180430954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":1180430954},{"type":"v1","addr":"172.21.15.98:6807","nonce":1180430954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":1180430954},{"type":"v1","addr":"172.21.15.98:6805","nonce":1180430954}]},"public_addr":"172.21.15.98:6801/1180430954","cluster_addr":"172.21.15.98:6803/1180430954","heartbeat_back_addr":"172.21.15.98:6807/1180430954","heartbeat_front_addr":"172.21.15.98:6805/1180430954","state":["exists","up"]},{"osd":2,"uuid":"0fc9d383-05b4-498d-b530-fccf7aeddd7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":494676216},{"type":"v1","addr":"172.21.15.103:6801","nonce":494676216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":494676216},{"type":"v1","addr":"172.21.15.103:6803","nonce":494676216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":494676216},{"type":"v1","addr":"172.21.15.103:6807","nonce":494676216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":494676216},{"type":"v1","addr":"172.21.15.103:6805","nonce":494676216}]},"public_addr":"172.21.15.103:6801/494676216","cluster_addr":"172.21.15.103:6803/494676216","heartbeat_back_addr":"172.21.15.103:6807/494676216","heartbeat_front_addr":"172.21.15.103:6805/494676216","state":["exists","up"]},{"osd":3,"uuid":"f6e0b0d5-0fb3-42d0-9e84-f7cfef21a998","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":1998369447},{"type":"v1","addr":"172.21.15.47:6811","nonce":1998369447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":1998369447},{"type":"v1","addr":"172.21.15.47:6813","nonce":1998369447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":1998369447},{"type":"v1","addr":"172.21.15.47:6817","nonce":1998369447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":1998369447},{"type":"v1","addr":"172.21.15.47:6815","nonce":1998369447}]},"public_addr":"172.21.15.47:6811/1998369447","cluster_addr":"172.21.15.47:6813/1998369447","heartbeat_back_addr":"172.21.15.47:6817/1998369447","heartbeat_front_addr":"172.21.15.47:6815/1998369447","state":["exists","up"]},{"osd":4,"uuid":"2c26217e-02df-466d-bc01-0149d4aec3c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":3060482485},{"type":"v1","addr":"172.21.15.98:6809","nonce":3060482485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":3060482485},{"type":"v1","addr":"172.21.15.98:6811","nonce":3060482485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":3060482485},{"type":"v1","addr":"172.21.15.98:6815","nonce":3060482485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":3060482485},{"type":"v1","addr":"172.21.15.98:6813","nonce":3060482485}]},"public_addr":"172.21.15.98:6809/3060482485","cluster_addr":"172.21.15.98:6811/3060482485","heartbeat_back_addr":"172.21.15.98:6815/3060482485","heartbeat_front_addr":"172.21.15.98:6813/3060482485","state":["exists","up"]},{"osd":5,"uuid":"740bf3e0-bea8-496b-8d6f-e42d0b017258","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":4181363567},{"type":"v1","addr":"172.21.15.103:6809","nonce":4181363567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":4181363567},{"type":"v1","addr":"172.21.15.103:6811","nonce":4181363567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":4181363567},{"type":"v1","addr":"172.21.15.103:6815","nonce":4181363567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":4181363567},{"type":"v1","addr":"172.21.15.103:6813","nonce":4181363567}]},"public_addr":"172.21.15.103:6809/4181363567","cluster_addr":"172.21.15.103:6811/4181363567","heartbeat_back_addr":"172.21.15.103:6815/4181363567","heartbeat_front_addr":"172.21.15.103:6813/4181363567","state":["exists","up"]},{"osd":6,"uuid":"356a6255-2c07-4f0d-a233-26be04c92b38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":2517529668},{"type":"v1","addr":"172.21.15.47:6819","nonce":2517529668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2517529668},{"type":"v1","addr":"172.21.15.47:6821","nonce":2517529668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2517529668},{"type":"v1","addr":"172.21.15.47:6825","nonce":2517529668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2517529668},{"type":"v1","addr":"172.21.15.47:6823","nonce":2517529668}]},"public_addr":"172.21.15.47:6819/2517529668","cluster_addr":"172.21.15.47:6821/2517529668","heartbeat_back_addr":"172.21.15.47:6825/2517529668","heartbeat_front_addr":"172.21.15.47:6823/2517529668","state":["exists","up"]},{"osd":7,"uuid":"349d0120-8290-417d-a800-5a8985f554f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":4225498496},{"type":"v1","addr":"172.21.15.98:6817","nonce":4225498496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":4225498496},{"type":"v1","addr":"172.21.15.98:6819","nonce":4225498496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":4225498496},{"type":"v1","addr":"172.21.15.98:6823","nonce":4225498496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":4225498496},{"type":"v1","addr":"172.21.15.98:6821","nonce":4225498496}]},"public_addr":"172.21.15.98:6817/4225498496","cluster_addr":"172.21.15.98:6819/4225498496","heartbeat_back_addr":"172.21.15.98:6823/4225498496","heartbeat_front_addr":"172.21.15.98:6821/4225498496","state":["exists","up"]},{"osd":8,"uuid":"cee4712f-321b-42c0-92ba-e15144c00f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":3433905520},{"type":"v1","addr":"172.21.15.103:6817","nonce":3433905520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":3433905520},{"type":"v1","addr":"172.21.15.103:6819","nonce":3433905520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":3433905520},{"type":"v1","addr":"172.21.15.103:6823","nonce":3433905520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":3433905520},{"type":"v1","addr":"172.21.15.103:6821","nonce":3433905520}]},"public_addr":"172.21.15.103:6817/3433905520","cluster_addr":"172.21.15.103:6819/3433905520","heartbeat_back_addr":"172.21.15.103:6823/3433905520","heartbeat_front_addr":"172.21.15.103:6821/3433905520","state":["exists","up"]},{"osd":9,"uuid":"63fc73c1-5a22-487d-80b8-9894e2ddd207","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":728359554},{"type":"v1","addr":"172.21.15.47:6827","nonce":728359554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":728359554},{"type":"v1","addr":"172.21.15.47:6829","nonce":728359554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":728359554},{"type":"v1","addr":"172.21.15.47:6833","nonce":728359554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":728359554},{"type":"v1","addr":"172.21.15.47:6831","nonce":728359554}]},"public_addr":"172.21.15.47:6827/728359554","cluster_addr":"172.21.15.47:6829/728359554","heartbeat_back_addr":"172.21.15.47:6833/728359554","heartbeat_front_addr":"172.21.15.47:6831/728359554","state":["exists","up"]},{"osd":10,"uuid":"9565bcc3-62ee-48eb-a07a-03a81e0cc8e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":468205689},{"type":"v1","addr":"172.21.15.98:6825","nonce":468205689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":468205689},{"type":"v1","addr":"172.21.15.98:6827","nonce":468205689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":468205689},{"type":"v1","addr":"172.21.15.98:6831","nonce":468205689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":468205689},{"type":"v1","addr":"172.21.15.98:6829","nonce":468205689}]},"public_addr":"172.21.15.98:6825/468205689","cluster_addr":"172.21.15.98:6827/468205689","heartbeat_back_addr":"172.21.15.98:6831/468205689","heartbeat_front_addr":"172.21.15.98:6829/468205689","state":["exists","up"]},{"osd":11,"uuid":"2e85e8d9-bf6c-4a12-b124-a4690dd61108","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":2639831013},{"type":"v1","addr":"172.21.15.103:6825","nonce":2639831013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":2639831013},{"type":"v1","addr":"172.21.15.103:6827","nonce":2639831013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":2639831013},{"type":"v1","addr":"172.21.15.103:6831","nonce":2639831013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":2639831013},{"type":"v1","addr":"172.21.15.103:6829","nonce":2639831013}]},"public_addr":"172.21.15.103:6825/2639831013","cluster_addr":"172.21.15.103:6827/2639831013","heartbeat_back_addr":"172.21.15.103:6831/2639831013","heartbeat_front_addr":"172.21.15.103:6829/2639831013","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:53:47.366438+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:54:10.953559+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:54:34.320332+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:54:59.751360+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:55:23.982052+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:55:48.425017+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:56:14.793368+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:56:40.440830+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:57:04.623680+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:57:31.215262+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:57:57.040346+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:58:22.671572+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:6800/3157902806":"2024-04-18T06:01:18.246671+0000","172.21.15.47:6801/3157902806":"2024-04-18T06:01:18.246671+0000","172.21.15.47:0/3681048987":"2024-04-18T06:01:18.246671+0000","172.21.15.47:0/3009926768":"2024-04-18T06:01:18.246671+0000","172.21.15.47:0/2970926607":"2024-04-18T06:01:18.246671+0000","172.21.15.47:6801/3632157175":"2024-04-18T05:52:13.645775+0000","172.21.15.47:0/4288219269":"2024-04-18T05:52:13.645775+0000","172.21.15.47:0/2584487544":"2024-04-18T05:52:13.645775+0000","172.21.15.47:6801/953064786":"2024-04-18T05:51:45.979351+0000","172.21.15.47:0/3484500927":"2024-04-18T05:51:45.979351+0000","172.21.15.47:6800/3632157175":"2024-04-18T05:52:13.645775+0000","172.21.15.47:0/112268670":"2024-04-18T05:51:45.979351+0000","172.21.15.47:0/1638423256":"2024-04-18T05:51:45.979351+0000","172.21.15.47:0/3275066738":"2024-04-18T06:01:18.246671+0000","172.21.15.47:6800/953064786":"2024-04-18T05:51:45.979351+0000","172.21.15.47:0/849479010":"2024-04-18T05:52:13.645775+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":7,"length":1},{"begin":19,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T06:13:34.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:34.766+0000 7fce7797c640 1 -- 172.21.15.47:0/4257291614 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fce5c0a3da0 msgr2=0x7fce5c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:13:34.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:34.766+0000 7fce7797c640 1 --2- 172.21.15.47:0/4257291614 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fce5c0a3da0 0x7fce5c0a6260 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fce6c0058c0 tx=0x7fce6c005cb0 comp rx=0 tx=0).stop 2024-04-17T06:13:34.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:34.767+0000 7fce7797c640 1 -- 172.21.15.47:0/4257291614 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce701080e0 msgr2=0x7fce701986e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:13:34.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:34.767+0000 7fce7797c640 1 --2- 172.21.15.47:0/4257291614 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce701080e0 0x7fce701986e0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fce68037040 tx=0x7fce68035ae0 comp rx=0 tx=0).stop 2024-04-17T06:13:34.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:34.767+0000 7fce7797c640 1 -- 172.21.15.47:0/4257291614 shutdown_connections 2024-04-17T06:13:34.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:34.767+0000 7fce7797c640 1 --2- 172.21.15.47:0/4257291614 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fce5c0a3da0 0x7fce5c0a6260 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:13:34.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:34.767+0000 7fce7797c640 1 --2- 172.21.15.47:0/4257291614 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce701080e0 0x7fce701986e0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:13:34.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:34.767+0000 7fce7797c640 1 --2- 172.21.15.47:0/4257291614 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fce70100e50 0x7fce7019eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:13:34.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:34.767+0000 7fce7797c640 1 --2- 172.21.15.47:0/4257291614 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fce701004d0 0x7fce7019e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:13:34.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:34.767+0000 7fce7797c640 1 -- 172.21.15.47:0/4257291614 >> 172.21.15.47:0/4257291614 conn(0x7fce70077490 msgr2=0x7fce70104f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:13:34.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:34.767+0000 7fce7797c640 1 -- 172.21.15.47:0/4257291614 shutdown_connections 2024-04-17T06:13:34.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:34.767+0000 7fce7797c640 1 -- 172.21.15.47:0/4257291614 wait complete. 2024-04-17T06:13:34.776 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T05:54:37.734306+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T05:59:12.458010+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T05:59:12.901339+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '105', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 19, 'snap_epoch': 105, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T06:13:34.776 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T06:13:34.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:34.890+0000 7f7763241640 1 -- 172.21.15.47:0/1196750674 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f775c106180 msgr2=0x7f775c10dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:13:34.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:34.890+0000 7f7763241640 1 --2- 172.21.15.47:0/1196750674 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f775c106180 0x7f775c10dd50 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f7754009ca0 tx=0x7f775402c0f0 comp rx=0 tx=0).stop 2024-04-17T06:13:34.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:34.890+0000 7f7763241640 1 -- 172.21.15.47:0/1196750674 shutdown_connections 2024-04-17T06:13:34.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:34.890+0000 7f7763241640 1 --2- 172.21.15.47:0/1196750674 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f775c106180 0x7f775c10dd50 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:13:34.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:34.890+0000 7f7763241640 1 --2- 172.21.15.47:0/1196750674 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f775c101a60 0x7f775c105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:13:34.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:34.890+0000 7f7763241640 1 --2- 172.21.15.47:0/1196750674 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f775c1010b0 0x7f775c101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:13:34.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:34.890+0000 7f7763241640 1 -- 172.21.15.47:0/1196750674 >> 172.21.15.47:0/1196750674 conn(0x7f775c077ec0 msgr2=0x7f775c0782d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:13:34.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:34.890+0000 7f7763241640 1 -- 172.21.15.47:0/1196750674 shutdown_connections 2024-04-17T06:13:34.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:34.890+0000 7f7763241640 1 -- 172.21.15.47:0/1196750674 wait complete. 2024-04-17T06:13:34.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:34.891+0000 7f7763241640 1 Processor -- start 2024-04-17T06:13:34.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:34.891+0000 7f7763241640 1 -- start start 2024-04-17T06:13:34.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:34.891+0000 7f7763241640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f775c1010b0 0x7f775c1976a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:13:34.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:34.891+0000 7f7763241640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f775c106180 0x7f775c197be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:13:34.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:34.891+0000 7f7763241640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f775c19bf10 0x7f775c19c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:13:34.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:34.891+0000 7f7763241640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f775c111240 con 0x7f775c1010b0 2024-04-17T06:13:34.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:34.891+0000 7f7763241640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f775c1110c0 con 0x7f775c106180 2024-04-17T06:13:34.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:34.891+0000 7f7763241640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f775c1113c0 con 0x7f775c19bf10 2024-04-17T06:13:34.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:34.891+0000 7f7760fdd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f775c1010b0 0x7f775c1976a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:13:34.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:34.891+0000 7f7760fdd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f775c1010b0 0x7f775c1976a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46644/0 (socket says 172.21.15.47:46644) 2024-04-17T06:13:34.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:34.891+0000 7f7760fdd640 1 -- 172.21.15.47:0/3992790273 learned_addr learned my addr 172.21.15.47:0/3992790273 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:13:34.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:34.891+0000 7f77617de640 1 --2- 172.21.15.47:0/3992790273 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f775c19bf10 0x7f775c19c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:13:34.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:34.891+0000 7f775bfff640 1 --2- 172.21.15.47:0/3992790273 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f775c106180 0x7f775c197be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:13:34.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:34.892+0000 7f7760fdd640 1 -- 172.21.15.47:0/3992790273 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f775c19bf10 msgr2=0x7f775c19c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:13:34.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:34.892+0000 7f7760fdd640 1 --2- 172.21.15.47:0/3992790273 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f775c19bf10 0x7f775c19c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:13:34.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:34.892+0000 7f7760fdd640 1 -- 172.21.15.47:0/3992790273 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f775c106180 msgr2=0x7f775c197be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:13:34.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:34.892+0000 7f7760fdd640 1 --2- 172.21.15.47:0/3992790273 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f775c106180 0x7f775c197be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:13:34.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:34.892+0000 7f7760fdd640 1 -- 172.21.15.47:0/3992790273 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f775c19ca00 con 0x7f775c1010b0 2024-04-17T06:13:34.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:34.892+0000 7f775bfff640 1 --2- 172.21.15.47:0/3992790273 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f775c106180 0x7f775c197be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:13:34.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:34.892+0000 7f7760fdd640 1 --2- 172.21.15.47:0/3992790273 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f775c1010b0 0x7f775c1976a0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f775000d770 tx=0x7f775000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:13:34.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:34.892+0000 7f7759ffb640 1 -- 172.21.15.47:0/3992790273 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7750017070 con 0x7f775c1010b0 2024-04-17T06:13:34.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:34.892+0000 7f7759ffb640 1 -- 172.21.15.47:0/3992790273 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7750005500 con 0x7f775c1010b0 2024-04-17T06:13:34.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:34.892+0000 7f7763241640 1 -- 172.21.15.47:0/3992790273 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f775c19ccf0 con 0x7f775c1010b0 2024-04-17T06:13:34.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:34.892+0000 7f7759ffb640 1 -- 172.21.15.47:0/3992790273 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7750004e00 con 0x7f775c1010b0 2024-04-17T06:13:34.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:34.892+0000 7f7763241640 1 -- 172.21.15.47:0/3992790273 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f775c1a4490 con 0x7f775c1010b0 2024-04-17T06:13:34.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:34.893+0000 7f7763241640 1 -- 172.21.15.47:0/3992790273 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f775c0691d0 con 0x7f775c1010b0 2024-04-17T06:13:34.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:34.894+0000 7f7759ffb640 1 -- 172.21.15.47:0/3992790273 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7750004fa0 con 0x7f775c1010b0 2024-04-17T06:13:34.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:34.895+0000 7f7759ffb640 1 --2- 172.21.15.47:0/3992790273 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f774c0a3d20 0x7f774c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:13:34.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:34.895+0000 7f775bfff640 1 --2- 172.21.15.47:0/3992790273 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f774c0a3d20 0x7f774c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:13:34.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:34.895+0000 7f7759ffb640 1 -- 172.21.15.47:0/3992790273 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8513+0+0 (secure 0 0 0) 0x7f77500d0280 con 0x7f775c1010b0 2024-04-17T06:13:34.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:34.895+0000 7f775bfff640 1 --2- 172.21.15.47:0/3992790273 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f774c0a3d20 0x7f774c0a61e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7748005e10 tx=0x7f7748005da0 comp rx=0 tx=0).ready entity=mgr.24923 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:13:34.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:34.899+0000 7f7759ffb640 1 -- 172.21.15.47:0/3992790273 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f775009c920 con 0x7f775c1010b0 2024-04-17T06:13:35.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.084+0000 7f7763241640 1 -- 172.21.15.47:0/3992790273 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f775c101bf0 con 0x7f775c1010b0 2024-04-17T06:13:35.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.085+0000 7f7759ffb640 1 -- 172.21.15.47:0/3992790273 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v106) ==== 93+0+10 (secure 0 0 0) 0x7f775009c220 con 0x7f775c1010b0 2024-04-17T06:13:35.083 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-04-17T06:13:35.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.087+0000 7f7763241640 1 -- 172.21.15.47:0/3992790273 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f774c0a3d20 msgr2=0x7f774c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:13:35.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.087+0000 7f7763241640 1 --2- 172.21.15.47:0/3992790273 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f774c0a3d20 0x7f774c0a61e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7748005e10 tx=0x7f7748005da0 comp rx=0 tx=0).stop 2024-04-17T06:13:35.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.087+0000 7f7763241640 1 -- 172.21.15.47:0/3992790273 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f775c1010b0 msgr2=0x7f775c1976a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:13:35.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.087+0000 7f7763241640 1 --2- 172.21.15.47:0/3992790273 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f775c1010b0 0x7f775c1976a0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f775000d770 tx=0x7f775000dc30 comp rx=0 tx=0).stop 2024-04-17T06:13:35.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.087+0000 7f7763241640 1 -- 172.21.15.47:0/3992790273 shutdown_connections 2024-04-17T06:13:35.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.087+0000 7f7763241640 1 --2- 172.21.15.47:0/3992790273 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f775c19bf10 0x7f775c19c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:13:35.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.087+0000 7f7763241640 1 --2- 172.21.15.47:0/3992790273 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f775c106180 0x7f775c197be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:13:35.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.087+0000 7f7763241640 1 --2- 172.21.15.47:0/3992790273 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f774c0a3d20 0x7f774c0a61e0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:13:35.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.087+0000 7f7763241640 1 --2- 172.21.15.47:0/3992790273 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f775c1010b0 0x7f775c1976a0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:13:35.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.087+0000 7f7763241640 1 -- 172.21.15.47:0/3992790273 >> 172.21.15.47:0/3992790273 conn(0x7f775c077ec0 msgr2=0x7f775c0fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:13:35.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.087+0000 7f7763241640 1 -- 172.21.15.47:0/3992790273 shutdown_connections 2024-04-17T06:13:35.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.087+0000 7f7763241640 1 -- 172.21.15.47:0/3992790273 wait complete. 2024-04-17T06:13:35.095 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:13:34 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/4257291614' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T06:13:35.096 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T06:13:35.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:13:34 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/4257291614' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T06:13:35.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.206+0000 7f6987074640 1 -- 172.21.15.47:0/678432565 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6980101b50 msgr2=0x7f698010dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:13:35.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.206+0000 7f6987074640 1 --2- 172.21.15.47:0/678432565 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6980101b50 0x7f698010dc90 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f6978009930 tx=0x7f697802bc30 comp rx=0 tx=0).stop 2024-04-17T06:13:35.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.206+0000 7f6987074640 1 -- 172.21.15.47:0/678432565 shutdown_connections 2024-04-17T06:13:35.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.206+0000 7f6987074640 1 --2- 172.21.15.47:0/678432565 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6980101b50 0x7f698010dc90 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:13:35.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.206+0000 7f6987074640 1 --2- 172.21.15.47:0/678432565 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f69801011b0 0x7f6980101610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:13:35.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.206+0000 7f6987074640 1 --2- 172.21.15.47:0/678432565 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6980107040 0x7f6980107420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:13:35.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.206+0000 7f6987074640 1 -- 172.21.15.47:0/678432565 >> 172.21.15.47:0/678432565 conn(0x7f6980076c60 msgr2=0x7f6980077070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:13:35.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.207+0000 7f6987074640 1 -- 172.21.15.47:0/678432565 shutdown_connections 2024-04-17T06:13:35.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.207+0000 7f6987074640 1 -- 172.21.15.47:0/678432565 wait complete. 2024-04-17T06:13:35.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.207+0000 7f6987074640 1 Processor -- start 2024-04-17T06:13:35.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.207+0000 7f6987074640 1 -- start start 2024-04-17T06:13:35.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.207+0000 7f6987074640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f69801011b0 0x7f6980199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:13:35.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.207+0000 7f6987074640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6980101b50 0x7f6980199ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:13:35.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.207+0000 7f6987074640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6980107040 0x7f69801a46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:13:35.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.207+0000 7f6987074640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f6980111240 con 0x7f69801011b0 2024-04-17T06:13:35.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.207+0000 7f6987074640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f69801110c0 con 0x7f6980107040 2024-04-17T06:13:35.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.207+0000 7f6987074640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f69801113c0 con 0x7f6980101b50 2024-04-17T06:13:35.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.207+0000 7f6984e10640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f69801011b0 0x7f6980199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:13:35.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.207+0000 7f6984e10640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f69801011b0 0x7f6980199660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46648/0 (socket says 172.21.15.47:46648) 2024-04-17T06:13:35.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.207+0000 7f6984e10640 1 -- 172.21.15.47:0/2236730144 learned_addr learned my addr 172.21.15.47:0/2236730144 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:13:35.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.208+0000 7f6985611640 1 --2- 172.21.15.47:0/2236730144 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6980107040 0x7f69801a46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:13:35.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.208+0000 7f697ffff640 1 --2- 172.21.15.47:0/2236730144 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6980101b50 0x7f6980199ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:13:35.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.208+0000 7f6984e10640 1 -- 172.21.15.47:0/2236730144 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6980101b50 msgr2=0x7f6980199ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:13:35.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.208+0000 7f6984e10640 1 --2- 172.21.15.47:0/2236730144 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6980101b50 0x7f6980199ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:13:35.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.208+0000 7f6984e10640 1 -- 172.21.15.47:0/2236730144 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6980107040 msgr2=0x7f69801a46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:13:35.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.208+0000 7f6984e10640 1 --2- 172.21.15.47:0/2236730144 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6980107040 0x7f69801a46e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:13:35.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.208+0000 7f6984e10640 1 -- 172.21.15.47:0/2236730144 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69801a4c20 con 0x7f69801011b0 2024-04-17T06:13:35.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.208+0000 7f6984e10640 1 --2- 172.21.15.47:0/2236730144 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f69801011b0 0x7f6980199660 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f697000e9f0 tx=0x7f697000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:13:35.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.208+0000 7f697dffb640 1 -- 172.21.15.47:0/2236730144 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f697000cd80 con 0x7f69801011b0 2024-04-17T06:13:35.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.208+0000 7f6987074640 1 -- 172.21.15.47:0/2236730144 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69801a4df0 con 0x7f69801011b0 2024-04-17T06:13:35.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.208+0000 7f697dffb640 1 -- 172.21.15.47:0/2236730144 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6970010910 con 0x7f69801011b0 2024-04-17T06:13:35.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.208+0000 7f697dffb640 1 -- 172.21.15.47:0/2236730144 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69700188f0 con 0x7f69801011b0 2024-04-17T06:13:35.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.208+0000 7f6987074640 1 -- 172.21.15.47:0/2236730144 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69801a52b0 con 0x7f69801011b0 2024-04-17T06:13:35.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.209+0000 7f6987074640 1 -- 172.21.15.47:0/2236730144 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69801025a0 con 0x7f69801011b0 2024-04-17T06:13:35.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.210+0000 7f697dffb640 1 -- 172.21.15.47:0/2236730144 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6970010ac0 con 0x7f69801011b0 2024-04-17T06:13:35.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.211+0000 7f697dffb640 1 --2- 172.21.15.47:0/2236730144 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f69680a3d20 0x7f69680a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:13:35.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.211+0000 7f697dffb640 1 -- 172.21.15.47:0/2236730144 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8513+0+0 (secure 0 0 0) 0x7f69700d0ae0 con 0x7f69801011b0 2024-04-17T06:13:35.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.211+0000 7f697ffff640 1 --2- 172.21.15.47:0/2236730144 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f69680a3d20 0x7f69680a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:13:35.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.212+0000 7f697ffff640 1 --2- 172.21.15.47:0/2236730144 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f69680a3d20 0x7f69680a61e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f698006b0b0 tx=0x7f6974009680 comp rx=0 tx=0).ready entity=mgr.24923 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:13:35.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.216+0000 7f697dffb640 1 -- 172.21.15.47:0/2236730144 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f697009cb30 con 0x7f69801011b0 2024-04-17T06:13:35.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:13:34 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/4257291614' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T06:13:35.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.402+0000 7f6987074640 1 -- 172.21.15.47:0/2236730144 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f6980108dc0 con 0x7f69801011b0 2024-04-17T06:13:35.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.402+0000 7f697dffb640 1 -- 172.21.15.47:0/2236730144 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v106) ==== 104+0+11 (secure 0 0 0) 0x7f69700a0e30 con 0x7f69801011b0 2024-04-17T06:13:35.400 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-04-17T06:13:35.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.404+0000 7f6987074640 1 -- 172.21.15.47:0/2236730144 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f69680a3d20 msgr2=0x7f69680a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:13:35.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.404+0000 7f6987074640 1 --2- 172.21.15.47:0/2236730144 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f69680a3d20 0x7f69680a61e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f698006b0b0 tx=0x7f6974009680 comp rx=0 tx=0).stop 2024-04-17T06:13:35.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.404+0000 7f6987074640 1 -- 172.21.15.47:0/2236730144 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f69801011b0 msgr2=0x7f6980199660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:13:35.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.404+0000 7f6987074640 1 --2- 172.21.15.47:0/2236730144 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f69801011b0 0x7f6980199660 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f697000e9f0 tx=0x7f697000eeb0 comp rx=0 tx=0).stop 2024-04-17T06:13:35.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.404+0000 7f6987074640 1 -- 172.21.15.47:0/2236730144 shutdown_connections 2024-04-17T06:13:35.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.404+0000 7f6987074640 1 --2- 172.21.15.47:0/2236730144 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6980107040 0x7f69801a46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:13:35.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.404+0000 7f6987074640 1 --2- 172.21.15.47:0/2236730144 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6980101b50 0x7f6980199ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:13:35.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.404+0000 7f6987074640 1 --2- 172.21.15.47:0/2236730144 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f69680a3d20 0x7f69680a61e0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:13:35.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.404+0000 7f6987074640 1 --2- 172.21.15.47:0/2236730144 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f69801011b0 0x7f6980199660 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:13:35.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.404+0000 7f6987074640 1 -- 172.21.15.47:0/2236730144 >> 172.21.15.47:0/2236730144 conn(0x7f6980076c60 msgr2=0x7f69800ffdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:13:35.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.404+0000 7f6987074640 1 -- 172.21.15.47:0/2236730144 shutdown_connections 2024-04-17T06:13:35.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.404+0000 7f6987074640 1 -- 172.21.15.47:0/2236730144 wait complete. 2024-04-17T06:13:35.413 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T06:13:35.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.524+0000 7fb245ec8640 1 -- 172.21.15.47:0/2098071760 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb24006b180 msgr2=0x7fb240106bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:13:35.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.524+0000 7fb245ec8640 1 --2- 172.21.15.47:0/2098071760 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb24006b180 0x7fb240106bb0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fb234009900 tx=0x7fb23402bc30 comp rx=0 tx=0).stop 2024-04-17T06:13:35.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.524+0000 7fb245ec8640 1 -- 172.21.15.47:0/2098071760 shutdown_connections 2024-04-17T06:13:35.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.524+0000 7fb245ec8640 1 --2- 172.21.15.47:0/2098071760 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb240107290 0x7fb24010ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:13:35.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.524+0000 7fb245ec8640 1 --2- 172.21.15.47:0/2098071760 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb24006b180 0x7fb240106bb0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:13:35.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.524+0000 7fb245ec8640 1 --2- 172.21.15.47:0/2098071760 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb24006a7d0 0x7fb24006abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:13:35.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.524+0000 7fb245ec8640 1 -- 172.21.15.47:0/2098071760 >> 172.21.15.47:0/2098071760 conn(0x7fb240076c60 msgr2=0x7fb240077070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:13:35.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.524+0000 7fb245ec8640 1 -- 172.21.15.47:0/2098071760 shutdown_connections 2024-04-17T06:13:35.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.524+0000 7fb245ec8640 1 -- 172.21.15.47:0/2098071760 wait complete. 2024-04-17T06:13:35.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.525+0000 7fb245ec8640 1 Processor -- start 2024-04-17T06:13:35.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.525+0000 7fb245ec8640 1 -- start start 2024-04-17T06:13:35.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.525+0000 7fb245ec8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb24006a7d0 0x7fb24019e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:13:35.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.525+0000 7fb245ec8640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb240107290 0x7fb24019e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:13:35.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.525+0000 7fb245ec8640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb2401985e0 0x7fb240198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:13:35.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.525+0000 7fb245ec8640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb240112300 con 0x7fb240107290 2024-04-17T06:13:35.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.525+0000 7fb245ec8640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb240112180 con 0x7fb2401985e0 2024-04-17T06:13:35.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.525+0000 7fb245ec8640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb240112480 con 0x7fb24006a7d0 2024-04-17T06:13:35.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.525+0000 7fb23effd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb240107290 0x7fb24019e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:13:35.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.525+0000 7fb23effd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb240107290 0x7fb24019e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46666/0 (socket says 172.21.15.47:46666) 2024-04-17T06:13:35.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.525+0000 7fb23effd640 1 -- 172.21.15.47:0/19212245 learned_addr learned my addr 172.21.15.47:0/19212245 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:13:35.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.525+0000 7fb23ffff640 1 --2- 172.21.15.47:0/19212245 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb2401985e0 0x7fb240198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:13:35.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.525+0000 7fb23f7fe640 1 --2- 172.21.15.47:0/19212245 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb24006a7d0 0x7fb24019e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:13:35.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.525+0000 7fb23effd640 1 -- 172.21.15.47:0/19212245 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb24006a7d0 msgr2=0x7fb24019e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:13:35.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.525+0000 7fb23effd640 1 --2- 172.21.15.47:0/19212245 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb24006a7d0 0x7fb24019e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:13:35.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.525+0000 7fb23effd640 1 -- 172.21.15.47:0/19212245 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb2401985e0 msgr2=0x7fb240198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:13:35.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.525+0000 7fb23effd640 1 --2- 172.21.15.47:0/19212245 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb2401985e0 0x7fb240198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:13:35.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.525+0000 7fb23effd640 1 -- 172.21.15.47:0/19212245 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb240199100 con 0x7fb240107290 2024-04-17T06:13:35.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.526+0000 7fb23effd640 1 --2- 172.21.15.47:0/19212245 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb240107290 0x7fb24019e970 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fb234036040 tx=0x7fb23402cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:13:35.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.526+0000 7fb23cff9640 1 -- 172.21.15.47:0/19212245 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb234041070 con 0x7fb240107290 2024-04-17T06:13:35.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.526+0000 7fb245ec8640 1 -- 172.21.15.47:0/19212245 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb240199390 con 0x7fb240107290 2024-04-17T06:13:35.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.526+0000 7fb23cff9640 1 -- 172.21.15.47:0/19212245 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb234004450 con 0x7fb240107290 2024-04-17T06:13:35.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.526+0000 7fb23cff9640 1 -- 172.21.15.47:0/19212245 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb234003e50 con 0x7fb240107290 2024-04-17T06:13:35.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.526+0000 7fb245ec8640 1 -- 172.21.15.47:0/19212245 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2401a55e0 con 0x7fb240107290 2024-04-17T06:13:35.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.527+0000 7fb245ec8640 1 -- 172.21.15.47:0/19212245 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb240103690 con 0x7fb240107290 2024-04-17T06:13:35.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.528+0000 7fb23cff9640 1 -- 172.21.15.47:0/19212245 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb234044070 con 0x7fb240107290 2024-04-17T06:13:35.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.529+0000 7fb23cff9640 1 --2- 172.21.15.47:0/19212245 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fb2280a3ec0 0x7fb2280a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:13:35.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.529+0000 7fb23f7fe640 1 --2- 172.21.15.47:0/19212245 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fb2280a3ec0 0x7fb2280a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:13:35.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.529+0000 7fb23cff9640 1 -- 172.21.15.47:0/19212245 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8513+0+0 (secure 0 0 0) 0x7fb2340f2650 con 0x7fb240107290 2024-04-17T06:13:35.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.529+0000 7fb23f7fe640 1 --2- 172.21.15.47:0/19212245 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fb2280a3ec0 0x7fb2280a6380 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb230005fd0 tx=0x7fb230005850 comp rx=0 tx=0).ready entity=mgr.24923 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:13:35.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.533+0000 7fb23cff9640 1 -- 172.21.15.47:0/19212245 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fb2340bee00 con 0x7fb240107290 2024-04-17T06:13:35.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.747+0000 7fb245ec8640 1 -- 172.21.15.47:0/19212245 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb24010a4c0 con 0x7fb240107290 2024-04-17T06:13:35.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.747+0000 7fb23cff9640 1 -- 172.21.15.47:0/19212245 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v106) ==== 100+0+11 (secure 0 0 0) 0x7fb2340be700 con 0x7fb240107290 2024-04-17T06:13:35.745 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-04-17T06:13:35.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.749+0000 7fb245ec8640 1 -- 172.21.15.47:0/19212245 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fb2280a3ec0 msgr2=0x7fb2280a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:13:35.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.749+0000 7fb245ec8640 1 --2- 172.21.15.47:0/19212245 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fb2280a3ec0 0x7fb2280a6380 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb230005fd0 tx=0x7fb230005850 comp rx=0 tx=0).stop 2024-04-17T06:13:35.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.749+0000 7fb245ec8640 1 -- 172.21.15.47:0/19212245 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb240107290 msgr2=0x7fb24019e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:13:35.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.749+0000 7fb245ec8640 1 --2- 172.21.15.47:0/19212245 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb240107290 0x7fb24019e970 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fb234036040 tx=0x7fb23402cbf0 comp rx=0 tx=0).stop 2024-04-17T06:13:35.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.749+0000 7fb245ec8640 1 -- 172.21.15.47:0/19212245 shutdown_connections 2024-04-17T06:13:35.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.749+0000 7fb245ec8640 1 --2- 172.21.15.47:0/19212245 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb2401985e0 0x7fb240198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:13:35.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.749+0000 7fb245ec8640 1 --2- 172.21.15.47:0/19212245 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fb2280a3ec0 0x7fb2280a6380 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:13:35.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.749+0000 7fb245ec8640 1 --2- 172.21.15.47:0/19212245 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb240107290 0x7fb24019e970 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:13:35.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.749+0000 7fb245ec8640 1 --2- 172.21.15.47:0/19212245 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb24006a7d0 0x7fb24019e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:13:35.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.749+0000 7fb245ec8640 1 -- 172.21.15.47:0/19212245 >> 172.21.15.47:0/19212245 conn(0x7fb240076c60 msgr2=0x7fb2400ff2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:13:35.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.749+0000 7fb245ec8640 1 -- 172.21.15.47:0/19212245 shutdown_connections 2024-04-17T06:13:35.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.749+0000 7fb245ec8640 1 -- 172.21.15.47:0/19212245 wait complete. 2024-04-17T06:13:35.759 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-17T06:13:35.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.871+0000 7fcad8111640 1 -- 172.21.15.47:0/1621933572 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcad01031e0 msgr2=0x7fcad01035c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:13:35.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.871+0000 7fcad8111640 1 --2- 172.21.15.47:0/1621933572 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcad01031e0 0x7fcad01035c0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fcacc009900 tx=0x7fcacc02bd60 comp rx=0 tx=0).stop 2024-04-17T06:13:35.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.871+0000 7fcad8111640 1 -- 172.21.15.47:0/1621933572 shutdown_connections 2024-04-17T06:13:35.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.871+0000 7fcad8111640 1 --2- 172.21.15.47:0/1621933572 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcad010c520 0x7fcad010e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:13:35.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.871+0000 7fcad8111640 1 --2- 172.21.15.47:0/1621933572 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcad0103b00 0x7fcad010bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:13:35.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.871+0000 7fcad8111640 1 --2- 172.21.15.47:0/1621933572 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcad01031e0 0x7fcad01035c0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:13:35.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.871+0000 7fcad8111640 1 -- 172.21.15.47:0/1621933572 >> 172.21.15.47:0/1621933572 conn(0x7fcad0077f60 msgr2=0x7fcad0078370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:13:35.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.871+0000 7fcad8111640 1 -- 172.21.15.47:0/1621933572 shutdown_connections 2024-04-17T06:13:35.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.871+0000 7fcad8111640 1 -- 172.21.15.47:0/1621933572 wait complete. 2024-04-17T06:13:35.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.872+0000 7fcad8111640 1 Processor -- start 2024-04-17T06:13:35.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.872+0000 7fcad8111640 1 -- start start 2024-04-17T06:13:35.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.872+0000 7fcad8111640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcad01031e0 0x7fcad01a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:13:35.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.872+0000 7fcad8111640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcad0103b00 0x7fcad01a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:13:35.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.872+0000 7fcad8111640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcad010c520 0x7fcad019a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:13:35.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.872+0000 7fcad8111640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fcad0112330 con 0x7fcad010c520 2024-04-17T06:13:35.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.872+0000 7fcad8111640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fcad01121b0 con 0x7fcad0103b00 2024-04-17T06:13:35.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.872+0000 7fcad8111640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcad01124b0 con 0x7fcad01031e0 2024-04-17T06:13:35.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.872+0000 7fcad66ae640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcad010c520 0x7fcad019a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:13:35.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.872+0000 7fcad66ae640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcad010c520 0x7fcad019a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46688/0 (socket says 172.21.15.47:46688) 2024-04-17T06:13:35.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.872+0000 7fcad5ead640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcad01031e0 0x7fcad01a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:13:35.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.872+0000 7fcad66ae640 1 -- 172.21.15.47:0/917258857 learned_addr learned my addr 172.21.15.47:0/917258857 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:13:35.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.872+0000 7fcad56ac640 1 --2- 172.21.15.47:0/917258857 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcad0103b00 0x7fcad01a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:13:35.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.872+0000 7fcad66ae640 1 -- 172.21.15.47:0/917258857 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcad01031e0 msgr2=0x7fcad01a06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:13:35.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.872+0000 7fcad66ae640 1 --2- 172.21.15.47:0/917258857 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcad01031e0 0x7fcad01a06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:13:35.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.872+0000 7fcad66ae640 1 -- 172.21.15.47:0/917258857 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcad0103b00 msgr2=0x7fcad01a0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:13:35.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.872+0000 7fcad66ae640 1 --2- 172.21.15.47:0/917258857 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcad0103b00 0x7fcad01a0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:13:35.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.872+0000 7fcad66ae640 1 -- 172.21.15.47:0/917258857 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcad019aff0 con 0x7fcad010c520 2024-04-17T06:13:35.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.872+0000 7fcad56ac640 1 --2- 172.21.15.47:0/917258857 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcad0103b00 0x7fcad01a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T06:13:35.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.873+0000 7fcad66ae640 1 --2- 172.21.15.47:0/917258857 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcad010c520 0x7fcad019a820 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fcac800eaa0 tx=0x7fcac800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:13:35.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.873+0000 7fcac6ffd640 1 -- 172.21.15.47:0/917258857 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac800cdb0 con 0x7fcad010c520 2024-04-17T06:13:35.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.873+0000 7fcac6ffd640 1 -- 172.21.15.47:0/917258857 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcac8005020 con 0x7fcad010c520 2024-04-17T06:13:35.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.873+0000 7fcad8111640 1 -- 172.21.15.47:0/917258857 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcad019b2e0 con 0x7fcad010c520 2024-04-17T06:13:35.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.873+0000 7fcac6ffd640 1 -- 172.21.15.47:0/917258857 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac80116b0 con 0x7fcad010c520 2024-04-17T06:13:35.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.873+0000 7fcad8111640 1 -- 172.21.15.47:0/917258857 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcad01a7500 con 0x7fcad010c520 2024-04-17T06:13:35.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.874+0000 7fcad8111640 1 -- 172.21.15.47:0/917258857 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcad00698f0 con 0x7fcad010c520 2024-04-17T06:13:35.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.875+0000 7fcac6ffd640 1 -- 172.21.15.47:0/917258857 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fcac8005560 con 0x7fcad010c520 2024-04-17T06:13:35.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.876+0000 7fcac6ffd640 1 --2- 172.21.15.47:0/917258857 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fcabc0a3d20 0x7fcabc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:13:35.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.876+0000 7fcad5ead640 1 --2- 172.21.15.47:0/917258857 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fcabc0a3d20 0x7fcabc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:13:35.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.876+0000 7fcac6ffd640 1 -- 172.21.15.47:0/917258857 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8513+0+0 (secure 0 0 0) 0x7fcac80d0d00 con 0x7fcad010c520 2024-04-17T06:13:35.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.876+0000 7fcad5ead640 1 --2- 172.21.15.47:0/917258857 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fcabc0a3d20 0x7fcabc0a61e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fcacc000c00 tx=0x7fcacc0058d0 comp rx=0 tx=0).ready entity=mgr.24923 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:13:35.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:35.880+0000 7fcac6ffd640 1 -- 172.21.15.47:0/917258857 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fcac8016030 con 0x7fcad010c520 2024-04-17T06:13:36.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:36.081+0000 7fcad8111640 1 -- 172.21.15.47:0/917258857 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fcad01a7800 con 0x7fcad010c520 2024-04-17T06:13:36.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:36.082+0000 7fcac6ffd640 1 -- 172.21.15.47:0/917258857 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v106) ==== 81+0+942 (secure 0 0 0) 0x7fcac809cd50 con 0x7fcad010c520 2024-04-17T06:13:36.080 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:6800/3157902806 2024-04-18T06:01:18.246671+0000 2024-04-17T06:13:36.080 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:6801/3157902806 2024-04-18T06:01:18.246671+0000 2024-04-17T06:13:36.081 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:0/3681048987 2024-04-18T06:01:18.246671+0000 2024-04-17T06:13:36.081 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:0/3009926768 2024-04-18T06:01:18.246671+0000 2024-04-17T06:13:36.081 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:0/2970926607 2024-04-18T06:01:18.246671+0000 2024-04-17T06:13:36.081 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:6801/3632157175 2024-04-18T05:52:13.645775+0000 2024-04-17T06:13:36.081 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:0/4288219269 2024-04-18T05:52:13.645775+0000 2024-04-17T06:13:36.081 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:0/2584487544 2024-04-18T05:52:13.645775+0000 2024-04-17T06:13:36.081 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:6801/953064786 2024-04-18T05:51:45.979351+0000 2024-04-17T06:13:36.081 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:0/3484500927 2024-04-18T05:51:45.979351+0000 2024-04-17T06:13:36.081 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:6800/3632157175 2024-04-18T05:52:13.645775+0000 2024-04-17T06:13:36.081 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:0/112268670 2024-04-18T05:51:45.979351+0000 2024-04-17T06:13:36.082 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:0/1638423256 2024-04-18T05:51:45.979351+0000 2024-04-17T06:13:36.082 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:0/3275066738 2024-04-18T06:01:18.246671+0000 2024-04-17T06:13:36.082 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:6800/953064786 2024-04-18T05:51:45.979351+0000 2024-04-17T06:13:36.082 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:0/849479010 2024-04-18T05:52:13.645775+0000 2024-04-17T06:13:36.083 INFO:teuthology.orchestra.run.smithi047.stderr:listed 16 entries 2024-04-17T06:13:36.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:36.084+0000 7fcad8111640 1 -- 172.21.15.47:0/917258857 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fcabc0a3d20 msgr2=0x7fcabc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:13:36.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:36.084+0000 7fcad8111640 1 --2- 172.21.15.47:0/917258857 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fcabc0a3d20 0x7fcabc0a61e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fcacc000c00 tx=0x7fcacc0058d0 comp rx=0 tx=0).stop 2024-04-17T06:13:36.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:36.084+0000 7fcad8111640 1 -- 172.21.15.47:0/917258857 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcad010c520 msgr2=0x7fcad019a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:13:36.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:36.084+0000 7fcad8111640 1 --2- 172.21.15.47:0/917258857 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcad010c520 0x7fcad019a820 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fcac800eaa0 tx=0x7fcac800ef60 comp rx=0 tx=0).stop 2024-04-17T06:13:36.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:36.084+0000 7fcad8111640 1 -- 172.21.15.47:0/917258857 shutdown_connections 2024-04-17T06:13:36.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:36.084+0000 7fcad8111640 1 --2- 172.21.15.47:0/917258857 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fcabc0a3d20 0x7fcabc0a61e0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:13:36.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:36.084+0000 7fcad8111640 1 --2- 172.21.15.47:0/917258857 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcad010c520 0x7fcad019a820 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:13:36.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:36.084+0000 7fcad8111640 1 --2- 172.21.15.47:0/917258857 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcad0103b00 0x7fcad01a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:13:36.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:36.084+0000 7fcad8111640 1 --2- 172.21.15.47:0/917258857 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcad01031e0 0x7fcad01a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:13:36.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:36.084+0000 7fcad8111640 1 -- 172.21.15.47:0/917258857 >> 172.21.15.47:0/917258857 conn(0x7fcad0077f60 msgr2=0x7fcad0108430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:13:36.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:36.084+0000 7fcad8111640 1 -- 172.21.15.47:0/917258857 shutdown_connections 2024-04-17T06:13:36.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:36.084+0000 7fcad8111640 1 -- 172.21.15.47:0/917258857 wait complete. 2024-04-17T06:13:36.092 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:13:35 smithi047 ceph-mon[32223]: pgmap v388: 97 pgs: 97 active+clean; 5.9 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 853 B/s wr, 8 op/s 2024-04-17T06:13:36.093 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:13:35 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/3992790273' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-17T06:13:36.093 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:13:35 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/2236730144' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-17T06:13:36.093 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:13:35 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T06:13:36.093 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:13:35 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/19212245' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-17T06:13:36.094 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-17T06:13:36.095 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T06:13:36.095 DEBUG:teuthology.orchestra.run.smithi047:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-17T06:13:36.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:13:35 smithi098 ceph-mon[36516]: pgmap v388: 97 pgs: 97 active+clean; 5.9 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 853 B/s wr, 8 op/s 2024-04-17T06:13:36.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:13:35 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/3992790273' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-17T06:13:36.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:13:35 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/2236730144' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-17T06:13:36.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:13:35 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T06:13:36.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:13:35 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/19212245' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-17T06:13:36.181 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi047.front.sepia.ceph.com 2024-04-17T06:13:36.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T06:13:36.181 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-17T06:13:36.207 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T06:13:36.207 DEBUG:teuthology.orchestra.run.smithi047:> ip netns list 2024-04-17T06:13:36.264 INFO:teuthology.orchestra.run.smithi047.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-17T06:13:36.264 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T06:13:36.264 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T06:13:36.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:13:35 smithi103 ceph-mon[36243]: pgmap v388: 97 pgs: 97 active+clean; 5.9 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 853 B/s wr, 8 op/s 2024-04-17T06:13:36.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:13:35 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/3992790273' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-17T06:13:36.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:13:35 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/2236730144' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-17T06:13:36.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:13:35 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T06:13:36.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:13:35 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/19212245' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-17T06:13:36.334 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T06:13:36.334 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link delete ceph-brx 2024-04-17T06:13:36.436 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-04-17T06:13:36.449 INFO:tasks.cephadm:Teardown begin 2024-04-17T06:13:36.449 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-17T06:13:36.449 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T06:13:36.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:36.559+0000 7fc5528e4640 1 -- 172.21.15.47:0/2144796120 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc54c1010b0 msgr2=0x7fc54c101490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:13:36.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:36.559+0000 7fc5528e4640 1 --2- 172.21.15.47:0/2144796120 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc54c1010b0 0x7fc54c101490 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fc53c009900 tx=0x7fc53c02bc30 comp rx=0 tx=0).stop 2024-04-17T06:13:36.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:36.560+0000 7fc5528e4640 1 -- 172.21.15.47:0/2144796120 shutdown_connections 2024-04-17T06:13:36.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:36.560+0000 7fc5528e4640 1 --2- 172.21.15.47:0/2144796120 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc54c106180 0x7fc54c10dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:13:36.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:36.560+0000 7fc5528e4640 1 --2- 172.21.15.47:0/2144796120 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc54c101a60 0x7fc54c105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:13:36.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:36.560+0000 7fc5528e4640 1 --2- 172.21.15.47:0/2144796120 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc54c1010b0 0x7fc54c101490 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:13:36.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:36.560+0000 7fc5528e4640 1 -- 172.21.15.47:0/2144796120 >> 172.21.15.47:0/2144796120 conn(0x7fc54c077f60 msgr2=0x7fc54c078370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:13:36.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:36.560+0000 7fc5528e4640 1 -- 172.21.15.47:0/2144796120 shutdown_connections 2024-04-17T06:13:36.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:36.560+0000 7fc5528e4640 1 -- 172.21.15.47:0/2144796120 wait complete. 2024-04-17T06:13:36.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:36.560+0000 7fc5528e4640 1 Processor -- start 2024-04-17T06:13:36.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:36.560+0000 7fc5528e4640 1 -- start start 2024-04-17T06:13:36.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:36.560+0000 7fc5528e4640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc54c1010b0 0x7fc54c1976f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:13:36.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:36.560+0000 7fc5528e4640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc54c106180 0x7fc54c197c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:13:36.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:36.560+0000 7fc5528e4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc54c19bfb0 0x7fc54c19c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:13:36.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:36.560+0000 7fc5528e4640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fc54c111210 con 0x7fc54c1010b0 2024-04-17T06:13:36.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:36.560+0000 7fc5528e4640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc54c111090 con 0x7fc54c106180 2024-04-17T06:13:36.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:36.560+0000 7fc5528e4640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc54c111390 con 0x7fc54c19bfb0 2024-04-17T06:13:36.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:36.561+0000 7fc54bfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc54c1010b0 0x7fc54c1976f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:13:36.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:36.561+0000 7fc54bfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc54c1010b0 0x7fc54c1976f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46698/0 (socket says 172.21.15.47:46698) 2024-04-17T06:13:36.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:36.561+0000 7fc54bfff640 1 -- 172.21.15.47:0/2589861122 learned_addr learned my addr 172.21.15.47:0/2589861122 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:13:36.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:36.561+0000 7fc550e81640 1 --2- 172.21.15.47:0/2589861122 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc54c19bfb0 0x7fc54c19c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:13:36.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:36.561+0000 7fc543fff640 1 --2- 172.21.15.47:0/2589861122 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc54c106180 0x7fc54c197c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:13:36.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:36.561+0000 7fc54bfff640 1 -- 172.21.15.47:0/2589861122 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc54c19bfb0 msgr2=0x7fc54c19c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:13:36.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:36.561+0000 7fc54bfff640 1 --2- 172.21.15.47:0/2589861122 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc54c19bfb0 0x7fc54c19c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:13:36.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:36.561+0000 7fc54bfff640 1 -- 172.21.15.47:0/2589861122 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc54c106180 msgr2=0x7fc54c197c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:13:36.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:36.561+0000 7fc54bfff640 1 --2- 172.21.15.47:0/2589861122 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc54c106180 0x7fc54c197c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:13:36.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:36.561+0000 7fc54bfff640 1 -- 172.21.15.47:0/2589861122 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc54c19caa0 con 0x7fc54c1010b0 2024-04-17T06:13:36.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:36.561+0000 7fc54bfff640 1 --2- 172.21.15.47:0/2589861122 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc54c1010b0 0x7fc54c1976f0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fc53c006010 tx=0x7fc53c02cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:13:36.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:36.561+0000 7fc549ffb640 1 -- 172.21.15.47:0/2589861122 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc53c041070 con 0x7fc54c1010b0 2024-04-17T06:13:36.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:36.561+0000 7fc549ffb640 1 -- 172.21.15.47:0/2589861122 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc53c004450 con 0x7fc54c1010b0 2024-04-17T06:13:36.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:36.561+0000 7fc5528e4640 1 -- 172.21.15.47:0/2589861122 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc54c19cd30 con 0x7fc54c1010b0 2024-04-17T06:13:36.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:36.561+0000 7fc549ffb640 1 -- 172.21.15.47:0/2589861122 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc53c034440 con 0x7fc54c1010b0 2024-04-17T06:13:36.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:36.561+0000 7fc5528e4640 1 -- 172.21.15.47:0/2589861122 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc54c1a4550 con 0x7fc54c1010b0 2024-04-17T06:13:36.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:36.562+0000 7fc5528e4640 1 -- 172.21.15.47:0/2589861122 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc54c068d80 con 0x7fc54c1010b0 2024-04-17T06:13:36.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:36.564+0000 7fc549ffb640 1 -- 172.21.15.47:0/2589861122 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc53c044070 con 0x7fc54c1010b0 2024-04-17T06:13:36.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:36.564+0000 7fc549ffb640 1 --2- 172.21.15.47:0/2589861122 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fc5380a3da0 0x7fc5380a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:13:36.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:36.564+0000 7fc543fff640 1 --2- 172.21.15.47:0/2589861122 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fc5380a3da0 0x7fc5380a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:13:36.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:36.564+0000 7fc549ffb640 1 -- 172.21.15.47:0/2589861122 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8513+0+0 (secure 0 0 0) 0x7fc53c0f2430 con 0x7fc54c1010b0 2024-04-17T06:13:36.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:36.565+0000 7fc543fff640 1 --2- 172.21.15.47:0/2589861122 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fc5380a3da0 0x7fc5380a6260 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc54c100980 tx=0x7fc53400b410 comp rx=0 tx=0).ready entity=mgr.24923 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:13:36.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:36.568+0000 7fc549ffb640 1 -- 172.21.15.47:0/2589861122 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fc53c0be590 con 0x7fc54c1010b0 2024-04-17T06:13:36.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:36.793+0000 7fc5528e4640 1 -- 172.21.15.47:0/2589861122 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc54c198a40 con 0x7fc54c1010b0 2024-04-17T06:13:36.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:36.794+0000 7fc549ffb640 1 -- 172.21.15.47:0/2589861122 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+11883 (secure 0 0 0) 0x7fc54c198a40 con 0x7fc54c1010b0 2024-04-17T06:13:36.792 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-17T06:13:36.792 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3189958720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3189958720},{"type":"v1","addr":"172.21.15.47:6835","nonce":3189958720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14595,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/3143529973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":3143529973},{"type":"v1","addr":"172.21.15.47:6837","nonce":3143529973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14652,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/2375823701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":2375823701},{"type":"v1","addr":"172.21.15.47:6839","nonce":2375823701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14664,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6841/3334309157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":3334309157},{"type":"v1","addr":"172.21.15.47:6841","nonce":3334309157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/2398363754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":2398363754},{"type":"v1","addr":"172.21.15.103:6833","nonce":2398363754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":25304,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/194247764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":194247764},{"type":"v1","addr":"172.21.15.98:6835","nonce":194247764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":37,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T05:59:13.922176+0000","modified":"2024-04-17T06:12:59.955751+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24518,"mds_1":24479,"mds_2":24466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6835/270543522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":270543522},{"type":"v1","addr":"172.21.15.103:6835","nonce":270543522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6837/116936024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":116936024},{"type":"v1","addr":"172.21.15.103:6837","nonce":116936024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.103:6839/1467157889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":1467157889},{"type":"v1","addr":"172.21.15.103:6839","nonce":1467157889}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.98:6837/1117440719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1117440719},{"type":"v1","addr":"172.21.15.98:6837","nonce":1117440719}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6839/2841287917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":2841287917},{"type":"v1","addr":"172.21.15.98:6839","nonce":2841287917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25688":{"gid":25688,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6833/4230863674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":4230863674},{"type":"v1","addr":"172.21.15.98:6833","nonce":4230863674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24518,"qdb_cluster":[24466,24479,24518]},"id":1}]} 2024-04-17T06:13:36.792 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 37 2024-04-17T06:13:36.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:36.796+0000 7fc5528e4640 1 -- 172.21.15.47:0/2589861122 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fc5380a3da0 msgr2=0x7fc5380a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:13:36.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:36.796+0000 7fc5528e4640 1 --2- 172.21.15.47:0/2589861122 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fc5380a3da0 0x7fc5380a6260 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc54c100980 tx=0x7fc53400b410 comp rx=0 tx=0).stop 2024-04-17T06:13:36.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:36.796+0000 7fc5528e4640 1 -- 172.21.15.47:0/2589861122 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc54c1010b0 msgr2=0x7fc54c1976f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:13:36.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:36.796+0000 7fc5528e4640 1 --2- 172.21.15.47:0/2589861122 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc54c1010b0 0x7fc54c1976f0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fc53c006010 tx=0x7fc53c02cc70 comp rx=0 tx=0).stop 2024-04-17T06:13:36.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:36.796+0000 7fc5528e4640 1 -- 172.21.15.47:0/2589861122 shutdown_connections 2024-04-17T06:13:36.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:36.796+0000 7fc5528e4640 1 --2- 172.21.15.47:0/2589861122 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc54c19bfb0 0x7fc54c19c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:13:36.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:36.796+0000 7fc5528e4640 1 --2- 172.21.15.47:0/2589861122 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc54c106180 0x7fc54c197c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:13:36.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:36.796+0000 7fc5528e4640 1 --2- 172.21.15.47:0/2589861122 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fc5380a3da0 0x7fc5380a6260 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:13:36.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:36.796+0000 7fc5528e4640 1 --2- 172.21.15.47:0/2589861122 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc54c1010b0 0x7fc54c1976f0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:13:36.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:36.796+0000 7fc5528e4640 1 -- 172.21.15.47:0/2589861122 >> 172.21.15.47:0/2589861122 conn(0x7fc54c077f60 msgr2=0x7fc54c0fefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:13:36.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:36.796+0000 7fc5528e4640 1 -- 172.21.15.47:0/2589861122 shutdown_connections 2024-04-17T06:13:36.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:36.796+0000 7fc5528e4640 1 -- 172.21.15.47:0/2589861122 wait complete. 2024-04-17T06:13:36.807 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T06:13:36.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:36.917+0000 7fa08e2ce640 1 -- 172.21.15.47:0/4270947623 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa088101a60 msgr2=0x7fa088105aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:13:36.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:36.917+0000 7fa08e2ce640 1 --2- 172.21.15.47:0/4270947623 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa088101a60 0x7fa088105aa0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fa07c009900 tx=0x7fa07c02bc30 comp rx=0 tx=0).stop 2024-04-17T06:13:36.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:36.917+0000 7fa08e2ce640 1 -- 172.21.15.47:0/4270947623 shutdown_connections 2024-04-17T06:13:36.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:36.917+0000 7fa08e2ce640 1 --2- 172.21.15.47:0/4270947623 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa088106180 0x7fa08810dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:13:36.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:36.917+0000 7fa08e2ce640 1 --2- 172.21.15.47:0/4270947623 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa088101a60 0x7fa088105aa0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:13:36.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:36.917+0000 7fa08e2ce640 1 --2- 172.21.15.47:0/4270947623 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa0881010b0 0x7fa088101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:13:36.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:36.917+0000 7fa08e2ce640 1 -- 172.21.15.47:0/4270947623 >> 172.21.15.47:0/4270947623 conn(0x7fa088077f60 msgr2=0x7fa088078370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:13:36.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:36.917+0000 7fa08e2ce640 1 -- 172.21.15.47:0/4270947623 shutdown_connections 2024-04-17T06:13:36.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:36.917+0000 7fa08e2ce640 1 -- 172.21.15.47:0/4270947623 wait complete. 2024-04-17T06:13:36.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:36.917+0000 7fa08e2ce640 1 Processor -- start 2024-04-17T06:13:36.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:36.917+0000 7fa08e2ce640 1 -- start start 2024-04-17T06:13:36.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:36.918+0000 7fa08e2ce640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa0881010b0 0x7fa088102b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:13:36.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:36.918+0000 7fa08e2ce640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa088106180 0x7fa088103080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:13:36.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:36.918+0000 7fa08e2ce640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa0881035c0 0x7fa0881a6a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:13:36.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:36.918+0000 7fa08e2ce640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa088111210 con 0x7fa088106180 2024-04-17T06:13:36.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:36.918+0000 7fa08e2ce640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa088111090 con 0x7fa0881035c0 2024-04-17T06:13:36.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:36.918+0000 7fa08e2ce640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa088111390 con 0x7fa0881010b0 2024-04-17T06:13:36.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:36.918+0000 7fa0877fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa088106180 0x7fa088103080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:13:36.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:36.918+0000 7fa0877fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa088106180 0x7fa088103080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46720/0 (socket says 172.21.15.47:46720) 2024-04-17T06:13:36.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:36.918+0000 7fa0877fe640 1 -- 172.21.15.47:0/3147758023 learned_addr learned my addr 172.21.15.47:0/3147758023 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:13:36.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:36.918+0000 7fa08c86b640 1 --2- 172.21.15.47:0/3147758023 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa0881035c0 0x7fa0881a6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:13:36.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:36.918+0000 7fa087fff640 1 --2- 172.21.15.47:0/3147758023 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa0881010b0 0x7fa088102b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:13:36.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:36.918+0000 7fa0877fe640 1 -- 172.21.15.47:0/3147758023 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa0881010b0 msgr2=0x7fa088102b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:13:36.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:36.918+0000 7fa0877fe640 1 --2- 172.21.15.47:0/3147758023 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa0881010b0 0x7fa088102b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:13:36.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:36.918+0000 7fa0877fe640 1 -- 172.21.15.47:0/3147758023 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa0881035c0 msgr2=0x7fa0881a6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:13:36.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:36.918+0000 7fa0877fe640 1 --2- 172.21.15.47:0/3147758023 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa0881035c0 0x7fa0881a6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:13:36.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:36.918+0000 7fa0877fe640 1 -- 172.21.15.47:0/3147758023 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0881a7110 con 0x7fa088106180 2024-04-17T06:13:36.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:36.919+0000 7fa0877fe640 1 --2- 172.21.15.47:0/3147758023 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa088106180 0x7fa088103080 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fa07c036040 tx=0x7fa07c02cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:13:36.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:36.919+0000 7fa0857fa640 1 -- 172.21.15.47:0/3147758023 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa07c041070 con 0x7fa088106180 2024-04-17T06:13:36.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:36.919+0000 7fa08e2ce640 1 -- 172.21.15.47:0/3147758023 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0881a73a0 con 0x7fa088106180 2024-04-17T06:13:36.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:36.919+0000 7fa0857fa640 1 -- 172.21.15.47:0/3147758023 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa07c0054e0 con 0x7fa088106180 2024-04-17T06:13:36.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:36.919+0000 7fa0857fa640 1 -- 172.21.15.47:0/3147758023 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa07c034440 con 0x7fa088106180 2024-04-17T06:13:36.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:36.919+0000 7fa08e2ce640 1 -- 172.21.15.47:0/3147758023 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0881a7890 con 0x7fa088106180 2024-04-17T06:13:36.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:36.920+0000 7fa08e2ce640 1 -- 172.21.15.47:0/3147758023 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa088068d80 con 0x7fa088106180 2024-04-17T06:13:36.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:36.921+0000 7fa0857fa640 1 -- 172.21.15.47:0/3147758023 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa07c044070 con 0x7fa088106180 2024-04-17T06:13:36.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:36.922+0000 7fa0857fa640 1 --2- 172.21.15.47:0/3147758023 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fa0700a3d50 0x7fa0700a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:13:36.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:36.922+0000 7fa087fff640 1 --2- 172.21.15.47:0/3147758023 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fa0700a3d50 0x7fa0700a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:13:36.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:36.922+0000 7fa0857fa640 1 -- 172.21.15.47:0/3147758023 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8513+0+0 (secure 0 0 0) 0x7fa07c0f23e0 con 0x7fa088106180 2024-04-17T06:13:36.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:36.923+0000 7fa087fff640 1 --2- 172.21.15.47:0/3147758023 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fa0700a3d50 0x7fa0700a6210 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa07800bbe0 tx=0x7fa07800b390 comp rx=0 tx=0).ready entity=mgr.24923 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:13:36.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:36.926+0000 7fa0857fa640 1 -- 172.21.15.47:0/3147758023 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fa07c0be540 con 0x7fa088106180 2024-04-17T06:13:37.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:13:36 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/917258857' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-17T06:13:37.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:13:36 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:13:37.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:13:36 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:13:37.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:13:36 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:13:37.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:13:36 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:13:37.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:13:36 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/2589861122' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T06:13:37.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:13:36 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/917258857' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-17T06:13:37.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:13:36 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:13:37.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:13:36 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:13:37.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:13:36 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:13:37.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:13:36 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:13:37.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:13:36 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/2589861122' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T06:13:37.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.153+0000 7fa08e2ce640 1 -- 172.21.15.47:0/3147758023 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa08810a3c0 con 0x7fa088106180 2024-04-17T06:13:37.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.154+0000 7fa0857fa640 1 -- 172.21.15.47:0/3147758023 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+11883 (secure 0 0 0) 0x7fa07c0c2840 con 0x7fa088106180 2024-04-17T06:13:37.153 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-17T06:13:37.153 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3189958720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3189958720},{"type":"v1","addr":"172.21.15.47:6835","nonce":3189958720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14595,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/3143529973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":3143529973},{"type":"v1","addr":"172.21.15.47:6837","nonce":3143529973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14652,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/2375823701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":2375823701},{"type":"v1","addr":"172.21.15.47:6839","nonce":2375823701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14664,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6841/3334309157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":3334309157},{"type":"v1","addr":"172.21.15.47:6841","nonce":3334309157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/2398363754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":2398363754},{"type":"v1","addr":"172.21.15.103:6833","nonce":2398363754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":25304,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/194247764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":194247764},{"type":"v1","addr":"172.21.15.98:6835","nonce":194247764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":37,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T05:59:13.922176+0000","modified":"2024-04-17T06:12:59.955751+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24518,"mds_1":24479,"mds_2":24466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6835/270543522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":270543522},{"type":"v1","addr":"172.21.15.103:6835","nonce":270543522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6837/116936024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":116936024},{"type":"v1","addr":"172.21.15.103:6837","nonce":116936024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.103:6839/1467157889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":1467157889},{"type":"v1","addr":"172.21.15.103:6839","nonce":1467157889}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.98:6837/1117440719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1117440719},{"type":"v1","addr":"172.21.15.98:6837","nonce":1117440719}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6839/2841287917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":2841287917},{"type":"v1","addr":"172.21.15.98:6839","nonce":2841287917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25688":{"gid":25688,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6833/4230863674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":4230863674},{"type":"v1","addr":"172.21.15.98:6833","nonce":4230863674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24518,"qdb_cluster":[24466,24479,24518]},"id":1}]} 2024-04-17T06:13:37.155 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 37 2024-04-17T06:13:37.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.156+0000 7fa08e2ce640 1 -- 172.21.15.47:0/3147758023 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fa0700a3d50 msgr2=0x7fa0700a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:13:37.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.156+0000 7fa08e2ce640 1 --2- 172.21.15.47:0/3147758023 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fa0700a3d50 0x7fa0700a6210 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa07800bbe0 tx=0x7fa07800b390 comp rx=0 tx=0).stop 2024-04-17T06:13:37.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.156+0000 7fa08e2ce640 1 -- 172.21.15.47:0/3147758023 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa088106180 msgr2=0x7fa088103080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:13:37.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.156+0000 7fa08e2ce640 1 --2- 172.21.15.47:0/3147758023 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa088106180 0x7fa088103080 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fa07c036040 tx=0x7fa07c02cbf0 comp rx=0 tx=0).stop 2024-04-17T06:13:37.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.156+0000 7fa08e2ce640 1 -- 172.21.15.47:0/3147758023 shutdown_connections 2024-04-17T06:13:37.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.156+0000 7fa08e2ce640 1 --2- 172.21.15.47:0/3147758023 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa0881035c0 0x7fa0881a6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:13:37.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.156+0000 7fa08e2ce640 1 --2- 172.21.15.47:0/3147758023 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fa0700a3d50 0x7fa0700a6210 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:13:37.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.156+0000 7fa08e2ce640 1 --2- 172.21.15.47:0/3147758023 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa088106180 0x7fa088103080 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:13:37.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.156+0000 7fa08e2ce640 1 --2- 172.21.15.47:0/3147758023 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa0881010b0 0x7fa088102b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:13:37.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.156+0000 7fa08e2ce640 1 -- 172.21.15.47:0/3147758023 >> 172.21.15.47:0/3147758023 conn(0x7fa088077f60 msgr2=0x7fa088069460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:13:37.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.156+0000 7fa08e2ce640 1 -- 172.21.15.47:0/3147758023 shutdown_connections 2024-04-17T06:13:37.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.156+0000 7fa08e2ce640 1 -- 172.21.15.47:0/3147758023 wait complete. 2024-04-17T06:13:37.166 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T06:13:37.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.275+0000 7f2c6ce0d640 1 -- 172.21.15.47:0/3460661739 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2c68106170 msgr2=0x7f2c6810dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:13:37.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.275+0000 7f2c6ce0d640 1 --2- 172.21.15.47:0/3460661739 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2c68106170 0x7f2c6810dd40 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f2c60009ca0 tx=0x7f2c6002c0f0 comp rx=0 tx=0).stop 2024-04-17T06:13:37.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.275+0000 7f2c6ce0d640 1 -- 172.21.15.47:0/3460661739 shutdown_connections 2024-04-17T06:13:37.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.275+0000 7f2c6ce0d640 1 --2- 172.21.15.47:0/3460661739 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2c68106170 0x7f2c6810dd40 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:13:37.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.275+0000 7f2c6ce0d640 1 --2- 172.21.15.47:0/3460661739 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2c68101a50 0x7f2c68105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:13:37.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.275+0000 7f2c6ce0d640 1 --2- 172.21.15.47:0/3460661739 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c681010a0 0x7f2c68101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:13:37.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.275+0000 7f2c6ce0d640 1 -- 172.21.15.47:0/3460661739 >> 172.21.15.47:0/3460661739 conn(0x7f2c680771e0 msgr2=0x7f2c680775f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:13:37.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.275+0000 7f2c6ce0d640 1 -- 172.21.15.47:0/3460661739 shutdown_connections 2024-04-17T06:13:37.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.275+0000 7f2c6ce0d640 1 -- 172.21.15.47:0/3460661739 wait complete. 2024-04-17T06:13:37.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.275+0000 7f2c6ce0d640 1 Processor -- start 2024-04-17T06:13:37.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.275+0000 7f2c6ce0d640 1 -- start start 2024-04-17T06:13:37.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.276+0000 7f2c6ce0d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2c681010a0 0x7f2c681995f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:13:37.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.276+0000 7f2c6ce0d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2c68106170 0x7f2c68199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:13:37.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.276+0000 7f2c6ce0d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c6819a110 0x7f2c681a4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:13:37.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.276+0000 7f2c6ce0d640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f2c68111210 con 0x7f2c681010a0 2024-04-17T06:13:37.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.276+0000 7f2c6ce0d640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2c68111090 con 0x7f2c6819a110 2024-04-17T06:13:37.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.276+0000 7f2c6ce0d640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2c68111390 con 0x7f2c68106170 2024-04-17T06:13:37.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.276+0000 7f2c6659c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2c681010a0 0x7f2c681995f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:13:37.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.276+0000 7f2c6659c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2c681010a0 0x7f2c681995f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46744/0 (socket says 172.21.15.47:46744) 2024-04-17T06:13:37.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.276+0000 7f2c6659c640 1 -- 172.21.15.47:0/2085290711 learned_addr learned my addr 172.21.15.47:0/2085290711 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:13:37.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.276+0000 7f2c65d9b640 1 --2- 172.21.15.47:0/2085290711 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2c68106170 0x7f2c68199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:13:37.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.276+0000 7f2c66d9d640 1 --2- 172.21.15.47:0/2085290711 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c6819a110 0x7f2c681a4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:13:37.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.276+0000 7f2c6659c640 1 -- 172.21.15.47:0/2085290711 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2c68106170 msgr2=0x7f2c68199b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:13:37.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.276+0000 7f2c6659c640 1 --2- 172.21.15.47:0/2085290711 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2c68106170 0x7f2c68199b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:13:37.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.276+0000 7f2c6659c640 1 -- 172.21.15.47:0/2085290711 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c6819a110 msgr2=0x7f2c681a4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:13:37.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.276+0000 7f2c6659c640 1 --2- 172.21.15.47:0/2085290711 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c6819a110 0x7f2c681a4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:13:37.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.276+0000 7f2c6659c640 1 -- 172.21.15.47:0/2085290711 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c681a4e90 con 0x7f2c681010a0 2024-04-17T06:13:37.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.276+0000 7f2c66d9d640 1 --2- 172.21.15.47:0/2085290711 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c6819a110 0x7f2c681a4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:13:37.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.276+0000 7f2c6659c640 1 --2- 172.21.15.47:0/2085290711 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2c681010a0 0x7f2c681995f0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f2c5800c890 tx=0x7f2c5800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:13:37.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.276+0000 7f2c577fe640 1 -- 172.21.15.47:0/2085290711 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c58004220 con 0x7f2c681010a0 2024-04-17T06:13:37.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.276+0000 7f2c6ce0d640 1 -- 172.21.15.47:0/2085290711 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c681a5180 con 0x7f2c681010a0 2024-04-17T06:13:37.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.276+0000 7f2c577fe640 1 -- 172.21.15.47:0/2085290711 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c580043c0 con 0x7f2c681010a0 2024-04-17T06:13:37.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.276+0000 7f2c577fe640 1 -- 172.21.15.47:0/2085290711 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c58004a00 con 0x7f2c681010a0 2024-04-17T06:13:37.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.277+0000 7f2c6ce0d640 1 -- 172.21.15.47:0/2085290711 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c681a56d0 con 0x7f2c681010a0 2024-04-17T06:13:37.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.277+0000 7f2c6ce0d640 1 -- 172.21.15.47:0/2085290711 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c680691d0 con 0x7f2c681010a0 2024-04-17T06:13:37.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.279+0000 7f2c577fe640 1 -- 172.21.15.47:0/2085290711 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2c580079b0 con 0x7f2c681010a0 2024-04-17T06:13:37.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.279+0000 7f2c577fe640 1 --2- 172.21.15.47:0/2085290711 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f2c500a3d20 0x7f2c500a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:13:37.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.279+0000 7f2c65d9b640 1 --2- 172.21.15.47:0/2085290711 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f2c500a3d20 0x7f2c500a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:13:37.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.279+0000 7f2c577fe640 1 -- 172.21.15.47:0/2085290711 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8513+0+0 (secure 0 0 0) 0x7f2c580d0900 con 0x7f2c681010a0 2024-04-17T06:13:37.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.280+0000 7f2c65d9b640 1 --2- 172.21.15.47:0/2085290711 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f2c500a3d20 0x7f2c500a61e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2c6806b1b0 tx=0x7f2c5c00b410 comp rx=0 tx=0).ready entity=mgr.24923 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:13:37.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.283+0000 7f2c577fe640 1 -- 172.21.15.47:0/2085290711 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f2c58014030 con 0x7f2c681010a0 2024-04-17T06:13:37.283 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:13:36 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/917258857' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-17T06:13:37.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:13:36 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:13:37.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:13:36 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:13:37.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:13:36 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:13:37.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:13:36 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:13:37.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:13:36 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/2589861122' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T06:13:37.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.472+0000 7f2c6ce0d640 1 -- 172.21.15.47:0/2085290711 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2c6806ac50 con 0x7f2c681010a0 2024-04-17T06:13:37.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.473+0000 7f2c577fe640 1 -- 172.21.15.47:0/2085290711 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) ==== 74+0+23073 (secure 0 0 0) 0x7f2c5809cfa0 con 0x7f2c681010a0 2024-04-17T06:13:37.471 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-17T06:13:37.472 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":106,"fsid":"565db2d8-fc7e-11ee-bc91-c7b262605968","created":"2024-04-17T05:51:19.888865+0000","modified":"2024-04-17T06:13:14.788483+0000","last_up_change":"2024-04-17T05:58:24.380131+0000","last_in_change":"2024-04-17T05:58:04.418236+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T05:54:37.734306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T05:59:12.458010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T05:59:12.901339+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":19,"snap_epoch":105,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8453f300-533d-4736-a343-55619c08e966","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":1290127544},{"type":"v1","addr":"172.21.15.47:6803","nonce":1290127544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":1290127544},{"type":"v1","addr":"172.21.15.47:6805","nonce":1290127544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":1290127544},{"type":"v1","addr":"172.21.15.47:6809","nonce":1290127544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":1290127544},{"type":"v1","addr":"172.21.15.47:6807","nonce":1290127544}]},"public_addr":"172.21.15.47:6803/1290127544","cluster_addr":"172.21.15.47:6805/1290127544","heartbeat_back_addr":"172.21.15.47:6809/1290127544","heartbeat_front_addr":"172.21.15.47:6807/1290127544","state":["exists","up"]},{"osd":1,"uuid":"80b0f03e-c1bc-4658-8028-1e6b8eedcc47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":1180430954},{"type":"v1","addr":"172.21.15.98:6801","nonce":1180430954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":1180430954},{"type":"v1","addr":"172.21.15.98:6803","nonce":1180430954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":1180430954},{"type":"v1","addr":"172.21.15.98:6807","nonce":1180430954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":1180430954},{"type":"v1","addr":"172.21.15.98:6805","nonce":1180430954}]},"public_addr":"172.21.15.98:6801/1180430954","cluster_addr":"172.21.15.98:6803/1180430954","heartbeat_back_addr":"172.21.15.98:6807/1180430954","heartbeat_front_addr":"172.21.15.98:6805/1180430954","state":["exists","up"]},{"osd":2,"uuid":"0fc9d383-05b4-498d-b530-fccf7aeddd7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":494676216},{"type":"v1","addr":"172.21.15.103:6801","nonce":494676216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":494676216},{"type":"v1","addr":"172.21.15.103:6803","nonce":494676216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":494676216},{"type":"v1","addr":"172.21.15.103:6807","nonce":494676216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":494676216},{"type":"v1","addr":"172.21.15.103:6805","nonce":494676216}]},"public_addr":"172.21.15.103:6801/494676216","cluster_addr":"172.21.15.103:6803/494676216","heartbeat_back_addr":"172.21.15.103:6807/494676216","heartbeat_front_addr":"172.21.15.103:6805/494676216","state":["exists","up"]},{"osd":3,"uuid":"f6e0b0d5-0fb3-42d0-9e84-f7cfef21a998","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":1998369447},{"type":"v1","addr":"172.21.15.47:6811","nonce":1998369447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":1998369447},{"type":"v1","addr":"172.21.15.47:6813","nonce":1998369447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":1998369447},{"type":"v1","addr":"172.21.15.47:6817","nonce":1998369447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":1998369447},{"type":"v1","addr":"172.21.15.47:6815","nonce":1998369447}]},"public_addr":"172.21.15.47:6811/1998369447","cluster_addr":"172.21.15.47:6813/1998369447","heartbeat_back_addr":"172.21.15.47:6817/1998369447","heartbeat_front_addr":"172.21.15.47:6815/1998369447","state":["exists","up"]},{"osd":4,"uuid":"2c26217e-02df-466d-bc01-0149d4aec3c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":3060482485},{"type":"v1","addr":"172.21.15.98:6809","nonce":3060482485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":3060482485},{"type":"v1","addr":"172.21.15.98:6811","nonce":3060482485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":3060482485},{"type":"v1","addr":"172.21.15.98:6815","nonce":3060482485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":3060482485},{"type":"v1","addr":"172.21.15.98:6813","nonce":3060482485}]},"public_addr":"172.21.15.98:6809/3060482485","cluster_addr":"172.21.15.98:6811/3060482485","heartbeat_back_addr":"172.21.15.98:6815/3060482485","heartbeat_front_addr":"172.21.15.98:6813/3060482485","state":["exists","up"]},{"osd":5,"uuid":"740bf3e0-bea8-496b-8d6f-e42d0b017258","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":4181363567},{"type":"v1","addr":"172.21.15.103:6809","nonce":4181363567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":4181363567},{"type":"v1","addr":"172.21.15.103:6811","nonce":4181363567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":4181363567},{"type":"v1","addr":"172.21.15.103:6815","nonce":4181363567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":4181363567},{"type":"v1","addr":"172.21.15.103:6813","nonce":4181363567}]},"public_addr":"172.21.15.103:6809/4181363567","cluster_addr":"172.21.15.103:6811/4181363567","heartbeat_back_addr":"172.21.15.103:6815/4181363567","heartbeat_front_addr":"172.21.15.103:6813/4181363567","state":["exists","up"]},{"osd":6,"uuid":"356a6255-2c07-4f0d-a233-26be04c92b38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":2517529668},{"type":"v1","addr":"172.21.15.47:6819","nonce":2517529668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2517529668},{"type":"v1","addr":"172.21.15.47:6821","nonce":2517529668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2517529668},{"type":"v1","addr":"172.21.15.47:6825","nonce":2517529668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2517529668},{"type":"v1","addr":"172.21.15.47:6823","nonce":2517529668}]},"public_addr":"172.21.15.47:6819/2517529668","cluster_addr":"172.21.15.47:6821/2517529668","heartbeat_back_addr":"172.21.15.47:6825/2517529668","heartbeat_front_addr":"172.21.15.47:6823/2517529668","state":["exists","up"]},{"osd":7,"uuid":"349d0120-8290-417d-a800-5a8985f554f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":4225498496},{"type":"v1","addr":"172.21.15.98:6817","nonce":4225498496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":4225498496},{"type":"v1","addr":"172.21.15.98:6819","nonce":4225498496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":4225498496},{"type":"v1","addr":"172.21.15.98:6823","nonce":4225498496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":4225498496},{"type":"v1","addr":"172.21.15.98:6821","nonce":4225498496}]},"public_addr":"172.21.15.98:6817/4225498496","cluster_addr":"172.21.15.98:6819/4225498496","heartbeat_back_addr":"172.21.15.98:6823/4225498496","heartbeat_front_addr":"172.21.15.98:6821/4225498496","state":["exists","up"]},{"osd":8,"uuid":"cee4712f-321b-42c0-92ba-e15144c00f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":3433905520},{"type":"v1","addr":"172.21.15.103:6817","nonce":3433905520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":3433905520},{"type":"v1","addr":"172.21.15.103:6819","nonce":3433905520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":3433905520},{"type":"v1","addr":"172.21.15.103:6823","nonce":3433905520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":3433905520},{"type":"v1","addr":"172.21.15.103:6821","nonce":3433905520}]},"public_addr":"172.21.15.103:6817/3433905520","cluster_addr":"172.21.15.103:6819/3433905520","heartbeat_back_addr":"172.21.15.103:6823/3433905520","heartbeat_front_addr":"172.21.15.103:6821/3433905520","state":["exists","up"]},{"osd":9,"uuid":"63fc73c1-5a22-487d-80b8-9894e2ddd207","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":728359554},{"type":"v1","addr":"172.21.15.47:6827","nonce":728359554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":728359554},{"type":"v1","addr":"172.21.15.47:6829","nonce":728359554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":728359554},{"type":"v1","addr":"172.21.15.47:6833","nonce":728359554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":728359554},{"type":"v1","addr":"172.21.15.47:6831","nonce":728359554}]},"public_addr":"172.21.15.47:6827/728359554","cluster_addr":"172.21.15.47:6829/728359554","heartbeat_back_addr":"172.21.15.47:6833/728359554","heartbeat_front_addr":"172.21.15.47:6831/728359554","state":["exists","up"]},{"osd":10,"uuid":"9565bcc3-62ee-48eb-a07a-03a81e0cc8e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":468205689},{"type":"v1","addr":"172.21.15.98:6825","nonce":468205689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":468205689},{"type":"v1","addr":"172.21.15.98:6827","nonce":468205689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":468205689},{"type":"v1","addr":"172.21.15.98:6831","nonce":468205689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":468205689},{"type":"v1","addr":"172.21.15.98:6829","nonce":468205689}]},"public_addr":"172.21.15.98:6825/468205689","cluster_addr":"172.21.15.98:6827/468205689","heartbeat_back_addr":"172.21.15.98:6831/468205689","heartbeat_front_addr":"172.21.15.98:6829/468205689","state":["exists","up"]},{"osd":11,"uuid":"2e85e8d9-bf6c-4a12-b124-a4690dd61108","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":2639831013},{"type":"v1","addr":"172.21.15.103:6825","nonce":2639831013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":2639831013},{"type":"v1","addr":"172.21.15.103:6827","nonce":2639831013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":2639831013},{"type":"v1","addr":"172.21.15.103:6831","nonce":2639831013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":2639831013},{"type":"v1","addr":"172.21.15.103:6829","nonce":2639831013}]},"public_addr":"172.21.15.103:6825/2639831013","cluster_addr":"172.21.15.103:6827/2639831013","heartbeat_back_addr":"172.21.15.103:6831/2639831013","heartbeat_front_addr":"172.21.15.103:6829/2639831013","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:53:47.366438+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:54:10.953559+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:54:34.320332+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:54:59.751360+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:55:23.982052+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:55:48.425017+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:56:14.793368+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:56:40.440830+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:57:04.623680+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:57:31.215262+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:57:57.040346+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T05:58:22.671572+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:6800/3157902806":"2024-04-18T06:01:18.246671+0000","172.21.15.47:6801/3157902806":"2024-04-18T06:01:18.246671+0000","172.21.15.47:0/3681048987":"2024-04-18T06:01:18.246671+0000","172.21.15.47:0/3009926768":"2024-04-18T06:01:18.246671+0000","172.21.15.47:0/2970926607":"2024-04-18T06:01:18.246671+0000","172.21.15.47:6801/3632157175":"2024-04-18T05:52:13.645775+0000","172.21.15.47:0/4288219269":"2024-04-18T05:52:13.645775+0000","172.21.15.47:0/2584487544":"2024-04-18T05:52:13.645775+0000","172.21.15.47:6801/953064786":"2024-04-18T05:51:45.979351+0000","172.21.15.47:0/3484500927":"2024-04-18T05:51:45.979351+0000","172.21.15.47:6800/3632157175":"2024-04-18T05:52:13.645775+0000","172.21.15.47:0/112268670":"2024-04-18T05:51:45.979351+0000","172.21.15.47:0/1638423256":"2024-04-18T05:51:45.979351+0000","172.21.15.47:0/3275066738":"2024-04-18T06:01:18.246671+0000","172.21.15.47:6800/953064786":"2024-04-18T05:51:45.979351+0000","172.21.15.47:0/849479010":"2024-04-18T05:52:13.645775+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":7,"length":1},{"begin":19,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T06:13:37.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.475+0000 7f2c6ce0d640 1 -- 172.21.15.47:0/2085290711 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f2c500a3d20 msgr2=0x7f2c500a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:13:37.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.475+0000 7f2c6ce0d640 1 --2- 172.21.15.47:0/2085290711 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f2c500a3d20 0x7f2c500a61e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2c6806b1b0 tx=0x7f2c5c00b410 comp rx=0 tx=0).stop 2024-04-17T06:13:37.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.475+0000 7f2c6ce0d640 1 -- 172.21.15.47:0/2085290711 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2c681010a0 msgr2=0x7f2c681995f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:13:37.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.475+0000 7f2c6ce0d640 1 --2- 172.21.15.47:0/2085290711 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2c681010a0 0x7f2c681995f0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f2c5800c890 tx=0x7f2c5800cd50 comp rx=0 tx=0).stop 2024-04-17T06:13:37.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.476+0000 7f2c6ce0d640 1 -- 172.21.15.47:0/2085290711 shutdown_connections 2024-04-17T06:13:37.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.476+0000 7f2c6ce0d640 1 --2- 172.21.15.47:0/2085290711 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c6819a110 0x7f2c681a4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:13:37.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.476+0000 7f2c6ce0d640 1 --2- 172.21.15.47:0/2085290711 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2c68106170 0x7f2c68199b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:13:37.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.476+0000 7f2c6ce0d640 1 --2- 172.21.15.47:0/2085290711 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f2c500a3d20 0x7f2c500a61e0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:13:37.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.476+0000 7f2c6ce0d640 1 --2- 172.21.15.47:0/2085290711 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2c681010a0 0x7f2c681995f0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:13:37.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.476+0000 7f2c6ce0d640 1 -- 172.21.15.47:0/2085290711 >> 172.21.15.47:0/2085290711 conn(0x7f2c680771e0 msgr2=0x7f2c680fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:13:37.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.476+0000 7f2c6ce0d640 1 -- 172.21.15.47:0/2085290711 shutdown_connections 2024-04-17T06:13:37.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.476+0000 7f2c6ce0d640 1 -- 172.21.15.47:0/2085290711 wait complete. 2024-04-17T06:13:37.486 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-17T06:13:37.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.595+0000 7fa49b888640 1 -- 172.21.15.47:0/1927548984 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa494105e70 msgr2=0x7fa494106250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:13:37.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.595+0000 7fa49b888640 1 --2- 172.21.15.47:0/1927548984 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa494105e70 0x7fa494106250 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fa490009ca0 tx=0x7fa49002c0f0 comp rx=0 tx=0).stop 2024-04-17T06:13:37.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.595+0000 7fa49b888640 1 -- 172.21.15.47:0/1927548984 shutdown_connections 2024-04-17T06:13:37.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.595+0000 7fa49b888640 1 --2- 172.21.15.47:0/1927548984 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa494101f50 0x7fa49410dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:13:37.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.595+0000 7fa49b888640 1 --2- 172.21.15.47:0/1927548984 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa494101590 0x7fa494101a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:13:37.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.595+0000 7fa49b888640 1 --2- 172.21.15.47:0/1927548984 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa494105e70 0x7fa494106250 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:13:37.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.595+0000 7fa49b888640 1 -- 172.21.15.47:0/1927548984 >> 172.21.15.47:0/1927548984 conn(0x7fa4940773f0 msgr2=0x7fa4940fe770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:13:37.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.595+0000 7fa49b888640 1 -- 172.21.15.47:0/1927548984 shutdown_connections 2024-04-17T06:13:37.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.595+0000 7fa49b888640 1 -- 172.21.15.47:0/1927548984 wait complete. 2024-04-17T06:13:37.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.596+0000 7fa49b888640 1 Processor -- start 2024-04-17T06:13:37.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.596+0000 7fa49b888640 1 -- start start 2024-04-17T06:13:37.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.596+0000 7fa49b888640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa494101590 0x7fa49410e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:13:37.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.596+0000 7fa49b888640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa494101f50 0x7fa49410ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:13:37.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.596+0000 7fa49b888640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa494105e70 0x7fa49410f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:13:37.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.596+0000 7fa49b888640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa494112310 con 0x7fa494101590 2024-04-17T06:13:37.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.596+0000 7fa49b888640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa494112190 con 0x7fa494105e70 2024-04-17T06:13:37.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.596+0000 7fa49b888640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa494112490 con 0x7fa494101f50 2024-04-17T06:13:37.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.596+0000 7fa499624640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa494101590 0x7fa49410e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:13:37.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.596+0000 7fa499624640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa494101590 0x7fa49410e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46758/0 (socket says 172.21.15.47:46758) 2024-04-17T06:13:37.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.596+0000 7fa499624640 1 -- 172.21.15.47:0/3844791178 learned_addr learned my addr 172.21.15.47:0/3844791178 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:13:37.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.596+0000 7fa498e23640 1 --2- 172.21.15.47:0/3844791178 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa494101f50 0x7fa49410ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:13:37.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.596+0000 7fa499e25640 1 --2- 172.21.15.47:0/3844791178 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa494105e70 0x7fa49410f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:13:37.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.597+0000 7fa499624640 1 -- 172.21.15.47:0/3844791178 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa494101f50 msgr2=0x7fa49410ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:13:37.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.597+0000 7fa499624640 1 --2- 172.21.15.47:0/3844791178 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa494101f50 0x7fa49410ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:13:37.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.597+0000 7fa499624640 1 -- 172.21.15.47:0/3844791178 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa494105e70 msgr2=0x7fa49410f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:13:37.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.597+0000 7fa499624640 1 --2- 172.21.15.47:0/3844791178 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa494105e70 0x7fa49410f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:13:37.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.597+0000 7fa499624640 1 -- 172.21.15.47:0/3844791178 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4941a6c50 con 0x7fa494101590 2024-04-17T06:13:37.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.597+0000 7fa499624640 1 --2- 172.21.15.47:0/3844791178 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa494101590 0x7fa49410e790 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fa490005e30 tx=0x7fa4900324f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:13:37.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.597+0000 7fa48a7fc640 1 -- 172.21.15.47:0/3844791178 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa490042020 con 0x7fa494101590 2024-04-17T06:13:37.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.597+0000 7fa48a7fc640 1 -- 172.21.15.47:0/3844791178 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa490003970 con 0x7fa494101590 2024-04-17T06:13:37.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.597+0000 7fa49b888640 1 -- 172.21.15.47:0/3844791178 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4941a6ee0 con 0x7fa494101590 2024-04-17T06:13:37.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.597+0000 7fa48a7fc640 1 -- 172.21.15.47:0/3844791178 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa490039040 con 0x7fa494101590 2024-04-17T06:13:37.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.597+0000 7fa49b888640 1 -- 172.21.15.47:0/3844791178 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4941a7340 con 0x7fa494101590 2024-04-17T06:13:37.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.598+0000 7fa49b888640 1 -- 172.21.15.47:0/3844791178 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4940691d0 con 0x7fa494101590 2024-04-17T06:13:37.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.599+0000 7fa48a7fc640 1 -- 172.21.15.47:0/3844791178 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa490045020 con 0x7fa494101590 2024-04-17T06:13:37.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.600+0000 7fa48a7fc640 1 --2- 172.21.15.47:0/3844791178 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fa4800a3d20 0x7fa4800a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:13:37.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.600+0000 7fa498e23640 1 --2- 172.21.15.47:0/3844791178 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fa4800a3d20 0x7fa4800a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:13:37.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.600+0000 7fa48a7fc640 1 -- 172.21.15.47:0/3844791178 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8513+0+0 (secure 0 0 0) 0x7fa4900f27b0 con 0x7fa494101590 2024-04-17T06:13:37.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.600+0000 7fa498e23640 1 --2- 172.21.15.47:0/3844791178 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fa4800a3d20 0x7fa4800a61e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa494110220 tx=0x7fa48400a2d0 comp rx=0 tx=0).ready entity=mgr.24923 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:13:37.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.605+0000 7fa48a7fc640 1 -- 172.21.15.47:0/3844791178 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fa49004c020 con 0x7fa494101590 2024-04-17T06:13:37.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.834+0000 7fa49b888640 1 -- 172.21.15.47:0/3844791178 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fa49402d5d0 con 0x7fa494101590 2024-04-17T06:13:37.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.836+0000 7fa48a7fc640 1 -- 172.21.15.47:0/3844791178 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v38) ==== 161+0+0 (secure 0 0 0) 0x7fa4900bef00 con 0x7fa494101590 2024-04-17T06:13:37.834 INFO:teuthology.orchestra.run.smithi047.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-17T06:13:37.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.838+0000 7fa49b888640 1 -- 172.21.15.47:0/3844791178 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fa4800a3d20 msgr2=0x7fa4800a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:13:37.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.838+0000 7fa49b888640 1 --2- 172.21.15.47:0/3844791178 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fa4800a3d20 0x7fa4800a61e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa494110220 tx=0x7fa48400a2d0 comp rx=0 tx=0).stop 2024-04-17T06:13:37.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.838+0000 7fa49b888640 1 -- 172.21.15.47:0/3844791178 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa494101590 msgr2=0x7fa49410e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:13:37.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.838+0000 7fa49b888640 1 --2- 172.21.15.47:0/3844791178 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa494101590 0x7fa49410e790 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fa490005e30 tx=0x7fa4900324f0 comp rx=0 tx=0).stop 2024-04-17T06:13:37.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.839+0000 7fa49b888640 1 -- 172.21.15.47:0/3844791178 shutdown_connections 2024-04-17T06:13:37.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.839+0000 7fa49b888640 1 --2- 172.21.15.47:0/3844791178 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa494105e70 0x7fa49410f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:13:37.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.839+0000 7fa49b888640 1 --2- 172.21.15.47:0/3844791178 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa494101f50 0x7fa49410ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:13:37.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.839+0000 7fa49b888640 1 --2- 172.21.15.47:0/3844791178 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fa4800a3d20 0x7fa4800a61e0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:13:37.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.839+0000 7fa49b888640 1 --2- 172.21.15.47:0/3844791178 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa494101590 0x7fa49410e790 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:13:37.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.839+0000 7fa49b888640 1 -- 172.21.15.47:0/3844791178 >> 172.21.15.47:0/3844791178 conn(0x7fa4940773f0 msgr2=0x7fa494103f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:13:37.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.839+0000 7fa49b888640 1 -- 172.21.15.47:0/3844791178 shutdown_connections 2024-04-17T06:13:37.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.839+0000 7fa49b888640 1 -- 172.21.15.47:0/3844791178 wait complete. 2024-04-17T06:13:37.848 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-17T06:13:37.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.960+0000 7fad5db24640 1 -- 172.21.15.47:0/2437239036 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad58107040 msgr2=0x7fad58107420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:13:37.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.960+0000 7fad5db24640 1 --2- 172.21.15.47:0/2437239036 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad58107040 0x7fad58107420 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fad48009930 tx=0x7fad4802bc30 comp rx=0 tx=0).stop 2024-04-17T06:13:37.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.960+0000 7fad5db24640 1 -- 172.21.15.47:0/2437239036 shutdown_connections 2024-04-17T06:13:37.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.960+0000 7fad5db24640 1 --2- 172.21.15.47:0/2437239036 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fad58101b50 0x7fad5810dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:13:37.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.960+0000 7fad5db24640 1 --2- 172.21.15.47:0/2437239036 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad581011b0 0x7fad58101610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:13:37.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.960+0000 7fad5db24640 1 --2- 172.21.15.47:0/2437239036 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad58107040 0x7fad58107420 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:13:37.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.960+0000 7fad5db24640 1 -- 172.21.15.47:0/2437239036 >> 172.21.15.47:0/2437239036 conn(0x7fad58076c60 msgr2=0x7fad58077070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:13:37.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.960+0000 7fad5db24640 1 -- 172.21.15.47:0/2437239036 shutdown_connections 2024-04-17T06:13:37.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.960+0000 7fad5db24640 1 -- 172.21.15.47:0/2437239036 wait complete. 2024-04-17T06:13:37.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.961+0000 7fad5db24640 1 Processor -- start 2024-04-17T06:13:37.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.961+0000 7fad5db24640 1 -- start start 2024-04-17T06:13:37.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.961+0000 7fad5db24640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad581011b0 0x7fad5819e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:13:37.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.961+0000 7fad5db24640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad58101b50 0x7fad5819e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:13:37.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.961+0000 7fad5db24640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fad58107040 0x7fad58198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:13:37.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.961+0000 7fad5db24640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fad58111220 con 0x7fad581011b0 2024-04-17T06:13:37.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.961+0000 7fad5db24640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fad581110a0 con 0x7fad58101b50 2024-04-17T06:13:37.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.961+0000 7fad5db24640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fad581113a0 con 0x7fad58107040 2024-04-17T06:13:37.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.961+0000 7fad577fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad581011b0 0x7fad5819e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:13:37.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.961+0000 7fad577fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad581011b0 0x7fad5819e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46788/0 (socket says 172.21.15.47:46788) 2024-04-17T06:13:37.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.961+0000 7fad577fe640 1 -- 172.21.15.47:0/1137007243 learned_addr learned my addr 172.21.15.47:0/1137007243 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:13:37.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.961+0000 7fad56ffd640 1 --2- 172.21.15.47:0/1137007243 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad58101b50 0x7fad5819e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:13:37.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.961+0000 7fad57fff640 1 --2- 172.21.15.47:0/1137007243 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fad58107040 0x7fad58198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:13:37.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.962+0000 7fad577fe640 1 -- 172.21.15.47:0/1137007243 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fad58107040 msgr2=0x7fad58198620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:13:37.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.962+0000 7fad577fe640 1 --2- 172.21.15.47:0/1137007243 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fad58107040 0x7fad58198620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:13:37.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.962+0000 7fad577fe640 1 -- 172.21.15.47:0/1137007243 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad58101b50 msgr2=0x7fad5819e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:13:37.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.962+0000 7fad577fe640 1 --2- 172.21.15.47:0/1137007243 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad58101b50 0x7fad5819e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:13:37.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.962+0000 7fad577fe640 1 -- 172.21.15.47:0/1137007243 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad58198da0 con 0x7fad581011b0 2024-04-17T06:13:37.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.962+0000 7fad57fff640 1 --2- 172.21.15.47:0/1137007243 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fad58107040 0x7fad58198620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:13:37.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.962+0000 7fad577fe640 1 --2- 172.21.15.47:0/1137007243 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad581011b0 0x7fad5819e470 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fad48009230 tx=0x7fad4802cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:13:37.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.962+0000 7fad54ff9640 1 -- 172.21.15.47:0/1137007243 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad48041070 con 0x7fad581011b0 2024-04-17T06:13:37.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.962+0000 7fad54ff9640 1 -- 172.21.15.47:0/1137007243 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad48004450 con 0x7fad581011b0 2024-04-17T06:13:37.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.962+0000 7fad5db24640 1 -- 172.21.15.47:0/1137007243 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad58199030 con 0x7fad581011b0 2024-04-17T06:13:37.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.962+0000 7fad54ff9640 1 -- 172.21.15.47:0/1137007243 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad48034440 con 0x7fad581011b0 2024-04-17T06:13:37.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.962+0000 7fad5db24640 1 -- 172.21.15.47:0/1137007243 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad581a5330 con 0x7fad581011b0 2024-04-17T06:13:37.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.963+0000 7fad5db24640 1 -- 172.21.15.47:0/1137007243 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad581025a0 con 0x7fad581011b0 2024-04-17T06:13:37.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.964+0000 7fad54ff9640 1 -- 172.21.15.47:0/1137007243 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fad48044070 con 0x7fad581011b0 2024-04-17T06:13:37.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.965+0000 7fad54ff9640 1 --2- 172.21.15.47:0/1137007243 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fad400a3d20 0x7fad400a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:13:37.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.965+0000 7fad56ffd640 1 --2- 172.21.15.47:0/1137007243 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fad400a3d20 0x7fad400a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:13:37.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.965+0000 7fad54ff9640 1 -- 172.21.15.47:0/1137007243 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8727+0+0 (secure 0 0 0) 0x7fad480f2570 con 0x7fad581011b0 2024-04-17T06:13:37.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.965+0000 7fad56ffd640 1 --2- 172.21.15.47:0/1137007243 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fad400a3d20 0x7fad400a61e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fad581066e0 tx=0x7fad4c005f90 comp rx=0 tx=0).ready entity=mgr.24923 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:13:37.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:37.969+0000 7fad54ff9640 1 -- 172.21.15.47:0/1137007243 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fad480bec50 con 0x7fad581011b0 2024-04-17T06:13:38.133 INFO:journalctl@ceph.mds.b.smithi098.stdout:Apr 17 06:13:37 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-b[67363]: ignoring --setuser ceph since I am not root 2024-04-17T06:13:38.133 INFO:journalctl@ceph.mds.b.smithi098.stdout:Apr 17 06:13:37 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-b[67363]: ignoring --setgroup ceph since I am not root 2024-04-17T06:13:38.133 INFO:journalctl@ceph.mds.b.smithi098.stdout:Apr 17 06:13:37 smithi098 ceph-mds[67367]: -- 172.21.15.98:0/4250527721 <== mon.2 v2:172.21.15.98:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e81601fa40 con 0x55e815ff3000 2024-04-17T06:13:38.133 INFO:journalctl@ceph.mds.b.smithi098.stdout:Apr 17 06:13:37 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-b[67363]: starting mds.b at 2024-04-17T06:13:38.133 INFO:journalctl@ceph.mds.h.smithi098.stdout:Apr 17 06:13:37 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-h[67059]: ignoring --setuser ceph since I am not root 2024-04-17T06:13:38.134 INFO:journalctl@ceph.mds.h.smithi098.stdout:Apr 17 06:13:37 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-h[67059]: ignoring --setgroup ceph since I am not root 2024-04-17T06:13:38.134 INFO:journalctl@ceph.mds.h.smithi098.stdout:Apr 17 06:13:37 smithi098 ceph-mds[67063]: -- 172.21.15.98:0/3480737351 <== mon.2 v2:172.21.15.98:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5623454eda40 con 0x5623454c0c00 2024-04-17T06:13:38.134 INFO:journalctl@ceph.mds.h.smithi098.stdout:Apr 17 06:13:37 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-h[67059]: starting mds.h at 2024-04-17T06:13:38.134 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:13:37 smithi098 ceph-mon[36516]: pgmap v389: 97 pgs: 97 active+clean; 5.9 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 853 B/s wr, 5 op/s 2024-04-17T06:13:38.134 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:13:37 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T06:13:38.134 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:13:37 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T06:13:38.134 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:13:37 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:13:38.134 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:13:37 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/3147758023' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T06:13:38.134 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:13:37 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/2085290711' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T06:13:38.135 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:13:37 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[73021]: ignoring --setuser ceph since I am not root 2024-04-17T06:13:38.135 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:13:37 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[73021]: ignoring --setgroup ceph since I am not root 2024-04-17T06:13:38.135 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:13:37 smithi098 ceph-mds[73025]: -- 172.21.15.98:0/902667482 <== mon.2 v2:172.21.15.98:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56056775ba40 con 0x56056772f000 2024-04-17T06:13:38.135 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:13:37 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[73021]: starting mds.k at 2024-04-17T06:13:38.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:13:37 smithi047 ceph-mon[32223]: pgmap v389: 97 pgs: 97 active+clean; 5.9 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 853 B/s wr, 5 op/s 2024-04-17T06:13:38.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:13:37 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T06:13:38.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:13:37 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T06:13:38.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:13:37 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:13:38.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:13:37 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/3147758023' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T06:13:38.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:13:37 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/2085290711' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T06:13:38.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:13:37 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mon-a[32219]: 2024-04-17T06:13:37.835+0000 7f699f558700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-17T06:13:38.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:38.195+0000 7fad5db24640 1 -- 172.21.15.47:0/1137007243 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fad581a5500 con 0x7fad581011b0 2024-04-17T06:13:38.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:13:37 smithi103 ceph-mon[36243]: pgmap v389: 97 pgs: 97 active+clean; 5.9 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 853 B/s wr, 5 op/s 2024-04-17T06:13:38.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:13:37 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T06:13:38.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:13:37 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T06:13:38.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:13:37 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' 2024-04-17T06:13:38.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:13:37 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/3147758023' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T06:13:38.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:13:37 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/2085290711' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T06:13:38.285 INFO:journalctl@ceph.mds.c.smithi103.stdout:Apr 17 06:13:37 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-c[66600]: ignoring --setuser ceph since I am not root 2024-04-17T06:13:38.285 INFO:journalctl@ceph.mds.c.smithi103.stdout:Apr 17 06:13:37 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-c[66600]: ignoring --setgroup ceph since I am not root 2024-04-17T06:13:38.285 INFO:journalctl@ceph.mds.c.smithi103.stdout:Apr 17 06:13:37 smithi103 ceph-mds[66604]: -- 172.21.15.103:0/4252976674 <== mon.2 v2:172.21.15.98:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fd8155ba40 con 0x55fd8152e800 2024-04-17T06:13:38.285 INFO:journalctl@ceph.mds.c.smithi103.stdout:Apr 17 06:13:37 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-c[66600]: starting mds.c at 2024-04-17T06:13:38.285 INFO:journalctl@ceph.mds.i.smithi103.stdout:Apr 17 06:13:37 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-i[66918]: ignoring --setuser ceph since I am not root 2024-04-17T06:13:38.285 INFO:journalctl@ceph.mds.i.smithi103.stdout:Apr 17 06:13:37 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-i[66918]: ignoring --setgroup ceph since I am not root 2024-04-17T06:13:38.285 INFO:journalctl@ceph.mds.i.smithi103.stdout:Apr 17 06:13:37 smithi103 ceph-mds[66922]: -- 172.21.15.103:0/2530840626 <== mon.0 v2:172.21.15.47:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55dadb9bfa40 con 0x55dadb993000 2024-04-17T06:13:38.285 INFO:journalctl@ceph.mds.i.smithi103.stdout:Apr 17 06:13:37 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-i[66918]: starting mds.i at 2024-04-17T06:13:38.286 INFO:journalctl@ceph.mds.f.smithi103.stdout:Apr 17 06:13:37 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-f[66292]: ignoring --setuser ceph since I am not root 2024-04-17T06:13:38.286 INFO:journalctl@ceph.mds.f.smithi103.stdout:Apr 17 06:13:37 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-f[66292]: ignoring --setgroup ceph since I am not root 2024-04-17T06:13:38.286 INFO:journalctl@ceph.mds.f.smithi103.stdout:Apr 17 06:13:37 smithi103 ceph-mds[66296]: -- 172.21.15.103:0/3608643973 <== mon.1 v2:172.21.15.103:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a7a94fba40 con 0x55a7a94cec00 2024-04-17T06:13:38.286 INFO:journalctl@ceph.mds.f.smithi103.stdout:Apr 17 06:13:37 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-f[66292]: starting mds.f at 2024-04-17T06:13:38.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:38.840+0000 7fad54ff9640 1 -- 172.21.15.47:0/1137007243 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v39) ==== 104+0+0 (secure 0 0 0) 0x7fad480be550 con 0x7fad581011b0 2024-04-17T06:13:38.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:38.843+0000 7fad5db24640 1 -- 172.21.15.47:0/1137007243 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fad400a3d20 msgr2=0x7fad400a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:13:38.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:38.843+0000 7fad5db24640 1 --2- 172.21.15.47:0/1137007243 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fad400a3d20 0x7fad400a61e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fad581066e0 tx=0x7fad4c005f90 comp rx=0 tx=0).stop 2024-04-17T06:13:38.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:38.843+0000 7fad5db24640 1 -- 172.21.15.47:0/1137007243 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad581011b0 msgr2=0x7fad5819e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:13:38.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:38.843+0000 7fad5db24640 1 --2- 172.21.15.47:0/1137007243 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad581011b0 0x7fad5819e470 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fad48009230 tx=0x7fad4802cba0 comp rx=0 tx=0).stop 2024-04-17T06:13:38.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:38.843+0000 7fad5db24640 1 -- 172.21.15.47:0/1137007243 shutdown_connections 2024-04-17T06:13:38.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:38.843+0000 7fad5db24640 1 --2- 172.21.15.47:0/1137007243 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fad58107040 0x7fad58198620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:13:38.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:38.843+0000 7fad5db24640 1 --2- 172.21.15.47:0/1137007243 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad58101b50 0x7fad5819e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:13:38.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:38.843+0000 7fad5db24640 1 --2- 172.21.15.47:0/1137007243 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7fad400a3d20 0x7fad400a61e0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:13:38.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:38.843+0000 7fad5db24640 1 --2- 172.21.15.47:0/1137007243 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad581011b0 0x7fad5819e470 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:13:38.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:38.843+0000 7fad5db24640 1 -- 172.21.15.47:0/1137007243 >> 172.21.15.47:0/1137007243 conn(0x7fad58076c60 msgr2=0x7fad5806a860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:13:38.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:38.843+0000 7fad5db24640 1 -- 172.21.15.47:0/1137007243 shutdown_connections 2024-04-17T06:13:38.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:38.843+0000 7fad5db24640 1 -- 172.21.15.47:0/1137007243 wait complete. 2024-04-17T06:13:38.858 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-17T06:13:38.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:38.976+0000 7f19f6ca5640 1 -- 172.21.15.47:0/3990142158 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f19f006b220 msgr2=0x7f19f0106bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:13:38.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:38.976+0000 7f19f6ca5640 1 --2- 172.21.15.47:0/3990142158 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f19f006b220 0x7f19f0106bf0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f19d8009900 tx=0x7f19d802bc30 comp rx=0 tx=0).stop 2024-04-17T06:13:38.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:38.976+0000 7f19f6ca5640 1 -- 172.21.15.47:0/3990142158 shutdown_connections 2024-04-17T06:13:38.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:38.976+0000 7f19f6ca5640 1 --2- 172.21.15.47:0/3990142158 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f19f0107320 0x7f19f010ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:13:38.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:38.976+0000 7f19f6ca5640 1 --2- 172.21.15.47:0/3990142158 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f19f006b220 0x7f19f0106bf0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:13:38.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:38.976+0000 7f19f6ca5640 1 --2- 172.21.15.47:0/3990142158 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19f006a870 0x7f19f006ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:13:38.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:38.976+0000 7f19f6ca5640 1 -- 172.21.15.47:0/3990142158 >> 172.21.15.47:0/3990142158 conn(0x7f19f0077c50 msgr2=0x7f19f0078060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:13:38.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:38.976+0000 7f19f6ca5640 1 -- 172.21.15.47:0/3990142158 shutdown_connections 2024-04-17T06:13:38.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:38.976+0000 7f19f6ca5640 1 -- 172.21.15.47:0/3990142158 wait complete. 2024-04-17T06:13:38.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:38.976+0000 7f19f6ca5640 1 Processor -- start 2024-04-17T06:13:38.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:38.976+0000 7f19f6ca5640 1 -- start start 2024-04-17T06:13:38.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:38.976+0000 7f19f6ca5640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19f006a870 0x7f19f019e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:13:38.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:38.976+0000 7f19f6ca5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f19f0107320 0x7f19f019e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:13:38.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:38.977+0000 7f19f6ca5640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f19f01985b0 0x7f19f0198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:13:38.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:38.977+0000 7f19f6ca5640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f19f0112330 con 0x7f19f01985b0 2024-04-17T06:13:38.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:38.977+0000 7f19f6ca5640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f19f01121b0 con 0x7f19f006a870 2024-04-17T06:13:38.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:38.977+0000 7f19f6ca5640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f19f01124b0 con 0x7f19f0107320 2024-04-17T06:13:38.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:38.977+0000 7f19f5242640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f19f01985b0 0x7f19f0198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:13:38.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:38.977+0000 7f19f5242640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f19f01985b0 0x7f19f0198a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46800/0 (socket says 172.21.15.47:46800) 2024-04-17T06:13:38.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:38.977+0000 7f19f5242640 1 -- 172.21.15.47:0/3506323713 learned_addr learned my addr 172.21.15.47:0/3506323713 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:13:38.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:38.977+0000 7f19effff640 1 --2- 172.21.15.47:0/3506323713 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f19f0107320 0x7f19f019e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:13:38.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:38.977+0000 7f19f4a41640 1 --2- 172.21.15.47:0/3506323713 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19f006a870 0x7f19f019e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:13:38.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:38.977+0000 7f19f5242640 1 -- 172.21.15.47:0/3506323713 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f19f0107320 msgr2=0x7f19f019e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:13:38.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:38.977+0000 7f19f5242640 1 --2- 172.21.15.47:0/3506323713 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f19f0107320 0x7f19f019e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:13:38.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:38.977+0000 7f19f5242640 1 -- 172.21.15.47:0/3506323713 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19f006a870 msgr2=0x7f19f019e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:13:38.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:38.977+0000 7f19f5242640 1 --2- 172.21.15.47:0/3506323713 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19f006a870 0x7f19f019e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:13:38.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:38.977+0000 7f19f5242640 1 -- 172.21.15.47:0/3506323713 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19f01990d0 con 0x7f19f01985b0 2024-04-17T06:13:38.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:38.977+0000 7f19f5242640 1 --2- 172.21.15.47:0/3506323713 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f19f01985b0 0x7f19f0198a60 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f19e800c970 tx=0x7f19e800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:13:38.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:38.977+0000 7f19edffb640 1 -- 172.21.15.47:0/3506323713 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e8007bb0 con 0x7f19f01985b0 2024-04-17T06:13:38.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:38.977+0000 7f19edffb640 1 -- 172.21.15.47:0/3506323713 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19e8007d50 con 0x7f19f01985b0 2024-04-17T06:13:38.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:38.977+0000 7f19f6ca5640 1 -- 172.21.15.47:0/3506323713 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19f01993c0 con 0x7f19f01985b0 2024-04-17T06:13:38.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:38.977+0000 7f19edffb640 1 -- 172.21.15.47:0/3506323713 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e8018800 con 0x7f19f01985b0 2024-04-17T06:13:38.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:38.978+0000 7f19f6ca5640 1 -- 172.21.15.47:0/3506323713 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19f01a5610 con 0x7f19f01985b0 2024-04-17T06:13:38.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:38.978+0000 7f19f6ca5640 1 -- 172.21.15.47:0/3506323713 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19f00691d0 con 0x7f19f01985b0 2024-04-17T06:13:38.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:38.979+0000 7f19edffb640 1 -- 172.21.15.47:0/3506323713 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f19e800fa50 con 0x7f19f01985b0 2024-04-17T06:13:38.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:38.980+0000 7f19edffb640 1 --2- 172.21.15.47:0/3506323713 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f19dc0a3da0 0x7f19dc0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:13:38.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:38.980+0000 7f19f4a41640 1 --2- 172.21.15.47:0/3506323713 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f19dc0a3da0 0x7f19dc0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:13:38.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:38.980+0000 7f19edffb640 1 -- 172.21.15.47:0/3506323713 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8727+0+0 (secure 0 0 0) 0x7f19e80d09c0 con 0x7f19f01985b0 2024-04-17T06:13:38.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:38.981+0000 7f19f4a41640 1 --2- 172.21.15.47:0/3506323713 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f19dc0a3da0 0x7f19dc0a6260 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f19e0005c40 tx=0x7f19e000a780 comp rx=0 tx=0).ready entity=mgr.24923 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:13:38.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:38.985+0000 7f19edffb640 1 -- 172.21.15.47:0/3506323713 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f19e8014030 con 0x7f19f01985b0 2024-04-17T06:13:39.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:13:38 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/3844791178' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-17T06:13:39.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:13:38 smithi098 ceph-mon[36516]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-17T06:13:39.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:13:38 smithi098 ceph-mon[36516]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-17T06:13:39.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:13:38 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/3844791178' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-17T06:13:39.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:13:38 smithi098 ceph-mon[36516]: osdmap e107: 12 total, 12 up, 12 in 2024-04-17T06:13:39.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:13:38 smithi098 ceph-mon[36516]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-17T06:13:39.134 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:13:38 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/1137007243' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-17T06:13:39.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:13:38 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/3844791178' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-17T06:13:39.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:13:38 smithi047 ceph-mon[32223]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-17T06:13:39.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:13:38 smithi047 ceph-mon[32223]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-17T06:13:39.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:13:38 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/3844791178' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-17T06:13:39.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:13:38 smithi047 ceph-mon[32223]: osdmap e107: 12 total, 12 up, 12 in 2024-04-17T06:13:39.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:13:38 smithi047 ceph-mon[32223]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-17T06:13:39.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:13:38 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/1137007243' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-17T06:13:39.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:39.172+0000 7f19f6ca5640 1 -- 172.21.15.47:0/3506323713 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f19f006b220 con 0x7f19f01985b0 2024-04-17T06:13:39.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:13:38 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/3844791178' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-17T06:13:39.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:13:38 smithi103 ceph-mon[36243]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-17T06:13:39.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:13:38 smithi103 ceph-mon[36243]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-17T06:13:39.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:13:38 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/3844791178' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-17T06:13:39.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:13:38 smithi103 ceph-mon[36243]: osdmap e107: 12 total, 12 up, 12 in 2024-04-17T06:13:39.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:13:38 smithi103 ceph-mon[36243]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-17T06:13:39.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:13:38 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/1137007243' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-17T06:13:39.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:39.841+0000 7f19edffb640 1 -- 172.21.15.47:0/3506323713 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v108) ==== 181+0+0 (secure 0 0 0) 0x7f19e809cf30 con 0x7f19f01985b0 2024-04-17T06:13:39.840 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_metadata' removed 2024-04-17T06:13:39.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:39.845+0000 7f19f6ca5640 1 -- 172.21.15.47:0/3506323713 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f19dc0a3da0 msgr2=0x7f19dc0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:13:39.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:39.845+0000 7f19f6ca5640 1 --2- 172.21.15.47:0/3506323713 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f19dc0a3da0 0x7f19dc0a6260 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f19e0005c40 tx=0x7f19e000a780 comp rx=0 tx=0).stop 2024-04-17T06:13:39.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:39.845+0000 7f19f6ca5640 1 -- 172.21.15.47:0/3506323713 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f19f01985b0 msgr2=0x7f19f0198a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:13:39.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:39.845+0000 7f19f6ca5640 1 --2- 172.21.15.47:0/3506323713 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f19f01985b0 0x7f19f0198a60 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f19e800c970 tx=0x7f19e800ce30 comp rx=0 tx=0).stop 2024-04-17T06:13:39.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:39.846+0000 7f19f6ca5640 1 -- 172.21.15.47:0/3506323713 shutdown_connections 2024-04-17T06:13:39.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:39.846+0000 7f19f6ca5640 1 --2- 172.21.15.47:0/3506323713 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f19dc0a3da0 0x7f19dc0a6260 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:13:39.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:39.846+0000 7f19f6ca5640 1 --2- 172.21.15.47:0/3506323713 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f19f01985b0 0x7f19f0198a60 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:13:39.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:39.846+0000 7f19f6ca5640 1 --2- 172.21.15.47:0/3506323713 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f19f0107320 0x7f19f019e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:13:39.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:39.846+0000 7f19f6ca5640 1 --2- 172.21.15.47:0/3506323713 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19f006a870 0x7f19f019e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:13:39.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:39.846+0000 7f19f6ca5640 1 -- 172.21.15.47:0/3506323713 >> 172.21.15.47:0/3506323713 conn(0x7f19f0077c50 msgr2=0x7f19f00ff1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:13:39.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:39.846+0000 7f19f6ca5640 1 -- 172.21.15.47:0/3506323713 shutdown_connections 2024-04-17T06:13:39.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:39.846+0000 7f19f6ca5640 1 -- 172.21.15.47:0/3506323713 wait complete. 2024-04-17T06:13:39.861 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-17T06:13:39.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:39.980+0000 7f137bfba640 1 -- 172.21.15.47:0/1704289431 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1374106170 msgr2=0x7f137410dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:13:39.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:39.980+0000 7f137bfba640 1 --2- 172.21.15.47:0/1704289431 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1374106170 0x7f137410dd40 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f136c009ca0 tx=0x7f136c02c0f0 comp rx=0 tx=0).stop 2024-04-17T06:13:39.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:39.980+0000 7f137bfba640 1 -- 172.21.15.47:0/1704289431 shutdown_connections 2024-04-17T06:13:39.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:39.980+0000 7f137bfba640 1 --2- 172.21.15.47:0/1704289431 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1374106170 0x7f137410dd40 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:13:39.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:39.980+0000 7f137bfba640 1 --2- 172.21.15.47:0/1704289431 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1374101a50 0x7f1374105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:13:39.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:39.980+0000 7f137bfba640 1 --2- 172.21.15.47:0/1704289431 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f13741010a0 0x7f1374101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:13:39.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:39.980+0000 7f137bfba640 1 -- 172.21.15.47:0/1704289431 >> 172.21.15.47:0/1704289431 conn(0x7f1374077f60 msgr2=0x7f1374078370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:13:39.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:39.980+0000 7f137bfba640 1 -- 172.21.15.47:0/1704289431 shutdown_connections 2024-04-17T06:13:39.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:39.980+0000 7f137bfba640 1 -- 172.21.15.47:0/1704289431 wait complete. 2024-04-17T06:13:39.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:39.980+0000 7f137bfba640 1 Processor -- start 2024-04-17T06:13:39.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:39.980+0000 7f137bfba640 1 -- start start 2024-04-17T06:13:39.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:39.981+0000 7f137bfba640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13741010a0 0x7f13741976f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:13:39.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:39.981+0000 7f137bfba640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1374106170 0x7f1374197c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:13:39.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:39.981+0000 7f137bfba640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f137419bfb0 0x7f137419c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:13:39.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:39.981+0000 7f137bfba640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f1374111210 con 0x7f13741010a0 2024-04-17T06:13:39.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:39.981+0000 7f137bfba640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1374111090 con 0x7f137419bfb0 2024-04-17T06:13:39.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:39.981+0000 7f137bfba640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1374111390 con 0x7f1374106170 2024-04-17T06:13:39.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:39.981+0000 7f1379d56640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13741010a0 0x7f13741976f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:13:39.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:39.981+0000 7f1379d56640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13741010a0 0x7f13741976f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46818/0 (socket says 172.21.15.47:46818) 2024-04-17T06:13:39.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:39.981+0000 7f1379d56640 1 -- 172.21.15.47:0/957533908 learned_addr learned my addr 172.21.15.47:0/957533908 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-17T06:13:39.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:39.981+0000 7f1379555640 1 --2- 172.21.15.47:0/957533908 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1374106170 0x7f1374197c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:13:39.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:39.981+0000 7f137a557640 1 --2- 172.21.15.47:0/957533908 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f137419bfb0 0x7f137419c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:13:39.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:39.981+0000 7f1379d56640 1 -- 172.21.15.47:0/957533908 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1374106170 msgr2=0x7f1374197c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:13:39.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:39.981+0000 7f1379d56640 1 --2- 172.21.15.47:0/957533908 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1374106170 0x7f1374197c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:13:39.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:39.981+0000 7f1379d56640 1 -- 172.21.15.47:0/957533908 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f137419bfb0 msgr2=0x7f137419c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:13:39.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:39.981+0000 7f1379d56640 1 --2- 172.21.15.47:0/957533908 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f137419bfb0 0x7f137419c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:13:39.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:39.981+0000 7f1379d56640 1 -- 172.21.15.47:0/957533908 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f137419caa0 con 0x7f13741010a0 2024-04-17T06:13:39.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:39.981+0000 7f1379555640 1 --2- 172.21.15.47:0/957533908 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1374106170 0x7f1374197c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:13:39.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:39.981+0000 7f137a557640 1 --2- 172.21.15.47:0/957533908 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f137419bfb0 0x7f137419c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:13:39.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:39.981+0000 7f1379d56640 1 --2- 172.21.15.47:0/957533908 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13741010a0 0x7f13741976f0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f137000c890 tx=0x7f137000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:13:39.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:39.981+0000 7f136affd640 1 -- 172.21.15.47:0/957533908 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1370004220 con 0x7f13741010a0 2024-04-17T06:13:39.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:39.981+0000 7f136affd640 1 -- 172.21.15.47:0/957533908 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13700043c0 con 0x7f13741010a0 2024-04-17T06:13:39.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:39.981+0000 7f137bfba640 1 -- 172.21.15.47:0/957533908 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f137419cd90 con 0x7f13741010a0 2024-04-17T06:13:39.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:39.981+0000 7f136affd640 1 -- 172.21.15.47:0/957533908 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13700049e0 con 0x7f13741010a0 2024-04-17T06:13:39.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:39.982+0000 7f137bfba640 1 -- 172.21.15.47:0/957533908 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13741a4580 con 0x7f13741010a0 2024-04-17T06:13:39.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:39.982+0000 7f137bfba640 1 -- 172.21.15.47:0/957533908 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1374068d80 con 0x7f13741010a0 2024-04-17T06:13:39.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:39.984+0000 7f136affd640 1 -- 172.21.15.47:0/957533908 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f13700079b0 con 0x7f13741010a0 2024-04-17T06:13:39.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:39.984+0000 7f136affd640 1 --2- 172.21.15.47:0/957533908 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f13600a3d20 0x7f13600a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:13:39.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:39.984+0000 7f1379555640 1 --2- 172.21.15.47:0/957533908 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f13600a3d20 0x7f13600a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:13:39.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:39.984+0000 7f136affd640 1 -- 172.21.15.47:0/957533908 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 8202+0+0 (secure 0 0 0) 0x7f13700d0800 con 0x7f13741010a0 2024-04-17T06:13:39.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:39.985+0000 7f1379555640 1 --2- 172.21.15.47:0/957533908 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f13600a3d20 0x7f13600a61e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f1364005fd0 tx=0x7f1364005850 comp rx=0 tx=0).ready entity=mgr.24923 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:13:39.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:39.988+0000 7f136affd640 1 -- 172.21.15.47:0/957533908 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f1370014030 con 0x7f13741010a0 2024-04-17T06:13:40.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:13:39 smithi098 ceph-mon[36516]: pgmap v391: 97 pgs: 97 active+clean; 5.9 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1023 B/s wr, 7 op/s 2024-04-17T06:13:40.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:13:39 smithi098 ceph-mon[36516]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-17T06:13:40.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:13:39 smithi098 ceph-mon[36516]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-17T06:13:40.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:13:39 smithi098 ceph-mon[36516]: Cluster is now healthy 2024-04-17T06:13:40.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:13:39 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/1137007243' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-17T06:13:40.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:13:39 smithi098 ceph-mon[36516]: mds.? [v2:172.21.15.98:6832/4280181545,v1:172.21.15.98:6833/4280181545] up:boot 2024-04-17T06:13:40.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:13:39 smithi098 ceph-mon[36516]: mds.? [v2:172.21.15.98:6836/3293808034,v1:172.21.15.98:6837/3293808034] up:boot 2024-04-17T06:13:40.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:13:39 smithi098 ceph-mon[36516]: mds.? [v2:172.21.15.98:6838/1795055910,v1:172.21.15.98:6839/1795055910] up:boot 2024-04-17T06:13:40.134 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:13:39 smithi098 ceph-mon[36516]: mds.? [v2:172.21.15.103:6836/3996439608,v1:172.21.15.103:6838/3996439608] up:boot 2024-04-17T06:13:40.134 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:13:39 smithi098 ceph-mon[36516]: mds.? [v2:172.21.15.103:6837/1528258807,v1:172.21.15.103:6839/1528258807] up:boot 2024-04-17T06:13:40.134 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:13:39 smithi098 ceph-mon[36516]: mds.? [v2:172.21.15.103:6834/641946772,v1:172.21.15.103:6835/641946772] up:boot 2024-04-17T06:13:40.134 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:13:39 smithi098 ceph-mon[36516]: fsmap 12 up:standby 2024-04-17T06:13:40.134 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:13:39 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-17T06:13:40.134 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:13:39 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-17T06:13:40.134 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:13:39 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-17T06:13:40.134 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:13:39 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-17T06:13:40.134 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:13:39 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-17T06:13:40.134 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:13:39 smithi098 ceph-mon[36516]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-17T06:13:40.134 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:13:39 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/3506323713' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-17T06:13:40.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:13:39 smithi047 ceph-mon[32223]: pgmap v391: 97 pgs: 97 active+clean; 5.9 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1023 B/s wr, 7 op/s 2024-04-17T06:13:40.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:13:39 smithi047 ceph-mon[32223]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-17T06:13:40.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:13:39 smithi047 ceph-mon[32223]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-17T06:13:40.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:13:39 smithi047 ceph-mon[32223]: Cluster is now healthy 2024-04-17T06:13:40.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:13:39 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/1137007243' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-17T06:13:40.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:13:39 smithi047 ceph-mon[32223]: mds.? [v2:172.21.15.98:6832/4280181545,v1:172.21.15.98:6833/4280181545] up:boot 2024-04-17T06:13:40.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:13:39 smithi047 ceph-mon[32223]: mds.? [v2:172.21.15.98:6836/3293808034,v1:172.21.15.98:6837/3293808034] up:boot 2024-04-17T06:13:40.140 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:13:39 smithi047 ceph-mon[32223]: mds.? [v2:172.21.15.98:6838/1795055910,v1:172.21.15.98:6839/1795055910] up:boot 2024-04-17T06:13:40.140 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:13:39 smithi047 ceph-mon[32223]: mds.? [v2:172.21.15.103:6836/3996439608,v1:172.21.15.103:6838/3996439608] up:boot 2024-04-17T06:13:40.140 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:13:39 smithi047 ceph-mon[32223]: mds.? [v2:172.21.15.103:6837/1528258807,v1:172.21.15.103:6839/1528258807] up:boot 2024-04-17T06:13:40.140 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:13:39 smithi047 ceph-mon[32223]: mds.? [v2:172.21.15.103:6834/641946772,v1:172.21.15.103:6835/641946772] up:boot 2024-04-17T06:13:40.140 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:13:39 smithi047 ceph-mon[32223]: fsmap 12 up:standby 2024-04-17T06:13:40.140 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:13:39 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-17T06:13:40.140 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:13:39 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-17T06:13:40.140 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:13:39 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-17T06:13:40.140 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:13:39 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-17T06:13:40.140 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:13:39 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-17T06:13:40.140 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:13:39 smithi047 ceph-mon[32223]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-17T06:13:40.140 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:13:39 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/3506323713' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-17T06:13:40.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:40.177+0000 7f137bfba640 1 -- 172.21.15.47:0/957533908 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f1374198a40 con 0x7f13741010a0 2024-04-17T06:13:40.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:13:39 smithi103 ceph-mon[36243]: pgmap v391: 97 pgs: 97 active+clean; 5.9 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1023 B/s wr, 7 op/s 2024-04-17T06:13:40.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:13:39 smithi103 ceph-mon[36243]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-17T06:13:40.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:13:39 smithi103 ceph-mon[36243]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-17T06:13:40.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:13:39 smithi103 ceph-mon[36243]: Cluster is now healthy 2024-04-17T06:13:40.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:13:39 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/1137007243' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-17T06:13:40.285 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:13:39 smithi103 ceph-mon[36243]: mds.? [v2:172.21.15.98:6832/4280181545,v1:172.21.15.98:6833/4280181545] up:boot 2024-04-17T06:13:40.285 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:13:39 smithi103 ceph-mon[36243]: mds.? [v2:172.21.15.98:6836/3293808034,v1:172.21.15.98:6837/3293808034] up:boot 2024-04-17T06:13:40.285 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:13:39 smithi103 ceph-mon[36243]: mds.? [v2:172.21.15.98:6838/1795055910,v1:172.21.15.98:6839/1795055910] up:boot 2024-04-17T06:13:40.285 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:13:39 smithi103 ceph-mon[36243]: mds.? [v2:172.21.15.103:6836/3996439608,v1:172.21.15.103:6838/3996439608] up:boot 2024-04-17T06:13:40.285 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:13:39 smithi103 ceph-mon[36243]: mds.? [v2:172.21.15.103:6837/1528258807,v1:172.21.15.103:6839/1528258807] up:boot 2024-04-17T06:13:40.285 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:13:39 smithi103 ceph-mon[36243]: mds.? [v2:172.21.15.103:6834/641946772,v1:172.21.15.103:6835/641946772] up:boot 2024-04-17T06:13:40.285 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:13:39 smithi103 ceph-mon[36243]: fsmap 12 up:standby 2024-04-17T06:13:40.285 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:13:39 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-17T06:13:40.285 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:13:39 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-17T06:13:40.285 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:13:39 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-17T06:13:40.285 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:13:39 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-17T06:13:40.286 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:13:39 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-17T06:13:40.286 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:13:39 smithi103 ceph-mon[36243]: from='mgr.24923 172.21.15.47:0/2751527419' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-17T06:13:40.286 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:13:39 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/3506323713' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-17T06:13:40.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:40.848+0000 7f136affd640 1 -- 172.21.15.47:0/957533908 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v109) ==== 169+0+0 (secure 0 0 0) 0x7f137009cfe0 con 0x7f13741010a0 2024-04-17T06:13:40.847 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_data' removed 2024-04-17T06:13:40.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:40.852+0000 7f137bfba640 1 -- 172.21.15.47:0/957533908 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f13600a3d20 msgr2=0x7f13600a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:13:40.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:40.852+0000 7f137bfba640 1 --2- 172.21.15.47:0/957533908 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f13600a3d20 0x7f13600a61e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f1364005fd0 tx=0x7f1364005850 comp rx=0 tx=0).stop 2024-04-17T06:13:40.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:40.852+0000 7f137bfba640 1 -- 172.21.15.47:0/957533908 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13741010a0 msgr2=0x7f13741976f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:13:40.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:40.852+0000 7f137bfba640 1 --2- 172.21.15.47:0/957533908 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13741010a0 0x7f13741976f0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f137000c890 tx=0x7f137000cd50 comp rx=0 tx=0).stop 2024-04-17T06:13:40.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:40.853+0000 7f137bfba640 1 -- 172.21.15.47:0/957533908 shutdown_connections 2024-04-17T06:13:40.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:40.853+0000 7f137bfba640 1 --2- 172.21.15.47:0/957533908 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f137419bfb0 0x7f137419c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:13:40.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:40.853+0000 7f137bfba640 1 --2- 172.21.15.47:0/957533908 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1374106170 0x7f1374197c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:13:40.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:40.853+0000 7f137bfba640 1 --2- 172.21.15.47:0/957533908 >> [v2:172.21.15.47:6800/255579447,v1:172.21.15.47:6801/255579447] conn(0x7f13600a3d20 0x7f13600a61e0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:13:40.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:40.853+0000 7f137bfba640 1 --2- 172.21.15.47:0/957533908 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13741010a0 0x7f13741976f0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:13:40.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:40.853+0000 7f137bfba640 1 -- 172.21.15.47:0/957533908 >> 172.21.15.47:0/957533908 conn(0x7f1374077f60 msgr2=0x7f13740fee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:13:40.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:40.853+0000 7f137bfba640 1 -- 172.21.15.47:0/957533908 shutdown_connections 2024-04-17T06:13:40.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-17T06:13:40.853+0000 7f137bfba640 1 -- 172.21.15.47:0/957533908 wait complete. 2024-04-17T06:13:40.868 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-17T06:13:40.901 DEBUG:teuthology.orchestra.run.smithi098:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-17T06:13:40.934 DEBUG:teuthology.orchestra.run.smithi103:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-17T06:13:40.969 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-04-17T06:13:40.970 DEBUG:teuthology.orchestra.run.smithi047:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-17T06:13:40.987 DEBUG:teuthology.orchestra.run.smithi098:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-17T06:13:41.004 DEBUG:teuthology.orchestra.run.smithi103:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-17T06:13:41.026 INFO:tasks.cephadm:Stopping all daemons... 2024-04-17T06:13:41.026 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-04-17T06:13:41.027 DEBUG:teuthology.orchestra.run.smithi047:> sudo systemctl stop ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@mon.a 2024-04-17T06:13:41.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:13:40 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/3506323713' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-17T06:13:41.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:13:40 smithi098 ceph-mon[36516]: osdmap e108: 12 total, 12 up, 12 in 2024-04-17T06:13:41.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:13:40 smithi098 ceph-mon[36516]: from='client.? 172.21.15.47:0/957533908' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-17T06:13:41.138 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:13:40 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/3506323713' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-17T06:13:41.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:13:40 smithi047 ceph-mon[32223]: osdmap e108: 12 total, 12 up, 12 in 2024-04-17T06:13:41.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:13:40 smithi047 ceph-mon[32223]: from='client.? 172.21.15.47:0/957533908' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-17T06:13:41.139 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:13:41 smithi047 systemd[1]: Stopping Ceph mon.a for 565db2d8-fc7e-11ee-bc91-c7b262605968... 2024-04-17T06:13:41.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:13:40 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/3506323713' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-17T06:13:41.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:13:40 smithi103 ceph-mon[36243]: osdmap e108: 12 total, 12 up, 12 in 2024-04-17T06:13:41.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:13:40 smithi103 ceph-mon[36243]: from='client.? 172.21.15.47:0/957533908' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-17T06:13:41.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:13:41 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mon-a[32219]: 2024-04-17T06:13:41.342+0000 7f69a4d63700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-17T06:13:41.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:13:41 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mon-a[32219]: 2024-04-17T06:13:41.342+0000 7f69a4d63700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-04-17T06:13:41.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:13:41 smithi047 podman[93485]: 2024-04-17 06:13:41.435212419 +0000 UTC m=+0.214744942 container died d01e5baa55cce7efcd28623ab8975076add23c7e219e8db2d41dc3fb0ba7723d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mon-a, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-04-17T06:13:42.075 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:13:41 smithi047 podman[93485]: 2024-04-17 06:13:41.834294636 +0000 UTC m=+0.613827157 container cleanup d01e5baa55cce7efcd28623ab8975076add23c7e219e8db2d41dc3fb0ba7723d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mon-a, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-04-17T06:13:42.075 INFO:journalctl@ceph.mon.a.smithi047.stdout:Apr 17 06:13:41 smithi047 bash[93485]: ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mon-a 2024-04-17T06:13:42.221 DEBUG:teuthology.orchestra.run.smithi047:> sudo pkill -f 'journalctl -f -n 0 -u ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@mon.a.service' 2024-04-17T06:13:42.273 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-17T06:13:42.273 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-04-17T06:13:42.274 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-04-17T06:13:42.274 DEBUG:teuthology.orchestra.run.smithi098:> sudo systemctl stop ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@mon.b 2024-04-17T06:13:42.600 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:13:42 smithi098 systemd[1]: Stopping Ceph mon.b for 565db2d8-fc7e-11ee-bc91-c7b262605968... 2024-04-17T06:13:42.886 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:13:42 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mon-b[36512]: 2024-04-17T06:13:42.601+0000 7f0b999ee700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-17T06:13:42.886 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:13:42 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mon-b[36512]: 2024-04-17T06:13:42.601+0000 7f0b999ee700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-04-17T06:13:42.886 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:13:42 smithi098 podman[74152]: 2024-04-17 06:13:42.622597972 +0000 UTC m=+0.155367256 container died f85004f63c848e31dda15d63f7cc7a009074a13eba7a1a72796f0c85fbce12a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mon-b, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-17T06:13:43.269 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:13:42 smithi098 podman[74152]: 2024-04-17 06:13:42.986137375 +0000 UTC m=+0.518906648 container cleanup f85004f63c848e31dda15d63f7cc7a009074a13eba7a1a72796f0c85fbce12a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mon-b, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-17T06:13:43.269 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 06:13:42 smithi098 bash[74152]: ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mon-b 2024-04-17T06:13:43.399 DEBUG:teuthology.orchestra.run.smithi098:> sudo pkill -f 'journalctl -f -n 0 -u ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@mon.b.service' 2024-04-17T06:13:43.470 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-17T06:13:43.470 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-04-17T06:13:43.471 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-04-17T06:13:43.471 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@mon.c 2024-04-17T06:13:43.784 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:13:43 smithi103 systemd[1]: Stopping Ceph mon.c for 565db2d8-fc7e-11ee-bc91-c7b262605968... 2024-04-17T06:13:44.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:13:43 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mon-c[36228]: 2024-04-17T06:13:43.788+0000 7fd05a1b7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-17T06:13:44.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:13:43 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mon-c[36228]: 2024-04-17T06:13:43.788+0000 7fd05a1b7700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-04-17T06:13:44.284 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:13:43 smithi103 podman[73335]: 2024-04-17 06:13:43.993911397 +0000 UTC m=+0.329831273 container died 382e5609334324f373df64b5437a9fcd721af091de127a749ee84c4da2bc8a01 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mon-c, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-17T06:13:44.649 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:13:44 smithi103 podman[73335]: 2024-04-17 06:13:44.416278289 +0000 UTC m=+0.752198143 container cleanup 382e5609334324f373df64b5437a9fcd721af091de127a749ee84c4da2bc8a01 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mon-c, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-04-17T06:13:44.649 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:13:44 smithi103 bash[73335]: ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mon-c 2024-04-17T06:13:44.889 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@mon.c.service' 2024-04-17T06:13:44.913 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:13:44 smithi103 podman[73349]: 2024-04-17 06:13:44.651422519 +0000 UTC m=+0.653171807 container remove 382e5609334324f373df64b5437a9fcd721af091de127a749ee84c4da2bc8a01 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mon-c, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-17T06:13:44.914 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:13:44 smithi103 systemd[1]: ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@mon.c.service: Deactivated successfully. 2024-04-17T06:13:44.914 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:13:44 smithi103 systemd[1]: Stopped Ceph mon.c for 565db2d8-fc7e-11ee-bc91-c7b262605968. 2024-04-17T06:13:44.914 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 06:13:44 smithi103 systemd[1]: ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@mon.c.service: Consumed 11.963s CPU time. 2024-04-17T06:13:44.938 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-17T06:13:44.939 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-04-17T06:13:44.939 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-04-17T06:13:44.939 DEBUG:teuthology.orchestra.run.smithi047:> sudo systemctl stop ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@mgr.x 2024-04-17T06:13:45.262 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 06:13:44 smithi047 systemd[1]: Stopping Ceph mgr.x for 565db2d8-fc7e-11ee-bc91-c7b262605968... 2024-04-17T06:13:45.580 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 06:13:45 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x[32453]: 2024-04-17T06:13:45.264+0000 7effbf2d2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-17T06:13:45.580 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 06:13:45 smithi047 podman[93605]: 2024-04-17 06:13:45.295128154 +0000 UTC m=+0.147406767 container died b8f280f6ff25fce571f13037d77ede78db5b2c8ce28f0135be7d9abc3d5c33ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240416) 2024-04-17T06:13:45.889 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 06:13:45 smithi047 podman[93605]: 2024-04-17 06:13:45.68153451 +0000 UTC m=+0.533813118 container cleanup b8f280f6ff25fce571f13037d77ede78db5b2c8ce28f0135be7d9abc3d5c33ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-04-17T06:13:45.889 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Apr 17 06:13:45 smithi047 bash[93605]: ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-x 2024-04-17T06:13:46.103 DEBUG:teuthology.orchestra.run.smithi047:> sudo pkill -f 'journalctl -f -n 0 -u ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@mgr.x.service' 2024-04-17T06:13:46.141 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-17T06:13:46.142 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-04-17T06:13:46.142 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-04-17T06:13:46.142 DEBUG:teuthology.orchestra.run.smithi098:> sudo systemctl stop ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@mgr.y 2024-04-17T06:13:46.469 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 17 06:13:46 smithi098 systemd[1]: Stopping Ceph mgr.y for 565db2d8-fc7e-11ee-bc91-c7b262605968... 2024-04-17T06:13:46.768 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 17 06:13:46 smithi098 podman[74275]: 2024-04-17 06:13:46.471878628 +0000 UTC m=+0.131665575 container died a400e56efbbfad06eec1931950d736347e830916175e8e61fe9389b80f60ebb7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-y, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-17T06:13:47.086 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 17 06:13:46 smithi098 podman[74275]: 2024-04-17 06:13:46.861949582 +0000 UTC m=+0.521736520 container cleanup a400e56efbbfad06eec1931950d736347e830916175e8e61fe9389b80f60ebb7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-y, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-17T06:13:47.086 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 17 06:13:46 smithi098 bash[74275]: ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-y 2024-04-17T06:13:47.213 DEBUG:teuthology.orchestra.run.smithi098:> sudo pkill -f 'journalctl -f -n 0 -u ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@mgr.y.service' 2024-04-17T06:13:47.250 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-17T06:13:47.250 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-04-17T06:13:47.250 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-04-17T06:13:47.250 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@mgr.z 2024-04-17T06:13:47.534 INFO:journalctl@ceph.mgr.z.smithi103.stdout:Apr 17 06:13:47 smithi103 systemd[1]: Stopping Ceph mgr.z for 565db2d8-fc7e-11ee-bc91-c7b262605968... 2024-04-17T06:13:47.862 INFO:journalctl@ceph.mgr.z.smithi103.stdout:Apr 17 06:13:47 smithi103 podman[73457]: 2024-04-17 06:13:47.575133071 +0000 UTC m=+0.138135012 container died 6af7082b7f514ef604e356ecf1c23d76aa639607f98383bf74b6e2f1ce3084aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-z, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, ceph=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-17T06:13:48.238 INFO:journalctl@ceph.mgr.z.smithi103.stdout:Apr 17 06:13:47 smithi103 podman[73457]: 2024-04-17 06:13:47.963674449 +0000 UTC m=+0.526676392 container cleanup 6af7082b7f514ef604e356ecf1c23d76aa639607f98383bf74b6e2f1ce3084aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-z, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-04-17T06:13:48.238 INFO:journalctl@ceph.mgr.z.smithi103.stdout:Apr 17 06:13:47 smithi103 bash[73457]: ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mgr-z 2024-04-17T06:13:48.371 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@mgr.z.service' 2024-04-17T06:13:48.410 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-17T06:13:48.410 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-04-17T06:13:48.411 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-04-17T06:13:48.411 DEBUG:teuthology.orchestra.run.smithi047:> sudo systemctl stop ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@osd.0 2024-04-17T06:13:48.735 INFO:journalctl@ceph.osd.0.smithi047.stdout:Apr 17 06:13:48 smithi047 systemd[1]: Stopping Ceph osd.0 for 565db2d8-fc7e-11ee-bc91-c7b262605968... 2024-04-17T06:13:49.139 INFO:journalctl@ceph.osd.0.smithi047.stdout:Apr 17 06:13:48 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-0[46923]: 2024-04-17T06:13:48.735+0000 7f4c88d17700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-17T06:13:49.139 INFO:journalctl@ceph.osd.0.smithi047.stdout:Apr 17 06:13:48 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-0[46923]: 2024-04-17T06:13:48.735+0000 7f4c88d17700 -1 osd.0 109 *** Got signal Terminated *** 2024-04-17T06:13:49.139 INFO:journalctl@ceph.osd.0.smithi047.stdout:Apr 17 06:13:48 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-0[46923]: 2024-04-17T06:13:48.735+0000 7f4c88d17700 -1 osd.0 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-17T06:13:54.541 INFO:journalctl@ceph.osd.0.smithi047.stdout:Apr 17 06:13:54 smithi047 podman[93730]: 2024-04-17 06:13:54.161973215 +0000 UTC m=+5.548601282 container died 57ab3478e010876ac9dab52e4ba43732c8cd124a9251edf2f45d42b28833a6de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-0, GIT_CLEAN=True, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-17T06:13:54.889 INFO:journalctl@ceph.osd.0.smithi047.stdout:Apr 17 06:13:54 smithi047 podman[93730]: 2024-04-17 06:13:54.634455793 +0000 UTC m=+6.021083864 container cleanup 57ab3478e010876ac9dab52e4ba43732c8cd124a9251edf2f45d42b28833a6de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-04-17T06:13:54.889 INFO:journalctl@ceph.osd.0.smithi047.stdout:Apr 17 06:13:54 smithi047 bash[93730]: ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-0 2024-04-17T06:13:55.251 INFO:journalctl@ceph.osd.0.smithi047.stdout:Apr 17 06:13:54 smithi047 podman[93750]: 2024-04-17 06:13:54.936366479 +0000 UTC m=+0.766694715 container remove 57ab3478e010876ac9dab52e4ba43732c8cd124a9251edf2f45d42b28833a6de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-17T06:13:55.514 INFO:journalctl@ceph.osd.0.smithi047.stdout:Apr 17 06:13:55 smithi047 podman[93839]: 2024-04-17 06:13:55.253865741 +0000 UTC m=+0.103983345 container create 9242813868b3f57b341cc7e3a1176afb479e004b5edc86850959901abcfdebff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-17T06:13:55.514 INFO:journalctl@ceph.osd.0.smithi047.stdout:Apr 17 06:13:55 smithi047 podman[93839]: 2024-04-17 06:13:55.168006531 +0000 UTC m=+0.018124135 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-17T06:13:55.514 INFO:journalctl@ceph.osd.0.smithi047.stdout:Apr 17 06:13:55 smithi047 podman[93839]: 2024-04-17 06:13:55.515893 +0000 UTC m=+0.366010601 container init 9242813868b3f57b341cc7e3a1176afb479e004b5edc86850959901abcfdebff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-17T06:13:55.817 INFO:journalctl@ceph.osd.0.smithi047.stdout:Apr 17 06:13:55 smithi047 podman[93839]: 2024-04-17 06:13:55.519731728 +0000 UTC m=+0.369849322 container start 9242813868b3f57b341cc7e3a1176afb479e004b5edc86850959901abcfdebff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-0-deactivate, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-17T06:13:55.817 INFO:journalctl@ceph.osd.0.smithi047.stdout:Apr 17 06:13:55 smithi047 podman[93839]: 2024-04-17 06:13:55.615025447 +0000 UTC m=+0.465143049 container attach 9242813868b3f57b341cc7e3a1176afb479e004b5edc86850959901abcfdebff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-17T06:13:56.139 INFO:journalctl@ceph.osd.0.smithi047.stdout:Apr 17 06:13:55 smithi047 podman[93839]: 2024-04-17 06:13:55.819914792 +0000 UTC m=+0.670032387 container died 9242813868b3f57b341cc7e3a1176afb479e004b5edc86850959901abcfdebff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_CLEAN=True, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-17T06:13:56.486 DEBUG:teuthology.orchestra.run.smithi047:> sudo pkill -f 'journalctl -f -n 0 -u ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@osd.0.service' 2024-04-17T06:13:56.532 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-17T06:13:56.532 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-04-17T06:13:56.532 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-04-17T06:13:56.532 DEBUG:teuthology.orchestra.run.smithi098:> sudo systemctl stop ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@osd.1 2024-04-17T06:13:56.848 INFO:journalctl@ceph.osd.1.smithi098.stdout:Apr 17 06:13:56 smithi098 systemd[1]: Stopping Ceph osd.1 for 565db2d8-fc7e-11ee-bc91-c7b262605968... 2024-04-17T06:13:57.133 INFO:journalctl@ceph.osd.1.smithi098.stdout:Apr 17 06:13:56 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-1[41120]: 2024-04-17T06:13:56.849+0000 7fbe7f9cc700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-17T06:13:57.133 INFO:journalctl@ceph.osd.1.smithi098.stdout:Apr 17 06:13:56 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-1[41120]: 2024-04-17T06:13:56.849+0000 7fbe7f9cc700 -1 osd.1 109 *** Got signal Terminated *** 2024-04-17T06:13:57.134 INFO:journalctl@ceph.osd.1.smithi098.stdout:Apr 17 06:13:56 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-1[41120]: 2024-04-17T06:13:56.849+0000 7fbe7f9cc700 -1 osd.1 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-17T06:14:02.557 INFO:journalctl@ceph.osd.1.smithi098.stdout:Apr 17 06:14:02 smithi098 podman[74397]: 2024-04-17 06:14:02.262876961 +0000 UTC m=+5.535693342 container died fb341992a6ef91e3821f9f082eb20c654dbf089cbf9db6623af67970b75778bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-1, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-17T06:14:02.883 INFO:journalctl@ceph.osd.1.smithi098.stdout:Apr 17 06:14:02 smithi098 podman[74397]: 2024-04-17 06:14:02.64240657 +0000 UTC m=+5.915222929 container cleanup fb341992a6ef91e3821f9f082eb20c654dbf089cbf9db6623af67970b75778bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-17T06:14:02.883 INFO:journalctl@ceph.osd.1.smithi098.stdout:Apr 17 06:14:02 smithi098 bash[74397]: ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-1 2024-04-17T06:14:03.251 INFO:journalctl@ceph.osd.1.smithi098.stdout:Apr 17 06:14:02 smithi098 podman[74414]: 2024-04-17 06:14:02.927358635 +0000 UTC m=+0.661496269 container remove fb341992a6ef91e3821f9f082eb20c654dbf089cbf9db6623af67970b75778bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-1, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-17T06:14:03.531 INFO:journalctl@ceph.osd.1.smithi098.stdout:Apr 17 06:14:03 smithi098 podman[74503]: 2024-04-17 06:14:03.156479426 +0000 UTC m=+0.017807832 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-17T06:14:03.531 INFO:journalctl@ceph.osd.1.smithi098.stdout:Apr 17 06:14:03 smithi098 podman[74503]: 2024-04-17 06:14:03.270254067 +0000 UTC m=+0.131582476 container create 9396e35e219363812c0cd3e49301d5e0f0af7a329719586b5f4a287ddf24e28c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-17T06:14:03.531 INFO:journalctl@ceph.osd.1.smithi098.stdout:Apr 17 06:14:03 smithi098 podman[74503]: 2024-04-17 06:14:03.532618106 +0000 UTC m=+0.393946489 container init 9396e35e219363812c0cd3e49301d5e0f0af7a329719586b5f4a287ddf24e28c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240416) 2024-04-17T06:14:03.823 INFO:journalctl@ceph.osd.1.smithi098.stdout:Apr 17 06:14:03 smithi098 podman[74503]: 2024-04-17 06:14:03.536011204 +0000 UTC m=+0.397339597 container start 9396e35e219363812c0cd3e49301d5e0f0af7a329719586b5f4a287ddf24e28c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-04-17T06:14:03.823 INFO:journalctl@ceph.osd.1.smithi098.stdout:Apr 17 06:14:03 smithi098 podman[74503]: 2024-04-17 06:14:03.608771591 +0000 UTC m=+0.470099985 container attach 9396e35e219363812c0cd3e49301d5e0f0af7a329719586b5f4a287ddf24e28c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5) 2024-04-17T06:14:04.133 INFO:journalctl@ceph.osd.1.smithi098.stdout:Apr 17 06:14:03 smithi098 podman[74503]: 2024-04-17 06:14:03.826466356 +0000 UTC m=+0.687794748 container died 9396e35e219363812c0cd3e49301d5e0f0af7a329719586b5f4a287ddf24e28c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-04-17T06:14:04.464 DEBUG:teuthology.orchestra.run.smithi098:> sudo pkill -f 'journalctl -f -n 0 -u ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@osd.1.service' 2024-04-17T06:14:04.506 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-17T06:14:04.506 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-04-17T06:14:04.507 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-04-17T06:14:04.507 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@osd.2 2024-04-17T06:14:04.816 INFO:journalctl@ceph.osd.2.smithi103.stdout:Apr 17 06:14:04 smithi103 systemd[1]: Stopping Ceph osd.2 for 565db2d8-fc7e-11ee-bc91-c7b262605968... 2024-04-17T06:14:05.077 INFO:journalctl@ceph.osd.2.smithi103.stdout:Apr 17 06:14:04 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-2[41376]: 2024-04-17T06:14:04.818+0000 7fcce1653700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-17T06:14:05.077 INFO:journalctl@ceph.osd.2.smithi103.stdout:Apr 17 06:14:04 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-2[41376]: 2024-04-17T06:14:04.818+0000 7fcce1653700 -1 osd.2 109 *** Got signal Terminated *** 2024-04-17T06:14:05.077 INFO:journalctl@ceph.osd.2.smithi103.stdout:Apr 17 06:14:04 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-2[41376]: 2024-04-17T06:14:04.818+0000 7fcce1653700 -1 osd.2 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-17T06:14:10.626 INFO:journalctl@ceph.osd.2.smithi103.stdout:Apr 17 06:14:10 smithi103 podman[73580]: 2024-04-17 06:14:10.32337269 +0000 UTC m=+5.625702733 container died 02645ff29b180891fb3aaad8931416e357d84ff1d7096781b59f57a29e77688b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-04-17T06:14:10.984 INFO:journalctl@ceph.osd.2.smithi103.stdout:Apr 17 06:14:10 smithi103 podman[73580]: 2024-04-17 06:14:10.727197898 +0000 UTC m=+6.029527908 container cleanup 02645ff29b180891fb3aaad8931416e357d84ff1d7096781b59f57a29e77688b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-17T06:14:10.985 INFO:journalctl@ceph.osd.2.smithi103.stdout:Apr 17 06:14:10 smithi103 bash[73580]: ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-2 2024-04-17T06:14:11.284 INFO:journalctl@ceph.osd.2.smithi103.stdout:Apr 17 06:14:10 smithi103 podman[73609]: 2024-04-17 06:14:10.987063488 +0000 UTC m=+0.661514206 container remove 02645ff29b180891fb3aaad8931416e357d84ff1d7096781b59f57a29e77688b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=) 2024-04-17T06:14:11.673 INFO:journalctl@ceph.osd.2.smithi103.stdout:Apr 17 06:14:11 smithi103 podman[73701]: 2024-04-17 06:14:11.244231591 +0000 UTC m=+0.018217575 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-17T06:14:11.673 INFO:journalctl@ceph.osd.2.smithi103.stdout:Apr 17 06:14:11 smithi103 podman[73701]: 2024-04-17 06:14:11.413740906 +0000 UTC m=+0.187726871 container create 3e53d104abc5f388117038b1c8903ceb351312b8b4fe6605fab317f37b2bc278 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS) 2024-04-17T06:14:11.673 INFO:journalctl@ceph.osd.2.smithi103.stdout:Apr 17 06:14:11 smithi103 podman[73701]: 2024-04-17 06:14:11.675439369 +0000 UTC m=+0.449425334 container init 3e53d104abc5f388117038b1c8903ceb351312b8b4fe6605fab317f37b2bc278 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-2-deactivate, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0) 2024-04-17T06:14:11.998 INFO:journalctl@ceph.osd.2.smithi103.stdout:Apr 17 06:14:11 smithi103 podman[73701]: 2024-04-17 06:14:11.678645825 +0000 UTC m=+0.452631799 container start 3e53d104abc5f388117038b1c8903ceb351312b8b4fe6605fab317f37b2bc278 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True, io.buildah.version=1.33.5) 2024-04-17T06:14:11.998 INFO:journalctl@ceph.osd.2.smithi103.stdout:Apr 17 06:14:11 smithi103 podman[73701]: 2024-04-17 06:14:11.853624956 +0000 UTC m=+0.627610992 container attach 3e53d104abc5f388117038b1c8903ceb351312b8b4fe6605fab317f37b2bc278 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-04-17T06:14:12.284 INFO:journalctl@ceph.osd.2.smithi103.stdout:Apr 17 06:14:12 smithi103 podman[73701]: 2024-04-17 06:14:12.000743838 +0000 UTC m=+0.774729831 container died 3e53d104abc5f388117038b1c8903ceb351312b8b4fe6605fab317f37b2bc278 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416) 2024-04-17T06:14:12.715 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@osd.2.service' 2024-04-17T06:14:12.759 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-17T06:14:12.759 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-04-17T06:14:12.759 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-04-17T06:14:12.759 DEBUG:teuthology.orchestra.run.smithi047:> sudo systemctl stop ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@osd.3 2024-04-17T06:14:13.087 INFO:journalctl@ceph.osd.3.smithi047.stdout:Apr 17 06:14:12 smithi047 systemd[1]: Stopping Ceph osd.3 for 565db2d8-fc7e-11ee-bc91-c7b262605968... 2024-04-17T06:14:13.389 INFO:journalctl@ceph.osd.3.smithi047.stdout:Apr 17 06:14:13 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-3[53909]: 2024-04-17T06:14:13.089+0000 7f62fc1c7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-17T06:14:13.389 INFO:journalctl@ceph.osd.3.smithi047.stdout:Apr 17 06:14:13 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-3[53909]: 2024-04-17T06:14:13.089+0000 7f62fc1c7700 -1 osd.3 109 *** Got signal Terminated *** 2024-04-17T06:14:13.389 INFO:journalctl@ceph.osd.3.smithi047.stdout:Apr 17 06:14:13 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-3[53909]: 2024-04-17T06:14:13.089+0000 7f62fc1c7700 -1 osd.3 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-17T06:14:15.383 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:15 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:15.132+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:53.138058+0000 front 2024-04-17T06:13:53.138083+0000 (oldest deadline 2024-04-17T06:14:14.238150+0000) 2024-04-17T06:14:16.139 INFO:journalctl@ceph.osd.3.smithi047.stdout:Apr 17 06:14:15 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-3[53909]: 2024-04-17T06:14:15.974+0000 7f62f87bb700 -1 osd.3 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:53.631744+0000 front 2024-04-17T06:13:53.631776+0000 (oldest deadline 2024-04-17T06:14:15.931955+0000) 2024-04-17T06:14:16.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:15 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:15.765+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.354382+0000 front 2024-04-17T06:13:52.354774+0000 (oldest deadline 2024-04-17T06:14:15.254487+0000) 2024-04-17T06:14:16.383 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:16 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:16.108+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:53.138058+0000 front 2024-04-17T06:13:53.138083+0000 (oldest deadline 2024-04-17T06:14:14.238150+0000) 2024-04-17T06:14:17.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:16 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:16.812+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.354382+0000 front 2024-04-17T06:13:52.354774+0000 (oldest deadline 2024-04-17T06:14:15.254487+0000) 2024-04-17T06:14:17.139 INFO:journalctl@ceph.osd.3.smithi047.stdout:Apr 17 06:14:16 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-3[53909]: 2024-04-17T06:14:16.958+0000 7f62f87bb700 -1 osd.3 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:53.631744+0000 front 2024-04-17T06:13:53.631776+0000 (oldest deadline 2024-04-17T06:14:15.931955+0000) 2024-04-17T06:14:17.139 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 17 06:14:16 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-6[60771]: 2024-04-17T06:14:16.823+0000 7f3aa99e3700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.024894+0000 front 2024-04-17T06:13:52.024917+0000 (oldest deadline 2024-04-17T06:14:16.725097+0000) 2024-04-17T06:14:17.284 INFO:journalctl@ceph.osd.5.smithi103.stdout:Apr 17 06:14:16 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-5[48375]: 2024-04-17T06:14:16.922+0000 7f6ecd773700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.033619+0000 front 2024-04-17T06:13:52.033577+0000 (oldest deadline 2024-04-17T06:14:16.733732+0000) 2024-04-17T06:14:17.383 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:17 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:17.094+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:53.138058+0000 front 2024-04-17T06:13:53.138083+0000 (oldest deadline 2024-04-17T06:14:14.238150+0000) 2024-04-17T06:14:18.139 INFO:journalctl@ceph.osd.3.smithi047.stdout:Apr 17 06:14:17 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-3[53909]: 2024-04-17T06:14:17.966+0000 7f62f87bb700 -1 osd.3 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:53.631744+0000 front 2024-04-17T06:13:53.631776+0000 (oldest deadline 2024-04-17T06:14:15.931955+0000) 2024-04-17T06:14:18.139 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 17 06:14:17 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-6[60771]: 2024-04-17T06:14:17.789+0000 7f3aa99e3700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.024894+0000 front 2024-04-17T06:13:52.024917+0000 (oldest deadline 2024-04-17T06:14:16.725097+0000) 2024-04-17T06:14:18.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:17 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:17.843+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.354382+0000 front 2024-04-17T06:13:52.354774+0000 (oldest deadline 2024-04-17T06:14:15.254487+0000) 2024-04-17T06:14:18.284 INFO:journalctl@ceph.osd.5.smithi103.stdout:Apr 17 06:14:17 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-5[48375]: 2024-04-17T06:14:17.913+0000 7f6ecd773700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.033619+0000 front 2024-04-17T06:13:52.033577+0000 (oldest deadline 2024-04-17T06:14:16.733732+0000) 2024-04-17T06:14:18.383 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:18 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:18.064+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:53.138058+0000 front 2024-04-17T06:13:53.138083+0000 (oldest deadline 2024-04-17T06:14:14.238150+0000) 2024-04-17T06:14:18.889 INFO:journalctl@ceph.osd.3.smithi047.stdout:Apr 17 06:14:18 smithi047 podman[93958]: 2024-04-17 06:14:18.615935622 +0000 UTC m=+5.646696359 container died 522e29f01f59e49289aeb910199a219b134347c02a4506d0e028e75287d99326 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-3, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-04-17T06:14:18.889 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 17 06:14:18 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-6[60771]: 2024-04-17T06:14:18.775+0000 7f3aa99e3700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.024894+0000 front 2024-04-17T06:13:52.024917+0000 (oldest deadline 2024-04-17T06:14:16.725097+0000) 2024-04-17T06:14:18.890 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:18 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:18.841+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.354382+0000 front 2024-04-17T06:13:52.354774+0000 (oldest deadline 2024-04-17T06:14:15.254487+0000) 2024-04-17T06:14:19.284 INFO:journalctl@ceph.osd.5.smithi103.stdout:Apr 17 06:14:18 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-5[48375]: 2024-04-17T06:14:18.869+0000 7f6ecd773700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.033619+0000 front 2024-04-17T06:13:52.033577+0000 (oldest deadline 2024-04-17T06:14:16.733732+0000) 2024-04-17T06:14:19.284 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:19 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:19.007+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.989762+0000 front 2024-04-17T06:13:52.989781+0000 (oldest deadline 2024-04-17T06:14:18.289957+0000) 2024-04-17T06:14:19.383 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:19 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:19.102+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:53.138058+0000 front 2024-04-17T06:13:53.138083+0000 (oldest deadline 2024-04-17T06:14:14.238150+0000) 2024-04-17T06:14:19.388 INFO:journalctl@ceph.osd.3.smithi047.stdout:Apr 17 06:14:18 smithi047 podman[93958]: 2024-04-17 06:14:18.930329328 +0000 UTC m=+5.961090064 container cleanup 522e29f01f59e49289aeb910199a219b134347c02a4506d0e028e75287d99326 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-3, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-04-17T06:14:19.389 INFO:journalctl@ceph.osd.3.smithi047.stdout:Apr 17 06:14:18 smithi047 bash[93958]: ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-3 2024-04-17T06:14:19.389 INFO:journalctl@ceph.osd.3.smithi047.stdout:Apr 17 06:14:19 smithi047 podman[93975]: 2024-04-17 06:14:19.148410657 +0000 UTC m=+0.525863180 container remove 522e29f01f59e49289aeb910199a219b134347c02a4506d0e028e75287d99326 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-3, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-04-17T06:14:19.809 INFO:journalctl@ceph.osd.3.smithi047.stdout:Apr 17 06:14:19 smithi047 podman[94065]: 2024-04-17 06:14:19.45849538 +0000 UTC m=+0.018029066 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-17T06:14:19.809 INFO:journalctl@ceph.osd.3.smithi047.stdout:Apr 17 06:14:19 smithi047 podman[94065]: 2024-04-17 06:14:19.57430827 +0000 UTC m=+0.133841998 container create 33529f5a7e1d146ebe1696861e9b9009c340bb48ae4de807d9a8697d1b184d18 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2) 2024-04-17T06:14:20.139 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 17 06:14:19 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-6[60771]: 2024-04-17T06:14:19.810+0000 7f3aa99e3700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.024894+0000 front 2024-04-17T06:13:52.024917+0000 (oldest deadline 2024-04-17T06:14:16.725097+0000) 2024-04-17T06:14:20.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:19 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:19.880+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.354382+0000 front 2024-04-17T06:13:52.354774+0000 (oldest deadline 2024-04-17T06:14:15.254487+0000) 2024-04-17T06:14:20.139 INFO:journalctl@ceph.osd.3.smithi047.stdout:Apr 17 06:14:19 smithi047 podman[94065]: 2024-04-17 06:14:19.852183069 +0000 UTC m=+0.411716752 container init 33529f5a7e1d146ebe1696861e9b9009c340bb48ae4de807d9a8697d1b184d18 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-3-deactivate, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416) 2024-04-17T06:14:20.139 INFO:journalctl@ceph.osd.3.smithi047.stdout:Apr 17 06:14:19 smithi047 podman[94065]: 2024-04-17 06:14:19.855677578 +0000 UTC m=+0.415211261 container start 33529f5a7e1d146ebe1696861e9b9009c340bb48ae4de807d9a8697d1b184d18 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-3-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-17T06:14:20.139 INFO:journalctl@ceph.osd.3.smithi047.stdout:Apr 17 06:14:19 smithi047 podman[94065]: 2024-04-17 06:14:19.952233895 +0000 UTC m=+0.511767586 container attach 33529f5a7e1d146ebe1696861e9b9009c340bb48ae4de807d9a8697d1b184d18 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416) 2024-04-17T06:14:20.284 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:20 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:20.005+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.989762+0000 front 2024-04-17T06:13:52.989781+0000 (oldest deadline 2024-04-17T06:14:18.289957+0000) 2024-04-17T06:14:20.284 INFO:journalctl@ceph.osd.5.smithi103.stdout:Apr 17 06:14:19 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-5[48375]: 2024-04-17T06:14:19.863+0000 7f6ecd773700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.033619+0000 front 2024-04-17T06:13:52.033577+0000 (oldest deadline 2024-04-17T06:14:16.733732+0000) 2024-04-17T06:14:20.383 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:20 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:20.101+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:53.138058+0000 front 2024-04-17T06:13:53.138083+0000 (oldest deadline 2024-04-17T06:14:14.238150+0000) 2024-04-17T06:14:20.452 INFO:journalctl@ceph.osd.3.smithi047.stdout:Apr 17 06:14:20 smithi047 podman[94065]: 2024-04-17 06:14:20.164022833 +0000 UTC m=+0.723556508 container died 33529f5a7e1d146ebe1696861e9b9009c340bb48ae4de807d9a8697d1b184d18 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True) 2024-04-17T06:14:20.772 DEBUG:teuthology.orchestra.run.smithi047:> sudo pkill -f 'journalctl -f -n 0 -u ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@osd.3.service' 2024-04-17T06:14:20.818 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-17T06:14:20.818 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-04-17T06:14:20.818 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-04-17T06:14:20.819 DEBUG:teuthology.orchestra.run.smithi098:> sudo systemctl stop ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@osd.4 2024-04-17T06:14:21.131 INFO:journalctl@ceph.osd.4.smithi098.stdout:Apr 17 06:14:20 smithi098 systemd[1]: Stopping Ceph osd.4 for 565db2d8-fc7e-11ee-bc91-c7b262605968... 2024-04-17T06:14:21.139 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 17 06:14:20 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-6[60771]: 2024-04-17T06:14:20.797+0000 7f3aa99e3700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.024894+0000 front 2024-04-17T06:13:52.024917+0000 (oldest deadline 2024-04-17T06:14:16.725097+0000) 2024-04-17T06:14:21.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:20 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:20.883+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.354382+0000 front 2024-04-17T06:13:52.354774+0000 (oldest deadline 2024-04-17T06:14:15.254487+0000) 2024-04-17T06:14:21.284 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:21 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:21.011+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.989762+0000 front 2024-04-17T06:13:52.989781+0000 (oldest deadline 2024-04-17T06:14:18.289957+0000) 2024-04-17T06:14:21.284 INFO:journalctl@ceph.osd.5.smithi103.stdout:Apr 17 06:14:20 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-5[48375]: 2024-04-17T06:14:20.846+0000 7f6ecd773700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.033619+0000 front 2024-04-17T06:13:52.033577+0000 (oldest deadline 2024-04-17T06:14:16.733732+0000) 2024-04-17T06:14:21.382 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:21 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:21.143+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:53.138058+0000 front 2024-04-17T06:13:53.138083+0000 (oldest deadline 2024-04-17T06:14:14.238150+0000) 2024-04-17T06:14:21.383 INFO:journalctl@ceph.osd.4.smithi098.stdout:Apr 17 06:14:21 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-4[48120]: 2024-04-17T06:14:21.131+0000 7f8cf1476700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-17T06:14:21.383 INFO:journalctl@ceph.osd.4.smithi098.stdout:Apr 17 06:14:21 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-4[48120]: 2024-04-17T06:14:21.131+0000 7f8cf1476700 -1 osd.4 109 *** Got signal Terminated *** 2024-04-17T06:14:21.383 INFO:journalctl@ceph.osd.4.smithi098.stdout:Apr 17 06:14:21 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-4[48120]: 2024-04-17T06:14:21.131+0000 7f8cf1476700 -1 osd.4 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-17T06:14:22.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:21 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:21.847+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.354382+0000 front 2024-04-17T06:13:52.354774+0000 (oldest deadline 2024-04-17T06:14:15.254487+0000) 2024-04-17T06:14:22.139 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 17 06:14:21 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-6[60771]: 2024-04-17T06:14:21.764+0000 7f3aa99e3700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.024894+0000 front 2024-04-17T06:13:52.024917+0000 (oldest deadline 2024-04-17T06:14:16.725097+0000) 2024-04-17T06:14:22.284 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:22 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:22.012+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.989762+0000 front 2024-04-17T06:13:52.989781+0000 (oldest deadline 2024-04-17T06:14:18.289957+0000) 2024-04-17T06:14:22.284 INFO:journalctl@ceph.osd.5.smithi103.stdout:Apr 17 06:14:21 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-5[48375]: 2024-04-17T06:14:21.859+0000 7f6ecd773700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.033619+0000 front 2024-04-17T06:13:52.033577+0000 (oldest deadline 2024-04-17T06:14:16.733732+0000) 2024-04-17T06:14:22.633 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:22 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:22.190+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:53.138058+0000 front 2024-04-17T06:13:53.138083+0000 (oldest deadline 2024-04-17T06:14:14.238150+0000) 2024-04-17T06:14:23.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:22 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:22.810+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.354382+0000 front 2024-04-17T06:13:52.354774+0000 (oldest deadline 2024-04-17T06:14:15.254487+0000) 2024-04-17T06:14:23.139 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 17 06:14:22 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-6[60771]: 2024-04-17T06:14:22.729+0000 7f3aa99e3700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.024894+0000 front 2024-04-17T06:13:52.024917+0000 (oldest deadline 2024-04-17T06:14:16.725097+0000) 2024-04-17T06:14:23.139 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 17 06:14:22 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-6[60771]: 2024-04-17T06:14:22.729+0000 7f3aa99e3700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:13:56.725456+0000 front 2024-04-17T06:13:56.725477+0000 (oldest deadline 2024-04-17T06:14:22.625606+0000) 2024-04-17T06:14:23.283 INFO:journalctl@ceph.osd.5.smithi103.stdout:Apr 17 06:14:22 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-5[48375]: 2024-04-17T06:14:22.889+0000 7f6ecd773700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.033619+0000 front 2024-04-17T06:13:52.033577+0000 (oldest deadline 2024-04-17T06:14:16.733732+0000) 2024-04-17T06:14:23.284 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:23 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:23.025+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.989762+0000 front 2024-04-17T06:13:52.989781+0000 (oldest deadline 2024-04-17T06:14:18.289957+0000) 2024-04-17T06:14:23.633 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:23 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:23.224+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:53.138058+0000 front 2024-04-17T06:13:53.138083+0000 (oldest deadline 2024-04-17T06:14:14.238150+0000) 2024-04-17T06:14:24.040 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:23 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:23.848+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.354382+0000 front 2024-04-17T06:13:52.354774+0000 (oldest deadline 2024-04-17T06:14:15.254487+0000) 2024-04-17T06:14:24.041 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 17 06:14:23 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-6[60771]: 2024-04-17T06:14:23.777+0000 7f3aa99e3700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.024894+0000 front 2024-04-17T06:13:52.024917+0000 (oldest deadline 2024-04-17T06:14:16.725097+0000) 2024-04-17T06:14:24.041 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 17 06:14:23 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-6[60771]: 2024-04-17T06:14:23.777+0000 7f3aa99e3700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:13:56.725456+0000 front 2024-04-17T06:13:56.725477+0000 (oldest deadline 2024-04-17T06:14:22.625606+0000) 2024-04-17T06:14:24.284 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:23 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:23.980+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.989762+0000 front 2024-04-17T06:13:52.989781+0000 (oldest deadline 2024-04-17T06:14:18.289957+0000) 2024-04-17T06:14:24.284 INFO:journalctl@ceph.osd.5.smithi103.stdout:Apr 17 06:14:23 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-5[48375]: 2024-04-17T06:14:23.862+0000 7f6ecd773700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.033619+0000 front 2024-04-17T06:13:52.033577+0000 (oldest deadline 2024-04-17T06:14:16.733732+0000) 2024-04-17T06:14:24.284 INFO:journalctl@ceph.osd.5.smithi103.stdout:Apr 17 06:14:23 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-5[48375]: 2024-04-17T06:14:23.862+0000 7f6ecd773700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:13:56.734215+0000 front 2024-04-17T06:13:56.734133+0000 (oldest deadline 2024-04-17T06:14:23.134815+0000) 2024-04-17T06:14:24.633 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:24 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:24.245+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:53.138058+0000 front 2024-04-17T06:13:53.138083+0000 (oldest deadline 2024-04-17T06:14:14.238150+0000) 2024-04-17T06:14:25.051 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:24 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:24.808+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.354382+0000 front 2024-04-17T06:13:52.354774+0000 (oldest deadline 2024-04-17T06:14:15.254487+0000) 2024-04-17T06:14:25.051 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 17 06:14:24 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-6[60771]: 2024-04-17T06:14:24.779+0000 7f3aa99e3700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.024894+0000 front 2024-04-17T06:13:52.024917+0000 (oldest deadline 2024-04-17T06:14:16.725097+0000) 2024-04-17T06:14:25.051 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 17 06:14:24 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-6[60771]: 2024-04-17T06:14:24.779+0000 7f3aa99e3700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:13:56.725456+0000 front 2024-04-17T06:13:56.725477+0000 (oldest deadline 2024-04-17T06:14:22.625606+0000) 2024-04-17T06:14:25.133 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 06:14:24 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-7[54731]: 2024-04-17T06:14:24.792+0000 7f1a08efe700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:13:57.505129+0000 front 2024-04-17T06:13:57.505157+0000 (oldest deadline 2024-04-17T06:14:23.905794+0000) 2024-04-17T06:14:25.284 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 06:14:25 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-8[54941]: 2024-04-17T06:14:25.081+0000 7f9c2d8b8700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:00.597947+0000 front 2024-04-17T06:14:00.598005+0000 (oldest deadline 2024-04-17T06:14:24.698026+0000) 2024-04-17T06:14:25.284 INFO:journalctl@ceph.osd.5.smithi103.stdout:Apr 17 06:14:24 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-5[48375]: 2024-04-17T06:14:24.852+0000 7f6ecd773700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.033619+0000 front 2024-04-17T06:13:52.033577+0000 (oldest deadline 2024-04-17T06:14:16.733732+0000) 2024-04-17T06:14:25.284 INFO:journalctl@ceph.osd.5.smithi103.stdout:Apr 17 06:14:24 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-5[48375]: 2024-04-17T06:14:24.852+0000 7f6ecd773700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:13:56.734215+0000 front 2024-04-17T06:13:56.734133+0000 (oldest deadline 2024-04-17T06:14:23.134815+0000) 2024-04-17T06:14:25.284 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:24 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:24.979+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.989762+0000 front 2024-04-17T06:13:52.989781+0000 (oldest deadline 2024-04-17T06:14:18.289957+0000) 2024-04-17T06:14:25.285 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:24 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:24.979+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:01.191270+0000 front 2024-04-17T06:14:01.191086+0000 (oldest deadline 2024-04-17T06:14:24.690981+0000) 2024-04-17T06:14:25.633 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:25 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:25.234+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:53.138058+0000 front 2024-04-17T06:13:53.138083+0000 (oldest deadline 2024-04-17T06:14:14.238150+0000) 2024-04-17T06:14:25.633 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:25 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:25.234+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:13:58.939213+0000 front 2024-04-17T06:13:58.939286+0000 (oldest deadline 2024-04-17T06:14:24.839434+0000) 2024-04-17T06:14:26.116 INFO:journalctl@ceph.osd.5.smithi103.stdout:Apr 17 06:14:25 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-5[48375]: 2024-04-17T06:14:25.808+0000 7f6ecd773700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.033619+0000 front 2024-04-17T06:13:52.033577+0000 (oldest deadline 2024-04-17T06:14:16.733732+0000) 2024-04-17T06:14:26.117 INFO:journalctl@ceph.osd.5.smithi103.stdout:Apr 17 06:14:25 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-5[48375]: 2024-04-17T06:14:25.808+0000 7f6ecd773700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:13:56.734215+0000 front 2024-04-17T06:13:56.734133+0000 (oldest deadline 2024-04-17T06:14:23.134815+0000) 2024-04-17T06:14:26.117 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:25 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:25.992+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.989762+0000 front 2024-04-17T06:13:52.989781+0000 (oldest deadline 2024-04-17T06:14:18.289957+0000) 2024-04-17T06:14:26.117 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:25 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:25.992+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:01.191270+0000 front 2024-04-17T06:14:01.191086+0000 (oldest deadline 2024-04-17T06:14:24.690981+0000) 2024-04-17T06:14:26.133 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 06:14:25 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-7[54731]: 2024-04-17T06:14:25.774+0000 7f1a08efe700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:13:57.505129+0000 front 2024-04-17T06:13:57.505157+0000 (oldest deadline 2024-04-17T06:14:23.905794+0000) 2024-04-17T06:14:26.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:25 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:25.783+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.354382+0000 front 2024-04-17T06:13:52.354774+0000 (oldest deadline 2024-04-17T06:14:15.254487+0000) 2024-04-17T06:14:26.139 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 17 06:14:25 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-6[60771]: 2024-04-17T06:14:25.769+0000 7f3aa99e3700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.024894+0000 front 2024-04-17T06:13:52.024917+0000 (oldest deadline 2024-04-17T06:14:16.725097+0000) 2024-04-17T06:14:26.139 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 17 06:14:25 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-6[60771]: 2024-04-17T06:14:25.769+0000 7f3aa99e3700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:13:56.725456+0000 front 2024-04-17T06:13:56.725477+0000 (oldest deadline 2024-04-17T06:14:22.625606+0000) 2024-04-17T06:14:26.534 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 06:14:26 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-8[54941]: 2024-04-17T06:14:26.117+0000 7f9c2d8b8700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:00.597947+0000 front 2024-04-17T06:14:00.598005+0000 (oldest deadline 2024-04-17T06:14:24.698026+0000) 2024-04-17T06:14:26.622 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:26 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:26.216+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:53.138058+0000 front 2024-04-17T06:13:53.138083+0000 (oldest deadline 2024-04-17T06:14:14.238150+0000) 2024-04-17T06:14:26.622 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:26 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:26.216+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:13:58.939213+0000 front 2024-04-17T06:13:58.939286+0000 (oldest deadline 2024-04-17T06:14:24.839434+0000) 2024-04-17T06:14:26.883 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 06:14:26 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-7[54731]: 2024-04-17T06:14:26.786+0000 7f1a08efe700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:13:57.505129+0000 front 2024-04-17T06:13:57.505157+0000 (oldest deadline 2024-04-17T06:14:23.905794+0000) 2024-04-17T06:14:26.883 INFO:journalctl@ceph.osd.4.smithi098.stdout:Apr 17 06:14:26 smithi098 podman[74619]: 2024-04-17 06:14:26.62470053 +0000 UTC m=+5.610998966 container died 523761e89cab3a378180f5fd403e783cc74dba834ecadf2178489ac681f9fc55 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-4, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0) 2024-04-17T06:14:27.110 INFO:journalctl@ceph.osd.5.smithi103.stdout:Apr 17 06:14:26 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-5[48375]: 2024-04-17T06:14:26.846+0000 7f6ecd773700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.033619+0000 front 2024-04-17T06:13:52.033577+0000 (oldest deadline 2024-04-17T06:14:16.733732+0000) 2024-04-17T06:14:27.110 INFO:journalctl@ceph.osd.5.smithi103.stdout:Apr 17 06:14:26 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-5[48375]: 2024-04-17T06:14:26.846+0000 7f6ecd773700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:13:56.734215+0000 front 2024-04-17T06:13:56.734133+0000 (oldest deadline 2024-04-17T06:14:23.134815+0000) 2024-04-17T06:14:27.110 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:26 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:26.981+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.989762+0000 front 2024-04-17T06:13:52.989781+0000 (oldest deadline 2024-04-17T06:14:18.289957+0000) 2024-04-17T06:14:27.110 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:26 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:26.981+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:01.191270+0000 front 2024-04-17T06:14:01.191086+0000 (oldest deadline 2024-04-17T06:14:24.690981+0000) 2024-04-17T06:14:27.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:26 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:26.766+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.354382+0000 front 2024-04-17T06:13:52.354774+0000 (oldest deadline 2024-04-17T06:14:15.254487+0000) 2024-04-17T06:14:27.139 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 17 06:14:26 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-6[60771]: 2024-04-17T06:14:26.732+0000 7f3aa99e3700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.024894+0000 front 2024-04-17T06:13:52.024917+0000 (oldest deadline 2024-04-17T06:14:16.725097+0000) 2024-04-17T06:14:27.139 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 17 06:14:26 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-6[60771]: 2024-04-17T06:14:26.732+0000 7f3aa99e3700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:13:56.725456+0000 front 2024-04-17T06:13:56.725477+0000 (oldest deadline 2024-04-17T06:14:22.625606+0000) 2024-04-17T06:14:27.226 INFO:journalctl@ceph.osd.4.smithi098.stdout:Apr 17 06:14:27 smithi098 podman[74619]: 2024-04-17 06:14:27.026483875 +0000 UTC m=+6.012782310 container cleanup 523761e89cab3a378180f5fd403e783cc74dba834ecadf2178489ac681f9fc55 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-4, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416) 2024-04-17T06:14:27.226 INFO:journalctl@ceph.osd.4.smithi098.stdout:Apr 17 06:14:27 smithi098 bash[74619]: ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-4 2024-04-17T06:14:27.534 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 06:14:27 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-8[54941]: 2024-04-17T06:14:27.110+0000 7f9c2d8b8700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:00.597947+0000 front 2024-04-17T06:14:00.598005+0000 (oldest deadline 2024-04-17T06:14:24.698026+0000) 2024-04-17T06:14:27.554 INFO:journalctl@ceph.osd.4.smithi098.stdout:Apr 17 06:14:27 smithi098 podman[74638]: 2024-04-17 06:14:27.261165271 +0000 UTC m=+0.636717551 container remove 523761e89cab3a378180f5fd403e783cc74dba834ecadf2178489ac681f9fc55 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-4, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-17T06:14:27.554 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:27 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:27.227+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:53.138058+0000 front 2024-04-17T06:13:53.138083+0000 (oldest deadline 2024-04-17T06:14:14.238150+0000) 2024-04-17T06:14:27.554 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:27 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:27.227+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:13:58.939213+0000 front 2024-04-17T06:13:58.939286+0000 (oldest deadline 2024-04-17T06:14:24.839434+0000) 2024-04-17T06:14:27.830 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 06:14:27 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-7[54731]: 2024-04-17T06:14:27.795+0000 7f1a08efe700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:13:57.505129+0000 front 2024-04-17T06:13:57.505157+0000 (oldest deadline 2024-04-17T06:14:23.905794+0000) 2024-04-17T06:14:27.830 INFO:journalctl@ceph.osd.4.smithi098.stdout:Apr 17 06:14:27 smithi098 podman[74727]: 2024-04-17 06:14:27.460040754 +0000 UTC m=+0.017781318 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-17T06:14:27.830 INFO:journalctl@ceph.osd.4.smithi098.stdout:Apr 17 06:14:27 smithi098 podman[74727]: 2024-04-17 06:14:27.579087074 +0000 UTC m=+0.136827626 container create 68474eafa7129443d1f920edd43f7b7295293510531bca9cfb420eec245d4e0c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-4-deactivate, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240416) 2024-04-17T06:14:27.830 INFO:journalctl@ceph.osd.4.smithi098.stdout:Apr 17 06:14:27 smithi098 podman[74727]: 2024-04-17 06:14:27.832080384 +0000 UTC m=+0.389820940 container init 68474eafa7129443d1f920edd43f7b7295293510531bca9cfb420eec245d4e0c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-04-17T06:14:28.089 INFO:journalctl@ceph.osd.5.smithi103.stdout:Apr 17 06:14:27 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-5[48375]: 2024-04-17T06:14:27.798+0000 7f6ecd773700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.033619+0000 front 2024-04-17T06:13:52.033577+0000 (oldest deadline 2024-04-17T06:14:16.733732+0000) 2024-04-17T06:14:28.089 INFO:journalctl@ceph.osd.5.smithi103.stdout:Apr 17 06:14:27 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-5[48375]: 2024-04-17T06:14:27.798+0000 7f6ecd773700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:13:56.734215+0000 front 2024-04-17T06:13:56.734133+0000 (oldest deadline 2024-04-17T06:14:23.134815+0000) 2024-04-17T06:14:28.090 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:27 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:27.991+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.989762+0000 front 2024-04-17T06:13:52.989781+0000 (oldest deadline 2024-04-17T06:14:18.289957+0000) 2024-04-17T06:14:28.090 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:27 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:27.991+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:01.191270+0000 front 2024-04-17T06:14:01.191086+0000 (oldest deadline 2024-04-17T06:14:24.690981+0000) 2024-04-17T06:14:28.127 INFO:journalctl@ceph.osd.4.smithi098.stdout:Apr 17 06:14:27 smithi098 podman[74727]: 2024-04-17 06:14:27.835170591 +0000 UTC m=+0.392911136 container start 68474eafa7129443d1f920edd43f7b7295293510531bca9cfb420eec245d4e0c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-17T06:14:28.127 INFO:journalctl@ceph.osd.4.smithi098.stdout:Apr 17 06:14:27 smithi098 podman[74727]: 2024-04-17 06:14:27.932214632 +0000 UTC m=+0.489955194 container attach 68474eafa7129443d1f920edd43f7b7295293510531bca9cfb420eec245d4e0c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-17T06:14:28.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:27 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:27.749+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.354382+0000 front 2024-04-17T06:13:52.354774+0000 (oldest deadline 2024-04-17T06:14:15.254487+0000) 2024-04-17T06:14:28.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:27 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:27.749+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:01.655772+0000 front 2024-04-17T06:14:01.655755+0000 (oldest deadline 2024-04-17T06:14:27.555801+0000) 2024-04-17T06:14:28.139 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 17 06:14:27 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-6[60771]: 2024-04-17T06:14:27.717+0000 7f3aa99e3700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.024894+0000 front 2024-04-17T06:13:52.024917+0000 (oldest deadline 2024-04-17T06:14:16.725097+0000) 2024-04-17T06:14:28.139 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 17 06:14:27 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-6[60771]: 2024-04-17T06:14:27.717+0000 7f3aa99e3700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:13:56.725456+0000 front 2024-04-17T06:13:56.725477+0000 (oldest deadline 2024-04-17T06:14:22.625606+0000) 2024-04-17T06:14:28.383 INFO:journalctl@ceph.osd.4.smithi098.stdout:Apr 17 06:14:28 smithi098 podman[74727]: 2024-04-17 06:14:28.130104842 +0000 UTC m=+0.687845408 container died 68474eafa7129443d1f920edd43f7b7295293510531bca9cfb420eec245d4e0c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-04-17T06:14:28.383 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:28 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:28.239+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:53.138058+0000 front 2024-04-17T06:13:53.138083+0000 (oldest deadline 2024-04-17T06:14:14.238150+0000) 2024-04-17T06:14:28.383 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:28 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:28.239+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:13:58.939213+0000 front 2024-04-17T06:13:58.939286+0000 (oldest deadline 2024-04-17T06:14:24.839434+0000) 2024-04-17T06:14:28.534 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 06:14:28 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-8[54941]: 2024-04-17T06:14:28.090+0000 7f9c2d8b8700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:00.597947+0000 front 2024-04-17T06:14:00.598005+0000 (oldest deadline 2024-04-17T06:14:24.698026+0000) 2024-04-17T06:14:28.807 DEBUG:teuthology.orchestra.run.smithi098:> sudo pkill -f 'journalctl -f -n 0 -u ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@osd.4.service' 2024-04-17T06:14:28.848 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-17T06:14:28.848 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-04-17T06:14:28.848 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-04-17T06:14:28.848 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@osd.5 2024-04-17T06:14:29.034 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:28 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:28.961+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.989762+0000 front 2024-04-17T06:13:52.989781+0000 (oldest deadline 2024-04-17T06:14:18.289957+0000) 2024-04-17T06:14:29.034 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:28 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:28.961+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:01.191270+0000 front 2024-04-17T06:14:01.191086+0000 (oldest deadline 2024-04-17T06:14:24.690981+0000) 2024-04-17T06:14:29.034 INFO:journalctl@ceph.osd.5.smithi103.stdout:Apr 17 06:14:28 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-5[48375]: 2024-04-17T06:14:28.767+0000 7f6ecd773700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.033619+0000 front 2024-04-17T06:13:52.033577+0000 (oldest deadline 2024-04-17T06:14:16.733732+0000) 2024-04-17T06:14:29.034 INFO:journalctl@ceph.osd.5.smithi103.stdout:Apr 17 06:14:28 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-5[48375]: 2024-04-17T06:14:28.767+0000 7f6ecd773700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:13:56.734215+0000 front 2024-04-17T06:13:56.734133+0000 (oldest deadline 2024-04-17T06:14:23.134815+0000) 2024-04-17T06:14:29.034 INFO:journalctl@ceph.osd.5.smithi103.stdout:Apr 17 06:14:28 smithi103 systemd[1]: Stopping Ceph osd.5 for 565db2d8-fc7e-11ee-bc91-c7b262605968... 2024-04-17T06:14:29.133 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 06:14:28 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-7[54731]: 2024-04-17T06:14:28.766+0000 7f1a08efe700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:13:57.505129+0000 front 2024-04-17T06:13:57.505157+0000 (oldest deadline 2024-04-17T06:14:23.905794+0000) 2024-04-17T06:14:29.138 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:28 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:28.710+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.354382+0000 front 2024-04-17T06:13:52.354774+0000 (oldest deadline 2024-04-17T06:14:15.254487+0000) 2024-04-17T06:14:29.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:28 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:28.710+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:01.655772+0000 front 2024-04-17T06:14:01.655755+0000 (oldest deadline 2024-04-17T06:14:27.555801+0000) 2024-04-17T06:14:29.139 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 17 06:14:28 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-6[60771]: 2024-04-17T06:14:28.675+0000 7f3aa99e3700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.024894+0000 front 2024-04-17T06:13:52.024917+0000 (oldest deadline 2024-04-17T06:14:16.725097+0000) 2024-04-17T06:14:29.139 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 17 06:14:28 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-6[60771]: 2024-04-17T06:14:28.675+0000 7f3aa99e3700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:13:56.725456+0000 front 2024-04-17T06:13:56.725477+0000 (oldest deadline 2024-04-17T06:14:22.625606+0000) 2024-04-17T06:14:29.534 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 06:14:29 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-8[54941]: 2024-04-17T06:14:29.076+0000 7f9c2d8b8700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:00.597947+0000 front 2024-04-17T06:14:00.598005+0000 (oldest deadline 2024-04-17T06:14:24.698026+0000) 2024-04-17T06:14:29.534 INFO:journalctl@ceph.osd.5.smithi103.stdout:Apr 17 06:14:29 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-5[48375]: 2024-04-17T06:14:29.175+0000 7f6ed117f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-17T06:14:29.534 INFO:journalctl@ceph.osd.5.smithi103.stdout:Apr 17 06:14:29 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-5[48375]: 2024-04-17T06:14:29.175+0000 7f6ed117f700 -1 osd.5 109 *** Got signal Terminated *** 2024-04-17T06:14:29.534 INFO:journalctl@ceph.osd.5.smithi103.stdout:Apr 17 06:14:29 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-5[48375]: 2024-04-17T06:14:29.175+0000 7f6ed117f700 -1 osd.5 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-17T06:14:29.633 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:29 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:29.208+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:53.138058+0000 front 2024-04-17T06:13:53.138083+0000 (oldest deadline 2024-04-17T06:14:14.238150+0000) 2024-04-17T06:14:29.633 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:29 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:29.208+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:13:58.939213+0000 front 2024-04-17T06:13:58.939286+0000 (oldest deadline 2024-04-17T06:14:24.839434+0000) 2024-04-17T06:14:30.034 INFO:journalctl@ceph.osd.5.smithi103.stdout:Apr 17 06:14:29 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-5[48375]: 2024-04-17T06:14:29.737+0000 7f6ecd773700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.033619+0000 front 2024-04-17T06:13:52.033577+0000 (oldest deadline 2024-04-17T06:14:16.733732+0000) 2024-04-17T06:14:30.034 INFO:journalctl@ceph.osd.5.smithi103.stdout:Apr 17 06:14:29 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-5[48375]: 2024-04-17T06:14:29.737+0000 7f6ecd773700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:13:56.734215+0000 front 2024-04-17T06:13:56.734133+0000 (oldest deadline 2024-04-17T06:14:23.134815+0000) 2024-04-17T06:14:30.034 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:29 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:29.961+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.989762+0000 front 2024-04-17T06:13:52.989781+0000 (oldest deadline 2024-04-17T06:14:18.289957+0000) 2024-04-17T06:14:30.034 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:29 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:29.961+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:01.191270+0000 front 2024-04-17T06:14:01.191086+0000 (oldest deadline 2024-04-17T06:14:24.690981+0000) 2024-04-17T06:14:30.132 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 06:14:29 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-7[54731]: 2024-04-17T06:14:29.794+0000 7f1a08efe700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:13:57.505129+0000 front 2024-04-17T06:13:57.505157+0000 (oldest deadline 2024-04-17T06:14:23.905794+0000) 2024-04-17T06:14:30.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:29 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:29.671+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.354382+0000 front 2024-04-17T06:13:52.354774+0000 (oldest deadline 2024-04-17T06:14:15.254487+0000) 2024-04-17T06:14:30.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:29 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:29.671+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:01.655772+0000 front 2024-04-17T06:14:01.655755+0000 (oldest deadline 2024-04-17T06:14:27.555801+0000) 2024-04-17T06:14:30.139 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 17 06:14:29 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-6[60771]: 2024-04-17T06:14:29.707+0000 7f3aa99e3700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.024894+0000 front 2024-04-17T06:13:52.024917+0000 (oldest deadline 2024-04-17T06:14:16.725097+0000) 2024-04-17T06:14:30.139 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 17 06:14:29 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-6[60771]: 2024-04-17T06:14:29.707+0000 7f3aa99e3700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:13:56.725456+0000 front 2024-04-17T06:13:56.725477+0000 (oldest deadline 2024-04-17T06:14:22.625606+0000) 2024-04-17T06:14:30.534 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 06:14:30 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-8[54941]: 2024-04-17T06:14:30.091+0000 7f9c2d8b8700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:00.597947+0000 front 2024-04-17T06:14:00.598005+0000 (oldest deadline 2024-04-17T06:14:24.698026+0000) 2024-04-17T06:14:30.633 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:30 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:30.253+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:53.138058+0000 front 2024-04-17T06:13:53.138083+0000 (oldest deadline 2024-04-17T06:14:14.238150+0000) 2024-04-17T06:14:30.633 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:30 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:30.253+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:13:58.939213+0000 front 2024-04-17T06:13:58.939286+0000 (oldest deadline 2024-04-17T06:14:24.839434+0000) 2024-04-17T06:14:30.889 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:30 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:30.629+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.354382+0000 front 2024-04-17T06:13:52.354774+0000 (oldest deadline 2024-04-17T06:14:15.254487+0000) 2024-04-17T06:14:30.889 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:30 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:30.629+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:01.655772+0000 front 2024-04-17T06:14:01.655755+0000 (oldest deadline 2024-04-17T06:14:27.555801+0000) 2024-04-17T06:14:30.889 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 17 06:14:30 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-6[60771]: 2024-04-17T06:14:30.728+0000 7f3aa99e3700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.024894+0000 front 2024-04-17T06:13:52.024917+0000 (oldest deadline 2024-04-17T06:14:16.725097+0000) 2024-04-17T06:14:30.889 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 17 06:14:30 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-6[60771]: 2024-04-17T06:14:30.728+0000 7f3aa99e3700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:13:56.725456+0000 front 2024-04-17T06:13:56.725477+0000 (oldest deadline 2024-04-17T06:14:22.625606+0000) 2024-04-17T06:14:31.034 INFO:journalctl@ceph.osd.5.smithi103.stdout:Apr 17 06:14:30 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-5[48375]: 2024-04-17T06:14:30.771+0000 7f6ecd773700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.033619+0000 front 2024-04-17T06:13:52.033577+0000 (oldest deadline 2024-04-17T06:14:16.733732+0000) 2024-04-17T06:14:31.034 INFO:journalctl@ceph.osd.5.smithi103.stdout:Apr 17 06:14:30 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-5[48375]: 2024-04-17T06:14:30.771+0000 7f6ecd773700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:13:56.734215+0000 front 2024-04-17T06:13:56.734133+0000 (oldest deadline 2024-04-17T06:14:23.134815+0000) 2024-04-17T06:14:31.034 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:30 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:30.994+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.989762+0000 front 2024-04-17T06:13:52.989781+0000 (oldest deadline 2024-04-17T06:14:18.289957+0000) 2024-04-17T06:14:31.034 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:30 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:30.994+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:01.191270+0000 front 2024-04-17T06:14:01.191086+0000 (oldest deadline 2024-04-17T06:14:24.690981+0000) 2024-04-17T06:14:31.133 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 06:14:30 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-7[54731]: 2024-04-17T06:14:30.838+0000 7f1a08efe700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:13:57.505129+0000 front 2024-04-17T06:13:57.505157+0000 (oldest deadline 2024-04-17T06:14:23.905794+0000) 2024-04-17T06:14:31.534 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 06:14:31 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-8[54941]: 2024-04-17T06:14:31.047+0000 7f9c2d8b8700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:00.597947+0000 front 2024-04-17T06:14:00.598005+0000 (oldest deadline 2024-04-17T06:14:24.698026+0000) 2024-04-17T06:14:31.633 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:31 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:31.294+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:53.138058+0000 front 2024-04-17T06:13:53.138083+0000 (oldest deadline 2024-04-17T06:14:14.238150+0000) 2024-04-17T06:14:31.633 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:31 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:31.294+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:13:58.939213+0000 front 2024-04-17T06:13:58.939286+0000 (oldest deadline 2024-04-17T06:14:24.839434+0000) 2024-04-17T06:14:31.633 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:31 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:31.294+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:07.040744+0000 front 2024-04-17T06:14:07.040770+0000 (oldest deadline 2024-04-17T06:14:30.441549+0000) 2024-04-17T06:14:31.889 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 17 06:14:31 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-6[60771]: 2024-04-17T06:14:31.682+0000 7f3aa99e3700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.024894+0000 front 2024-04-17T06:13:52.024917+0000 (oldest deadline 2024-04-17T06:14:16.725097+0000) 2024-04-17T06:14:31.889 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 17 06:14:31 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-6[60771]: 2024-04-17T06:14:31.682+0000 7f3aa99e3700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:13:56.725456+0000 front 2024-04-17T06:13:56.725477+0000 (oldest deadline 2024-04-17T06:14:22.625606+0000) 2024-04-17T06:14:31.889 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:31 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:31.638+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.354382+0000 front 2024-04-17T06:13:52.354774+0000 (oldest deadline 2024-04-17T06:14:15.254487+0000) 2024-04-17T06:14:31.889 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:31 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:31.638+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:01.655772+0000 front 2024-04-17T06:14:01.655755+0000 (oldest deadline 2024-04-17T06:14:27.555801+0000) 2024-04-17T06:14:31.890 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:31 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:31.638+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:09.257128+0000 front 2024-04-17T06:14:09.257345+0000 (oldest deadline 2024-04-17T06:14:31.557233+0000) 2024-04-17T06:14:32.034 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:31 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:31.967+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.989762+0000 front 2024-04-17T06:13:52.989781+0000 (oldest deadline 2024-04-17T06:14:18.289957+0000) 2024-04-17T06:14:32.034 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:31 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:31.967+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:01.191270+0000 front 2024-04-17T06:14:01.191086+0000 (oldest deadline 2024-04-17T06:14:24.690981+0000) 2024-04-17T06:14:32.034 INFO:journalctl@ceph.osd.5.smithi103.stdout:Apr 17 06:14:31 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-5[48375]: 2024-04-17T06:14:31.751+0000 7f6ecd773700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.033619+0000 front 2024-04-17T06:13:52.033577+0000 (oldest deadline 2024-04-17T06:14:16.733732+0000) 2024-04-17T06:14:32.034 INFO:journalctl@ceph.osd.5.smithi103.stdout:Apr 17 06:14:31 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-5[48375]: 2024-04-17T06:14:31.751+0000 7f6ecd773700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:13:56.734215+0000 front 2024-04-17T06:13:56.734133+0000 (oldest deadline 2024-04-17T06:14:23.134815+0000) 2024-04-17T06:14:32.133 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 06:14:31 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-7[54731]: 2024-04-17T06:14:31.841+0000 7f1a08efe700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:13:57.505129+0000 front 2024-04-17T06:13:57.505157+0000 (oldest deadline 2024-04-17T06:14:23.905794+0000) 2024-04-17T06:14:32.534 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 06:14:32 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-8[54941]: 2024-04-17T06:14:32.045+0000 7f9c2d8b8700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:00.597947+0000 front 2024-04-17T06:14:00.598005+0000 (oldest deadline 2024-04-17T06:14:24.698026+0000) 2024-04-17T06:14:32.633 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:32 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:32.254+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:53.138058+0000 front 2024-04-17T06:13:53.138083+0000 (oldest deadline 2024-04-17T06:14:14.238150+0000) 2024-04-17T06:14:32.633 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:32 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:32.254+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:13:58.939213+0000 front 2024-04-17T06:13:58.939286+0000 (oldest deadline 2024-04-17T06:14:24.839434+0000) 2024-04-17T06:14:32.633 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:32 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:32.254+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:07.040744+0000 front 2024-04-17T06:14:07.040770+0000 (oldest deadline 2024-04-17T06:14:30.441549+0000) 2024-04-17T06:14:33.031 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:32 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:32.921+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.989762+0000 front 2024-04-17T06:13:52.989781+0000 (oldest deadline 2024-04-17T06:14:18.289957+0000) 2024-04-17T06:14:33.031 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:32 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:32.921+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:01.191270+0000 front 2024-04-17T06:14:01.191086+0000 (oldest deadline 2024-04-17T06:14:24.690981+0000) 2024-04-17T06:14:33.031 INFO:journalctl@ceph.osd.5.smithi103.stdout:Apr 17 06:14:32 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-5[48375]: 2024-04-17T06:14:32.713+0000 7f6ecd773700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.033619+0000 front 2024-04-17T06:13:52.033577+0000 (oldest deadline 2024-04-17T06:14:16.733732+0000) 2024-04-17T06:14:33.032 INFO:journalctl@ceph.osd.5.smithi103.stdout:Apr 17 06:14:32 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-5[48375]: 2024-04-17T06:14:32.713+0000 7f6ecd773700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:13:56.734215+0000 front 2024-04-17T06:13:56.734133+0000 (oldest deadline 2024-04-17T06:14:23.134815+0000) 2024-04-17T06:14:33.133 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 06:14:32 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-7[54731]: 2024-04-17T06:14:32.846+0000 7f1a08efe700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:13:57.505129+0000 front 2024-04-17T06:13:57.505157+0000 (oldest deadline 2024-04-17T06:14:23.905794+0000) 2024-04-17T06:14:33.139 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 17 06:14:32 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-6[60771]: 2024-04-17T06:14:32.687+0000 7f3aa99e3700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.024894+0000 front 2024-04-17T06:13:52.024917+0000 (oldest deadline 2024-04-17T06:14:16.725097+0000) 2024-04-17T06:14:33.139 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 17 06:14:32 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-6[60771]: 2024-04-17T06:14:32.687+0000 7f3aa99e3700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:13:56.725456+0000 front 2024-04-17T06:13:56.725477+0000 (oldest deadline 2024-04-17T06:14:22.625606+0000) 2024-04-17T06:14:33.139 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 17 06:14:32 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-6[60771]: 2024-04-17T06:14:32.687+0000 7f3aa99e3700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:06.726829+0000 front 2024-04-17T06:14:06.726785+0000 (oldest deadline 2024-04-17T06:14:32.026942+0000) 2024-04-17T06:14:33.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:32 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:32.680+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.354382+0000 front 2024-04-17T06:13:52.354774+0000 (oldest deadline 2024-04-17T06:14:15.254487+0000) 2024-04-17T06:14:33.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:32 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:32.680+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:01.655772+0000 front 2024-04-17T06:14:01.655755+0000 (oldest deadline 2024-04-17T06:14:27.555801+0000) 2024-04-17T06:14:33.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:32 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:32.680+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:09.257128+0000 front 2024-04-17T06:14:09.257345+0000 (oldest deadline 2024-04-17T06:14:31.557233+0000) 2024-04-17T06:14:33.284 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 06:14:33 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-8[54941]: 2024-04-17T06:14:33.032+0000 7f9c2d8b8700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:00.597947+0000 front 2024-04-17T06:14:00.598005+0000 (oldest deadline 2024-04-17T06:14:24.698026+0000) 2024-04-17T06:14:33.284 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 06:14:33 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-8[54941]: 2024-04-17T06:14:33.032+0000 7f9c2d8b8700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:06.998880+0000 front 2024-04-17T06:14:06.998908+0000 (oldest deadline 2024-04-17T06:14:32.899054+0000) 2024-04-17T06:14:33.404 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:33 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:33.231+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:53.138058+0000 front 2024-04-17T06:13:53.138083+0000 (oldest deadline 2024-04-17T06:14:14.238150+0000) 2024-04-17T06:14:33.404 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:33 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:33.231+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:13:58.939213+0000 front 2024-04-17T06:13:58.939286+0000 (oldest deadline 2024-04-17T06:14:24.839434+0000) 2024-04-17T06:14:33.404 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:33 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:33.231+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:07.040744+0000 front 2024-04-17T06:14:07.040770+0000 (oldest deadline 2024-04-17T06:14:30.441549+0000) 2024-04-17T06:14:34.032 INFO:journalctl@ceph.osd.5.smithi103.stdout:Apr 17 06:14:33 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-5[48375]: 2024-04-17T06:14:33.732+0000 7f6ecd773700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.033619+0000 front 2024-04-17T06:13:52.033577+0000 (oldest deadline 2024-04-17T06:14:16.733732+0000) 2024-04-17T06:14:34.032 INFO:journalctl@ceph.osd.5.smithi103.stdout:Apr 17 06:14:33 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-5[48375]: 2024-04-17T06:14:33.732+0000 7f6ecd773700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:13:56.734215+0000 front 2024-04-17T06:13:56.734133+0000 (oldest deadline 2024-04-17T06:14:23.134815+0000) 2024-04-17T06:14:34.032 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:33 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:33.934+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.989762+0000 front 2024-04-17T06:13:52.989781+0000 (oldest deadline 2024-04-17T06:14:18.289957+0000) 2024-04-17T06:14:34.033 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:33 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:33.934+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:01.191270+0000 front 2024-04-17T06:14:01.191086+0000 (oldest deadline 2024-04-17T06:14:24.690981+0000) 2024-04-17T06:14:34.133 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 06:14:33 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-7[54731]: 2024-04-17T06:14:33.825+0000 7f1a08efe700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:13:57.505129+0000 front 2024-04-17T06:13:57.505157+0000 (oldest deadline 2024-04-17T06:14:23.905794+0000) 2024-04-17T06:14:34.139 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 17 06:14:33 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-6[60771]: 2024-04-17T06:14:33.725+0000 7f3aa99e3700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.024894+0000 front 2024-04-17T06:13:52.024917+0000 (oldest deadline 2024-04-17T06:14:16.725097+0000) 2024-04-17T06:14:34.139 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 17 06:14:33 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-6[60771]: 2024-04-17T06:14:33.725+0000 7f3aa99e3700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:13:56.725456+0000 front 2024-04-17T06:13:56.725477+0000 (oldest deadline 2024-04-17T06:14:22.625606+0000) 2024-04-17T06:14:34.139 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 17 06:14:33 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-6[60771]: 2024-04-17T06:14:33.725+0000 7f3aa99e3700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:06.726829+0000 front 2024-04-17T06:14:06.726785+0000 (oldest deadline 2024-04-17T06:14:32.026942+0000) 2024-04-17T06:14:34.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:33 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:33.656+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.354382+0000 front 2024-04-17T06:13:52.354774+0000 (oldest deadline 2024-04-17T06:14:15.254487+0000) 2024-04-17T06:14:34.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:33 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:33.656+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:01.655772+0000 front 2024-04-17T06:14:01.655755+0000 (oldest deadline 2024-04-17T06:14:27.555801+0000) 2024-04-17T06:14:34.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:33 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:33.656+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:09.257128+0000 front 2024-04-17T06:14:09.257345+0000 (oldest deadline 2024-04-17T06:14:31.557233+0000) 2024-04-17T06:14:34.284 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 06:14:34 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-8[54941]: 2024-04-17T06:14:34.032+0000 7f9c2d8b8700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:00.597947+0000 front 2024-04-17T06:14:00.598005+0000 (oldest deadline 2024-04-17T06:14:24.698026+0000) 2024-04-17T06:14:34.284 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 06:14:34 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-8[54941]: 2024-04-17T06:14:34.032+0000 7f9c2d8b8700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:06.998880+0000 front 2024-04-17T06:14:06.998908+0000 (oldest deadline 2024-04-17T06:14:32.899054+0000) 2024-04-17T06:14:34.633 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:34 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:34.244+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:53.138058+0000 front 2024-04-17T06:13:53.138083+0000 (oldest deadline 2024-04-17T06:14:14.238150+0000) 2024-04-17T06:14:34.633 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:34 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:34.244+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:13:58.939213+0000 front 2024-04-17T06:13:58.939286+0000 (oldest deadline 2024-04-17T06:14:24.839434+0000) 2024-04-17T06:14:34.633 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:34 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:34.244+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:07.040744+0000 front 2024-04-17T06:14:07.040770+0000 (oldest deadline 2024-04-17T06:14:30.441549+0000) 2024-04-17T06:14:34.784 INFO:journalctl@ceph.osd.5.smithi103.stdout:Apr 17 06:14:34 smithi103 podman[73807]: 2024-04-17 06:14:34.515060862 +0000 UTC m=+5.466748668 container died 8071cfefa25b11f01690ad79cd06e3fada57fac93aba893e9a818a67e2c64d04 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS) 2024-04-17T06:14:35.093 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 17 06:14:34 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-6[60771]: 2024-04-17T06:14:34.703+0000 7f3aa99e3700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.024894+0000 front 2024-04-17T06:13:52.024917+0000 (oldest deadline 2024-04-17T06:14:16.725097+0000) 2024-04-17T06:14:35.094 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 17 06:14:34 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-6[60771]: 2024-04-17T06:14:34.703+0000 7f3aa99e3700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:13:56.725456+0000 front 2024-04-17T06:13:56.725477+0000 (oldest deadline 2024-04-17T06:14:22.625606+0000) 2024-04-17T06:14:35.094 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 17 06:14:34 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-6[60771]: 2024-04-17T06:14:34.703+0000 7f3aa99e3700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:06.726829+0000 front 2024-04-17T06:14:06.726785+0000 (oldest deadline 2024-04-17T06:14:32.026942+0000) 2024-04-17T06:14:35.094 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:34 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:34.657+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.354382+0000 front 2024-04-17T06:13:52.354774+0000 (oldest deadline 2024-04-17T06:14:15.254487+0000) 2024-04-17T06:14:35.094 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:34 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:34.657+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:01.655772+0000 front 2024-04-17T06:14:01.655755+0000 (oldest deadline 2024-04-17T06:14:27.555801+0000) 2024-04-17T06:14:35.094 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:34 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:34.657+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:09.257128+0000 front 2024-04-17T06:14:09.257345+0000 (oldest deadline 2024-04-17T06:14:31.557233+0000) 2024-04-17T06:14:35.133 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 06:14:34 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-7[54731]: 2024-04-17T06:14:34.830+0000 7f1a08efe700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:13:57.505129+0000 front 2024-04-17T06:13:57.505157+0000 (oldest deadline 2024-04-17T06:14:23.905794+0000) 2024-04-17T06:14:35.150 INFO:journalctl@ceph.osd.5.smithi103.stdout:Apr 17 06:14:34 smithi103 podman[73807]: 2024-04-17 06:14:34.951376221 +0000 UTC m=+5.903063999 container cleanup 8071cfefa25b11f01690ad79cd06e3fada57fac93aba893e9a818a67e2c64d04 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-17T06:14:35.151 INFO:journalctl@ceph.osd.5.smithi103.stdout:Apr 17 06:14:34 smithi103 bash[73807]: ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-5 2024-04-17T06:14:35.151 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 06:14:35 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-8[54941]: 2024-04-17T06:14:35.034+0000 7f9c2d8b8700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:00.597947+0000 front 2024-04-17T06:14:00.598005+0000 (oldest deadline 2024-04-17T06:14:24.698026+0000) 2024-04-17T06:14:35.151 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 06:14:35 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-8[54941]: 2024-04-17T06:14:35.034+0000 7f9c2d8b8700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:06.998880+0000 front 2024-04-17T06:14:06.998908+0000 (oldest deadline 2024-04-17T06:14:32.899054+0000) 2024-04-17T06:14:35.151 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:34 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:34.977+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.989762+0000 front 2024-04-17T06:13:52.989781+0000 (oldest deadline 2024-04-17T06:14:18.289957+0000) 2024-04-17T06:14:35.151 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:34 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:34.977+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:01.191270+0000 front 2024-04-17T06:14:01.191086+0000 (oldest deadline 2024-04-17T06:14:24.690981+0000) 2024-04-17T06:14:35.534 INFO:journalctl@ceph.osd.5.smithi103.stdout:Apr 17 06:14:35 smithi103 podman[73826]: 2024-04-17 06:14:35.153117365 +0000 UTC m=+0.637766905 container remove 8071cfefa25b11f01690ad79cd06e3fada57fac93aba893e9a818a67e2c64d04 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-17T06:14:35.633 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:35 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:35.202+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:53.138058+0000 front 2024-04-17T06:13:53.138083+0000 (oldest deadline 2024-04-17T06:14:14.238150+0000) 2024-04-17T06:14:35.633 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:35 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:35.202+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:13:58.939213+0000 front 2024-04-17T06:13:58.939286+0000 (oldest deadline 2024-04-17T06:14:24.839434+0000) 2024-04-17T06:14:35.633 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:35 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:35.202+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:07.040744+0000 front 2024-04-17T06:14:07.040770+0000 (oldest deadline 2024-04-17T06:14:30.441549+0000) 2024-04-17T06:14:35.863 INFO:journalctl@ceph.osd.5.smithi103.stdout:Apr 17 06:14:35 smithi103 podman[73920]: 2024-04-17 06:14:35.468237882 +0000 UTC m=+0.018563147 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-17T06:14:35.863 INFO:journalctl@ceph.osd.5.smithi103.stdout:Apr 17 06:14:35 smithi103 podman[73920]: 2024-04-17 06:14:35.596135368 +0000 UTC m=+0.146460633 container create 5e1fbefd3465bbb17fd5cae91cebd1d7fab9dcf35f33cb0983f404c297655c51 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-17T06:14:35.863 INFO:journalctl@ceph.osd.5.smithi103.stdout:Apr 17 06:14:35 smithi103 podman[73920]: 2024-04-17 06:14:35.865527926 +0000 UTC m=+0.415853188 container init 5e1fbefd3465bbb17fd5cae91cebd1d7fab9dcf35f33cb0983f404c297655c51 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=) 2024-04-17T06:14:35.889 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 17 06:14:35 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-6[60771]: 2024-04-17T06:14:35.678+0000 7f3aa99e3700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.024894+0000 front 2024-04-17T06:13:52.024917+0000 (oldest deadline 2024-04-17T06:14:16.725097+0000) 2024-04-17T06:14:35.889 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 17 06:14:35 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-6[60771]: 2024-04-17T06:14:35.678+0000 7f3aa99e3700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:13:56.725456+0000 front 2024-04-17T06:13:56.725477+0000 (oldest deadline 2024-04-17T06:14:22.625606+0000) 2024-04-17T06:14:35.889 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 17 06:14:35 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-6[60771]: 2024-04-17T06:14:35.678+0000 7f3aa99e3700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:06.726829+0000 front 2024-04-17T06:14:06.726785+0000 (oldest deadline 2024-04-17T06:14:32.026942+0000) 2024-04-17T06:14:35.889 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:35 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:35.611+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.354382+0000 front 2024-04-17T06:13:52.354774+0000 (oldest deadline 2024-04-17T06:14:15.254487+0000) 2024-04-17T06:14:35.890 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:35 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:35.611+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:01.655772+0000 front 2024-04-17T06:14:01.655755+0000 (oldest deadline 2024-04-17T06:14:27.555801+0000) 2024-04-17T06:14:35.890 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:35 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:35.611+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:09.257128+0000 front 2024-04-17T06:14:09.257345+0000 (oldest deadline 2024-04-17T06:14:31.557233+0000) 2024-04-17T06:14:36.133 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 06:14:35 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-7[54731]: 2024-04-17T06:14:35.799+0000 7f1a08efe700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:13:57.505129+0000 front 2024-04-17T06:13:57.505157+0000 (oldest deadline 2024-04-17T06:14:23.905794+0000) 2024-04-17T06:14:36.133 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 06:14:35 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-7[54731]: 2024-04-17T06:14:35.799+0000 7f1a08efe700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:09.707328+0000 front 2024-04-17T06:14:09.707119+0000 (oldest deadline 2024-04-17T06:14:35.007071+0000) 2024-04-17T06:14:36.150 INFO:journalctl@ceph.osd.5.smithi103.stdout:Apr 17 06:14:35 smithi103 podman[73920]: 2024-04-17 06:14:35.868728786 +0000 UTC m=+0.419054050 container start 5e1fbefd3465bbb17fd5cae91cebd1d7fab9dcf35f33cb0983f404c297655c51 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-5-deactivate, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-04-17T06:14:36.150 INFO:journalctl@ceph.osd.5.smithi103.stdout:Apr 17 06:14:35 smithi103 podman[73920]: 2024-04-17 06:14:35.965845624 +0000 UTC m=+0.516170906 container attach 5e1fbefd3465bbb17fd5cae91cebd1d7fab9dcf35f33cb0983f404c297655c51 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-5-deactivate, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-04-17T06:14:36.151 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 06:14:36 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-8[54941]: 2024-04-17T06:14:36.047+0000 7f9c2d8b8700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:00.597947+0000 front 2024-04-17T06:14:00.598005+0000 (oldest deadline 2024-04-17T06:14:24.698026+0000) 2024-04-17T06:14:36.151 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 06:14:36 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-8[54941]: 2024-04-17T06:14:36.047+0000 7f9c2d8b8700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:06.998880+0000 front 2024-04-17T06:14:06.998908+0000 (oldest deadline 2024-04-17T06:14:32.899054+0000) 2024-04-17T06:14:36.151 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:35 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:35.996+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.989762+0000 front 2024-04-17T06:13:52.989781+0000 (oldest deadline 2024-04-17T06:14:18.289957+0000) 2024-04-17T06:14:36.151 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:35 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:35.996+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:01.191270+0000 front 2024-04-17T06:14:01.191086+0000 (oldest deadline 2024-04-17T06:14:24.690981+0000) 2024-04-17T06:14:36.151 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:35 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:35.996+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:09.292281+0000 front 2024-04-17T06:14:09.292431+0000 (oldest deadline 2024-04-17T06:14:35.192525+0000) 2024-04-17T06:14:36.492 INFO:journalctl@ceph.osd.5.smithi103.stdout:Apr 17 06:14:36 smithi103 podman[73920]: 2024-04-17 06:14:36.153131729 +0000 UTC m=+0.703457009 container died 5e1fbefd3465bbb17fd5cae91cebd1d7fab9dcf35f33cb0983f404c297655c51 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-17T06:14:36.633 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:36 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:36.193+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:53.138058+0000 front 2024-04-17T06:13:53.138083+0000 (oldest deadline 2024-04-17T06:14:14.238150+0000) 2024-04-17T06:14:36.633 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:36 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:36.193+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:13:58.939213+0000 front 2024-04-17T06:13:58.939286+0000 (oldest deadline 2024-04-17T06:14:24.839434+0000) 2024-04-17T06:14:36.633 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:36 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:36.193+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:07.040744+0000 front 2024-04-17T06:14:07.040770+0000 (oldest deadline 2024-04-17T06:14:30.441549+0000) 2024-04-17T06:14:36.791 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@osd.5.service' 2024-04-17T06:14:36.833 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-17T06:14:36.833 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-04-17T06:14:36.833 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-04-17T06:14:36.833 DEBUG:teuthology.orchestra.run.smithi047:> sudo systemctl stop ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@osd.6 2024-04-17T06:14:37.034 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:37 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:37.011+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.989762+0000 front 2024-04-17T06:13:52.989781+0000 (oldest deadline 2024-04-17T06:14:18.289957+0000) 2024-04-17T06:14:37.034 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:37 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:37.011+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:01.191270+0000 front 2024-04-17T06:14:01.191086+0000 (oldest deadline 2024-04-17T06:14:24.690981+0000) 2024-04-17T06:14:37.034 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:37 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:37.011+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:09.292281+0000 front 2024-04-17T06:14:09.292431+0000 (oldest deadline 2024-04-17T06:14:35.192525+0000) 2024-04-17T06:14:37.133 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 06:14:36 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-7[54731]: 2024-04-17T06:14:36.782+0000 7f1a08efe700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:13:57.505129+0000 front 2024-04-17T06:13:57.505157+0000 (oldest deadline 2024-04-17T06:14:23.905794+0000) 2024-04-17T06:14:37.133 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 06:14:36 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-7[54731]: 2024-04-17T06:14:36.782+0000 7f1a08efe700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:09.707328+0000 front 2024-04-17T06:14:09.707119+0000 (oldest deadline 2024-04-17T06:14:35.007071+0000) 2024-04-17T06:14:37.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:36 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:36.644+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.354382+0000 front 2024-04-17T06:13:52.354774+0000 (oldest deadline 2024-04-17T06:14:15.254487+0000) 2024-04-17T06:14:37.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:36 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:36.644+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:01.655772+0000 front 2024-04-17T06:14:01.655755+0000 (oldest deadline 2024-04-17T06:14:27.555801+0000) 2024-04-17T06:14:37.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:36 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:36.644+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:09.257128+0000 front 2024-04-17T06:14:09.257345+0000 (oldest deadline 2024-04-17T06:14:31.557233+0000) 2024-04-17T06:14:37.139 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 17 06:14:36 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-6[60771]: 2024-04-17T06:14:36.706+0000 7f3aa99e3700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.024894+0000 front 2024-04-17T06:13:52.024917+0000 (oldest deadline 2024-04-17T06:14:16.725097+0000) 2024-04-17T06:14:37.139 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 17 06:14:36 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-6[60771]: 2024-04-17T06:14:36.706+0000 7f3aa99e3700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:13:56.725456+0000 front 2024-04-17T06:13:56.725477+0000 (oldest deadline 2024-04-17T06:14:22.625606+0000) 2024-04-17T06:14:37.139 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 17 06:14:36 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-6[60771]: 2024-04-17T06:14:36.706+0000 7f3aa99e3700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:06.726829+0000 front 2024-04-17T06:14:06.726785+0000 (oldest deadline 2024-04-17T06:14:32.026942+0000) 2024-04-17T06:14:37.139 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 17 06:14:36 smithi047 systemd[1]: Stopping Ceph osd.6 for 565db2d8-fc7e-11ee-bc91-c7b262605968... 2024-04-17T06:14:37.534 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 06:14:37 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-8[54941]: 2024-04-17T06:14:37.068+0000 7f9c2d8b8700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:00.597947+0000 front 2024-04-17T06:14:00.598005+0000 (oldest deadline 2024-04-17T06:14:24.698026+0000) 2024-04-17T06:14:37.534 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 06:14:37 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-8[54941]: 2024-04-17T06:14:37.068+0000 7f9c2d8b8700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:06.998880+0000 front 2024-04-17T06:14:06.998908+0000 (oldest deadline 2024-04-17T06:14:32.899054+0000) 2024-04-17T06:14:37.633 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:37 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:37.184+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:53.138058+0000 front 2024-04-17T06:13:53.138083+0000 (oldest deadline 2024-04-17T06:14:14.238150+0000) 2024-04-17T06:14:37.633 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:37 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:37.184+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:13:58.939213+0000 front 2024-04-17T06:13:58.939286+0000 (oldest deadline 2024-04-17T06:14:24.839434+0000) 2024-04-17T06:14:37.633 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:37 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:37.184+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:07.040744+0000 front 2024-04-17T06:14:07.040770+0000 (oldest deadline 2024-04-17T06:14:30.441549+0000) 2024-04-17T06:14:37.639 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 17 06:14:37 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-6[60771]: 2024-04-17T06:14:37.157+0000 7f3aad3ef700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-17T06:14:37.639 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 17 06:14:37 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-6[60771]: 2024-04-17T06:14:37.157+0000 7f3aad3ef700 -1 osd.6 109 *** Got signal Terminated *** 2024-04-17T06:14:37.639 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 17 06:14:37 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-6[60771]: 2024-04-17T06:14:37.157+0000 7f3aad3ef700 -1 osd.6 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-17T06:14:38.133 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 06:14:37 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-7[54731]: 2024-04-17T06:14:37.816+0000 7f1a08efe700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:13:57.505129+0000 front 2024-04-17T06:13:57.505157+0000 (oldest deadline 2024-04-17T06:14:23.905794+0000) 2024-04-17T06:14:38.133 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 06:14:37 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-7[54731]: 2024-04-17T06:14:37.816+0000 7f1a08efe700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:09.707328+0000 front 2024-04-17T06:14:09.707119+0000 (oldest deadline 2024-04-17T06:14:35.007071+0000) 2024-04-17T06:14:38.139 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 17 06:14:37 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-6[60771]: 2024-04-17T06:14:37.696+0000 7f3aa99e3700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.024894+0000 front 2024-04-17T06:13:52.024917+0000 (oldest deadline 2024-04-17T06:14:16.725097+0000) 2024-04-17T06:14:38.139 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 17 06:14:37 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-6[60771]: 2024-04-17T06:14:37.696+0000 7f3aa99e3700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:13:56.725456+0000 front 2024-04-17T06:13:56.725477+0000 (oldest deadline 2024-04-17T06:14:22.625606+0000) 2024-04-17T06:14:38.139 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 17 06:14:37 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-6[60771]: 2024-04-17T06:14:37.696+0000 7f3aa99e3700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:06.726829+0000 front 2024-04-17T06:14:06.726785+0000 (oldest deadline 2024-04-17T06:14:32.026942+0000) 2024-04-17T06:14:38.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:37 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:37.671+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.354382+0000 front 2024-04-17T06:13:52.354774+0000 (oldest deadline 2024-04-17T06:14:15.254487+0000) 2024-04-17T06:14:38.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:37 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:37.671+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:01.655772+0000 front 2024-04-17T06:14:01.655755+0000 (oldest deadline 2024-04-17T06:14:27.555801+0000) 2024-04-17T06:14:38.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:37 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:37.671+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:09.257128+0000 front 2024-04-17T06:14:09.257345+0000 (oldest deadline 2024-04-17T06:14:31.557233+0000) 2024-04-17T06:14:38.284 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 06:14:38 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-8[54941]: 2024-04-17T06:14:38.047+0000 7f9c2d8b8700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:00.597947+0000 front 2024-04-17T06:14:00.598005+0000 (oldest deadline 2024-04-17T06:14:24.698026+0000) 2024-04-17T06:14:38.284 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 06:14:38 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-8[54941]: 2024-04-17T06:14:38.047+0000 7f9c2d8b8700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:06.998880+0000 front 2024-04-17T06:14:06.998908+0000 (oldest deadline 2024-04-17T06:14:32.899054+0000) 2024-04-17T06:14:38.284 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:38 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:38.004+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.989762+0000 front 2024-04-17T06:13:52.989781+0000 (oldest deadline 2024-04-17T06:14:18.289957+0000) 2024-04-17T06:14:38.284 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:38 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:38.004+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:01.191270+0000 front 2024-04-17T06:14:01.191086+0000 (oldest deadline 2024-04-17T06:14:24.690981+0000) 2024-04-17T06:14:38.284 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:38 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:38.004+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:09.292281+0000 front 2024-04-17T06:14:09.292431+0000 (oldest deadline 2024-04-17T06:14:35.192525+0000) 2024-04-17T06:14:38.633 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:38 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:38.167+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:53.138058+0000 front 2024-04-17T06:13:53.138083+0000 (oldest deadline 2024-04-17T06:14:14.238150+0000) 2024-04-17T06:14:38.633 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:38 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:38.167+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:13:58.939213+0000 front 2024-04-17T06:13:58.939286+0000 (oldest deadline 2024-04-17T06:14:24.839434+0000) 2024-04-17T06:14:38.633 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:38 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:38.167+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:07.040744+0000 front 2024-04-17T06:14:07.040770+0000 (oldest deadline 2024-04-17T06:14:30.441549+0000) 2024-04-17T06:14:39.124 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 06:14:38 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-7[54731]: 2024-04-17T06:14:38.834+0000 7f1a08efe700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:13:57.505129+0000 front 2024-04-17T06:13:57.505157+0000 (oldest deadline 2024-04-17T06:14:23.905794+0000) 2024-04-17T06:14:39.124 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 06:14:38 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-7[54731]: 2024-04-17T06:14:38.834+0000 7f1a08efe700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:09.707328+0000 front 2024-04-17T06:14:09.707119+0000 (oldest deadline 2024-04-17T06:14:35.007071+0000) 2024-04-17T06:14:39.139 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 17 06:14:38 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-6[60771]: 2024-04-17T06:14:38.711+0000 7f3aa99e3700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.024894+0000 front 2024-04-17T06:13:52.024917+0000 (oldest deadline 2024-04-17T06:14:16.725097+0000) 2024-04-17T06:14:39.139 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 17 06:14:38 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-6[60771]: 2024-04-17T06:14:38.711+0000 7f3aa99e3700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:13:56.725456+0000 front 2024-04-17T06:13:56.725477+0000 (oldest deadline 2024-04-17T06:14:22.625606+0000) 2024-04-17T06:14:39.139 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 17 06:14:38 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-6[60771]: 2024-04-17T06:14:38.711+0000 7f3aa99e3700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:06.726829+0000 front 2024-04-17T06:14:06.726785+0000 (oldest deadline 2024-04-17T06:14:32.026942+0000) 2024-04-17T06:14:39.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:38 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:38.718+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.354382+0000 front 2024-04-17T06:13:52.354774+0000 (oldest deadline 2024-04-17T06:14:15.254487+0000) 2024-04-17T06:14:39.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:38 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:38.718+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:01.655772+0000 front 2024-04-17T06:14:01.655755+0000 (oldest deadline 2024-04-17T06:14:27.555801+0000) 2024-04-17T06:14:39.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:38 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:38.718+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:09.257128+0000 front 2024-04-17T06:14:09.257345+0000 (oldest deadline 2024-04-17T06:14:31.557233+0000) 2024-04-17T06:14:39.284 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:38 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:38.989+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.989762+0000 front 2024-04-17T06:13:52.989781+0000 (oldest deadline 2024-04-17T06:14:18.289957+0000) 2024-04-17T06:14:39.284 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:38 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:38.989+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:01.191270+0000 front 2024-04-17T06:14:01.191086+0000 (oldest deadline 2024-04-17T06:14:24.690981+0000) 2024-04-17T06:14:39.284 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:38 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:38.989+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:09.292281+0000 front 2024-04-17T06:14:09.292431+0000 (oldest deadline 2024-04-17T06:14:35.192525+0000) 2024-04-17T06:14:39.284 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 06:14:39 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-8[54941]: 2024-04-17T06:14:39.034+0000 7f9c2d8b8700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:00.597947+0000 front 2024-04-17T06:14:00.598005+0000 (oldest deadline 2024-04-17T06:14:24.698026+0000) 2024-04-17T06:14:39.284 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 06:14:39 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-8[54941]: 2024-04-17T06:14:39.034+0000 7f9c2d8b8700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:06.998880+0000 front 2024-04-17T06:14:06.998908+0000 (oldest deadline 2024-04-17T06:14:32.899054+0000) 2024-04-17T06:14:39.284 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 06:14:39 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-8[54941]: 2024-04-17T06:14:39.034+0000 7f9c2d8b8700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-17T06:14:12.899673+0000 front 2024-04-17T06:14:12.899701+0000 (oldest deadline 2024-04-17T06:14:38.799720+0000) 2024-04-17T06:14:39.383 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:39 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:39.126+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:53.138058+0000 front 2024-04-17T06:13:53.138083+0000 (oldest deadline 2024-04-17T06:14:14.238150+0000) 2024-04-17T06:14:39.383 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:39 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:39.126+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:13:58.939213+0000 front 2024-04-17T06:13:58.939286+0000 (oldest deadline 2024-04-17T06:14:24.839434+0000) 2024-04-17T06:14:39.383 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:39 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:39.126+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:07.040744+0000 front 2024-04-17T06:14:07.040770+0000 (oldest deadline 2024-04-17T06:14:30.441549+0000) 2024-04-17T06:14:39.383 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:39 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:39.126+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-17T06:14:15.743014+0000 front 2024-04-17T06:14:15.742885+0000 (oldest deadline 2024-04-17T06:14:38.642710+0000) 2024-04-17T06:14:40.133 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 06:14:39 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-7[54731]: 2024-04-17T06:14:39.844+0000 7f1a08efe700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:13:57.505129+0000 front 2024-04-17T06:13:57.505157+0000 (oldest deadline 2024-04-17T06:14:23.905794+0000) 2024-04-17T06:14:40.133 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 06:14:39 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-7[54731]: 2024-04-17T06:14:39.844+0000 7f1a08efe700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:09.707328+0000 front 2024-04-17T06:14:09.707119+0000 (oldest deadline 2024-04-17T06:14:35.007071+0000) 2024-04-17T06:14:40.133 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 06:14:39 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-7[54731]: 2024-04-17T06:14:39.844+0000 7f1a08efe700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-17T06:14:15.007691+0000 front 2024-04-17T06:14:15.007814+0000 (oldest deadline 2024-04-17T06:14:39.707750+0000) 2024-04-17T06:14:40.139 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 17 06:14:39 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-6[60771]: 2024-04-17T06:14:39.743+0000 7f3aa99e3700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.024894+0000 front 2024-04-17T06:13:52.024917+0000 (oldest deadline 2024-04-17T06:14:16.725097+0000) 2024-04-17T06:14:40.139 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 17 06:14:39 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-6[60771]: 2024-04-17T06:14:39.743+0000 7f3aa99e3700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:13:56.725456+0000 front 2024-04-17T06:13:56.725477+0000 (oldest deadline 2024-04-17T06:14:22.625606+0000) 2024-04-17T06:14:40.139 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 17 06:14:39 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-6[60771]: 2024-04-17T06:14:39.743+0000 7f3aa99e3700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:06.726829+0000 front 2024-04-17T06:14:06.726785+0000 (oldest deadline 2024-04-17T06:14:32.026942+0000) 2024-04-17T06:14:40.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:39 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:39.709+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.354382+0000 front 2024-04-17T06:13:52.354774+0000 (oldest deadline 2024-04-17T06:14:15.254487+0000) 2024-04-17T06:14:40.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:39 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:39.709+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:01.655772+0000 front 2024-04-17T06:14:01.655755+0000 (oldest deadline 2024-04-17T06:14:27.555801+0000) 2024-04-17T06:14:40.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:39 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:39.709+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:09.257128+0000 front 2024-04-17T06:14:09.257345+0000 (oldest deadline 2024-04-17T06:14:31.557233+0000) 2024-04-17T06:14:40.284 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 06:14:40 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-8[54941]: 2024-04-17T06:14:40.021+0000 7f9c2d8b8700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:00.597947+0000 front 2024-04-17T06:14:00.598005+0000 (oldest deadline 2024-04-17T06:14:24.698026+0000) 2024-04-17T06:14:40.284 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 06:14:40 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-8[54941]: 2024-04-17T06:14:40.021+0000 7f9c2d8b8700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:06.998880+0000 front 2024-04-17T06:14:06.998908+0000 (oldest deadline 2024-04-17T06:14:32.899054+0000) 2024-04-17T06:14:40.284 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 06:14:40 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-8[54941]: 2024-04-17T06:14:40.021+0000 7f9c2d8b8700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-17T06:14:12.899673+0000 front 2024-04-17T06:14:12.899701+0000 (oldest deadline 2024-04-17T06:14:38.799720+0000) 2024-04-17T06:14:40.284 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:39 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:39.965+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.989762+0000 front 2024-04-17T06:13:52.989781+0000 (oldest deadline 2024-04-17T06:14:18.289957+0000) 2024-04-17T06:14:40.284 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:39 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:39.965+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:01.191270+0000 front 2024-04-17T06:14:01.191086+0000 (oldest deadline 2024-04-17T06:14:24.690981+0000) 2024-04-17T06:14:40.285 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:39 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:39.965+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:09.292281+0000 front 2024-04-17T06:14:09.292431+0000 (oldest deadline 2024-04-17T06:14:35.192525+0000) 2024-04-17T06:14:40.633 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:40 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:40.149+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:53.138058+0000 front 2024-04-17T06:13:53.138083+0000 (oldest deadline 2024-04-17T06:14:14.238150+0000) 2024-04-17T06:14:40.633 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:40 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:40.149+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:13:58.939213+0000 front 2024-04-17T06:13:58.939286+0000 (oldest deadline 2024-04-17T06:14:24.839434+0000) 2024-04-17T06:14:40.633 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:40 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:40.149+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:07.040744+0000 front 2024-04-17T06:14:07.040770+0000 (oldest deadline 2024-04-17T06:14:30.441549+0000) 2024-04-17T06:14:40.633 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:40 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:40.149+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-17T06:14:15.743014+0000 front 2024-04-17T06:14:15.742885+0000 (oldest deadline 2024-04-17T06:14:38.642710+0000) 2024-04-17T06:14:41.099 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 06:14:40 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-7[54731]: 2024-04-17T06:14:40.824+0000 7f1a08efe700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:13:57.505129+0000 front 2024-04-17T06:13:57.505157+0000 (oldest deadline 2024-04-17T06:14:23.905794+0000) 2024-04-17T06:14:41.100 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 06:14:40 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-7[54731]: 2024-04-17T06:14:40.824+0000 7f1a08efe700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:09.707328+0000 front 2024-04-17T06:14:09.707119+0000 (oldest deadline 2024-04-17T06:14:35.007071+0000) 2024-04-17T06:14:41.100 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 06:14:40 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-7[54731]: 2024-04-17T06:14:40.824+0000 7f1a08efe700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-17T06:14:15.007691+0000 front 2024-04-17T06:14:15.007814+0000 (oldest deadline 2024-04-17T06:14:39.707750+0000) 2024-04-17T06:14:41.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:40 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:40.703+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.354382+0000 front 2024-04-17T06:13:52.354774+0000 (oldest deadline 2024-04-17T06:14:15.254487+0000) 2024-04-17T06:14:41.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:40 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:40.703+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:01.655772+0000 front 2024-04-17T06:14:01.655755+0000 (oldest deadline 2024-04-17T06:14:27.555801+0000) 2024-04-17T06:14:41.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:40 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:40.703+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:09.257128+0000 front 2024-04-17T06:14:09.257345+0000 (oldest deadline 2024-04-17T06:14:31.557233+0000) 2024-04-17T06:14:41.139 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 17 06:14:40 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-6[60771]: 2024-04-17T06:14:40.742+0000 7f3aa99e3700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.024894+0000 front 2024-04-17T06:13:52.024917+0000 (oldest deadline 2024-04-17T06:14:16.725097+0000) 2024-04-17T06:14:41.139 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 17 06:14:40 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-6[60771]: 2024-04-17T06:14:40.742+0000 7f3aa99e3700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:13:56.725456+0000 front 2024-04-17T06:13:56.725477+0000 (oldest deadline 2024-04-17T06:14:22.625606+0000) 2024-04-17T06:14:41.139 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 17 06:14:40 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-6[60771]: 2024-04-17T06:14:40.742+0000 7f3aa99e3700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:06.726829+0000 front 2024-04-17T06:14:06.726785+0000 (oldest deadline 2024-04-17T06:14:32.026942+0000) 2024-04-17T06:14:41.139 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 17 06:14:40 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-6[60771]: 2024-04-17T06:14:40.742+0000 7f3aa99e3700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-17T06:14:17.228691+0000 front 2024-04-17T06:14:17.228612+0000 (oldest deadline 2024-04-17T06:14:40.728694+0000) 2024-04-17T06:14:41.284 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:40 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:40.936+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.989762+0000 front 2024-04-17T06:13:52.989781+0000 (oldest deadline 2024-04-17T06:14:18.289957+0000) 2024-04-17T06:14:41.284 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:40 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:40.936+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:01.191270+0000 front 2024-04-17T06:14:01.191086+0000 (oldest deadline 2024-04-17T06:14:24.690981+0000) 2024-04-17T06:14:41.284 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:40 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:40.936+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:09.292281+0000 front 2024-04-17T06:14:09.292431+0000 (oldest deadline 2024-04-17T06:14:35.192525+0000) 2024-04-17T06:14:41.284 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 06:14:41 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-8[54941]: 2024-04-17T06:14:41.034+0000 7f9c2d8b8700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:00.597947+0000 front 2024-04-17T06:14:00.598005+0000 (oldest deadline 2024-04-17T06:14:24.698026+0000) 2024-04-17T06:14:41.284 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 06:14:41 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-8[54941]: 2024-04-17T06:14:41.034+0000 7f9c2d8b8700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:06.998880+0000 front 2024-04-17T06:14:06.998908+0000 (oldest deadline 2024-04-17T06:14:32.899054+0000) 2024-04-17T06:14:41.285 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 06:14:41 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-8[54941]: 2024-04-17T06:14:41.034+0000 7f9c2d8b8700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-17T06:14:12.899673+0000 front 2024-04-17T06:14:12.899701+0000 (oldest deadline 2024-04-17T06:14:38.799720+0000) 2024-04-17T06:14:41.383 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:41 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:41.101+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:53.138058+0000 front 2024-04-17T06:13:53.138083+0000 (oldest deadline 2024-04-17T06:14:14.238150+0000) 2024-04-17T06:14:41.383 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:41 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:41.101+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:13:58.939213+0000 front 2024-04-17T06:13:58.939286+0000 (oldest deadline 2024-04-17T06:14:24.839434+0000) 2024-04-17T06:14:41.383 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:41 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:41.101+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:07.040744+0000 front 2024-04-17T06:14:07.040770+0000 (oldest deadline 2024-04-17T06:14:30.441549+0000) 2024-04-17T06:14:41.383 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:41 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:41.101+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-17T06:14:15.743014+0000 front 2024-04-17T06:14:15.742885+0000 (oldest deadline 2024-04-17T06:14:38.642710+0000) 2024-04-17T06:14:42.133 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 06:14:41 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-7[54731]: 2024-04-17T06:14:41.866+0000 7f1a08efe700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:13:57.505129+0000 front 2024-04-17T06:13:57.505157+0000 (oldest deadline 2024-04-17T06:14:23.905794+0000) 2024-04-17T06:14:42.133 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 06:14:41 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-7[54731]: 2024-04-17T06:14:41.866+0000 7f1a08efe700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:09.707328+0000 front 2024-04-17T06:14:09.707119+0000 (oldest deadline 2024-04-17T06:14:35.007071+0000) 2024-04-17T06:14:42.133 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 06:14:41 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-7[54731]: 2024-04-17T06:14:41.866+0000 7f1a08efe700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-17T06:14:15.007691+0000 front 2024-04-17T06:14:15.007814+0000 (oldest deadline 2024-04-17T06:14:39.707750+0000) 2024-04-17T06:14:42.133 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:42 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:42.095+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:53.138058+0000 front 2024-04-17T06:13:53.138083+0000 (oldest deadline 2024-04-17T06:14:14.238150+0000) 2024-04-17T06:14:42.133 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:42 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:42.095+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:13:58.939213+0000 front 2024-04-17T06:13:58.939286+0000 (oldest deadline 2024-04-17T06:14:24.839434+0000) 2024-04-17T06:14:42.133 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:42 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:42.095+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:07.040744+0000 front 2024-04-17T06:14:07.040770+0000 (oldest deadline 2024-04-17T06:14:30.441549+0000) 2024-04-17T06:14:42.133 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:42 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:42.095+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-17T06:14:15.743014+0000 front 2024-04-17T06:14:15.742885+0000 (oldest deadline 2024-04-17T06:14:38.642710+0000) 2024-04-17T06:14:42.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:41 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:41.686+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.354382+0000 front 2024-04-17T06:13:52.354774+0000 (oldest deadline 2024-04-17T06:14:15.254487+0000) 2024-04-17T06:14:42.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:41 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:41.686+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:01.655772+0000 front 2024-04-17T06:14:01.655755+0000 (oldest deadline 2024-04-17T06:14:27.555801+0000) 2024-04-17T06:14:42.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:41 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:41.686+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:09.257128+0000 front 2024-04-17T06:14:09.257345+0000 (oldest deadline 2024-04-17T06:14:31.557233+0000) 2024-04-17T06:14:42.139 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 17 06:14:41 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-6[60771]: 2024-04-17T06:14:41.709+0000 7f3aa99e3700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.024894+0000 front 2024-04-17T06:13:52.024917+0000 (oldest deadline 2024-04-17T06:14:16.725097+0000) 2024-04-17T06:14:42.139 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 17 06:14:41 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-6[60771]: 2024-04-17T06:14:41.709+0000 7f3aa99e3700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:13:56.725456+0000 front 2024-04-17T06:13:56.725477+0000 (oldest deadline 2024-04-17T06:14:22.625606+0000) 2024-04-17T06:14:42.139 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 17 06:14:41 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-6[60771]: 2024-04-17T06:14:41.709+0000 7f3aa99e3700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:06.726829+0000 front 2024-04-17T06:14:06.726785+0000 (oldest deadline 2024-04-17T06:14:32.026942+0000) 2024-04-17T06:14:42.139 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 17 06:14:41 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-6[60771]: 2024-04-17T06:14:41.709+0000 7f3aa99e3700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-17T06:14:17.228691+0000 front 2024-04-17T06:14:17.228612+0000 (oldest deadline 2024-04-17T06:14:40.728694+0000) 2024-04-17T06:14:42.284 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:41 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:41.941+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.989762+0000 front 2024-04-17T06:13:52.989781+0000 (oldest deadline 2024-04-17T06:14:18.289957+0000) 2024-04-17T06:14:42.284 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:41 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:41.941+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:01.191270+0000 front 2024-04-17T06:14:01.191086+0000 (oldest deadline 2024-04-17T06:14:24.690981+0000) 2024-04-17T06:14:42.284 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:41 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:41.941+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:09.292281+0000 front 2024-04-17T06:14:09.292431+0000 (oldest deadline 2024-04-17T06:14:35.192525+0000) 2024-04-17T06:14:42.284 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 06:14:41 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-8[54941]: 2024-04-17T06:14:41.994+0000 7f9c2d8b8700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:00.597947+0000 front 2024-04-17T06:14:00.598005+0000 (oldest deadline 2024-04-17T06:14:24.698026+0000) 2024-04-17T06:14:42.284 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 06:14:41 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-8[54941]: 2024-04-17T06:14:41.994+0000 7f9c2d8b8700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:06.998880+0000 front 2024-04-17T06:14:06.998908+0000 (oldest deadline 2024-04-17T06:14:32.899054+0000) 2024-04-17T06:14:42.284 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 06:14:41 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-8[54941]: 2024-04-17T06:14:41.994+0000 7f9c2d8b8700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-17T06:14:12.899673+0000 front 2024-04-17T06:14:12.899701+0000 (oldest deadline 2024-04-17T06:14:38.799720+0000) 2024-04-17T06:14:42.999 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 17 06:14:42 smithi047 podman[94181]: 2024-04-17 06:14:42.694177945 +0000 UTC m=+5.653575799 container died b477be316c817b9ebe18b56ddcbf5ab1ed36e068f26787512b8e131cebfe88f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-6, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-04-17T06:14:42.999 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:42 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:42.701+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.354382+0000 front 2024-04-17T06:13:52.354774+0000 (oldest deadline 2024-04-17T06:14:15.254487+0000) 2024-04-17T06:14:42.999 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:42 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:42.701+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:01.655772+0000 front 2024-04-17T06:14:01.655755+0000 (oldest deadline 2024-04-17T06:14:27.555801+0000) 2024-04-17T06:14:42.999 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:42 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:42.701+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:09.257128+0000 front 2024-04-17T06:14:09.257345+0000 (oldest deadline 2024-04-17T06:14:31.557233+0000) 2024-04-17T06:14:42.999 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:42 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:42.701+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-17T06:14:17.958812+0000 front 2024-04-17T06:14:17.958952+0000 (oldest deadline 2024-04-17T06:14:42.658979+0000) 2024-04-17T06:14:43.133 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 06:14:42 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-7[54731]: 2024-04-17T06:14:42.869+0000 7f1a08efe700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:13:57.505129+0000 front 2024-04-17T06:13:57.505157+0000 (oldest deadline 2024-04-17T06:14:23.905794+0000) 2024-04-17T06:14:43.133 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 06:14:42 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-7[54731]: 2024-04-17T06:14:42.869+0000 7f1a08efe700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:09.707328+0000 front 2024-04-17T06:14:09.707119+0000 (oldest deadline 2024-04-17T06:14:35.007071+0000) 2024-04-17T06:14:43.133 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 06:14:42 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-7[54731]: 2024-04-17T06:14:42.869+0000 7f1a08efe700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-17T06:14:15.007691+0000 front 2024-04-17T06:14:15.007814+0000 (oldest deadline 2024-04-17T06:14:39.707750+0000) 2024-04-17T06:14:43.133 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:43 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:43.050+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:53.138058+0000 front 2024-04-17T06:13:53.138083+0000 (oldest deadline 2024-04-17T06:14:14.238150+0000) 2024-04-17T06:14:43.133 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:43 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:43.050+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:13:58.939213+0000 front 2024-04-17T06:13:58.939286+0000 (oldest deadline 2024-04-17T06:14:24.839434+0000) 2024-04-17T06:14:43.133 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:43 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:43.050+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:07.040744+0000 front 2024-04-17T06:14:07.040770+0000 (oldest deadline 2024-04-17T06:14:30.441549+0000) 2024-04-17T06:14:43.134 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:43 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:43.050+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-17T06:14:15.743014+0000 front 2024-04-17T06:14:15.742885+0000 (oldest deadline 2024-04-17T06:14:38.642710+0000) 2024-04-17T06:14:43.284 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:42 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:42.894+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.989762+0000 front 2024-04-17T06:13:52.989781+0000 (oldest deadline 2024-04-17T06:14:18.289957+0000) 2024-04-17T06:14:43.284 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:42 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:42.894+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:01.191270+0000 front 2024-04-17T06:14:01.191086+0000 (oldest deadline 2024-04-17T06:14:24.690981+0000) 2024-04-17T06:14:43.284 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:42 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:42.894+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:09.292281+0000 front 2024-04-17T06:14:09.292431+0000 (oldest deadline 2024-04-17T06:14:35.192525+0000) 2024-04-17T06:14:43.284 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 06:14:43 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-8[54941]: 2024-04-17T06:14:43.006+0000 7f9c2d8b8700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:00.597947+0000 front 2024-04-17T06:14:00.598005+0000 (oldest deadline 2024-04-17T06:14:24.698026+0000) 2024-04-17T06:14:43.284 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 06:14:43 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-8[54941]: 2024-04-17T06:14:43.006+0000 7f9c2d8b8700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:06.998880+0000 front 2024-04-17T06:14:06.998908+0000 (oldest deadline 2024-04-17T06:14:32.899054+0000) 2024-04-17T06:14:43.284 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 06:14:43 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-8[54941]: 2024-04-17T06:14:43.006+0000 7f9c2d8b8700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-17T06:14:12.899673+0000 front 2024-04-17T06:14:12.899701+0000 (oldest deadline 2024-04-17T06:14:38.799720+0000) 2024-04-17T06:14:43.357 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 17 06:14:43 smithi047 podman[94181]: 2024-04-17 06:14:43.091947337 +0000 UTC m=+6.051345172 container cleanup b477be316c817b9ebe18b56ddcbf5ab1ed36e068f26787512b8e131cebfe88f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-6, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.schema-version=1.0, ceph=True) 2024-04-17T06:14:43.357 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 17 06:14:43 smithi047 bash[94181]: ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-6 2024-04-17T06:14:43.639 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 17 06:14:43 smithi047 podman[94198]: 2024-04-17 06:14:43.359714483 +0000 UTC m=+0.658375119 container remove b477be316c817b9ebe18b56ddcbf5ab1ed36e068f26787512b8e131cebfe88f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-6, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux ) 2024-04-17T06:14:43.989 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 17 06:14:43 smithi047 podman[94290]: 2024-04-17 06:14:43.628954453 +0000 UTC m=+0.018612525 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-17T06:14:43.990 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 17 06:14:43 smithi047 podman[94290]: 2024-04-17 06:14:43.869052714 +0000 UTC m=+0.258710826 container create 2c5d76cab0c645fdff1f5bd0b8f64edfc3ac5bba21a9bbac1e5ee62cae4a03f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-6-deactivate, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-17T06:14:43.990 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:43 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:43.706+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.354382+0000 front 2024-04-17T06:13:52.354774+0000 (oldest deadline 2024-04-17T06:14:15.254487+0000) 2024-04-17T06:14:43.990 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:43 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:43.706+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:01.655772+0000 front 2024-04-17T06:14:01.655755+0000 (oldest deadline 2024-04-17T06:14:27.555801+0000) 2024-04-17T06:14:43.990 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:43 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:43.706+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:09.257128+0000 front 2024-04-17T06:14:09.257345+0000 (oldest deadline 2024-04-17T06:14:31.557233+0000) 2024-04-17T06:14:43.990 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:43 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:43.706+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-17T06:14:17.958812+0000 front 2024-04-17T06:14:17.958952+0000 (oldest deadline 2024-04-17T06:14:42.658979+0000) 2024-04-17T06:14:44.133 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 06:14:43 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-7[54731]: 2024-04-17T06:14:43.821+0000 7f1a08efe700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:13:57.505129+0000 front 2024-04-17T06:13:57.505157+0000 (oldest deadline 2024-04-17T06:14:23.905794+0000) 2024-04-17T06:14:44.133 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 06:14:43 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-7[54731]: 2024-04-17T06:14:43.821+0000 7f1a08efe700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:09.707328+0000 front 2024-04-17T06:14:09.707119+0000 (oldest deadline 2024-04-17T06:14:35.007071+0000) 2024-04-17T06:14:44.133 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 06:14:43 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-7[54731]: 2024-04-17T06:14:43.821+0000 7f1a08efe700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-17T06:14:15.007691+0000 front 2024-04-17T06:14:15.007814+0000 (oldest deadline 2024-04-17T06:14:39.707750+0000) 2024-04-17T06:14:44.134 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:44 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:44.026+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:53.138058+0000 front 2024-04-17T06:13:53.138083+0000 (oldest deadline 2024-04-17T06:14:14.238150+0000) 2024-04-17T06:14:44.134 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:44 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:44.026+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:13:58.939213+0000 front 2024-04-17T06:13:58.939286+0000 (oldest deadline 2024-04-17T06:14:24.839434+0000) 2024-04-17T06:14:44.134 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:44 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:44.026+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:07.040744+0000 front 2024-04-17T06:14:07.040770+0000 (oldest deadline 2024-04-17T06:14:30.441549+0000) 2024-04-17T06:14:44.134 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:44 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:44.026+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-17T06:14:15.743014+0000 front 2024-04-17T06:14:15.742885+0000 (oldest deadline 2024-04-17T06:14:38.642710+0000) 2024-04-17T06:14:44.284 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:43 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:43.940+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.989762+0000 front 2024-04-17T06:13:52.989781+0000 (oldest deadline 2024-04-17T06:14:18.289957+0000) 2024-04-17T06:14:44.284 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:43 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:43.940+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:01.191270+0000 front 2024-04-17T06:14:01.191086+0000 (oldest deadline 2024-04-17T06:14:24.690981+0000) 2024-04-17T06:14:44.284 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:43 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:43.940+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:09.292281+0000 front 2024-04-17T06:14:09.292431+0000 (oldest deadline 2024-04-17T06:14:35.192525+0000) 2024-04-17T06:14:44.284 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 06:14:44 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-8[54941]: 2024-04-17T06:14:44.017+0000 7f9c2d8b8700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:00.597947+0000 front 2024-04-17T06:14:00.598005+0000 (oldest deadline 2024-04-17T06:14:24.698026+0000) 2024-04-17T06:14:44.284 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 06:14:44 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-8[54941]: 2024-04-17T06:14:44.017+0000 7f9c2d8b8700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:06.998880+0000 front 2024-04-17T06:14:06.998908+0000 (oldest deadline 2024-04-17T06:14:32.899054+0000) 2024-04-17T06:14:44.285 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 06:14:44 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-8[54941]: 2024-04-17T06:14:44.017+0000 7f9c2d8b8700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-17T06:14:12.899673+0000 front 2024-04-17T06:14:12.899701+0000 (oldest deadline 2024-04-17T06:14:38.799720+0000) 2024-04-17T06:14:44.389 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 17 06:14:44 smithi047 podman[94290]: 2024-04-17 06:14:44.130602402 +0000 UTC m=+0.520260459 container init 2c5d76cab0c645fdff1f5bd0b8f64edfc3ac5bba21a9bbac1e5ee62cae4a03f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-04-17T06:14:44.389 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 17 06:14:44 smithi047 podman[94290]: 2024-04-17 06:14:44.134391834 +0000 UTC m=+0.524049882 container start 2c5d76cab0c645fdff1f5bd0b8f64edfc3ac5bba21a9bbac1e5ee62cae4a03f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-6-deactivate, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-04-17T06:14:44.389 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 17 06:14:44 smithi047 podman[94290]: 2024-04-17 06:14:44.213326849 +0000 UTC m=+0.602984897 container attach 2c5d76cab0c645fdff1f5bd0b8f64edfc3ac5bba21a9bbac1e5ee62cae4a03f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-6-deactivate, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.vendor=CentOS) 2024-04-17T06:14:44.714 INFO:journalctl@ceph.osd.6.smithi047.stdout:Apr 17 06:14:44 smithi047 podman[94290]: 2024-04-17 06:14:44.443154897 +0000 UTC m=+0.832812958 container died 2c5d76cab0c645fdff1f5bd0b8f64edfc3ac5bba21a9bbac1e5ee62cae4a03f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, ceph=True, org.label-schema.schema-version=1.0) 2024-04-17T06:14:45.015 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:44 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:44.719+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.354382+0000 front 2024-04-17T06:13:52.354774+0000 (oldest deadline 2024-04-17T06:14:15.254487+0000) 2024-04-17T06:14:45.015 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:44 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:44.719+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:01.655772+0000 front 2024-04-17T06:14:01.655755+0000 (oldest deadline 2024-04-17T06:14:27.555801+0000) 2024-04-17T06:14:45.015 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:44 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:44.719+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:09.257128+0000 front 2024-04-17T06:14:09.257345+0000 (oldest deadline 2024-04-17T06:14:31.557233+0000) 2024-04-17T06:14:45.015 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:44 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:44.719+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-17T06:14:17.958812+0000 front 2024-04-17T06:14:17.958952+0000 (oldest deadline 2024-04-17T06:14:42.658979+0000) 2024-04-17T06:14:45.043 DEBUG:teuthology.orchestra.run.smithi047:> sudo pkill -f 'journalctl -f -n 0 -u ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@osd.6.service' 2024-04-17T06:14:45.089 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-17T06:14:45.089 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-04-17T06:14:45.090 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-04-17T06:14:45.090 DEBUG:teuthology.orchestra.run.smithi098:> sudo systemctl stop ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@osd.7 2024-04-17T06:14:45.120 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 06:14:44 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-7[54731]: 2024-04-17T06:14:44.857+0000 7f1a08efe700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:13:57.505129+0000 front 2024-04-17T06:13:57.505157+0000 (oldest deadline 2024-04-17T06:14:23.905794+0000) 2024-04-17T06:14:45.121 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 06:14:44 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-7[54731]: 2024-04-17T06:14:44.857+0000 7f1a08efe700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:09.707328+0000 front 2024-04-17T06:14:09.707119+0000 (oldest deadline 2024-04-17T06:14:35.007071+0000) 2024-04-17T06:14:45.121 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 06:14:44 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-7[54731]: 2024-04-17T06:14:44.857+0000 7f1a08efe700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-17T06:14:15.007691+0000 front 2024-04-17T06:14:15.007814+0000 (oldest deadline 2024-04-17T06:14:39.707750+0000) 2024-04-17T06:14:45.121 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:45 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:45.025+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:53.138058+0000 front 2024-04-17T06:13:53.138083+0000 (oldest deadline 2024-04-17T06:14:14.238150+0000) 2024-04-17T06:14:45.121 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:45 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:45.025+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:13:58.939213+0000 front 2024-04-17T06:13:58.939286+0000 (oldest deadline 2024-04-17T06:14:24.839434+0000) 2024-04-17T06:14:45.121 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:45 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:45.025+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:07.040744+0000 front 2024-04-17T06:14:07.040770+0000 (oldest deadline 2024-04-17T06:14:30.441549+0000) 2024-04-17T06:14:45.121 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:45 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:45.025+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-17T06:14:15.743014+0000 front 2024-04-17T06:14:15.742885+0000 (oldest deadline 2024-04-17T06:14:38.642710+0000) 2024-04-17T06:14:45.284 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:44 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:44.902+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.989762+0000 front 2024-04-17T06:13:52.989781+0000 (oldest deadline 2024-04-17T06:14:18.289957+0000) 2024-04-17T06:14:45.284 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:44 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:44.902+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:01.191270+0000 front 2024-04-17T06:14:01.191086+0000 (oldest deadline 2024-04-17T06:14:24.690981+0000) 2024-04-17T06:14:45.284 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:44 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:44.902+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:09.292281+0000 front 2024-04-17T06:14:09.292431+0000 (oldest deadline 2024-04-17T06:14:35.192525+0000) 2024-04-17T06:14:45.284 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 06:14:45 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-8[54941]: 2024-04-17T06:14:45.018+0000 7f9c2d8b8700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:00.597947+0000 front 2024-04-17T06:14:00.598005+0000 (oldest deadline 2024-04-17T06:14:24.698026+0000) 2024-04-17T06:14:45.284 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 06:14:45 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-8[54941]: 2024-04-17T06:14:45.018+0000 7f9c2d8b8700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:06.998880+0000 front 2024-04-17T06:14:06.998908+0000 (oldest deadline 2024-04-17T06:14:32.899054+0000) 2024-04-17T06:14:45.284 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 06:14:45 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-8[54941]: 2024-04-17T06:14:45.018+0000 7f9c2d8b8700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-17T06:14:12.899673+0000 front 2024-04-17T06:14:12.899701+0000 (oldest deadline 2024-04-17T06:14:38.799720+0000) 2024-04-17T06:14:45.383 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 06:14:45 smithi098 systemd[1]: Stopping Ceph osd.7 for 565db2d8-fc7e-11ee-bc91-c7b262605968... 2024-04-17T06:14:45.835 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 06:14:45 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-7[54731]: 2024-04-17T06:14:45.388+0000 7f1a0c90a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-17T06:14:45.836 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 06:14:45 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-7[54731]: 2024-04-17T06:14:45.388+0000 7f1a0c90a700 -1 osd.7 109 *** Got signal Terminated *** 2024-04-17T06:14:45.836 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 06:14:45 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-7[54731]: 2024-04-17T06:14:45.388+0000 7f1a0c90a700 -1 osd.7 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-17T06:14:45.836 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 06:14:45 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-7[54731]: 2024-04-17T06:14:45.837+0000 7f1a08efe700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:13:57.505129+0000 front 2024-04-17T06:13:57.505157+0000 (oldest deadline 2024-04-17T06:14:23.905794+0000) 2024-04-17T06:14:46.133 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 06:14:45 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-7[54731]: 2024-04-17T06:14:45.837+0000 7f1a08efe700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:09.707328+0000 front 2024-04-17T06:14:09.707119+0000 (oldest deadline 2024-04-17T06:14:35.007071+0000) 2024-04-17T06:14:46.133 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 06:14:45 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-7[54731]: 2024-04-17T06:14:45.837+0000 7f1a08efe700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-17T06:14:15.007691+0000 front 2024-04-17T06:14:15.007814+0000 (oldest deadline 2024-04-17T06:14:39.707750+0000) 2024-04-17T06:14:46.134 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:46 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:46.021+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:53.138058+0000 front 2024-04-17T06:13:53.138083+0000 (oldest deadline 2024-04-17T06:14:14.238150+0000) 2024-04-17T06:14:46.134 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:46 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:46.021+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:13:58.939213+0000 front 2024-04-17T06:13:58.939286+0000 (oldest deadline 2024-04-17T06:14:24.839434+0000) 2024-04-17T06:14:46.134 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:46 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:46.021+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:07.040744+0000 front 2024-04-17T06:14:07.040770+0000 (oldest deadline 2024-04-17T06:14:30.441549+0000) 2024-04-17T06:14:46.134 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:46 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:46.021+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-17T06:14:15.743014+0000 front 2024-04-17T06:14:15.742885+0000 (oldest deadline 2024-04-17T06:14:38.642710+0000) 2024-04-17T06:14:46.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:45 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:45.759+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.354382+0000 front 2024-04-17T06:13:52.354774+0000 (oldest deadline 2024-04-17T06:14:15.254487+0000) 2024-04-17T06:14:46.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:45 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:45.759+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:01.655772+0000 front 2024-04-17T06:14:01.655755+0000 (oldest deadline 2024-04-17T06:14:27.555801+0000) 2024-04-17T06:14:46.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:45 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:45.759+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:09.257128+0000 front 2024-04-17T06:14:09.257345+0000 (oldest deadline 2024-04-17T06:14:31.557233+0000) 2024-04-17T06:14:46.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:45 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:45.759+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-17T06:14:17.958812+0000 front 2024-04-17T06:14:17.958952+0000 (oldest deadline 2024-04-17T06:14:42.658979+0000) 2024-04-17T06:14:46.284 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:45 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:45.941+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.989762+0000 front 2024-04-17T06:13:52.989781+0000 (oldest deadline 2024-04-17T06:14:18.289957+0000) 2024-04-17T06:14:46.284 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:45 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:45.941+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:01.191270+0000 front 2024-04-17T06:14:01.191086+0000 (oldest deadline 2024-04-17T06:14:24.690981+0000) 2024-04-17T06:14:46.284 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:45 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:45.941+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:09.292281+0000 front 2024-04-17T06:14:09.292431+0000 (oldest deadline 2024-04-17T06:14:35.192525+0000) 2024-04-17T06:14:46.284 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 06:14:46 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-8[54941]: 2024-04-17T06:14:46.045+0000 7f9c2d8b8700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:00.597947+0000 front 2024-04-17T06:14:00.598005+0000 (oldest deadline 2024-04-17T06:14:24.698026+0000) 2024-04-17T06:14:46.284 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 06:14:46 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-8[54941]: 2024-04-17T06:14:46.045+0000 7f9c2d8b8700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:06.998880+0000 front 2024-04-17T06:14:06.998908+0000 (oldest deadline 2024-04-17T06:14:32.899054+0000) 2024-04-17T06:14:46.284 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 06:14:46 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-8[54941]: 2024-04-17T06:14:46.045+0000 7f9c2d8b8700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-17T06:14:12.899673+0000 front 2024-04-17T06:14:12.899701+0000 (oldest deadline 2024-04-17T06:14:38.799720+0000) 2024-04-17T06:14:47.133 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 06:14:46 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-7[54731]: 2024-04-17T06:14:46.857+0000 7f1a08efe700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:13:57.505129+0000 front 2024-04-17T06:13:57.505157+0000 (oldest deadline 2024-04-17T06:14:23.905794+0000) 2024-04-17T06:14:47.133 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 06:14:46 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-7[54731]: 2024-04-17T06:14:46.857+0000 7f1a08efe700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:09.707328+0000 front 2024-04-17T06:14:09.707119+0000 (oldest deadline 2024-04-17T06:14:35.007071+0000) 2024-04-17T06:14:47.133 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 06:14:46 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-7[54731]: 2024-04-17T06:14:46.857+0000 7f1a08efe700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-17T06:14:15.007691+0000 front 2024-04-17T06:14:15.007814+0000 (oldest deadline 2024-04-17T06:14:39.707750+0000) 2024-04-17T06:14:47.133 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:47 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:47.046+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:53.138058+0000 front 2024-04-17T06:13:53.138083+0000 (oldest deadline 2024-04-17T06:14:14.238150+0000) 2024-04-17T06:14:47.133 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:47 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:47.046+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:13:58.939213+0000 front 2024-04-17T06:13:58.939286+0000 (oldest deadline 2024-04-17T06:14:24.839434+0000) 2024-04-17T06:14:47.133 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:47 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:47.046+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:07.040744+0000 front 2024-04-17T06:14:07.040770+0000 (oldest deadline 2024-04-17T06:14:30.441549+0000) 2024-04-17T06:14:47.134 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:47 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:47.046+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-17T06:14:15.743014+0000 front 2024-04-17T06:14:15.742885+0000 (oldest deadline 2024-04-17T06:14:38.642710+0000) 2024-04-17T06:14:47.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:46 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:46.801+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.354382+0000 front 2024-04-17T06:13:52.354774+0000 (oldest deadline 2024-04-17T06:14:15.254487+0000) 2024-04-17T06:14:47.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:46 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:46.801+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:01.655772+0000 front 2024-04-17T06:14:01.655755+0000 (oldest deadline 2024-04-17T06:14:27.555801+0000) 2024-04-17T06:14:47.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:46 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:46.801+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:09.257128+0000 front 2024-04-17T06:14:09.257345+0000 (oldest deadline 2024-04-17T06:14:31.557233+0000) 2024-04-17T06:14:47.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:46 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:46.801+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-17T06:14:17.958812+0000 front 2024-04-17T06:14:17.958952+0000 (oldest deadline 2024-04-17T06:14:42.658979+0000) 2024-04-17T06:14:47.284 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 06:14:47 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-8[54941]: 2024-04-17T06:14:47.006+0000 7f9c2d8b8700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:00.597947+0000 front 2024-04-17T06:14:00.598005+0000 (oldest deadline 2024-04-17T06:14:24.698026+0000) 2024-04-17T06:14:47.284 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 06:14:47 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-8[54941]: 2024-04-17T06:14:47.006+0000 7f9c2d8b8700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:06.998880+0000 front 2024-04-17T06:14:06.998908+0000 (oldest deadline 2024-04-17T06:14:32.899054+0000) 2024-04-17T06:14:47.284 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 06:14:47 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-8[54941]: 2024-04-17T06:14:47.006+0000 7f9c2d8b8700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-17T06:14:12.899673+0000 front 2024-04-17T06:14:12.899701+0000 (oldest deadline 2024-04-17T06:14:38.799720+0000) 2024-04-17T06:14:47.284 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:46 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:46.907+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.989762+0000 front 2024-04-17T06:13:52.989781+0000 (oldest deadline 2024-04-17T06:14:18.289957+0000) 2024-04-17T06:14:47.284 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:46 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:46.907+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:01.191270+0000 front 2024-04-17T06:14:01.191086+0000 (oldest deadline 2024-04-17T06:14:24.690981+0000) 2024-04-17T06:14:47.284 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:46 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:46.907+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:09.292281+0000 front 2024-04-17T06:14:09.292431+0000 (oldest deadline 2024-04-17T06:14:35.192525+0000) 2024-04-17T06:14:47.285 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:46 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:46.907+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T06:14:23.294796+0000 front 2024-04-17T06:14:23.294673+0000 (oldest deadline 2024-04-17T06:14:46.695089+0000) 2024-04-17T06:14:48.133 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 06:14:47 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-7[54731]: 2024-04-17T06:14:47.811+0000 7f1a08efe700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:13:57.505129+0000 front 2024-04-17T06:13:57.505157+0000 (oldest deadline 2024-04-17T06:14:23.905794+0000) 2024-04-17T06:14:48.133 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 06:14:47 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-7[54731]: 2024-04-17T06:14:47.811+0000 7f1a08efe700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:09.707328+0000 front 2024-04-17T06:14:09.707119+0000 (oldest deadline 2024-04-17T06:14:35.007071+0000) 2024-04-17T06:14:48.133 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 06:14:47 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-7[54731]: 2024-04-17T06:14:47.811+0000 7f1a08efe700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-17T06:14:15.007691+0000 front 2024-04-17T06:14:15.007814+0000 (oldest deadline 2024-04-17T06:14:39.707750+0000) 2024-04-17T06:14:48.133 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:48 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:48.009+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:53.138058+0000 front 2024-04-17T06:13:53.138083+0000 (oldest deadline 2024-04-17T06:14:14.238150+0000) 2024-04-17T06:14:48.133 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:48 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:48.009+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:13:58.939213+0000 front 2024-04-17T06:13:58.939286+0000 (oldest deadline 2024-04-17T06:14:24.839434+0000) 2024-04-17T06:14:48.134 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:48 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:48.009+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:07.040744+0000 front 2024-04-17T06:14:07.040770+0000 (oldest deadline 2024-04-17T06:14:30.441549+0000) 2024-04-17T06:14:48.134 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:48 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:48.009+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-17T06:14:15.743014+0000 front 2024-04-17T06:14:15.742885+0000 (oldest deadline 2024-04-17T06:14:38.642710+0000) 2024-04-17T06:14:48.143 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:47 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:47.771+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.354382+0000 front 2024-04-17T06:13:52.354774+0000 (oldest deadline 2024-04-17T06:14:15.254487+0000) 2024-04-17T06:14:48.144 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:47 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:47.771+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:01.655772+0000 front 2024-04-17T06:14:01.655755+0000 (oldest deadline 2024-04-17T06:14:27.555801+0000) 2024-04-17T06:14:48.144 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:47 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:47.771+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:09.257128+0000 front 2024-04-17T06:14:09.257345+0000 (oldest deadline 2024-04-17T06:14:31.557233+0000) 2024-04-17T06:14:48.144 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:47 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:47.771+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-17T06:14:17.958812+0000 front 2024-04-17T06:14:17.958952+0000 (oldest deadline 2024-04-17T06:14:42.658979+0000) 2024-04-17T06:14:48.144 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:47 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:47.771+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T06:14:22.659666+0000 front 2024-04-17T06:14:22.659556+0000 (oldest deadline 2024-04-17T06:14:47.359461+0000) 2024-04-17T06:14:48.284 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:47 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:47.888+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.989762+0000 front 2024-04-17T06:13:52.989781+0000 (oldest deadline 2024-04-17T06:14:18.289957+0000) 2024-04-17T06:14:48.284 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:47 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:47.888+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:01.191270+0000 front 2024-04-17T06:14:01.191086+0000 (oldest deadline 2024-04-17T06:14:24.690981+0000) 2024-04-17T06:14:48.284 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:47 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:47.888+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:09.292281+0000 front 2024-04-17T06:14:09.292431+0000 (oldest deadline 2024-04-17T06:14:35.192525+0000) 2024-04-17T06:14:48.284 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:47 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:47.888+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T06:14:23.294796+0000 front 2024-04-17T06:14:23.294673+0000 (oldest deadline 2024-04-17T06:14:46.695089+0000) 2024-04-17T06:14:48.284 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 06:14:48 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-8[54941]: 2024-04-17T06:14:48.017+0000 7f9c2d8b8700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:00.597947+0000 front 2024-04-17T06:14:00.598005+0000 (oldest deadline 2024-04-17T06:14:24.698026+0000) 2024-04-17T06:14:48.284 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 06:14:48 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-8[54941]: 2024-04-17T06:14:48.017+0000 7f9c2d8b8700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:06.998880+0000 front 2024-04-17T06:14:06.998908+0000 (oldest deadline 2024-04-17T06:14:32.899054+0000) 2024-04-17T06:14:48.284 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 06:14:48 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-8[54941]: 2024-04-17T06:14:48.017+0000 7f9c2d8b8700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-17T06:14:12.899673+0000 front 2024-04-17T06:14:12.899701+0000 (oldest deadline 2024-04-17T06:14:38.799720+0000) 2024-04-17T06:14:48.284 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 06:14:48 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-8[54941]: 2024-04-17T06:14:48.017+0000 7f9c2d8b8700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T06:14:24.501648+0000 front 2024-04-17T06:14:24.501798+0000 (oldest deadline 2024-04-17T06:14:48.001492+0000) 2024-04-17T06:14:49.133 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 06:14:48 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-7[54731]: 2024-04-17T06:14:48.856+0000 7f1a08efe700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:13:57.505129+0000 front 2024-04-17T06:13:57.505157+0000 (oldest deadline 2024-04-17T06:14:23.905794+0000) 2024-04-17T06:14:49.133 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 06:14:48 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-7[54731]: 2024-04-17T06:14:48.856+0000 7f1a08efe700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:09.707328+0000 front 2024-04-17T06:14:09.707119+0000 (oldest deadline 2024-04-17T06:14:35.007071+0000) 2024-04-17T06:14:49.133 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 06:14:48 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-7[54731]: 2024-04-17T06:14:48.856+0000 7f1a08efe700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-17T06:14:15.007691+0000 front 2024-04-17T06:14:15.007814+0000 (oldest deadline 2024-04-17T06:14:39.707750+0000) 2024-04-17T06:14:49.133 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 06:14:48 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-7[54731]: 2024-04-17T06:14:48.856+0000 7f1a08efe700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T06:14:22.509319+0000 front 2024-04-17T06:14:22.509336+0000 (oldest deadline 2024-04-17T06:14:48.309769+0000) 2024-04-17T06:14:49.134 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:49 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:49.057+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:53.138058+0000 front 2024-04-17T06:13:53.138083+0000 (oldest deadline 2024-04-17T06:14:14.238150+0000) 2024-04-17T06:14:49.134 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:49 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:49.057+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:13:58.939213+0000 front 2024-04-17T06:13:58.939286+0000 (oldest deadline 2024-04-17T06:14:24.839434+0000) 2024-04-17T06:14:49.134 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:49 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:49.057+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:07.040744+0000 front 2024-04-17T06:14:07.040770+0000 (oldest deadline 2024-04-17T06:14:30.441549+0000) 2024-04-17T06:14:49.134 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:49 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:49.057+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-17T06:14:15.743014+0000 front 2024-04-17T06:14:15.742885+0000 (oldest deadline 2024-04-17T06:14:38.642710+0000) 2024-04-17T06:14:49.134 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:49 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:49.057+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T06:14:23.343598+0000 front 2024-04-17T06:14:23.343627+0000 (oldest deadline 2024-04-17T06:14:48.043713+0000) 2024-04-17T06:14:49.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:48 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:48.744+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.354382+0000 front 2024-04-17T06:13:52.354774+0000 (oldest deadline 2024-04-17T06:14:15.254487+0000) 2024-04-17T06:14:49.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:48 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:48.744+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:01.655772+0000 front 2024-04-17T06:14:01.655755+0000 (oldest deadline 2024-04-17T06:14:27.555801+0000) 2024-04-17T06:14:49.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:48 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:48.744+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:09.257128+0000 front 2024-04-17T06:14:09.257345+0000 (oldest deadline 2024-04-17T06:14:31.557233+0000) 2024-04-17T06:14:49.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:48 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:48.744+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-17T06:14:17.958812+0000 front 2024-04-17T06:14:17.958952+0000 (oldest deadline 2024-04-17T06:14:42.658979+0000) 2024-04-17T06:14:49.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:48 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:48.744+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T06:14:22.659666+0000 front 2024-04-17T06:14:22.659556+0000 (oldest deadline 2024-04-17T06:14:47.359461+0000) 2024-04-17T06:14:49.284 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:48 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:48.881+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.989762+0000 front 2024-04-17T06:13:52.989781+0000 (oldest deadline 2024-04-17T06:14:18.289957+0000) 2024-04-17T06:14:49.284 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:48 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:48.881+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:01.191270+0000 front 2024-04-17T06:14:01.191086+0000 (oldest deadline 2024-04-17T06:14:24.690981+0000) 2024-04-17T06:14:49.284 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:48 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:48.881+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:09.292281+0000 front 2024-04-17T06:14:09.292431+0000 (oldest deadline 2024-04-17T06:14:35.192525+0000) 2024-04-17T06:14:49.284 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:48 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:48.881+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T06:14:23.294796+0000 front 2024-04-17T06:14:23.294673+0000 (oldest deadline 2024-04-17T06:14:46.695089+0000) 2024-04-17T06:14:49.284 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 06:14:49 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-8[54941]: 2024-04-17T06:14:48.998+0000 7f9c2d8b8700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:00.597947+0000 front 2024-04-17T06:14:00.598005+0000 (oldest deadline 2024-04-17T06:14:24.698026+0000) 2024-04-17T06:14:49.284 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 06:14:49 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-8[54941]: 2024-04-17T06:14:48.998+0000 7f9c2d8b8700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:06.998880+0000 front 2024-04-17T06:14:06.998908+0000 (oldest deadline 2024-04-17T06:14:32.899054+0000) 2024-04-17T06:14:49.284 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 06:14:49 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-8[54941]: 2024-04-17T06:14:48.998+0000 7f9c2d8b8700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-17T06:14:12.899673+0000 front 2024-04-17T06:14:12.899701+0000 (oldest deadline 2024-04-17T06:14:38.799720+0000) 2024-04-17T06:14:49.285 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 06:14:49 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-8[54941]: 2024-04-17T06:14:48.998+0000 7f9c2d8b8700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T06:14:24.501648+0000 front 2024-04-17T06:14:24.501798+0000 (oldest deadline 2024-04-17T06:14:48.001492+0000) 2024-04-17T06:14:50.133 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 06:14:49 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-7[54731]: 2024-04-17T06:14:49.862+0000 7f1a08efe700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:13:57.505129+0000 front 2024-04-17T06:13:57.505157+0000 (oldest deadline 2024-04-17T06:14:23.905794+0000) 2024-04-17T06:14:50.133 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 06:14:49 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-7[54731]: 2024-04-17T06:14:49.862+0000 7f1a08efe700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:09.707328+0000 front 2024-04-17T06:14:09.707119+0000 (oldest deadline 2024-04-17T06:14:35.007071+0000) 2024-04-17T06:14:50.133 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 06:14:49 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-7[54731]: 2024-04-17T06:14:49.862+0000 7f1a08efe700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-17T06:14:15.007691+0000 front 2024-04-17T06:14:15.007814+0000 (oldest deadline 2024-04-17T06:14:39.707750+0000) 2024-04-17T06:14:50.133 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 06:14:49 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-7[54731]: 2024-04-17T06:14:49.862+0000 7f1a08efe700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T06:14:22.509319+0000 front 2024-04-17T06:14:22.509336+0000 (oldest deadline 2024-04-17T06:14:48.309769+0000) 2024-04-17T06:14:50.133 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:50 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:50.062+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:53.138058+0000 front 2024-04-17T06:13:53.138083+0000 (oldest deadline 2024-04-17T06:14:14.238150+0000) 2024-04-17T06:14:50.133 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:50 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:50.062+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:13:58.939213+0000 front 2024-04-17T06:13:58.939286+0000 (oldest deadline 2024-04-17T06:14:24.839434+0000) 2024-04-17T06:14:50.133 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:50 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:50.062+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:07.040744+0000 front 2024-04-17T06:14:07.040770+0000 (oldest deadline 2024-04-17T06:14:30.441549+0000) 2024-04-17T06:14:50.134 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:50 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:50.062+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-17T06:14:15.743014+0000 front 2024-04-17T06:14:15.742885+0000 (oldest deadline 2024-04-17T06:14:38.642710+0000) 2024-04-17T06:14:50.134 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:50 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:50.062+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T06:14:23.343598+0000 front 2024-04-17T06:14:23.343627+0000 (oldest deadline 2024-04-17T06:14:48.043713+0000) 2024-04-17T06:14:50.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:49 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:49.709+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.354382+0000 front 2024-04-17T06:13:52.354774+0000 (oldest deadline 2024-04-17T06:14:15.254487+0000) 2024-04-17T06:14:50.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:49 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:49.709+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:01.655772+0000 front 2024-04-17T06:14:01.655755+0000 (oldest deadline 2024-04-17T06:14:27.555801+0000) 2024-04-17T06:14:50.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:49 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:49.709+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:09.257128+0000 front 2024-04-17T06:14:09.257345+0000 (oldest deadline 2024-04-17T06:14:31.557233+0000) 2024-04-17T06:14:50.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:49 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:49.709+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-17T06:14:17.958812+0000 front 2024-04-17T06:14:17.958952+0000 (oldest deadline 2024-04-17T06:14:42.658979+0000) 2024-04-17T06:14:50.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:49 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:49.709+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T06:14:22.659666+0000 front 2024-04-17T06:14:22.659556+0000 (oldest deadline 2024-04-17T06:14:47.359461+0000) 2024-04-17T06:14:50.284 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:49 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:49.889+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.989762+0000 front 2024-04-17T06:13:52.989781+0000 (oldest deadline 2024-04-17T06:14:18.289957+0000) 2024-04-17T06:14:50.284 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:49 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:49.889+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:01.191270+0000 front 2024-04-17T06:14:01.191086+0000 (oldest deadline 2024-04-17T06:14:24.690981+0000) 2024-04-17T06:14:50.284 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:49 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:49.889+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:09.292281+0000 front 2024-04-17T06:14:09.292431+0000 (oldest deadline 2024-04-17T06:14:35.192525+0000) 2024-04-17T06:14:50.284 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:49 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:49.889+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T06:14:23.294796+0000 front 2024-04-17T06:14:23.294673+0000 (oldest deadline 2024-04-17T06:14:46.695089+0000) 2024-04-17T06:14:50.284 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 06:14:49 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-8[54941]: 2024-04-17T06:14:49.963+0000 7f9c2d8b8700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:00.597947+0000 front 2024-04-17T06:14:00.598005+0000 (oldest deadline 2024-04-17T06:14:24.698026+0000) 2024-04-17T06:14:50.285 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 06:14:49 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-8[54941]: 2024-04-17T06:14:49.963+0000 7f9c2d8b8700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:06.998880+0000 front 2024-04-17T06:14:06.998908+0000 (oldest deadline 2024-04-17T06:14:32.899054+0000) 2024-04-17T06:14:50.285 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 06:14:49 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-8[54941]: 2024-04-17T06:14:49.963+0000 7f9c2d8b8700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-17T06:14:12.899673+0000 front 2024-04-17T06:14:12.899701+0000 (oldest deadline 2024-04-17T06:14:38.799720+0000) 2024-04-17T06:14:50.285 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 06:14:49 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-8[54941]: 2024-04-17T06:14:49.963+0000 7f9c2d8b8700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T06:14:24.501648+0000 front 2024-04-17T06:14:24.501798+0000 (oldest deadline 2024-04-17T06:14:48.001492+0000) 2024-04-17T06:14:51.070 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 06:14:50 smithi098 podman[74844]: 2024-04-17 06:14:50.821754382 +0000 UTC m=+5.532365844 container died 2dbd1aae384dec8b7570377cddb8fee570e9e6781ec50b570ce17ad20d39e19d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-7, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-04-17T06:14:51.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:50 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:50.720+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.354382+0000 front 2024-04-17T06:13:52.354774+0000 (oldest deadline 2024-04-17T06:14:15.254487+0000) 2024-04-17T06:14:51.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:50 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:50.720+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:01.655772+0000 front 2024-04-17T06:14:01.655755+0000 (oldest deadline 2024-04-17T06:14:27.555801+0000) 2024-04-17T06:14:51.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:50 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:50.720+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:09.257128+0000 front 2024-04-17T06:14:09.257345+0000 (oldest deadline 2024-04-17T06:14:31.557233+0000) 2024-04-17T06:14:51.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:50 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:50.720+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-17T06:14:17.958812+0000 front 2024-04-17T06:14:17.958952+0000 (oldest deadline 2024-04-17T06:14:42.658979+0000) 2024-04-17T06:14:51.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:50 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:50.720+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T06:14:22.659666+0000 front 2024-04-17T06:14:22.659556+0000 (oldest deadline 2024-04-17T06:14:47.359461+0000) 2024-04-17T06:14:51.284 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 06:14:50 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-8[54941]: 2024-04-17T06:14:50.959+0000 7f9c2d8b8700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:00.597947+0000 front 2024-04-17T06:14:00.598005+0000 (oldest deadline 2024-04-17T06:14:24.698026+0000) 2024-04-17T06:14:51.284 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 06:14:50 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-8[54941]: 2024-04-17T06:14:50.959+0000 7f9c2d8b8700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:06.998880+0000 front 2024-04-17T06:14:06.998908+0000 (oldest deadline 2024-04-17T06:14:32.899054+0000) 2024-04-17T06:14:51.284 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 06:14:50 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-8[54941]: 2024-04-17T06:14:50.959+0000 7f9c2d8b8700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-17T06:14:12.899673+0000 front 2024-04-17T06:14:12.899701+0000 (oldest deadline 2024-04-17T06:14:38.799720+0000) 2024-04-17T06:14:51.284 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 06:14:50 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-8[54941]: 2024-04-17T06:14:50.959+0000 7f9c2d8b8700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T06:14:24.501648+0000 front 2024-04-17T06:14:24.501798+0000 (oldest deadline 2024-04-17T06:14:48.001492+0000) 2024-04-17T06:14:51.284 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:50 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:50.852+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.989762+0000 front 2024-04-17T06:13:52.989781+0000 (oldest deadline 2024-04-17T06:14:18.289957+0000) 2024-04-17T06:14:51.284 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:50 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:50.852+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:01.191270+0000 front 2024-04-17T06:14:01.191086+0000 (oldest deadline 2024-04-17T06:14:24.690981+0000) 2024-04-17T06:14:51.284 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:50 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:50.852+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:09.292281+0000 front 2024-04-17T06:14:09.292431+0000 (oldest deadline 2024-04-17T06:14:35.192525+0000) 2024-04-17T06:14:51.285 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:50 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:50.852+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T06:14:23.294796+0000 front 2024-04-17T06:14:23.294673+0000 (oldest deadline 2024-04-17T06:14:46.695089+0000) 2024-04-17T06:14:51.354 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 06:14:51 smithi098 podman[74844]: 2024-04-17 06:14:51.164183733 +0000 UTC m=+5.874795187 container cleanup 2dbd1aae384dec8b7570377cddb8fee570e9e6781ec50b570ce17ad20d39e19d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-7, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-04-17T06:14:51.355 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 06:14:51 smithi098 bash[74844]: ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-7 2024-04-17T06:14:51.355 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:51 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:51.072+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:53.138058+0000 front 2024-04-17T06:13:53.138083+0000 (oldest deadline 2024-04-17T06:14:14.238150+0000) 2024-04-17T06:14:51.355 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:51 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:51.072+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:13:58.939213+0000 front 2024-04-17T06:13:58.939286+0000 (oldest deadline 2024-04-17T06:14:24.839434+0000) 2024-04-17T06:14:51.355 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:51 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:51.072+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:07.040744+0000 front 2024-04-17T06:14:07.040770+0000 (oldest deadline 2024-04-17T06:14:30.441549+0000) 2024-04-17T06:14:51.355 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:51 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:51.072+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-17T06:14:15.743014+0000 front 2024-04-17T06:14:15.742885+0000 (oldest deadline 2024-04-17T06:14:38.642710+0000) 2024-04-17T06:14:51.355 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:51 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:51.072+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T06:14:23.343598+0000 front 2024-04-17T06:14:23.343627+0000 (oldest deadline 2024-04-17T06:14:48.043713+0000) 2024-04-17T06:14:51.633 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 06:14:51 smithi098 podman[74862]: 2024-04-17 06:14:51.357166902 +0000 UTC m=+0.533963274 container remove 2dbd1aae384dec8b7570377cddb8fee570e9e6781ec50b570ce17ad20d39e19d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-7, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True) 2024-04-17T06:14:52.019 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 06:14:51 smithi098 podman[74954]: 2024-04-17 06:14:51.667094445 +0000 UTC m=+0.017526393 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-17T06:14:52.019 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 06:14:51 smithi098 podman[74954]: 2024-04-17 06:14:51.791624239 +0000 UTC m=+0.142056187 container create a7f03d051e0f2326d7760a4e1f38882e0dbe8371ab2d66e3775a98f3fd397c73 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-17T06:14:52.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:51 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:51.719+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.354382+0000 front 2024-04-17T06:13:52.354774+0000 (oldest deadline 2024-04-17T06:14:15.254487+0000) 2024-04-17T06:14:52.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:51 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:51.719+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:01.655772+0000 front 2024-04-17T06:14:01.655755+0000 (oldest deadline 2024-04-17T06:14:27.555801+0000) 2024-04-17T06:14:52.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:51 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:51.719+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:09.257128+0000 front 2024-04-17T06:14:09.257345+0000 (oldest deadline 2024-04-17T06:14:31.557233+0000) 2024-04-17T06:14:52.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:51 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:51.719+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-17T06:14:17.958812+0000 front 2024-04-17T06:14:17.958952+0000 (oldest deadline 2024-04-17T06:14:42.658979+0000) 2024-04-17T06:14:52.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:51 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:51.719+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T06:14:22.659666+0000 front 2024-04-17T06:14:22.659556+0000 (oldest deadline 2024-04-17T06:14:47.359461+0000) 2024-04-17T06:14:52.284 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 06:14:51 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-8[54941]: 2024-04-17T06:14:51.940+0000 7f9c2d8b8700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:00.597947+0000 front 2024-04-17T06:14:00.598005+0000 (oldest deadline 2024-04-17T06:14:24.698026+0000) 2024-04-17T06:14:52.284 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 06:14:51 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-8[54941]: 2024-04-17T06:14:51.940+0000 7f9c2d8b8700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:06.998880+0000 front 2024-04-17T06:14:06.998908+0000 (oldest deadline 2024-04-17T06:14:32.899054+0000) 2024-04-17T06:14:52.285 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 06:14:51 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-8[54941]: 2024-04-17T06:14:51.940+0000 7f9c2d8b8700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-17T06:14:12.899673+0000 front 2024-04-17T06:14:12.899701+0000 (oldest deadline 2024-04-17T06:14:38.799720+0000) 2024-04-17T06:14:52.285 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 06:14:51 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-8[54941]: 2024-04-17T06:14:51.940+0000 7f9c2d8b8700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T06:14:24.501648+0000 front 2024-04-17T06:14:24.501798+0000 (oldest deadline 2024-04-17T06:14:48.001492+0000) 2024-04-17T06:14:52.285 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:51 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:51.854+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.989762+0000 front 2024-04-17T06:13:52.989781+0000 (oldest deadline 2024-04-17T06:14:18.289957+0000) 2024-04-17T06:14:52.285 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:51 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:51.854+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:01.191270+0000 front 2024-04-17T06:14:01.191086+0000 (oldest deadline 2024-04-17T06:14:24.690981+0000) 2024-04-17T06:14:52.285 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:51 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:51.854+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:09.292281+0000 front 2024-04-17T06:14:09.292431+0000 (oldest deadline 2024-04-17T06:14:35.192525+0000) 2024-04-17T06:14:52.285 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:51 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:51.854+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T06:14:23.294796+0000 front 2024-04-17T06:14:23.294673+0000 (oldest deadline 2024-04-17T06:14:46.695089+0000) 2024-04-17T06:14:52.307 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 06:14:52 smithi098 podman[74954]: 2024-04-17 06:14:52.021269326 +0000 UTC m=+0.371701273 container init a7f03d051e0f2326d7760a4e1f38882e0dbe8371ab2d66e3775a98f3fd397c73 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-7-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-17T06:14:52.308 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 06:14:52 smithi098 podman[74954]: 2024-04-17 06:14:52.024708501 +0000 UTC m=+0.375140452 container start a7f03d051e0f2326d7760a4e1f38882e0dbe8371ab2d66e3775a98f3fd397c73 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-7-deactivate, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-17T06:14:52.308 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 06:14:52 smithi098 podman[74954]: 2024-04-17 06:14:52.129714384 +0000 UTC m=+0.480146352 container attach a7f03d051e0f2326d7760a4e1f38882e0dbe8371ab2d66e3775a98f3fd397c73 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-7-deactivate, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-17T06:14:52.308 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:52 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:52.030+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:53.138058+0000 front 2024-04-17T06:13:53.138083+0000 (oldest deadline 2024-04-17T06:14:14.238150+0000) 2024-04-17T06:14:52.308 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:52 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:52.030+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:13:58.939213+0000 front 2024-04-17T06:13:58.939286+0000 (oldest deadline 2024-04-17T06:14:24.839434+0000) 2024-04-17T06:14:52.308 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:52 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:52.030+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:07.040744+0000 front 2024-04-17T06:14:07.040770+0000 (oldest deadline 2024-04-17T06:14:30.441549+0000) 2024-04-17T06:14:52.308 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:52 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:52.030+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-17T06:14:15.743014+0000 front 2024-04-17T06:14:15.742885+0000 (oldest deadline 2024-04-17T06:14:38.642710+0000) 2024-04-17T06:14:52.308 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:52 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:52.030+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T06:14:23.343598+0000 front 2024-04-17T06:14:23.343627+0000 (oldest deadline 2024-04-17T06:14:48.043713+0000) 2024-04-17T06:14:52.633 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 06:14:52 smithi098 podman[74954]: 2024-04-17 06:14:52.310467028 +0000 UTC m=+0.660899004 container died a7f03d051e0f2326d7760a4e1f38882e0dbe8371ab2d66e3775a98f3fd397c73 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-7-deactivate, ceph=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-04-17T06:14:52.941 DEBUG:teuthology.orchestra.run.smithi098:> sudo pkill -f 'journalctl -f -n 0 -u ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@osd.7.service' 2024-04-17T06:14:53.024 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-17T06:14:53.024 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-04-17T06:14:53.024 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-04-17T06:14:53.024 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@osd.8 2024-04-17T06:14:53.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:52 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:52.698+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.354382+0000 front 2024-04-17T06:13:52.354774+0000 (oldest deadline 2024-04-17T06:14:15.254487+0000) 2024-04-17T06:14:53.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:52 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:52.698+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:01.655772+0000 front 2024-04-17T06:14:01.655755+0000 (oldest deadline 2024-04-17T06:14:27.555801+0000) 2024-04-17T06:14:53.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:52 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:52.698+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:09.257128+0000 front 2024-04-17T06:14:09.257345+0000 (oldest deadline 2024-04-17T06:14:31.557233+0000) 2024-04-17T06:14:53.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:52 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:52.698+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-17T06:14:17.958812+0000 front 2024-04-17T06:14:17.958952+0000 (oldest deadline 2024-04-17T06:14:42.658979+0000) 2024-04-17T06:14:53.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:52 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:52.698+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T06:14:22.659666+0000 front 2024-04-17T06:14:22.659556+0000 (oldest deadline 2024-04-17T06:14:47.359461+0000) 2024-04-17T06:14:53.284 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:52 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:52.884+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.989762+0000 front 2024-04-17T06:13:52.989781+0000 (oldest deadline 2024-04-17T06:14:18.289957+0000) 2024-04-17T06:14:53.284 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:52 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:52.884+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:01.191270+0000 front 2024-04-17T06:14:01.191086+0000 (oldest deadline 2024-04-17T06:14:24.690981+0000) 2024-04-17T06:14:53.284 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:52 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:52.884+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:09.292281+0000 front 2024-04-17T06:14:09.292431+0000 (oldest deadline 2024-04-17T06:14:35.192525+0000) 2024-04-17T06:14:53.284 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:52 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:52.884+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T06:14:23.294796+0000 front 2024-04-17T06:14:23.294673+0000 (oldest deadline 2024-04-17T06:14:46.695089+0000) 2024-04-17T06:14:53.284 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 06:14:52 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-8[54941]: 2024-04-17T06:14:52.972+0000 7f9c2d8b8700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:00.597947+0000 front 2024-04-17T06:14:00.598005+0000 (oldest deadline 2024-04-17T06:14:24.698026+0000) 2024-04-17T06:14:53.284 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 06:14:52 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-8[54941]: 2024-04-17T06:14:52.972+0000 7f9c2d8b8700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:06.998880+0000 front 2024-04-17T06:14:06.998908+0000 (oldest deadline 2024-04-17T06:14:32.899054+0000) 2024-04-17T06:14:53.284 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 06:14:52 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-8[54941]: 2024-04-17T06:14:52.972+0000 7f9c2d8b8700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-17T06:14:12.899673+0000 front 2024-04-17T06:14:12.899701+0000 (oldest deadline 2024-04-17T06:14:38.799720+0000) 2024-04-17T06:14:53.285 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 06:14:52 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-8[54941]: 2024-04-17T06:14:52.972+0000 7f9c2d8b8700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T06:14:24.501648+0000 front 2024-04-17T06:14:24.501798+0000 (oldest deadline 2024-04-17T06:14:48.001492+0000) 2024-04-17T06:14:53.285 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 06:14:53 smithi103 systemd[1]: Stopping Ceph osd.8 for 565db2d8-fc7e-11ee-bc91-c7b262605968... 2024-04-17T06:14:53.383 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:53 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:53.052+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:53.138058+0000 front 2024-04-17T06:13:53.138083+0000 (oldest deadline 2024-04-17T06:14:14.238150+0000) 2024-04-17T06:14:53.383 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:53 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:53.052+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:13:58.939213+0000 front 2024-04-17T06:13:58.939286+0000 (oldest deadline 2024-04-17T06:14:24.839434+0000) 2024-04-17T06:14:53.383 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:53 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:53.052+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:07.040744+0000 front 2024-04-17T06:14:07.040770+0000 (oldest deadline 2024-04-17T06:14:30.441549+0000) 2024-04-17T06:14:53.383 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:53 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:53.052+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-17T06:14:15.743014+0000 front 2024-04-17T06:14:15.742885+0000 (oldest deadline 2024-04-17T06:14:38.642710+0000) 2024-04-17T06:14:53.383 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:53 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:53.052+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T06:14:23.343598+0000 front 2024-04-17T06:14:23.343627+0000 (oldest deadline 2024-04-17T06:14:48.043713+0000) 2024-04-17T06:14:53.784 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 06:14:53 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-8[54941]: 2024-04-17T06:14:53.340+0000 7f9c312c4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-17T06:14:53.784 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 06:14:53 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-8[54941]: 2024-04-17T06:14:53.340+0000 7f9c312c4700 -1 osd.8 109 *** Got signal Terminated *** 2024-04-17T06:14:53.784 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 06:14:53 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-8[54941]: 2024-04-17T06:14:53.340+0000 7f9c312c4700 -1 osd.8 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-17T06:14:54.040 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:53 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:53.698+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.354382+0000 front 2024-04-17T06:13:52.354774+0000 (oldest deadline 2024-04-17T06:14:15.254487+0000) 2024-04-17T06:14:54.040 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:53 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:53.698+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:01.655772+0000 front 2024-04-17T06:14:01.655755+0000 (oldest deadline 2024-04-17T06:14:27.555801+0000) 2024-04-17T06:14:54.041 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:53 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:53.698+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:09.257128+0000 front 2024-04-17T06:14:09.257345+0000 (oldest deadline 2024-04-17T06:14:31.557233+0000) 2024-04-17T06:14:54.041 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:53 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:53.698+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-17T06:14:17.958812+0000 front 2024-04-17T06:14:17.958952+0000 (oldest deadline 2024-04-17T06:14:42.658979+0000) 2024-04-17T06:14:54.041 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:53 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:53.698+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T06:14:22.659666+0000 front 2024-04-17T06:14:22.659556+0000 (oldest deadline 2024-04-17T06:14:47.359461+0000) 2024-04-17T06:14:54.284 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 06:14:53 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-8[54941]: 2024-04-17T06:14:53.924+0000 7f9c2d8b8700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:00.597947+0000 front 2024-04-17T06:14:00.598005+0000 (oldest deadline 2024-04-17T06:14:24.698026+0000) 2024-04-17T06:14:54.284 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 06:14:53 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-8[54941]: 2024-04-17T06:14:53.924+0000 7f9c2d8b8700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:06.998880+0000 front 2024-04-17T06:14:06.998908+0000 (oldest deadline 2024-04-17T06:14:32.899054+0000) 2024-04-17T06:14:54.284 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 06:14:53 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-8[54941]: 2024-04-17T06:14:53.924+0000 7f9c2d8b8700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-17T06:14:12.899673+0000 front 2024-04-17T06:14:12.899701+0000 (oldest deadline 2024-04-17T06:14:38.799720+0000) 2024-04-17T06:14:54.284 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 06:14:53 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-8[54941]: 2024-04-17T06:14:53.924+0000 7f9c2d8b8700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T06:14:24.501648+0000 front 2024-04-17T06:14:24.501798+0000 (oldest deadline 2024-04-17T06:14:48.001492+0000) 2024-04-17T06:14:54.284 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:53 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:53.844+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.989762+0000 front 2024-04-17T06:13:52.989781+0000 (oldest deadline 2024-04-17T06:14:18.289957+0000) 2024-04-17T06:14:54.285 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:53 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:53.844+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:01.191270+0000 front 2024-04-17T06:14:01.191086+0000 (oldest deadline 2024-04-17T06:14:24.690981+0000) 2024-04-17T06:14:54.285 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:53 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:53.844+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:09.292281+0000 front 2024-04-17T06:14:09.292431+0000 (oldest deadline 2024-04-17T06:14:35.192525+0000) 2024-04-17T06:14:54.285 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:53 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:53.844+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T06:14:23.294796+0000 front 2024-04-17T06:14:23.294673+0000 (oldest deadline 2024-04-17T06:14:46.695089+0000) 2024-04-17T06:14:54.383 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:54 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:54.100+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:53.138058+0000 front 2024-04-17T06:13:53.138083+0000 (oldest deadline 2024-04-17T06:14:14.238150+0000) 2024-04-17T06:14:54.383 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:54 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:54.100+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:13:58.939213+0000 front 2024-04-17T06:13:58.939286+0000 (oldest deadline 2024-04-17T06:14:24.839434+0000) 2024-04-17T06:14:54.383 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:54 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:54.100+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:07.040744+0000 front 2024-04-17T06:14:07.040770+0000 (oldest deadline 2024-04-17T06:14:30.441549+0000) 2024-04-17T06:14:54.384 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:54 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:54.100+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-17T06:14:15.743014+0000 front 2024-04-17T06:14:15.742885+0000 (oldest deadline 2024-04-17T06:14:38.642710+0000) 2024-04-17T06:14:54.384 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:54 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:54.100+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T06:14:23.343598+0000 front 2024-04-17T06:14:23.343627+0000 (oldest deadline 2024-04-17T06:14:48.043713+0000) 2024-04-17T06:14:55.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:54 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:54.738+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.354382+0000 front 2024-04-17T06:13:52.354774+0000 (oldest deadline 2024-04-17T06:14:15.254487+0000) 2024-04-17T06:14:55.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:54 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:54.738+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:01.655772+0000 front 2024-04-17T06:14:01.655755+0000 (oldest deadline 2024-04-17T06:14:27.555801+0000) 2024-04-17T06:14:55.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:54 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:54.738+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:09.257128+0000 front 2024-04-17T06:14:09.257345+0000 (oldest deadline 2024-04-17T06:14:31.557233+0000) 2024-04-17T06:14:55.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:54 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:54.738+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-17T06:14:17.958812+0000 front 2024-04-17T06:14:17.958952+0000 (oldest deadline 2024-04-17T06:14:42.658979+0000) 2024-04-17T06:14:55.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:54 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:54.738+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T06:14:22.659666+0000 front 2024-04-17T06:14:22.659556+0000 (oldest deadline 2024-04-17T06:14:47.359461+0000) 2024-04-17T06:14:55.284 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 06:14:54 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-8[54941]: 2024-04-17T06:14:54.901+0000 7f9c2d8b8700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:00.597947+0000 front 2024-04-17T06:14:00.598005+0000 (oldest deadline 2024-04-17T06:14:24.698026+0000) 2024-04-17T06:14:55.284 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 06:14:54 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-8[54941]: 2024-04-17T06:14:54.901+0000 7f9c2d8b8700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:06.998880+0000 front 2024-04-17T06:14:06.998908+0000 (oldest deadline 2024-04-17T06:14:32.899054+0000) 2024-04-17T06:14:55.284 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 06:14:54 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-8[54941]: 2024-04-17T06:14:54.901+0000 7f9c2d8b8700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-17T06:14:12.899673+0000 front 2024-04-17T06:14:12.899701+0000 (oldest deadline 2024-04-17T06:14:38.799720+0000) 2024-04-17T06:14:55.284 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 06:14:54 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-8[54941]: 2024-04-17T06:14:54.901+0000 7f9c2d8b8700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T06:14:24.501648+0000 front 2024-04-17T06:14:24.501798+0000 (oldest deadline 2024-04-17T06:14:48.001492+0000) 2024-04-17T06:14:55.285 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:54 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:54.827+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.989762+0000 front 2024-04-17T06:13:52.989781+0000 (oldest deadline 2024-04-17T06:14:18.289957+0000) 2024-04-17T06:14:55.285 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:54 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:54.827+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:01.191270+0000 front 2024-04-17T06:14:01.191086+0000 (oldest deadline 2024-04-17T06:14:24.690981+0000) 2024-04-17T06:14:55.285 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:54 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:54.827+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:09.292281+0000 front 2024-04-17T06:14:09.292431+0000 (oldest deadline 2024-04-17T06:14:35.192525+0000) 2024-04-17T06:14:55.285 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:54 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:54.827+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T06:14:23.294796+0000 front 2024-04-17T06:14:23.294673+0000 (oldest deadline 2024-04-17T06:14:46.695089+0000) 2024-04-17T06:14:55.383 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:55 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:55.084+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:53.138058+0000 front 2024-04-17T06:13:53.138083+0000 (oldest deadline 2024-04-17T06:14:14.238150+0000) 2024-04-17T06:14:55.383 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:55 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:55.084+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:13:58.939213+0000 front 2024-04-17T06:13:58.939286+0000 (oldest deadline 2024-04-17T06:14:24.839434+0000) 2024-04-17T06:14:55.383 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:55 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:55.084+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:07.040744+0000 front 2024-04-17T06:14:07.040770+0000 (oldest deadline 2024-04-17T06:14:30.441549+0000) 2024-04-17T06:14:55.383 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:55 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:55.084+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-17T06:14:15.743014+0000 front 2024-04-17T06:14:15.742885+0000 (oldest deadline 2024-04-17T06:14:38.642710+0000) 2024-04-17T06:14:55.383 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:55 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:55.084+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T06:14:23.343598+0000 front 2024-04-17T06:14:23.343627+0000 (oldest deadline 2024-04-17T06:14:48.043713+0000) 2024-04-17T06:14:55.383 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:55 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:55.084+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T06:14:29.144854+0000 front 2024-04-17T06:14:29.144791+0000 (oldest deadline 2024-04-17T06:14:55.044695+0000) 2024-04-17T06:14:56.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:55 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:55.723+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.354382+0000 front 2024-04-17T06:13:52.354774+0000 (oldest deadline 2024-04-17T06:14:15.254487+0000) 2024-04-17T06:14:56.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:55 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:55.723+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:01.655772+0000 front 2024-04-17T06:14:01.655755+0000 (oldest deadline 2024-04-17T06:14:27.555801+0000) 2024-04-17T06:14:56.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:55 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:55.723+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:09.257128+0000 front 2024-04-17T06:14:09.257345+0000 (oldest deadline 2024-04-17T06:14:31.557233+0000) 2024-04-17T06:14:56.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:55 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:55.723+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-17T06:14:17.958812+0000 front 2024-04-17T06:14:17.958952+0000 (oldest deadline 2024-04-17T06:14:42.658979+0000) 2024-04-17T06:14:56.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:55 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:55.723+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T06:14:22.659666+0000 front 2024-04-17T06:14:22.659556+0000 (oldest deadline 2024-04-17T06:14:47.359461+0000) 2024-04-17T06:14:56.284 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 06:14:55 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-8[54941]: 2024-04-17T06:14:55.911+0000 7f9c2d8b8700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:00.597947+0000 front 2024-04-17T06:14:00.598005+0000 (oldest deadline 2024-04-17T06:14:24.698026+0000) 2024-04-17T06:14:56.284 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 06:14:55 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-8[54941]: 2024-04-17T06:14:55.911+0000 7f9c2d8b8700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:06.998880+0000 front 2024-04-17T06:14:06.998908+0000 (oldest deadline 2024-04-17T06:14:32.899054+0000) 2024-04-17T06:14:56.284 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 06:14:55 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-8[54941]: 2024-04-17T06:14:55.911+0000 7f9c2d8b8700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-17T06:14:12.899673+0000 front 2024-04-17T06:14:12.899701+0000 (oldest deadline 2024-04-17T06:14:38.799720+0000) 2024-04-17T06:14:56.284 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 06:14:55 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-8[54941]: 2024-04-17T06:14:55.911+0000 7f9c2d8b8700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T06:14:24.501648+0000 front 2024-04-17T06:14:24.501798+0000 (oldest deadline 2024-04-17T06:14:48.001492+0000) 2024-04-17T06:14:56.284 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:55 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:55.798+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.989762+0000 front 2024-04-17T06:13:52.989781+0000 (oldest deadline 2024-04-17T06:14:18.289957+0000) 2024-04-17T06:14:56.285 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:55 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:55.798+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:01.191270+0000 front 2024-04-17T06:14:01.191086+0000 (oldest deadline 2024-04-17T06:14:24.690981+0000) 2024-04-17T06:14:56.285 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:55 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:55.798+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:09.292281+0000 front 2024-04-17T06:14:09.292431+0000 (oldest deadline 2024-04-17T06:14:35.192525+0000) 2024-04-17T06:14:56.285 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:55 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:55.798+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T06:14:23.294796+0000 front 2024-04-17T06:14:23.294673+0000 (oldest deadline 2024-04-17T06:14:46.695089+0000) 2024-04-17T06:14:56.285 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:55 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:55.798+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T06:14:30.696345+0000 front 2024-04-17T06:14:30.696394+0000 (oldest deadline 2024-04-17T06:14:55.396489+0000) 2024-04-17T06:14:56.383 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:56 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:56.067+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:53.138058+0000 front 2024-04-17T06:13:53.138083+0000 (oldest deadline 2024-04-17T06:14:14.238150+0000) 2024-04-17T06:14:56.383 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:56 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:56.067+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:13:58.939213+0000 front 2024-04-17T06:13:58.939286+0000 (oldest deadline 2024-04-17T06:14:24.839434+0000) 2024-04-17T06:14:56.383 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:56 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:56.067+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:07.040744+0000 front 2024-04-17T06:14:07.040770+0000 (oldest deadline 2024-04-17T06:14:30.441549+0000) 2024-04-17T06:14:56.383 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:56 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:56.067+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-17T06:14:15.743014+0000 front 2024-04-17T06:14:15.742885+0000 (oldest deadline 2024-04-17T06:14:38.642710+0000) 2024-04-17T06:14:56.383 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:56 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:56.067+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T06:14:23.343598+0000 front 2024-04-17T06:14:23.343627+0000 (oldest deadline 2024-04-17T06:14:48.043713+0000) 2024-04-17T06:14:56.383 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:56 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:56.067+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T06:14:29.144854+0000 front 2024-04-17T06:14:29.144791+0000 (oldest deadline 2024-04-17T06:14:55.044695+0000) 2024-04-17T06:14:57.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:56 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:56.726+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.354382+0000 front 2024-04-17T06:13:52.354774+0000 (oldest deadline 2024-04-17T06:14:15.254487+0000) 2024-04-17T06:14:57.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:56 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:56.726+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:01.655772+0000 front 2024-04-17T06:14:01.655755+0000 (oldest deadline 2024-04-17T06:14:27.555801+0000) 2024-04-17T06:14:57.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:56 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:56.726+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:09.257128+0000 front 2024-04-17T06:14:09.257345+0000 (oldest deadline 2024-04-17T06:14:31.557233+0000) 2024-04-17T06:14:57.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:56 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:56.726+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-17T06:14:17.958812+0000 front 2024-04-17T06:14:17.958952+0000 (oldest deadline 2024-04-17T06:14:42.658979+0000) 2024-04-17T06:14:57.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:56 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:56.726+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T06:14:22.659666+0000 front 2024-04-17T06:14:22.659556+0000 (oldest deadline 2024-04-17T06:14:47.359461+0000) 2024-04-17T06:14:57.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:56 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:56.726+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T06:14:31.361344+0000 front 2024-04-17T06:14:31.361395+0000 (oldest deadline 2024-04-17T06:14:56.661258+0000) 2024-04-17T06:14:57.284 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 06:14:56 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-8[54941]: 2024-04-17T06:14:56.886+0000 7f9c2d8b8700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:00.597947+0000 front 2024-04-17T06:14:00.598005+0000 (oldest deadline 2024-04-17T06:14:24.698026+0000) 2024-04-17T06:14:57.284 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 06:14:56 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-8[54941]: 2024-04-17T06:14:56.886+0000 7f9c2d8b8700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:06.998880+0000 front 2024-04-17T06:14:06.998908+0000 (oldest deadline 2024-04-17T06:14:32.899054+0000) 2024-04-17T06:14:57.284 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 06:14:56 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-8[54941]: 2024-04-17T06:14:56.886+0000 7f9c2d8b8700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-17T06:14:12.899673+0000 front 2024-04-17T06:14:12.899701+0000 (oldest deadline 2024-04-17T06:14:38.799720+0000) 2024-04-17T06:14:57.284 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 06:14:56 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-8[54941]: 2024-04-17T06:14:56.886+0000 7f9c2d8b8700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T06:14:24.501648+0000 front 2024-04-17T06:14:24.501798+0000 (oldest deadline 2024-04-17T06:14:48.001492+0000) 2024-04-17T06:14:57.284 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 06:14:56 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-8[54941]: 2024-04-17T06:14:56.886+0000 7f9c2d8b8700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T06:14:33.902551+0000 front 2024-04-17T06:14:33.902576+0000 (oldest deadline 2024-04-17T06:14:56.202619+0000) 2024-04-17T06:14:57.284 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:56 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:56.803+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.989762+0000 front 2024-04-17T06:13:52.989781+0000 (oldest deadline 2024-04-17T06:14:18.289957+0000) 2024-04-17T06:14:57.284 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:56 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:56.803+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:01.191270+0000 front 2024-04-17T06:14:01.191086+0000 (oldest deadline 2024-04-17T06:14:24.690981+0000) 2024-04-17T06:14:57.285 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:56 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:56.803+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:09.292281+0000 front 2024-04-17T06:14:09.292431+0000 (oldest deadline 2024-04-17T06:14:35.192525+0000) 2024-04-17T06:14:57.285 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:56 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:56.803+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T06:14:23.294796+0000 front 2024-04-17T06:14:23.294673+0000 (oldest deadline 2024-04-17T06:14:46.695089+0000) 2024-04-17T06:14:57.285 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:56 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:56.803+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T06:14:30.696345+0000 front 2024-04-17T06:14:30.696394+0000 (oldest deadline 2024-04-17T06:14:55.396489+0000) 2024-04-17T06:14:57.383 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:57 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:57.085+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:53.138058+0000 front 2024-04-17T06:13:53.138083+0000 (oldest deadline 2024-04-17T06:14:14.238150+0000) 2024-04-17T06:14:57.383 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:57 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:57.085+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:13:58.939213+0000 front 2024-04-17T06:13:58.939286+0000 (oldest deadline 2024-04-17T06:14:24.839434+0000) 2024-04-17T06:14:57.383 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:57 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:57.085+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:07.040744+0000 front 2024-04-17T06:14:07.040770+0000 (oldest deadline 2024-04-17T06:14:30.441549+0000) 2024-04-17T06:14:57.383 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:57 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:57.085+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-17T06:14:15.743014+0000 front 2024-04-17T06:14:15.742885+0000 (oldest deadline 2024-04-17T06:14:38.642710+0000) 2024-04-17T06:14:57.383 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:57 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:57.085+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T06:14:23.343598+0000 front 2024-04-17T06:14:23.343627+0000 (oldest deadline 2024-04-17T06:14:48.043713+0000) 2024-04-17T06:14:57.383 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:57 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:57.085+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T06:14:29.144854+0000 front 2024-04-17T06:14:29.144791+0000 (oldest deadline 2024-04-17T06:14:55.044695+0000) 2024-04-17T06:14:58.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:57 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:57.748+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.354382+0000 front 2024-04-17T06:13:52.354774+0000 (oldest deadline 2024-04-17T06:14:15.254487+0000) 2024-04-17T06:14:58.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:57 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:57.748+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:01.655772+0000 front 2024-04-17T06:14:01.655755+0000 (oldest deadline 2024-04-17T06:14:27.555801+0000) 2024-04-17T06:14:58.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:57 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:57.748+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:09.257128+0000 front 2024-04-17T06:14:09.257345+0000 (oldest deadline 2024-04-17T06:14:31.557233+0000) 2024-04-17T06:14:58.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:57 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:57.748+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-17T06:14:17.958812+0000 front 2024-04-17T06:14:17.958952+0000 (oldest deadline 2024-04-17T06:14:42.658979+0000) 2024-04-17T06:14:58.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:57 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:57.748+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T06:14:22.659666+0000 front 2024-04-17T06:14:22.659556+0000 (oldest deadline 2024-04-17T06:14:47.359461+0000) 2024-04-17T06:14:58.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:57 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:57.748+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T06:14:31.361344+0000 front 2024-04-17T06:14:31.361395+0000 (oldest deadline 2024-04-17T06:14:56.661258+0000) 2024-04-17T06:14:58.284 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 06:14:57 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-8[54941]: 2024-04-17T06:14:57.868+0000 7f9c2d8b8700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:00.597947+0000 front 2024-04-17T06:14:00.598005+0000 (oldest deadline 2024-04-17T06:14:24.698026+0000) 2024-04-17T06:14:58.284 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 06:14:57 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-8[54941]: 2024-04-17T06:14:57.868+0000 7f9c2d8b8700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:06.998880+0000 front 2024-04-17T06:14:06.998908+0000 (oldest deadline 2024-04-17T06:14:32.899054+0000) 2024-04-17T06:14:58.284 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 06:14:57 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-8[54941]: 2024-04-17T06:14:57.868+0000 7f9c2d8b8700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-17T06:14:12.899673+0000 front 2024-04-17T06:14:12.899701+0000 (oldest deadline 2024-04-17T06:14:38.799720+0000) 2024-04-17T06:14:58.284 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 06:14:57 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-8[54941]: 2024-04-17T06:14:57.868+0000 7f9c2d8b8700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T06:14:24.501648+0000 front 2024-04-17T06:14:24.501798+0000 (oldest deadline 2024-04-17T06:14:48.001492+0000) 2024-04-17T06:14:58.284 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 06:14:57 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-8[54941]: 2024-04-17T06:14:57.868+0000 7f9c2d8b8700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T06:14:33.902551+0000 front 2024-04-17T06:14:33.902576+0000 (oldest deadline 2024-04-17T06:14:56.202619+0000) 2024-04-17T06:14:58.284 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:57 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:57.826+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.989762+0000 front 2024-04-17T06:13:52.989781+0000 (oldest deadline 2024-04-17T06:14:18.289957+0000) 2024-04-17T06:14:58.285 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:57 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:57.826+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:01.191270+0000 front 2024-04-17T06:14:01.191086+0000 (oldest deadline 2024-04-17T06:14:24.690981+0000) 2024-04-17T06:14:58.285 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:57 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:57.826+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:09.292281+0000 front 2024-04-17T06:14:09.292431+0000 (oldest deadline 2024-04-17T06:14:35.192525+0000) 2024-04-17T06:14:58.285 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:57 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:57.826+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T06:14:23.294796+0000 front 2024-04-17T06:14:23.294673+0000 (oldest deadline 2024-04-17T06:14:46.695089+0000) 2024-04-17T06:14:58.285 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:57 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:57.826+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T06:14:30.696345+0000 front 2024-04-17T06:14:30.696394+0000 (oldest deadline 2024-04-17T06:14:55.396489+0000) 2024-04-17T06:14:58.383 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:58 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:58.103+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:53.138058+0000 front 2024-04-17T06:13:53.138083+0000 (oldest deadline 2024-04-17T06:14:14.238150+0000) 2024-04-17T06:14:58.383 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:58 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:58.103+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:13:58.939213+0000 front 2024-04-17T06:13:58.939286+0000 (oldest deadline 2024-04-17T06:14:24.839434+0000) 2024-04-17T06:14:58.383 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:58 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:58.103+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:07.040744+0000 front 2024-04-17T06:14:07.040770+0000 (oldest deadline 2024-04-17T06:14:30.441549+0000) 2024-04-17T06:14:58.383 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:58 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:58.103+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-17T06:14:15.743014+0000 front 2024-04-17T06:14:15.742885+0000 (oldest deadline 2024-04-17T06:14:38.642710+0000) 2024-04-17T06:14:58.383 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:58 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:58.103+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T06:14:23.343598+0000 front 2024-04-17T06:14:23.343627+0000 (oldest deadline 2024-04-17T06:14:48.043713+0000) 2024-04-17T06:14:58.383 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:58 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:58.103+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T06:14:29.144854+0000 front 2024-04-17T06:14:29.144791+0000 (oldest deadline 2024-04-17T06:14:55.044695+0000) 2024-04-17T06:14:58.964 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 06:14:58 smithi103 podman[74025]: 2024-04-17 06:14:58.683871429 +0000 UTC m=+5.468665066 container died 9186747c0e9e99d9a423da898ad16d9f322f2705ac6f3c5017cf1429d05e9d5b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-8, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-04-17T06:14:58.964 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:58 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:58.800+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.989762+0000 front 2024-04-17T06:13:52.989781+0000 (oldest deadline 2024-04-17T06:14:18.289957+0000) 2024-04-17T06:14:58.964 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:58 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:58.800+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:01.191270+0000 front 2024-04-17T06:14:01.191086+0000 (oldest deadline 2024-04-17T06:14:24.690981+0000) 2024-04-17T06:14:58.964 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:58 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:58.800+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:09.292281+0000 front 2024-04-17T06:14:09.292431+0000 (oldest deadline 2024-04-17T06:14:35.192525+0000) 2024-04-17T06:14:58.964 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:58 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:58.800+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T06:14:23.294796+0000 front 2024-04-17T06:14:23.294673+0000 (oldest deadline 2024-04-17T06:14:46.695089+0000) 2024-04-17T06:14:58.964 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:58 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:58.800+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T06:14:30.696345+0000 front 2024-04-17T06:14:30.696394+0000 (oldest deadline 2024-04-17T06:14:55.396489+0000) 2024-04-17T06:14:59.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:58 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:58.721+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.354382+0000 front 2024-04-17T06:13:52.354774+0000 (oldest deadline 2024-04-17T06:14:15.254487+0000) 2024-04-17T06:14:59.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:58 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:58.721+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:01.655772+0000 front 2024-04-17T06:14:01.655755+0000 (oldest deadline 2024-04-17T06:14:27.555801+0000) 2024-04-17T06:14:59.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:58 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:58.721+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:09.257128+0000 front 2024-04-17T06:14:09.257345+0000 (oldest deadline 2024-04-17T06:14:31.557233+0000) 2024-04-17T06:14:59.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:58 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:58.722+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-17T06:14:17.958812+0000 front 2024-04-17T06:14:17.958952+0000 (oldest deadline 2024-04-17T06:14:42.658979+0000) 2024-04-17T06:14:59.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:58 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:58.722+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T06:14:22.659666+0000 front 2024-04-17T06:14:22.659556+0000 (oldest deadline 2024-04-17T06:14:47.359461+0000) 2024-04-17T06:14:59.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:58 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:58.722+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T06:14:31.361344+0000 front 2024-04-17T06:14:31.361395+0000 (oldest deadline 2024-04-17T06:14:56.661258+0000) 2024-04-17T06:14:59.284 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 06:14:59 smithi103 podman[74025]: 2024-04-17 06:14:59.057377351 +0000 UTC m=+5.842170977 container cleanup 9186747c0e9e99d9a423da898ad16d9f322f2705ac6f3c5017cf1429d05e9d5b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-8, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True) 2024-04-17T06:14:59.284 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 06:14:59 smithi103 bash[74025]: ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-8 2024-04-17T06:14:59.615 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 06:14:59 smithi103 podman[74042]: 2024-04-17 06:14:59.316961485 +0000 UTC m=+0.632616105 container remove 9186747c0e9e99d9a423da898ad16d9f322f2705ac6f3c5017cf1429d05e9d5b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-8, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-17T06:14:59.633 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:59 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:59.136+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:53.138058+0000 front 2024-04-17T06:13:53.138083+0000 (oldest deadline 2024-04-17T06:14:14.238150+0000) 2024-04-17T06:14:59.633 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:59 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:59.136+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:13:58.939213+0000 front 2024-04-17T06:13:58.939286+0000 (oldest deadline 2024-04-17T06:14:24.839434+0000) 2024-04-17T06:14:59.633 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:59 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:59.136+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:07.040744+0000 front 2024-04-17T06:14:07.040770+0000 (oldest deadline 2024-04-17T06:14:30.441549+0000) 2024-04-17T06:14:59.633 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:59 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:59.136+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-17T06:14:15.743014+0000 front 2024-04-17T06:14:15.742885+0000 (oldest deadline 2024-04-17T06:14:38.642710+0000) 2024-04-17T06:14:59.633 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:59 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:59.136+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T06:14:23.343598+0000 front 2024-04-17T06:14:23.343627+0000 (oldest deadline 2024-04-17T06:14:48.043713+0000) 2024-04-17T06:14:59.633 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:14:59 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:14:59.136+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T06:14:29.144854+0000 front 2024-04-17T06:14:29.144791+0000 (oldest deadline 2024-04-17T06:14:55.044695+0000) 2024-04-17T06:14:59.885 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 06:14:59 smithi103 podman[74133]: 2024-04-17 06:14:59.618068012 +0000 UTC m=+0.111310074 container create d7afbe3dbf1457fd886ab830e5c4dbcf89aee0ff340d18741981910fb51a1a88 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-8-deactivate, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-17T06:14:59.885 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 06:14:59 smithi103 podman[74133]: 2024-04-17 06:14:59.524482122 +0000 UTC m=+0.017724183 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-17T06:14:59.885 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:59 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:59.841+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.989762+0000 front 2024-04-17T06:13:52.989781+0000 (oldest deadline 2024-04-17T06:14:18.289957+0000) 2024-04-17T06:14:59.885 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:59 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:59.841+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:01.191270+0000 front 2024-04-17T06:14:01.191086+0000 (oldest deadline 2024-04-17T06:14:24.690981+0000) 2024-04-17T06:14:59.886 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:59 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:59.841+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:09.292281+0000 front 2024-04-17T06:14:09.292431+0000 (oldest deadline 2024-04-17T06:14:35.192525+0000) 2024-04-17T06:14:59.886 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:59 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:59.841+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T06:14:23.294796+0000 front 2024-04-17T06:14:23.294673+0000 (oldest deadline 2024-04-17T06:14:46.695089+0000) 2024-04-17T06:14:59.886 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:14:59 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:14:59.841+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T06:14:30.696345+0000 front 2024-04-17T06:14:30.696394+0000 (oldest deadline 2024-04-17T06:14:55.396489+0000) 2024-04-17T06:15:00.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:59 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:59.686+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.354382+0000 front 2024-04-17T06:13:52.354774+0000 (oldest deadline 2024-04-17T06:14:15.254487+0000) 2024-04-17T06:15:00.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:59 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:59.686+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:01.655772+0000 front 2024-04-17T06:14:01.655755+0000 (oldest deadline 2024-04-17T06:14:27.555801+0000) 2024-04-17T06:15:00.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:59 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:59.686+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:09.257128+0000 front 2024-04-17T06:14:09.257345+0000 (oldest deadline 2024-04-17T06:14:31.557233+0000) 2024-04-17T06:15:00.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:59 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:59.686+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-17T06:14:17.958812+0000 front 2024-04-17T06:14:17.958952+0000 (oldest deadline 2024-04-17T06:14:42.658979+0000) 2024-04-17T06:15:00.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:59 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:59.686+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T06:14:22.659666+0000 front 2024-04-17T06:14:22.659556+0000 (oldest deadline 2024-04-17T06:14:47.359461+0000) 2024-04-17T06:15:00.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:14:59 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:14:59.686+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T06:14:31.361344+0000 front 2024-04-17T06:14:31.361395+0000 (oldest deadline 2024-04-17T06:14:56.661258+0000) 2024-04-17T06:15:00.181 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 06:14:59 smithi103 podman[74133]: 2024-04-17 06:14:59.887578548 +0000 UTC m=+0.380820591 container init d7afbe3dbf1457fd886ab830e5c4dbcf89aee0ff340d18741981910fb51a1a88 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-8-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.schema-version=1.0) 2024-04-17T06:15:00.182 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 06:14:59 smithi103 podman[74133]: 2024-04-17 06:14:59.891484926 +0000 UTC m=+0.384726978 container start d7afbe3dbf1457fd886ab830e5c4dbcf89aee0ff340d18741981910fb51a1a88 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-8-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, ceph=True, org.label-schema.schema-version=1.0) 2024-04-17T06:15:00.182 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 06:15:00 smithi103 podman[74133]: 2024-04-17 06:15:00.01282006 +0000 UTC m=+0.506062121 container attach d7afbe3dbf1457fd886ab830e5c4dbcf89aee0ff340d18741981910fb51a1a88 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-8-deactivate, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-04-17T06:15:00.447 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 06:15:00 smithi103 podman[74133]: 2024-04-17 06:15:00.18395662 +0000 UTC m=+0.677198672 container died d7afbe3dbf1457fd886ab830e5c4dbcf89aee0ff340d18741981910fb51a1a88 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-8-deactivate, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-17T06:15:00.633 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:15:00 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:15:00.154+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:53.138058+0000 front 2024-04-17T06:13:53.138083+0000 (oldest deadline 2024-04-17T06:14:14.238150+0000) 2024-04-17T06:15:00.633 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:15:00 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:15:00.154+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:13:58.939213+0000 front 2024-04-17T06:13:58.939286+0000 (oldest deadline 2024-04-17T06:14:24.839434+0000) 2024-04-17T06:15:00.633 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:15:00 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:15:00.154+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:07.040744+0000 front 2024-04-17T06:14:07.040770+0000 (oldest deadline 2024-04-17T06:14:30.441549+0000) 2024-04-17T06:15:00.633 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:15:00 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:15:00.154+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-17T06:14:15.743014+0000 front 2024-04-17T06:14:15.742885+0000 (oldest deadline 2024-04-17T06:14:38.642710+0000) 2024-04-17T06:15:00.633 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:15:00 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:15:00.154+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T06:14:23.343598+0000 front 2024-04-17T06:14:23.343627+0000 (oldest deadline 2024-04-17T06:14:48.043713+0000) 2024-04-17T06:15:00.633 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:15:00 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:15:00.154+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T06:14:29.144854+0000 front 2024-04-17T06:14:29.144791+0000 (oldest deadline 2024-04-17T06:14:55.044695+0000) 2024-04-17T06:15:00.786 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@osd.8.service' 2024-04-17T06:15:00.827 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-17T06:15:00.827 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-04-17T06:15:00.827 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-04-17T06:15:00.827 DEBUG:teuthology.orchestra.run.smithi047:> sudo systemctl stop ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@osd.9 2024-04-17T06:15:01.034 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:00 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:00.869+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.989762+0000 front 2024-04-17T06:13:52.989781+0000 (oldest deadline 2024-04-17T06:14:18.289957+0000) 2024-04-17T06:15:01.034 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:00 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:00.869+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:01.191270+0000 front 2024-04-17T06:14:01.191086+0000 (oldest deadline 2024-04-17T06:14:24.690981+0000) 2024-04-17T06:15:01.034 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:00 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:00.869+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:09.292281+0000 front 2024-04-17T06:14:09.292431+0000 (oldest deadline 2024-04-17T06:14:35.192525+0000) 2024-04-17T06:15:01.034 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:00 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:00.869+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T06:14:23.294796+0000 front 2024-04-17T06:14:23.294673+0000 (oldest deadline 2024-04-17T06:14:46.695089+0000) 2024-04-17T06:15:01.034 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:00 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:00.869+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T06:14:30.696345+0000 front 2024-04-17T06:14:30.696394+0000 (oldest deadline 2024-04-17T06:14:55.396489+0000) 2024-04-17T06:15:01.126 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:15:00 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:15:00.664+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.354382+0000 front 2024-04-17T06:13:52.354774+0000 (oldest deadline 2024-04-17T06:14:15.254487+0000) 2024-04-17T06:15:01.126 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:15:00 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:15:00.664+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:01.655772+0000 front 2024-04-17T06:14:01.655755+0000 (oldest deadline 2024-04-17T06:14:27.555801+0000) 2024-04-17T06:15:01.126 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:15:00 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:15:00.664+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:09.257128+0000 front 2024-04-17T06:14:09.257345+0000 (oldest deadline 2024-04-17T06:14:31.557233+0000) 2024-04-17T06:15:01.126 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:15:00 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:15:00.664+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-17T06:14:17.958812+0000 front 2024-04-17T06:14:17.958952+0000 (oldest deadline 2024-04-17T06:14:42.658979+0000) 2024-04-17T06:15:01.126 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:15:00 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:15:00.664+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T06:14:22.659666+0000 front 2024-04-17T06:14:22.659556+0000 (oldest deadline 2024-04-17T06:14:47.359461+0000) 2024-04-17T06:15:01.126 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:15:00 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:15:00.664+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T06:14:31.361344+0000 front 2024-04-17T06:14:31.361395+0000 (oldest deadline 2024-04-17T06:14:56.661258+0000) 2024-04-17T06:15:01.126 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:15:00 smithi047 systemd[1]: Stopping Ceph osd.9 for 565db2d8-fc7e-11ee-bc91-c7b262605968... 2024-04-17T06:15:01.126 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:15:01 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:15:01.127+0000 7fd512589700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-17T06:15:01.126 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:15:01 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:15:01.127+0000 7fd512589700 -1 osd.9 109 *** Got signal Terminated *** 2024-04-17T06:15:01.389 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:15:01 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:15:01.127+0000 7fd512589700 -1 osd.9 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-17T06:15:01.633 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:15:01 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:15:01.201+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:53.138058+0000 front 2024-04-17T06:13:53.138083+0000 (oldest deadline 2024-04-17T06:14:14.238150+0000) 2024-04-17T06:15:01.633 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:15:01 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:15:01.201+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:13:58.939213+0000 front 2024-04-17T06:13:58.939286+0000 (oldest deadline 2024-04-17T06:14:24.839434+0000) 2024-04-17T06:15:01.633 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:15:01 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:15:01.201+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:07.040744+0000 front 2024-04-17T06:14:07.040770+0000 (oldest deadline 2024-04-17T06:14:30.441549+0000) 2024-04-17T06:15:01.633 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:15:01 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:15:01.201+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-17T06:14:15.743014+0000 front 2024-04-17T06:14:15.742885+0000 (oldest deadline 2024-04-17T06:14:38.642710+0000) 2024-04-17T06:15:01.633 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:15:01 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:15:01.201+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T06:14:23.343598+0000 front 2024-04-17T06:14:23.343627+0000 (oldest deadline 2024-04-17T06:14:48.043713+0000) 2024-04-17T06:15:01.633 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:15:01 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:15:01.201+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T06:14:29.144854+0000 front 2024-04-17T06:14:29.144791+0000 (oldest deadline 2024-04-17T06:14:55.044695+0000) 2024-04-17T06:15:02.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:15:01 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:15:01.711+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.354382+0000 front 2024-04-17T06:13:52.354774+0000 (oldest deadline 2024-04-17T06:14:15.254487+0000) 2024-04-17T06:15:02.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:15:01 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:15:01.711+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:01.655772+0000 front 2024-04-17T06:14:01.655755+0000 (oldest deadline 2024-04-17T06:14:27.555801+0000) 2024-04-17T06:15:02.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:15:01 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:15:01.711+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:09.257128+0000 front 2024-04-17T06:14:09.257345+0000 (oldest deadline 2024-04-17T06:14:31.557233+0000) 2024-04-17T06:15:02.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:15:01 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:15:01.711+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-17T06:14:17.958812+0000 front 2024-04-17T06:14:17.958952+0000 (oldest deadline 2024-04-17T06:14:42.658979+0000) 2024-04-17T06:15:02.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:15:01 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:15:01.711+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T06:14:22.659666+0000 front 2024-04-17T06:14:22.659556+0000 (oldest deadline 2024-04-17T06:14:47.359461+0000) 2024-04-17T06:15:02.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:15:01 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:15:01.711+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T06:14:31.361344+0000 front 2024-04-17T06:14:31.361395+0000 (oldest deadline 2024-04-17T06:14:56.661258+0000) 2024-04-17T06:15:02.284 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:01 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:01.828+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.989762+0000 front 2024-04-17T06:13:52.989781+0000 (oldest deadline 2024-04-17T06:14:18.289957+0000) 2024-04-17T06:15:02.284 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:01 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:01.828+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:01.191270+0000 front 2024-04-17T06:14:01.191086+0000 (oldest deadline 2024-04-17T06:14:24.690981+0000) 2024-04-17T06:15:02.284 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:01 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:01.828+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:09.292281+0000 front 2024-04-17T06:14:09.292431+0000 (oldest deadline 2024-04-17T06:14:35.192525+0000) 2024-04-17T06:15:02.284 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:01 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:01.828+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T06:14:23.294796+0000 front 2024-04-17T06:14:23.294673+0000 (oldest deadline 2024-04-17T06:14:46.695089+0000) 2024-04-17T06:15:02.284 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:01 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:01.828+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T06:14:30.696345+0000 front 2024-04-17T06:14:30.696394+0000 (oldest deadline 2024-04-17T06:14:55.396489+0000) 2024-04-17T06:15:02.633 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:15:02 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:15:02.230+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:53.138058+0000 front 2024-04-17T06:13:53.138083+0000 (oldest deadline 2024-04-17T06:14:14.238150+0000) 2024-04-17T06:15:02.633 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:15:02 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:15:02.230+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:13:58.939213+0000 front 2024-04-17T06:13:58.939286+0000 (oldest deadline 2024-04-17T06:14:24.839434+0000) 2024-04-17T06:15:02.633 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:15:02 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:15:02.230+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:07.040744+0000 front 2024-04-17T06:14:07.040770+0000 (oldest deadline 2024-04-17T06:14:30.441549+0000) 2024-04-17T06:15:02.633 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:15:02 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:15:02.230+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-17T06:14:15.743014+0000 front 2024-04-17T06:14:15.742885+0000 (oldest deadline 2024-04-17T06:14:38.642710+0000) 2024-04-17T06:15:02.633 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:15:02 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:15:02.230+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T06:14:23.343598+0000 front 2024-04-17T06:14:23.343627+0000 (oldest deadline 2024-04-17T06:14:48.043713+0000) 2024-04-17T06:15:02.633 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:15:02 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:15:02.230+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T06:14:29.144854+0000 front 2024-04-17T06:14:29.144791+0000 (oldest deadline 2024-04-17T06:14:55.044695+0000) 2024-04-17T06:15:03.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:15:02 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:15:02.684+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.354382+0000 front 2024-04-17T06:13:52.354774+0000 (oldest deadline 2024-04-17T06:14:15.254487+0000) 2024-04-17T06:15:03.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:15:02 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:15:02.685+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:01.655772+0000 front 2024-04-17T06:14:01.655755+0000 (oldest deadline 2024-04-17T06:14:27.555801+0000) 2024-04-17T06:15:03.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:15:02 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:15:02.685+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:09.257128+0000 front 2024-04-17T06:14:09.257345+0000 (oldest deadline 2024-04-17T06:14:31.557233+0000) 2024-04-17T06:15:03.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:15:02 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:15:02.685+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-17T06:14:17.958812+0000 front 2024-04-17T06:14:17.958952+0000 (oldest deadline 2024-04-17T06:14:42.658979+0000) 2024-04-17T06:15:03.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:15:02 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:15:02.685+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T06:14:22.659666+0000 front 2024-04-17T06:14:22.659556+0000 (oldest deadline 2024-04-17T06:14:47.359461+0000) 2024-04-17T06:15:03.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:15:02 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:15:02.685+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T06:14:31.361344+0000 front 2024-04-17T06:14:31.361395+0000 (oldest deadline 2024-04-17T06:14:56.661258+0000) 2024-04-17T06:15:03.284 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:02 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:02.828+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.989762+0000 front 2024-04-17T06:13:52.989781+0000 (oldest deadline 2024-04-17T06:14:18.289957+0000) 2024-04-17T06:15:03.284 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:02 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:02.828+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:01.191270+0000 front 2024-04-17T06:14:01.191086+0000 (oldest deadline 2024-04-17T06:14:24.690981+0000) 2024-04-17T06:15:03.284 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:02 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:02.828+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:09.292281+0000 front 2024-04-17T06:14:09.292431+0000 (oldest deadline 2024-04-17T06:14:35.192525+0000) 2024-04-17T06:15:03.284 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:02 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:02.828+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T06:14:23.294796+0000 front 2024-04-17T06:14:23.294673+0000 (oldest deadline 2024-04-17T06:14:46.695089+0000) 2024-04-17T06:15:03.284 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:02 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:02.828+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T06:14:30.696345+0000 front 2024-04-17T06:14:30.696394+0000 (oldest deadline 2024-04-17T06:14:55.396489+0000) 2024-04-17T06:15:03.534 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:15:03 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:15:03.216+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:53.138058+0000 front 2024-04-17T06:13:53.138083+0000 (oldest deadline 2024-04-17T06:14:14.238150+0000) 2024-04-17T06:15:03.535 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:15:03 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:15:03.216+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:13:58.939213+0000 front 2024-04-17T06:13:58.939286+0000 (oldest deadline 2024-04-17T06:14:24.839434+0000) 2024-04-17T06:15:03.535 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:15:03 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:15:03.216+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:07.040744+0000 front 2024-04-17T06:14:07.040770+0000 (oldest deadline 2024-04-17T06:14:30.441549+0000) 2024-04-17T06:15:03.535 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:15:03 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:15:03.216+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-17T06:14:15.743014+0000 front 2024-04-17T06:14:15.742885+0000 (oldest deadline 2024-04-17T06:14:38.642710+0000) 2024-04-17T06:15:03.535 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:15:03 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:15:03.216+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T06:14:23.343598+0000 front 2024-04-17T06:14:23.343627+0000 (oldest deadline 2024-04-17T06:14:48.043713+0000) 2024-04-17T06:15:03.535 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:15:03 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:15:03.216+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T06:14:29.144854+0000 front 2024-04-17T06:14:29.144791+0000 (oldest deadline 2024-04-17T06:14:55.044695+0000) 2024-04-17T06:15:04.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:15:03 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:15:03.668+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.354382+0000 front 2024-04-17T06:13:52.354774+0000 (oldest deadline 2024-04-17T06:14:15.254487+0000) 2024-04-17T06:15:04.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:15:03 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:15:03.668+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:01.655772+0000 front 2024-04-17T06:14:01.655755+0000 (oldest deadline 2024-04-17T06:14:27.555801+0000) 2024-04-17T06:15:04.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:15:03 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:15:03.668+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:09.257128+0000 front 2024-04-17T06:14:09.257345+0000 (oldest deadline 2024-04-17T06:14:31.557233+0000) 2024-04-17T06:15:04.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:15:03 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:15:03.668+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-17T06:14:17.958812+0000 front 2024-04-17T06:14:17.958952+0000 (oldest deadline 2024-04-17T06:14:42.658979+0000) 2024-04-17T06:15:04.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:15:03 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:15:03.668+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T06:14:22.659666+0000 front 2024-04-17T06:14:22.659556+0000 (oldest deadline 2024-04-17T06:14:47.359461+0000) 2024-04-17T06:15:04.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:15:03 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:15:03.668+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T06:14:31.361344+0000 front 2024-04-17T06:14:31.361395+0000 (oldest deadline 2024-04-17T06:14:56.661258+0000) 2024-04-17T06:15:04.139 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:15:03 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:15:03.668+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-04-17T06:14:41.362153+0000 front 2024-04-17T06:14:41.362187+0000 (oldest deadline 2024-04-17T06:15:03.562743+0000) 2024-04-17T06:15:04.185 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:03 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:03.828+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.989762+0000 front 2024-04-17T06:13:52.989781+0000 (oldest deadline 2024-04-17T06:14:18.289957+0000) 2024-04-17T06:15:04.185 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:03 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:03.828+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:01.191270+0000 front 2024-04-17T06:14:01.191086+0000 (oldest deadline 2024-04-17T06:14:24.690981+0000) 2024-04-17T06:15:04.185 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:03 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:03.828+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:09.292281+0000 front 2024-04-17T06:14:09.292431+0000 (oldest deadline 2024-04-17T06:14:35.192525+0000) 2024-04-17T06:15:04.185 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:03 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:03.828+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T06:14:23.294796+0000 front 2024-04-17T06:14:23.294673+0000 (oldest deadline 2024-04-17T06:14:46.695089+0000) 2024-04-17T06:15:04.185 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:03 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:03.828+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T06:14:30.696345+0000 front 2024-04-17T06:14:30.696394+0000 (oldest deadline 2024-04-17T06:14:55.396489+0000) 2024-04-17T06:15:04.548 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:15:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:15:04.233+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:53.138058+0000 front 2024-04-17T06:13:53.138083+0000 (oldest deadline 2024-04-17T06:14:14.238150+0000) 2024-04-17T06:15:04.548 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:15:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:15:04.233+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:13:58.939213+0000 front 2024-04-17T06:13:58.939286+0000 (oldest deadline 2024-04-17T06:14:24.839434+0000) 2024-04-17T06:15:04.548 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:15:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:15:04.233+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:07.040744+0000 front 2024-04-17T06:14:07.040770+0000 (oldest deadline 2024-04-17T06:14:30.441549+0000) 2024-04-17T06:15:04.548 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:15:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:15:04.233+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-17T06:14:15.743014+0000 front 2024-04-17T06:14:15.742885+0000 (oldest deadline 2024-04-17T06:14:38.642710+0000) 2024-04-17T06:15:04.548 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:15:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:15:04.233+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T06:14:23.343598+0000 front 2024-04-17T06:14:23.343627+0000 (oldest deadline 2024-04-17T06:14:48.043713+0000) 2024-04-17T06:15:04.548 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:15:04 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:15:04.233+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T06:14:29.144854+0000 front 2024-04-17T06:14:29.144791+0000 (oldest deadline 2024-04-17T06:14:55.044695+0000) 2024-04-17T06:15:04.889 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:15:04 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:15:04.626+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.354382+0000 front 2024-04-17T06:13:52.354774+0000 (oldest deadline 2024-04-17T06:14:15.254487+0000) 2024-04-17T06:15:04.889 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:15:04 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:15:04.626+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:01.655772+0000 front 2024-04-17T06:14:01.655755+0000 (oldest deadline 2024-04-17T06:14:27.555801+0000) 2024-04-17T06:15:04.889 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:15:04 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:15:04.626+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:09.257128+0000 front 2024-04-17T06:14:09.257345+0000 (oldest deadline 2024-04-17T06:14:31.557233+0000) 2024-04-17T06:15:04.889 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:15:04 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:15:04.626+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-17T06:14:17.958812+0000 front 2024-04-17T06:14:17.958952+0000 (oldest deadline 2024-04-17T06:14:42.658979+0000) 2024-04-17T06:15:04.889 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:15:04 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:15:04.626+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T06:14:22.659666+0000 front 2024-04-17T06:14:22.659556+0000 (oldest deadline 2024-04-17T06:14:47.359461+0000) 2024-04-17T06:15:04.889 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:15:04 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:15:04.626+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T06:14:31.361344+0000 front 2024-04-17T06:14:31.361395+0000 (oldest deadline 2024-04-17T06:14:56.661258+0000) 2024-04-17T06:15:04.890 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:15:04 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:15:04.626+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-04-17T06:14:41.362153+0000 front 2024-04-17T06:14:41.362187+0000 (oldest deadline 2024-04-17T06:15:03.562743+0000) 2024-04-17T06:15:05.284 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:04 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:04.841+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.989762+0000 front 2024-04-17T06:13:52.989781+0000 (oldest deadline 2024-04-17T06:14:18.289957+0000) 2024-04-17T06:15:05.284 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:04 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:04.841+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:01.191270+0000 front 2024-04-17T06:14:01.191086+0000 (oldest deadline 2024-04-17T06:14:24.690981+0000) 2024-04-17T06:15:05.284 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:04 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:04.841+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:09.292281+0000 front 2024-04-17T06:14:09.292431+0000 (oldest deadline 2024-04-17T06:14:35.192525+0000) 2024-04-17T06:15:05.284 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:04 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:04.841+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T06:14:23.294796+0000 front 2024-04-17T06:14:23.294673+0000 (oldest deadline 2024-04-17T06:14:46.695089+0000) 2024-04-17T06:15:05.284 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:04 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:04.841+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T06:14:30.696345+0000 front 2024-04-17T06:14:30.696394+0000 (oldest deadline 2024-04-17T06:14:55.396489+0000) 2024-04-17T06:15:05.633 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:15:05 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:15:05.196+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:53.138058+0000 front 2024-04-17T06:13:53.138083+0000 (oldest deadline 2024-04-17T06:14:14.238150+0000) 2024-04-17T06:15:05.633 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:15:05 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:15:05.196+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:13:58.939213+0000 front 2024-04-17T06:13:58.939286+0000 (oldest deadline 2024-04-17T06:14:24.839434+0000) 2024-04-17T06:15:05.633 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:15:05 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:15:05.196+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:07.040744+0000 front 2024-04-17T06:14:07.040770+0000 (oldest deadline 2024-04-17T06:14:30.441549+0000) 2024-04-17T06:15:05.633 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:15:05 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:15:05.196+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-17T06:14:15.743014+0000 front 2024-04-17T06:14:15.742885+0000 (oldest deadline 2024-04-17T06:14:38.642710+0000) 2024-04-17T06:15:05.634 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:15:05 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:15:05.196+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T06:14:23.343598+0000 front 2024-04-17T06:14:23.343627+0000 (oldest deadline 2024-04-17T06:14:48.043713+0000) 2024-04-17T06:15:05.634 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:15:05 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:15:05.196+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T06:14:29.144854+0000 front 2024-04-17T06:14:29.144791+0000 (oldest deadline 2024-04-17T06:14:55.044695+0000) 2024-04-17T06:15:05.889 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:15:05 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:15:05.627+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.354382+0000 front 2024-04-17T06:13:52.354774+0000 (oldest deadline 2024-04-17T06:14:15.254487+0000) 2024-04-17T06:15:05.889 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:15:05 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:15:05.627+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:01.655772+0000 front 2024-04-17T06:14:01.655755+0000 (oldest deadline 2024-04-17T06:14:27.555801+0000) 2024-04-17T06:15:05.889 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:15:05 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:15:05.627+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:09.257128+0000 front 2024-04-17T06:14:09.257345+0000 (oldest deadline 2024-04-17T06:14:31.557233+0000) 2024-04-17T06:15:05.890 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:15:05 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:15:05.627+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-17T06:14:17.958812+0000 front 2024-04-17T06:14:17.958952+0000 (oldest deadline 2024-04-17T06:14:42.658979+0000) 2024-04-17T06:15:05.890 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:15:05 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:15:05.627+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T06:14:22.659666+0000 front 2024-04-17T06:14:22.659556+0000 (oldest deadline 2024-04-17T06:14:47.359461+0000) 2024-04-17T06:15:05.890 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:15:05 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:15:05.627+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T06:14:31.361344+0000 front 2024-04-17T06:14:31.361395+0000 (oldest deadline 2024-04-17T06:14:56.661258+0000) 2024-04-17T06:15:05.890 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:15:05 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9[67823]: 2024-04-17T06:15:05.627+0000 7fd50ab75700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-04-17T06:14:41.362153+0000 front 2024-04-17T06:14:41.362187+0000 (oldest deadline 2024-04-17T06:15:03.562743+0000) 2024-04-17T06:15:06.284 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:05 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:05.791+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.989762+0000 front 2024-04-17T06:13:52.989781+0000 (oldest deadline 2024-04-17T06:14:18.289957+0000) 2024-04-17T06:15:06.284 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:05 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:05.791+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:01.191270+0000 front 2024-04-17T06:14:01.191086+0000 (oldest deadline 2024-04-17T06:14:24.690981+0000) 2024-04-17T06:15:06.284 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:05 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:05.791+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:09.292281+0000 front 2024-04-17T06:14:09.292431+0000 (oldest deadline 2024-04-17T06:14:35.192525+0000) 2024-04-17T06:15:06.284 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:05 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:05.791+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T06:14:23.294796+0000 front 2024-04-17T06:14:23.294673+0000 (oldest deadline 2024-04-17T06:14:46.695089+0000) 2024-04-17T06:15:06.285 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:05 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:05.791+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T06:14:30.696345+0000 front 2024-04-17T06:14:30.696394+0000 (oldest deadline 2024-04-17T06:14:55.396489+0000) 2024-04-17T06:15:06.633 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:15:06 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:15:06.211+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:53.138058+0000 front 2024-04-17T06:13:53.138083+0000 (oldest deadline 2024-04-17T06:14:14.238150+0000) 2024-04-17T06:15:06.633 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:15:06 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:15:06.211+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:13:58.939213+0000 front 2024-04-17T06:13:58.939286+0000 (oldest deadline 2024-04-17T06:14:24.839434+0000) 2024-04-17T06:15:06.634 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:15:06 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:15:06.211+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:07.040744+0000 front 2024-04-17T06:14:07.040770+0000 (oldest deadline 2024-04-17T06:14:30.441549+0000) 2024-04-17T06:15:06.634 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:15:06 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:15:06.211+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-17T06:14:15.743014+0000 front 2024-04-17T06:14:15.742885+0000 (oldest deadline 2024-04-17T06:14:38.642710+0000) 2024-04-17T06:15:06.634 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:15:06 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:15:06.211+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T06:14:23.343598+0000 front 2024-04-17T06:14:23.343627+0000 (oldest deadline 2024-04-17T06:14:48.043713+0000) 2024-04-17T06:15:06.634 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:15:06 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:15:06.211+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T06:14:29.144854+0000 front 2024-04-17T06:14:29.144791+0000 (oldest deadline 2024-04-17T06:14:55.044695+0000) 2024-04-17T06:15:06.909 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:15:06 smithi047 podman[94407]: 2024-04-17 06:15:06.646871448 +0000 UTC m=+5.617840500 container died f37324656282cbcbc24d7de6395bb86dcabc87cf3a9b1d6569b4aaaae52e88a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-04-17T06:15:07.284 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:06 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:06.797+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.989762+0000 front 2024-04-17T06:13:52.989781+0000 (oldest deadline 2024-04-17T06:14:18.289957+0000) 2024-04-17T06:15:07.284 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:06 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:06.797+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:01.191270+0000 front 2024-04-17T06:14:01.191086+0000 (oldest deadline 2024-04-17T06:14:24.690981+0000) 2024-04-17T06:15:07.284 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:06 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:06.797+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:09.292281+0000 front 2024-04-17T06:14:09.292431+0000 (oldest deadline 2024-04-17T06:14:35.192525+0000) 2024-04-17T06:15:07.284 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:06 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:06.797+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T06:14:23.294796+0000 front 2024-04-17T06:14:23.294673+0000 (oldest deadline 2024-04-17T06:14:46.695089+0000) 2024-04-17T06:15:07.284 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:06 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:06.797+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T06:14:30.696345+0000 front 2024-04-17T06:14:30.696394+0000 (oldest deadline 2024-04-17T06:14:55.396489+0000) 2024-04-17T06:15:07.284 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:06 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:06.797+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-04-17T06:14:41.098570+0000 front 2024-04-17T06:14:41.098676+0000 (oldest deadline 2024-04-17T06:15:05.798605+0000) 2024-04-17T06:15:07.285 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:15:06 smithi047 podman[94407]: 2024-04-17 06:15:06.993635366 +0000 UTC m=+5.964604408 container cleanup f37324656282cbcbc24d7de6395bb86dcabc87cf3a9b1d6569b4aaaae52e88a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-17T06:15:07.285 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:15:06 smithi047 bash[94407]: ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9 2024-04-17T06:15:07.633 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:15:07 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:15:07.260+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:53.138058+0000 front 2024-04-17T06:13:53.138083+0000 (oldest deadline 2024-04-17T06:14:14.238150+0000) 2024-04-17T06:15:07.633 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:15:07 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:15:07.260+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:13:58.939213+0000 front 2024-04-17T06:13:58.939286+0000 (oldest deadline 2024-04-17T06:14:24.839434+0000) 2024-04-17T06:15:07.633 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:15:07 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:15:07.260+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:07.040744+0000 front 2024-04-17T06:14:07.040770+0000 (oldest deadline 2024-04-17T06:14:30.441549+0000) 2024-04-17T06:15:07.633 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:15:07 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:15:07.260+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-17T06:14:15.743014+0000 front 2024-04-17T06:14:15.742885+0000 (oldest deadline 2024-04-17T06:14:38.642710+0000) 2024-04-17T06:15:07.633 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:15:07 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:15:07.260+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T06:14:23.343598+0000 front 2024-04-17T06:14:23.343627+0000 (oldest deadline 2024-04-17T06:14:48.043713+0000) 2024-04-17T06:15:07.633 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:15:07 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:15:07.260+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T06:14:29.144854+0000 front 2024-04-17T06:14:29.144791+0000 (oldest deadline 2024-04-17T06:14:55.044695+0000) 2024-04-17T06:15:07.633 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:15:07 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:15:07.260+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-04-17T06:14:42.046364+0000 front 2024-04-17T06:14:42.046419+0000 (oldest deadline 2024-04-17T06:15:07.246847+0000) 2024-04-17T06:15:07.638 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:15:07 smithi047 podman[94424]: 2024-04-17 06:15:07.286724762 +0000 UTC m=+0.631313129 container remove f37324656282cbcbc24d7de6395bb86dcabc87cf3a9b1d6569b4aaaae52e88a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-04-17T06:15:08.062 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:15:07 smithi047 podman[94516]: 2024-04-17 06:15:07.670230349 +0000 UTC m=+0.018028413 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-17T06:15:08.063 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:15:07 smithi047 podman[94516]: 2024-04-17 06:15:07.770838631 +0000 UTC m=+0.118636692 container create 6888d8a40992ddd92124d7074ebdae7e0a865d396e0b07c74b45f3f2c5fe58fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9-deactivate, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-04-17T06:15:08.063 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:15:07 smithi047 podman[94516]: 2024-04-17 06:15:07.990116509 +0000 UTC m=+0.337914574 container init 6888d8a40992ddd92124d7074ebdae7e0a865d396e0b07c74b45f3f2c5fe58fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-04-17T06:15:08.063 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:15:07 smithi047 podman[94516]: 2024-04-17 06:15:07.994142212 +0000 UTC m=+0.341940282 container start 6888d8a40992ddd92124d7074ebdae7e0a865d396e0b07c74b45f3f2c5fe58fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-17T06:15:08.063 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:15:08 smithi047 podman[94516]: 2024-04-17 06:15:08.064753609 +0000 UTC m=+0.412551682 container attach 6888d8a40992ddd92124d7074ebdae7e0a865d396e0b07c74b45f3f2c5fe58fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9-deactivate, ceph=True, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-17T06:15:08.284 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:07 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:07.801+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.989762+0000 front 2024-04-17T06:13:52.989781+0000 (oldest deadline 2024-04-17T06:14:18.289957+0000) 2024-04-17T06:15:08.284 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:07 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:07.801+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:01.191270+0000 front 2024-04-17T06:14:01.191086+0000 (oldest deadline 2024-04-17T06:14:24.690981+0000) 2024-04-17T06:15:08.284 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:07 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:07.801+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:09.292281+0000 front 2024-04-17T06:14:09.292431+0000 (oldest deadline 2024-04-17T06:14:35.192525+0000) 2024-04-17T06:15:08.284 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:07 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:07.801+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T06:14:23.294796+0000 front 2024-04-17T06:14:23.294673+0000 (oldest deadline 2024-04-17T06:14:46.695089+0000) 2024-04-17T06:15:08.284 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:07 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:07.801+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T06:14:30.696345+0000 front 2024-04-17T06:14:30.696394+0000 (oldest deadline 2024-04-17T06:14:55.396489+0000) 2024-04-17T06:15:08.284 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:07 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:07.801+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-04-17T06:14:41.098570+0000 front 2024-04-17T06:14:41.098676+0000 (oldest deadline 2024-04-17T06:15:05.798605+0000) 2024-04-17T06:15:08.323 INFO:journalctl@ceph.osd.9.smithi047.stdout:Apr 17 06:15:08 smithi047 podman[94516]: 2024-04-17 06:15:08.305476611 +0000 UTC m=+0.653274677 container died 6888d8a40992ddd92124d7074ebdae7e0a865d396e0b07c74b45f3f2c5fe58fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-9-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-17T06:15:08.633 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:15:08 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:15:08.276+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:53.138058+0000 front 2024-04-17T06:13:53.138083+0000 (oldest deadline 2024-04-17T06:14:14.238150+0000) 2024-04-17T06:15:08.633 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:15:08 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:15:08.276+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:13:58.939213+0000 front 2024-04-17T06:13:58.939286+0000 (oldest deadline 2024-04-17T06:14:24.839434+0000) 2024-04-17T06:15:08.633 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:15:08 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:15:08.276+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:07.040744+0000 front 2024-04-17T06:14:07.040770+0000 (oldest deadline 2024-04-17T06:14:30.441549+0000) 2024-04-17T06:15:08.633 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:15:08 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:15:08.276+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-17T06:14:15.743014+0000 front 2024-04-17T06:14:15.742885+0000 (oldest deadline 2024-04-17T06:14:38.642710+0000) 2024-04-17T06:15:08.634 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:15:08 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:15:08.276+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T06:14:23.343598+0000 front 2024-04-17T06:14:23.343627+0000 (oldest deadline 2024-04-17T06:14:48.043713+0000) 2024-04-17T06:15:08.634 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:15:08 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:15:08.276+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T06:14:29.144854+0000 front 2024-04-17T06:14:29.144791+0000 (oldest deadline 2024-04-17T06:14:55.044695+0000) 2024-04-17T06:15:08.634 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:15:08 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:15:08.276+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-04-17T06:14:42.046364+0000 front 2024-04-17T06:14:42.046419+0000 (oldest deadline 2024-04-17T06:15:07.246847+0000) 2024-04-17T06:15:08.875 DEBUG:teuthology.orchestra.run.smithi047:> sudo pkill -f 'journalctl -f -n 0 -u ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@osd.9.service' 2024-04-17T06:15:08.917 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-17T06:15:08.917 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-04-17T06:15:08.917 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-04-17T06:15:08.917 DEBUG:teuthology.orchestra.run.smithi098:> sudo systemctl stop ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@osd.10 2024-04-17T06:15:09.209 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:15:08 smithi098 systemd[1]: Stopping Ceph osd.10 for 565db2d8-fc7e-11ee-bc91-c7b262605968... 2024-04-17T06:15:09.284 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:08 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:08.787+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.989762+0000 front 2024-04-17T06:13:52.989781+0000 (oldest deadline 2024-04-17T06:14:18.289957+0000) 2024-04-17T06:15:09.284 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:08 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:08.787+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:01.191270+0000 front 2024-04-17T06:14:01.191086+0000 (oldest deadline 2024-04-17T06:14:24.690981+0000) 2024-04-17T06:15:09.284 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:08 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:08.787+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:09.292281+0000 front 2024-04-17T06:14:09.292431+0000 (oldest deadline 2024-04-17T06:14:35.192525+0000) 2024-04-17T06:15:09.284 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:08 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:08.787+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T06:14:23.294796+0000 front 2024-04-17T06:14:23.294673+0000 (oldest deadline 2024-04-17T06:14:46.695089+0000) 2024-04-17T06:15:09.284 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:08 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:08.787+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T06:14:30.696345+0000 front 2024-04-17T06:14:30.696394+0000 (oldest deadline 2024-04-17T06:14:55.396489+0000) 2024-04-17T06:15:09.284 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:08 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:08.787+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-04-17T06:14:41.098570+0000 front 2024-04-17T06:14:41.098676+0000 (oldest deadline 2024-04-17T06:15:05.798605+0000) 2024-04-17T06:15:09.633 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:15:09 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:15:09.210+0000 7fdb3a37d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-17T06:15:09.634 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:15:09 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:15:09.210+0000 7fdb3a37d700 -1 osd.10 109 *** Got signal Terminated *** 2024-04-17T06:15:09.634 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:15:09 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:15:09.210+0000 7fdb3a37d700 -1 osd.10 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-17T06:15:09.634 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:15:09 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:15:09.244+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:53.138058+0000 front 2024-04-17T06:13:53.138083+0000 (oldest deadline 2024-04-17T06:14:14.238150+0000) 2024-04-17T06:15:09.634 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:15:09 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:15:09.244+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:13:58.939213+0000 front 2024-04-17T06:13:58.939286+0000 (oldest deadline 2024-04-17T06:14:24.839434+0000) 2024-04-17T06:15:09.634 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:15:09 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:15:09.244+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:07.040744+0000 front 2024-04-17T06:14:07.040770+0000 (oldest deadline 2024-04-17T06:14:30.441549+0000) 2024-04-17T06:15:09.634 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:15:09 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:15:09.244+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-17T06:14:15.743014+0000 front 2024-04-17T06:14:15.742885+0000 (oldest deadline 2024-04-17T06:14:38.642710+0000) 2024-04-17T06:15:09.634 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:15:09 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:15:09.244+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T06:14:23.343598+0000 front 2024-04-17T06:14:23.343627+0000 (oldest deadline 2024-04-17T06:14:48.043713+0000) 2024-04-17T06:15:09.634 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:15:09 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:15:09.244+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T06:14:29.144854+0000 front 2024-04-17T06:14:29.144791+0000 (oldest deadline 2024-04-17T06:14:55.044695+0000) 2024-04-17T06:15:09.634 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:15:09 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:15:09.244+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-04-17T06:14:42.046364+0000 front 2024-04-17T06:14:42.046419+0000 (oldest deadline 2024-04-17T06:15:07.246847+0000) 2024-04-17T06:15:10.034 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:09 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:09.783+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.989762+0000 front 2024-04-17T06:13:52.989781+0000 (oldest deadline 2024-04-17T06:14:18.289957+0000) 2024-04-17T06:15:10.034 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:09 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:09.783+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:01.191270+0000 front 2024-04-17T06:14:01.191086+0000 (oldest deadline 2024-04-17T06:14:24.690981+0000) 2024-04-17T06:15:10.034 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:09 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:09.783+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:09.292281+0000 front 2024-04-17T06:14:09.292431+0000 (oldest deadline 2024-04-17T06:14:35.192525+0000) 2024-04-17T06:15:10.034 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:09 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:09.783+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T06:14:23.294796+0000 front 2024-04-17T06:14:23.294673+0000 (oldest deadline 2024-04-17T06:14:46.695089+0000) 2024-04-17T06:15:10.035 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:09 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:09.783+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T06:14:30.696345+0000 front 2024-04-17T06:14:30.696394+0000 (oldest deadline 2024-04-17T06:14:55.396489+0000) 2024-04-17T06:15:10.035 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:09 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:09.783+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-04-17T06:14:41.098570+0000 front 2024-04-17T06:14:41.098676+0000 (oldest deadline 2024-04-17T06:15:05.798605+0000) 2024-04-17T06:15:10.633 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:15:10 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:15:10.249+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:53.138058+0000 front 2024-04-17T06:13:53.138083+0000 (oldest deadline 2024-04-17T06:14:14.238150+0000) 2024-04-17T06:15:10.633 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:15:10 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:15:10.249+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:13:58.939213+0000 front 2024-04-17T06:13:58.939286+0000 (oldest deadline 2024-04-17T06:14:24.839434+0000) 2024-04-17T06:15:10.633 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:15:10 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:15:10.249+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:07.040744+0000 front 2024-04-17T06:14:07.040770+0000 (oldest deadline 2024-04-17T06:14:30.441549+0000) 2024-04-17T06:15:10.633 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:15:10 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:15:10.249+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-17T06:14:15.743014+0000 front 2024-04-17T06:14:15.742885+0000 (oldest deadline 2024-04-17T06:14:38.642710+0000) 2024-04-17T06:15:10.633 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:15:10 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:15:10.249+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T06:14:23.343598+0000 front 2024-04-17T06:14:23.343627+0000 (oldest deadline 2024-04-17T06:14:48.043713+0000) 2024-04-17T06:15:10.633 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:15:10 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:15:10.249+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T06:14:29.144854+0000 front 2024-04-17T06:14:29.144791+0000 (oldest deadline 2024-04-17T06:14:55.044695+0000) 2024-04-17T06:15:10.633 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:15:10 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:15:10.249+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-04-17T06:14:42.046364+0000 front 2024-04-17T06:14:42.046419+0000 (oldest deadline 2024-04-17T06:15:07.246847+0000) 2024-04-17T06:15:11.034 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:10 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:10.745+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.989762+0000 front 2024-04-17T06:13:52.989781+0000 (oldest deadline 2024-04-17T06:14:18.289957+0000) 2024-04-17T06:15:11.034 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:10 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:10.745+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:01.191270+0000 front 2024-04-17T06:14:01.191086+0000 (oldest deadline 2024-04-17T06:14:24.690981+0000) 2024-04-17T06:15:11.034 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:10 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:10.745+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:09.292281+0000 front 2024-04-17T06:14:09.292431+0000 (oldest deadline 2024-04-17T06:14:35.192525+0000) 2024-04-17T06:15:11.034 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:10 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:10.745+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T06:14:23.294796+0000 front 2024-04-17T06:14:23.294673+0000 (oldest deadline 2024-04-17T06:14:46.695089+0000) 2024-04-17T06:15:11.034 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:10 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:10.745+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T06:14:30.696345+0000 front 2024-04-17T06:14:30.696394+0000 (oldest deadline 2024-04-17T06:14:55.396489+0000) 2024-04-17T06:15:11.034 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:10 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:10.745+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-04-17T06:14:41.098570+0000 front 2024-04-17T06:14:41.098676+0000 (oldest deadline 2024-04-17T06:15:05.798605+0000) 2024-04-17T06:15:11.633 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:15:11 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:15:11.287+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:53.138058+0000 front 2024-04-17T06:13:53.138083+0000 (oldest deadline 2024-04-17T06:14:14.238150+0000) 2024-04-17T06:15:11.633 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:15:11 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:15:11.287+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:13:58.939213+0000 front 2024-04-17T06:13:58.939286+0000 (oldest deadline 2024-04-17T06:14:24.839434+0000) 2024-04-17T06:15:11.633 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:15:11 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:15:11.287+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:07.040744+0000 front 2024-04-17T06:14:07.040770+0000 (oldest deadline 2024-04-17T06:14:30.441549+0000) 2024-04-17T06:15:11.633 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:15:11 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:15:11.287+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-17T06:14:15.743014+0000 front 2024-04-17T06:14:15.742885+0000 (oldest deadline 2024-04-17T06:14:38.642710+0000) 2024-04-17T06:15:11.633 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:15:11 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:15:11.287+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T06:14:23.343598+0000 front 2024-04-17T06:14:23.343627+0000 (oldest deadline 2024-04-17T06:14:48.043713+0000) 2024-04-17T06:15:11.633 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:15:11 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:15:11.287+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T06:14:29.144854+0000 front 2024-04-17T06:14:29.144791+0000 (oldest deadline 2024-04-17T06:14:55.044695+0000) 2024-04-17T06:15:11.633 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:15:11 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:15:11.287+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-04-17T06:14:42.046364+0000 front 2024-04-17T06:14:42.046419+0000 (oldest deadline 2024-04-17T06:15:07.246847+0000) 2024-04-17T06:15:12.034 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:11 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:11.720+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.989762+0000 front 2024-04-17T06:13:52.989781+0000 (oldest deadline 2024-04-17T06:14:18.289957+0000) 2024-04-17T06:15:12.034 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:11 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:11.720+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:01.191270+0000 front 2024-04-17T06:14:01.191086+0000 (oldest deadline 2024-04-17T06:14:24.690981+0000) 2024-04-17T06:15:12.034 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:11 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:11.720+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:09.292281+0000 front 2024-04-17T06:14:09.292431+0000 (oldest deadline 2024-04-17T06:14:35.192525+0000) 2024-04-17T06:15:12.034 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:11 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:11.720+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T06:14:23.294796+0000 front 2024-04-17T06:14:23.294673+0000 (oldest deadline 2024-04-17T06:14:46.695089+0000) 2024-04-17T06:15:12.034 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:11 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:11.720+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T06:14:30.696345+0000 front 2024-04-17T06:14:30.696394+0000 (oldest deadline 2024-04-17T06:14:55.396489+0000) 2024-04-17T06:15:12.034 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:11 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:11.720+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-04-17T06:14:41.098570+0000 front 2024-04-17T06:14:41.098676+0000 (oldest deadline 2024-04-17T06:15:05.798605+0000) 2024-04-17T06:15:12.035 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:11 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:11.720+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.98:6820 osd.7 since back 2024-04-17T06:14:45.799172+0000 front 2024-04-17T06:14:45.799153+0000 (oldest deadline 2024-04-17T06:15:11.699127+0000) 2024-04-17T06:15:12.633 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:15:12 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:15:12.259+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:53.138058+0000 front 2024-04-17T06:13:53.138083+0000 (oldest deadline 2024-04-17T06:14:14.238150+0000) 2024-04-17T06:15:12.633 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:15:12 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:15:12.259+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:13:58.939213+0000 front 2024-04-17T06:13:58.939286+0000 (oldest deadline 2024-04-17T06:14:24.839434+0000) 2024-04-17T06:15:12.633 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:15:12 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:15:12.259+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:07.040744+0000 front 2024-04-17T06:14:07.040770+0000 (oldest deadline 2024-04-17T06:14:30.441549+0000) 2024-04-17T06:15:12.633 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:15:12 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:15:12.259+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-17T06:14:15.743014+0000 front 2024-04-17T06:14:15.742885+0000 (oldest deadline 2024-04-17T06:14:38.642710+0000) 2024-04-17T06:15:12.633 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:15:12 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:15:12.259+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T06:14:23.343598+0000 front 2024-04-17T06:14:23.343627+0000 (oldest deadline 2024-04-17T06:14:48.043713+0000) 2024-04-17T06:15:12.633 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:15:12 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:15:12.259+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T06:14:29.144854+0000 front 2024-04-17T06:14:29.144791+0000 (oldest deadline 2024-04-17T06:14:55.044695+0000) 2024-04-17T06:15:12.633 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:15:12 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:15:12.259+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-04-17T06:14:42.046364+0000 front 2024-04-17T06:14:42.046419+0000 (oldest deadline 2024-04-17T06:15:07.246847+0000) 2024-04-17T06:15:13.034 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:12 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:12.719+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.989762+0000 front 2024-04-17T06:13:52.989781+0000 (oldest deadline 2024-04-17T06:14:18.289957+0000) 2024-04-17T06:15:13.034 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:12 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:12.719+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:01.191270+0000 front 2024-04-17T06:14:01.191086+0000 (oldest deadline 2024-04-17T06:14:24.690981+0000) 2024-04-17T06:15:13.034 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:12 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:12.719+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:09.292281+0000 front 2024-04-17T06:14:09.292431+0000 (oldest deadline 2024-04-17T06:14:35.192525+0000) 2024-04-17T06:15:13.034 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:12 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:12.719+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T06:14:23.294796+0000 front 2024-04-17T06:14:23.294673+0000 (oldest deadline 2024-04-17T06:14:46.695089+0000) 2024-04-17T06:15:13.035 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:12 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:12.719+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T06:14:30.696345+0000 front 2024-04-17T06:14:30.696394+0000 (oldest deadline 2024-04-17T06:14:55.396489+0000) 2024-04-17T06:15:13.035 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:12 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:12.719+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-04-17T06:14:41.098570+0000 front 2024-04-17T06:14:41.098676+0000 (oldest deadline 2024-04-17T06:15:05.798605+0000) 2024-04-17T06:15:13.035 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:12 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:12.719+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.98:6820 osd.7 since back 2024-04-17T06:14:45.799172+0000 front 2024-04-17T06:14:45.799153+0000 (oldest deadline 2024-04-17T06:15:11.699127+0000) 2024-04-17T06:15:13.633 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:15:13 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:15:13.292+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:53.138058+0000 front 2024-04-17T06:13:53.138083+0000 (oldest deadline 2024-04-17T06:14:14.238150+0000) 2024-04-17T06:15:13.634 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:15:13 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:15:13.292+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:13:58.939213+0000 front 2024-04-17T06:13:58.939286+0000 (oldest deadline 2024-04-17T06:14:24.839434+0000) 2024-04-17T06:15:13.634 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:15:13 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:15:13.292+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:07.040744+0000 front 2024-04-17T06:14:07.040770+0000 (oldest deadline 2024-04-17T06:14:30.441549+0000) 2024-04-17T06:15:13.634 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:15:13 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:15:13.292+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-04-17T06:14:15.743014+0000 front 2024-04-17T06:14:15.742885+0000 (oldest deadline 2024-04-17T06:14:38.642710+0000) 2024-04-17T06:15:13.634 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:15:13 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:15:13.292+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T06:14:23.343598+0000 front 2024-04-17T06:14:23.343627+0000 (oldest deadline 2024-04-17T06:14:48.043713+0000) 2024-04-17T06:15:13.634 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:15:13 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:15:13.292+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T06:14:29.144854+0000 front 2024-04-17T06:14:29.144791+0000 (oldest deadline 2024-04-17T06:14:55.044695+0000) 2024-04-17T06:15:13.634 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:15:13 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10[61660]: 2024-04-17T06:15:13.292+0000 7fdb32969700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-04-17T06:14:42.046364+0000 front 2024-04-17T06:14:42.046419+0000 (oldest deadline 2024-04-17T06:15:07.246847+0000) 2024-04-17T06:15:14.034 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:13 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:13.762+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.989762+0000 front 2024-04-17T06:13:52.989781+0000 (oldest deadline 2024-04-17T06:14:18.289957+0000) 2024-04-17T06:15:14.034 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:13 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:13.762+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:01.191270+0000 front 2024-04-17T06:14:01.191086+0000 (oldest deadline 2024-04-17T06:14:24.690981+0000) 2024-04-17T06:15:14.034 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:13 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:13.762+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:09.292281+0000 front 2024-04-17T06:14:09.292431+0000 (oldest deadline 2024-04-17T06:14:35.192525+0000) 2024-04-17T06:15:14.034 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:13 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:13.762+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T06:14:23.294796+0000 front 2024-04-17T06:14:23.294673+0000 (oldest deadline 2024-04-17T06:14:46.695089+0000) 2024-04-17T06:15:14.034 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:13 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:13.762+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T06:14:30.696345+0000 front 2024-04-17T06:14:30.696394+0000 (oldest deadline 2024-04-17T06:14:55.396489+0000) 2024-04-17T06:15:14.035 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:13 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:13.762+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-04-17T06:14:41.098570+0000 front 2024-04-17T06:14:41.098676+0000 (oldest deadline 2024-04-17T06:15:05.798605+0000) 2024-04-17T06:15:14.035 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:13 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:13.762+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.98:6820 osd.7 since back 2024-04-17T06:14:45.799172+0000 front 2024-04-17T06:14:45.799153+0000 (oldest deadline 2024-04-17T06:15:11.699127+0000) 2024-04-17T06:15:14.883 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:15:14 smithi098 podman[75070]: 2024-04-17 06:15:14.594657653 +0000 UTC m=+5.481447168 container died 904727f23cbf3ca1e39c73c598bebfecfd5ae527372b8ac870ebc032097000de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True) 2024-04-17T06:15:15.034 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:14 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:14.756+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.989762+0000 front 2024-04-17T06:13:52.989781+0000 (oldest deadline 2024-04-17T06:14:18.289957+0000) 2024-04-17T06:15:15.034 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:14 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:14.756+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:01.191270+0000 front 2024-04-17T06:14:01.191086+0000 (oldest deadline 2024-04-17T06:14:24.690981+0000) 2024-04-17T06:15:15.034 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:14 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:14.756+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:09.292281+0000 front 2024-04-17T06:14:09.292431+0000 (oldest deadline 2024-04-17T06:14:35.192525+0000) 2024-04-17T06:15:15.034 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:14 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:14.756+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T06:14:23.294796+0000 front 2024-04-17T06:14:23.294673+0000 (oldest deadline 2024-04-17T06:14:46.695089+0000) 2024-04-17T06:15:15.034 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:14 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:14.756+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T06:14:30.696345+0000 front 2024-04-17T06:14:30.696394+0000 (oldest deadline 2024-04-17T06:14:55.396489+0000) 2024-04-17T06:15:15.035 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:14 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:14.756+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-04-17T06:14:41.098570+0000 front 2024-04-17T06:14:41.098676+0000 (oldest deadline 2024-04-17T06:15:05.798605+0000) 2024-04-17T06:15:15.035 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:14 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:14.756+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.98:6820 osd.7 since back 2024-04-17T06:14:45.799172+0000 front 2024-04-17T06:14:45.799153+0000 (oldest deadline 2024-04-17T06:15:11.699127+0000) 2024-04-17T06:15:15.226 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:15:14 smithi098 podman[75070]: 2024-04-17 06:15:14.97745984 +0000 UTC m=+5.864249377 container cleanup 904727f23cbf3ca1e39c73c598bebfecfd5ae527372b8ac870ebc032097000de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10, org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-17T06:15:15.226 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:15:14 smithi098 bash[75070]: ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10 2024-04-17T06:15:15.574 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:15:15 smithi098 podman[75088]: 2024-04-17 06:15:15.228624546 +0000 UTC m=+0.627773257 container remove 904727f23cbf3ca1e39c73c598bebfecfd5ae527372b8ac870ebc032097000de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-17T06:15:15.855 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:15:15 smithi098 podman[75178]: 2024-04-17 06:15:15.479715301 +0000 UTC m=+0.017638241 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-17T06:15:15.855 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:15:15 smithi098 podman[75178]: 2024-04-17 06:15:15.579459321 +0000 UTC m=+0.117382236 container create 4fd238e9c9ea16020ac5c2dc3487e353d2c7b28d9967cd0f1eced6ea431caee3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-04-17T06:15:15.855 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:15:15 smithi098 podman[75178]: 2024-04-17 06:15:15.85737964 +0000 UTC m=+0.395302572 container init 4fd238e9c9ea16020ac5c2dc3487e353d2c7b28d9967cd0f1eced6ea431caee3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True) 2024-04-17T06:15:16.034 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:15 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:15.738+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.989762+0000 front 2024-04-17T06:13:52.989781+0000 (oldest deadline 2024-04-17T06:14:18.289957+0000) 2024-04-17T06:15:16.034 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:15 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:15.738+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:01.191270+0000 front 2024-04-17T06:14:01.191086+0000 (oldest deadline 2024-04-17T06:14:24.690981+0000) 2024-04-17T06:15:16.034 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:15 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:15.738+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:09.292281+0000 front 2024-04-17T06:14:09.292431+0000 (oldest deadline 2024-04-17T06:14:35.192525+0000) 2024-04-17T06:15:16.034 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:15 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:15.738+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T06:14:23.294796+0000 front 2024-04-17T06:14:23.294673+0000 (oldest deadline 2024-04-17T06:14:46.695089+0000) 2024-04-17T06:15:16.034 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:15 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:15.738+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T06:14:30.696345+0000 front 2024-04-17T06:14:30.696394+0000 (oldest deadline 2024-04-17T06:14:55.396489+0000) 2024-04-17T06:15:16.034 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:15 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:15.738+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-04-17T06:14:41.098570+0000 front 2024-04-17T06:14:41.098676+0000 (oldest deadline 2024-04-17T06:15:05.798605+0000) 2024-04-17T06:15:16.035 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:15 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:15.738+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.98:6820 osd.7 since back 2024-04-17T06:14:45.799172+0000 front 2024-04-17T06:14:45.799153+0000 (oldest deadline 2024-04-17T06:15:11.699127+0000) 2024-04-17T06:15:16.133 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:15:15 smithi098 podman[75178]: 2024-04-17 06:15:15.860462686 +0000 UTC m=+0.398385622 container start 4fd238e9c9ea16020ac5c2dc3487e353d2c7b28d9967cd0f1eced6ea431caee3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-04-17T06:15:16.133 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:15:15 smithi098 podman[75178]: 2024-04-17 06:15:15.940810547 +0000 UTC m=+0.478733494 container attach 4fd238e9c9ea16020ac5c2dc3487e353d2c7b28d9967cd0f1eced6ea431caee3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416) 2024-04-17T06:15:16.433 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 06:15:16 smithi098 podman[75178]: 2024-04-17 06:15:16.149716379 +0000 UTC m=+0.687639316 container died 4fd238e9c9ea16020ac5c2dc3487e353d2c7b28d9967cd0f1eced6ea431caee3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-10-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.license=GPLv2, ceph=True) 2024-04-17T06:15:16.748 DEBUG:teuthology.orchestra.run.smithi098:> sudo pkill -f 'journalctl -f -n 0 -u ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@osd.10.service' 2024-04-17T06:15:16.791 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-17T06:15:16.791 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-04-17T06:15:16.791 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-04-17T06:15:16.792 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@osd.11 2024-04-17T06:15:17.034 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:16 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:16.714+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.989762+0000 front 2024-04-17T06:13:52.989781+0000 (oldest deadline 2024-04-17T06:14:18.289957+0000) 2024-04-17T06:15:17.034 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:16 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:16.714+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:01.191270+0000 front 2024-04-17T06:14:01.191086+0000 (oldest deadline 2024-04-17T06:14:24.690981+0000) 2024-04-17T06:15:17.034 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:16 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:16.714+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:09.292281+0000 front 2024-04-17T06:14:09.292431+0000 (oldest deadline 2024-04-17T06:14:35.192525+0000) 2024-04-17T06:15:17.034 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:16 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:16.714+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T06:14:23.294796+0000 front 2024-04-17T06:14:23.294673+0000 (oldest deadline 2024-04-17T06:14:46.695089+0000) 2024-04-17T06:15:17.034 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:16 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:16.714+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T06:14:30.696345+0000 front 2024-04-17T06:14:30.696394+0000 (oldest deadline 2024-04-17T06:14:55.396489+0000) 2024-04-17T06:15:17.035 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:16 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:16.714+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-04-17T06:14:41.098570+0000 front 2024-04-17T06:14:41.098676+0000 (oldest deadline 2024-04-17T06:15:05.798605+0000) 2024-04-17T06:15:17.035 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:16 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:16.714+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.98:6820 osd.7 since back 2024-04-17T06:14:45.799172+0000 front 2024-04-17T06:14:45.799153+0000 (oldest deadline 2024-04-17T06:15:11.699127+0000) 2024-04-17T06:15:17.035 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:16 smithi103 systemd[1]: Stopping Ceph osd.11 for 565db2d8-fc7e-11ee-bc91-c7b262605968... 2024-04-17T06:15:17.534 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:17 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:17.086+0000 7f4e5c453700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-17T06:15:17.534 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:17 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:17.086+0000 7f4e5c453700 -1 osd.11 109 *** Got signal Terminated *** 2024-04-17T06:15:17.534 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:17 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:17.086+0000 7f4e5c453700 -1 osd.11 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-17T06:15:18.034 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:17 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:17.728+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.989762+0000 front 2024-04-17T06:13:52.989781+0000 (oldest deadline 2024-04-17T06:14:18.289957+0000) 2024-04-17T06:15:18.034 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:17 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:17.728+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:01.191270+0000 front 2024-04-17T06:14:01.191086+0000 (oldest deadline 2024-04-17T06:14:24.690981+0000) 2024-04-17T06:15:18.034 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:17 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:17.728+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:09.292281+0000 front 2024-04-17T06:14:09.292431+0000 (oldest deadline 2024-04-17T06:14:35.192525+0000) 2024-04-17T06:15:18.034 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:17 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:17.728+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T06:14:23.294796+0000 front 2024-04-17T06:14:23.294673+0000 (oldest deadline 2024-04-17T06:14:46.695089+0000) 2024-04-17T06:15:18.034 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:17 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:17.728+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T06:14:30.696345+0000 front 2024-04-17T06:14:30.696394+0000 (oldest deadline 2024-04-17T06:14:55.396489+0000) 2024-04-17T06:15:18.034 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:17 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:17.728+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-04-17T06:14:41.098570+0000 front 2024-04-17T06:14:41.098676+0000 (oldest deadline 2024-04-17T06:15:05.798605+0000) 2024-04-17T06:15:18.034 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:17 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:17.728+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.98:6820 osd.7 since back 2024-04-17T06:14:45.799172+0000 front 2024-04-17T06:14:45.799153+0000 (oldest deadline 2024-04-17T06:15:11.699127+0000) 2024-04-17T06:15:19.034 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:18 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:18.725+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.989762+0000 front 2024-04-17T06:13:52.989781+0000 (oldest deadline 2024-04-17T06:14:18.289957+0000) 2024-04-17T06:15:19.035 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:18 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:18.725+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:01.191270+0000 front 2024-04-17T06:14:01.191086+0000 (oldest deadline 2024-04-17T06:14:24.690981+0000) 2024-04-17T06:15:19.035 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:18 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:18.725+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:09.292281+0000 front 2024-04-17T06:14:09.292431+0000 (oldest deadline 2024-04-17T06:14:35.192525+0000) 2024-04-17T06:15:19.035 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:18 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:18.725+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T06:14:23.294796+0000 front 2024-04-17T06:14:23.294673+0000 (oldest deadline 2024-04-17T06:14:46.695089+0000) 2024-04-17T06:15:19.035 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:18 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:18.725+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T06:14:30.696345+0000 front 2024-04-17T06:14:30.696394+0000 (oldest deadline 2024-04-17T06:14:55.396489+0000) 2024-04-17T06:15:19.035 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:18 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:18.725+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-04-17T06:14:41.098570+0000 front 2024-04-17T06:14:41.098676+0000 (oldest deadline 2024-04-17T06:15:05.798605+0000) 2024-04-17T06:15:19.035 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:18 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:18.725+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.98:6820 osd.7 since back 2024-04-17T06:14:45.799172+0000 front 2024-04-17T06:14:45.799153+0000 (oldest deadline 2024-04-17T06:15:11.699127+0000) 2024-04-17T06:15:20.034 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:19 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:19.701+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.989762+0000 front 2024-04-17T06:13:52.989781+0000 (oldest deadline 2024-04-17T06:14:18.289957+0000) 2024-04-17T06:15:20.034 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:19 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:19.701+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:01.191270+0000 front 2024-04-17T06:14:01.191086+0000 (oldest deadline 2024-04-17T06:14:24.690981+0000) 2024-04-17T06:15:20.034 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:19 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:19.701+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:09.292281+0000 front 2024-04-17T06:14:09.292431+0000 (oldest deadline 2024-04-17T06:14:35.192525+0000) 2024-04-17T06:15:20.034 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:19 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:19.701+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T06:14:23.294796+0000 front 2024-04-17T06:14:23.294673+0000 (oldest deadline 2024-04-17T06:14:46.695089+0000) 2024-04-17T06:15:20.034 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:19 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:19.701+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T06:14:30.696345+0000 front 2024-04-17T06:14:30.696394+0000 (oldest deadline 2024-04-17T06:14:55.396489+0000) 2024-04-17T06:15:20.034 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:19 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:19.701+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-04-17T06:14:41.098570+0000 front 2024-04-17T06:14:41.098676+0000 (oldest deadline 2024-04-17T06:15:05.798605+0000) 2024-04-17T06:15:20.035 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:19 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:19.701+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.98:6820 osd.7 since back 2024-04-17T06:14:45.799172+0000 front 2024-04-17T06:14:45.799153+0000 (oldest deadline 2024-04-17T06:15:11.699127+0000) 2024-04-17T06:15:21.034 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:20 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:20.666+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.989762+0000 front 2024-04-17T06:13:52.989781+0000 (oldest deadline 2024-04-17T06:14:18.289957+0000) 2024-04-17T06:15:21.034 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:20 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:20.666+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:01.191270+0000 front 2024-04-17T06:14:01.191086+0000 (oldest deadline 2024-04-17T06:14:24.690981+0000) 2024-04-17T06:15:21.034 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:20 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:20.666+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:09.292281+0000 front 2024-04-17T06:14:09.292431+0000 (oldest deadline 2024-04-17T06:14:35.192525+0000) 2024-04-17T06:15:21.034 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:20 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:20.666+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T06:14:23.294796+0000 front 2024-04-17T06:14:23.294673+0000 (oldest deadline 2024-04-17T06:14:46.695089+0000) 2024-04-17T06:15:21.034 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:20 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:20.666+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T06:14:30.696345+0000 front 2024-04-17T06:14:30.696394+0000 (oldest deadline 2024-04-17T06:14:55.396489+0000) 2024-04-17T06:15:21.035 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:20 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:20.666+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-04-17T06:14:41.098570+0000 front 2024-04-17T06:14:41.098676+0000 (oldest deadline 2024-04-17T06:15:05.798605+0000) 2024-04-17T06:15:21.035 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:20 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:20.666+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.98:6820 osd.7 since back 2024-04-17T06:14:45.799172+0000 front 2024-04-17T06:14:45.799153+0000 (oldest deadline 2024-04-17T06:15:11.699127+0000) 2024-04-17T06:15:21.035 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:20 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:20.666+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.103:6820 osd.8 since back 2024-04-17T06:14:58.100417+0000 front 2024-04-17T06:14:58.100437+0000 (oldest deadline 2024-04-17T06:15:19.800457+0000) 2024-04-17T06:15:22.034 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:21 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:21.667+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-04-17T06:13:52.989762+0000 front 2024-04-17T06:13:52.989781+0000 (oldest deadline 2024-04-17T06:14:18.289957+0000) 2024-04-17T06:15:22.034 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:21 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:21.667+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T06:14:01.191270+0000 front 2024-04-17T06:14:01.191086+0000 (oldest deadline 2024-04-17T06:14:24.690981+0000) 2024-04-17T06:15:22.034 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:21 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:21.667+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T06:14:09.292281+0000 front 2024-04-17T06:14:09.292431+0000 (oldest deadline 2024-04-17T06:14:35.192525+0000) 2024-04-17T06:15:22.035 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:21 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:21.667+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T06:14:23.294796+0000 front 2024-04-17T06:14:23.294673+0000 (oldest deadline 2024-04-17T06:14:46.695089+0000) 2024-04-17T06:15:22.035 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:21 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:21.667+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T06:14:30.696345+0000 front 2024-04-17T06:14:30.696394+0000 (oldest deadline 2024-04-17T06:14:55.396489+0000) 2024-04-17T06:15:22.035 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:21 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:21.667+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-04-17T06:14:41.098570+0000 front 2024-04-17T06:14:41.098676+0000 (oldest deadline 2024-04-17T06:15:05.798605+0000) 2024-04-17T06:15:22.035 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:21 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:21.667+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.98:6820 osd.7 since back 2024-04-17T06:14:45.799172+0000 front 2024-04-17T06:14:45.799153+0000 (oldest deadline 2024-04-17T06:15:11.699127+0000) 2024-04-17T06:15:22.035 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:21 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11[61684]: 2024-04-17T06:15:21.667+0000 7f4e54a3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.103:6820 osd.8 since back 2024-04-17T06:14:58.100417+0000 front 2024-04-17T06:14:58.100437+0000 (oldest deadline 2024-04-17T06:15:19.800457+0000) 2024-04-17T06:15:22.942 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:22 smithi103 podman[74251]: 2024-04-17 06:15:22.620133948 +0000 UTC m=+5.637294226 container died 2dac99cfead4d91283e2242741ce8e76b46ff2a17592b471933b1d36d1d298d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-17T06:15:22.942 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:22 smithi103 podman[74251]: 2024-04-17 06:15:22.944203549 +0000 UTC m=+5.961363820 container cleanup 2dac99cfead4d91283e2242741ce8e76b46ff2a17592b471933b1d36d1d298d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux ) 2024-04-17T06:15:23.284 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:22 smithi103 bash[74251]: ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11 2024-04-17T06:15:23.284 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:23 smithi103 podman[74268]: 2024-04-17 06:15:23.128993234 +0000 UTC m=+0.503586093 container remove 2dac99cfead4d91283e2242741ce8e76b46ff2a17592b471933b1d36d1d298d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11, ceph=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-17T06:15:23.814 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:23 smithi103 podman[74359]: 2024-04-17 06:15:23.461981385 +0000 UTC m=+0.017746419 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-17T06:15:23.814 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:23 smithi103 podman[74359]: 2024-04-17 06:15:23.588288794 +0000 UTC m=+0.144053851 container create fe901a1ea4649c2758aedb8ed686447d4866999109e99e32842023afb20ef7c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-04-17T06:15:24.105 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:23 smithi103 podman[74359]: 2024-04-17 06:15:23.816591285 +0000 UTC m=+0.372356317 container init fe901a1ea4649c2758aedb8ed686447d4866999109e99e32842023afb20ef7c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11-deactivate, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_CLEAN=True) 2024-04-17T06:15:24.105 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:23 smithi103 podman[74359]: 2024-04-17 06:15:23.819806928 +0000 UTC m=+0.375571938 container start fe901a1ea4649c2758aedb8ed686447d4866999109e99e32842023afb20ef7c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11-deactivate, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-17T06:15:24.105 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:23 smithi103 podman[74359]: 2024-04-17 06:15:23.908063486 +0000 UTC m=+0.463828520 container attach fe901a1ea4649c2758aedb8ed686447d4866999109e99e32842023afb20ef7c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-04-17T06:15:24.400 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 06:15:24 smithi103 podman[74359]: 2024-04-17 06:15:24.107377448 +0000 UTC m=+0.663142468 container died fe901a1ea4649c2758aedb8ed686447d4866999109e99e32842023afb20ef7c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-osd-11-deactivate, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-17T06:15:24.722 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@osd.11.service' 2024-04-17T06:15:24.763 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-17T06:15:24.763 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-04-17T06:15:24.763 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-04-17T06:15:24.763 DEBUG:teuthology.orchestra.run.smithi047:> sudo systemctl stop ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@mds.a 2024-04-17T06:15:25.054 INFO:journalctl@ceph.mds.a.smithi047.stdout:Apr 17 06:15:24 smithi047 systemd[1]: Stopping Ceph mds.a for 565db2d8-fc7e-11ee-bc91-c7b262605968... 2024-04-17T06:15:25.324 INFO:journalctl@ceph.mds.a.smithi047.stdout:Apr 17 06:15:25 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-a[74679]: 2024-04-17T06:15:25.054+0000 7f00bd0f4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-17T06:15:25.324 INFO:journalctl@ceph.mds.a.smithi047.stdout:Apr 17 06:15:25 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-a[74679]: 2024-04-17T06:15:25.054+0000 7f00bd0f4700 -1 mds.a *** got signal Terminated *** 2024-04-17T06:15:29.320 INFO:journalctl@ceph.mds.a.smithi047.stdout:Apr 17 06:15:29 smithi047 podman[94634]: 2024-04-17 06:15:29.067990782 +0000 UTC m=+4.103921931 container died ba359aa70ff5c0e4f19bcf246156f6ac9df49cb842d01fbbda88f0f1fd38ea98 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-a, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-04-17T06:15:29.639 INFO:journalctl@ceph.mds.a.smithi047.stdout:Apr 17 06:15:29 smithi047 podman[94634]: 2024-04-17 06:15:29.538333137 +0000 UTC m=+4.574264285 container cleanup ba359aa70ff5c0e4f19bcf246156f6ac9df49cb842d01fbbda88f0f1fd38ea98 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-a, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240416) 2024-04-17T06:15:29.639 INFO:journalctl@ceph.mds.a.smithi047.stdout:Apr 17 06:15:29 smithi047 bash[94634]: ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-a 2024-04-17T06:15:29.873 DEBUG:teuthology.orchestra.run.smithi047:> sudo pkill -f 'journalctl -f -n 0 -u ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@mds.a.service' 2024-04-17T06:15:29.913 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-17T06:15:29.914 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-04-17T06:15:29.914 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-04-17T06:15:29.914 DEBUG:teuthology.orchestra.run.smithi047:> sudo systemctl stop ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@mds.d 2024-04-17T06:15:30.139 INFO:journalctl@ceph.mds.d.smithi047.stdout:Apr 17 06:15:29 smithi047 systemd[1]: Stopping Ceph mds.d for 565db2d8-fc7e-11ee-bc91-c7b262605968... 2024-04-17T06:15:30.639 INFO:journalctl@ceph.mds.d.smithi047.stdout:Apr 17 06:15:30 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-d[73710]: 2024-04-17T06:15:30.316+0000 7f817cce7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-17T06:15:30.639 INFO:journalctl@ceph.mds.d.smithi047.stdout:Apr 17 06:15:30 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-d[73710]: 2024-04-17T06:15:30.316+0000 7f817cce7700 -1 mds.d *** got signal Terminated *** 2024-04-17T06:15:33.889 INFO:journalctl@ceph.mds.d.smithi047.stdout:Apr 17 06:15:33 smithi047 podman[94766]: 2024-04-17 06:15:33.571499034 +0000 UTC m=+3.435014203 container died 5dad8c0cc8741985ab623a302834d9239cb1efc7c43ab3c872fe6fd8700b6124 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True, GIT_CLEAN=True) 2024-04-17T06:15:34.191 INFO:journalctl@ceph.mds.d.smithi047.stdout:Apr 17 06:15:34 smithi047 podman[94766]: 2024-04-17 06:15:34.01692016 +0000 UTC m=+3.880435328 container cleanup 5dad8c0cc8741985ab623a302834d9239cb1efc7c43ab3c872fe6fd8700b6124 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, CEPH_POINT_RELEASE=) 2024-04-17T06:15:34.191 INFO:journalctl@ceph.mds.d.smithi047.stdout:Apr 17 06:15:34 smithi047 bash[94766]: ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-d 2024-04-17T06:15:34.443 INFO:journalctl@ceph.mds.d.smithi047.stdout:Apr 17 06:15:34 smithi047 podman[94780]: 2024-04-17 06:15:34.193172652 +0000 UTC m=+0.612125555 container remove 5dad8c0cc8741985ab623a302834d9239cb1efc7c43ab3c872fe6fd8700b6124 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-d, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-17T06:15:34.447 DEBUG:teuthology.orchestra.run.smithi047:> sudo pkill -f 'journalctl -f -n 0 -u ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@mds.d.service' 2024-04-17T06:15:34.484 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-17T06:15:34.485 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-04-17T06:15:34.485 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-04-17T06:15:34.485 DEBUG:teuthology.orchestra.run.smithi047:> sudo systemctl stop ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@mds.g 2024-04-17T06:15:34.827 INFO:journalctl@ceph.mds.g.smithi047.stdout:Apr 17 06:15:34 smithi047 systemd[1]: Stopping Ceph mds.g for 565db2d8-fc7e-11ee-bc91-c7b262605968... 2024-04-17T06:15:35.139 INFO:journalctl@ceph.mds.g.smithi047.stdout:Apr 17 06:15:34 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-g[74323]: 2024-04-17T06:15:34.828+0000 7f9f67cf8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-17T06:15:35.139 INFO:journalctl@ceph.mds.g.smithi047.stdout:Apr 17 06:15:34 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-g[74323]: 2024-04-17T06:15:34.828+0000 7f9f67cf8700 -1 mds.g *** got signal Terminated *** 2024-04-17T06:15:39.094 INFO:journalctl@ceph.mds.g.smithi047.stdout:Apr 17 06:15:38 smithi047 podman[94887]: 2024-04-17 06:15:38.838001484 +0000 UTC m=+4.125403272 container died 9b688587ce629a4e4a8f887e2cf637fac63876838623abb07f3e97e04ba10488 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-g, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-17T06:15:39.389 INFO:journalctl@ceph.mds.g.smithi047.stdout:Apr 17 06:15:39 smithi047 podman[94887]: 2024-04-17 06:15:39.304314878 +0000 UTC m=+4.591716661 container cleanup 9b688587ce629a4e4a8f887e2cf637fac63876838623abb07f3e97e04ba10488 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-g, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-04-17T06:15:39.389 INFO:journalctl@ceph.mds.g.smithi047.stdout:Apr 17 06:15:39 smithi047 bash[94887]: ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-g 2024-04-17T06:15:39.688 DEBUG:teuthology.orchestra.run.smithi047:> sudo pkill -f 'journalctl -f -n 0 -u ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@mds.g.service' 2024-04-17T06:15:39.725 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-17T06:15:39.725 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-04-17T06:15:39.725 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-04-17T06:15:39.725 DEBUG:teuthology.orchestra.run.smithi047:> sudo systemctl stop ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@mds.j 2024-04-17T06:15:39.889 INFO:journalctl@ceph.mds.j.smithi047.stdout:Apr 17 06:15:39 smithi047 systemd[1]: Stopping Ceph mds.j for 565db2d8-fc7e-11ee-bc91-c7b262605968... 2024-04-17T06:15:40.639 INFO:journalctl@ceph.mds.j.smithi047.stdout:Apr 17 06:15:40 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-j[73332]: 2024-04-17T06:15:40.149+0000 7fcfda40b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-17T06:15:40.639 INFO:journalctl@ceph.mds.j.smithi047.stdout:Apr 17 06:15:40 smithi047 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-j[73332]: 2024-04-17T06:15:40.149+0000 7fcfda40b700 -1 mds.j *** got signal Terminated *** 2024-04-17T06:15:42.715 INFO:journalctl@ceph.mds.j.smithi047.stdout:Apr 17 06:15:42 smithi047 podman[95008]: 2024-04-17 06:15:42.430956056 +0000 UTC m=+2.481477373 container died 90546cf558c860dec36fdb766e61c2079eae1f0984f96ff480c6f6dfb61b8ad1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-j, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD) 2024-04-17T06:15:43.139 INFO:journalctl@ceph.mds.j.smithi047.stdout:Apr 17 06:15:42 smithi047 podman[95008]: 2024-04-17 06:15:42.907863768 +0000 UTC m=+2.958385083 container cleanup 90546cf558c860dec36fdb766e61c2079eae1f0984f96ff480c6f6dfb61b8ad1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-j, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416) 2024-04-17T06:15:43.139 INFO:journalctl@ceph.mds.j.smithi047.stdout:Apr 17 06:15:42 smithi047 bash[95008]: ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-j 2024-04-17T06:15:43.282 DEBUG:teuthology.orchestra.run.smithi047:> sudo pkill -f 'journalctl -f -n 0 -u ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@mds.j.service' 2024-04-17T06:15:43.318 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-17T06:15:43.318 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-04-17T06:15:43.318 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-04-17T06:15:43.318 DEBUG:teuthology.orchestra.run.smithi098:> sudo systemctl stop ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@mds.b 2024-04-17T06:15:43.633 INFO:journalctl@ceph.mds.b.smithi098.stdout:Apr 17 06:15:43 smithi098 systemd[1]: Stopping Ceph mds.b for 565db2d8-fc7e-11ee-bc91-c7b262605968... 2024-04-17T06:15:43.895 INFO:journalctl@ceph.mds.b.smithi098.stdout:Apr 17 06:15:43 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-b[67363]: 2024-04-17T06:15:43.661+0000 7f1893636700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-17T06:15:43.895 INFO:journalctl@ceph.mds.b.smithi098.stdout:Apr 17 06:15:43 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-b[67363]: 2024-04-17T06:15:43.661+0000 7f1893636700 -1 mds.b *** got signal Terminated *** 2024-04-17T06:15:46.177 INFO:journalctl@ceph.mds.b.smithi098.stdout:Apr 17 06:15:45 smithi098 podman[75284]: 2024-04-17 06:15:45.897193191 +0000 UTC m=+2.384325099 container died b45b802cb363de6617f0822ce68347498cc102bf1f82c39d693cf8712f74717d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-b, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-17T06:15:46.527 INFO:journalctl@ceph.mds.b.smithi098.stdout:Apr 17 06:15:46 smithi098 podman[75284]: 2024-04-17 06:15:46.253512494 +0000 UTC m=+2.740644371 container cleanup b45b802cb363de6617f0822ce68347498cc102bf1f82c39d693cf8712f74717d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-b, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-17T06:15:46.528 INFO:journalctl@ceph.mds.b.smithi098.stdout:Apr 17 06:15:46 smithi098 bash[75284]: ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-b 2024-04-17T06:15:46.680 DEBUG:teuthology.orchestra.run.smithi098:> sudo pkill -f 'journalctl -f -n 0 -u ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@mds.b.service' 2024-04-17T06:15:46.718 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-17T06:15:46.718 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-04-17T06:15:46.718 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-04-17T06:15:46.718 DEBUG:teuthology.orchestra.run.smithi098:> sudo systemctl stop ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@mds.e 2024-04-17T06:15:47.039 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:15:46 smithi098 systemd[1]: Stopping Ceph mds.e for 565db2d8-fc7e-11ee-bc91-c7b262605968... 2024-04-17T06:15:47.383 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:15:47 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[71345]: 2024-04-17T06:15:47.040+0000 7f68c79f1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-17T06:15:47.383 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:15:47 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e[71345]: 2024-04-17T06:15:47.040+0000 7f68c79f1700 -1 mds.e *** got signal Terminated *** 2024-04-17T06:15:51.306 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:15:51 smithi098 podman[75416]: 2024-04-17 06:15:51.054310624 +0000 UTC m=+4.119114907 container died c10b1e0b1416beb3d91d8206ac63000f89b005e2582c3e40b474bb19a69df613 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-04-17T06:15:51.633 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:15:51 smithi098 podman[75416]: 2024-04-17 06:15:51.414051615 +0000 UTC m=+4.478855892 container cleanup c10b1e0b1416beb3d91d8206ac63000f89b005e2582c3e40b474bb19a69df613 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416) 2024-04-17T06:15:51.633 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 06:15:51 smithi098 bash[75416]: ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-e 2024-04-17T06:15:51.837 DEBUG:teuthology.orchestra.run.smithi098:> sudo pkill -f 'journalctl -f -n 0 -u ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@mds.e.service' 2024-04-17T06:15:51.873 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-17T06:15:51.873 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-04-17T06:15:51.873 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-04-17T06:15:51.873 DEBUG:teuthology.orchestra.run.smithi098:> sudo systemctl stop ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@mds.h 2024-04-17T06:15:52.383 INFO:journalctl@ceph.mds.h.smithi098.stdout:Apr 17 06:15:51 smithi098 systemd[1]: Stopping Ceph mds.h for 565db2d8-fc7e-11ee-bc91-c7b262605968... 2024-04-17T06:15:52.383 INFO:journalctl@ceph.mds.h.smithi098.stdout:Apr 17 06:15:52 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-h[67059]: 2024-04-17T06:15:52.186+0000 7f8e9fc09700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-17T06:15:52.383 INFO:journalctl@ceph.mds.h.smithi098.stdout:Apr 17 06:15:52 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-h[67059]: 2024-04-17T06:15:52.186+0000 7f8e9fc09700 -1 mds.h *** got signal Terminated *** 2024-04-17T06:15:54.152 INFO:journalctl@ceph.mds.h.smithi098.stdout:Apr 17 06:15:53 smithi098 podman[75538]: 2024-04-17 06:15:53.895652117 +0000 UTC m=+1.807734582 container died 94c7c390af79f132dbf9d065fd5f3f5ee1c9fe6bfcd12dcf8b55821b1ca8b045 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-h, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux ) 2024-04-17T06:15:54.403 INFO:journalctl@ceph.mds.h.smithi098.stdout:Apr 17 06:15:54 smithi098 podman[75538]: 2024-04-17 06:15:54.245119424 +0000 UTC m=+2.157201864 container cleanup 94c7c390af79f132dbf9d065fd5f3f5ee1c9fe6bfcd12dcf8b55821b1ca8b045 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-h, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416) 2024-04-17T06:15:54.403 INFO:journalctl@ceph.mds.h.smithi098.stdout:Apr 17 06:15:54 smithi098 bash[75538]: ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-h 2024-04-17T06:15:54.598 DEBUG:teuthology.orchestra.run.smithi098:> sudo pkill -f 'journalctl -f -n 0 -u ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@mds.h.service' 2024-04-17T06:15:54.632 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-17T06:15:54.632 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-04-17T06:15:54.632 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-04-17T06:15:54.632 DEBUG:teuthology.orchestra.run.smithi098:> sudo systemctl stop ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@mds.k 2024-04-17T06:15:54.972 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:15:54 smithi098 systemd[1]: Stopping Ceph mds.k for 565db2d8-fc7e-11ee-bc91-c7b262605968... 2024-04-17T06:15:55.383 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:15:54 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[73021]: 2024-04-17T06:15:54.974+0000 7f97d5c13700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-17T06:15:55.383 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:15:54 smithi098 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k[73021]: 2024-04-17T06:15:54.974+0000 7f97d5c13700 -1 mds.k *** got signal Terminated *** 2024-04-17T06:15:58.165 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:15:57 smithi098 podman[75658]: 2024-04-17 06:15:57.894163998 +0000 UTC m=+3.029089943 container died 49fcadeeae1a3608b497973c8cfdf03c4da9def26e9c8779af7839fb67df4f9f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-04-17T06:15:58.506 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:15:58 smithi098 podman[75658]: 2024-04-17 06:15:58.249251062 +0000 UTC m=+3.384177006 container cleanup 49fcadeeae1a3608b497973c8cfdf03c4da9def26e9c8779af7839fb67df4f9f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-17T06:15:58.506 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 06:15:58 smithi098 bash[75658]: ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-k 2024-04-17T06:15:58.661 DEBUG:teuthology.orchestra.run.smithi098:> sudo pkill -f 'journalctl -f -n 0 -u ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@mds.k.service' 2024-04-17T06:15:58.694 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-17T06:15:58.694 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-04-17T06:15:58.694 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-04-17T06:15:58.694 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@mds.c 2024-04-17T06:15:59.034 INFO:journalctl@ceph.mds.c.smithi103.stdout:Apr 17 06:15:58 smithi103 systemd[1]: Stopping Ceph mds.c for 565db2d8-fc7e-11ee-bc91-c7b262605968... 2024-04-17T06:15:59.534 INFO:journalctl@ceph.mds.c.smithi103.stdout:Apr 17 06:15:59 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-c[66600]: 2024-04-17T06:15:59.037+0000 7fb3338eb700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-17T06:15:59.534 INFO:journalctl@ceph.mds.c.smithi103.stdout:Apr 17 06:15:59 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-c[66600]: 2024-04-17T06:15:59.037+0000 7fb3338eb700 -1 mds.c *** got signal Terminated *** 2024-04-17T06:16:02.244 INFO:journalctl@ceph.mds.c.smithi103.stdout:Apr 17 06:16:01 smithi103 podman[74463]: 2024-04-17 06:16:01.91108522 +0000 UTC m=+3.022284639 container died 957484b8ec18dd512d85ee0307b416fabc87f42ce23c7a99a08c732ae4f5154f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-c, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-17T06:16:02.528 INFO:journalctl@ceph.mds.c.smithi103.stdout:Apr 17 06:16:02 smithi103 podman[74463]: 2024-04-17 06:16:02.353730414 +0000 UTC m=+3.464929858 container cleanup 957484b8ec18dd512d85ee0307b416fabc87f42ce23c7a99a08c732ae4f5154f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-c, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-04-17T06:16:02.528 INFO:journalctl@ceph.mds.c.smithi103.stdout:Apr 17 06:16:02 smithi103 bash[74463]: ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-c 2024-04-17T06:16:02.764 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@mds.c.service' 2024-04-17T06:16:02.784 INFO:journalctl@ceph.mds.c.smithi103.stdout:Apr 17 06:16:02 smithi103 podman[74489]: 2024-04-17 06:16:02.530188951 +0000 UTC m=+0.613377538 container remove 957484b8ec18dd512d85ee0307b416fabc87f42ce23c7a99a08c732ae4f5154f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-c, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-17T06:16:02.784 INFO:journalctl@ceph.mds.c.smithi103.stdout:Apr 17 06:16:02 smithi103 systemd[1]: ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@mds.c.service: Deactivated successfully. 2024-04-17T06:16:02.784 INFO:journalctl@ceph.mds.c.smithi103.stdout:Apr 17 06:16:02 smithi103 systemd[1]: Stopped Ceph mds.c for 565db2d8-fc7e-11ee-bc91-c7b262605968. 2024-04-17T06:16:02.784 INFO:journalctl@ceph.mds.c.smithi103.stdout:Apr 17 06:16:02 smithi103 systemd[1]: ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@mds.c.service: Consumed 2.328s CPU time. 2024-04-17T06:16:02.802 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-17T06:16:02.802 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-04-17T06:16:02.802 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-04-17T06:16:02.803 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@mds.f 2024-04-17T06:16:03.123 INFO:journalctl@ceph.mds.f.smithi103.stdout:Apr 17 06:16:02 smithi103 systemd[1]: Stopping Ceph mds.f for 565db2d8-fc7e-11ee-bc91-c7b262605968... 2024-04-17T06:16:03.534 INFO:journalctl@ceph.mds.f.smithi103.stdout:Apr 17 06:16:03 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-f[66292]: 2024-04-17T06:16:03.124+0000 7fa3444fa700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-17T06:16:03.534 INFO:journalctl@ceph.mds.f.smithi103.stdout:Apr 17 06:16:03 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-f[66292]: 2024-04-17T06:16:03.124+0000 7fa3444fa700 -1 mds.f *** got signal Terminated *** 2024-04-17T06:16:06.247 INFO:journalctl@ceph.mds.f.smithi103.stdout:Apr 17 06:16:05 smithi103 podman[74596]: 2024-04-17 06:16:05.916964857 +0000 UTC m=+2.897393458 container died fa8c8ed243365c17696df32ed92348728d19f73ea9d4adef1fcdc2528d78e3c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-f, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-17T06:16:06.534 INFO:journalctl@ceph.mds.f.smithi103.stdout:Apr 17 06:16:06 smithi103 podman[74596]: 2024-04-17 06:16:06.324626124 +0000 UTC m=+3.305054733 container cleanup fa8c8ed243365c17696df32ed92348728d19f73ea9d4adef1fcdc2528d78e3c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-f, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240416) 2024-04-17T06:16:06.537 INFO:journalctl@ceph.mds.f.smithi103.stdout:Apr 17 06:16:06 smithi103 bash[74596]: ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-f 2024-04-17T06:16:06.747 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@mds.f.service' 2024-04-17T06:16:06.783 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-17T06:16:06.783 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-04-17T06:16:06.783 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-04-17T06:16:06.783 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@mds.i 2024-04-17T06:16:07.284 INFO:journalctl@ceph.mds.i.smithi103.stdout:Apr 17 06:16:06 smithi103 systemd[1]: Stopping Ceph mds.i for 565db2d8-fc7e-11ee-bc91-c7b262605968... 2024-04-17T06:16:07.284 INFO:journalctl@ceph.mds.i.smithi103.stdout:Apr 17 06:16:07 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-i[66918]: 2024-04-17T06:16:07.095+0000 7f9e23f02700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-17T06:16:07.284 INFO:journalctl@ceph.mds.i.smithi103.stdout:Apr 17 06:16:07 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-i[66918]: 2024-04-17T06:16:07.095+0000 7f9e23f02700 -1 mds.i *** got signal Terminated *** 2024-04-17T06:16:10.177 INFO:journalctl@ceph.mds.i.smithi103.stdout:Apr 17 06:16:09 smithi103 podman[74719]: 2024-04-17 06:16:09.903249758 +0000 UTC m=+2.902691030 container died e6268839d96ab24bf4e242b908e7fb4da2286439cb6c3744a39a484d97b8e85f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-i, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_CLEAN=True) 2024-04-17T06:16:10.427 INFO:journalctl@ceph.mds.i.smithi103.stdout:Apr 17 06:16:10 smithi103 podman[74719]: 2024-04-17 06:16:10.253649833 +0000 UTC m=+3.253091087 container cleanup e6268839d96ab24bf4e242b908e7fb4da2286439cb6c3744a39a484d97b8e85f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-i, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5) 2024-04-17T06:16:10.427 INFO:journalctl@ceph.mds.i.smithi103.stdout:Apr 17 06:16:10 smithi103 bash[74719]: ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-i 2024-04-17T06:16:10.649 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@mds.i.service' 2024-04-17T06:16:10.683 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-17T06:16:10.683 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-04-17T06:16:10.683 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-04-17T06:16:10.683 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@mds.l 2024-04-17T06:16:11.005 INFO:journalctl@ceph.mds.l.smithi103.stdout:Apr 17 06:16:10 smithi103 systemd[1]: Stopping Ceph mds.l for 565db2d8-fc7e-11ee-bc91-c7b262605968... 2024-04-17T06:16:11.281 INFO:journalctl@ceph.mds.l.smithi103.stdout:Apr 17 06:16:11 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-l[65981]: 2024-04-17T06:16:11.006+0000 7f92765f3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-17T06:16:11.281 INFO:journalctl@ceph.mds.l.smithi103.stdout:Apr 17 06:16:11 smithi103 ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-l[65981]: 2024-04-17T06:16:11.006+0000 7f92765f3700 -1 mds.l *** got signal Terminated *** 2024-04-17T06:16:13.014 INFO:journalctl@ceph.mds.l.smithi103.stdout:Apr 17 06:16:12 smithi103 podman[74841]: 2024-04-17 06:16:12.74858571 +0000 UTC m=+1.857752548 container died 903cc67b7c0d4393f7f453966226c0bb8aaabb45dde78318626a9b5374627c0f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-l, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2) 2024-04-17T06:16:13.284 INFO:journalctl@ceph.mds.l.smithi103.stdout:Apr 17 06:16:13 smithi103 podman[74841]: 2024-04-17 06:16:13.116237851 +0000 UTC m=+2.225404689 container cleanup 903cc67b7c0d4393f7f453966226c0bb8aaabb45dde78318626a9b5374627c0f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-l, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-17T06:16:13.284 INFO:journalctl@ceph.mds.l.smithi103.stdout:Apr 17 06:16:13 smithi103 bash[74841]: ceph-565db2d8-fc7e-11ee-bc91-c7b262605968-mds-l 2024-04-17T06:16:13.543 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-565db2d8-fc7e-11ee-bc91-c7b262605968@mds.l.service' 2024-04-17T06:16:13.576 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-17T06:16:13.576 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-04-17T06:16:13.576 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 565db2d8-fc7e-11ee-bc91-c7b262605968 --force --keep-logs 2024-04-17T06:16:13.778 INFO:teuthology.orchestra.run.smithi047.stdout:Deleting cluster with fsid: 565db2d8-fc7e-11ee-bc91-c7b262605968 2024-04-17T06:16:16.183 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 565db2d8-fc7e-11ee-bc91-c7b262605968 --force --keep-logs 2024-04-17T06:16:16.375 INFO:teuthology.orchestra.run.smithi098.stdout:Deleting cluster with fsid: 565db2d8-fc7e-11ee-bc91-c7b262605968 2024-04-17T06:16:18.866 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 565db2d8-fc7e-11ee-bc91-c7b262605968 --force --keep-logs 2024-04-17T06:16:19.059 INFO:teuthology.orchestra.run.smithi103.stdout:Deleting cluster with fsid: 565db2d8-fc7e-11ee-bc91-c7b262605968 2024-04-17T06:16:21.518 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-17T06:16:21.555 DEBUG:teuthology.orchestra.run.smithi098:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-17T06:16:21.589 DEBUG:teuthology.orchestra.run.smithi103:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-17T06:16:21.618 INFO:tasks.cephadm:Archiving crash dumps... 2024-04-17T06:16:21.620 DEBUG:teuthology.misc:Transferring archived files from smithi047:/var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-04-17_04:58:44-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7659857/remote/smithi047/crash 2024-04-17T06:16:21.621 DEBUG:teuthology.orchestra.run.smithi047:> sudo tar c -f - -C /var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/crash -- . 2024-04-17T06:16:21.666 INFO:teuthology.orchestra.run.smithi047.stderr:tar: /var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/crash: Cannot open: No such file or directory 2024-04-17T06:16:21.666 INFO:teuthology.orchestra.run.smithi047.stderr:tar: Error is not recoverable: exiting now 2024-04-17T06:16:21.668 DEBUG:teuthology.misc:Transferring archived files from smithi098:/var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-04-17_04:58:44-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7659857/remote/smithi098/crash 2024-04-17T06:16:21.669 DEBUG:teuthology.orchestra.run.smithi098:> sudo tar c -f - -C /var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/crash -- . 2024-04-17T06:16:21.714 INFO:teuthology.orchestra.run.smithi098.stderr:tar: /var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/crash: Cannot open: No such file or directory 2024-04-17T06:16:21.714 INFO:teuthology.orchestra.run.smithi098.stderr:tar: Error is not recoverable: exiting now 2024-04-17T06:16:21.716 DEBUG:teuthology.misc:Transferring archived files from smithi103:/var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-04-17_04:58:44-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7659857/remote/smithi103/crash 2024-04-17T06:16:21.717 DEBUG:teuthology.orchestra.run.smithi103:> sudo tar c -f - -C /var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/crash -- . 2024-04-17T06:16:21.756 INFO:teuthology.orchestra.run.smithi103.stderr:tar: /var/lib/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/crash: Cannot open: No such file or directory 2024-04-17T06:16:21.756 INFO:teuthology.orchestra.run.smithi103.stderr:tar: Error is not recoverable: exiting now 2024-04-17T06:16:21.757 INFO:tasks.cephadm:Checking cluster log for badness... 2024-04-17T06:16:21.757 DEBUG:teuthology.orchestra.run.smithi047:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | head -n 1 2024-04-17T06:16:21.794 INFO:tasks.cephadm:Compressing logs... 2024-04-17T06:16:21.794 DEBUG:teuthology.orchestra.run.smithi047:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-17T06:16:21.838 DEBUG:teuthology.orchestra.run.smithi098:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-17T06:16:21.840 DEBUG:teuthology.orchestra.run.smithi103:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-17T06:16:21.865 INFO:teuthology.orchestra.run.smithi047.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph-osd.6.log 2024-04-17T06:16:21.865 INFO:teuthology.orchestra.run.smithi047.stderr:: No such file or directory 2024-04-17T06:16:21.865 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph-mon.a.log 2024-04-17T06:16:21.866 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph.audit.log 2024-04-17T06:16:21.866 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph-osd.6.log: /var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph-mds.g.log 2024-04-17T06:16:21.867 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph-mds.j.log 2024-04-17T06:16:21.867 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph-osd.3.log 2024-04-17T06:16:21.867 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph-mds.g.log: /var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph-mgr.x.log 2024-04-17T06:16:21.873 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph-osd.0.log 2024-04-17T06:16:21.874 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph.cephadm.log 2024-04-17T06:16:21.874 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph-osd.0.log: 90.3% -- replaced with /var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph.audit.log.gz 2024-04-17T06:16:21.874 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph.log 2024-04-17T06:16:21.874 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph.cephadm.log: 84.1% -- replaced with /var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph.cephadm.log.gz 2024-04-17T06:16:21.874 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 94.6% -- replaced with /var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph-mds.g.log.gz 2024-04-17T06:16:21.874 INFO:teuthology.orchestra.run.smithi047.stderr: --verbose -- /var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph-volume.log 2024-04-17T06:16:21.874 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph-osd.9.log 2024-04-17T06:16:21.874 INFO:teuthology.orchestra.run.smithi047.stderr: 94.7% -- replaced with /var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph-mds.j.log.gz 2024-04-17T06:16:21.879 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph-volume.log: 87.4% -- replaced with /var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph.log.gz 2024-04-17T06:16:21.879 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph-mds.d.log 2024-04-17T06:16:21.879 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph-mds.a.log 2024-04-17T06:16:21.881 INFO:teuthology.orchestra.run.smithi103.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph-mds.l.log 2024-04-17T06:16:21.882 INFO:teuthology.orchestra.run.smithi103.stderr:: No such file or directory 2024-04-17T06:16:21.882 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph.audit.log 2024-04-17T06:16:21.882 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph-mds.i.log 2024-04-17T06:16:21.882 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph-mds.l.log: /var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph-osd.11.log 2024-04-17T06:16:21.882 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph-mgr.z.log 2024-04-17T06:16:21.882 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph-osd.8.log 2024-04-17T06:16:21.882 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph-osd.11.log: /var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph-osd.5.log 2024-04-17T06:16:21.882 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph-mds.f.log 2024-04-17T06:16:21.882 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph.cephadm.log 2024-04-17T06:16:21.882 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph-mds.f.log: 90.5% -- replaced with /var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph.audit.log.gz 2024-04-17T06:16:21.882 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph.log 2024-04-17T06:16:21.882 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph.cephadm.log: 83.8% -- replaced with /var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph.cephadm.log.gz 2024-04-17T06:16:21.882 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph-osd.2.log 2024-04-17T06:16:21.882 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph.log: 94.6% -- replaced with /var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph-mds.l.log.gz 2024-04-17T06:16:21.883 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph-mds.c.log 2024-04-17T06:16:21.883 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph-osd.2.log: 87.5% -- replaced with /var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph.log.gz 2024-04-17T06:16:21.883 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph-volume.log 2024-04-17T06:16:21.883 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph-mds.c.log: 92.0% -- replaced with /var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph-mgr.z.log.gz 2024-04-17T06:16:21.883 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph-mon.c.log 2024-04-17T06:16:21.883 INFO:teuthology.orchestra.run.smithi098.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph-osd.10.log 2024-04-17T06:16:21.884 INFO:teuthology.orchestra.run.smithi098.stderr:: No such file or directory 2024-04-17T06:16:21.884 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph.audit.log 2024-04-17T06:16:21.884 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph-mds.b.log 2024-04-17T06:16:21.884 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph-osd.4.log 2024-04-17T06:16:21.884 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph-osd.1.log 2024-04-17T06:16:21.884 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph-mds.k.log 2024-04-17T06:16:21.884 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph-osd.4.log: /var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph-mds.h.log 2024-04-17T06:16:21.884 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph-osd.7.log 2024-04-17T06:16:21.884 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph-mds.e.log 2024-04-17T06:16:21.884 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph-osd.7.log: 90.5% -- replaced with /var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph.audit.log.gz 2024-04-17T06:16:21.884 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph-mgr.y.log 2024-04-17T06:16:21.885 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph-mds.d.log: 94.5% -- replaced with /var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph-mds.d.log.gz 2024-04-17T06:16:21.886 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-17T06:16:21.887 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph-mds.a.log: 94.6% -- replaced with /var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph-mds.a.log.gz 2024-04-17T06:16:21.887 INFO:teuthology.orchestra.run.smithi047.stderr: 91.0% -- replaced with /var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph-volume.log.gz 2024-04-17T06:16:21.890 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-17T06:16:21.891 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph-mon.c.log: 90.9% -- replaced with /var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph-volume.log.gz 2024-04-17T06:16:21.896 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/cephadm.log: 93.3% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-17T06:16:21.903 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph.cephadm.log 2024-04-17T06:16:21.911 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/cephadm.log: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-17T06:16:21.913 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph-mgr.y.log: 91.9% -- replaced with /var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph-mgr.y.log.gz 2024-04-17T06:16:21.916 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph.log 2024-04-17T06:16:21.917 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph.cephadm.log: 83.8% -- replaced with /var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph.cephadm.log.gz 2024-04-17T06:16:21.933 INFO:teuthology.orchestra.run.smithi098.stderr: 93.2% -- replaced with /var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph-mds.h.log.gz 2024-04-17T06:16:21.934 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph-volume.log 2024-04-17T06:16:21.937 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph.log: 87.5% -- replaced with /var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph.log.gz 2024-04-17T06:16:21.938 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph-mon.b.log 2024-04-17T06:16:21.944 INFO:teuthology.orchestra.run.smithi103.stderr: 94.5% -- replaced with /var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph-mds.c.log.gz 2024-04-17T06:16:21.946 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-17T06:16:21.952 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph-mon.b.log: 90.9% -- replaced with /var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph-volume.log.gz 2024-04-17T06:16:21.954 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/cephadm.log: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-17T06:16:21.972 INFO:teuthology.orchestra.run.smithi098.stderr: 92.0% -- replaced with /var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph-mds.e.log.gz 2024-04-17T06:16:21.980 INFO:teuthology.orchestra.run.smithi098.stderr: 91.4% -- replaced with /var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph-mds.k.log.gz 2024-04-17T06:16:22.030 INFO:teuthology.orchestra.run.smithi103.stderr: 94.9% -- replaced with /var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph-mds.f.log.gz 2024-04-17T06:16:22.208 INFO:teuthology.orchestra.run.smithi098.stderr: 92.6% -- replaced with /var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph-mds.b.log.gz 2024-04-17T06:16:22.258 INFO:teuthology.orchestra.run.smithi047.stderr: 90.1% -- replaced with /var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph-mgr.x.log.gz 2024-04-17T06:16:22.293 INFO:teuthology.orchestra.run.smithi103.stderr: 92.0% -- replaced with /var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph-mon.c.log.gz 2024-04-17T06:16:22.329 INFO:teuthology.orchestra.run.smithi098.stderr: 92.0% -- replaced with /var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph-mon.b.log.gz 2024-04-17T06:16:22.719 INFO:teuthology.orchestra.run.smithi047.stderr: 92.4% -- replaced with /var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph-osd.3.log.gz 2024-04-17T06:16:22.790 INFO:teuthology.orchestra.run.smithi103.stderr: 93.0% -- replaced with /var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph-mds.i.log.gz 2024-04-17T06:16:22.836 INFO:teuthology.orchestra.run.smithi103.stderr: 93.3% -- replaced with /var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph-osd.11.log.gz 2024-04-17T06:16:22.839 INFO:teuthology.orchestra.run.smithi047.stderr: 90.4% -- replaced with /var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph-mon.a.log.gz 2024-04-17T06:16:22.862 INFO:teuthology.orchestra.run.smithi103.stderr: 92.5% -- replaced with /var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph-osd.8.log.gz 2024-04-17T06:16:22.867 INFO:teuthology.orchestra.run.smithi103.stderr: 92.4% -- replaced with /var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph-osd.2.log.gz 2024-04-17T06:16:22.936 INFO:teuthology.orchestra.run.smithi098.stderr: 91.6% -- replaced with /var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph-osd.1.log.gz 2024-04-17T06:16:22.952 INFO:teuthology.orchestra.run.smithi098.stderr: 92.0% -- replaced with /var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph-osd.7.log.gz 2024-04-17T06:16:23.053 INFO:teuthology.orchestra.run.smithi047.stderr: 91.2% -- replaced with /var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph-osd.6.log.gz 2024-04-17T06:16:23.141 INFO:teuthology.orchestra.run.smithi103.stderr: 91.7% -- replaced with /var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph-osd.5.log.gz 2024-04-17T06:16:23.143 INFO:teuthology.orchestra.run.smithi103.stderr: 2024-04-17T06:16:23.143 INFO:teuthology.orchestra.run.smithi103.stderr:real 0m1.289s 2024-04-17T06:16:23.143 INFO:teuthology.orchestra.run.smithi103.stderr:user 0m5.640s 2024-04-17T06:16:23.143 INFO:teuthology.orchestra.run.smithi103.stderr:sys 0m0.203s 2024-04-17T06:16:23.157 INFO:teuthology.orchestra.run.smithi047.stderr: 92.9% -- replaced with /var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph-osd.9.log.gz 2024-04-17T06:16:23.283 INFO:teuthology.orchestra.run.smithi047.stderr: 93.0% -- replaced with /var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph-osd.0.log.gz 2024-04-17T06:16:23.285 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-17T06:16:23.285 INFO:teuthology.orchestra.run.smithi047.stderr:real 0m1.434s 2024-04-17T06:16:23.285 INFO:teuthology.orchestra.run.smithi047.stderr:user 0m5.963s 2024-04-17T06:16:23.285 INFO:teuthology.orchestra.run.smithi047.stderr:sys 0m0.216s 2024-04-17T06:16:23.932 INFO:teuthology.orchestra.run.smithi098.stderr: 92.3% -- replaced with /var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph-osd.10.log.gz 2024-04-17T06:16:24.134 INFO:teuthology.orchestra.run.smithi098.stderr: 91.6% -- replaced with /var/log/ceph/565db2d8-fc7e-11ee-bc91-c7b262605968/ceph-osd.4.log.gz 2024-04-17T06:16:24.135 INFO:teuthology.orchestra.run.smithi098.stderr: 2024-04-17T06:16:24.136 INFO:teuthology.orchestra.run.smithi098.stderr:real 0m2.282s 2024-04-17T06:16:24.136 INFO:teuthology.orchestra.run.smithi098.stderr:user 0m7.259s 2024-04-17T06:16:24.136 INFO:teuthology.orchestra.run.smithi098.stderr:sys 0m0.246s 2024-04-17T06:16:24.136 INFO:tasks.cephadm:Archiving logs... 2024-04-17T06:16:24.136 DEBUG:teuthology.misc:Transferring archived files from smithi047:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-17_04:58:44-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7659857/remote/smithi047/log 2024-04-17T06:16:24.137 DEBUG:teuthology.orchestra.run.smithi047:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-17T06:16:25.031 DEBUG:teuthology.misc:Transferring archived files from smithi098:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-17_04:58:44-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7659857/remote/smithi098/log 2024-04-17T06:16:25.032 DEBUG:teuthology.orchestra.run.smithi098:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-17T06:16:26.199 DEBUG:teuthology.misc:Transferring archived files from smithi103:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-17_04:58:44-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7659857/remote/smithi103/log 2024-04-17T06:16:26.200 DEBUG:teuthology.orchestra.run.smithi103:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-17T06:16:26.998 INFO:tasks.cephadm:Removing cluster... 2024-04-17T06:16:26.998 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 565db2d8-fc7e-11ee-bc91-c7b262605968 --force 2024-04-17T06:16:27.201 INFO:teuthology.orchestra.run.smithi047.stdout:Deleting cluster with fsid: 565db2d8-fc7e-11ee-bc91-c7b262605968 2024-04-17T06:16:27.459 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 565db2d8-fc7e-11ee-bc91-c7b262605968 --force 2024-04-17T06:16:27.653 INFO:teuthology.orchestra.run.smithi098.stdout:Deleting cluster with fsid: 565db2d8-fc7e-11ee-bc91-c7b262605968 2024-04-17T06:16:27.921 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 565db2d8-fc7e-11ee-bc91-c7b262605968 --force 2024-04-17T06:16:28.112 INFO:teuthology.orchestra.run.smithi103.stdout:Deleting cluster with fsid: 565db2d8-fc7e-11ee-bc91-c7b262605968 2024-04-17T06:16:28.368 INFO:tasks.cephadm:Removing cephadm ... 2024-04-17T06:16:28.368 DEBUG:teuthology.orchestra.run.smithi047:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-17T06:16:28.387 DEBUG:teuthology.orchestra.run.smithi098:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-17T06:16:28.403 DEBUG:teuthology.orchestra.run.smithi103:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-17T06:16:28.419 INFO:tasks.cephadm:Teardown complete 2024-04-17T06:16:28.419 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-17T06:16:28.432 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-17T06:16:28.432 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-17T06:16:28.435 DEBUG:teuthology.orchestra.run.smithi098:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-17T06:16:28.445 DEBUG:teuthology.orchestra.run.smithi103:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-17T06:16:28.555 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-17T06:16:28.565 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-17T06:16:28.565 DEBUG:teuthology.orchestra.run.smithi047:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-17T06:16:28.568 DEBUG:teuthology.orchestra.run.smithi098:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-17T06:16:28.571 DEBUG:teuthology.orchestra.run.smithi103:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-17T06:16:28.582 INFO:teuthology.orchestra.run.smithi098.stderr:bash: line 1: ntpq: command not found 2024-04-17T06:16:28.583 INFO:teuthology.orchestra.run.smithi047.stderr:bash: line 1: ntpq: command not found 2024-04-17T06:16:28.584 INFO:teuthology.orchestra.run.smithi103.stderr:bash: line 1: ntpq: command not found 2024-04-17T06:16:28.587 INFO:teuthology.orchestra.run.smithi098.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-17T06:16:28.587 INFO:teuthology.orchestra.run.smithi098.stdout:=============================================================================== 2024-04-17T06:16:28.587 INFO:teuthology.orchestra.run.smithi098.stdout:^* hv01.front.sepia.ceph.com 3 6 377 10 +736us[ +745us] +/- 65ms 2024-04-17T06:16:28.587 INFO:teuthology.orchestra.run.smithi098.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 12 -899us[ -889us] +/- 64ms 2024-04-17T06:16:28.587 INFO:teuthology.orchestra.run.smithi098.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 11 +210us[ +220us] +/- 89ms 2024-04-17T06:16:28.587 INFO:teuthology.orchestra.run.smithi098.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-17T06:16:28.589 INFO:teuthology.orchestra.run.smithi047.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-17T06:16:28.589 INFO:teuthology.orchestra.run.smithi047.stdout:=============================================================================== 2024-04-17T06:16:28.589 INFO:teuthology.orchestra.run.smithi047.stdout:^* hv01.front.sepia.ceph.com 3 6 377 11 +769us[ +761us] +/- 65ms 2024-04-17T06:16:28.590 INFO:teuthology.orchestra.run.smithi047.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 13 -871us[ -879us] +/- 64ms 2024-04-17T06:16:28.590 INFO:teuthology.orchestra.run.smithi047.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 13 +240us[ +232us] +/- 89ms 2024-04-17T06:16:28.590 INFO:teuthology.orchestra.run.smithi047.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-17T06:16:28.590 INFO:teuthology.orchestra.run.smithi103.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-17T06:16:28.590 INFO:teuthology.orchestra.run.smithi103.stdout:=============================================================================== 2024-04-17T06:16:28.590 INFO:teuthology.orchestra.run.smithi103.stdout:^* hv01.front.sepia.ceph.com 3 6 377 14 +716us[ +726us] +/- 65ms 2024-04-17T06:16:28.590 INFO:teuthology.orchestra.run.smithi103.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 14 -896us[ -886us] +/- 64ms 2024-04-17T06:16:28.590 INFO:teuthology.orchestra.run.smithi103.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 13 +211us[ +211us] +/- 89ms 2024-04-17T06:16:28.590 INFO:teuthology.orchestra.run.smithi103.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-17T06:16:28.590 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-17T06:16:28.601 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-17T06:16:28.602 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-17T06:16:28.630 DEBUG:teuthology.orchestra.run.smithi047:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-17T06:16:28.719 DEBUG:teuthology.orchestra.run.smithi098:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-17T06:16:28.791 DEBUG:teuthology.orchestra.run.smithi103:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-17T06:16:28.862 DEBUG:teuthology.orchestra.run.smithi047:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-17T06:16:28.896 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T06:16:28.896 DEBUG:teuthology.orchestra.run.smithi098:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-17T06:16:28.926 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T06:16:28.927 DEBUG:teuthology.orchestra.run.smithi103:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-17T06:16:28.957 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T06:16:28.957 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-17T06:16:28.969 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-17T06:16:29.024 INFO:teuthology.task.internal:Duration was 1979.954595 seconds 2024-04-17T06:16:29.024 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-17T06:16:29.051 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-17T06:16:29.052 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-17T06:16:29.054 DEBUG:teuthology.orchestra.run.smithi098:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-17T06:16:29.057 DEBUG:teuthology.orchestra.run.smithi103:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-17T06:16:29.097 INFO:teuthology.orchestra.run.smithi098.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-17T06:16:29.100 INFO:teuthology.orchestra.run.smithi103.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-17T06:16:29.100 INFO:teuthology.orchestra.run.smithi047.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-17T06:16:29.563 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-17T06:16:29.563 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi047.front.sepia.ceph.com 2024-04-17T06:16:29.563 DEBUG:teuthology.orchestra.run.smithi047:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-17T06:16:29.589 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi098.front.sepia.ceph.com 2024-04-17T06:16:29.589 DEBUG:teuthology.orchestra.run.smithi098:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-17T06:16:29.609 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi103.front.sepia.ceph.com 2024-04-17T06:16:29.609 DEBUG:teuthology.orchestra.run.smithi103:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-17T06:16:29.632 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-17T06:16:29.632 DEBUG:teuthology.orchestra.run.smithi047:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-17T06:16:29.634 DEBUG:teuthology.orchestra.run.smithi098:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-17T06:16:29.652 DEBUG:teuthology.orchestra.run.smithi103:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-17T06:16:29.701 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-17T06:16:29.701 DEBUG:teuthology.orchestra.run.smithi047:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-17T06:16:29.709 DEBUG:teuthology.orchestra.run.smithi098:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-17T06:16:29.720 DEBUG:teuthology.orchestra.run.smithi103:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-17T06:16:30.090 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-17T06:16:30.102 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-17T06:16:30.102 DEBUG:teuthology.orchestra.run.smithi047:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-17T06:16:30.131 DEBUG:teuthology.orchestra.run.smithi098:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-17T06:16:30.157 DEBUG:teuthology.orchestra.run.smithi103:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-17T06:16:30.186 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-17T06:16:30.238 DEBUG:teuthology.orchestra.run.smithi047:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-17T06:16:30.241 DEBUG:teuthology.orchestra.run.smithi098:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-17T06:16:30.244 DEBUG:teuthology.orchestra.run.smithi103:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-17T06:16:30.266 INFO:teuthology.orchestra.run.smithi047.stdout:kernel.core_pattern = core 2024-04-17T06:16:30.267 INFO:teuthology.orchestra.run.smithi098.stdout:kernel.core_pattern = core 2024-04-17T06:16:30.270 INFO:teuthology.orchestra.run.smithi103.stdout:kernel.core_pattern = core 2024-04-17T06:16:30.312 DEBUG:teuthology.orchestra.run.smithi047:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-17T06:16:30.337 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T06:16:30.338 DEBUG:teuthology.orchestra.run.smithi098:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-17T06:16:30.355 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T06:16:30.356 DEBUG:teuthology.orchestra.run.smithi103:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-17T06:16:30.369 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T06:16:30.370 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-17T06:16:30.387 INFO:teuthology.task.internal:Transferring archived files... 2024-04-17T06:16:30.388 DEBUG:teuthology.misc:Transferring archived files from smithi047:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-17_04:58:44-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7659857/remote/smithi047 2024-04-17T06:16:30.388 DEBUG:teuthology.orchestra.run.smithi047:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-17T06:16:30.433 DEBUG:teuthology.misc:Transferring archived files from smithi098:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-17_04:58:44-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7659857/remote/smithi098 2024-04-17T06:16:30.433 DEBUG:teuthology.orchestra.run.smithi098:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-17T06:16:30.471 DEBUG:teuthology.misc:Transferring archived files from smithi103:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-17_04:58:44-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7659857/remote/smithi103 2024-04-17T06:16:30.472 DEBUG:teuthology.orchestra.run.smithi103:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-17T06:16:30.512 INFO:teuthology.task.internal:Removing archive directory... 2024-04-17T06:16:30.512 DEBUG:teuthology.orchestra.run.smithi047:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-17T06:16:30.514 DEBUG:teuthology.orchestra.run.smithi098:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-17T06:16:30.517 DEBUG:teuthology.orchestra.run.smithi103:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-17T06:16:30.559 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-17T06:16:30.573 INFO:teuthology.task.internal:Not uploading archives. 2024-04-17T06:16:30.573 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-17T06:16:30.608 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-17T06:16:30.608 DEBUG:teuthology.orchestra.run.smithi047:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-17T06:16:30.610 DEBUG:teuthology.orchestra.run.smithi098:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-17T06:16:30.613 DEBUG:teuthology.orchestra.run.smithi103:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-17T06:16:30.623 INFO:teuthology.orchestra.run.smithi047.stdout: 262166 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 17 06:16 /home/ubuntu/cephtest 2024-04-17T06:16:30.625 INFO:teuthology.orchestra.run.smithi098.stdout: 262162 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 17 06:16 /home/ubuntu/cephtest 2024-04-17T06:16:30.627 INFO:teuthology.orchestra.run.smithi103.stdout: 262162 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 17 06:16 /home/ubuntu/cephtest 2024-04-17T06:16:30.628 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-17T06:16:30.714 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-17T06:16:30.795 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-workunit/fs/test_o_trunc}} duration: 1979.9545948505402 flavor: default owner: scheduled_rishabh@teuthology success: true 2024-04-17T06:16:30.795 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-17T06:16:30.879 INFO:teuthology.run:pass